Your types already know what to test: Property-Based Testing in PHPUnit 14

Example-based tests check the cases their author thought of. Property-based tests state a rule that must hold for all inputs and leave it to the tool to search for an input that breaks it. PHPUnit 14 brings this technique into the test runner itself: a property is an ordinary test method, marked with #[Property], whose arguments PHPUnit generates. In many cases it does so from the PHPDoc types that PHPStan and Psalm already read, so integer ranges, non-empty strings, array shapes, enums, and value objects need no hand-written generator.

This talk shows how to find properties worth stating in real PHP code, from round-trips and invariants to stateful models of an object’s life cycle. It then explains the mechanism: every random decision is recorded as a choice, a falsified property is shrunk to a minimal counterexample by simplifying that record of choices rather than the value, and the counterexample can be stored and replayed on the next run until the bug is fixed. Along the way, it covers what running each generated example as a full test costs, what the implementation deliberately does not do, and where example-based tests remain the better tool.

This presentation currently exists only as an idea and has not yet been accepted by a conference.

About me

I am the creator and maintainer of PHPUnit, the de facto standard testing framework within the PHP ecosystem. Used by millions of developers, it is embedded in the build pipelines of start-ups, Fortune 500 companies, and public sector organisations alike. I serve on the board of the PHP Foundation and am a co-founder of thePHP.cc, where I advise organisations on testing strategy, test design, and testing beyond hand-picked examples.

I started writing PHPUnit in 2000 and have contributed to the PHP language itself as well as the tools on which its ecosystem depends. I designed and implemented the property-based testing of PHPUnit 14. Which PHPDoc types a generator can be derived from, why every generated example runs as a full test, and why shrinking simplifies the record of choices rather than the value are decisions I had to make for a test runner that millions of projects depend on. That is what I talk about here: what these decisions cost, and what I deliberately left out.