Shaking PHP code until it rattles

Unit tests check the inputs we thought of. Fuzzing finds the ones we did not.

This talk introduces a new coverage-guided fuzzer for PHP that instruments your code while it is loaded, executes it with inputs that it keeps mutating, and keeps every input that reaches code no input reached before.

This fuzzer and PHPUnit's property-based testing are built on the same foundation, choice sequences, so the properties in your test suite become fuzz targets. Every failure is minimised into a small counterexample that becomes a regression test. You will learn how the fuzzer works, how to use it, and what it finds.

This presentation currently exists only as an idea and has not yet been accepted by a conference.

About me

I am the creator and maintainer of PHPUnit, the de facto standard testing framework within the PHP ecosystem. Used by millions of developers, it is embedded in the build pipelines of start-ups, Fortune 500 companies, and public sector organisations alike. I serve on the board of the PHP Foundation and am a co-founder of thePHP.cc, where I advise organisations on testing strategy, test design, and testing code that processes data it cannot trust.

I started writing PHPUnit in 2000 and have contributed to the PHP language itself as well as the tools on which its ecosystem depends. My newest tool, rattle, fuzzes the properties of a PHPUnit test suite as they are, because a property already has what a fuzz target needs: a description of its valid inputs and assertions that tell whether the code behaves as it should. A fuzz target of a few lines and 30 minutes of fuzzing were enough for rattle to find exponential parse times in symfony/yaml, which the Symfony security team has since fixed, and an input that HTML Purifier does not purify in one pass.