First, an important clarification: I do not like cheese. Period. I find these yellowish, sometimes smelly blocks that have been left to mature for too long in dark caves suspicious. However, there are two exceptions that shake my entire worldview: pizza and pasta. Suddenly, cheese is no longer my enemy, but my best friend. Am I a hypocrite? Perhaps. But that is not what this article is about.
So why does a picture of Swiss cheese adorn this article?
The answer is the Swiss Cheese Model. And no, it is not about taste. It is a famous security concept in which multiple layers of security overlap like slices of cheese. Each slice has holes (weak points), but together they protect against the “enemy” breaking through unchecked.
I really hope that the picture above shows real Swiss cheese. I know absolutely nothing about cheese, as should be clear by now. It could also be Dutch or French cheese, or just any random yellow block from the supermarket. But one thing I know for sure: it should have large holes. Large, characteristic holes. Because the holes are the most important thing about this cheese – just like in our safety nets in software development.
Why one approach is not enough
Hillel Wayne observes:
[U]nit tests are not enough. Type systems are not enough. Contracts are not enough, formal specs are not enough, code review isn't enough, nothing is enough. We have to use everything we have to even hope of writing correct code, because there's only one way a program is right and infinite ways a program can be wrong, and we can't assume that any tool we use will prevent more than a narrow slice of all those wrong ways.
This fundamental truth is driving modern PHP development towards a multi-layered approach to software quality and security. Understanding the relationship between static analysis, dynamic testing with PHPUnit, property-based testing, mutation testing, and fuzz testing reveals a comprehensive strategy that identifies different types of bugs at various stages of development. Each technique addresses its own narrow slice of the infinite ways in which code can fail.
The traditional testing pyramid positions unit tests at the base, integration tests in the middle, and end-to-end tests at the top. However, when modern techniques such as property-based testing and fuzzing are incorporated, a continuum of test granularity and bug-finding capability emerges rather than a strict hierarchy.
Each layer operates at a different level of abstraction, identifies different types of bugs, and serves a distinct purpose in your quality assurance strategy.
The first line of defence
A static analyser checks your code without executing it, identifying type errors, undefined methods and logical inconsistencies before runtime. Operating at the fastest and most specific end of the spectrum, it analyses individual units of code and their call graphs to verify their correctness against PHP's type system and your documented contracts.
Relationship to other techniques
- Complements dynamic tests by catching bugs that would require dozens of test cases to surface (e.g., passing wrong types through multiple call layers)
- Guides property-based testing by identifying edge cases in type signatures and function contracts
- Informs fuzzing targets by highlighting complex code paths that deserve deeper dynamic analysis
- Reduces mutation testing cost by eliminating obviously incorrect code before expensive test execution
The foundation
Dynamic tests implemented using PHPUnit verify specific behaviours with specific inputs, forming the bedrock of your test suite. Each test case represents a specific scenario, such as “When I call this method with these exact arguments, I expect this exact result”.
Relationship to other techniques
- Validates static analysis findings by confirming that theoretically correct code actually behaves correctly
- Provides seeds for property-based tests: your dynamic test cases become starting points for generators
- Serves as baseline for mutation testing: if mutants survive, your dynamic tests are not good enough
- Creates harnesses for fuzzing: dynamic tests can be adapted into fuzz targets by replacing fixed inputs with mutated data
The limitation of traditional dynamic testing is the human imagination: you only test what you think of. This is where subsequent layers add value.
Generalisation
Property-based testing turns the traditional approach to dynamic testing on its head. Rather than writing specific test cases, you define properties that should apply to all inputs, and then automatically generate hundreds of random test cases.
Relationship to other techniques
- Extends dynamic testing by exploring input spaces humans would not consider
- Validates static analysis by testing type constraints dynamically across random inputs
- Informs fuzzing strategies: generators can be adapted for fuzzing corpora
- Complements mutation testing: property-based testing finds logical bugs, mutation testing validates test quality
Property-based testing operates at a more general level than traditional dynamic tests, yet is more structured than fuzzing in that it still requires you to specify properties. The key insight is that property-based testing bridges the gap between tests specified by humans and fully automated exploration.
Validating test quality
Mutation testing does not find bugs in your code; it finds bugs in your tests. If your tests still pass after your code has been mutated, it means that they lack adequate coverage of that code path.
Relationship to other techniques
- Validates dynamic tests and property-based tests by measuring their ability to detect behavioural changes
- Guides fuzzing efforts: uncovered mutants indicate code paths needing fuzzing attention
- Complements static analysis: static testing ensures code is type-safe, mutation testing ensures tests exercise that code meaningfully
- Quality gate for test suite maturity before investing in expensive fuzzing campaigns
Mutation testing can be slow, as it runs the entire test suite multiple times. However, it provides objective metrics about test quality. A test suite with high mutation coverage is ready for fuzzing, whereas one with low coverage requires more dynamic tests.
Exploratory bug hunting
Fuzzers operate at the most exploratory and slowest end of the spectrum. They generate random inputs to identify crashes, hangs, and unexpected behaviours, without requiring you to specify anticipated results.
Key differences from other techniques
- No oracle required: unlike dynamic tests and property-based tests
- Coverage-guided: unlike random testing
- Finds unknown unknowns: unlike static analysis which finds known bug patterns
Relationship to other techniques
- Explores beyond traditional dynamic tests and finds edge cases you did not think to test
- Validates static analysis and confirms that potential vulnerabilities are actually exploitable
- Benefits from property-based generators which provide excellent seed corpora
- Benefits from mutation testing as fuzzing only reaches its full value once your test suite can reliably detect behavioural changes
The circle is complete: Static analysis identifies complex code, traditional dynamic tests cover basic paths, property-based testing explores input spaces, mutation testing validates coverage, and fuzzing discovers edge cases that inform new static analysis rules.
Conclusion
No single technique is sufficient. Static analysis detects obvious errors at low cost. Traditional dynamic testing with PHPUnit verifies specific behaviours. Property-based testing generalises these behaviours. Mutation testing validates the quality of the test suite. Fuzz testing explores the unknown.
These methods are not hierarchical, but rather form a circular, reinforcing relationship. Begin with static analysis and unit tests, then add property-based testing for critical algorithms. Use mutation testing to measure quality and deploy fuzzing for security-sensitive code. Each layer informs and strengthens the others, creating a comprehensive defence against bugs that no single approach could achieve alone.
