I originally sent this article to my subscribers as a newsletter on August 7, 2026. See the end of this article for details.
PHPUnit 13.3 has two focal points. The first is dealing with unreliable tests: the test runner can now run each test repeatedly to track down flaky tests, and retry failing tests where flakiness cannot be eliminated. The second is the HTML report for code coverage: a new view of namespaces and classes, a much better presentation of branch coverage and path coverage, and a filter by test size.
Detecting flaky tests
A flaky test sometimes fails and sometimes does not, even though the code under test has not changed. Typical causes are shared mutable state, time-dependent logic, non-deterministic ordering, and resources that are not released. The most reliable way to track down such tests is remarkably simple: run the same test many times in a row.
Up to PHPUnit 9 there was an option called --repeat. It repeated the entire test suite, however, which made it more of a benchmarking tool. That is why I removed it in PHPUnit 10. The request to bring it back (#5718) became one of the most heavily discussed issues in PHPUnit's issue tracker. At the Code Sprint in Munich in October 2024, a new consensus emerged: --repeat should return, but repeat each individual test rather than the suite.
That is exactly what PHPUnit 13.3 does (#6591). With --repeat 100, every test runs up to one hundred times back to back. When a repetition fails, the remaining repetitions of that test are skipped. Each repetition runs as a test in its own right and identifies itself in the output as (repetition N of M). This helps with more than flakiness: code that manages connections, caches, or open files may behave correctly on the first call and only leak state over many calls.
With the #[Repeat] attribute you mark individual test methods; it was modelled on the @RepeatedTest annotation from JUnit 5. #[Repeat(100)] repeats a test up to one hundred times and stops at the first failure. The optional second parameter controls how many failures may accumulate before the remaining repetitions are skipped. #[Repeat(100, 5)] keeps repeating until five failures have been observed. That is useful when the failure pattern itself is the information you are after: how often, and on which repetitions, a test fails. Importantly, the threshold does not make failures acceptable. Every failed repetition is reported as a failure and fails the test run.
Not every test can be repeated. The return value of a test method can be passed to other tests via #[Depends], and repeating it would produce several, potentially different return values. That is why only test methods that explicitly declare void as their return type and do not depend on other tests are repeated. With --repeat, all other tests run exactly once, as before. When the #[Repeat] attribute is placed on a method that does not meet these conditions, however, the test runner emits a warning. When a test uses a data provider, each data set is repeated independently of the others.
Tolerating flaky tests
Not every flaky test can be fixed. Some tests interact with network services, hardware, or processes whose timing they cannot control. Such a test failing once is not information. It failing repeatedly is. Until now, restarting the entire CI job was often the only option. That is expensive, and it obscures which test was the problem.
PHPUnit 13.3 offers a better alternative (#6742). A test with the #[Retry(3)] attribute is run up to three times. The first attempt that is neither a failure nor an error decides the result. A skipped or incomplete attempt therefore ends the loop as well. Every attempt runs on a fresh instance of the test class, in line with PHPUnit's “one instance per test” guarantee. With the --retry command-line option, this behaviour applies to all eligible tests, and the conditions are the same as for repetition. An attribute on the test always takes precedence over the command-line option. Using --repeat and --retry together results in a warning, as does putting #[Repeat] and #[Retry] on the same method.
The usual objection to built-in retries goes: they institutionalise flaky tests. A race condition that strikes thirty percent of the time passes CI forever and is never fixed. The implementation in PHPUnit 13.3 answers this objection with unconditional visibility. Every test that only passed after one or more failed attempts is listed in the summary of the test run, together with the number of its failed attempts. This cannot be disabled. A test that exhausts all attempts fails just like any other; the failure message of the final attempt identifies itself as (attempt 3 of 3). A test that passes on the first attempt, which should be the normal case, produces no retry-related output anywhere.
Code coverage through the lens of your classes
Until now, the HTML report for code coverage showed your code the way it sits on disk: as directories and files. But you rarely think about code in terms of files. You think in namespaces, classes, and methods.
PHPUnit 13.3 therefore adds a class-oriented view (#1140). Every page of the report now shows two tabs: Files and Classes. The Files tab shows the report as before. The Classes tab organises the same data by namespaces and classes. The two views are linked to each other: from the page of a source file, the Classes tab takes you straight to the page of the class declared in it, and vice versa.
The heart of the feature is the class pages. Every class gets its own page with a summary table of its metrics, including the CRAP index per method, and its annotated source code. Only the lines that belong to the class are shown, not the whole file. Added to that are separate sections for every trait the class uses and for every method it inherits from parent classes; overridden methods are not repeated. A class page thus answers the question “How well is this class tested?” for the code the class actually consists of at runtime, without you having to jump between several file pages.
Namespace pages aggregate the metrics of their classes and sub-namespaces, just like the directory pages of the file-oriented view do. When all code shares a common root namespace, redundant intermediate levels are collapsed. And like the file-oriented view, the class-oriented view has a dashboard with bubble charts for coverage and complexity.
Two things are worth knowing. The two views deliberately aggregate differently: in the class-oriented view, code from traits and parent classes counts towards the class in question. The totals can therefore deviate from those of the file-oriented view, and functions as well as code outside of classes are only visible in the file-oriented view. Furthermore, the class-oriented view is generated by default and roughly doubles the size of the report. With the new --without-class-view and --without-file-view command-line options, or the classView and fileView attributes in the XML configuration file, you can disable either of the two views.
Making branch and path coverage visible
The branch coverage view now makes decision points in the code visible (#1141). A new gutter column shows, on every line where control flow branches, one dot per possible outcome: in the colour for covered code when a test executed that outcome, and in the colour for uncovered code otherwise. This lets you spot at a glance an if whose else case never runs, even when the line itself appears covered. The section below the source code has been redesigned. Instead of repeating the source code of every branch, there is now a compact table per method with one row per branch, plus a badge such as 3/4 on every method heading.
The path coverage view has been reworked in the same way and additionally gains a control flow graph per method. The graph shows the method's branches as nodes, connected by arrows from entry to exit, coloured by coverage. When you click on a row in the paths table, the corresponding path is highlighted in the graph. This shows you exactly which route through the method a path describes. Rendering the graphs requires the dot tool from Graphviz; when it is not available, the report is generated without graphs. Methods with more than 100 paths were previously not listed at all. Now their first 100 paths are shown in a collapsed section, together with a note stating the total number.
Filtering code coverage by test size
With the #[Small], #[Medium], and #[Large] attributes you declare the size of your tests. PHPUnit has always recorded this information in the code coverage data, but until now the HTML report only used it to colour covered lines. Now you can filter the entire report by it (#1153).
Every directory and file page shows a button group with Small, Medium, Large, and All. When you select one or more sizes, all numbers on the page are recalculated: bars, percentages, and the counters for lines, methods, and classes. A class can show one hundred percent coverage overall even though only a fraction of it comes from small tests and the rest is only reached through large end-to-end tests. One click on Small shows you what your unit tests cover on their own. Under an active filter, a method only counts as tested when tests of the selected sizes fully cover it on their own, and a class only counts as tested when that holds for all of its methods.
All and the combination of Small, Medium, and Large are deliberately not the same. All ignores test sizes entirely and counts every covered line. The combination of the three sizes only counts lines that are covered by at least one test with a declared size. The difference between the two shows you how much of your coverage comes from tests that do not declare a size.
Exclusive insights delivered to your inbox
Every two months, alongside each PHPUnit feature release, I send subscribers a detailed walkthrough of the new features: what they do, how they are implemented, and why they were added. It is the context that the ChangeLog does not cover.
I publish the content of the newsletter here on this website a month after my subscribers have received it. Subscribe now to get it as early as possible.
