Order and Focus: What is new in PHPUnit 13.4?

I originally sent this article to my subscribers as a newsletter on 2 October 2026. See the end of this article for details.

PHPUnit 13.4 lets mock objects record the order in which their methods are invoked, a feature I turned down more than three years ago. I added it because the replacement I had documented for id() and after() did not work, and because some classes promise an order. I still consider most tests that assert such an order a mistake. The other two changes shorten the test run you start dozens of times an hour while working on a test: PHPUnit no longer has to load every test file before it can run the few tests you selected, and it can run the tests you changed last first.

The journal records, the test asserts

In May 2023, someone asked for a way to specify the expected order of method calls across several mock objects (#5357). jMock, Mockito and Mockery all have such a feature, the issue argued. I closed it within the hour with a single sentence: “I do not think that relying on call order makes sense.” For most tests that want such a feature, I still think so.

When the request came up again in September (#6959), two things made me revisit that decision. The first is my own advice. When I soft-deprecated id() and after() in PHPUnit 13.1 (#6537), I recommended withParameterSetsInOrder() for tests in which the order of invocations is significant. That advice was wrong. withParameterSetsInOrder() orders the parameter sets of one expectation and says nothing about which method was invoked. Even when a single expectation covers both beginTransaction() and commit(), invoking commit() first does not make the test fail. So id() and after() were on their way out with a documented replacement that did not replace them. The second is that there are classes whose observable behaviour is an order, and for those the order is the specification.

PHPUnit 13.4 therefore adds invocation journals (#6960). createInvocationJournal() creates one, and recordInvocationsIn() registers it with a mock object. From then on, every invocation of a method of that mock object is recorded in the journal, in the order in which the invocations happen:

<?php declare(strict_types=1);
use PHPUnit\Framework\TestCase;

final class ExporterTest extends TestCase
{
    public function testWritesAllRecordsWithinOneSession(): void
    {
        $journal = $this->createInvocationJournal();

        $writer = $this->createMock(Writer::class);
        $writer->recordInvocationsIn($journal, 'writer');

        $writer->expects($this->once())->method('open')->willReturn(true);
        $writer->expects($this->exactly(2))->method('write')->willReturn(1);
        $writer->expects($this->once())->method('close');

        $exporter = new Exporter($writer);

        $exporter->export(['a', 'b']);

        $this->assertSame(
            [
                'writer::open()',
                'writer::write()',
                'writer::write()',
                'writer::close()',
            ],
            $journal->asArray(),
        );
    }
}

Each entry consists of a label for the mock object and the name of the invoked method. Without a label, the fully qualified name of the doubled type is used. The optional third argument of recordInvocationsIn() maps method names to labels of their own. Any number of mock objects can share a journal, which puts their invocations on one common timeline. asArray() returns all recorded invocations. only() returns just the ones with the labels you pass, which is what you need when only the relative order of a few invocations matters.

Until now, willReturnCallback() was the only way to observe an invocation. It takes the place of willReturn(), however, so a test that wanted to know the order had to return values by hand, from closures that also appended to an array passed by reference. With a journal, recording is independent of how a method is configured. willReturn() and the other will*() methods keep working, and so do once(), with() and the rest of the expectations.

A journal asserts nothing. It records what happened, and you state what should have happened with an ordinary assertion. This was the most important design decision. jMock and Mockery attach expectations to a sequence object instead. With such a sequence, the order in which the expects() lines of a test are written silently becomes part of the assertion. PHPUnit would also have to decide what a sequence means for a method that is expected more than once, for a sequence that is only partially consumed, and for several sequences that interleave. withParameterSetsInOrder(), withParameterSetsInAnyOrder() and withParameterSetsInPartialOrder() have shown me how quickly this problem space grows once PHPUnit owns the assertion. With a journal, you choose how strict the test is: assertSame() for a total order, assertEqualsCanonicalizing() when only the set of invocations matters, only() for a relative order. When the assertion fails, you get a diff of the two lists, reported for a line of your test. A sequence could only report that it expected a different step, from deep inside PHPUnit's invocation handler.

My first implementation attached recording to an expectation, as proposed in the pull request. Before the release, I moved it to the mock object. The journal now records every invocation, before the configured behaviour runs, whether or not an expectation matches it, and even when the invocation violates one. That keeps the journal a faithful log of what happened, independent of how the expectations are configured. Journals are available for mock objects only, not for test stubs. The only reason to observe the order of invocations is to assert it, and an assertion about how a test double was used belongs on a mock object.

Assert an order only where it is part of the contract

Before you write such a test, ask yourself one question: is the order part of a documented contract, of the class under test or of the interface of its collaborator, or is it an accident of how the method happens to be written today? If the order changes and no documented promise is broken, the test should not fail.

I see two cases in which the answer is clear. The first is a class that exists to put things in order, for example an event dispatcher that promises to invoke its listeners in order of their priority. There, the order is the behaviour under test. The second is a collaborator whose interface documents a protocol, like the Writer from the example above, which has to be opened before anything is written to it and closed afterwards.

The example that led to #6959 is a middleware that begins a database transaction, hands control to the next middleware and commits the transaction afterwards. A journal-based test for this looks obvious, but it is a weak one. A test that asserts that beginTransaction() was invoked before next() also passes for a middleware that begins a transaction, commits it, and begins a second one before it delegates. What the test actually wants to know is whether a transaction is open, and not yet committed, while the next middleware runs, and that nothing is committed when the next middleware throws an exception. Both can be asserted directly: the first from within the test double for the next middleware, the second with expects($this->never()) for commit(). The order is only a symptom of that invariant.

The other case I consider a mistake is pinning down the order of collaborators that have nothing to do with each other. When a class writes to a log, puts a value into a cache and increments a metric, nothing breaks if these three calls change places. A test that asserts their order fails on refactorings that change no behaviour. It also tells whoever reads it that an accident of the current implementation is a promise. That PHPUnit now has invocation journals does not make such a test a good one.

If your tests still use id() and after(), an invocation journal is now their replacement. Both methods will trigger a deprecation in PHPUnit 14 and will be removed in PHPUnit 15.

Focused test runs no longer have to load every test file

When you run a small part of a large test suite, with --group or --filter, PHPUnit has so far spent most of its time loading test files whose tests it then filtered away. Before it can decide that a test file has nothing to contribute, it has to load that file, and loading test files, including running their data providers, is most of the work.

PHPUnit 13.4 can remember which tests each test file contains and which groups they belong to (#6863). On a later run, it does not load a file at all when it can prove that none of its tests would be selected. This test index is off by default. You enable it with cacheTestIndex="true" on the <phpunit> element of the XML configuration file or with the --cache-test-index option, and --do-not-cache-test-index disables it for a single run. The index is stored in the cache directory that the test run history and the static analysis cache for code coverage use as well. When no cache directory is configured, PHPUnit emits a warning instead of quietly doing nothing.

I measured the effect on PHPUnit's own unit tests, 524 test files, with a warm index and without one:

For the whole test suite, there was no measurable difference. The saving is in test discovery: the fewer test files can contribute to a run, the fewer have to be loaded. That is why it is most noticeable in the narrow runs you start again and again while you work on a test, and why a pipeline that runs every test gains nothing from it. Building the index on the first run costs time, and the second run pays that back.

A cache that makes a test run faster is worthless if it changes which tests run. A file is only skipped when the index can prove that nothing in it could have been selected, and both the selection by group and the selection by name are consulted for this. For every measurement above, I checked that the same tests ran in the same order as without the index. An entry is only used as long as every source file it was derived from still has the content it had when the entry was written. This includes parent classes and traits, so adding a test to an abstract base class or changing a group on a trait is noticed in every test class that inherits from it. A file that PHPUnit warns about while loading it is never skipped. Otherwise, a warning would appear on the first run and silently disappear on the second. And a file that could not be loaded is never remembered, so a run that fails does so again instead of skipping past the problem.

What the index keeps the same is which tests run and what PHPUnit reports about them. The events a test run emits are a different matter. A skipped file runs none of its data providers, so the events for these data providers are not emitted either, which you will notice with --debug or in an event log.

The test index is also groundwork. It is the first part of PHPUnit that knows which source files each test class is made of, and that notices by their content when one of these files has changed. Test impact analysis (#6919), which I am working on for PHPUnit 14.0, needs exactly this knowledge. It records which source files each test depends on and then runs only the tests that a change can affect. A test whose own code has changed has to run no matter what else is known about it. To find out which files a test class is made of and whether one of them has changed, test impact analysis uses the code that was written for the test index.

The tests you just changed run first

While you write a test, it might be at the end of the queue. To find out whether it passes, you wait for everything you did not change to run first. --stop-on-failure does not help, because it stops at the first failure, not at the first one you are interested in. None of the orders PHPUnit offered so far express the work you are doing right now: duration-ascending and defects describe the previous test run, size-ascending and size-descending describe what a test declares about itself, and random and reverse describe nothing.

PHPUnit 13.4 adds two orders for the --order-by option and the executionOrder attribute of the XML configuration file (#6957). modified-descending runs the tests whose source files were modified most recently first, and modified-ascending does the opposite. With phpunit --order-by modified-descending --stop-on-failure, the tests you have just worked on run first, and the run stops at the first failure among them. Like every other order, it can be combined with moving the tests that failed in the previous run to the front: --order-by modified-descending,defects. Mind the order of the two tokens. PHPUnit 13.4 deprecates writing defects first, because in PHPUnit 14 the tests will be reordered in the order in which the tokens are written.

What counts are the file that declares the test class, the files of the classes it extends, and the files of the traits that any of them use. This is the same answer to the question of which files a test class is made of that the test index works with. When you change an abstract base class or a shared trait, every test class built on it moves to the front. For a PHPT test, the .phpt file is the test. A test suite is as new as the newest test it contains, so directories and <testsuite> elements are ordered sensibly as well. Unlike ordering by defects or by duration, this order needs no test run history. It works on the very first run and with --do-not-record-test-run-history.

The modification time of a file records when it was last written, not who wrote it or why. Checking out a branch, cloning a repository, installing dependencies and deploying files all rewrite it. On a continuous integration server, where all files are usually written at the same time, this order carries no information at all. I therefore consider it a tool for local development that belongs on the command line and not in the XML configuration file you commit. Many file systems store modification times with a resolution of one second, so tests whose files were written in the same second are equally new and keep the order they had before.

Exclusive insights delivered to your inbox

Every two months, alongside each PHPUnit feature release, I send subscribers a detailed walkthrough of the new features: what they do, how they are implemented, and why they were added. It is the context that the ChangeLog does not cover.

I publish the content of the newsletter here on this website a month after my subscribers have received it. Subscribe now to get it as early as possible.