Your tests are lying to you

Your tests are lying to you

Your test suite passes. Your code coverage is at 100%. Your CI pipeline is green. You feel safe.

You shouldn't.

Most test suites create a dangerous illusion: they prove that code runs, not that it works. They execute lines without verifying behaviour. They assert the obvious and ignore the critical. And now, AI coding assistants are exacerbating this problem by generating tests at an unprecedented speed while prioritising coverage metrics over fault detection.

In this presentation, I draw on over 25 years of building PHPUnit, the testing framework behind one of the world's largest developer ecosystems, to show you exactly how tests lie, and what you can do about it.

You will learn how test oracles determine whether your assertions are meaningful or merely decorative. You will see how mutation testing interrogates your test suite by systematically breaking your code to find out which tests actually notice. And you will discover how property-based testing lets you test what you never thought to test.

These are not theoretical ideas. They are practical techniques with mature tooling in every major language ecosystem. And they matter more now than ever: in a world where AI generates code and tests faster than any human can review them, the ability to verify that your safety net is real, not just green, is the skill that separates developers who move fast from those who merely feel fast.

You will leave with a concrete answer to the question every developer should be asking right now: How do I know my tests would actually catch a bug?

This presentation currently exists only as an idea and has not yet been accepted by a conference.

About me

My name is Sebastian Bergmann, and I have been working as a software developer and consultant in the PHP ecosystem for over 25 years. I support development teams in creating better software through modern practices, tools and training.

I am deeply committed to Open Source, maintain critical Open Source projects in the PHP ecosystem, and help shape the work of the PHP Foundation as a member of its board. I have also represented the interests of Open Source at the European Commission and am active in the Open Source Working Group ("Arbeitskreis Open Source Software") of the German Informatics Society ("Gesellschaft für Informatik").

I live and breathe Open Source and help shape how developers around the world test, write, and maintain their code.

More events where you can meet me are listed here.

My interactive online training courses are designed to provide you with practical knowledge that you can apply immediately. The next ones starting soon are listed here.

Upcoming events

More events
Back to overview