In Faster than understanding I described an experiment: I let an AI coding agent implement the ACPATH metric, an algorithm from a domain in which I am not an expert. The agent needed 15 minutes. I then spent hours trying to determine whether the implementation was correct, and I still could not be certain. My conclusion was that a fundamental asymmetry emerges when domain expertise is missing: an AI agent can generate code faster than we can understand the domain the code belongs to.
Since publishing that article, I have come across three observations. None of them were written in response to mine, but each looks at the same phenomenon from a different direction, and each made me see something I had not made explicit.
Alberto Brandolini looks at the organisational level. Owning a codebase, he argues, is a balance of confidence and accountability: writing and testing software is what gave developers the confidence to accept the accountability. AI-assisted development disturbs this balance, because codebases now grow faster than confidence does, while at the same time some organisations shrink their development teams and concentrate ever more accountability in fewer senior roles. His conclusion is blunt: “increased accountability with lowered confidence isn't sustainable”. When the balance breaks, leaving becomes a rational career move.
Zsófia Herendi, responding to Brandolini, adds the individual level. AI, she observes, does not only increase accountability faster than confidence. It also increases the feeling of confidence faster than actual competence. A product manager can now generate impressive-looking roadmaps and research summaries with ease, without becoming any better at understanding the problem. Worse, AI removes many of the moments in which judgement used to develop: challenging assumptions, discovering that you had the problem wrong, having difficult conversations. Those moments were slow and often frustrating, but they quietly built the intuition professionals later relied on. As she puts it: “AI can accelerate output but it cannot shortcut experience”.
Andreas Bulling, who researches and teaches artificial intelligence at a German university, adds the generational level, and it is the most sobering one. In a widely shared post he describes what he calls a “wave of incompetence” rolling towards the labour market: coursework, programming exercises, even entire theses are now largely AI-generated. Ask these students a single follow-up question about their own code, and there is no answer. A generation is entering the workforce, he warns, whose only genuine skill is operating an AI. His prediction: the few who resist this trend, who genuinely work things out, understand them, and think them through, will soon be able to pick any job they want.
The organisation, the individual, the next generation: these are three different levels, but I see the same pattern on all of them. I think this pattern deserves a name.
Borrowed confidence
There are two kinds of confidence in software development, and they are easy to confuse because they feel identical from the inside.
Earned confidence is the product of friction. You wrote the test first and watched it fail. You were sure you knew where the defect was, and debugged your way to the place where you were wrong. You explained your design to a colleague and could not answer their third question. These are exactly the slow, frustrating moments Zsófia Herendi describes, and they are where judgement comes from.
Borrowed confidence is what my ACPATH experiment produced. The code looks clean. The tests pass. The visualisations are plausible. The explanation is articulate. But every one of these signals was generated by the same agent that wrote the implementation. The tests do not verify the code against the paper; they merely show that the implementation agrees with itself.
I have started to think of this as confidence laundering: plausibility goes in, apparent certainty comes out, and at no point does independent evidence enter the loop.
The asymmetry has an age
Here is what these three observations made me realise: borrowed confidence plays out differently depending on where you are in your career.
I experienced the asymmetry for a few hours, and I noticed it. I noticed it because I have more than 35 years of earned confidence to compare against. I know what solid ground feels like, which is why I could feel it missing under the ACPATH implementation.
The mid-career practitioner, Brandolini's senior developer with a growing scope or Herendi's product manager under delivery pressure, still has judgement, but AI removes the occasions to exercise it. Earned confidence is less a savings account than a muscle: when it is not exercised, it does not simply stop growing, it decays.
And the newcomer Bulling describes has never earned any confidence at all. For someone whose entire education consisted of relaying prompts, looking right and being right have never been separate categories in the first place. My uncomfortable few hours are their permanent condition, minus the discomfort. The asymmetry does not hurt those it damages most.
Brandolini's observation closes this loop grimly: the organisations that shrink their teams are removing exactly the senior people who could have supervised the newcomers into competence.
Earning confidence at AI speed
So far, so bleak. But confidence can be earned even in a domain you do not master. You just cannot borrow it from the system you are trying to verify. You need independent evidence. Since publishing the first article, I have been building exactly that for the ACPATH implementation.
The first source of independent evidence is a brute-force oracle. The ACPATH algorithm is clever; that is what makes it hard to verify. But a naive path enumerator, one that literally walks every acyclic path through a small control flow graph and counts them, is the kind of program Sir Tony Hoare had in mind when he spoke of software so simple that there are obviously no deficiencies. I can write that one myself, and I can understand every line of it. It is far too slow for real code, but it does not need to be fast. It needs to be obviously correct. Running both implementations against thousands of small, generated functions and comparing the results is differential testing: the clever implementation is checked against the simple one, not against itself.
The second source is properties derived from the paper rather than from the code. Sequential decisions multiply path counts: appending an if statement to a straight-line function must double its ACPATH value. Wrapping a function body in a condition must change the count in a predictable way. These metamorphic relations come from the specification, so they are independent of the implementation by construction. An implementation that satisfies hundreds of such relations across generated inputs may still be wrong, but it is wrong in a much smaller space.
The third source is mutation testing. If I introduce small defects into the AI-generated implementation, does the AI-generated test suite notice? If the answer is no, then the passing test suite was never evidence of anything, and the confidence it produced was borrowed all along.
None of this is free. It is slower than accepting the agent's output, and much slower than the 15 minutes the agent needed. But it converts borrowed confidence into earned confidence, and it is dramatically faster than becoming a domain expert first. That, I think, is the honest version of the productivity promise: AI does not remove the intellectual work of verification. It changes what that work looks like.
The scarce skill
Andreas Bulling predicts that people who genuinely understand things will soon be fiercely sought after. For software development, I would make that prediction more concrete. The scarce skill will be producing justified confidence in code: knowing what would count as evidence, constructing oracles, deriving properties from specifications, and recognising when your certainty comes from understanding and when it merely comes from a very convincing answer.
This puts a practice many consider old-fashioned in a new light. Test-driven development was never primarily about catching bugs. Writing the test first forces you to write down the expected behaviour before a solution exists to seduce you. It is deliberate practice for judgement: the kind of slow moment that AI-assisted workflows optimise away, and that we must now preserve on purpose.
The typing can be delegated, and perhaps even the design. The accountability cannot, and confidence, the only thing that makes accountability bearable, is not transferable. It has to be earned by each person, in each domain, every time.
