The Productivity Illusion

Why your sharpest engineers feel fastest at the exact moment they slow down

In early 2025 a research outfit called METR ran the experiment the vendors would never have paid for. They took sixteen experienced open-source developers, the kind who had lived inside the same repositories for five years and a thousand commits, and gave them 246 real tasks from their own projects. Before each task, a coin decided one thing: AI assistant allowed, or not.

The developers made a forecast first. AI would speed them up by about 24 percent. A safe bet. By then everyone knew the tools were remarkable.

The stopwatch said otherwise. With AI, they were 19 percent slower.

Now the detail worth framing on the wall. After the study, having just lived through the slowdown, those same developers reported that AI had made them roughly 20 percent faster. They had been measured. The measurement missed their intuition by almost forty points. And they believed the intuition.

This is the Productivity Illusion, and it has a simple engine. What the tools remove is friction, not time. The blank page fills itself. The boilerplate appears. Your fingers stop aching and your mind stops straining, and the brain, a poor accountant, files "less effort" under "more speed." Meanwhile the real clock keeps running through the parts nobody counts: reading the suggestion, distrusting it, re-prompting, throwing it out. In the METR study the developers discarded more than half of what the AI produced. The work felt smooth. It measured long.

We have known for a century that the inner sense of how a task went is a cheap instrument. Pilots are trained to trust the dials over the seat of their pants, because in cloud the inner ear invents a horizon that isn't there. The developer with an AI assistant is flying in cloud. The feeling of banking gently level is exactly when the plane is in a spiral.

Here is the cruel part for anyone running a team. The illusion is strongest precisely where you would least choose to be fooled: on large, old, complicated codebases, the ones your best people know cold. That is where METR found AI helped the least, and it is where the fluent, confident feeling runs highest, because now two fluencies stack, the expert's ease with the tool and their ease with the code. The person most certain the tools are transforming their work is often the person they are quietly slowing down.

Watch what this does to how you learn about your own organization. You ask your engineers how the rollout is going. They tell you, sincerely, eyes bright, that it is a revolution. That is not evidence. That is the illusion filing its own status report. When a chief executive stands on an earnings call and describes an engineering team remade by AI, listen carefully. You are usually hearing the inner ear, amplified to shareholders.

So the discipline is dull and non-negotiable. Measure outcomes, not sentiment. Cycle time, throughput, escaped defects, on real tickets, with the tool and without. Survey the feeling if you like, but never mistake the survey for the truth. Feeling is the thing the illusion controls. The numbers are the thing it cannot touch.

And then, the part that saves this from mere cynicism. A year later METR revisited the question and found the same category of tools now producing something like an 18 percent speedup. The tools had improved. More to the point, the people had learned to use them. Both findings are real. The slowdown was real, and so was the eventual gain. The error is to let the warm early feeling talk you out of the cold measurement, and so skip the year of unglamorous practice that turns the first number into the second.

The feeling of speed is handed to you for free, on the first day, whether or not you have earned it. Actual speed you have to earn, and then prove, because on the day you finally have it you will not feel a thing.