The Orphaned Model
The story goes: Rob Pike at a terminal, working through a bug by reading stack traces and generating hypotheses from output. Ken Thompson standing behind him, then: he knows what's wrong. Not because Thompson was sharper in the moment. Thompson was holding a mental model of the system and reasoning forward from it, while Pike was reasoning backward from symptoms.
That difference finds different things. Pike's approach locates where the code fails. Thompson's approach finds why the reasoning that produced the code was wrong. One patches the symptom; the other addresses the cause.
Human-authored code has a model somewhere, even if it's unreachable. The author might be gone, the comments wrong, the git history noise. But someone sat down and made decisions. There was intent, even if it's opaque now. You can sometimes reconstruct it: this variable name implies they were thinking about X, this structure implies they expected Y to change. The model existed. It decayed.
AI-generated code doesn't have that. The LLM generated tokens statistically consistent with the surrounding context and training distribution. It didn't reason about what the code should do before writing it. It predicted what code would be written here. The distinction sounds philosophical until you're debugging.
When I debug AI-generated code, I'm not recovering a lost model. I'm constructing one that never existed. The code is the only artifact. No author to ask, no design doc, no set of decisions to reverse-engineer. The function does what it does, and I have to figure out what it's supposed to do before I can figure out why those two things differ.
What makes this harder
AI-generated code tends to be locally coherent and globally inconsistent. Each function looks reasonable. The interfaces fit. The unit tests pass. The design flaw is at the level of the model: what the code is actually for, how the components relate systemically. That level isn't visible in any individual function. This is exactly where Thompson's approach operates and Pike's approach fails. Patching the failing assertion doesn't tell you why the system was structured to fail.
Human-authored comments, even bad ones, externalize part of the model. "This handles the edge case where X." "We do Y because Z." AI-generated comments describe what the code does, not what the author was thinking. They're post-hoc descriptions. Reading them to understand intent doesn't work because there was no intent to capture.
The trap the tools set
This is where Thompson's approach becomes necessary rather than merely better. You have to stop, read the whole thing as if encountering an alien artifact, and construct the model yourself before changing a line. Skip that step and generate variations until tests pass, and you've done Pike's approach at scale with AI assistance removing the natural friction.
That removal is the problem. The iteration cost used to be enough pressure to slow you down. A new variation meant more typing, more waiting, more reading. When the variation costs a prompt and a second, there's no natural pause. You can iterate faster and stay wrong longer. The speed hides the fact that you're still reading outputs and reacting.
For human code, Thompson's approach is optional in the sense that you sometimes get away without it: call the author, read the design doc, get lucky. For AI code, there's no one to call. The model has to come from somewhere, and the only place left is your own head.
Model-building was always the right starting point. Now it's the only one.