The Boundary Was the Hard Part
A benchmark table sometimes carries two rows for the same model: one score with the task boxed in by a schema, an allowlist, a validator that rejects malformed output before it reaches a grader, and one score with all of that stripped away. The first number often lands in the high 90s. The second can sit forty points lower. Read as a measure of capability, that gap looks damning. The model was never that good. The guardrails were carrying it.
That reading treats the guardrail like scaffolding, removed once the building stands on its own. It's closer to a second worker on the same job.
Think about what a guardrail actually does across a multi-step task. It enumerates which moves are legal, so the model never has to reconstruct the boundary from context clues. It catches a malformed step before it compounds into three more malformed steps downstream. It restates the target after every turn, so drift gets corrected within one step instead of accumulating over ten. None of that is the model reasoning about the problem. It's a second process doing continuous maintenance on the shape of the task, running in parallel with whatever the model is doing to solve it.
Take the guardrail away and that maintenance doesn't disappear. It has to happen somewhere, and now the only thing left to do it is the model. It has to infer the boundary it used to be handed. It has to notice, without an external check, when it has wandered past that boundary. It has to correct course using its own judgment about what "past the boundary" even looked like, which is a much harder thing to detect from the inside than from a validator sitting outside the loop with the spec in hand. That's not one job anymore. It's two jobs, stacked in the same context window, and the second one was never being graded when the rail was up.
So the forty-point gap isn't asking "how much smarter is the model with rails on." It's asking how much of that first number was boundary maintenance rather than task reasoning, wearing the same score. A guarded run and an unguarded run aren't the same test with different difficulty settings. They're different tests. One measures whether the model can solve the problem given the shape of the problem. The other measures whether it can solve the problem and discover the shape at the same time, unassisted, and it's fair to expect the second to look a lot worse even when the underlying reasoning hasn't moved an inch.
Call it borrowed accuracy: the portion of a guarded score that belongs to the harness, credited to the model because nobody separated the two. Once you name it, a few things fall out. Two models can post identical guarded scores while one would barely dip without the rail and the other would fall in half, because the rail was absorbing a much bigger share of the first model's job than the second's. The guarded number can't tell them apart. It's not built to. The gap between the two runs, guarded minus unguarded, is where that difference actually shows up, and it's arguably a more honest signal than either score taken alone.
There's a design implication buried in this, not just a scoring one. Handing a system more autonomy doesn't just remove a constraint, it removes a worker. Every bit of boundary discipline that used to live in the harness now has to live in the model, on top of whatever it was already doing. A team deciding how much rail to strip off an agent is really deciding how much of that second job to reassign, and the honest way to check the decision is to look at exactly this gap, not the guarded score in isolation, which will keep looking fine right up until the rail comes off.
The guardrail was never just a fence keeping the model inside the field. It was out there walking the fence line, and the field only looked easy because someone was doing that.