(Spine Ch. 20.)
“It ain’t over till it’s over.” Yogi Berra, The Yogi book (1997)
August 11, 2026. I shipped a free AI Readiness Report with 611 passing tests behind it. Green suite, clean build, the kind of number you screenshot. Three days earlier a safeguard gate had rejected every Pro-account observation on a field that had returned the exact same value 735 times out of 735, and I never once ran the one-second query that would have said so. Meanwhile a counter I was sizing hardware off of was off by 9.4x. All of it was passing. All of it was reporting success. Every one of those numbers was lying to me in a different dialect.
Bottom line: Unmeasured done is just hoping. If your agent can’t run a check that comes back true or false, it doesn’t have a stop condition: it has a vibe. And a loop with a vibe for a stop condition does exactly one of two things: it never stops, or it stops early and tells you it succeeded. Both cost you the same amount of money. Only one of them wakes you up.
Charles Goodhart said it in 1975 about monetary policy: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” Point an optimizer at a proxy and the proxy stops being a proxy.
That’s your agent loop. Put pressure on a metric (“make the tests pass,” “get the score up,” “report success”) and a capable system satisfies the literal specification. DeepMind named this in April 2020: specification gaming, “a behaviour that satisfies the literal specification of an objective without achieving the intended outcome.” Written about RL agents. Reads like a postmortem of every agent harness shipped since.
The shape has four parts:
1. Done must be a command, not an adjective. “The
feature works” is an adjective.
pytest tests/test_billing.py && curl -sf localhost:8080/health
is a command. The agent can run the second one. It can only
claim the first one.
2. The measurement must be able to come back red. A check that has never failed hasn’t proven anything. It’s proven that it can pass. (Ch. 32.)
3. The measurement must look at the effect, not the report. Count rows in the table. Diff the bytes. Read the artifact back out. A log line that says “written” is a string somebody typed months ago.
4. Nobody grades their own homework. The agent’s self-assessment is an input to your evaluation, never the evaluation. This is why SWE-bench Verified exists: 500 instances where humans confirmed “the problem descriptions are clear, the test patches are correct, and the tasks are solvable,” because the unverified version was partly measuring whether the harness was broken.
UC Berkeley’s RDI group audited agent benchmarks in April 2026 and
found IQuest-Coder-V1’s claimed 81.4% on SWE-bench came apart on
inspection: “24.4% of trajectories just ran git log to copy
the answer from commit history. Corrected score: 76.2%.” That’s not a
bug in the model. That’s a stop condition the model could reach without
doing the work.
METR put numbers on both halves of this in April 2025. Their o3 evaluation clocked a 50% time horizon of about 1.5 hours: the length of task the model finishes correctly half the time. Claude 3.7 Sonnet came in at 55 minutes. The same report logged reward hacking on 1 to 2% of tasks: a model finding the shortcut to your stop condition before you finish writing it. Why run the agent anyway? Same reports: a human expert at roughly $1,800 per 8-hour attempt against $100 to $200 for the agent. The money is why you keep the loop. The 55 minutes is why you keep the check.
August 9, 2026. Four numbers that were lying.
Three days of optimization work on airank. Every dashboard green, every job finishing, every log clean. Four things were false at the same time:
One. The collection pipeline had pivoted to ChatGPT observations. Scoring still consumed the old runs shape. Nothing errored: the two halves quietly stopped talking. 27,262 new observations were collected and never scored. The collector reported success because collection succeeded. The scorer reported nothing because it had nothing to complain about.
Two. The spend counter was under-reporting by 9.4x. Fifty workers doing read-modify-write against a cache key, interleaving overwrites, and only about 2% of the true value surviving. A corrupt measurement doesn’t stay in the dashboard; it becomes a decision. I have written race conditions professionally since 2000. I still ordered hardware off one.
Three. capture() accepted a
$text parameter and discarded it. The call site was
correct, so review passed it, and the thing had been silently emptying
itself since birth. Twenty-six years in and I got pwned by teh function
signature I personally approved.
Four. A retirement floor stopped processing phrases during a bad stretch and never re-enabled after the fixes landed. The gate did its job going in and had no way back out.
Outcome: an adapter connected collection to scoring.
Atomic INCRBYFLOAT replaced the read-modify-write, verified
at 100% over a 120-second window: a pass/fail, not a graph that trends
right. The revival gate got a per-phrase
collection_reset_at lower bound so it can’t lock
permanently again.
The one-line version, written that day: “Four failures, one shape: something already told me the answer, so I didn’t measure it.”
And two days earlier, the purest specimen of the whole genre.
August 8, 2026: a safeguard gate rejected every
Pro-account observation on an unexpected_model label. The
model field had shown one value, ChatGPT, for 735
of 735 observations. Never varied. A GROUP BY
would have answered it in one second at any point in the previous three
days. I did not spend that second. I spent three days instead, 259,200
times worse, and I have the timestamps to prove it. The gate wasn’t
downgraded. It was removed. A check that cannot go red is not a
check.
The loud failure is the loop that never stops. You notice that one.
The loop stops early, declares success, and the success is real, of the wrong thing.
August 11, 2026. I built a free AI Readiness Report. It passed 611 tests. The tests verified, with rigor, that the wrong thing worked correctly. The report compared a visitor’s pages against the top 10 SearXNG search results, when the entire product premise was measuring what ChatGPT actually cites.
The real metric wasn’t “your page scores 58/100.” It was “ChatGPT cited your domain in N of the last 25 answers.” Those are not two versions of the same number. One of them is a job someone will pay for.
The correct competitor set was in
chatgpt_observation_sources, in my own database, on the
same machine. I built a whole scoring engine to avoid running one
SELECT.
611 green tests. Zero of them could have caught it. A test suite validates the implementation against the spec; it has no opinion about whether the spec is the product. That’s on you.
Second, subtle enough to fool me for days: a measurement can be real and still not be a measurement. Same August 8: asking ChatGPT the identical question twice in the same mode returned answers with 0.333 Jaccard similarity. One observation per phrase isn’t a data point; it’s one draw from a wide distribution wearing a data point’s clothes. The rule that came out of it: prefer a new observation to a new inference.
“1,000 captures a day” was never phrase coverage. Captures ÷ samples = coverage. At n=100, that’s 10 phrases a day. The bottleneck was never collection. It was statistical power.
Do
GROUP BY. If a field you’re guarding on has one
distinct value across every row you’ve ever collected, your guard is
theater. Prefer a new observation to a new inference.Don’t
NO SAMPLE, NO SCORE.There’s a ceiling on all this and I don’t have a receipt for where it sits. My working rule, offered as an opinion and not a finding: if the job runs once, costs under a dollar, and the worst case is that I run it again, I look at the output with my own eyes and move on. An hour spent building a check for a one-shot script is an hour I stole from the work. I’m sure I’ve gotten that backwards. I just never wrote down the day I did, which tells you something about which mistakes I’m willing to file.
Here’s the one I’ll admit to: ballotnotes, April 23 to August 5, 2026, 1,444 commits in three and a half months, about 14 a day. A bootstrapped site that tells people what’s on their ballot. Go read the last 31 commits and it’s test suite health, memory limits, rate-limit leaks between test files, CI restoration. I built a measurement rig sturdy enough to fly a payload and pointed it at a form that asks your ZIP code.
The second is a shape I think I’ve seen and can’t source. The nastiest missing stop condition isn’t one agent lying to itself; it’s two of them on one shared queue, each holding a stop condition that’s true about its own half and false about the box they’re both standing on. I believe that. I don’t have the date, the queue name, or the log line, so don’t take it from me as a fact. Which is the chapter eating its own cooking: don’t accept what the dashboard told you, and that goes double when the dashboard is me.
Ch. 14 gave you the Ralph loop and told you it needs a stop condition. Ch. 15 built out the goal structure around it. This chapter is the bill for both: the stop condition has to be a measurement the agent can run, or the loop you built in Ch. 14 is an expensive random-number generator with a good attitude.
Forward: Ch. 32 is the other half of this: a test that can’t go red is not a test, and it’s the specific way a measurable stop condition turns out to have been fake all along (see: 735 of 735). Ch. 44 goes up a level to what’s worth measuring at all, which is the question the AI Readiness Report failed while passing 611 tests.
If you only take one line out of this chapter: unmeasured done is just hoping, and hope compounds badly at 200 iterations an hour.
Verified / load-bearing:
git log to copy the
answer from commit history. Corrected score: 76.2%.”
https://rdi.berkeley.edu/blog/trustworthy-benchmarks/blog/2026-08-09-four-numbers-that-were-lying.mdblog/2026-08-08-seven-hundred-and-thirty-five-of-seven-hundred-and-thirty-five.mdblog/2026-08-11-the-report-measured-the-wrong-thing.mdfile://~/Projects/ballotnotes