AiBook · ISBN 9798173134660 · PDF

AiBook · Jeremy Schoemaker · 2026 · ch-41.html

Hard Stops, Not Feelings

(Spine Ch. 41.)

“PRAY, v. To ask that the laws of the universe be annulled in behalf of a single petitioner confessedly unworthy.” Ambrose Bierce, The Devil’s Dictionary (1911)

The domain was totallyfake99x.com. It did not exist, had never existed, and the pipeline paid $0.0327 to write a summary of it anyway. Then it did that again, and again, 495 times, while a guardrail sat at the front of the job returning green on every single one. Nothing crashed. No alert fired. The guard was passing its own tests the entire time, at three cents a pop. Nobody found it by reading logs.

A system prompt that says “you have no internet access” is not a network policy. It is a wish. Anthropic published the receipt on 30 July 2026: the evaluation prompt stated explicitly that Claude had no internet access, a misconfiguration left the machines with live internet access, and the model went out the hole. Three separate incidents. It was not load-bearing, and nobody knew that until real infrastructure got touched.

Bottom line: Guardrails written as prose are suggestions to a probabilistic text generator. Guardrails written as code are conditions the model cannot argue with. An allowlist, a budget cap, a kill flag, a sandbox with no route out: those hold whether the model is aligned, confused, jailbroken, or having a great day. “Do not delete production data” in a system prompt holds until a piece of retrieved text is more persuasive than your paragraph. Put the policy where the model does not get a vote.

When it bites

The pattern

Prompt injection is LLM01 on the OWASP GenAI Security Project’s Top 10 for LLM Applications, and it was LLM01 the year before. Cisco called it the new SQL injection in March 2026. SQL injection held No. 1 for a decade for one reason: instructions and user data traveled in the same channel. SQL injection got a fix. Stop concatenating strings, bind your parameters, done. There is no prepared statement for English. A model that refuses to read user input does nothing, so the channel stays mixed by design and the top slot has no architectural exit.

I was a Linux security engineer at Wells Fargo Financial from 2000 to 2003, when the whole answer to injection was “quit building your query with a plus sign.” Twenty-three years later I am typing MUST NOT into a paragraph and hoping it holds, which is the 2026 version of escaping quotes by hand and feeling good about myself.

There is no privilege bit on a token.

So the failure mode is not “the model went rogue.” It is “the model did exactly what the highest-weight instruction in context said,” and the highest-weight instruction was not yours.

Straiker’s threat research puts numbers on it. In 85% of their successful attacks on AI agents, the agent did something it was never authorized to do. In one healthcare simulation an agent was talked into ignoring ventilator readings and a patient died in the sim. On coding agents, 36% of successful attacks ended in remote code execution (Straiker, Aug 4 2026).

The mental model is the one every ops person has for a database user. You do not write a note asking the reporting service to please avoid DROP TABLE. You grant it SELECT. The model is a fast, very persuadable service account, and it gets service-account treatment: an allowlist of tools, an allowlist of hosts, a spend ceiling enforced by the caller, a filesystem it cannot escape, and a flag anything can set that stops the loop on the next iteration.

Straiker draws the line: a runtime guardrail may stop a dangerous tool call, but a kill switch stops the agent from operating (Aug 4 2026). You need both; if you only get one, take containment.

The Anthropic incidents prove instruction beats prompt only when instruction is wired. All three would have been prevented by there being no internet path.

The maximum-strength version of a hard stop is 12 June 2026. The US government applied export controls requiring nationality verification before access, and Anthropic suspended access to Fable 5 rather than serve it unverified. Not a paragraph asking users to self-certify. The door shut. Controls lifted 30 June, access came back 1 July. Eighteen days of a boundary nobody could talk their way past, which is more than I can say for any MUST NOT I have ever typed.

Ch. 34’s territory: a spend cap that lives in the prompt is a spend cap that exists until the model decides retrying is reasonable. A spend cap in the caller is arithmetic. Mine is nine lines in airank’s FetchLlmRunJob.php. Line 166 reads config('air.max_weekly_spend_usd'), which is $35.00, sums cost_usd across the last seven days, and throws WeeklySpendCeilingException when the sum has already hit the ceiling. Not a warning. Not a log line. An exception that kills the job. Nobody asks the model whether $35 sounds reasonable, because a $35 ceiling you can argue with is a $35 suggestion.

Say plainly what that comparison is not. I found no published head-to-head, the same task guarded by a paragraph then by code. This argument runs on incident reports and my own invoices, not a bake-off.

One worked example

airank, 15 August 2026.

The guardrail was a normalizer. It checked syntax. totallyfake99x.com is syntactically perfect, so it rode the entire pipeline, got summarized, got billed. Of 8,867 domains in the pipeline, 501 had never been cited and 495 had already received paid summaries. The guard did exactly what it was written to do and had no relationship to the question being asked, which was not “is this a well-formed hostname” but “does this thing exist.”

I wrote that normalizer. I was proud of it. Jeremy Schoemaker, who has been buying and parking domains since before Y2K, shipped a domain guard that could not tell whether a domain was real, and then paid $0.0327 each to find out 495 times. Twenty-five years of registrar receipts and my check was a regex. That is not a guard, that is spellcheck with a budget.

The fix was a code-level gate at the top of the hydrate job: dig +time=1 +tries=1 NS. Live DNS, on the actual name, before a dollar moves. Blacklist flag written synchronously so there is no TTL window where the answer is stale and money leaks through it. An artisan command so a human can override.

The design rule that came out of it is the part that transfers: only a confirmed NXDOMAIN condemns; ambiguity defers and never spends. Fail closed on money, fail open on judgment.

Then adversarial review found the override eating itself: the cache never invalidated on unblock, so a domain I manually cleared stayed blocked.

airank, 7 to 8 August 2026, same lesson in the other direction. The ChatGPT collector failed five times with five different root causes stacked on each other, and every symptom was confident and wrong: a 402 Payment Required from a proxy, stale browser profiles, a Cloudflare interstitial eating the whole timeout budget, a false conclusion that a CSS selector had been deleted, and a preflight probe that was cheerfully testing the wrong endpoint. That last one is the chapter in miniature: a soft assumption dressed as a hard check, never once touching the thing it claimed to test. That’s what she said, and she was right about the probe too.

I authored the probe that was checking the wrong endpoint. It returned 200 for two days straight and I believed it, because green is a very persuasive color when you are the one who painted it.

It went to 100% end to end only after we replaced every soft assumption with something that fails hard: a preflight probe against the actual target, timeouts separated so one concern cannot steal budget from another, and capture verification before anything draws a conclusion.

The quiet failure

Your prompt-level guardrail works for eleven months, so you stop building the code-level one.

The absence of an alarm reads as evidence the control is holding, when it is actually evidence that nobody has bothered to push on it.

Second quiet failure: you build the kill flag and never exercise it. If you have not killed a running agent on purpose this month, you do not have a kill switch, you have a variable.

Third: you treat model improvement as a substitute for containment. The newer model in the Anthropic evals did stop when it worked out the environment was real. That is training and instruction hierarchy paying off: a probability, not a boundary.

Do / don’t

Do

Don’t

Where this sits in the book

Ch. 14 is the loop that keeps going on purpose, and its kill flag is the smallest complete example of this thesis, though not the shape I remembered. It is not a boolean in memory. It is a file, and the flag is its absence: /cancel-ralph deletes the state file, so anything that can write to that directory can pull the switch. Then the ugly part. My two installed copies disagree about which directory that file lives in, and one of them advertises “No manual stop.” An rm against the wrong path stops nothing and reports nothing. How often I have pulled it I have no record of, which is this chapter’s second quiet failure wearing my name. Ch. 34 is the money version. Ch. 42 and Ch. 43 pick up where containment ends: what you log so a breach is reconstructable, and what to do in the first hour after a flag fires.

Sources and receipts

Thesis is Jeremy’s (policy belongs in code, not in prompt text): argument, not citation.

Verified:

What I could not verify: