AiBook · ISBN 9798173134660 · PDF

AiBook · Jeremy Schoemaker · 2026 · ch-12.html

It’s the Context, Stupid

(Spine Ch. 12.)

“There are only two hard things in Computer Science: cache invalidation and naming things.” Attributed to Phil Karlton, but nobody can produce Karlton saying it. Tim Bray published it December 23, 2005 and said he first heard it around 1996-7; Martin Fowler wrote it up July 14, 2009 and admitted he could not find a source either. The most quoted line in computer science, and the punchline belongs to a guy nobody can find.

I opened a handoff document on August 7, 2026, written by a previous session, in good faith, three priorities, clean and confident. I did exactly what this chapter tells you to do: cleared the window, carried the artifact, got to work. Priority one said the brand gate was the top failure source. I spent the morning on it. The real number was 1.8%. Priority two told me to merge seventeen branches. Priority three told me to scale production up. One of those two would have reverted a bug fix, and the other would have turned off prod. Every sentence in that document was true when somebody wrote it, and I could have caught all three with a single query each.

Bottom line: Prompt engineering is a subset of the real job. The real job is deciding what tokens are in the window at the moment the model has to think: system prompt, tool schemas, retrieved docs, memory files, prior turns, tool outputs, the 400 lines of stack trace that scrolled by twenty minutes ago. All of it counts. None of it counts equally. The model does not read your window; it attends to it, unevenly, and it attends worst to the middle. Your job is curation, not incantation. Stop rewriting the prompt. Start auditing the window.

When it bites

The pattern

Here’s what’s in the window on a real agent turn.

  1. System prompt. Yours plus the harness’s. Usually bigger than you think.
  2. Tool schemas. Every tool you registered, whether or not this turn needs one. Full JSON, every parameter description.
  3. Memory / project files. CLAUDE.md, AGENTS.md, whatever the harness auto-loads. The file you wrote in a hurry six weeks ago and never re-read.
  4. Retrieved documents. Whatever RAG dumped in, ranked by an embedding model that never saw your codebase.
  5. Prior turns. Everything said in both directions, including the three failed attempts.
  6. Tool outputs. The 12,000-token npm install log. I have personally paid frontier-model rates to have a language model read deprecation warnings about a package I do not use. Almost always the biggest single chunk, almost always the least useful per token.
  7. The actual question. Last. Usually about 40 tokens.

Item 7 is what you spent your afternoon wordsmithing. Items 1 through 6 decide the answer.

Now the half people skip: position matters and it is not close. Liu et al. at Stanford and Samaya AI put a number on it in July 2023: models perform best when the key information is at the beginning or the end of the input, and degrade significantly when they have to reach into the middle (“Lost in the Middle,” arXiv:2307.03172). The obvious assumption was that bigger windows would fix it.

They did not. Chroma re-ran the question on July 14, 2025 across 18 models (GPT-4.1, Claude Sonnet 4 and Opus 4, Gemini 2.5, Qwen3) and found performance varies significantly as input length changes, on tasks the model handles fine when short. They called it context rot. Both Claudes sat in that battery and neither got a published per-model curve, so nobody can hand you a Claude number, me included. The two curves somebody plotted are TianPan’s: GPT-4-1106 retrieves at 96.6% accuracy at 4K and 81.2% at 128K, and LLaMA 3.1-70B falls from 96.5% to 66.6% over that same stretch. Effective window for retrieval lands around 30 to 60 percent of the number printed on the box, a general shape, not a promise about whichever model you’re paying today. I paid for the 128K and used the 4K, which is the same math as every gym membership I have ever bought.

Then multi-turn. TianPan’s April 2026 write-up puts an average 39% performance drop across frontier models on multi-turn versus single-turn, same tasks. Your agent gets a third dumber by having been running a while. Nobody’s changelog mentions this.

The discipline, which Anthropic named properly on September 29, 2025: “context engineering is the natural progression of prompt engineering: strategies for curating and maintaining optimal tokens during inference.” That post publishes no degradation numbers, so take the shape from Anthropic and the numbers from Chroma and TianPan. Three moves for long-horizon work: compaction (summarize, then reset), structured note-taking (the agent writes down what it must not forget, outside the window), and sub-agents (send somebody else to read the 900 files and bring back the answer instead of the transcript).

Curate on the way in. Every token needs a reason to be there. Tool output goes through a filter, not straight into the transcript. You needed the status code, not the 900-line response body.

Position on purpose. The thing that must not be forgotten goes at the top or the bottom. The middle is where good instructions go to die quietly.

Clear at boundaries. This one is my opinion and my field practice, not doctrine, and I have never written it down anywhere you can go read it, which in a book about receipts is exactly the kind of irony I have earned. Clear the context between phases. PRD → milestones → phases → tasks → tests → prompts. Each stage takes the output of the last one, not the conversation that produced it. The reasoning that got you the milestone list is not an asset for writing the milestones.

Get comfortable throwing away conversations you enjoyed having.

Compact deliberately, not accidentally. Auto-compaction is real and documented: Anthropic’s cookbook (Nov 24, 2025) covers monitoring token usage and injecting a summary at a threshold. It is also a lossy compression performed by the same model that’s about to be confused, at a moment you didn’t choose. How lossy, nobody has published: not the Sept 29, 2025 post, not the Nov 24, 2025 cookbook, and not anything my searching turned up. I’m not handing you a percentage I made up in the shower. Compaction you triggered at a clean boundary beats compaction that ambushes you mid-task.

Prompt-craft still matters, and it’s the last 10%. The other 90% is deciding what’s in the room.

One worked example

Real, August 7, 2026 (airank blog: ~/Projects/airank/blog/2026-08-07-seventeen-branches-zero-merges.md). Full Mahir “I Kiss You!!!”: maximum enthusiasm, zero verification.

All three were wrong. Jeremy, career highlight: I got pwned by good formatting.

Priority one: the brand gate is the top failure source. Reality: 1.8% of failures. The dominant failure was navigation, which the handoff never mentioned, because when it was written, navigation was fine.

Priority two: seventeen unmerged branches, go merge them. Reality: they’d already landed through other routes. Merging would have reverted a known bug fix.

Priority three: production runs a single replica, scale it up. Reality: production had been imperatively scaled 20x, and Git still described one replica because nobody wrote the change back. Applying the manifest would have scaled production down. Somebody set up us the bomb, and the someone was a manifest that was accurate at write time.

Each of these took one query to expose. Three queries, call it eleven seconds of typing, against a day I spent optimizing a 1.8% problem. I was thorough. About the wrong thing.

The rule that came out of it: before acting on an inherited priority, re-measure the number it rests on. Not review it. Re-measure it. Cost of skipping: a day on a 1.8% problem, a merge that reverts a fix, a deploy that shuts off prod.

That’s not a context-management failure. Clearing context was correct. It’s a failure to treat the summary as evidence instead of as a claim.

The quiet failure

The loud failure is blowing the window: you hit the limit, you get an error, you fix it. Errors are a gift.

The quiet failure:

Your window is full of things that were true.

Not lies. Not hallucinations. Stale truths. The file you read before you edited it. The plan from before the requirements changed. The test output from two fixes ago. Every one of those tokens is a GeoCities under-construction GIF that nobody ever took down, and the model has no mechanism to tell “true” from “was true.” I certainly don’t. At hour four I’m arguing with an agent about a file I edited at hour two, and one of us is holding a stale copy, and it is not always the machine.

The Ch. 4 version of this was not a poisoned line of code at all, it was a poisoned setting. August 7, 2026, airank: Host 192.168.1.33 is blocked because of many connection errors. max_connect_errors was still at the default 100 and skip_name_resolve was off, so MariaDB reverse-DNS’d every connection on a LAN with no PTR records, and my own workers cheerfully retried themselves into the lockout. The config was fine when written. It was true, then stale, then it ate production.

Same disease as the August 8, 2026 outage in the airank blog: nine separate systems reporting success while delivering nothing. A log line said “no phrases matched,” which was true, but true about the wrong thing: the database was empty. The fix is the same shape as the context fix: check the effect, not the report.

Second quiet failure: you tune the last 40 tokens and never audit the first 40,000. Nine iterations on the wording of your instruction while a contradictory line in a memory file from March sits at position 2,000, winning every argument silently. Print the window. Mine was a line I wrote in a CLAUDE.md in March, forgotten, overruling every careful instruction I had written that week. I had been blaming the model for months. The model had been following my orders exactly.

Do / don’t

Do

Don’t

Where this sits in the book

Ch. 8 said decompose the job. This is the other half: decomposition only pays if you clear the window between the pieces, otherwise you split the work and kept all the ballast.

Ch. 13 is this chapter applied to tools. Ch. 21 is memory: what deserves to survive a cleared window. Ch. 22 is RAG, which is a context-engineering decision wearing a database costume. Ch. 28 (routing) and Part VII (evals) are downstream: you cannot eval a system whose window you can’t reproduce.

Sources and receipts

Thesis is Jeremy’s: argument, not citation.

Verified / load-bearing:

What I could not verify: