(Spine Ch. 22.)
“There is something fascinating about science. One gets such wholesale returns of conjecture out of such a trifling investment of fact.” Mark Twain, Life on the Mississippi, Chapter 17 (1883)
August 2026. A collector I built looked at a page with five CRM brands, six citations and a fat comparison table on it, reported zero products extracted, and tripped a fail-closed abort that killed 1,461 queued sessions in a single shot. I spent the first hour of that morning certain the source had gone empty. The source was fine. Every brand was sitting right there in the HTML, in bold, above the fold. The thing that had gone blind was mine, and the way it told me so was by reporting a number that looked like an answer. It took a second incident, in the same month, before I understood what I was actually staring at.
Bottom line: Retrieve, then generate. If the model doesn’t have the document in front of it, it isn’t answering: it’s rehearsing. The expensive part isn’t the model that admits it doesn’t know. It’s the pipeline that hands the model the wrong document, gets a fluent confident answer built on it, and staples a citation to the bottom so everybody downstream stops checking. That’s not RAG. That’s citation theater with a footnote budget.
contracts/2024-msa-v3.pdf, and the actual answer lived in
v4. Nobody catches it, because the citation looks like
diligence.RAG is not a feature. It’s a two-stage system: stage one is a search problem, stage two is a writing problem, and the industry spent five years obsessing over stage two while stage one silently ate the accuracy.
The original paper (Lewis et al., May 2020) was clear about the shape: a pre-trained generator, a dense vector index, retrieval as a first-class component rather than a prompt-stuffing trick. What shipped everywhere afterward was the trick.
Every stage is a place you lose:
1. Ingestion. Bad parse, bad chunking, bad boundaries. You split a table in half; now neither half has the answer.
2. Representation. Embeddings encode what they encode. Your jargon, your product names, your internal acronyms: the embedding model has barely seen them, and multi-word brand names get truncated by naive matchers.
3. Retrieval. The wrong doc comes back. Or the right doc comes back ranked fourth.
4. Generation. The model does exactly what we trained it to do: produce a fluent answer from whatever’s in the window. It’s completing text on a bad premise (same output shape as lying, none of the intent). Clippy at least popped up and asked. Your generator just writes.
5. Evaluation. You measure whether the answer sounds right. You never measure whether the retrieved chunk contained the answer. That’s the load-bearing miss.
Watch the taxonomy breed. Barnett et al. (January 2024) documented 7 failure points in production RAG. Cresswell et al. (2025) found 16 error types inside partial pipeline runs. Garani (July 2026) laid out 33 failure modes across 7 stages: ingestion, representation, retrieval, generation, evaluation, deployment, agentic orchestration. That’s 7 to 16 to 33 in thirty months, and it isn’t the field getting more paranoid, it’s the field catching up to what was already breaking. The part that should worry you is the grading: 12 of Garani’s 33 modes have no dedicated peer-reviewed evidence under them, and all 8 agentic orchestration modes are in that unsupported dozen. Nobody has published what happens when your agents pass each other the wrong document. We’re finding that out live, in production, on customers.
And the fallback everyone reaches for, “just use a bigger context, dump everything in,” is measurably shaky. “Lost in the Middle” (July 2023) is the standard cite for models reading past what sits mid-context rather than at either end, and I am taking that on its reputation. Chroma’s context-rot work (July 2025) reported non-uniform degradation as input length grows across a spread of models I have not counted. Whether that decay causes retrieval false negatives or merely rides along beside them is not established anywhere I can find, so don’t let anyone sell it to you as a mechanism. Stuffing more into the window doesn’t fix bad retrieval; it hides bad retrieval in a bigger haystack and charges you tokens for the privilege.
Confirm the retrieved chunk actually contains the answer, before you let the model write a word.
Not “did retrieval return something.” Does this text support this claim. Separate check, cheap, and it’s the entire difference between a RAG system and a confidently-wrong machine with references.
August 2026. airank. I wrote this collector. I also wrote the abort logic that trusted it. It reported zero products extracted on the phrase “best crm for small business.”
My extractor reported zero with the total serenity of a smoke detector with the battery pulled out. No alarm, no warning, not even the little ICQ “uh-oh” that told you in 1996 that something had arrived and you should probably look at it.
The extractor wasn’t measuring the source. It was measuring its own coverage of the source, which was zero, so the pipeline read “the source is empty” and slammed the door. Query shapes had drifted underneath it: product carousels, inconsistent columns, an unlabeled medallist format (🥇🥈🥉). And the gazetteer truncated multi-word brands: Land Rover matched as Land, BofA as Bank.
Three fixes, in order of how much they hurt to learn:
Save the raw pages. Re-running is not reproducing. If you didn’t bank the HTML, you cannot debug what the retriever saw last Tuesday. Cheapest insurance in the stack, and almost nobody buys it, me included, which is why I got to debug 1,461 dead sessions using nothing but vibes and a log line that said zero.
Stop fail-closed on absence. The abort logic asked “did this query produce results?” The right question was “has this batch banked any success at all?” Verified against 300 consecutive skips after one success: zero legitimate phases halted.
Move to entities. Segment text became first-class
(brand, attribute) pairs instead of a free-text
use_case_segment blob. That made claims like “HubSpot
owns best-overall-for-small-business in 67% of sessions” possible,
a statement no brand can quickly change.
There’s a companion incident from the same month. The measurement caught a false positive: the brand Drip (a real email-marketing company) was ranking #2 on every coffee query, via the alias “drip coffee.” Obviously absurd. Fixed by scoping brand matches to the phrase category. I fixed it in ten minutes and felt great about myself.
That fix then made HubSpot invisible. HubSpot was mentioned in 149 out of 149 CRM responses, but its stored category was email-marketing, so the new scope filtered it out. The published CRM winners silently reverted to the wrong set. Nothing indicated failure. No error, no alert, just a quietly wrong answer where a right one used to be. Twenty-some years building systems and I got n00bed by my own ten-minute fix.
The correct fix was to scope the ambiguous alias, not the brand. HubSpot came back. The lesson is the one to tattoo somewhere:
A false positive shows up as an absurd result. A false negative shows up as nothing at all.
False negatives walk straight through every guard you built to detect “this looks wrong,” because they don’t look like anything. You need active proof of success, not the absence of complaints.
The loud failure is the hallucinated citation. Google pulled Gemma from AI Studio in November 2025 after it generated fictitious allegations against a sitting US senator with citations to news articles that did not exist. That’s Ch. 23’s territory.
The quiet failure is worse:
You measure the machinery instead of the outcome, and the machinery works fine.
Same month, same shop. We built an elaborately instrumented accounting system and defended a position with evidence E-018: 57 verified ClaudeBot markdown fetches, zero forged. I put that number in a customer deliverable.
Then the floor gave out. Proof of fetch is not proof of citation effect. If serving markdown provided exactly zero citation benefit, those 57 fetches would look identical. We had measured a precondition (the crawler retrieved the page) sitting upstream of the claim we cared about.
The markdown check got downgraded to UNPROVEN, 13 points, with a two-sided note in the customer report, because writing “we were wrong” into a deliverable is the tax on having been confident. Then we measured the thing we’d actually claimed: median 6.81 days from publication to first AI citation, 90% inside 37 days, across roughly 900 pages (Profound). Semrush found only 42% within 30 days. Live retrieval, not model release cycle.
The cheapest guard against this whole class of error is one question, asked out loud, before you write the conclusion:
“What would this evidence look like if the effect were exactly zero?”
If the answer is “exactly like this,” congratulations: you measured the plumbing, not the water.
I’d love to hand you a second one out of aigate, and my memory says maybe there’s one in there: a retriever handing an agent the wrong doc, an answer that shipped, me finding out late. But a shrug is not a receipt, and I’m not printing a date and a dollar figure I can’t pull out of a log in the one chapter arguing that a confident number with nothing underneath it is how this goes sideways. That would be a hell of a way to lose the plot on page four.
Do
(brand, attribute) beats a free-text field every time you
need a defensible claim.Don’t
Ch. 12 was context: what fits in the window and what the window does to it. This chapter is what you put in the window and how you prove it belongs there. Ch. 21 was memory: retrieval with a longer half-life and all the same failure modes.
Ch. 23 is the sharp end: what happens when the model invents the source outright. This chapter is the system handing over the wrong document; the next is the model manufacturing one that never existed. Different mechanisms, identical smell downstream: fluent, sourced, wrong.
Part VII is where the containment check becomes automated, because doing it by hand works exactly until volume.
Thesis is Jeremy’s (retrieve then verify containment; citation theater is not retrieval): that stays argument, not citation.
Verified:
Production stories (airank, August 2026):
~/Projects/airank/blog/2026-08-06-the-data-was-there-the-whole-time.md~/Projects/airank/blog/2026-08-06-the-day-we-stopped-paying-to-guess.md~/Projects/airank/blog/2026-08-22-fetch-is-not-cite.mdCut for lack of a source: the “73% of production RAG failures are retrieval, not generation” figure (repeated in the airank blog, primary source never located, searched September 2026); any live-deployment failure attributed to context rot, since Chroma measured bench decline and nothing more; and a Profound breakdown by source type and domain drift, which does not exist publicly beyond the Tom’s Guide 9-of-10 coffee note.