(Spine Ch. 14.)
“Fanaticism consists in redoubling your effort when you have forgotten your aim.” George Santayana, The Life of Reason, Introduction (1905)
The loop had been awake for hours doing absolutely nothing useful.
Every five minutes it woke, checked, found the condition unmet, and
fired the agent again. Every run returned
{"error": "missing API key"} into a log nobody opened. By
the time I looked there were 120 wasted runs stacked under each other
and about 100 identical drafts of the same chapter on disk, each one
billed. The loop was healthy. The agent was healthy. The silence was the
problem, and the cause was small enough to be humiliating.
Bottom line: An agent loop that re-wakes itself until a verifiable stop condition wins. Idle ≠ done. The pattern is old (cron, retry loops, polling) and the shape is universal: schedule a re-wake, check if done, re-dispatch if not. Do it right and work finishes without your hand. Do it wrong and you rebuild the same task forever.
Ralph is a re-waking loop with these parts:
1. Schedule a wake. Cron, delay queue, polling interval. Every N minutes, wake and check. The loop never assumes the agent will finish; it assumes the agent does one thing and stops.
2. Check if done. One verifiable criterion: count rows, grep for a marker file, call an API and read the response. Binary. Ground truth, not “the agent said it was done.”
3. If not done: re-dispatch. Same prompt, different input, one unit of work per run. The loop controls pacing; the agent stays dumb about when to stop.
4. If done: shut up and go home. Remove the cron, clear the queue, stop polling.
5. Idle ≠ done. An agent that runs silently and
returns is not done. An agent that says “I’m done” is ESPECIALLY not
done; agents hallucinate completion like candy. I shipped a loop that
trusted {"status": "done"} and it congratulated me 60 times
in a row for finishing nothing, so take this one from the guy who paid
for it.
The nastiest version is when your tests agree with the lie. On August 28, 2026 the airank report generator had 1,592 passing tests behind it and printed a document stamped “CHECKED: Measured” about a page whose fetch had failed, then scored that page 25/100. Six independent reviewers read it and all six said the same thing: this document is lying. The suite was green because it tested that the badge rendered, never that it was true. It caught a CSS bug before it caught a report inventing its own measurements.
The name comes from the /ralph-loop skill. Other projects call it keep-alive, poller, retry daemon.
Real scenario: this manuscript. Sixty chapters, each on a fixed template. An agent drafts one chapter cleanly and stops, because the agent’s job ends at “write the file and return.”
Setup: cron every 5 minutes
(*/5 * * * * /usr/local/bin/ralph-loop continue), state in
/manuscript/.opencode/ralph-loop.local.md tracking
iteration: 14, maxIterations: 60,
chapters_verified: 14 (files passing a structure check:
word count over threshold, all 6 sections),
kill_flag: false.
The timeline: 1. Minute 0: Cron
wakes. Iteration 14, verified 14, not done. Fetches the chapter 15 spec,
spawns an agent, which writes /manuscript/15-*.md. Script
runs the structure check; if it passes, chapters_verified
and iteration both go to 15. Commit state, exit. Elapsed: 3
minutes.
Minute 5 through 245: Repeat, same cycle each time.
Minute 250: Iteration 60, verified 60. STOP
CONDITION MET. Sets kill_flag: true, exits.
Forever after: Cron wakes every 5 minutes, reads the flag, does nothing. A GeoCities page with an under-construction GIF on it: still technically live, costing nobody anything, never changing again.
What this buys you: 60 chapters with zero manual re-invocation. If chapter 37 times out mid-draft, minute 185 re-tries it, because iteration doesn’t advance until verification passes. And every run is its own log, so when chapter 43 breaks you read chapter 42’s log.
The loud failure is obvious: no stop condition at all, loop runs forever, easy to see and kill.
The quiet failure is subtler, more expensive, and how you end up in production at 2am:
Failure A: You define a stop condition that never triggers.
The loop checks for /work/DONE, but the agent writes
/work/done.txt. I wrote both halves of that mismatch in one
sitting, then blamed the model for 40 minutes.
On August 8, 2026 nine separate systems on airank announced success in one night and delivered nothing, and two were this exact bug wearing a suit. One was a config key the code read faithfully and nobody had ever defined. Both checks ran, both came back clean, and neither was physically capable of coming back dirty. That same night handed me a 4KB dump of a 900MB database. Every one got caught the same boring way: stop reading the report, measure the effect. Byte counts instead of log lines.
Result: the task finished hours ago. The agent re-runs every 5 minutes anyway, hammering the filesystem, database, and API.
Failure B: The agent stops, but doesn’t signal why.
It runs, produces nothing, returns empty. The loop checks the condition, sees nothing (empty table, no marker file, zero count), and wakes again on schedule. Loop alive, work dead, nobody knows.
On August 7, 2026 the airank collector ran 407 browser sessions in 25 minutes and produced zero artifacts and zero database rows. The retry logic counted attempts by reading ledger rows, and a failed session never wrote one, so the counter believed nothing had ever been tried. Every phrase stayed eligible and got picked right back up. Phrase 92 got attempted six times in 25 minutes. I had put the upload and the attempt record inside one conditional, so “we tried” and “we got something back” were the same fact. They are not. Record the attempt, not the artifact.
This is how you burn a pile of money in LLM costs debugging a typo in an environment variable. The loop did its job. The agent did its job, failing loudly into a log I never read. The mistake was mine and one character long.
Failure C: Race condition on the stop check.
You write files to a network mount. The condition checks file existence, but NFS attribute caching can show the loop “file not there” for seconds after the agent finished writing. Loop wakes, re-dispatches, and now two agents are grinding away on one file they both think they own. Caveat, because I would rather be boring than caught: I have no dated incident of my own for this one. It is a known shape, not a story I can put a timestamp on, and I am not going to invent one to make the section symmetric.
The fix for all three:
Pair the stop check with a diagnostic log. Every wake, log what the condition looked for, what it found, why it didn’t trigger. One log line inside a shared HTTP client’s back-off returned 2,161 hits in nine hours on September 6, 2026, and turned a theory into a number I could grep.
Bonus failure: Silent retries on transient errors.
The agent hits a 429, returns empty. Loop re-wakes, agent re-runs immediately, hits the same 429. By the time the limit expires it has been queued 60 times, and then it succeeds. You see 120 identical runs and spend an hour deciding whether that is an infinite loop or a win. Neither. It is a retry storm.
Mine cost $815.38 a day across 34,083 API calls, and I have four days of invoices proving I did nothing about it. The tell beat the symptom: on August 20, 2026 the queue sat at about 385 jobs while workers chewed 22 a minute against a queue that would not drain. I had already written the guard. Cron called artisan directly instead of the wrapper, so the guard never ran once, and a scheduled job on every web server re-issued the gated work every minute. The pin was written, installed, correct, and completely inert. What caught it was the number: spend stayed at exactly $815.38 after I “fixed” it, and a delta of zero is not a fix, it is a fix that never loaded.
Do
maxIterations with teeth: “finishes by 150 or page
somebody,” not “could finish by 1000 if everything breaks right.”Don’t
Part I was how to talk to the model: split work (Ch. 8), persona (Ch. 9), wrappers and their lies (Ch. 11), context (Ch. 12), skills (Ch. 13). Part II opens with the outer loop, the ring that keeps silence from looking like success, then works through the machinery inside it: planning (Ch. 15), routing (Ch. 16), tools (Ch. 17), reasoning (Ch. 18), reflection (Ch. 19), goal setting (Ch. 20). Every one assumes the loop is alive and checking. Without a verifiable stop, a loop is just a way to burn money repeatedly.
Thesis is mine, field-tested on airank, aigate, and the loop that drafted this book.
Verified: - /ralph-loop skill file:
~/.claude/skills/ralph-loop/, referenced by the pstack and
Claude Code hooks on this machine. - Cron polling pattern: standard on
Linux, systemd, and Kubernetes. - Hallucinated completion, certified by
a green suite: “The honesty layer lied, and 1,592 green tests watched it
happen,” airank build log, August 28, 2026.
file://~/Projects/airank/blog/2026-08-28-the-honesty-layer-lied.md
- Failure A (a stop check that could never fire): “Everything reported
success,” airank build log, August 8, 2026.
file://~/Projects/airank/blog/2026-08-08-everything-reported-success.md.
Nine systems, one night. - Failure B (silent agent errors, empty
results): “Four hundred and seven sessions, no evidence,” airank build
log, August 7, 2026.
file://~/Projects/airank/blog/2026-08-07-four-hundred-and-seven-sessions-no-evidence.md
- Rate-limit and re-issue storms: “Nothing called it,” airank build log,
August 20, 2026.
file://~/Projects/airank/blog/2026-08-20-nothing-called-it.md
- Silent back-off inside a shared HTTP client:
~/.claude/skills/shared-rate-limit-silent-backoff/SKILL.md,
verified September 6, 2026. A loop gapped 2-5 minutes five times an hour
with nothing in its own log; the cause was an unlogged 2/4/8/16/30
second back-off, and the log line that exposed it hit 2,161 in nine
hours.
Honest gaps: - Failure C (NFS race on file checks):
illustrative only, and labeled that way in the text. No dated incident
of mine exists; nothing in the airank build logs or fleet notes covers
it. - No academic papers cited; thesis is field-tested, not
peer-reviewed. - Ralph name: coined here, not industry-standard.
Industry says “retry loop” or “outer loop.” - Backoff timing (5min,
30min, 120min): heuristics, not derived, and possibly wrong for your
traffic shape. The mechanism is documented practice (on MariaDB SQLSTATE
1129: mysqladmin flush-hosts once, then reconnect with
backoff, never retry into the wall), and one real curve is written down
(2/4/8/16/30 seconds, verified September 6, 2026). What I do not have is
a per-project curve where each step is justified by a measured failure
rate or a spend number, so treat my minutes as a starting guess. -
“Never trust agent output” is strong: six of six reviewers caught one
generated report claiming measurements it never took. That is a hard
failure, not a base rate.
Cross-references kept honest: - Ch. 8 (decompose work into units): why one unit per run - Ch. 15 (planning is a bet): the loop re-dispatches the same bet until it lands - Ch. 20 (goal setting): the stop condition IS the goal, made verifiable - Ch. 35 (ledger over diary): log state, not full agent output - Ch. 60 (contradiction): “agent is done” contradicts “loop is awake”; loop wins