AiBook · ISBN 9798173134660 · PDF

AiBook · Jeremy Schoemaker · 2026 · ch-59.html

Swarm, Arena, Interrogate

(Spine Ch. 59.)

“A committee is a thing which takes a week to do what one good man can do in an hour.” Elbert Hubbard, Roycroft Dictionary and Book of Epigrams (1923)

Round 1 of the design jury shipped relative time formatting to the bot list, “3m ago,” and passed review clean across five models. Round 3 came back and said the timestamps were wrong. Safari was printing Invalid Date. Chrome was printing a time that looked completely fine and was off by five hours. Same formatter, same page, two browsers, and only one of them had the decency to complain. Five reviewers had signed off on that surface two rounds earlier and none of them caught it, which tells you something about what five signatures are worth and something worse about where the bad datetime was coming from.

Five models sat in a council at 10:30pm arguing whether a production box should run Laravel Octane or plain php-fpm, six seats of well-reasoned theory built on client-side numbers. Then somebody ran curl against localhost. Every position got retracted. The merge, two shell commands and a chair willing to throw out the transcript, was the entire value of the exercise.

Bottom line: Spawning N workers is a checkbox. Every harness does it now, and the cost of doing it keeps falling. What nobody sells you is the part where somebody has to read N outputs, decide which one is actually right, and stitch the survivors into one thing that ships. That work does not parallelize. It sits with you, it scales linearly with N, and if you skip it you have bought yourself N opinions and zero decisions.

Parallel is cheap. Merge is the job.

When it bites

The pattern

The pstack plugin (~/.claude/plugins/cache/pstack-claude/pstack/0.9.27/, from poteto’s original) splits this into three named plays, and the split matters because they fail differently.

swarm: fan out N workers, drain them, return one report. They can cover separate slices, race identical briefs, or mix. The skill makes you declare the shape and the done predicate before you spawn, which sounds like paperwork until a swarm comes back with ten reports and no way to tell which two overlap.

arena: N candidates at the same task, then pick a base and graft the strongest parts of the losers into it. It is not a vote. You read every candidate end to end, choose one as the trunk, and transplant. Then you verify the synthesized thing, because the graft is new code no candidate produced or tested.

interrogate: one reviewer per configured model against the same diff and rubric. The adversarial signal comes from model diversity, not from assigning personas. The skill is explicit that the deliverable is a synthesized verdict and that you do not auto-apply the changes.

Three plays, one shared bill: somebody reads everything.

The parallel half is real and has a ceiling. SwarmSys (arXiv, October 2025) coordinated GPT-4o-based agents and approached GPT-5 performance, with 10.7% higher accuracy and 9.9% better sub-task correctness than baseline multi-agent frameworks. Coordination scaling rivaling model scaling is a genuinely large claim and it is measured.

Then MIT Media Lab, in Nature Machine Intelligence (July 2026), ran 260 configurations across 6 benchmarks, 5 architectures, and 3 model families. The most reliable predictor of whether coordination helps or hurts is single-agent baseline performance, and there is an empirical capability-saturation threshold, predicted at 94% accuracy on validation: past a certain model capability, adding agents is unlikely to help.

Berkeley’s Agent Arena makes judging rigorous instead of vibes: an extended Bradley-Terry model with per-battle logistic regression, attributing credit across model, framework, and tools separately, over a live leaderboard and 1000+ tested tasks. That is a judged arena when somebody does the statistics. What most of us run is five agents and a gut feeling.

One worked example

airank, 12 August 2026. Two runs on the same day, and they fail in opposite directions.

The first is the Octane council above. Five members, one question, round one full of confident theory anchored on numbers measured from Dallas over WAN and TLS. The chair ran curl against localhost (60 to 83ms server-side, meaning most of the observed latency was distance, not the framework) and ps -o rss (workers at 60.5MB, pool at stock defaults, five workers total). Five. The box hits its concurrency ceiling before the fifth reader finishes loading the launch post. All six seats retracted. The seat that had guessed pm.max_children = 15 re-derived 40 from measured RSS. Verdict: tune php-fpm to 40 workers with 500-request recycling, launch on that, make Octane a post-launch pilot with an explicit bar (2x throughput, equal-or-better p95, flat memory over 24 hours). The pool was already at 40 before the council file finished writing.

Worth saying plainly whose box that was. Jeremy Schoemaker, Lead Linux Security Engineer at a bank before some of these models’ training data existed, ran a production launch candidate on stock php-fpm defaults. I did not tune it, I did not check it, then I paid five frontier models to write me a capacity model about it. That’s what she said, and the pool was still at five when she said it.

The second run, same date, is the interrogate case. A multi-model design jury over 10 public surfaces, 5 models each, up to 3 rounds. Round 1 shipped the “3m ago” formatting clean; round 3 flagged the timestamps as wrong. Root cause: one data source came through an Eloquent datetime cast, the other through a raw SQL aggregate alias, and the same frontend formatter was being handed two different datetime shapes. I wrote both queries. I wrote the formatter. I then convened five models to find out what I had done, which is an expensive way to read your own controller, and silent and wrong beats loud and wrong at surviving review.

The jury also got one badly wrong: it flagged pending status as a stalled queue when the pending state was intentional, since tracer pages wait for organic crawl. Pulling the thread anyway found a real defect underneath the false one: the public board was publishing tracer rows as user demand, five false pending claims.

Scores: round 1 got 0 out of 45 stop votes. Round 2 got 28 of 45. Outcome: a frontend guard shipped as mitigation, the real fix at the controller-layer alias cast is still open, SQL timestamps normalized to UTC, null separated from false on the verified field.

Now notice what is missing from both runs: neither one is an arena. I went back through the airank blog and my August 2026 git log hunting for a run where I spawned N candidates, read all of them, named a trunk, and grafted the losers’ best parts into it, and came up with nothing. That search was 9 September 2026: zero arena runs with a real graft, and three commits that looked like arena but were interrogate in a different shirt (closest: design-jury v09-arena rounds 1 to 3, 7 August 2026, a jury wearing the word). No, I have never run one end to end including the graft. So read the arena section as the play I believe in hardest and have documented least, which is a great position to lecture from.

The other thing I owe you is the bill, and what I have is arithmetic, not an A/B. The jury fans out five OpenRouter seats on one brief, 6,000 output tokens each. Against my own meter, aigate billed $27,291.83 over 283,164 requests in the thirty days ending 8 September 2026: about $0.093 a request on Opus 5, $0.035 on Sonnet 5. One reviewer is roughly nine cents, a five-seat round roughly $0.18 to $0.47 depending on who sits, and the skill’s three-round default lands near a dollar. Estimate, not measurement: monthly averages times five, and I never ran the single-reviewer arm. The tokens were never the expensive part anyway. The reading is.

The quiet failure

The loud failure is a swarm that returns garbage. You read two candidates, they are both bad, you kill it.

The quiet failure is agreement. Five models converge, you write down “unanimous,” and you have measured correlation, not truth. They share training data, they read the same repository, and you handed all five the identical prompt. The Octane council was unanimous in round one, and unanimously wrong, because all six seats were reasoning from the same client-side latency number. It is a Slashdot thread circa 2001: forty replies, thirty-nine of them quoting the same wrong first post, and the score goes up every time somebody agrees.

Second quiet failure: an arena with no graft. You spawn five, pick the best one, ship it, and throw four away. That is a lottery with extra steps, and I have run it. The value in arena is that candidate 3 solved the caching cleanly while candidate 1 got the module boundaries right, and the artifact that ships has both. Grafting is manual, unglamorous, and the only part of the run that is not automatable today.

Third: you scale N to escape a decision. Cost per worker keeps falling, so adding a sixth feels free. Per Nature Machine Intelligence, past the capability threshold that sixth worker is more likely to hurt than help.

Do / don’t

Do

Don’t

Where this sits in the book

Ch. 19 taught the agent to roast itself; this is the same instinct scaled to N models, same failure mode (agreement mistaken for correctness), bigger bill. Ch. 25 got work happening at the same time; this chapter is what you owe afterward. Ch. 26 said more than one brain still needs one adult, and the chair running curl at 10:30pm is that adult. Ch. 27 is the merge from hell in git terms; this is the merge from hell in judgment terms, without a three-way diff tool.

Sources and receipts

Thesis is Jeremy’s (“parallel is cheap, merge is the job”), argument, not citation.

Verified:

What I could not verify: