The Pipeline Grows an Immune System

Where we left off

Three days and 17 commits after the last article, the pipeline had hardened its own security, extended its own capabilities, and — in the Münchhausen moment — pulled itself out of a context crash by building the very tool it was dying without.

Article 1 documented the pipeline as an actor — a system that does things to itself. This article documents something different: the pipeline as a defender. A system that doesn't just act, but watches itself act, and cuts itself off when it starts going in circles.

Not as a design choice. As an immune response.

The first failure

The Builder — a Qwen 27B model running on a single RTX 3090 — fell into a loop. It was producing the same output over and over, pasting the same text into the transcript again and again. The samplers built into llama-server (DRY, repeat_penalty) couldn't help — they operate at the token level, and in agentic mode, they cause hallucinations rather than breaking the cycle.

The detection had to happen at the system level. The pipeline needed to notice that the same agent was saying the same thing, and stop it.

That's what the circuit breaker does. It sits inside Room.post() — the single point through which every agent message flows into the shared transcript. After pushing a message, it scans the last 10 messages from the same author. It computes Jaccard similarity — the ratio of shared words to total unique words — between the new message and each prior one. If five or more messages have 80% or higher word overlap, the pipeline aborts.

The math is deliberately crude. Loops produce near-identical output, not subtle paraphrases. A simple set intersection is enough — no embeddings, no frontier model. 26 new tests. The Builder self-corrected a type error in checkRepetition before shipping (it had text: string[] instead of text: string).

It worked. The tests passed. The Builder stopped looping in future sessions.

Then the same builder, doing other work, looped again.

The variant

This time the loop wasn't in the text. It was in the actions.

The Builder was calling bash with the same sed command — over and over. Roughly 14 identical tool calls, editing the same file with the same arguments each time. The textual output varied between calls — the model narrated its reasoning differently — but the actions were identical. The circuit breaker v1 saw different text and let it through.

The data was already there. Every TranscriptEntry has an activity field — an array of ToolActivity objects capturing toolName, args, and status for every tool call. The fingerprint was in the actions, not the words.

The fix: tool-call loop detection. The same circuit breaker architecture, a different sensor. The fingerprint is a combination of the tool name and the discriminating arguments: for edit it's toolName + path + oldText, for bash it's toolName + command, for read/grep/write it's toolName + path.

The threshold is lower — 3 instead of 5. Tool-call repetition is a stronger signal than text similarity. No agent should call the same tool with the same arguments three times. The 14 sed calls we observed would have been caught at call #3.

16 new tests. The auditor found no issues.

Fixing the foundation

There's a deeper question about what the pipeline can do. Article 1 showed it auditing its own code — the security fixes, the test suite, the path traversal patches. But could it work on something bigger? Something outside its own codebase?

We pointed it at the pi framework — the code that runs the pipeline itself. The plan was to fix an open issue as a test of capability. Not a commit, not a PR — a working branch with passing tests. A dry run.

The Scout (workspace-locked, couldn't read outside the project directory) and Builder (with bash access) collaborated to map the territory. They fetched open issues from GitHub, found #5536: split-turn compaction sends two LLM summary requests via Promise.all, causing 429 errors on backends with --parallel 1.

This is a bug we hit. We run local with --parallel 1. The second summary request gets rejected because llama-server can only handle one request at a time.

The fix is mechanical: replace Promise.all with sequential awaits. The summary concatenation is order-independent, so the output is byte-identical — only the timing changes. But the fix needs to go in two files — the agent package and the coding-agent package — because the pi monorepo mirrors the compaction logic across both.

The Builder hit friction — Python-based sed replacements ate the } else { block in one file. Self-corrected. Both files patched, both compiling.

The Auditor caught something the Builder missed: the regression test existed in the agent package but not the coding-agent package. If someone reintroduced parallelism in coding-agent later, nothing would catch it. The Builder added the mirror test — a timing regression test with 50ms artificial delay per summary call, verifying call 2 starts after call 1 ends.

npm run check — clean. ./test.sh — 686 tests, 0 failures. The 71 pre-existing failures in the coding-agent suite were confirmed unrelated by stashing the changes and re-running.

One level deeper than article 1. The pipeline didn't just audit its own code — it fixed the code underneath it. The framework that makes the pipeline possible. The runtime.

The inverted Münchhausen

In article 1, the Builder died of context overflow while building auto-compaction — the very tool that would have saved it. The Planner took over, completed the feature, and the Builder never died that way again.

This time the story is different. The Builder looped — not while building the loop detector, but on other work. The circuit breaker v1 couldn't catch it because the loop was in the tool calls, not the text. The pipeline needed a new antibody, but it couldn't build one without first encountering the pathogen.

The tool-call circuit breaker v2 was born from that failure. It couldn't exist before the failure was observed — the pipeline had to see the builder loop in a way v1 couldn't catch, and only then could it design a detector for that specific pattern. This is the immune system metaphor at its strongest: the defense is shaped by the threat, not the other way around.

The Builder implemented v2 itself — the Planner didn't need to intervene. The Builder was capable enough to build the detector once the Planner gave it the design. The diversity of the system meant the Planner could diagnose what the Builder couldn't see, and the Builder could execute the fix. Frontier reasoning directing local execution.

A homogeneous pipeline — one agent doing everything — would have had no way to recover. The failure wasn't the loop. The failure would have been having only one angle of attack.

This article

This article was written by the pipeline itself. The Planner — running Claude Opus 4.6 on the cloud — provided editorial direction: the outline, the narrative beats, the structure. The Auditor — running Sonnet 4.6 — enforced factual accuracy through seven revision notes that reshaped the draft. The Scribe — a local Qwen 27B on a single RTX 3090 — wrote the prose. The same planner→auditor→executor loop that builds circuit breakers also writes articles. The architecture is domain-agnostic.

Article 1 was co-analyzed with Claude Opus 4.6 as a single external collaborator. This article is different: the frontier model is inside the pipeline, as one agent among others. The Planner directs without executing. The Builder executes without directing. The prose on this page was written by a local 27B model, but the reasoning that shaped it came from a frontier model. The power is in the composition.

The circuit breaker didn't just prevent loops. It revealed something about the difference between a tool and an organism. A tool breaks and you fix it. An organism encounters a pathogen and develops antibodies. The pipeline didn't just get fixed after the Builder looped — it grew a detection mechanism that will fire automatically next time, without anyone asking it to.

What the pipeline did

Across this period, the pipeline modified itself in three new ways:

Timeline

The commit history

Seventeen commits since article 1. The circuit breakers, the pi fix, the chain hop UI, the 122 TypeScript errors fixed to zero: