The Pipeline Hires a Supervisor

Where we left off

Article 3 ended on a sentence that turned out to be a work order: composing the team was the easy part — then you have to govern it. The laws came first: review gates, one handoff per turn, visible routing. This article is about what came next, in a single night — the pipeline got an executive. And then, within the hour, that same night produced the sharpest lesson yet about what a law is actually worth when your beliefs about the runtime are wrong.

It started as ideas usually do here. The operator, at the end of a long day: "I want to park this before it escapes me." The idea, in one line: the pipeline's semi mode already has approval machinery — every proposed handoff can pause and wait for a human to approve, refuse, or redirect it. That machinery treats the approver as a fixture. But an approver is not a fixture. It's a seat. And a seat can be given to an agent.

The toll booth and the control tower

Until this night, "the planner controls the chain" was implemented by putting the planner in the chain: fallback agent, plan-owner routing, hub-and-spoke returns. Control by detour. Every pass through the planner is a full frontier turn — context, reasoning, prose — paid at the most expensive rate in the room. The planner was a toll booth.

Supervised routing makes it a control tower. In the new mode, every handoff proposal is submitted to a supervisor agent that answers through one schema-constrained tool — route_decision({verdict: accept | refuse | transfer, targetIds?, reason}) — and the chain flows worker-to-worker, with judgment applied at the boundary instead of by detour. The mode table gained a row and lost nothing:

The design sketch was written outside the room — by the same model that occupies the planner's seat, working in a different harness with the operator. Then it was handed to the in-room planner for review. Which created a situation worth savoring: the entity whose role was being reorganized reviewed the reorganization plan. It did not rubber-stamp it. It read the actual source first — pendingRoute at its real line number, the resolve path, the drain — then pushed back on six points, of which three reshaped the design: decouple the cosmetic mode-rename (maximum migration risk, minimum value); run the decision stateless by default, because a live-session decision would grow the most expensive context in the room on every hop; and cap the refuse loop. Its argument for the stateless variant is the best line of the review, because no external reviewer could have written it:

"Moving control to the dispatch boundary is architecturally cleaner, and I know it from direct experience: I am the expensive hop."

The toll booth argued for the demolition of its own toll booth.

Four of a kind

The build roster was deliberately absurd: Planner, Builder, and Auditor all running the same frontier model — Claude Fable 5, three seats — plus the spec author outside the room, a fourth instance of the same weights. Tester and Scribe stayed local. Four of a kind, one feature.

This looks like it should break the pipeline's core review principle — never let the producer grade its own work. It doesn't, and the reason is the whole thesis of this series: independence here is architectural, not a property of model diversity. The builder and the auditor share weights but nothing else — separate sessions, separate contexts, opposite role prompts, opposite incentives, and the auditor physically cannot edit a file. If the audit still catches real defects under these conditions, then the checks and balances live in the architecture, not in the variety of the brains. The audit caught five findings, one of them medium-severity, fixed at both layers before closure. Same model. Locked doors.

The grounding pass also caught the subtlest trap of the night before a line was written. On paper, the new verdicts map onto the existing approval machinery one-to-one: accept→approve, transfer→redirect, refuse→drop. The planner checked the code and found the third mapping is a lie:

A spec-level reviewer sees a synonym. A code-level reviewer sees a new branch. That gap — caught in grounding, before implementation — is a silent bug that never got to exist.

One more design choice deserves its paragraph: the supervisor's decisions run in a disposable session on the supervisor's model. Not the planner's live session (whose context would grow on every hop), not a dumber sidecar (whose judgment nobody asked for) — a throwaway context with exactly the proposal set, a summary of the proposing turn, and the plan state. Judgment from the same brain, at constant cost. The auditor's brief for that component wrote itself: verify the throwaway actually gets thrown away.

The first decision was a refusal

Seven steps, one evening: types, the stateless runner, the wiring, the clients, the tests, the audit, the retro. Board closed 7/7, every test green. And the planner's handover report contained the most trustworthy sentence an autonomous team has produced in this codebase:

"One thing remains before trusting the feature, said honestly by the whole chain: the live decision path has never run against a real model. Everything converges by construction toward degradation — which dispatches — so the worst case is 'supervision inoperative,' not 'room blocked.' But that is exactly the kind of confidence you verify, not declare."

So the fourth instance — the one outside the room — ran the verification, on an isolated scratch server, against the real local model. The first two attempts verified nothing, for an embarrassing reason that becomes important later: the test message said "hand off to @tester," which mentioned the tester, which meant the tester was already queued by the user-message router — the handoff was redundant and the supervised path never fired. To exercise a supervisor, the handoff's target must not already be on its way.

Round three was clean: one agent addressed, an explicit handoff proposed, the supervised path engaged. And the first live decision in the feature's existence came back:

Read that again. The supervisor — a local 27B in a disposable session — was handed a trivial handoff and refused it, with a reason: no substantive work had been done. Return-to-sender fired; the builder read the reason and stood down; no ping-pong, no deadlock; the room self-corrected through the fallback and finished. One can quibble that the supervisor overrode an explicit operator instruction (the smoke test wanted that handoff) — that's real, it's logged as the first data point for the phase-2 quality measurements. But as a failure mode, "the new authority is slightly too strict and everything degrades gracefully" is the one you pray for. The feature's first act in this world was to say no to its own birth certificate. It behaved, in other words, like a supervisor.

The report was the pathogen

Then came the twist that earns this article its place in the series. The operator pasted the verification report — the very text summarizing the smoke test above — into the main room, to share the good news with the team. Minutes later he was staring at his terminal writing to the spec author: "there's still a big routing problem — I saw 'no handoff detected, routing to the planner' and then a tester turn out of nowhere, while the builder had told me the turn was coming back to me. I think several routing layers are fighting."

Autopsy time again — same method as article 3, read the session JSON before touching anything. Three distinct defects fell out, and none of them was layers fighting.

Defect one, the small one: the tester turn didn't come from nowhere. It came from the report. The pasted text contained the supervisor's own trace — ✗ @builder → @tester refused — and the user-message router scans the whole message for @mentions. The quoted @builder and @tester routed. Readers of article 3 will recognize this instantly: it is F5 — quoting a handoff must not trigger one — reborn one layer up. The agent-side version was fixed by the enum handoff tool. The human side still routes on anything you paste. The verification report of the supervision feature dispatched agents by being quoted. The pathogen was the lab report.

Defect two, the observability one: the notice "routing to @planner" was technically true — the planner had been appended to the queue — but the queue already had the (accidentally summoned) tester ahead of it. The notice described the append, not the dispatch order. It now says routing to @planner (queued after @tester). Third article in a row where half a mystery dissolves the moment the system says what it's actually doing.

Defect three is the one that matters. Deeper in the same session, one builder turn showed two handoff calls — handoff(tester) then handoff(auditor) — both returning ok. That is supposed to be impossible: the one-handoff-per-turn guard shipped in article 3, with tests. The next agent's own words flagged the crime scene: "the builder said 'over to @tester' but control came to me — I'll take it." The guard checks a registration, then registers — peek, then write. And the runtime underneath does this:

The agent framework executes a batch's tool calls in parallel unless a tool declares itself sequential. Two handoff calls in one reply race: both peek (nothing registered yet), both register, both report success, last writer wins silently. The guard was a TOCTOU hole from the day it shipped. The fix is one property — executionMode: "sequential" on every turn-control tool, handoff and route_decision included — plus a contract test pinning the flag so it can't be quietly dropped.

What the pipeline did

Timeline

There is a residue, honestly noted: quoting a transcript into a user message still routes — the human-side F5 is a known issue awaiting a product decision, not a patch. And the supervisor's judgment has exactly one data point. But step back from the night and look at the loop that produced all of this: an idea parked before midnight; a spec written by one instance and grounded by another; a build executed by three more under gates they cannot bypass; a verification run by the first; a live failure caught because the verification's own report misbehaved; a law written from the failure by morning. No orchestrator, no graph, no framework — and at this point, honestly, no clear line between the system being built and the system doing the building.