Beast Mode — a behavioral auditor for your coding agent

Raising agents is frustrating.

Not in the way you expect when you first build with them. The early frustration is obvious: hallucinations, wrong APIs, confidently broken code. You build evals, add retry logic, instrument the outputs. That frustration is solvable.

The later frustration is quieter and harder to name. The agent is passing. The code compiles. The test is green. The file is written. And something still feels wrong.

You look at the transcript and you see it: the agent announced it was going to read three files and then read them one by one. It could have read them in parallel. It didn't. The agent was asked to fix a bug and patched the symptom instead of the root cause — not because it couldn't reach the root cause, but because the symptom patch was faster and the test passed. The agent asked you which migration tool to use when the repo's package.json was right there.

You correct it. You move on. The session ends.

And then it happens again in the next session. And the one after that.

That is the frustration. Not that the agent fails, but that the agent succeeds in a way that costs you something the output doesn't show. You spend cognitive load catching behavioral drift that the transcript never flags. The eval never touches it. The CI gate never sees it. It is invisible unless you are watching — and watching is expensive.

This is the OPBR problem. On your local machine.

I spent a year studying this failure mode in production AI agents: output-passing behavioral regressions. The agent completes the task. The output is correct by every metric in your eval suite. And the behavior — the path it took — was wrong in a way that would matter if you could see it.

In production, this manifests as a support agent that commits a refund before checking fraud risk, a coding agent that skips a precondition because the output state happens to be correct anyway, a memory agent that revises a record without verifying the requesting scope.

On my laptop, it manifests as sequential tool calls, scope shrinkage, and unnecessary asks. Smaller stakes, same structure: the output passes, the behavior doesn't, and the gap is invisible to standard tooling.

The difference is that in production, you have infra to instrument. On your laptop, you have your own attention. Which means the behavioral governance cost falls entirely on you, every session, with no accumulation — each correction evaporates when the conversation ends.

I kept making the same corrections. I had no record of having made them. The agent had no record of having been corrected. We were both starting from zero, every session.

So I built an auditor.

The first version was a prompt. I wrote a Haiku prompt that read the Claude Code transcript and scored it on six dimensions. I ran it manually after sessions I thought had been particularly bad. It helped. It named things I'd noticed but not articulated: "sequential when parallel was available," "scope shrunk without blocker cited."

Then I automated it. Then I added a ledger. Then I built a Constitution — a document that defined, precisely, what full-power agent behavior looks like and what human-contractor-mimic behavior looks like. Then I hooked the Constitution into the Claude Code context so the agent was reading its own operating doctrine at every session start.

I wrote the first Constitution with 22 rules. The agent found soft exceptions to all of them. I threw out fourteen and rewrote the rest as principles with one enforcement mechanism each. That version stuck.

The system has seven layers now. None of them block a response. The auditor fires after the turn completes, scores it, and writes one entry to a JSONL file. The evolution loop runs weekly — Opus reads the last seven days of the ledger and proposes Constitution amendments based on observed leak patterns. I review the proposals manually. Some land. Some don't.

The full architecture is in the Beast Mode workbench post. What matters here is the shape: it is a feedback loop with an append-only evidence base. The machine corrects the machine. I review the meta-level, not the individual turns.

What the ledger shows.

After running this system on my machine since February 2026, the ledger has 20,906 entries. The rolling Beast Index — the fraction of applicable dimension scores that hit 1 — is 76.3%. That is 47,443 beast-mode scores out of 62,156 applicable evaluations.

The top failure dimension is parallelism, by almost 2:1 over the next. This surprised me. I expected scope shrinkage or deferral leakage to dominate — those feel more visible in session. But the structural collector doesn't lie: 4,784 turns where independent work was dispatched sequentially when it didn't need to be.

The reason parallelism tops the list is structural. Sequential execution is invisible at the output level — the files still get written, the results are the same. And unlike scope shrinkage, it does not require a judgment call from the LM judge: the tool-call trace either shows parallel dispatch or it doesn't. So there is no measurement noise. Every 0 in this dimension is a confirmed miss.

Scope comes second (2,540 failures). Sequencing third (2,574). Boldness fourth (2,227). Deferrals and depth trail significantly — 1,484 and 1,104 respectively. Depth being lowest makes sense: root-cause reasoning is where the model is strongest. Deferral discipline has improved most visibly since adding the Constitution; the ledger shows a downward trend in deferral failures over the last 90 days.

One real entry from the ledger, unedited. This is what the evidence base looks like:

The null dims were N/A for that turn — no sequencing or deferral opportunity. Score is {beast}/{applicable}. This was a clean turn with one contextual note: announced deployment without showing the evidence in transcript. Not scored as a failure, but flagged by the auditor for pattern tracking.

76.3% is not a pass rate. It is a baseline. Three months ago it was lower. The direction matters more than the number.

What it changed.

The obvious thing: I no longer do the auditor's job. The machine catches the turn, writes the entry, and the ledger accumulates. I look at the weekly evolution proposals and decide which ones to apply to the Constitution. The behavioral governance loop is now mostly automated.

The less obvious thing: the agent reads its own correction history. Not literally — the ledger is not in context. But the Constitution is. And the Constitution is built from the ledger's patterns. When the evolution loop proposes an amendment because parallelism failures spiked in the last seven days, and I apply that amendment, the agent's operating doctrine updates. The loop closes.

The other thing it changed: I stopped correcting by feel. The frustration I described at the start — noticing something is wrong without being able to name it — that has a name now. Ten names, one per dimension. When a turn feels off, I can look at the ledger entry and see exactly which dimension the auditor flagged and why. The feeling became legible.

That legibility is, I think, the most underrated part of building this kind of system. The output evaluation gap is real and measurable. But the secondary value is that measuring it gives you vocabulary for something you were only feeling before.