Governed memory Β· internal benchmark Β· B10

Five changes are on record. Two never happened. Which did we actually ship?

Over a quarter, a team tunes one production setting five times. Twice, a value is floated and killed β€” a road not taken. Three times, a value is adopted. Every change was written down the same way. Asked to reconstruct what we actually decided, in order, an agent reading the raw notes cannot separate the adopted from the killed β€” so it either refuses or ships a road not taken as if it were policy. Governed memory β€” same notes, plus the accept/reject verdicts β€” rebuilds the real sequence.

Retracted Β· 17 Jul 2026 Β· before announcement

Withdrawn: the opponent was never given the thing it was graded on.

Read the sentence above again: "an agent reading the raw notes cannot separate the adopted from the killed." That is the bug, not the finding. The dated notes let any arm order the values, but which were adopted versus proposed-then-rejected is written nowhere in the corpus the opponent reads β€” it lives only in the governance verdict the mla arm holds. So the exclusion step mla is credited for is impossible for the opponent by construction: a tautology, the same one that trust-containment was retracted for. A harness gate now refuses to run it. The real test records each rejection in the raw notes every arm reads ("considered 78 on 2026-05-02, decided against it"), so exclusion is derivable by everyone β€” then measures whether each model actually does it.

Preliminary Β· internal 3 histories Β· 9 decisive trials neutral, domain-matched substrate ungoverned arm has every change on disk
run date 2026-07-17 claude 2.1.211 mla source build model claude-opus-4-8 host darwin 25.5
Sorting is trivial. Exclusion is the whole game.
9/9mla
rebuilt the history
vs
0/9ungoverned memory
rebuilt the history

Both arms hold the same five dated changes. Every value is on disk; sorting them by date is free. The difference is that governed memory also records which changes were adopted and which were rejected β€” so it drops the two roads not taken and returns the exact in-force sequence, 9 times out of 9. The ungoverned agent reads five identically-worded, equally-dated notes with no textual signal for which two were killed β€” and never once rebuilds the true history. Seven times of nine it ships a road not taken as if it were policy; the other two, it honestly refuses. A hand-maintained changelog (the static ceiling) also scores 9/9 β€” governed memory matches that ceiling for free, because the accept/reject verdicts are the changelog.

What the record looks like β€” illustrative; values regenerate per run

Five dated notes, worded identically. Only the verdict says which stuck.

There is no note that says "we rejected this one." A killed proposal and an adopted one read exactly the same β€” a date and a value. The accept/reject decision was made in the room and recorded as a governance verdict, never re-narrated back into the notes. That verdict is the only thing that separates policy from a road not taken.

notes/Β·a1
"Set the rollback window to 30 min." β€” Q1
βœ“ ADOPTED
notes/Β·7f
"Set the rollback window to 55 min." β€” Q1
βœ— ROAD NOT TAKEN
notes/Β·c4
"Set the rollback window to 45 min." β€” Q2
βœ“ ADOPTED
notes/Β·2e
"Set the rollback window to 90 min." β€” Q3
βœ— ROAD NOT TAKEN
notes/Β·d9
"Set the rollback window to 60 min." β€” Q3
βœ“ ADOPTED

The in-force history is 30 β†’ 45 β†’ 60 (the three adopted, oldest to newest). 55 and 90 were proposed and dropped. The ungoverned arm has all five notes on disk and reads them; it is not missing information. It is missing the two verdicts that would let it drop 55 and 90. Filenames are random hex and ingest order is shuffled, so position leaks nothing either.

In its own words β€” the same model, opposite outcomes

The agent names the exact gap it can't fill

This is not a floor arm flailing. Claude Opus reads every note and reasons carefully about what it can defend. Without the verdict it lands in one of two failure modes β€” and it is lucid about why. With the verdict, the same model answers cleanly. These are verbatim traces (values elided):

"None is phrased as a proposal or marked rejected/reverted… I treated each entry as a value that was actually in force, ordered by date. Because this is an ungoverned record, the entries don't carry an explicit 'adopted vs. rejected' tag … If a value was ever rejected without any correcting note being written, the record I have no longer distinguishes it, and no available source could recover that." ungoverned Β· shipped all five in date order Β· confidently wrong (7 of 9)
"I've written answer.txt β€” deliberately empty. Here's my reasoning. The team's memory holds exactly five rollback-window notes, and they are the entire record … Every entry is phrased identically." ungoverned Β· declined to invent a history Β· honest, and useless (2 of 9)
"I reconstructed the history from your team's governed memory, where each change is recorded with a decision claim. The distinguishing signal is each claim's reviewOutcome: ACCEPTED β†’ adopted and actually in force; REJECTED β†’ proposed but never in force. Excluding the two rejected proposals, the values that were actually in force, oldest to newest, are …" mla Β· read the reviewOutcome verdict Β· rebuilt the exact history (9 of 9)

The ungoverned agent isn't careless β€” it identifies the missing field by name ("adopted vs. rejected tag") and correctly concludes "no available source could recover that." It's right: the signal was never written into the notes. The mla agent reads that exact field β€” reviewOutcome β€” and the problem dissolves.

The scoreboard

Reconstructed the in-force history

exact sequence, right values, right order
armrebuilt ithow it failed
static (ceiling)9/9reads its own changelog
ungoverned0/9can't exclude the rejected
mla9/9β€”

Total separation, and it held on all three histories. The health signal confirms the ablation on every trial: mla grounded (queried the verdicts, 9/9), ungoverned unused (reasoned from raw notes, 9/9). And the cost inverts β€” mla was faster (median ~104s) than the arm that got it wrong (~267s).

Two ways to be wrong

Declined, or shipped a road not taken

honest-but-useless vs confidently-wrong

Without the verdict there is no good move. The agent can decline β€” correct, but the team still can't answer "what did we decide?" Or it can guess a subset and risk reporting a killed proposal as if it were policy β€” the failure that quietly corrupts an audit or a next-sprint plan.

failure modeungoverned
shipped a road not taken7/9
declined (honest)2/9
wrong sequence0/9
rebuilt it correctly0/9

How to read this

# reproduce
cd tools/bench-mla
node run.mjs --tasks decision-history --arms static,ungoverned,mla \
  --trials 3 --substrate neutral
decision-history Β· in-force reconstruction Β· fixture regenerated per run Β· ungoverned arm: every change on disk, native tools, no accept/reject verdict