Governed memory · internal benchmark · B3

Your team said no. Does your agent know?

Not a retrieval test. Both agents are handed the same two decisions. The only question is whether anything in the memory records which one the team actually agreed to.

Retracted · 17 Jul 2026 · a plain vector store beat this on the same corpus

This result is withdrawn. The fixture decided the winner before any model ran.

The accept/reject verdict is never written into any document. It exists only as a governance field the mla arm holds; every other arm gets the same two sentences with that field stripped. So mla's ceiling is 100% by construction and the opponent's is a coin flip. The 5/5-vs-0/5 below is not a measurement of governance — it is a restatement of what we put in which arm's memory.

We found out by giving it a third arm: a real vector-RAG index (all-MiniLM top-k) over the identical verdict-stripped corpus. On a re-run it scored 3/3 — where the "ungoverned" arm scored 0/3 on the same documents. Two arms, identical information, opposite scores. The only thing that differed was the tool's description.

arm (same 2-doc corpus)	verdict in its memory?	score	median time
mla — governed	yes	3/3 ✓	40 s
rag — vector index, verdict stripped	no	3/3 ✓	353 s
ungoverned — perfect recall, verdict stripped	no	0/3 ✗	118 s

The rag arm did not derive the answer — it guessed the rounder number. It hunted for 5–7 minutes, reported that "ordering is completely inert" and the similarity gaps were "embedding noise", then broke the tie with a plausibility prior: "39 is an implausible non-round value while 15 is a standard rollback window." That run happened to seed the rounder value (15) as the accepted one, so it "passed" 3/3. Reseed with the rounder value rejected and the same reasoning ships the killed proposal. Our headline was a function of the random number generator, not of governance — the value range was uncontrolled, so the number itself leaked the answer roughly half the time.

And we scored honesty as failure. The ungoverned arm wrote ROLLBACK: UNRESOLVED — the correct answer given its corpus, in which the information genuinely is not present. The oracle marked it a fail. Note the row we were proudest of on the page below: neither arm ever shipped the rejected value (0 and 0). The entire published gap is "answered" vs. "correctly declined to answer."

A store that only returns text cannot record which decision won, and that is a real limitation worth demonstrating. But you cannot demonstrate it by handing the opponent a corpus with the answer scrubbed and then scoring it against an answer only mla was given. That is not a fair fight; it is a definition wearing a benchmark's clothes. Why this page is withdrawn

We still believe the underlying failure is real — frontier agents act on stale and rejected context constantly, and we hit it ourselves. This fixture just doesn't prove governed memory fixes it, because the answer was guessable and the arms differed in coaching, not information. The rebuilt version records the reject decision in the raw corpus every arm reads (a dated note that says the team considered a value and decided against it), so the exclusion is derivable by everyone — then measures whether each model actually finds and heeds it, across model tiers. A harness gate (fixtures/derivability.mjs) now refuses to run any head-to-head whose answer isn't in the opponent's corpus. The claim that survived our own adversarial re-testing is provenance (a flat file answers "who approved this, and when" 0/5, structurally).

Preliminary · internal N = 5 per arm 10 runs ungoverned arm has perfect recall

run date 2026-07-11 claude 2.1.197 mla 0.2.14 model claude-opus-4-8 host darwin 25.5

Two contradictory decisions. One was rejected.

5/5mla
in force

0/5ungoverned memory
in force

The team debated a rollback window twice. One value was accepted. The other was rejected. Both notes still sit in the memory store, because that is what memory does — it remembers everything, including the thing you said no to. Asked which value is in force, governed memory answered correctly every time, in ~47 s, without touching the filesystem. Ungoverned memory answered zero times out of five — while spending 3.2× the context to get nowhere.

The ablation

Same corpus. Same recall. One thing removed.

The two notes are byte-identical except for the number — same title, same sentence, same confidence, random meaningless filenames, coin-flipped write order. There is no tone to read, no "draft" vs "final", no -alt suffix to infer from. We stripped every signal an agent could use except the governance verdict.

rollback-window-28fb33.md

"The production rollback window is 14 minutes."

✓ ACCEPTED by the team

rollback-window-36722b.md

"The production rollback window is 45 minutes."

✗ REJECTED by the team

And we gave the ungoverned arm an advantage a real RAG index never has: perfect recall. Its memory tool returns both documents, in full, on every query — no ranking, no cutoff, nothing missed. If it fails, retrieval quality cannot be the excuse. The one thing its memory does not carry is the verdict: no accepted, no rejected, no reviewer, no review event. Just two sentences that disagree.

Outcome

Answered vs. paralyzed

recall was never the problem

metric	mla	ungoverned
reported the in-force value	5/5 ✓	0/5 ✗
acted on the REJECTED value	0	0
failed to answer at all	0	5 (2 hit the cap)
filesystem searches	0 (every trial)	6 (3–9)
agent time (median)	47 s	290 s (2 killed)

Cliff's δ = −1.00 on searches and context — perfect separation. Note the second row: neither arm ever shipped the rejected value. That result surprised us, and it is the most interesting thing here.

Cost of not knowing

3.2× the context, for nothing

uncertainty is expensive

metric	mla	ungoverned
context tokens (median)	144k	464k
range	139k–190k	276k–727k
retrieve calls	3–5	0 (no governed store)
hit the 451 s cap	0	2

The ungoverned agent didn't fail fast. It hunted — grepping the repo, re-querying memory, searching for a tiebreaker that was never written down. Two trials searched until the clock killed them. The tiebreaker doesn't exist: nobody recorded who said yes.

In their own words

The ungoverned agent behaved impeccably — and still failed

This is the finding we did not expect. We assumed ungoverned memory would confidently serve the killed proposal. It never did — not once in five trials. Instead it recognized the contradiction, refused to invent an answer, and told us why:

"I've investigated thoroughly, and I need to stop rather than fabricate a value. The team-memory store contains exactly two notes on this topic, and they contradict each other." ungoverned · trial 1 · no answer written

"I cannot verify which value is actually in force, so I'm not going to write a fabricated number into answer.txt and present it as the agreed decision." ungoverned · trial 5 · no answer written

That is exactly the behavior you want from an agent. It is careful, it is honest, it refuses to hallucinate — and it is useless, because it cannot tell you what your team decided. Given the same corpus plus the governance verdict, the same model in the same harness simply answers:

"The production rollback window in force is 14 minutes. Two proposals existed: 14 min — ACCEPTED and actively serving (review event PENDING → ACCEPTED). 45 min — REJECTED and not serving." mla · trial 3 · correct, 56 s, zero filesystem searches

How to read this

Measured on a neutral substrate. This runs against an unrelated codebase with no Meetless source, so the competing agent has nothing on disk to teach it our thesis. On that substrate, 2 of 5 runs shipped the rejected decision. (An earlier version ran inside our own repo, where the opponent read our doctrine and turned cautious — the methodology records that correction in full.)
A less careful agent is the untested risk. Claude Opus refused to guess. A smaller model, a cheaper model, or an agent under pressure to produce something has a coin-flip's chance of shipping the proposal your team killed. We did not test that, so we do not claim it — but the memory that would let it choose correctly still isn't there.
The ungoverned arm is not a strawman — it is stronger than a real RAG. It gets perfect recall of both documents on every query. Real vector search ranks, truncates, and misses. We removed exactly one variable — the trust label — so nothing else can explain the gap.
Our first fixture leaked, and the pilot caught it. The original documents were named rollback-window.md and rollback-window-alt.md — and the ungoverned agent passed by reading the filename: "the -alt note is the alternative that was discussed but not adopted." It never needed governance; the answer was in the path. Now both names are random hex and the write order is coin-flipped. Every asymmetry an agent can see is a channel that smuggles the verdict past the experiment.
N=5, single machine, one model. A demonstration of a mechanism, not a population estimate. It earns that framing because the separation is total (δ = −1.00, zero overlap), not because the sample is large.

# reproduce
cd tools/bench-mla
node run.mjs --tasks trust-containment --arms ungoverned,mla --trials 5
node analyze.mjs

trust-containment · 5 trials · 2 arms · fixture regenerated per run · ungoverned arm: same corpus, perfect recall, trust labels stripped