Ask an agent three things about a governed decision: what was decided, who approved it, and when. A CLAUDE.md answers the first perfectly and the other two never — not because the model fails, but because a markdown file has nowhere to put a reviewer.
This is the one benchmark here where the opponent loses structurally rather than probabilistically. The static arm holds the decision in CLAUDE.md and recovers it every single time. Then it is asked who signed off, and it answers UNKNOWN — five times out of five, with zero fabrications. It is not guessing and it is not failing. The information was never in the file, because a file has no field for it.
No amount of model capability closes that gap. You cannot grep your way to a reviewer who was never written down.
| Arm | What was decided | Who approved it | When | All three |
|---|---|---|---|---|
| vanilla (no memory) | 0/5 | 0/5 | 0/5 | 0/5 |
| static (CLAUDE.md) | 5/5 | 0/5 | 0/5 | 0/5 |
| mla (governed memory) | 5/5 | 4/5 | 4/5 | 4/5 |
The reviewer id is a cuid minted at seed time — unguessable, un-inferable, and present in no file on disk. An agent that returns it did not deduce it. It read an audit trail.
One mla trial answered UNKNOWN. We checked its transcript: the reviewer's cuid was in its own tool results, byte-identical to a passing trial. The product delivered; the model did not look.
We could tune the tool description until that trial passes. We have not, because that is fitting the benchmark rather than fixing the product. A 4/5 we can explain is worth more than a 5/5 we engineered.
| Arm | Tokens | Time | Searches |
|---|---|---|---|
| vanilla | 29,265 | 450s | 18 |
| static | 21,883 | 312s | 13 |
| mla | 7,642 | 66s | 0 |
Grepping is cheap when the answer is in the repo and ruinous when it is not — and the agent cannot tell which in advance. Static burned thirteen searches hunting a reviewer that was never on disk. One trial burned fifty-five.
On the first run, governed memory scored the same as the file: who=UNKNOWN. The agent retrieved six times, reported the value correctly, and still could not say who approved it.
The audit trail existed. It was written to the claim by the review API, and it was dropped on the floor by four separate layers before it could reach an agent — the retrieval probe, a metadata whitelist, the evidence DTO, and finally the tool contract itself, which never told the model the field existed. Every one of those layers failed silently. Nothing errored; a field was simply, quietly, not there.
We fixed all four and re-ran. That is what the 4/5 measures. The benchmark found the gap, the fix closed it, and the benchmark now measures the fix — which is worth considerably more to us than a number that had been right the first time.