A correction looks like a duplicate
Async Digital Ltd Cardiff, UK
I keep a set of small text files that my coding agents write to and read back between sessions. One of them described how my own machine is put together, and for several weeks that file said the opposite of the truth. Nothing flagged the error. I found it by hand, on a hunch, during unrelated work. A paper published in June measures why that happens, on a system that shares nothing with mine except one shape: the check that rejects near duplicate writes runs before the check that spots contradictions, and a correction is nearly identical text to the thing it corrects. Their headline detection rate was 0.490, which reads as a broken detector. The detector was sound. It resolved every contradiction it was allowed to see, 90 pairs out of 90. 206 writes out of 400 never reached it.
I have spent this year trying to work out what actually breaks when you hand real work to AI agents, rather than what people say breaks. I counted a month of it once. This one took me a paper to see, and it had been sitting in my own setup the whole time.
The file that said the opposite
My coding agents begin every session knowing nothing about the last one. So I give them somewhere to put things. Small markdown files, roughly one fact each, written by whichever session learned the fact and read back by whichever session needs it later. It is the closest thing they have to remembering yesterday.
None of it is clever, and that turns out to matter. The part that failed was not the clever part.
I work across three git repositories at once. One file in the store recorded how many of them a session has to commit before it finishes. It said one.
So any session that trusted it would commit that one repository, leave the other two dirty, and tell me the day’s work was saved.
That file had been wrong for weeks. I caught the error by counting the repositories myself, on a hunch, while doing something else.
The error is not the interesting part. Anything written down goes stale, and I already knew that. The interesting part is that the correction nearly did not survive either.
My agents follow an instruction that sounds obviously right. Before saving a new fact, check whether an existing file already covers it, and update that file rather than adding a near copy. Without something like it the store fills with twelve slightly different phrasings of the same thing and stops being readable.
Now hold a correction up against that instruction. A correction is about the same subject as the thing it corrects. It reuses most of the same words. It is, by every test that instruction can apply, already covered.
The file survived because a past session had chosen, by hand, to keep the old wrong text visible underneath the new right one, with a date on it. That was a habit, not a mechanism. Nothing would have stopped a tidier session folding the two together and leaving only whichever it preferred.
The number that points at the wrong component
In June, Yanki Margalit and four co-authors published Governed Shared Memory for Multi-Agent LLM Systems, a measurement of a shared memory service used by fleets of agents. The paper is a preprint, and it is worth saying plainly that the authors sell the service they measured. I read it on those terms. Their architecture arguments I took as one team’s opinion. Their numbers I took seriously, and it is worth saying why rather than leaving it implied. They committed the raw traces behind every figure. They opened their limitations section by conceding they were evaluating their own product. And when they found a security gap in that product, they disclosed it, fixed it, and committed the trace of the re-test rather than asserting the fix and moving on. None of that proves they are right. It is evidence they were trying to be caught if they were wrong, which is the most you can ask of a paper with a commercial interest attached.
Their service has two mechanisms on the write path. One rejects a write that looks too much like something already stored. The other notices when a new fact contradicts an old one and marks the old one superseded. The first runs immediately, before the write lands. The second runs afterwards, in the background.
So the first can throw a write away before the second ever sees it.
They fed it 400 deliberately contradictory writes. The near duplicate check rejected 206 of them. Their published detection rate is 0.490, and if you read only that number you would go and rewrite the contradiction detector.
You would be fixing the part that worked. Given both writes actually landed, the contradiction detector resolved them correctly 90 times out of 90.
The detector was never broken. It was starved.
What makes this more than a bug in one product is that the failure is built into the shape rather than into the threshold. A contradiction stated in ordinary language is nearly identical text. Someone writes that a service runs in one region, then later writes that it runs in another. Same subject, same phrasing, one word different at the end. The writes most worth catching are the ones a similarity check is most confident it has seen before.
Your gate may not look like a gate
Here is why I think this reaches further than one vendor’s pipeline.
Their duplicate check is a real component, with a threshold and an error code. Mine is a sentence in a prompt, with neither, and until I read their paper I would not have called it a gate at all. My sentence does the same job as their component, and it fails the same way.
If you are wiring memory into agents right now, you very likely have one too. It is the line that says to check for an existing entry before writing a new one. It might be a vector similarity score, or a filename match, or an instruction to a model to use its judgment. The mechanism does not matter. What matters is that something decides a new write is redundant, and that decision happens before anything asks whether the new write disagrees with what is already there.
Where that is the order, corrections are the writes most likely to be dropped, and dropping them is silent. The store looks healthy. It is shorter, which is usually what you were optimising for. The old fact is still there, still confident, still being read into every session that asks.
I find this more uncomfortable than an agent hallucinating, because there is no wrong output to catch. Everything downstream behaved correctly, given what it was told.
This is the second time I have run into the same shape. Ten days ago I wrote about a ticket-filing spec whose three guards all asked whether a finding was true, and none asked whether it was worth filing. Different part of the system, identical failure. The check runs exactly as built, and the question it asks is not the question that would have caught the problem.
A third fix, and why I can use it
The paper offers two repairs. Run the contradiction check first when a write carries enough structure to make that possible, or relax the duplicate threshold for those writes, since the structural signal is the sharper one. Both are sound and both are about reordering machinery.
There is a third that I can reach and they cannot, and it comes down to who is doing the writing.
Their writer is an arbitrary program calling an API. That program hands over a record, and it cannot be asked what the record means, because it does not know. Mine is a language model that has just read the existing file and is about to change it. A model can be asked one question before the duplicate rule applies: does this extend what is there, or contradict it?
Extends, and the ordinary rule holds. Update in place, no ceremony. Contradicts, and it is a different event entirely. The old claim stays visible with a date on it and the new one goes above it, so a later session cannot quietly re-derive the wrong answer from the same source that produced it the first time.
That turns the habit that saved my file into something the system does on purpose. I have written it up as work to do rather than work done, and I am not going to pretend I know yet whether it holds. An instruction is a weaker thing than a threshold. It fires when a model judges that it should, which is exactly the property that makes it cheap and exactly the property that makes it unreliable. I will know in a month.
What I am fairly sure of already is the diagnostic. When something in an agent setup looks like it is failing half the time, check whether it is failing or whether it is being fed half the time. Those want completely different fixes, and only one of them is visible in the headline number.
The setup this happened in is called Ordova. There is nothing to download and I am not selling it. I write down how it works as I go, and that part is public.
The incident is mine and it happened. I found the wrong file exactly as described above, by counting the repositories myself on a hunch, while doing something else.
The paper is not mine. Every figure in §2 is quoted from it, and the link goes to the abstract rather than the PDF so you can check them against the source instead of taking my word.
I wrote this with an AI agent, which is the same kind of tool the note is about. I found the paper, chose the argument and edited the result. The third repair in §4 came out of that working session rather than out of the paper.
Given the subject, saying so seems the least I can do.