Anthropic's agent memory study: fixed save-rules cost Claude its gains, and 9 files beat 413

Anthropic published a research post yesterday called The components of agent memory (Lance Martin, Nov 6, 2026). It lays out how they think about memory for agents, backed by three evaluation settings: the CL-Bench 1.0 database task run with a fresh session per question, Claude playing Pokémon Red with a writable memory folder, and a corpus of roughly 350,000 files (about 420M tokens) of internal documents from Stainless, a company Anthropic acquired in May. I read it twice, because several findings land directly on files you probably maintain: memory directories, the memory section of CLAUDE.md or AGENTS.md, and instruction files generally. Here is what they measured and what I am changing. What they mean by memory Anthropic defines agent memory narrowly: knowledge Claude gains from experience, kept between sessions, saved as context it can read later. The definition explicitly excludes three neighbors people confuse it with: changes to model weights, in-context learning inside one conversation, and compaction summaries. This is about persistent files you could open in an editor. We explained how Claude Code loads these files in CLAUDE.md and Claude Code memory, explained. This post is the first hard evaluation evidence I have seen about what to put in them. The setup: fresh sessions, one persistent folder On the CL-Bench database task, every question starts a fresh session. The only thing that carries over is a memory directory. The progression across model generations is the interesting part: Sonnet 4.6 generated theories about wrong answers and saved them untested, scattered across files. Opus 4.7 consolidated into a single reference document and marked which facts it had checked and which were still unconfirmed. Sonnet 5 marked each answer as confirmed after the grader checked it, and wrote corrections when it had been wrong. Opus 5 was the first to close the loop: most entries carried a rule, how it knew the rule was true, and warnings about traps it had hit. Opus 5.5 and Sonnet 5.5 also closed the loop and scored about the same as Opus 5 with memory. Newer models build their own learning loop with less scaffolding from you. Fixed save-rules made the strongest models worse This is the finding to sit with. On CL-Bench, simply telling Opus 4.7 and Opus 5 that a memory folder existed was enough for a noticeable performance boost. Then Anthropic tested specific rules about what to write, one rule per run. A "record only failures" rule made both models lose much of that gain. Their summary: it helps to tell Claude what memory is for, but not how to use it. For stronger models, freedom to judge salience beat fixed rules, and the rules could actively weaken performance. If your config file carries a long list of what to save, that list is the thing to reconsider. The purpose-first shape that worked: # Memory This folder holds knowledge future sessions need and cannot cheaply re-derive. Write a note when someone asks you to remember something, or when you verified something the hard way. Record how you verified it. The rule-list shape that lost performance in the eval: # Memory rules - only record failures - one file per bug - never store commands or paths One caveat, and it matters for hybrid stacks: steering helped the older models most. Sonnet 4.6 often never read its memory files and did not test its theories. Prompting it to read memory before answering, then explicitly test its theories, nearly doubled its benchmark score. Sonnet 5 does this on its own, reading memory before answering 9 times out of 10 without prompting, where Sonnet 4.6 did it less than half the time. Rule of thumb: frontier model, state the purpose and stop. Older model, spell out the read-then-test loop. 413 files earned one badge. 9 files earned four. The salience test was Pokémon Red with a writable memory folder. Sonnet 3.7 wrote everything down: a file per battle, per item found, per NPC met. That came to 413 memory files across 225 folders, and it earned one badge. Around step 10,000 it spent 17 steps creating six files and deleting nine others without taking a single action in the game. Fable 5 wrote 9 files in 2 folders plus a single global.md dashboard, and earned four badges. Opus 5.5 kept a small folder tracking its current goal and next steps, and beat the game. Anthropic's distinction: state you can read off the screen every turn, like money or map position, is redundant in memory and goes stale immediately. Memory belongs to things that only exist if the agent works them out and writes them down: strategy, plans, tested rules. A good note from the Pokémon run, verbatim: "If 3+ balls fail in a row, VARY the timing (insert extra 'wait' presses between menu steps)". That is a rule, its trigger condition, and the fix, in two lines. Audit your agent's memory directory for screen-readable facts: current branch names, package versions, CI status, directory listings. All of it is stale by the next session. Wrong notes compound. Dreaming is the correction pass. A note written confidently wrong gets read and trusted later. In the Pokémon run, Claude wrote down the wrong side of the map for a PokéCenter it was using as a landmark, walked off a one-way ledge three times because of it, and eventually gave up without ever reaching the next city. Dreaming fixed it. Dreaming, in Anthropic's usage, is a background pass that checks memory notes against the original session transcripts, which serve as ground truth. Claude caught the error and rewrote the note, then went straight back through the cave and reached Cerulean City. The precondition is the part everyone skips: dreaming can only verify notes against transcripts you still have. If your setup deletes session transcripts when the session ends, nothing can audit the notes. We watched the same ground-truth problem eat agent state in What survives compaction. Anthropic shipped a cookbook workflow for this. You export finished sessions as transcript files, an analyzer agent verifies what memory claims against each transcript, enriches it with exact identifiers and staleness notes, and organizes it into small linked files with an index. An organizer pass tidies the whole store, and the next session mounts it read-only. The layer that loads every session is a budget Dreaming can also promote notes to the top of a memory hierarchy, where the harness injects them into every session. Anthropic is blunt about the cost: injected memories consume tokens, an incorrect promoted note rides along in every session, and loading a note does not guarantee the model acts on it. Their Claude Code documentation warns that longer instruction files reduce adherence, which matches what we measured for the 300-line instruction budget. Claude Code's auto memory follows the pattern the post recommends: only the first 200 lines or 25 KB of MEMORY.md load into context, and that header acts as an index pointing at everything below it. The wiki numbers back the layered approach. On 25 questions over the Stainless corpus, wiki-only answering grew from 13% to 48% of graded criteria as dreaming scaled from 2 to 128 sessions. Search plus the wiki scored 61% where search alone scored 53%, and the combination saved about 20,000 tokens per question. On 24 merger questions where more than half required combining several documents: 87% with the wiki against 73% with search. But on tasks answerable from a single document the wiki showed no clear gain, and on one task the wiki grew larger than the source documents themselves. Representations need pruning, or they stop abstracting anything. A layout consistent with all of that: memory/ MEMORY.md # index only, loads every session,