How Much Memory Should You Give an AI Agent? What an 8-Model Study Found (2026)

"Memory" here means distilled guidelines, not transcripts

This is the first thing to get straight, because the word covers several different things. The authors are explicit: "'Memory' here doesn't mean replaying a past transcript. It means a guideline set — strategies that worked, mistakes to avoid, and edge cases — distilled from the agent's own prior trajectories."

The loop is: the agent attempts tasks and produces trajectories; the system extracts behavioral guidelines "from both its successful and unsuccessful runs"; it consolidates them into a reusable set; and at inference time the agent receives either the full set or a selection from it. Critically, "No model weights are updated." What changes is "the guidance available to the agent," not the model.

So this is not a study about conversation history, and not a study about long context windows — a distinction worth keeping separate from why a large context window isn't memory.

The benchmark and the two metrics

Evaluation ran on AppWorld — "585 multi-step tasks (168 test_normal + 417 test_challenge) across 9 simulated apps (calendars, messaging, payments, and so on)."

Two scores, and the gap between them turns out to be the interesting part:

TGC — Task Goal Completion. "The share of individual tasks the agent completes fully and correctly." The headline did-it-work number.

SGC — Scenario Goal Completion. "A stricter, all-or-nothing metric." Each scenario bundles several variants of the same task, and "SGC counts a scenario as passing only if the agent succeeds on every variant. It measures reliability."

The three configurations

Both memory configurations draw from the same guideline set, "mined once… from AppWorld's training split only," and the authors note "no test-split data ever goes into building it." What differs is delivery:

Baseline — "No memory — the agent as shipped."

Full guideline set — "Every mined guideline, injected on every ReAct step."

Curated retrieval — "A fixed, high-confidence core of those same guidelines plus a few task-relevant ones retrieved for each task (a fixed portion + a variable portion)."

Three things stand out.

Strong models absorbed everything. Models with headroom did best on the full guideline set, "including rare edge-case lessons," because "They have the capacity to absorb and apply all of it."

Weaker models were hurt by volume. "Smaller or weaker models get drowned by a large guideline set." For gpt-oss-120b, the full set "gained less and cost ~50% more tokens" than curated retrieval did.

The strict metric moved more than the headline one. DeepSeek gained +9.5 TGC but +16.1 SGC, because "good guidelines especially help an agent clear every variant of a scenario, not just the average case." Even the near-ceiling models kept gaining on reliability: GPT-5.5 and Opus added +7.2 and +7.1pp SGC. The authors' formulation is the one to remember — "Memory keeps paying off as long as a model has a remaining failure mode to target."

And one model showed nothing. GLM-5 scored identically with and without memory on these tasks. What that means is where the authors are most careful, and it's covered below.

The reflexive objection to injecting guidelines on every step is cost, and it's a fair one. Their measurements:

So for the weaker model, selection won on both axes at once: "+16.1pp TGC at only +5% tokens." As the authors put it, "Better performance here does not require more inference cost."

Two supporting findings matter for anyone budgeting this. First, the overhead is input inflation rather than more work: "DeepSeek runs about the same number of ReAct steps with memory as without (≈18–19 on average), so the added cost is input-token inflation, not longer trajectories." Second, there's a standard lever for it — "The real efficiency lever in production is prompt caching: the static portion of the guideline set is identical across steps and can be cached, cutting effective cost substantially." They recommend designing for it: "keeping the shared guideline-set prefix stable so it stays cacheable."

That last point has a direct design implication. A memory layer that reshuffles its output on every call defeats caching. One with a stable core and a small variable tail is cheaper to run — and it's also the configuration that won on accuracy for the weaker model.

The authors are unusually disciplined about their own limits, and quoting them properly is the honest way to read the result.

"Saturated" describes an observation, not a cause. On GLM-5's flat result: "We call this the saturated pattern — the label describes what we observed, not a proven cause. The model may already have been near its ceiling on these tasks, the guidelines may not have addressed its remaining failures, or it may not have applied the guidance effectively." That is not a finding about the model's quality. It's one benchmark, one guideline set, one configuration.

Capability tier isn't parameter count. "What puts a model into one pattern rather than another isn't simply parameter count. Benchmark headroom, context-window size, architecture, guideline quality, and task distribution all appear to shape where a model lands, and separating those factors is ongoing work." Note that GLM-5 at 745B and DeepSeek at 671B landed in different patterns.

The context-window explanation is a hypothesis. They think larger windows may absorb full sets better, and say directly: "We have not yet run controlled experiments isolating this factor."

It's one benchmark. "These results are validated on AppWorld — a rigorous multi-step benchmark, but a single one." Broader benchmarks and real deployments are listed as in progress.

The retrieval itself is admittedly imperfect. "Our current retrieval ranks guidelines by cosine similarity, which we've shown doesn't perfectly predict which guidelines help a given task." A learned selector is named as the next step — which means the curated-retrieval numbers were achieved despite a blunt selection method.

Very weak models are out of scope. "Below a minimum capability baseline, self-distillation lacks signal."

Read together: the direction of the finding is well supported, and the specific numbers belong to this benchmark and this method. Don't quote the percentages as what your system will do.

Treating storage as the whole problem. Most memory setups optimise for capturing more. This study says the capture is the easy half — what gets injected, per task, is where the accuracy and the cost both live.

Injecting everything because context is cheap now. Cheap isn't free, it was +51% to +78% here, and for the weaker model more context made results worse.

Assuming a bigger model needs less help. The near-ceiling models still gained meaningfully on the reliability metric. Headline accuracy hid it; the strict metric didn't.

Judging memory by average-case success. TGC understated the benefit throughout. If your agent is "usually right," the thing memory fixes may be the variant it fails, which is exactly what SGC measures.

Concluding that memory doesn't work when one model shows no gain. The authors explicitly decline to draw that conclusion from their own data.

The practical translation of this study is not "store less." It's that a memory layer has to be able to select, keep a stable core, and be prunable — because the dose is a decision you make per model and per task, not once at setup.

Split your memory into a core and a tail. A small, high-confidence set that's always injected, plus task-relevant entries retrieved per task. That's the configuration that won on both accuracy and cost for the weaker model, and it's the one that stays cacheable.

Keep the core stable so prompt caching works. If the always-on portion is byte-identical across steps, you get the discount. Reordering it silently throws that away.

Test both doses on your own model. Full set versus core-plus-retrieval, measured on your tasks. The study's own conclusion is that the answer differs by model.

Measure the strict metric. Don't just count tasks completed; count how often every variant of the same task passes. That's where these gains showed up.

Prune, since guideline quality is named as a factor. Wrong or stale entries aren't neutral — they're injected with the good ones.

Doing that requires memory that lives somewhere you can inspect and edit, rather than accumulating inside one assistant. That's what MemoryLake is: a memory layer your agents read from over MCP or an API, where entries are individual and editable rather than an opaque growing blob — which is what makes calibrating the dose possible at all. Setup is three steps.

Step 1: Create an API key

Sign in to MemoryLake and create an API key. One credential across the tools you connect.

Step 2: Upload your first memories

Write entries the way the study's guideline sets are shaped — one claim each, behavioral where possible: what worked, what to avoid, and the edge case that broke a previous attempt. The categories that earn their place:

Corrections you've made more than once. These are your highest-confidence core entries, by definition.

Failure modes and how to avoid them. The study's gains came largely from clearing the variant that used to fail.

Constraints that look arbitrary. Rate limits, ordering requirements, the API that behaves differently than documented.

What's already been ruled out. The category no artifact contains, and the one every fresh agent proposes again.

Step 3: Connect your AI & agents

Connect the tools you use. MemoryLake is reachable over MCP and over an API, so MCP-native agents — Claude Code, Codex, and OpenClaw among them — connect by pointing at the MCP server, while other assistants read the same memory through the API.

Three honest limits. MemoryLake was not part of this study, and none of the percentages above are claims about it — the results belong to the authors' own method on their own benchmark. It holds what you or your agents write into it, so quality is your input. And it doesn't change a model's capability; if your agent's remaining failures aren't the kind guidance fixes, better memory won't fix them either.

"More context" stops being the default fix. For a weaker model, more made it worse. Selection is the lever.

Reliability becomes the metric you watch. The strict all-variants measure moved more than headline accuracy at every tier, including the near-ceiling models.

Cost stops being the argument against memory. +5% tokens for +16.1pp on one model, and prompt caching on a stable core for the rest.

Model swaps become re-calibrations. Change models and the right dose may change with it, since capability tier isn't predictable from size.

Pruning becomes a performance activity. Guideline quality is listed among the factors that decide which pattern a model lands in — which makes deleting stale entries real work, not tidying. The token side of this is covered in reducing token usage with a memory layer.

Start with a small high-confidence core. Grow it only when you can show the addition helps.

Retrieve per task for everything else. A few relevant entries beat the whole set for models without headroom.

Keep the always-on prefix byte-stable. Cacheability is the difference between affordable and not.

Run the A/B on your own tasks. Full versus curated, on your workload. The study's headline finding is precisely that the answer isn't universal.

Score all-or-nothing, not average. Reliability is where guidance pays.

Date and delete. Stale entries get injected too.

Don't read a flat result as proof memory is useless. The authors named three possible explanations for their one zero-gain case and committed to none.

Keep memory outside the model. No weights were updated in this study — that's why the approach was "cheap to adopt and portable across the eight models we tested." Portability is a property of keeping it external, the general case in what persistent memory means.

The finding worth carrying is the one the authors put in a section heading: "Memory Should Be Calibrated, Not Merely Accumulated." Strong models with headroom used the full self-distilled guideline set. Weaker models did better with a compact core plus per-task retrieval — and did it at +5% tokens instead of +51%. One model showed no measurable change, and the authors deliberately declined to explain why from a single benchmark.

For your own agents that translates into three habits: separate a stable always-on core from a retrieved tail, measure reliability rather than average success, and treat pruning as part of performance. And keep the memory itself somewhere inspectable and editable, because "how much" is a dial you'll be turning every time you change models — not a switch you flip once.