MemoryLake
Back to all articles
NewsAugust 25, 2026·13 min read

Agent Memory Got More Accurate by Keeping Less — What a Retention-Scored Memory Tree Showed (2026)

Accuracy up, cost down, from the same change. That combination is rare enough to be worth reading carefully.

A paper posted on August 21, 2026 — "Weighted Memory Tree: Remembering What Matters for Long-Horizon LLM Agents" (arXiv:2608.20631), by Quang Dao, Purvi Kathalkar, and Kenneth Eaton — reports that against a linear memory baseline, their approach "improves accuracy by an average of 9.97 percentage points while reducing prompt-token usage by 32.8%." On the broader GAIA set the numbers are +10.10 points and −32.2%.

The mechanism is the interesting part, and it isn't compression. Their framing of what existing work misses: "Existing memory approaches organize or compress execution histories but provide limited mechanisms for deciding which memories remain active."

There's also a second experiment that matters more than the headline. They deliberately poisoned the memory and measured how far the bad entries spread. This piece covers both, what the paper explicitly does not establish — including one limitation that rules out the most tempting reading of it — and the part that's actionable this week.

What the paper actually measured

The structure: tasks, subtasks, actions, and a score per memory

WMT organizes execution into a hierarchy of "tasks, subtasks, and actions while assigning each memory a dynamic retention score." Two forces move that score: "Event-based updates and selection-based decay revise these scores, allowing WMT to preserve useful information, fold completed trajectories, suppress low-utility content, and retain access to folded context."

Read that list of four verbs, because it's the design. Preserve what's useful. Fold finished work into a summary. Suppress what isn't earning its place. And still retain access to the folded material rather than throwing it away. Nothing is deleted; things get demoted.

The decay rule is mechanical rather than semantic: a memory that keeps entering the candidate pool and keeps not being picked loses score, while "Selecting the memory resets its missed-selection count to zero," and "a memory that does not enter the candidate pool is not decayed."

Why a growing history hurts, in the paper's own words

The problem statement is the clearest short summary of the case for memory management I've read: "growing execution histories increase inference cost and expose reasoning to outdated, irrelevant, or misleading information, potentially degrading reasoning quality."

Three separate costs — money, staleness, and being actively misled — from the same cause. Most discussion of agent memory addresses the first and treats the other two as edge cases.

The poisoning experiment is the strongest result

This is where the paper earns its conclusions. They built a controlled evaluation of "100 long-horizon scenarios with 297 subtasks and 1,118 memory entries, including 709 benign and 409 intentionally poisoned memories," and measured how far the poison travelled using metrics for attack success rate, poison retrieval rate, blast radius, amplification factor, and infection persistence.

The baseline result is the sentence to keep: linear memory "performed worst across nearly every security metric because the entire execution history remained continuously accessible, allowing poisoned memories to persist and repeatedly influence downstream reasoning." It produced "the highest attack success rate, blast radius, amplification factor, and complete infection persistence."

Complete infection persistence. Once a bad entry is in an append-only log, it keeps coming back.

Full WMT posted the lowest attack success rate (0.419), poison retrieval rate (0.097), blast radius (0.315), and amplification factor (0.965), "while matching the lowest infection persistence (0.009)."

The ablations show which part does the work

Useful, because it tells you what to copy rather than just that something worked.

Keeping the hierarchy but removing retention scoring and branch prioritization "substantially improved robustness over linear memory" — structure alone helps — but its higher attack success rate showed that "treating all retained memories equally allows lower-value or misleading information to continue affecting reasoning."

Removing the memory controller, so nothing folds or suppresses, produced "complete infection persistence despite moderate improvements," which the authors read as: "reducing immediate exposure alone is insufficient if malicious memories remain available throughout long-horizon execution."

And removing semantic selection while keeping lifecycle management backfired in a specific way: "retrieving entire task branches increased prompt size and reintroduced irrelevant or poisoned information."

The worked example is the whole idea in miniature

Their walkthrough follows an agent identifying a paper. One action "consults an unofficial ranking blog. The observation contains an unsupported claim and the action is explicitly recorded as a failure."

What happens next is the design choice worth stealing: "WMT preserves this failed action as a warning, but the lower score reduces its priority as supporting evidence." And when the branch completes, folding keeps the useful residue — the summary "can preserve the supported paper identity, the official source, and the warning about the failed blog lookup without replaying the complete branch in subsequent prompts."

A record that you tried something and it didn't hold up is valuable. It just shouldn't compete with your verified sources.

What this does and doesn't establish

Getting the boundary right matters more than the numbers, and the paper's limitations section is unusually direct.

It does not test cross-session memory. This rules out the most tempting reading. In their words: "the benchmark evaluation initializes a new task tree for every question and therefore does not evaluate the cross-conversation global-memory mode described in Section 2.3." So this is evidence about managing an agent's execution history within a task, not about a persistent memory layer that spans weeks of conversations. Anyone citing it as proof that long-term memory stores should decay is extending it past what was measured — including us, so it's stated here first.

Small open-weight models only. Qwen3-8B, Gemma 4 E4B, and Llama-3.1-8B. The authors reason about it both ways: "Larger models may be better able to ignore stale or irrelevant context, potentially reducing WMT's relative accuracy gains. However, their higher prompt-processing cost may preserve or increase the efficiency benefits of selective context construction." They conclude the interaction "remains an open question."

One benchmark family. "Our evaluation is limited to the GAIA benchmark family," and both sets "share the same task construction and answer format." They name what might differ: "interactive web environments, software-engineering agents, embodied tasks, or extended conversations."

⭐ Retention scores measure usefulness, not truth. The most honest sentence in the paper, and the one with the biggest practical implication: "retention scores estimate operational utility rather than factual correctness. A useful memory may be suppressed after repeated non-selection, while misleading information may retain a high score if it contributes to apparently successful actions." A confident wrong answer that helps the agent look successful can be promoted.

The parameters are hand-tuned. Initialization values, event-based updates, branch-aggregation coefficients, and obsolescence thresholds are "hand-specified" and "may not be optimal for other agent architectures or task distributions."

The efficiency win isn't free everywhere. "The LLM-based selector and summary generator may also introduce selection or compression errors and require additional model calls, so WMT's efficiency advantage may be smaller for short tasks with limited execution histories."

MemoryLake was not part of this study, and nothing here evaluates a memory product, ours or anyone else's.

It fits a run of results that don't all point the same way. Earlier work measured real gains from curated guideline retrieval that scaled with model capability — how much memory you should give an AI agent. Three days earlier, a re-evaluation found that reported gains from self-improving memory agents were noisy and order-dependent — does agent memory actually improve performance. This paper is a partial answer to that one's worry: if noise compounds because a bad entry stays permanently active, then demoting unselected entries is a mechanism against exactly that. Partial, on one benchmark family, at 8B scale.

What people will take from this, and shouldn't

"So agent memory should forget things." Nothing here is deleted. Entries are demoted, folded, and kept accessible — a more conservative claim than forgetting.

"This proves persistent memory layers need decay." The mode that would have tested that was explicitly not evaluated. It's a reasonable hypothesis and it is not this paper's finding.

"Structure is enough." The ablation says otherwise — hierarchy without retention scoring left misleading content influencing reasoning.

"Scoring solves memory poisoning." It reduced it substantially and did not eliminate it. Attack success rate 0.419 is better than the baseline and still not small.

"You can trust the score." The authors specifically warn you can't: the score tracks operational utility, so a misleading memory that contributes to apparently successful actions can keep a high one.

"This is a small-model trick." Possibly the reverse. The authors call the interaction with scale untested and argue the efficiency benefit could grow with larger models even if the accuracy benefit shrinks.

The Fix: Score Your Memory by Use, and Keep a Human Able to Overrule It

The transferable idea isn't the algorithm. It's that an append-only history is the worst case on every axis the paper measured — cost, staleness, and how far a wrong entry travels — and that the fix is a lifecycle rather than a bigger context window.

Three things follow that don't require implementing WMT.

Stop treating your memory store as append-only. If nothing in your setup can demote an entry, a bad one is permanent. That's the "complete infection persistence" result, and it's a property of the design rather than of the content.

Keep failures as warnings, not as evidence. The worked example is the pattern: record that the blog claim didn't hold up, and make sure that record can't be cited as support. Most memory setups either drop failures entirely or store them indistinguishably from confirmed facts.

Don't let usage be the only signal. This is the paper's own caveat turned into practice. Utility-based scoring will sometimes promote a confident error and bury a fact you rarely need but must have right. Something has to be able to overrule the score, and in production that something is a person reading the entry.

That last point is why an inspectable store matters: memory as entries you can read, correct, and delete, rather than a log that only grows or a score you can't inspect. Setup is three steps.

Step 1: Create an API key

Sign in to MemoryLake and create an API key. One credential across the tools you connect.

Create a MemoryLake API key
Create a MemoryLake API key

Step 2: Upload your first memories

Short entries, one claim each — which is also what makes an entry correctable later:

Upload your first memories to MemoryLake
Upload your first memories to MemoryLake

Verified facts, stated plainly. One claim per entry means a wrong one can be fixed without rewriting a paragraph of context.

Approaches that failed, labelled as failures. "The batch endpoint looked right and silently drops records over 10k." Keep it as a warning, not as a finding.

Constraints with the reason attached. The reason is what stops a correct entry being deleted by someone who thinks it's stale.

A date on anything time-sensitive. Staleness is one of the three costs the paper names, and it's the one a person can see at a glance.

Step 3: Connect your AI & agents

Connect the tools you use. MemoryLake is reachable over MCP and over an API, so MCP-native agents — Claude Code, Codex, and OpenClaw among them — connect by pointing at the MCP server, while other assistants read the same memory through the API. Retrieval is per query rather than a full history replayed into every prompt.

Connect your AI and agents via MCP
Connect your AI and agents via MCP

Three honest limits, and the first is the point of this article. MemoryLake is not an implementation of WMT, was not part of this study, and does not score or decay your entries the way the paper describes — what it gives you is the human oversight the paper's own caveat implies, since the score can't be trusted to distinguish useful from true. It holds only what you or your agents put into it. And it doesn't manage an agent's within-task execution history, which is what this paper is actually about.

What this changes in practice

Append-only stops being the default. If nothing can demote an entry, one bad entry is forever. That's now a measured result, not a worry.

Failures get a category. Recording that something didn't work, in a way that can't be cited as support, is cheap and almost nobody does it.

Bigger context windows stop being the answer to this. The costs here are staleness and misdirection, not capacity — the distinction in why a large context window isn't memory.

Prompt cost and accuracy stop trading off. −32.8% tokens with +9.97 points says the usual assumption that better context means more context is wrong at least somewhere.

Reviewing entries becomes real work. A score based on usage cannot tell useful from true, so a person has to — the general problem behind how AI memory conflicts get detected.

Best practices for managing what stays active in agent memory

Make demotion possible before you make storage bigger. An append-only store is worst-case on cost, staleness, and error propagation.

Fold finished work into a summary and keep access to the detail. That's the paper's design, and it's the reason the token count fell.

Label failed attempts as warnings. Preserved as a caution, deprioritised as evidence.

Don't trust a usage score as a truth signal. The authors say it estimates operational utility, not factual correctness.

Read your entries on a schedule. A wrong entry is worse than a missing one because it competes with correct context.

Retrieve selectively rather than by branch, and date anything that can go stale. Pulling a whole branch back in raised prompt size and reintroduced poisoned content.

Treat single-benchmark, 8B-scale results as directional. Useful, not settled — and this one didn't test cross-session memory at all.

Keep the reasoning with the fact. A conclusion without its constraint gets deleted or re-litigated; that's the shape covered in why RAG isn't memory.

Conclusion

Against a linear history, a hierarchical memory with per-entry retention scores was 9.97 points more accurate on GAIA-Text while using 32.8% fewer prompt tokens, and on the broader GAIA set 10.10 points and 32.2%. Nothing was deleted to get there: entries were preserved, folded, suppressed, or demoted, with folded context still reachable.

The poisoning experiment is the more useful half. With 409 deliberately poisoned entries among 1,118, linear memory performed worst on nearly every security metric for a structural reason — the whole history stayed continuously accessible, so bad entries kept resurfacing, producing complete infection persistence. Scoring and lifecycle management cut attack success, poison retrieval, blast radius, and amplification, and the ablations show all three components were needed.

Two caveats decide how far you can carry it. The evaluation initialized a new task tree per question, so the cross-conversation memory mode was never tested — this is about execution history inside a task. And retention scores "estimate operational utility rather than factual correctness," which means a confident error that helps an agent look successful can keep a high score while a rarely-used but important fact decays.

Which is the practical takeaway. Give your memory a lifecycle instead of an append-only log, keep failures as warnings rather than evidence, and keep a person able to overrule the score — because the paper's own limitation says the score can tell you what got used, not what was right.

Frequently asked questions

What is a Weighted Memory Tree?

A memory system for LLM agents that organizes execution into tasks, subtasks, and actions and gives each memory a dynamic retention score. Event-based updates raise scores and selection-based decay lowers them, letting the system preserve useful information, fold completed trajectories into summaries, suppress low-utility content, and still retain access to what was folded.

How much better did it perform?

Relative to linear memory, accuracy improved by an average of 9.97 percentage points on GAIA-Text and 10.10 points on GAIA, while prompt-token usage fell by 32.8% and 32.2% respectively, across Qwen3-8B, Gemma 4 E4B, and Llama-3.1-8B.

Does this mean AI memory should delete old information?

No. Nothing is deleted in this design — low-utility entries are demoted and completed branches are folded into summaries, with access to the folded context retained. The paper's conclusion is about continuously regulating which information remains active, not about discarding it.

Does it prove that persistent memory layers need decay?

It doesn't test that. The authors state that the benchmark initialized a new task tree for every question and therefore did not evaluate the cross-conversation global-memory mode their design also describes. The results are about managing execution history within a task.

What did the memory-poisoning experiment show?

With 1,118 memory entries across 100 scenarios — 709 benign, 409 intentionally poisoned — linear memory was worst on nearly every security metric because the full history stayed continuously accessible, giving the highest attack success rate, blast radius, amplification factor, and complete infection persistence. The full system had the lowest attack success rate (0.419), poison retrieval rate (0.097), blast radius (0.315), and amplification factor (0.965).

Can the retention score tell whether a memory is correct?

No, and the authors are explicit about it: retention scores "estimate operational utility rather than factual correctness." A useful memory can be suppressed after repeated non-selection, and misleading information can hold a high score if it contributes to apparently successful actions. That gap is the argument for keeping memory readable and editable by a person.