Anthropic's Own Agents Overloaded Its Test Selection Service — The Case to Rebuild Was Argued in One Months-Long Session (2026)

The setup is documented plainly. Anthropic engineers "ship 8x as much code per quarter as they did from 2021-2025," and "Writing code is no longer the constraint, and once PR review gets accelerated, CI starts feeling the pressure." Meanwhile "the amount of tests across our codebase grew 10x and we added a nominal amount of engineers." The result: "Our CI job volume increased 25x over 6 months."

Rather than run every test on every change, Anthropic built what the post calls a "deterministic test impact analysis or test selection service that determines which tests run on each change based on past performance and package relevance." It has two halves. "A “listener” records the test results from every CI run." "A “selector” reads the test result history and determines which tests run on which opened PRs."

That two-part design is the whole story. One component writes down what happened. The other reads what was written and decides from it. While the writer keeps pace with reality, the reader's decisions are grounded. When the writer falls behind, nothing breaks — the reader simply keeps deciding, confidently, from a record that no longer matches the world.

And it did fall behind. "the listener starts to increasingly fall behind the PR queue." The magnitude is specific: "20 minutes of listener lag can translate into tens of thousands of test updates not being applied to the selector." So is the cause: "All of this ran as a single process because keeping a running history per test meant a single writer needed to apply the results," which "prevented us from being able to horizontally shard."

Then comes the paragraph. Describing how he drove the long-term fix, Malhotra writes: "I started a long-running session in an internal version of Claude Tag dedicated to monitoring the service." It was event-driven — "Anytime the listener lag would get more than 50,000 jobs behind, Claude would ping me and resume our conversation on next steps." And then the line that matters: "This would go on for months, and it was helpful not having to constantly remind it of past efforts or context."

He adds one more sentence worth sitting with: "Claude often argued for an overhaul, but we usually settled on another patch."

Let us be precise about what is being claimed, because it is easy to inflate.

Anthropic is not announcing a memory product here. The post is an engineering retrospective about test selection, and the session in question ran in an internal build of an internal tool. Nothing in it says this is how anyone else should work, and nothing in it describes a feature you can go turn on.

Anthropic is also careful not to overstate the damage. When the service fell behind badly, the post says: "To be clear, this doesn’t mean CI never ran on those PRs, or that untested code was pushed to production." What happened was narrower: the selector was "using stale data to decide what to run and what not to on PRs." That is an honest, bounded description of a failure, and it deserves to be quoted rather than dramatized.

What the post does establish is two things, both first-hand.

The first is that a decision which took months to make was held together by a conversation that did not restart. The value named is not intelligence and not speed. It is "not having to constantly remind it of past efforts or context" — the cost of re-establishing where you already were. Anyone who has reopened a thread from six weeks ago and spent twenty minutes reconstructing why the obvious answer was rejected in week two knows that cost.

The second is that the same post, entirely independently, describes what goes wrong when a written record lags behind the decisions made from it. The lag was invisible. It produced no errors. It produced confident decisions from stale state.

Those two observations are the same observation, pointed in opposite directions. When the record keeps up, months of argument stay coherent. When it falls behind, every downstream decision quietly degrades and nothing tells you.

"Long context solves this." It doesn't, and the post accidentally shows why. A session running for months across an unknown number of alerts is not a context window problem; it is a question of what survives between activations. We wrote that distinction up separately in why a long context window isn't memory.

"So retrieval over the chat logs would have worked." Retrieval finds text resembling your query. What carried this argument forward was a settled position — that an overhaul was warranted, that three patches had been tried, that each bought less time than the last. That is a conclusion, not a passage, and searching transcripts for it is a different operation than holding it. We drew that line in why RAG isn't memory.

"Anthropic proved agents should decide architecture." The post says the opposite, gently. Claude argued for the overhaul repeatedly and was repeatedly overruled by humans with other priorities, and the post frames the eventual agreement as the engineer's own lesson — "always plan for the exponential." The session's contribution was continuity, not authority.

"This is a coding-agent story." The mechanism has nothing to do with code. A recording layer falling behind a decision layer is what happens when your assistant's stored summary of your preferences was written in June and you changed your mind in August. No error appears. The answers just get subtly wrong.

The reason a months-long session worked is that it preserved a small number of settled facts across a large number of interruptions. You can get that property deliberately, without depending on any one vendor's session staying open.

Step 1: Separate the running log from the settled position

At the end of any argument spanning more than one sitting, there are two artifacts. There is the transcript, which is long and mostly consists of options you rejected. And there is the position, which is short: what was decided, what was tried, what it cost, and what would change the answer.

Write the position as prose, in your own words, separate from the transcript. Four or five sentences is usually enough. "We tried a bigger machine; it lasted about seventy days. We tried sharding by package; it lasted about a month. We tried daily restarts; it lasted a day. The next step is a redesign, not a fourth patch."

That paragraph is what a months-long session actually carried. The rest was scaffolding.

Step 2: Date the position and record what it replaced

The failure mode in the Anthropic post is a record that fell behind without announcing it. Your own notes fail the same way: they get older while you get newer, and nothing marks the gap.

So when a position changes, do not silently overwrite it. Write the new position, note the date, and keep one line about what it supersedes. "As of March, restarting is no longer considered a fix." A stored fact that knows what it replaced can be checked; one that doesn't can only be believed. If you have never looked at how contradictory versions in your own store resolve against each other, memory conflict detection is worth understanding first.

Step 3: Put it somewhere the next session can read without being told

The last property makes it durable. The Anthropic session worked because the assistant already had the context when it pinged; nobody re-briefed it at 2 a.m. when the lag alert fired.

Reproduce that by keeping the position in a store your tools read at the start of work, rather than a document you remember to paste. The test is simple: open a brand-new conversation, in any assistant, and ask what was decided. If you have to explain it first, the position is not stored — it is just written down. That distinction is the subject of sharing context between sessions.

MemoryLake exists to be that separate, addressable layer: a store for the decisions you want every assistant to already have, rather than a place that reaches into anyone else's product. You write the entries yourself, in your own words. Nothing is pulled out of Anthropic's systems, OpenAI's systems, or any other vendor's store, and nothing here reads, restores, or modifies what those vendors hold.

Step 1: Create an API key

Sign in and generate a key from the dashboard. The key is what lets your assistants and agents reach the same layer, so every surface reads one set of positions rather than whatever is in that tool's own history.

Step 2: Upload your first memories

Start with the positions, not the transcripts. Take the two or three arguments currently live in your work — the architectural one, the process one, the one about which vendor you are standardizing on — and write each as a short dated paragraph, with what was tried and what it cost. That is the highest-value thing you can store, because it takes longest to reconstruct.

Step 3: Connect your AI & agents

Point your assistants at the layer so positions load at the start of a session rather than being pasted into it. Then verify by opening a genuinely fresh conversation and asking what the current position is. Reading it back is the only proof the connection works; a panel showing a green state is not the same thing.

The practical difference shows up at the moments when nothing is happening.

A months-long argument is not a months-long conversation. It is perhaps fifteen real interactions across a hundred days, with weeks of silence between them. The silence is where positions rot. Someone tries the thing, it doesn't work, they move on, and three weeks later the same suggestion comes back because the cost of the last attempt was never written anywhere durable.

Anthropic's post makes the same point about a machine rather than a person: when the record lagged for more than an hour, "a ton of job results weren’t recorded by the listener," and the selector went on deciding anyway. Nothing alerted. The decisions just got worse.

The second difference is scale of attention. The post notes that "agents push overnight and on weekends, but it remains bursty as human engineers still drive and approve a significant amount of PRs." The floor of activity has risen; the humans have not gained hours. If you are supervising more work across more surfaces, the ratio of re-briefing to thinking determines whether you can do it at all — which is really a question about how much memory an agent should be given.

The third is auditability. The redesign worked because the team could finally see the lag as a number. Your positions deserve the same treatment: a store you can read end to end and say "this is current, this is stale, this one contradicts that one." If you have never done that pass, auditing what your AI actually remembers is where to start.

Write the position at the end of the sitting, not the start of the next one. You will never have more context about why an option was rejected than in the ten minutes after rejecting it.

Keep it in prose. A bullet saying "sharding — didn't work" is unusable in eight weeks. "Sharding by package let each worker own one section; it held about a month before the lag returned" is still usable in a year.

Store the cost, not just the outcome. The Anthropic post is memorable because it says how long each patch lasted. "It didn't work" invites a retry. "It bought twenty-nine days" ends the argument.

Name what would change your mind. A position with a stated trigger is self-invalidating in a useful way. Without one, you relitigate it whenever someone new arrives.

Do not store the transcript as the record. The transcript is evidence. The position is the record. Conflating them is how stores get large and useless at once.

Re-read the positions on a schedule. Anthropic's lag was invisible until it was measured. Yours is too.

The headline finding is that agentic coding put twenty-five times more load on Anthropic's CI in half a year, and that patching a single-writer service three times was a worse use of engineering time than rebuilding it once — a rebuild that "took three weeks for a single engineer."

The quieter finding is in the method. The argument for that rebuild was made over months, by an assistant that did not have to be reminded where the argument stood, against a human who kept choosing the patch. It won because it stayed coherent longer than the objections did.

That is not a feature of any particular tool. It is a property of keeping the conclusion separate from the conversation, dated, and readable by whatever you open next. Anthropic got it from a session that happened not to end. You can get it on purpose.