What Is Context Engineering — and Why It Needs a Memory Layer That Outlives Any Single Agent (2026)

Context engineering is the practice of deciding what information reaches a language model at each step of its work — instructions, tools, examples, retrieved documents, conversation history and memory — so it has what it needs and little else. Prompt engineering is one part of it. Memory is where the context comes from when the work spans more than one session.

Anthropic's engineering team gave one of the clearest definitions in a September 2025 post. "At Anthropic, we view context engineering as the natural progression of prompt engineering." Prompt engineering, they wrote, is about writing and organizing instructions. Context engineering "refers to the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference, including all the other information that may land there outside of the prompts."

The shift happened because agents work in loops. The same post explains that "An agent running in a loop generates more and more data that could be relevant for the next turn of inference, and this information must be cyclically refined." That makes the job continuous: "Context engineering is the art and science of curating what will go into the limited context window from that constantly evolving universe of possible information."

So the difference between the two is scope and timing. Prompt engineering is something you do once, before a conversation. Context engineering happens every time the agent decides what to look at next.

Anthropic lists the pieces an agent works with: "system instructions, tools, Model Context Protocol (MCP), external data, message history, etc." In practice, most agents draw on five kinds of context.

Instructions. The system prompt, plus files like CLAUDE.md or AGENTS.md that coding agents load at the start of a session.

Tools. The definitions of what the agent can do, which themselves take up space. Anthropic warns that "One of the most common failure modes we see is bloated tool sets that cover too much functionality or lead to ambiguous decision points about which tool to use."

Examples. A small set of canonical examples, rather than an exhaustive list of edge cases.

Retrieved information. Documents, search results and data pulled in when needed.

History and memory. What happened earlier in this session, and what was learned in previous ones.

Good context engineering is mostly about balance between these. Anthropic's guiding principle is that "good context engineering means finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome."

It is tempting to assume larger context windows make all of this unnecessary. Anthropic argues otherwise, pointing to what it calls context rot: "as the number of tokens in the context window increases, the model's ability to accurately recall information from that context decreases." Their conclusion is that "Context, therefore, must be treated as a finite resource with diminishing marginal returns."

The Claude Developer Platform's documentation on context editing makes the same point in product terms: "context is a finite resource with diminishing returns, and irrelevant content degrades model focus." And Anthropic's engineering post adds that "it's likely that for the foreseeable future, context windows of all sizes will be subject to context pollution and information relevance concerns."

Alibaba reached the same conclusion from the enterprise side. In the Yunqi keynote, the argument was that you cannot pour a company's data into an agent and expect it to work; overloading the window wastes the space the agent needs for the task, so the data has to be compressed into layers first.

This is also why shorter prompts alone do not solve cost or quality problems, a point explored in why shorter prompts are not enough. The question is not how little you send, but whether what you send is the right thing.

For long tasks, Anthropic describes three approaches: "compaction, structured note-taking, and multi-agent architectures."

Compaction. "Compaction is the practice of taking a conversation nearing the context window limit, summarizing its contents, and reinitiating a new context window with the summary." It keeps work going, with a known risk: "overly aggressive compaction can result in the loss of subtle but critical context whose importance only becomes apparent later."

Structured note-taking. "Structured note-taking, or agentic memory, is a technique where the agent regularly writes notes persisted to memory outside of the context window. These notes get pulled back into the context window at later times." A to-do list or a NOTES.md file are the simple versions.

Sub-agents. "Specialized sub-agents can handle focused tasks with clean context windows," returning only a condensed result to the lead agent.

There is also a retrieval pattern that runs through all three. Instead of loading everything up front, agents "maintain lightweight identifiers (file paths, stored queries, web links, etc.) and use these references to dynamically load data into context at runtime using tools." Claude Code is Anthropic's own example of a hybrid: "CLAUDE.md files are naively dropped into context up front, while primitives like glob and grep allow it to navigate its environment and retrieve files just-in-time."

Retrieval is not the same as memory, though the two are often confused. AI memory versus RAG covers the difference in detail.

In September 2026, context engineering moved from an engineering blog to a corporate strategy. Alibaba Cloud described its agentic cloud as "built around three core scenarios — model, harness and context," and launched an Agent Context service "that gives AI agents real-time context and long-term memory." Qianwen Office launched a companion product called Enterprise Context, which is what a context layer for AI agents looks like when the scope is a whole company.

Qianwen Office's keynote framed the enterprise version of the problem in three parts: agents that do not know the business, useful know-how that stays with individuals, and hesitation about data security. The proposed answer was to connect company data, understand it, and make it reusable, with compression at the center.

The most interesting line came from an interview afterwards. Shu Junliang, a Qianwen Office vice president, said context "is decoupled from the Agent you finally use," and that a company's context "must belong to this enterprise." He added that this infrastructure "has not been standardized like the database yet."

That is the enterprise form of a principle every context engineer eventually meets. Context is assembled per step, but the knowledge it is assembled from has an owner, and that owner is rarely the agent.

Context engineering decides what goes into the window. Memory is where much of it comes from once work spans more than one session. Some writers now call this second half memory engineering: context engineering operates at inference time, memory across time.

The catch is that most memory mechanisms today are tied to one tool or one place.

Claude Code's auto memory is a good example of a sensible design with a clear boundary: "Auto memory is machine-local," and "Files are not shared across machines or cloud environments." Anthropic's memory tool for developers goes further in putting you in charge: "The memory tool operates client-side: Claude requests file operations, and your application executes them. You control where and how the data is stored through your own infrastructure." In the same page, the boundary is stated plainly: "Memory lives entirely in your application." How Anthropic's managed stores approach this is covered in Claude agent memory stores explained.

Consumer tools draw their own lines too. In ChatGPT, for example, shared projects "do not have access to an individual member's context, custom instructions, or memories outside the project."

None of these are flaws. Each boundary protects something — privacy, scope, predictability. But together they mean that the knowledge an agent needs is scattered across as many places as the tools you use. A team running three agents has three partial memories, each shaped by one product's rules.

That is the gap the Alibaba interview pointed at. If context should be decoupled from the agent, then the memory it draws on should be too. Which kinds of memory matter is its own subject, laid out in the six types of AI memory; the practical point here is where they live.

You do not need a framework to start. Three steps cover most of the value.

Step 1: Separate standing context from working context

List what every session should start with: who the work is for, the conventions, the settled decisions, the definitions no model can guess. That is standing context. Then list what a task pulls in as it goes: files, search results, recent history. That is working context.

Standing context belongs in instructions and memory. Working context should be retrieved just in time. Mixing the two is the most common reason agents either miss key facts or drown in irrelevant ones.

Step 2: Compress standing context into layers

Write a short index first, then one compact entry per project, decision or process, then the source documents. Date entries so newer decisions visibly replace older ones. Keep examples canonical rather than exhaustive.

Resist the urge to save everything. Whether more memory helps an agent at all is a live question, examined in how much memory to give an AI agent.

Step 3: Store standing context where every agent can reach it

This is the step most setups skip. If your standing context lives in one tool's memory, every other agent starts without it. Keep it in a layer outside any single product, and test it by asking two different agents the same question about your work. If only one knows the answer, the context is living in that tool.

Step 3 describes a memory layer that belongs to you rather than to an agent. MemoryLake is built for that job: long-term memory for AI agents that sits outside any single tool, so the standing context you engineered once is available to every assistant you use.

You write the entries yourself, in your own words. Nothing is read out of, written to, or deleted from Claude Code's memory directory, your agents' own stores, or any vendor's store.

Step 1: Create an API key

Sign in and generate a key from the dashboard. The key belongs to your workspace in the memory layer, independent of any one agent or model.

Step 2: Upload your first memories

Start with the standing context from Step 1 and the layered entries from Step 2: definitions, settled decisions, conventions. One fact per entry, dated.

Step 3: Connect your AI & agents

Connect the assistants and agents you use. Each one can then draw on the same standing context, which is the decoupling test passing by design.

Treat context as a budget. Every token competes for the model's attention; include what earns its place.

Keep tools few and distinct. Overlapping tools create ambiguous choices for the agent.

Retrieve working context just in time. Load files and data when the task needs them, not up front.

Compact carefully. Summaries keep work going but can drop details whose importance appears later.

Write notes outside the window. Structured notes let an agent pick up where it left off.

Keep standing context independent of any one tool. Memory that lives inside one product only helps that product. The distinction between what an agent said and what it did, covered in episodic memory for AI agents, is worth keeping in mind when deciding what to save.

Context engineering is the natural successor to prompt engineering: the ongoing work of deciding what an agent sees at each step, within a window that is large but finite. Anthropic's principle sums it up — find "the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome."

The techniques are well understood: compaction, structured notes, sub-agents and just-in-time retrieval. What is less settled is where the knowledge behind all of it should live. Alibaba's phrase for it, "Context is All You Need," comes with the admission that context infrastructure is not yet standardized.

Until it is, the practical rule is simple. Engineer the context per step, but keep the standing knowledge it draws on in one place you own, reachable by every agent you use. For a broader introduction to that layer, see what AI memory is.