Episodic Memory for AI Agents: Why Your Agent Knows What You Said, Not What You Did (2026)

Memory products are built on the transcript

Look at where memory comes from in the tools you use. ChatGPT's memory is derived from your chats. Claude's memory is derived from your conversations. Codex generates memories from prior chats — its own documentation describes what it keeps as "summaries, durable entries, recent inputs, and supporting evidence from prior chats." Coding agents read files you wrote.

Every one of those is downstream of language you produced for the model. That's a real and useful kind of memory, and it's roughly what psychologists would call semantic or declarative: facts, preferences, standing rules. What it isn't is a record of events — the sequence of things you actually did, in what order, in which application, for how long.

The doing half has no natural capture point

When you type a preference into a chat, capture is free: the text already exists. When you work — clicking through an admin panel, reconciling two spreadsheets, checking a dashboard before approving a payout — nothing is written down anywhere the agent can reach.

That's why the missing half stays missing. It isn't that vendors don't want it; it's that collecting it means capturing your screen, which is an entirely different product with entirely different consent, storage, and security questions. Note that this is also why the gap is stable: it won't close as a side effect of better models.

When a model summarizes your activity, the evidence goes with it

The obvious way to turn screen capture into memory is to have a model watch and write summaries. The August 6 preprint — Activity Frames: Deterministic Screen-Activity Compilation for Agent Memory and Replay (Nossa Iyamu, arXiv 2608.05784) — argues for the opposite, and measures the difference.

Its pipeline uses no model at all. It segments a local capture stream into "typed activity frames, bounded episodes carrying application, site, timing, input volume, and evidence pointers back to the raw rows, with no model in the loop, so the output is byte-identical, cacheable, and mechanically auditable." On the author's corpus of 128,756 frames across 51 active days, compiling one day of raw capture produced a prompt-ready context block 86x smaller in 68 ms, and an agent reading that block answered questions about the day at 98.4% accuracy (Wilson 95% CI 91.7–99.7%) against an independent oracle — "versus 66-80% for an LLM summary of the same capture."

Two caveats that matter more than the headline. This is a preprint, not peer-reviewed work, and the corpus is one professional's own single-user data — the numbers describe that setup, not your team. And the 66–80% figure is the paper's own comparison against a language-model summary of the same capture stream; it is not a measurement of any shipping product.

What survives those caveats is the structural point, and it's the same one that makes document handling work: a summary is a lossy artifact with no pointer back to what produced it. Frames keep evidence pointers. You can check them.

Where activity is captured, it lands in exactly one tool

OpenAI already ships this category. Chronicle, documented as an opt-in research preview for ChatGPT Pro subscribers on macOS, builds Codex memories from recent screen activity. Its documentation is refreshingly specific: screen captures live locally under a temp directory and are deleted after six hours, while the memories it generates are "unencrypted markdown files that you can read and modify if needed," stored under the Codex home directory. To produce them, "screenshot frames, OCR text extracted from screenshots, timing information, and local file paths" are processed on OpenAI's servers. OpenAI also states plainly that the background agents "consume rate limits quickly" and that using Chronicle "increases risk to prompt injection attacks from screen content."

That is a vendor being honest about a hard feature, and the design choices are defensible. But note where the output ends up: in Codex's local memory store, on one machine, readable by Codex. The open-source project accompanying the paper makes the mirror-image choice — capture to a local SQLite database, compile locally, serve over MCP — and its output likewise lives on the machine that captured it.

So even in the best case today, the record of what you did is legible to one client on one laptop. Meanwhile the assistant you'll actually ask about Tuesday morning might be a different one.

Re-explaining the routine. The default. You describe the eleven steps, the agent does a decent job, and next week you describe them again. The paper's whole motivation is the cost of this: agents "pay full frontier inference to re-derive routines their user has already performed."

Pointing the agent at artifacts. Git log, ticket history, browser history, exported reports. Genuinely useful, and the closest cheap approximation — but artifacts record outcomes, not sequences. A commit tells you what changed, not the twenty minutes of checking that preceded it.

General-purpose screen recorders. Rewind-style tools and the newer local-first recorders produce searchable history for you. Some now expose it to agents. Worth trying, and the question to ask each one is the same: does the output carry pointers back to the raw capture, or is it a model's impression of your afternoon?

Chronicle. If you're a Pro subscriber on a Mac and your work is mostly in Codex, this is the shortest path to the doing half, and you should read its documentation about rate limits and prompt injection before enabling it. Its scope is Codex.

Writing your own runbook after the fact. The highest-fidelity option and the one nobody keeps up. Which is the honest reason automatic capture keeps being attempted.

Notice the shape of the whole list: either the record isn't collected, or it's collected into a place only one tool can read.

Two things are true and they're often confused. Episodic capture is a capture problem — it needs something watching, with your consent, and no memory layer changes that. What happens to the record afterward is a storage and access problem, and that's where most of the value leaks today.

So split the work honestly. If you want the doing half, pick a capture tool and read its privacy documentation properly. Then make sure its compiled output — plus the declarative half you already have, the specs and decisions and constraints — lives somewhere every assistant you use can read, rather than in one client's local directory or in a chat transcript.

MemoryLake is a memory layer for that second job: one store your documents, decisions, and compiled records live in, readable from MCP-capable tools like Claude and Codex directly and from ChatGPT through the API. To be exact about what it isn't — it doesn't watch your screen, and it isn't a substitute for a capture tool if episodic history is what you need.

Step 1: Create an API key

Generate a key and make your first request in about 30 seconds. Keep it in your environment or a secret manager rather than pasting it into a chat window.

Step 2: Upload your first memories

Drop in the documents, images, and files you currently re-explain: the runbook, the spec, the constraints, the decision records, the exported reports. If you run a capture tool that produces compiled markdown or structured output, that output belongs here too — and upload it as produced rather than as a summary of a summary.

Step 3: Connect your AI & agents

Give Claude, Codex, OpenClaw, and other AI agents access to memory via MCP or the API. ChatGPT has no MCP client, so retrieve what you need through the API and inject it into the prompt or the workflow that calls the model. The point of this step is that the same record answers to more than one tool — which is the part local capture stores can't do on their own.

The first difference is that the expensive half stops being re-derived. Whether the record of a routine came from a capture tool or from ten minutes of you writing it down once, the agent reading it starts from what happened instead of reconstructing it. The paper's demonstration is the extreme version of this: a compiled routine that "replays deterministically with the model out of the loop," at zero model tokens.

The second is that provenance survives. This is the same argument that shows up whenever an AI system tells you something confidently: the useful record distinguishes what was measured from what was inferred. That distinction is exactly what a summary destroys, and it's why AI systems that invent plausible facts about you are so hard to catch — the inference and the observation arrive in the same sentence, with the same confidence.

The third is that the record outlives the tool that made it. Capture stores are machine-local by design. A shared layer means the notes you accumulated this quarter are still there after you switch editors, change laptops, or add a second agent — and that one memory reachable from several tools is a different property than any single client's memory feature.

And it composes with what you already run. Codex memories, Claude Code's project memory, and ChatGPT's saved memory keep doing their local jobs. What they can't do is be read by each other, which is the whole reason multi-agent setups lose context at every handoff.

Decide what you're capturing before you turn anything on

Screen capture is not a neutral default. Read the storage and retention specifics, check what leaves the machine, and consider whether client work, credentials, or regulated data appear on your screen during the day. Chronicle's own documentation warning about prompt injection from screen content is the kind of sentence worth reading twice — a page you visit can carry instructions your agent then follows.

Prefer records that point back to something

Whether it's frames with evidence pointers, a runbook citing file paths, or a decision record citing the incident that forced it, a memory you can verify is worth several you can't. Summaries feel efficient right up to the moment you need to know whether something actually happened.

Keep the two halves separate but adjacent

What you decided and what you did are different records and shouldn't be merged into one narrative. Store both, label which is which, and let retrieval pull whichever the question needs. Compressing them together is how "I checked the dashboard first" turns into "prefers checking dashboards."

Don't let the routine live only in the capture store

If a routine matters enough to automate, write the short version down as a document too. Capture stores are machine-local, opt-in, and sometimes research previews. A five-line runbook in a shared store survives all three of those facts, and it's also what a new teammate can read.

Treat "the agent already knows this" as a claim to check

The failure mode here is quiet. An agent with no episodic record doesn't announce it; it produces a plausible version of your workflow. Before delegating something procedural, ask it to state the steps back, and watch specifically for the steps you never wrote down anywhere.

Episodic memory is the half of agent memory that nobody has finished building. Today's features are built on transcripts, so they hold what you said and not what you did — the paper's own framing, and the reason it exists. The two live attempts at the doing half both work by capturing your screen, and both leave the result in one tool on one machine: OpenAI's Chronicle in Codex's local memory store, the open-source compiler in a local database served over MCP.

The practical takeaway isn't to wait. Enable capture if the privacy tradeoff works for you, prefer records that keep evidence pointers over model summaries that don't, and make sure everything you accumulate — the compiled activity and the specs and decisions that were never on screen at all — lands somewhere more than one assistant can read. The record is worth more than the tool that produced it.