Why ChatGPT Forgets Your Metric Definitions — and How to Fix It (2026)

A metric name is not a metric

"Gross margin" is four syllables that could mean six things. AtScale opens its write-up on the cost of semantic drift with three teams answering "What was our gross margin last quarter?" and returning 30%, 32%, and 31% — three defensible numbers from three definitions of what counts as cost of revenue.

Your assistant inherits this ambiguity without inheriting your resolution of it. When you say "churn," the model has a strong prior about what churn means in general and no information about the fact that your company measures it on revenue rather than accounts, monthly rather than annually, excluding downgrades. Absent your definition, it uses the prior. That's not a memory failure in the narrow sense — it's a memory failure that presents as a confident answer.

The definition lives in people, not in the data

This is the part that makes metric definitions harder than schema. A table has a shape you can inspect; the model can read column names and types. A definition is a decision — someone chose, at some point, that trials don't count — and that decision is recorded in a Slack thread, a dashboard's SQL, or someone's head.

The industry consensus on this is unusually blunt. As AtScale puts it: "AI systems don't just fail to reconcile conflicting definitions. They make the problem worse by confidently presenting inconsistent answers." The failure isn't the model inventing math. It's the model faithfully executing whichever of your organization's several definitions it happened to reconstruct.

Note how this differs from ChatGPT losing track of your data schema. Schema loss produces errors you notice — a column that doesn't exist, a join that fails. Definition loss produces numbers that reconcile to nothing, and you find out in the meeting.

Nothing carries a definition between sessions

Even when you get it right in one conversation, the mechanism that would carry it forward doesn't exist for this kind of content. Built-in memory holds a couple of pages of short facts about you; a nine-clause definition of qualified pipeline with its exclusions is not what it's sized for. Custom instructions can hold two or three definitions before crowding out everything else. Project files help, and they're scoped to that Project and to that tool.

So the definition gets re-supplied by hand, which means it gets re-supplied approximately. The version you type on Thursday is shorter than Monday's, because you're typing from memory and the exclusions are the boring part. Drift enters through your own summarizing.

Definitions change, and the old version doesn't get retracted

The compounding problem: you revise the definition in Q3, and every prior analysis was computed on the Q2 version. A chat-based workflow has no place to record that a definition superseded another one on a date. So you end up with numbers computed under different rules and nothing that tells you which — the analytics equivalent of undated documentation.

Pasting the definitions at the top of every chat. Effective, and the reason it's the most common approach. It also degrades: the pasted block gets shorter over weeks, and it's per-chat, so anyone else on the team is pasting their own version.

Putting definitions in custom instructions. Better for the two or three metrics you use daily, and genuinely worth doing. Then it runs out of room, and you've spent your instructions budget on definitions rather than on how you want the assistant to behave.

Using a Project with a definitions document attached. The best of the in-product options. Scoped to that Project, though, and if you also work in a coding agent or a BI tool, that document isn't visible from there.

Letting the assistant read the SQL. Clever and partially effective — the definition really is in the query. What it misses is intent and exceptions: SQL shows a WHERE status != 'internal' clause but not that the clause exists because of an audit finding, and it shows one dashboard's version without telling you it's the disputed one.

Building a semantic layer. This is the actual professional answer, and it should be said plainly: a semantic layer — dbt's Semantic Layer, AtScale, Cube, and the others in that category — exists precisely to define a metric once and have every tool compute it the same way. AtScale describes the goal as: "When someone asks, 'What's our revenue?' there's one answer because the definition is applied consistently everywhere." If your organization runs on shared metrics, this is the investment to make, and a memory layer does not substitute for it.

AtScale also reports that LLMs querying databases directly reach roughly 20% accuracy on business questions, rising to essentially full accuracy when a semantic layer supplies governed definitions, multidimensional business logic, and contextual relationships. That figure comes from a vendor with an interest in the conclusion and shouldn't be treated as an independent benchmark — but the direction is not controversial, and it's the strongest available argument for governed definitions.

The gap the semantic layer doesn't close is the one you're actually standing in: the assistant you paste a CSV into at 6pm, the ad-hoc question that never goes through the modeled layer, the definition that exists as a decision but hasn't been implemented as a metric yet, and every analysis done by someone who doesn't have BI access. That's where the definitions get lost, and it's most of the analysis that actually happens.

Keep the definitions outside the chat, in a store the assistant reads on every request. Not a note you paste from — a layer that supplies the definition as part of answering.

What makes this work isn't storage; it's that the definition arrives with its context. A stored definition can carry the exclusions, the reason for each exclusion, who approved it, when it last changed, and what the previous version was. That's the difference between "active = paid in 90 days" and a definition someone can actually be held to.

MemoryLake is a memory layer built for that — your definitions, decision records, and source documents in one store that ChatGPT reads through the API and MCP-capable tools like Claude and Codex read directly. So the same definition reaches the ad-hoc chat, the coding agent writing the query, and the analyst who joined last week.

Two boundaries, stated clearly because this is a domain where overselling causes real damage:

• A memory layer makes definitions available, consistent, and traceable. It does not enforce them. Enforcement is what a semantic layer does — computing every query through one definition. A memory layer supplies the definition to whoever is asking; it can't stop a model from being handed a raw table and doing its own arithmetic. If a number is going in a board deck or a filing, it should come through your governed layer, not out of a chat.
• Definitions need their metadata, or you've moved the problem. A two-column table of metric names and formulas is exactly the artifact that caused the drift. What has to travel is the grain, the time window, the exclusions and why, the owner, and the effective date.

Step 1: Create an API key

Generate a key and make your first request in about 30 seconds. Keep it in your environment or a secret manager rather than inline in a config or notebook that gets shared.

Step 2: Upload your first memories

Drop in the documents, images, and files where your definitions actually live — the metrics doc, the dashboard spec, the Slack thread where the exclusion was decided, the audit note that caused it. Upload the sources, not a tidied summary. The tidied summary is where "excluding internal accounts" quietly disappears.

Step 3: Connect your AI & agents

Give Claude, Codex, OpenClaw, and other AI agents access to memory via MCP or the API. ChatGPT has no MCP client, so retrieve the relevant definitions through the API and inject them into the prompt, a custom GPT, or the workflow that calls the model. For tools that speak MCP, add the server to that tool's configuration and they read the same store — which is the point: one definition, several consumers.

Thursday's number matches Monday's. That's the whole headline, and it's worth stating without embellishment: the same question computed against the same definition produces the same answer, because the definition didn't have to be retyped.

The second change is that disagreement becomes productive. Right now when two numbers disagree you spend an hour reconstructing what each one measured. When the definition is stored with its exclusions and effective date, you compare definitions instead of reverse-engineering arithmetic — and often discover the two numbers were both right and answered different questions.

The third is onboarding. A new analyst's first month is largely spent learning that trials don't count. When that's in a store their tools read, they start with the institutional version instead of rebuilding it from dashboards.

And revisions become legible. A definition with an effective date lets you say "this analysis used the pre-August definition" instead of hoping nobody asks. That single property — knowing which rules a number was computed under — is most of what makes analysis defensible.

Write the exclusions and the reason, not just the formula

"Active = paid invoice in trailing 90 days" is half a definition. The half that prevents drift is: excluding trials (they inflated the number before the 2025 pricing change), excluding internal accounts (audit finding, March), counted at account level not seat level. The reasons matter because they're what stops someone re-litigating the exclusion every quarter.

Date every definition and keep the old one

When a definition changes, add the new version with an effective date and keep the previous one marked as superseded. Deleting the old definition destroys your ability to interpret last quarter's numbers. This is the single highest-value habit in the whole practice and the most commonly skipped.

Name the owner

Every definition should say who decides it. Not for bureaucracy — because the question "is this still right?" needs an addressee. Definitions without owners drift silently; definitions with owners get corrected.

Keep the governed layer as the source of record

Use the memory layer so definitions are available everywhere, and keep your semantic layer or warehouse models as the thing that actually computes reported numbers. When the two disagree, the governed layer wins and the memory entry gets fixed. Reversing that hierarchy is how you end up with a well-documented wrong number.

ChatGPT forgets your metric definitions because a definition is a decision, not a property of your data, and nothing in a chat-based workflow carries decisions between sessions. What you get instead is the model's prior about what the metric name usually means, executed confidently — which is why the failure surfaces as a number that doesn't reconcile rather than an error you can see.

Keeping definitions in a store your tools read fixes the availability problem: the definition, its exclusions, its owner, and its effective date arrive with the question. Keep your semantic layer for enforcement and for anything that gets reported. Between the two, the version of "active customer" you explained on Monday is still the one in use on Thursday, and you can prove which version any given number used.