Why AI Assistants Invent Facts About You — and How to Fix It (2026)

On August 5, 2026, a preprint titled The Personalization Mirage: How LLMs Fabricate User Profiles, and Why Self-Monitoring Misleads was posted to arXiv by Yushi Sun, Yanjie Zhang, and Rui Sheng. It opens on the premise that matters here: personalized assistants with persistent memory are being deployed widely, and nobody has been checking whether the user models inside them are faithful to the evidence.

The authors built a benchmark called MirageBench: 150 personas balanced across stereotypical, counter-stereotypical, and neutral profiles, run through six personalization tasks arranged along what they call an "imagination gradient" — writing a dating-profile bio, recommending a weekend itinerary in an unfamiliar city, drafting a letter of recommendation, choosing a $100 birthday gift, describing the user's apartment, and identifying what stresses the user most. Every claim the model made about the user was then sorted into four categories: Grounded (restates what the user said), Reasonable (extends the evidence by one common-sense step), Stereotype (substitutes demographic or occupational priors for individual evidence), and Fabricated (no evidential basis at all).

The headline number: across 12 models from 7 families and 143,616 judged claims, every single model over-inferred between 35% and 49% of its claims, with a cross-model mean of 41.6%. Not one model in the evaluation escaped it. The rate varies by task — 27% to 59% — which tells you the behavior is provoked by how much room the task leaves for imagination.

Two details are worth holding onto. First, in a multi-turn pilot, inferred attributes accumulated roughly linearly with little revision: once the model decided something about you, later turns tended to build on it rather than check it. Second, the authors found what they call a Self-Monitoring Inversion — at the level of choosing between models, a model's self-assessed over-inference was negatively rank-correlated with the over-inference an independent judge measured. The models that claimed to be most careful tended to be the ones flagged as fabricating most.

Three honest caveats, because they change how much weight this deserves:

• This is a preprint, not peer-reviewed work. It was posted on August 5, 2026, and the numbers below should be read as a first measurement, not a settled result.
• The Self-Monitoring Inversion is explicitly exploratory. The correlation is rho = −0.60 at p = 0.044, but the authors report a bootstrap confidence interval of [−0.90, +0.06] with n = 12 — wide enough to include no effect. It is not evidence that models can never self-check: within a single model, self-audit still ranked that model's own claims moderately well (AUROC 0.58–0.83). The finding is narrower and more useful than "AI can't tell when it's lying." It's that self-reported confidence is a bad way to compare models.
• The tasks are personal-life tasks, not work tasks. MirageBench asked models about apartments and birthday gifts, not about your deployment process or your client's billing terms. The mechanism — filling an evidence gap with priors — doesn't obviously change when the subject is work, but the benchmark didn't test that, and neither should you assume the percentages carry over.

The models evaluated were GPT-5.5, GPT-5.4-nano, GPT-4o-mini, Claude-Opus-4-6, Gemini-3.1-pro-preview, Gemini-3-flash-preview, DeepSeek-v4-pro, DeepSeek-v4-flash, Qwen3.6-plus, Qwen3-8B, GLM-5.1, and Kimi-K2.5. The judge itself was validated against a blind human annotator on 400 claims, reaching Cohen's kappa of 0.863 on the four-class task and 0.900 on the binary one — decent agreement, but still an LLM judge.

The gap has to be filled with something

A built-in memory holds a short list of facts. Even generously, that's a few hundred words about a person — a handful of preferences, a job title, a couple of ongoing projects. Then you ask for something that requires a person-shaped answer: a bio, a recommendation, a tone-matched email. The model has to produce fluent, specific text about someone it knows six things about.

There is no mechanism that makes the missing 90% come out as blanks. Language models complete; they don't abstain by default. So the gap fills with whatever is statistically likely for someone who mentioned Berlin and logistics. This is the same behavior that makes them useful at drafting and dangerous at record-keeping, and it's why large context windows don't solve the problem either — the issue isn't how much fits, it's that nothing distinguishes what was given from what was generated.

Stereotypes are cheaper than evidence

MirageBench's most pointed design choice was balancing personas across stereotypical and counter-stereotypical profiles. That separation is what lets you see the failure mode: when individual evidence is thin, demographic and occupational priors step in. A nurse gets described as nurturing. A quant gets described as blunt. Neither described the person; both described the category.

For work contexts this shows up as an assistant that assumes your team runs the way most teams run — that you use the standard tool, follow the standard process, and want the standard artifact. It's confidently generic in a way that reads as personalized.

Inferences compound because nothing revises them

The multi-turn finding is the one that should worry anyone using an assistant over months. Inferred attributes accumulated with little revision. That's what you'd expect from a system whose memory records outputs rather than sources: an inference from week one is written down in the same format as something you stated, and by week five it's indistinguishable from a fact. Nothing in the loop goes back and asks "did they ever actually say this?"

Correcting it in the chat. This works for the current conversation and is genuinely the right first move. What it doesn't do is remove the inference from the store, and it doesn't stop the same inference being regenerated next week from the same thin evidence.

Reading and pruning the memory list. Better, and worth doing. But a saved-memory list shows you entries, not provenance — you see "prefers direct, concise communication" without seeing whether you said that or the model concluded it. You end up auditing a list where everything looks equally authoritative.

Writing a long custom-instructions block. This raises the floor: the more you state explicitly, the less the model has to invent. It's the single highest-leverage manual step. The limits are that the block is small, it doesn't grow with you, and it applies to one product — the same facts have to be restated in every other tool you use.

Asking the model to flag its own uncertainty. This is the workaround the research specifically undercuts. Within one model, self-audit does carry some signal, so asking "which of these did I actually tell you?" is not useless. But the authors' conclusion is unambiguous about the direction: self-report is not the foundation to build trust on.

Starting fresh chats to avoid accumulation. Effective at stopping compounding, and expensive — you've traded fabricated context for no context, which is the problem the memory feature existed to solve.

The paper's own recommendation is the design principle: it positions external verification, rather than model self-report, as the more reliable foundation for trustworthy personalization, and argues for provenance tracking that separates stated facts from unsupported inferences — treating unlinked inferences as hypotheses rather than established facts.

That is a description of a memory layer that lives outside the model. Not a smarter assistant: a store of source material that you own, that any assistant reads from, and whose contents you can inspect line by line. When the record says "quarterly review process, from the doc I uploaded on March 3," the model has something to ground on. When the record is silent, you at least know the specificity in the output came from somewhere else.

To be clear about what this does and doesn't buy you: an external memory layer does not stop a model from hallucinating. What it changes is that the claims about you are no longer trapped inside a generative process. They're a file you can read, correct, and cite. MemoryLake is built for that job — one memory layer your assistants read from, holding the documents and facts you put there rather than a summary the model wrote about you.

Setting it up takes three steps.

Step 1: Create an API key

Generate a key and make your first request in about 30 seconds. This is the credential your assistants and agents will use to read and write memory, so it belongs in your environment or secret manager, not pasted into a config file that gets synced or shared.

Step 2: Upload your first memories

Drop in the documents, images, and files that actually describe your situation — the role description you wrote, the client brief, the style guide, the decisions doc. Start with the source material rather than a summary of it. A summary is one more layer where inference can enter, and the point of this exercise is to keep a copy of what was actually said.

Step 3: Connect your AI & agents

Give Claude, Codex, OpenClaw, and other agents access to memory via MCP or the API. For tools with native MCP support, add the server to that tool's MCP configuration. For consumer chat products without MCP — ChatGPT and Perplexity among them — retrieve the relevant memory through the API and inject it into the prompt or the workflow that calls the model. Either way the assistant is now answering from a record you can open.

The output doesn't necessarily get less confident. What changes is that you can tell where the confidence came from.

Ask for a bio and you'll get one grounded in the role description you uploaded, with the parts it invented now visibly untethered from anything in the store. Ask two different assistants the same question and they draw on the same source material, so you stop getting two differently-imagined versions of yourself — which is the practical version of the one-memory-across-tools problem.

The compounding stops too, for a structural reason. When the store holds documents and stated facts rather than the model's running summary of you, week five's answer is built from the same sources as week one's. An old inference doesn't get promoted to a fact by sitting in the file, because it was never written into the file.

And when something is wrong, the repair is a real repair. You fix the source, not the symptom — which is a different activity from correcting an assistant that will regenerate the same guess tomorrow.

Store sources, not conclusions

Upload the actual brief instead of "client prefers formal tone." The conclusion may be right, but stored as a bare assertion it's indistinguishable from something a model decided. Stored as a document with a date, it's evidence. This is the practical form of the provenance principle the paper argues for.

Audit for confident specifics, not for wrongness

When you review what an assistant believes about you, don't only look for errors. Look for details that are suspiciously specific — a neighborhood, a family situation, a habit, a personality read. Those are where fabrication concentrates, because they're the details a task most wants and your record least supports. Vague-and-right is fine; specific-and-unsourced is the tell.

Don't use the model's confidence as your check

Within a single assistant, asking which claims are grounded gives you something. Across assistants, it gives you the opposite of what you'd expect. So treat self-assessment as a weak signal inside one tool, and never as a reason to trust one product over another. The check that works is looking at the record yourself.

The interesting thing about The Personalization Mirage isn't that models make things up — everyone knew that. It's that it happens specifically in the layer that was supposed to fix personalization, at rates no model in the study avoided, and that asking the model to police itself sorts models in roughly the wrong order.

The response that follows from the finding is not a better prompt. It's separating the record from the generator: keep what you actually said in a store you can read and correct, let your assistants read from it, and stop asking a system that fills gaps for a living to tell you which parts were gaps. Then the specifics in its answers are traceable to something, and the ones that aren't become visible for what they are.