Why AI memory is cringe

There's a moment that most regular AI users have experienced by now. You're in the middle of a conversation with an AI assistant, working through a problem, and the assistant drops in a reference to something from a previous conversation. "Since you mentioned you're working on a healthcare startup..." or "Given your preference for Python..." The reference is technically accurate. It's also slightly jarring, like talking to someone who read your file before the meeting and is trying a little too hard to prove they did.

Every major AI lab is working on memory. ChatGPT has it. Claude has it. Gemini has it. The pitch is compelling: an AI that remembers you across conversations, builds up context over time, and gets more useful the longer you use it. In theory, this is the path to the kind of compounding context that transforms a generic assistant into a personalised thinking partner.

In practice, after more than a year of these features being live, the consensus among heavy users is underwhelming. The memories feel forced and out of context. The assistant surfaces them to demonstrate that it remembers rather than because the memory is useful in the current conversation. It's the AI equivalent of a waiter who memorises your name and uses it three times during the meal: technically impressive, socially uncomfortable, and not actually making the food better.

What current memory actually does

The way most AI memory systems work is roughly the same across providers. During a conversation, the system identifies facts that seem worth remembering and stores them as flat assertions. "User is a software engineer." "User prefers concise responses." "User is working on a project involving climate data." "User has two children."

These extracted facts sit in a memory store and get injected into future conversations as additional context. The model sees the memories alongside your new message and can reference them if they seem relevant.

The problems with this approach are structural rather than implementational.

Conversations are a terrible knowledge source. You mention something once in passing, a joke about your dog, a brief reference to a project you abandoned, and it becomes a permanent memory with the same weight as something you discussed in depth for an hour. The system can't distinguish between a throwaway comment and a core part of your identity, because in the flat text of a conversation they look identical. You spend three hours working through a complex research problem and the memory system extracts "User is interested in attention mechanisms," which is roughly as useful as summarising War and Peace as "book about Russia."

Extraction is lossy by design. Converting a rich, contextual conversation into a flat factual assertion strips away everything that made the information meaningful. The assertion "User prefers functional programming" doesn't capture whether this is a strong conviction or a mild preference, whether it applies to all contexts or just certain types of work, whether it's a current view or one you held three years ago, or whether you said it seriously or ironically. The memory stores the fact and discards the context, which means the fact is often less useful than having no memory at all, because it gives the system false confidence about something it doesn't actually understand.

Relevance detection is crude. The system has to decide, in real time, which of your stored memories are relevant to the current conversation. This is a surprisingly hard problem, because relevance depends on intent, context, and the specific question being asked, all of which are ambiguous. The result is memories being surfaced when they're tangentially related but not useful, which is the "since you mentioned your healthcare startup" phenomenon. The system has a hammer and everything looks like a nail.

The memories feel like surveillance. There's an uncanny valley effect when an AI references something you said weeks ago in a different context. It triggers the same discomfort as when a company uses your browsing history to target an ad: the data is accurate, the use is technically legitimate, and the feeling is still that you're being watched. The more personal the memory, the stronger the discomfort, which creates a perverse incentive where the memories that would be most useful (deeply personal context) are the ones that feel most invasive when surfaced.

The wrong abstraction

The deeper issue is that auto-generated memories from conversations are the wrong abstraction for what people actually need from AI memory.

What people want is an AI that understands their work, their thinking, their accumulated knowledge, and their current context well enough to be useful without being told everything from scratch each time. What they're getting is an AI that remembers scattered facts extracted from chat and deploys them with the subtlety of a first-year psychology student practising active listening.

The gap between these two things is enormous, and it's not going to be closed by better extraction algorithms or smarter relevance detection. The problem is the source material. Conversations are a narrow, ephemeral, context-dependent channel. They capture a tiny fraction of what you know and care about, filtered through the specific purpose of whatever you were discussing at the time. Building a model of someone from their chat history is like trying to understand a city by reading its text messages.

The information that would actually make an AI useful as a long-term thinking partner is the information you've deliberately accumulated over months and years: the articles you've read and annotated, the notes you've written in your own words, the research you've collected, the documents you've saved, the voice memos you've recorded when an idea struck you. This is curated memory rather than extracted memory, and the difference in quality is the difference between reading someone's published work and reading their text messages.

Curated memory vs extracted memory

The distinction maps onto something the knowledge management field has understood for decades. There are different types of knowledge, and they require different approaches to capture and retrieval.

Extracted memories from conversations are mostly shallow factual assertions, the kind of explicit, declarative knowledge that's easy to state but low in value. "User works at a fintech company." "User is interested in education." These are profile facts, not working knowledge.

A curated knowledge library contains something richer: the accumulated conceptual understanding, the research trail, the documented reasoning, the connections between ideas that you've drawn over time. When an AI assistant draws on this kind of memory, the result feels natural rather than performative, because it's referencing your actual work rather than performing remembrance of things you mentioned in passing.

The practical difference: ask an AI with auto-generated memories to help you think through a research problem, and it might note that you're interested in the topic (useful for about three seconds). Ask an AI that has access to your research library, your annotated papers, your literature notes, and your research log to help with the same problem, and it can engage with your specific reading, your specific thinking, and the specific gaps in your understanding. One of these is a memory trick. The other is a thinking partnership.

The ownership dimension

There's also a sovereignty question worth noting. Auto-generated memories are controlled entirely by the provider. They decide what to extract, how to store it, what to surface, and what to forget. You can usually view and delete individual memories, but you can't export them, you can't use them with a different provider, and you have limited visibility into how they influence the model's behaviour.

A knowledge library that you own, stored in infrastructure you control, accessible through open protocols to any AI agent, inverts this. The memory is yours. You built it deliberately through months of capturing, annotating, and connecting. You can switch AI providers without losing any of it. And the boundaries of what the AI can see are set by you rather than by the provider's extraction algorithm.

The auto-generated memory model says: use our product and we'll gradually learn about you. The curated library model says: build your knowledge system and any AI can draw on it, on your terms. The first creates lock-in (the longer you use one provider, the more it knows, the harder it is to leave). The second creates portability (the memory belongs to you regardless of which provider you're using today).

What good AI memory will look like

The labs will improve their memory implementations. The extraction will get better, the relevance detection will get sharper, and the uncanny valley effect will diminish as the systems learn when to surface memories and when to stay quiet. These are engineering problems and they'll be solved incrementally.

But the fundamental limitation of building memory from conversation extracts will remain, because conversations will always be a narrow window onto what a person knows and cares about. The richest, most useful AI memory will come from what you've deliberately built rather than from what a system has passively observed. Your notes, your annotations, your saved sources, your documented thinking, your research trail: this is the memory that makes AI interactions feel like working with someone who knows your work, rather than someone who's read your file.

The person who starts building that library today has a form of AI memory that's already better than anything the labs have shipped, and it improves every day they use it, independently of which model they're running, which provider they're using, or how the next generation of auto-generated memory features turns out.

Frequently asked questions

Should I turn off AI memory features? Not necessarily. They're mildly useful for basic personalisation (response format, language preferences) and they'll improve over time. The point is that they're a weak form of memory compared to a curated knowledge library, and investing effort in building the library produces far more value than passively hoping the auto-generated memories become useful.

What's the difference between AI memory and a second brain? AI memory is extracted by the system from your conversations, controlled by the provider, and limited to flat factual assertions. A second brain is built by you from your reading, research, and thinking, controlled by you, and rich with context, connection, and nuance. The second brain is a better form of AI memory because it contains the kind of knowledge that actually makes AI interactions more useful.

Will the labs eventually solve the memory problem? They'll improve the extraction and relevance, and the experience will get less awkward. But the fundamental limitation, that conversations are a narrow source of knowledge, won't change. The richest AI memory will always come from deliberately curated knowledge rather than passively observed chat. The labs are trying to build memory from the exhaust. You can build it from the engine.

How does this connect to the context compounding argument? Directly. The context that compounds is the context you've built deliberately through months of capturing, annotating, and connecting knowledge. Auto-generated memories compound slowly if at all, because each extracted fact is thin and disconnected. A curated library compounds richly, because each new piece of knowledge connects to more existing pieces and creates more retrieval paths for the AI to draw on.

Related reading: The memory is the moat, Where does your knowledge live?, The data wallet, What is knowledge management, The different types of knowledge. Related guides: Building a Second Brain, Zettelkasten, Research workflow.