The context window is not memory

Context windows have grown roughly three orders of magnitude in a few years, and each expansion arrives with the suggestion that retrieval is now unnecessary: just put everything in the prompt. It is an appealing idea and it is wrong in a specific, load-bearing way.

A context window is working memory. It is scoped to a single turn, it is paid for on every call, it degrades in the middle, and it is gone when the conversation ends. Calling it memory confuses a buffer with a system. What people actually want when they say memory is something else entirely: a body of knowledge that survives the session, accumulates from experience, and is retrieved selectively when relevant.

Working memory is what the model is holding. Context is what the system knows. Only one of them persists.

Three things that are all called context

Part of the confusion is that one word is doing three jobs.

• The window: the tokens in front of the model right now. Finite, expensive, ephemeral.
• The corpus: the documents, records, and connected systems the agent can reach. Large, durable, mostly irrelevant at any given moment.
• The accumulated record: what this agent has learned from doing the work. Small, high value, and the piece almost nobody builds.

Most stacks build the first two and skip the third. That is why agents that have run a task four hundred times approach the four hundred and first exactly as they approached the first. They have access to everything and have learned nothing.

Retrieval is a judgment problem, not a search problem

Once the corpus is bigger than the window, something has to choose what goes in. That choice is the actual product. Semantic similarity is a reasonable first pass and a poor final answer: the most similar document is often not the most useful one, and the most useful one is frequently the record of what happened last time someone tried this.

Good selection weighs recency, provenance, and outcome, not just closeness. It should be able to prefer a short note that says "this approach failed for this reason" over five pages of documentation that are topically adjacent and practically useless. That preference has to be learned from results, which means it depends on measurement, which means context and proof are the same problem wearing different clothes.

The part that compounds

The reason to care about the third category, the accumulated record, is that it is the only part of the system that gets better on its own. Models improve on someone else's schedule. Prompts improve when a person edits them. But a corpus of what worked, what failed, and under which conditions grows every time the system runs, and it belongs to whoever operated it.

This is also the honest answer to the question of what a company owns when it builds on someone else's model. Not the weights. Not the architecture. What it owns is the accumulated context: the record of its own work, structured well enough to be retrieved. That is the asset, and it is worth being deliberate about where it lives.

How we think about it

We treat context as infrastructure rather than prompt engineering. Memory is a stored, queryable thing with an owner and a scope, not a blob concatenated at the top of a request. Agents write to it deliberately, including recording their own evaluations of how a task went. Retrieval is a first-class operation you can inspect, which means when an agent does something surprising you can ask what it was looking at when it decided.

The window will keep growing, and that is genuinely useful. It just answers a different question than the one most people are asking.