Memory & State of AI Agents
Building an AI agent can be tricky. Keeping it in working condition for six months is very difficult.
LLMs don’t say anything about design. Every call starts from the beginning, without remembering what came before. Early agent developers worked around this by dumping all chat history into the content window and hoping for the best.
So far, we know that this method of treatment is growing rapidly. Latency spikes, and the model’s ability to do that use it the context degrades: important facts are buried, and if two versions of the truth are both in the window, there is no guarantee that it chooses the current one. Token prices have ballooned as well, though the immediate buffering has softened that blow to stable startups. The fix is not a larger context window; it treats memory and form as deliberate architectural decisions, not afterthoughts.
Before getting into the patterns, it’s worth being precise about what those two words mean, because it’s easy to mix them up.
The country summary. It’s everything the agent currently knows about the job right now: what step it is, what the last tool call returned, what variables it’s tracking. Think of it as a whiteboard. It is constantly updated as the task continues, and when the session ends it is gone, unless you deliberately persist, which is what Pattern 2 is all about.
Memory it is a machine that carries information across a boundary: the next opportunity, the next session, or a completely different agent that works later. Working memory is a very short horizon case (turn to turn); semantic periods and episodic memory span.
The two interact in a specific cycle. At the beginning of a task, the agent learns by heart to create its initial state: loading important facts, applicable rules of conduct, and records of past failures in similar tasks. During a task, the agent updates the state continuously as it runs. As work progresses and ends, you write certain pieces of that situation back into memory so that the next opportunity or session can benefit from what just happened. Memory transfers to the situation; the state returns to memory.
This distinction is important because the failure modes are different. A broken state means that the agent loses track of what it is doing during the task. Fractured memory means the agent can’t learn, can’t personalize, and treats every interaction as a blank slate. Both failures are common in production systems, and require separate solutions.
The five patterns below address both: Patterns 1 and 2 deal with the situation; 3 and 4 form a continuous memory layer for all sessions; and 5 suppresses both.
1. In-Content Performance Buffer (Short Term Use)
The concept
Working memory holds the ephemeral state of the current session: active information, recent conversational turns, and live tool outputs. Think of it as a temporary scratch pad for the agent, which gets hit when time runs out.
How It Works
Rather than letting the list of messages grow indefinitely, the active buffer acts as a sliding window. The agent writes the steps to think quickly on the scratchpad. As the buffer approaches the token limit, the compression process compresses the old conversion into a dense background compression, which preserves logical ends and reduces the output of the raw tool. When work is wrapped up, the buffer is flexible: whatever needs to be saved is moved out of long-term stores, and the rest is discarded.
It should be noted: that a mid-conversation summary may overwrite the beginning of the message, which clogs the KV cache and causes a latency spike on the next call. It’s a real tradeoff to design around.
When is it used?
All agents need this. It is the basis for managing multi-step thinking during a session.
2. Functional Testing (Fault Tolerance and Pause)
Once you have a strategy for managing what the agent keeps in mind during a session, the next question is what happens if that session is interrupted.
The concept
Long-running operations fail. The agent may time out, reach a rate limit, or pause waiting for someone to approve an action. Checkpointing saves the state of an agent’s workflow in the database so that it can resume where it stopped, without restarting a job that has already completed.
How It Works
A graph-based framework model flows as nodes and edges. After each step, the framework persists the state of the workflow, including variables, history, and current state, to a robust store such as PostgreSQL or SQLite. If the agent crashes, it reloads the last checkpoint and takes over from there.
One thing doctors often get burned by: resumes don’t give you semantics all at once. If a node has been used for a while before crashing (say it sent an email or wrote a query to a database), it may restart if it continues. Side-effecting nodes must be weak. Also remember that open file handles and client objects cannot be checked, which limits what you can safely configure.
When is it used?
It is important for human-in-the-loop systems, controlled workflows where actions need to be authorized, and any work horizon that may be susceptible to network failure.
3. Semantic Memory (Past Knowledge)
Checkpointing handles continuity between operations. But what about information that needs to survive all completely different times?
The concept
Semantic memory is what the agent is you know: facts, user preferences, and domain information that persist across independent sessions.
How It Works
Facts are extracted asynchronously and stored in an external database, usually a vector store filtered by metadata, sometimes paired with an information graph where relational maturity is critical. When a query arrives, the system finds the most relevant facts and injects them into the notification before the model sees it. Note that the rollout may cost an additional LLM call or more, depending on the structure, and is usually a single opportunity.
One design conflict around: if a user mentions “I use Postgres” in March and “we moved to Snowflake” in July, both facts end up in the store. Retrieval may appear either. The disallowance of truth, with the weight of the latter, supersession logic, or TTLs, is what actually solves the problem of the old supersession logic.
And it is worth shouting clearly: credentials and secrets are not semantic memory. Do not store API keys in a non-retrievable store. A rapid injection or overly eager retrieval can throw them out of the model response. Secrets are part of the secret manager, where the agent receives an authentication handle without seeing its value.
The opposite risk is also important: untrusted content (scraped page, user message, tool output) extracted from the semantic memory as the “truth” can drive the agent to the wrong place. Because there is no immediate equivalent of parameter placement, no hard separation between instructions and content, provenance marking does the work instead: track where the truth comes from and measure its impact accordingly.
When is it used?
Personal assistants, coders, or business agents need to remember the user’s preferred coding style, architectural guidelines, or database schema conventions at all times.
4. Episode Event Logs (Historical Reflection)
Semantic memory stores what the agent knows; episodic memory stores what the agent he did.
The concept
Episodic memory serves as a chronological ledger of an agent’s course of action: Goal, Plan, Tool Calls, Result.
How It Works
When the workflow ends, the background process enters this full path. Before the agent performs the same task, it queries this log. If it has previously failed a database query due to a syntax error, the episode memory displays that context so that the agent does not repeat the error.
One caveat: the failure traces returned are hints, not constraints. The model ignores them. There is also the danger of toxicity: if a single environmental failure passes as a strategic failure, you are persistently teaching the agent the wrong lesson. Go in with that in mind.
When is it used?
Automated coding agents, data engineering pipelines, and programming systems that need to learn from past mistakes without human intervention.
5. Multi-Scope Classification (Business Privacy)
If the memory persists, the question is who can see it. When your system serves more than one user, memory must be saved.
The concept
Memory is not a single shared bucket. A fact learned while helping User A must not be revealed to User B.
How It Works
All memory writes are marked with proprietary scopes: User ID, session_id, org_id. Retrieving filters strictly based on the active user’s auth token. Where possible, use this at the storage layer, using per-tenant namespaces or row-level security, rather than relying solely on application-layer query filters. A you forgot THERE clause fails to open; The storage layer partition fails to close.
This is a requirement for data privacy compliance, not the bottom line. The most difficult issue is deletion: if the user exercises his right to erasure, he needs to delete not only his raw data but also the embeddings, summaries, and extracted facts derived from it.
When is it used?
Any SaaS product, multi-tenant application, or enterprise deployment where data limits must be enforced.
Summary
One thing that these patterns do not cover is growth parameters. In more than six months (the draft of this article was opened), the semantic and episodic stores will accumulate almost duplicates, outdated entries, and noise. The quality of returns decreases as the stores fill up, and the cost scales with them. TTLs, aggregation functions, and pruning policies are not optional; they are part of the working memory scale.
The content window is not a database. When you divide the memory into different parts, short-term buffers of usage, experience episode logs, and semantic stores for facts, you get systems that really learn, stay within data boundaries, and stick to production.
