Why is that? From a 10,000-foot perspective, we can quickly state that agentic memory is about agents "remembering" past interactions to perform better in the future. Without memory, AI systems suffer amnesia, starting blank with every interaction.
But when exactly do agentic systems need memory? What are the business use (and do not use) cases? How does agentic memory differ from other types of memory, and how can we define it? This is what this article is about.
When to Use Agentic Memory?
To better reason about when to use agentic memory, I've found it helpful to start from first principles. My initial approach is to examine the types of information that should and should not be stored in agentic memory. The following considerations act as a thought experiment: a strong case for agentic memory begins to emerge when the information to be stored within agentic memory meets the following criteria and avoids the listed counterexamples:
First Principles Perspective: What to store in agentic memory and what not. The information is...
| Counterexample |
| Valuable across multiple interactions, carrying significance beyond a single session. | A one-off query for a customer's order status. This fact should be retrieved via a live API call. |
| Novel, meaning it is not already contained within or contradicted by the base LLM's pre-trained knowledge. | A request for the definition of "SWOT analysis." This is standard business knowledge already known by the model, making memory storage redundant. |
| Emergent and unspecified, meaning it is unknown at design time and is only discovered through interaction. | A company's official list of approved software vendors. This is pre-defined information, best handled by a dedicated retrieval (RAG) system or prompting. |
| Unavailable from external systems like databases or APIs that could be queried directly. | A customer's billing address. This structured, sensitive data must live in and be queried from a secure CRM, not duplicated in the agent's memory. |
| Constructed by the agent from its inputs, feedback, and self-reflection. | None, this is a prerequisite. If this cannot be achieved, memory cannot be built. |
Conversely, if any of these conditions are not met, we should consider whether a more conventional tool, such as a direct database query or a standard RAG system, would be a better solution.
In contrast, the following scenarios are prime examples of situations in which agentic memory is not just helpful but often necessary for an agent to perform its tasks effectively.
Use Cases for Agentic Memory | Use Case | Description | Example |
| Personalization | Tailor the agent's workflow, communication style, or outputs to the needs, preferences, and history of a specific user. | A customer service agent remembers a user's technical expertise level and provides advanced troubleshooting steps instead of basic ones. |
| Agile Contextualization | Provide the agent with narrow-scope, limited-lifespan information to be used across multiple interactions within a specific project, case, or workflow. | An agent helping a team with a two-week software remembers conversations, decisions, and code snippets related only to that sprint. |
| Extended Working Memory | Provide an agent with relevant facts from a long history of interactions and tool use, overcoming the fixed-size limitations of an LLM's context window. | During a long-running legal case analysis, the agent pulls key precedents from conversations held months ago, even if they exceed the context window. |
| Multi-Agent Orchestration | Serve as a shared knowledge exchange layer, or "blackboard," for multiple agents to coordinate their work on a mutual goal. | In a research team, one agent finds a key statistic and saves it to shared memory, allowing another agent to incorporate it into the final report immediately. |
| Curation of Latent Knowledge | Extract and make persistent the undocumented "tribal knowledge" of experts on business procedures or semantics from their interactions with the agent. | By working with a senior engineer through a system diagnostics case, a support agent learns an undocumented process and saves it for future use. |
| Agentic Self-Improvement | Allow an agent to reflect on its own performance and environmental feedback to refine its approach for future tasks systematically. | After a user marks a summary as "unhelpful," the agent analyzes the feedback and adjusts its summarization strategy to be more detailed next time. |
These use cases — from personalization to self-improvement — suggest that agentic memory behaves more like human memory than computer databases. Let’s explore that distinction to arrive at a clear definition.
Trying to Define "Agentic Memory"
To delineate Agentic Memory further, I find it useful to examine its relationship to the concepts of Computer Memory and Human Memory. Here is a glimpse at my current point of view on these terms and their characteristics:
Computer Memory is often described and discussed through the technical lens of read-write speeds, size, volatility, scalability, and cost. These are also the dimensions used to cluster and characterize types of computer memory (e.g., cache vs. RAM=memory vs. secondary storage, vs. cloud storage). In computer science, the term is driven by "how" to store things. "What" to do with the memory is often discussed within the context of files, file systems, and databases. In computer science, memory is closely tied to the underlying substrate.
Based on my intuitive understanding (not a neuroscientist!) of Human Memory, I tend to discuss it from the perspective of "what" or through the lens of its lifecycle phases and big-picture purpose ("why"). Terms like certainty, connection, validity (e.g., time, scope), importance, attention, meaning, or interpretation come to mind. The lines between different types of memory, such as long-term and short-term, are less distinct than in computer memory. I also strongly associate human memory with learning mechanisms like self-reflection. E.g., humans start to actively search for confirmation of memories after a certain amount of time, and they constantly reconnect and restructure memories. Thus, my understanding of human memory also assumes a natural connection to learning.
I know of no singular, widely accepted definition for Agentic Memory. Defining it is a matter of perspective and emphasis. From my point of view, I find it helpful to employ a working definition closer to my perception of human memory, focusing on "why", "what", and lifecycle aspects instead of the memory substrate:
Agentic Memory is a system designed to manage agent-specific, emergent information that holds persistent value. It provides this service to an AI agent by managing the complete memory lifecycle:
- Formation – Capturing novel insights from interactions and experiences.
- Evolution – Refining, synthesizing, and connecting stored knowledge through reflection.
- Retrieval – Intelligently surfacing relevant information to enhance the agent's context and decision-making for future tasks.
Memory evolution may not just involve memory updates but also strategic consolidation, optimization, rephrasing, interpretation, and hierarchization of memories aimed at improving agentic performance. In contrast to memory formation and retrieval, I consider it optional.
But even without memory evolution, I do not see agentic memory without learning! Instead, the very essence of engineering agentic memory systems is to define the principles and guidelines that govern the formation (and evolution) of memories and to provide a suitable substrate for storing and retrieving the anticipated information.
What's Next?
In my next blog post, I will share a summary of core agentic memory concepts, including a deep dive into memory lifecycle phases, an overview of different memory types, and memory architectures. Stay tuned and feel free to share your thoughts on this point of view.
# CTO Research and Innovation Munich
Bluesky
