Multi-agent systems are increasingly used for complex enterprise tasks. Yet, they still communicate mostly the way humans do: by translating internal reasoning into human language and asking other agents to reconstruct meaning from it. This works, but it is also lossy, ambiguous, and expensive.
As agents become more specialized and workflows more distributed, the next bottleneck is no longer single-model capability, but how computational state itself is shared, preserved, and handed off. Latent memory points to a new paradigm in which agents exchange not only words but richer internal representations, potentially changing both the economics and the capabilities of multi-agent collaboration.
In this deep-dive article based on cutting-edge research, we break down how agents coordinate today, why latent memory is gaining traction, how it could help improve enterprise agent systems, and the key challenges that remain.
Note: This article assumes you are familiar with the fundamentals of AI agents. It is part of our article series on agentic memory that provides the bigger picture: see What is Agentic Memory? for a definition of agentic memory, and Agentic Memory Unpacked to understand different memory families (parametric, latent, and token-level memory) and types (factual, experiential, and working memory) on a high level.
How do agents coordinate?
Currently, agents coordinate through a mix of communication, shared memory, protocol layers, and environment-mediated state. In all of these cases, an agent’s internal reasoning is turned into an explicit representation that other agents can access, interpret, or build on. These representations vary in structure and how they are exchanged, in particular:
Forms of representation and mechanisms used to coordinate multi-agent systems
| Coordination Mechanism | Example | Strengths | Limitations | |
| Free-form text | Message passing | Chat turns | Flexible | Verbose, lossy |
| Structured data | Tool / API calls | JSON payload | Precise | Rigid |
| Shared memory | Shared storage | Vector store | Persistent | Stale/noisy |
| Protocol messages | Standardized A2A layer | JSON-RPC message | Interoperable | Still text-bound |
| Environment state | Indirect coordination | Ticket status | Decoupled | Weak shared context |
These coordination patterns are easy to inspect and orchestrate, but they also create a coordination tax. Towards a Science of Scaling Agent Systems shows how multi-agent variants experience degraded performance by 39 to 70% on sequential reasoning tasks. CooperBench arrives at a complementary diagnosis from collaborative coding: agents often perform worse together than alone (curse of coordination), and failures cluster around broken expectations, broken commitments, and communication breakdowns.
Taken together, these results suggest that coordination becomes increasingly difficult as agents must continuously maintain, exchange, and reconstruct context across multiple steps. This broader coordination problem has several dimensions, but one of the major bottlenecks appears at the communication layer.
At a deeper level, this points to a structural mismatch between how models process information and how they communicate. Models reason in high-dimensional hidden states (i.e., latent representations), but communication forces them to externalize that reasoning into tokens, exchange it, and reconstruct it again. In practice, this leads to repeated cycles of decode -> serialize -> re-tokenize -> re-encode across agents. This introduces overhead, increases latency, and creates opportunities for information loss and drift.
This naturally raises the question: can agents communicate in a more model-native way than repeatedly exchanging tokens?
Latent Memory: Communicating Beyond Words
To understand why latent memory is gaining traction, it helps to look at how LLMs represent and encode information:
Tokens are discrete symbols with limited information per unit, requiring reasoning to be serialized into language before it can be shared. Latent representations, by contrast, are high-dimensional continuous vectors that can encode richer information and be reused directly within the model’s computation — closer to how the model already “thinks” internally (kind of like telepathy).
This difference becomes critical at the communication interface. In token-based systems, reasoning must be externalized, transmitted, and reconstructed at every step, introducing both overhead and opportunities for information loss. Latent approaches, in contrast, allow agents to pass their internal state directly, avoiding this repeated translation and preserving more of the underlying signal. In effect, tokens act as a lower-bandwidth interface between agents, while latent memory enables more direct, model-native communication.
Three core trends regarding latent memory are becoming increasingly relevant for practical applications:
- Latent reasoning: Models can operate over continuous hidden states without emitting intermediate text (e.g., Coconut).
- Inter-agent latent communication: Agents begin to share working memory directly. LatentMAS passes KV cache across agents, while ThoughtComm explores structured latent exchange.
- KV-cache infrastructure: KV cache is increasingly treated as a reusable systems primitive. Advances in compression, quantization (e.g., TurboQuant), and cache reuse (e.g., LMCache) make latent memory more practical.
To get a sense of this in practice, @Akshat6 tested a setup that combines latent reasoning and inter-agent latent communication (inspired by LatentMAS) on the FinQA reasoning benchmark using Qwen3-8B on a NVIDIA A40 GPU:
FinQA-Experiment: Using Latent Communication in a MAS Increases Accuracy and Efficiency Over Text-Based Coordination
Compared to the same multi-agent setup communicating in text, this yielded ~5% higher accuracy and a 1.7× speedup, partly from avoiding repeated tokenization of intermediate reasoning. But these gains also come with significant constraints:
- Limited observability: Most reasoning happens in latent space, making systems harder to inspect, debug, and audit compared to text-based traces.
- Infrastructure constraints: Requires direct access to KV cache, which today effectively limits this to self-hosted setups and excludes standard API-based models.
- Memory and transfer overhead: KV caches can become large, creating pressure on VRAM and increasing bandwidth requirements when transferring state between agents.
Nevertheless, we believe latent memory is a promising approach to reducing costs and latency in agent swarms. It can be used wherever in-depth auditability and explainability of individual reasoning loops are a second-order concern.
Outlook and Open Questions
Despite the potential of latent memory, practical applications are still in their infancy, and it is not yet clear what the full stack around it will look like. We see a whole ecosystem that could evolve: latent APIs for models that expose hidden state, shared latent working memory between agents, protocols for computational state transfer, latent checkpoints, latent debuggers, KV-cache compression to reduce transfer overhead, and so forth.
Aside from these technical considerations, several foundational questions remain:
- How can latent exchanges be made observable and auditable without relying entirely on text traces?
- How stable is shared latent memory over longer workflows, especially across multiple handoffs and tool use?
- When does latent collaboration actually outperform text-based systems in practice, and how should that be evaluated?
- How well does latent collaboration generalize across different models, and as model size scales?
Bluesky
