TRACE Open Source: Turning Agent Memory into a Topic Tree, Taking Down Mem0

A Memory System That Doesn’t Use RAG — Yet Still Crushes RAG-Based Approaches

In early July, an open-source project called TRACE was released by its developer on r/MachineLearning, along with a benchmark result that immediately caught attention: on the EventQA precise retrieval task from MemoryAgentBench (ICLR 2026), TRACE achieved an impressive 82.5% F1 using gpt-oss-20B, and pushed that further to 83.8% with the 120B model.

For comparison, the official numbers reported in the benchmark paper for the same task were:

• Mem0 (GPT-4o-mini): 37.5%
• MemGPT / Letta (GPT-4o-mini): 26.2%

An approach running on local open-source weights outperformed two dedicated Agent memory systems running on proprietary commercial models by roughly 45 percentage points. At first glance, the number looks almost absurd. But anyone who has recently struggled with long-context Agent memory systems will recognize that it points to a question the field has avoided seriously answering for a long time: how should memory actually be organized?

Every AI Memory System Eventually Ends Up Looking the Same

First, let’s state something that has already become an unspoken consensus among developers: nearly all AI memory systems on the market today follow essentially the same template.

Chat histories, user preferences, uploaded documents, intermediate workflow outputs — everything gets chunked, embedded into a vector database, then retrieved via similarity search when a new query arrives, and finally stitched back into the prompt. Mem0, MemGPT, Letta, Zep, including many in-house solutions, all revolve around this same architecture.

The problem is not that this approach doesn’t work. The issue is that it breaks down in scenarios requiring reasoning across long time spans and multiple topics. The reasons are straightforward:

1. Chunks are structurally meaningless. Once content is split into 500-token segments, semantic boundaries disappear. It’s common for a single event to be cut in half.
2. Vector retrieval finds “most similar,” not “most relevant.” Ask something like “Did I say last time that I wanted to visit Kyoto or Osaka?” and the vector store retrieves every chunk mentioning Japan travel, including irrelevant casual conversations from three months ago.
3. There’s no timeline or causality. RAG assumes knowledge is static, but conversational memory is fundamentally a dynamic timeline: statement B may override statement A.

MemoryAgentBench was able to push Mem0 down to 37.5% precisely because it specifically tests this ability: accurately locating a particular event inside a long conversation history. That is not what RAG is good at.

TRACE’s Approach: Organizing Conversations Into a Growing Topic Tree

The core idea behind TRACE can be summarized in one sentence: instead of fragmenting conversations into chunks, it organizes them into a topic tree, where every node maintains its own summary.

More specifically, TRACE does the following:

• Topic branching. When a new message arrives, the agent decides whether it belongs to an existing topic branch or requires creating a new one. This process is LLM-driven, not clustering-based.
• Hierarchical summarization. Each branch maintains a dynamically updated summary, while parent node summaries abstract over their child summaries. Think of it as an automatically generated, continuously maintained semantic table of contents.
• Tree-based retrieval. Queries start at the root summary, which guides traversal down the appropriate branch layer by layer until reaching leaf nodes containing detailed information. This is a top-down semantic navigation process, not global vector matching.

Developers familiar with B-Trees or file systems will immediately recognize what this really is: an indexing structure for conversational memory. RAG is full-text search; TRACE is a directory hierarchy. One relies on brute force, the other on divide-and-conquer.

Installation is as simple as:

pip install trace-memory

The author also released complete JSON logs from two runs, including every LLM input and output at each step — far more transparent than many papers that only publish final metrics.

How Trustworthy Is the Benchmark?

A caveat is necessary here: this comparison is not entirely apples-to-apples.

The author explicitly acknowledged this in the original post. TRACE used locally hosted gpt-oss-20B/120B models, while the Mem0 and MemGPT numbers came from their paper’s official GPT-4o-mini results. Strictly speaking, comparing different backend models is not fully fair.

The author attempted to run Mem0 on gpt-oss-20B for a same-backbone comparison, but encountered issues in Mem0’s fact-extraction stage. That component requires strict JSON outputs, and gpt-oss occasionally fails to maintain the required formatting. This issue is not unique to gpt-oss — the same problem can be reproduced with Gemini and Mistral models as well. Letta was skipped entirely because it requires spinning up a full server.

So strictly speaking, the benchmark proves the following:

• TRACE reaches 82.5% on gpt-oss, which is a solid absolute result.
• Mem0/MemGPT only achieve around 30% on the same benchmark even when using stronger commercial models, according to their published paper results.
• Part of the performance gap comes from methodological differences, and part may come from differences in model capability. But with a 45-point gap, methodology is clearly the dominant factor.

In other words, even if you discount the result heavily, it still demonstrates that hierarchical tree structures hold a substantial advantage over flat RAG architectures for event retrieval tasks.

Why This Matters for Agent Developers

Over the past year, teams building Agents — especially long-term companion systems, customer service bots, and personal assistants that need to “remember users” — have all struggled with memory management. Common symptoms include:

• User information continuously accumulates, causing token costs to rise linearly
• Vector retrieval accuracy degrades as conversations grow longer, becoming nearly random after 50 turns
• Attempts to implement features like “summarize what we discussed last week” fail because chunks are unordered

TRACE-like approaches offer an alternative direction: stop treating conversations as a retrievable knowledge base, and instead treat them as a semantic tree whose structure must be maintained. This is especially valuable for:

1. Multi-turn conversational Agents: preserving memory continuity across days and sessions
2. Multi-task Agents: a single agent handling coding, scheduling, writing, etc., where topics naturally branch
3. Auditability and interpretability: once visualized as a tree, you can directly see what the agent “remembers,” whereas embedding space is inherently opaque

Problems That Still Haven’t Been Solved

To be fair, TRACE still has several significant challenges:

• Topic splitting accuracy depends heavily on LLM judgment. With weaker models, the tree structure degrades and summary quality suffers. There’s a reason the author started with 20B models.
• Write latency is high. Every incoming message requires an LLM pass to determine branch placement, making it an order of magnitude slower than simply inserting vectors into a database. Suitable for offline batching, not per-token updates.
• Wrong retrieval paths are hard to recover from. If top-down navigation chooses the wrong branch midway, the answer becomes unreachable. RAG’s brute-force retrieval is at least more robust in terms of recall.
• What happens when the tree grows too deep? The author did not discuss pruning or archival strategies in detail, and the tree will inevitably expand over long-term operation.

These are unavoidable engineering challenges, but the important point is that the architectural direction itself appears correct — and that matters more than the benchmark numbers.

A Quick Note on Backend Models

TRACE uses OpenAI’s recently released open-source gpt-oss series, available in 20B and 120B sizes. These models perform well on structured reasoning and tool usage, while remaining relatively practical for local deployment. A quantized 20B version can fit within 24GB VRAM. For developers wanting to reproduce TRACE without burning through GPT-4o token costs, it’s a reasonable option.

For those who don’t want to run models locally, the gpt-oss series is also available on OpenAI Hub alongside GPT-4o, Claude, Gemini, and DeepSeek. Everything can be accessed through a single API key using OpenAI-compatible formats, making A/B testing convenient — for example, running the same TRACE logic across different backends to observe how tree quality changes. Aggregation platforms are genuinely convenient for this kind of horizontal experimentation.

A Directional Shift

From 2025 into 2026, the two most persistent keywords in the Agent ecosystem have been “context” and “memory.” The former has been addressed by model providers through ever-larger context windows (Gemini 2M, Claude 1M), while the latter requires engineering teams to build entirely new infrastructure themselves.

TRACE is not the first project exploring hierarchical memory structures, but it clearly explains the method, open-sources the implementation, and demonstrates benchmark results. Memory systems are evolving from “better RAG” toward “more structured knowledge organization” — and that may be a necessary step for Agents to evolve from demos into genuinely usable products.

RAG isn’t going away, but it shouldn’t be the entirety of Agent memory.

References

• TRACE: open-source hierarchical memory for LLM agents - Reddit - Original post by the TRACE project author on r/MachineLearning, including benchmark data and implementation details.