Naive RAG matches or beats complex agentic memory systems

The problem

Large language models are increasingly used in long-term interactions where they must track information across many conversations and tasks, according to the authors (arxiv.org). Because these interaction histories can far exceed a model's context window, recent research has introduced complex agentic memory systems that store, organize, and retrieve past information (arxiv.org). These systems use operations like extraction, summarization, mutation, consolidation, clustering, graph construction, and iterative or trainable search strategies (arxiv.org). The authors note that retrieval-augmented generation (RAG) is often dismissed as insufficient for long-term memory because critics view it as suited only for static corpora rather than dynamic state changes, and because prior evaluations have reported RAG trailing behind agentic memory systems (arxiv.org).

The idea

The authors investigate whether this added complexity is truly necessary by testing Raw-RAG, a naive retrieval-augmented generation baseline (arxiv.org). Unlike complex architectures, Raw-RAG indexes source history without using large language model-based extraction, summarization, graph construction, or mutation (arxiv.org). Each conversation is serialized with message timestamps and image captions, split into 256-token windows with a 32-token overlap, and placed in a flat dense retrieval index (arxiv.org). At query time, the system uses cosine similarity to retrieve the 128 closest windows, and a cross-encoder then scores and selects the top 15 windows as evidence for a reader model (arxiv.org). The authors evaluate Raw-RAG across four benchmarks—LoCoMo, LongMemEval, MEME, and MemoryAgentBench—comparing it against baseline systems such as EverMemOS, MemOS, EMem, SimpleMem, MemU, MemMachine, Mem0, Memora, Nemori, MemoryOS, A-MEM, and RecMem (arxiv.org, arxiv.org).

The results

The authors report that Raw-RAG ranks first among the compared memory systems in six out of seven benchmark-model settings, excluding full-context models (arxiv.org). Specifically, Raw-RAG outperforms the strongest compared agentic memory system by 0.13 to 5.53 points across the tested settings (arxiv.org). It leads LoCoMo by 0.65 to 0.95 points, LongMemEval by 0.13 points, GPT-4o-mini MEME by 2.24 points, and MemoryAgentBench by 4.48 to 5.53 points (arxiv.org). In contrast, MemMachine leads DeepSeek-V4-Flash MEME among memory systems by 1.52 points, while full-context leads Raw-RAG by 19.98 points in that specific setting (arxiv.org). In ablation tests on MemoryAgentBench, removing reranking lowers the total score by 3.92 points, while returning only one item lowers it significantly (arxiv.org). Furthermore, adding atomic-fact extraction, summary augmentation, or graph-based retrieval expansion does not improve Raw-RAG's performance, with atomic-fact extraction reducing total accuracy by 19.62 points (arxiv.org). In efficiency comparisons on a 10-episode MEME slice, Raw-RAG finishes its ingestion and search tasks in 305.91 seconds, making it 5.1 times faster than the next system and 30.4 times faster than EverMemOS (arxiv.org).

Why it matters

These findings challenge the presumed necessity of complex agentic memory architectures for the evaluated conversation tasks (arxiv.org). The authors demonstrate that capable reader models can often resolve dynamic state changes directly from sufficient raw history (arxiv.org). By showing that a straightforward retrieval pipeline matches or exceeds the performance of much more intricate systems, the work establishes naive RAG as a strong and essential baseline for future research in long-term conversation understanding (arxiv.org).

Caveats

The authors state that performance can vary across different reader models, noting that while capable readers resolve changing states directly from raw evidence, performance still depends heavily on the chosen reader model (arxiv.org). Additionally, this study is a preprint shared on arXiv and has not yet undergone formal peer review.