# Memory engines for long-running agents

URL: https://www.thedeepfeed.ai/posts/2026-04-30-memory-engines-for-agents/
Category: Agents
Published: 2026-04-08
Updated: 2026-05-30
Author: the-deep-feed
Tags: agents, memory, infrastructure, vector-databases, sqlite
Kind: deep

> Every agent-memory tutorial opens with a vector DB. That is a product opinion, not a default. Here is what mature agent runtimes actually use, and the narrow window where Qdrant or sqlite-vec earns its keep.

## TL;DR

- Every agent-memory tutorial starts with a vector database. That is a product opinion, not a default. Most long-running agents need a different memory model.
- The boring stack (files, SQLite, git) runs in-process, survives restarts, and gives you the audit log for free. The thesis aged well: in February 2026 Letta shipped **MemFS** — markdown files in a local git repo — enabled by default, and Anthropic's memory tool is flat files with client-side CRUD. The boring stack is now the product.
- Vector search is the right answer when episodic memory crosses **roughly 50K items**. Below that, BM25 plus a label index beats a vector DB on latency and operability.
- Engines worth knowing: mem0 (in-process extractor, ~57K stars), Zep/Graphiti (temporal knowledge graph), Cognee (GraphRAG control plane), Letta MemFS (git-backed markdown filesystem, default-on), and Supermemory — whose experimental ASMR technique drops vector search entirely for an agent swarm and claims ~99% on LongMemEval.
- Default to files and a git commit per write. Reach for vector only when you can name the corpus, the query shape, and the failure cost out loud.

![Layered memory architecture rendered as a tilted cake-stack of tiers](/post-images/memory-engines-for-agents/hero-memory-cake.jpg)

Every "agent memory" tutorial opens the same way. Pick a vector database. Embed your conversations. Top-k on every turn. The diagram has Pinecone or Qdrant in the middle, the LLM on the right, and an arrow labelled "context" connecting them. It is the most-copied architecture in the agent stack, and for most teams it is the wrong one.

The architecture is downstream of a marketing problem. Hallucination is the AI industry's central anxiety, "memory" is the word that sells the fix, and embeddings plus a vector store is the most demoable answer. The pitch fits on a slide. The slide does not survive a long-running agent. The most-copied diagram in the category (LLM plus vector DB plus arrows) is also the most over-applied. Long-running agents need a state machine, not a similarity search; vector DBs feel like "memory" because they are the only piece you can buy as a SKU.

The disillusionment happens in production. banteg, after a real evaluation:

> choosing a memory system for an assistant. so far mem0 truly lives up to its name, send it to zero. it can't recall shit and i had to patch severe ranking bugs for it to be usable. do not use this one.
>
> — [@banteg](https://x.com/banteg/status/2049124081853718612), Apr 28, 2026

What survives is more boring. After auditing agent codebases that have been live for months, the memory layers that hold up share a pattern: a small set of always-rendered files for identity and active state, a SQLite database for episodic events, a git repo for the audit trail, and a vector index added later, narrowly, when a specific recall query requires it. That is the shape Letta ships, the shape Mastra ships, the shape mem0 defaults to under in-process config. The vector DB is a feature, not the substrate.

## What agent memory is

Agent memory is three different problems wearing the same word.

**Ephemeral memory** is the turn buffer. The last N messages, current tool-call results, the scratchpad the model writes during a single task. Lives in the context window, dies at the end of the run. Solved by the LLM's own KV cache and whatever your agent loop hands back into the next turn. Nobody confuses this with persistence.

**Semantic memory** is cross-session recall. The trade note from six weeks ago, the Slack thread the agent half-remembers from yesterday, the user's stated preference about coffee. This is what the vector-DB tutorials are pretending to solve. The honest version: given a free-text query, surface relevant items from a corpus of episodic events the agent has accumulated. Vector search is one way to do that. It is rarely the first way.

**Structural memory** is the state machine. The agent's identity, the rules it must follow, the current plan, open positions, active workflow. This memory has to be in context every turn. Retrieval cannot be allowed to miss it. A trading agent that forgets its own risk policy because the embedding query surfaced a different chunk is a trading agent that gets liquidated.

The three need different storage. Treat them the same and you get a system that retrieves the user's coffee preference reliably and the stop-loss rule never.

![Concentric tiers showing working memory, session memory, and long-term memory](/post-images/memory-engines-for-agents/memory-tiers.jpg)

Treat ephemeral, semantic, and structural memory as one system and you ship an agent that forgets its own constitution but remembers what color shirt the user mentioned in May.

> **3** distinct memory problems
>
> **1** word everyone is using to describe them

## The boring stack

What mature agent runtimes actually run is files, SQLite, and git. Once you have seen it three times you stop being surprised.

Christian Tzolov, who ships agent infra for Spring AI, described exactly this shape from a real deployment:

> File-based long-term memory for @SpringAI agents - no vector store, no database, just files.
>
> Plain Markdown files, MEMORY.md index, the agent manages it all via tool calls. Claude auto-memory compatible but portable to any LLM provider.
> 📖
>
> #Java #AI
>
> — [@christzolov](https://x.com/christzolov/status/2041439491131392289), Apr 7, 2026

Files for structural memory. The agent has a handful of named markdown blobs loaded into the system prompt at startup and re-rendered every turn: identity, rules, active plan, current focus. Letta calls these "core memory blocks." Mastra calls them "working memory." Anthropic's own [memory tool reference](https://docs.claude.com/en/docs/agents/tool-use/memory) describes the same shape. The blobs are small (kilobytes, not megabytes), labelled, and edited by surgical tools the LLM can call. Edits go to disk atomically. A git commit fires after every successful write.

SQLite for episodic memory. Every turn writes a row: timestamp, user input, agent output, tool calls, outcome. SQLite handles a million rows on a laptop without complaint. The schema is boring on purpose. A `messages` table, a `tool_calls` table, a `decisions` table if the agent makes them. This is the database that runs underneath ChromaDB's `PersistentClient` anyway. You can either use ChromaDB and inherit the SQLite or skip the wrapper and write the rows yourself.

Git for the audit trail. The blocks directory is a git repo. Every block edit produces a commit with a structured message. A bad LLM turn that overwrites the risk policy is one `git revert` away from being undone. Blame, history, rollback, free. The pattern is older than Letta — but in February 2026 Letta made it the headline. The company behind the original memory paper shipped [MemFS](https://docs.letta.com/letta-code/memfs): "a directory of markdown files in a local git repository, giving you full version history, conflict resolution, and the ability to inspect or edit memory files directly," and every new agent gets it switched on as the default. The boring stack stopped being the thing mature teams quietly converged on and became the thing the reference runtime ships out of the box. Anthropic's own memory tool tells the same story from the other end: it is flat files in a directory with client-side create, read, update, and delete — no embedding store anywhere in the design.

The reason this stack works is that the three memory problems sort cleanly onto it. Structural memory goes in files, gets git-versioned, lives in the prompt. Episodic memory goes in SQLite, gets queried with `WHERE` clauses or full-text search. Ephemeral memory stays in the agent loop and never touches disk. The architecture is honest about which memory has which lifecycle, which is what most vector-first tutorials miss.

Git-as-memory specifically is fine for any agent whose total structural state fits in a few hundred kilobytes of markdown. That covers personal assistants, coding agents, customer-support agents, trading daemons running under a few dozen entities. A FAANG-grade compliance backend buys you nothing you do not already get from `git log`.

![Side-by-side: humble files+SQL+git tools versus an industrial vector apparatus](/post-images/memory-engines-for-agents/boring-stack-vs-vector.jpg)

Teams that have run this stack for a year or more describe the same arc: they picked files plus SQLite plus git in 2024 expecting to "graduate" to a vector DB by year-end. Several million turns later, the graduate moment never came. The killer feature of git-as-memory is `git revert` — and the cost of rebuilding that primitive on top of a vector database is exactly why the boring stack is hard to dislodge.

## When to reach for vector

Vector search earns its keep when three conditions hold at once.

First, the corpus is large. By "large" we mean at least 50,000 items the agent might recall, growing week-over-week. Below that threshold, BM25 over SQLite's FTS5 plus a labels index returns the right rows in single-digit milliseconds and never has an embedding-drift problem. ChromaDB quietly degrades past 100K items on a single machine. Qdrant or pgvector start making sense around the same scale.

Second, the queries are genuinely semantic. "Find the conversation where the user complained about onboarding" is semantic. "Find all tool calls that errored last Tuesday" is a `WHERE` clause. A surprising fraction of the queries agents actually run are the second kind, dressed up as the first. If you can write the query in SQL, do not embed it.

Third, the cost of missing a relevant item is acceptable. Vector search is probabilistic. Top-k sometimes misses the document you needed. For semantic recall over a large corpus, "the agent missed a relevant note" is usually a non-event; the next turn surfaces it. For structural state the same failure is catastrophic. Anything that has to be in the prompt every turn does not belong behind a similarity search.

Most agents in production have fewer than ten thousand episodic items at any meaningful filter. SQLite with FTS5 handles that with a `MATCH` query in microseconds. The vector index is a six-week-out problem dressed up as a day-one decision.

![Two scatter panels: SQL primitives win at small scale, vectors win at large scale](/post-images/memory-engines-for-agents/vector-vs-primitives-scatter.jpg)

The honest threshold map:

| Corpus size | Query shape | Right primitive |
|---|---|---|
| < 1K items | any | In-memory list, linear scan |
| 1K – 50K items | structured (`WHERE`-able) | SQLite + indexes |
| 1K – 50K items | free-text keyword | SQLite FTS5 + BM25 |
| 1K – 50K items | genuinely semantic | sqlite-vec (in-process) |
| 50K – 1M items | semantic | mem0 + ChromaDB / pgvector |
| > 1M items, multi-tenant | semantic, ANN-tuned | Qdrant / Pinecone / Weaviate |

> **50,000** items — the rough threshold where vector search starts to earn its keep
>
> **0** — number of vector DBs needed by most agents in their first 18 months

Jerry Liu at LlamaIndex makes the same case from the retrieval side — ingest, structured store, then optionally vector:

> This is a great tutorial (credits @itsclelia + @lancedb) on how to build a practical retrieval pipeline that integrates directly with your agent harness.
>
> 1. Ingest a massive pile of docs with liteparse.
> 2. Store data in a vector db (despite my memes to the contrary, you will
>
> — [@jerryjliu0](https://x.com/jerryjliu0/status/2041665979261108418), Apr 7, 2026

If you can write the question as `SELECT … WHERE …`, do not embed it. A meaningful share of vector-search workloads in production are SQL queries dressed up as similarity search; the recall is "worse" because the question was never semantic to begin with.

## Five engines worth knowing

![Five memory engines worth knowing — mem0, Zep/Graphiti, Cognee (the GraphRAG control plane, in editorial red), MemFS, and Letta. Each represents a different bet on how agent memory should be structured.](/post-images/memory-engines-for-agents/five-engines.jpg)

The honest map of the space. Five named engines pictured above, each a different bet on how agent memory should be structured — plus a sixth, Supermemory, that arrived loud enough to belong on the same map. The primitives under all of them (SQLite, FTS5, sqlite-vec) are the same; what differs is how much machinery each stacks on top.

**mem0.** [mem0ai/mem0](https://github.com/mem0ai/mem0), Apache 2.0, ~57K stars. The most-deployed open-source memory layer for agents. Raw chat turns go in, an extractor decides what to persist, the result lands in a pluggable vector backend (ChromaDB by default, swappable to pgvector, Qdrant, or Milvus) plus a local SQLite history file. The [Mem0 paper](https://arxiv.org/abs/2504.19413) reports a 26% improvement on the LOCOMO benchmark over OpenAI's memory while using 90% fewer tokens at inference, and that number still headlines the README. The April 2026 redesign (Python `v2`, Node `v3`) rebuilt the core: a single-pass, ADD-only extractor at roughly half the latency, multi-signal hybrid retrieval (semantic plus BM25 plus entity matching fused into one score), and built-in entity linking that *removed* the optional graph store entirely — about 4,000 lines of Neo4j/Memgraph driver code deleted from the SDK. Because the new extractor never overwrites, superseded facts chain through `linked_memory_ids` rather than being mutated in place; `v2.0.4` (late May) added `delete_linked` to prune those chains transitively. This is the memory layer most teams should reach for first when episodic recall starts mattering. Failure mode: the extractor is non-deterministic, so identical inputs can produce different memories across runs. Log raw input separately if audit is on the table.

**Zep / Graphiti.** [getzep/graphiti](https://github.com/getzep/graphiti), Apache 2.0, ~27K stars — the largest of the five by stars, and the only one whose central data structure is a graph rather than a flat store. Graphiti builds a *temporal* knowledge graph: as facts change, the old ones are not deleted but *invalidated*, with a valid-from/valid-to interval attached. Retrieval fuses vector, full-text, and graph traversal. The payoff is the case the boring stack handles worst: a user says their shipping address changed, and the system invalidates the old address automatically instead of returning two contradictory facts ranked by cosine distance. Graphiti is the open-source engine; **Zep Cloud** is the managed layer on top (Flex tier starts at $125/month, metered at $1.25 per 1,000 credits, with bring-your-own-cloud and SOC 2 / HIPAA for regulated deployments). Reach for it when evolving entity relationships and "when was this true" are the product, not a footnote.

**Cognee.** [topoteretes/cognee](https://github.com/topoteretes/cognee), Apache 2.0, ~17.5K stars, "memory control plane for AI agents in 6 lines of code." Where mem0 extracts facts into a vector store, Cognee runs an *extract–cognify–load* pipeline that builds a full GraphRAG layer: `cognify()` turns raw sources (documents, Slack threads, images) into a graph of entities and relationships, and retrieval combines the graph with vector search. In April 2026 it shipped a `cognee.agent_memory` decorator that wraps an async agent function to retrieve relevant context before the call and persist the function's output as a searchable trace after it — the agent's own execution history becomes memory for future calls. It is the heaviest of the five conceptually, and the right choice when the *structure* between memories matters more than the memories themselves. Founder Vasilije Markovic; a Bayer case study anchors the enterprise pitch.

**MemFS (Letta).** The most consequential shift since this piece first ran. In February 2026 Letta replaced its in-context "memory blocks" with [Context Repositories](https://www.letta.com/blog/context-repositories) and shipped [MemFS](https://docs.letta.com/letta-code/memfs) — "a directory of markdown files in a local git repository, giving you full version history, conflict resolution, and the ability to inspect or edit memory files directly." A `system/` subdirectory is loaded in full into the prompt every turn; everything else is visible to the agent but pulled in on demand. MemFS is *enabled by default* for all new agents on the Letta API and in Local mode. This is the boring stack (files plus git) shipped as the headline product of the company that wrote the original memory paper. Background "memory reflection" subagents work in isolated git worktrees and merge changes back through standard git conflict resolution. The advice this piece gave in April is, by May, the default in the reference runtime.

**Letta (the blocks lineage).** [letta-ai/letta](https://github.com/letta-ai/letta), Apache 2.0, ~23K stars, the descendant of the [original MemGPT paper](https://arxiv.org/abs/2310.08560) and the $10M-seed company (Felicis-led; angels include Jeff Dean and Clem Delangue) behind MemFS. The durable contribution is the *block* pattern: N labelled memory segments, each rendered into the system prompt every turn, edited by surgical tools where a replace is exact-match-or-fail — the model has to quote the existing text byte-for-byte before it can edit, so a bad turn cannot silently wipe the block. The 2025 "memory omni-tool" let agents create and delete blocks on the fly; MemFS then projected the whole thing onto a git filesystem. The legacy blocks still run on Docker deployments. The *pattern* is the export, and every mature agent runtime steals a version of it. Letta Code, the memory-first coding harness built on this, is the top model-agnostic harness on Terminal-Bench.

**Supermemory.** [supermemoryai/supermemory](https://github.com/supermemoryai/supermemory), MIT, ~23K stars, the sixth name that belongs on this map and the loudest argument for the whole thesis. Founder Dhravya Shah raised $2.6M from Google and Cloudflare executives, and the product is a hosted "universal memory API": a six-stage pipeline (queue, extract, chunk, embed, build graph, store) that turns documents, URLs, images, and audio into a *living knowledge graph* where memories are "facts built on top of other facts" rather than flat entity-relation triples, with automatic updates and decay built in. The architecture leans on Cloudflare (Workers, KV, the edge), which is exactly the bet its investors are making. The headline is the one this piece would write itself: in March 2026 Supermemory claimed roughly **99% on LongMemEval** with a technique it calls ASMR (Agentic Search and Memory Retrieval) that *ditches the vector database entirely* in favour of a swarm of parallel LLM agents that read, search, and reason over raw conversation history. Vindication, with an asterisk Shah flags himself: ASMR is experimental, and throwing eight agents at every query trades the vector DB's millisecond lookup for the latency and token cost of a brute-force read. It proves the direction (structure and reasoning beat similarity search on the hard questions) without yet being the thing you put on a production hot path. Note the shape, though: like mem0, Zep, and OpenAI, the polished version is a SaaS endpoint, which is the trade the next section is about.

Two more worth naming, even though they are primitives rather than engines. **Mastra** ([mastra-ai/mastra](https://github.com/mastra-ai/mastra), Apache 2.0 since July 2025, ~24K stars) is the TypeScript-native equivalent of Letta's discipline; its [memory module](https://mastra.ai/docs/memory/overview) now leads with *observational memory* (a background agent maintaining a dense observation log that replaces raw history as it grows), alongside Zod-typed working memory and vector-backed semantic recall, over LibSQL or Postgres. And **sqlite-vec** ([asg017/sqlite-vec](https://github.com/asg017/sqlite-vec), Apache 2.0) is the in-process vector primitive most of these reach for at small scale: pure C, no external dependencies, a single `.db` file for an agent's entire memory. After a quiet 2025 ("`sqlite-vec` is back," the author wrote in March 2026), it gained experimental ANN and DiskANN indexes — though it is still pre-1.0 at `v0.1.x` two years in, which is itself the tell about how rarely agents actually need it. Its Faiss-based predecessor **sqlite-vss** has had no commits since May 2024; treat it as retired.

What you do not see on this list: Pinecone, Weaviate, standalone Qdrant. Good products. Not the right substrate for a single long-running agent. They are the right answer for a *fleet* of agents: multi-tenant, cross-customer, hundreds of millions of vectors, a team that operates databases as a job. If that is your situation you already know it.

A side-by-side, in the shape we'd want on a whiteboard:

| Engine | License | Backbone | Best for | Failure mode to know |
|---|---|---|---|---|
| mem0 | Apache 2.0 | Pluggable vector (ChromaDB default) + local SQLite history | First reach when episodic recall starts mattering | Non-deterministic extractor; log raw input separately |
| Zep / Graphiti | Apache 2.0 (Zep Cloud managed) | Temporal knowledge graph + vector + full-text | Evolving entities where "when was this true" matters | Graph overhead if your facts never change |
| Cognee | Apache 2.0 | GraphRAG pipeline (graph + vector) | When structure *between* memories is the product | Heaviest pipeline; more to operate and reason about |
| MemFS (Letta) | Apache 2.0 | Markdown files in a local git repo, default-on | Always-in-context structural state with full git history | Per-agent filesystem to manage; not on Docker server |
| Letta blocks | Apache 2.0 | Labelled blocks; Postgres + Redis server runtime | Surgical exact-match edits to in-context state | Server-side complexity if you adopt the full runtime |
| Supermemory | MIT (hosted API) | Living knowledge graph on Cloudflare; ASMR agent swarm | Hosted "universal memory" with top benchmark scores | SaaS hot-path dependency; ASMR latency/cost still experimental |

## The benchmarks the vendors fight over

If the engines have a battleground, it is three benchmarks that emerged as the standard set for comparing memory architectures over the past year. Worth knowing what they actually measure before you read a leaderboard.

**LoCoMo** (Snap, 2024) is the oldest and most-cited: a dataset of very long conversations, around 300 turns and 9K tokens each across up to 35 sessions, scored on question answering, event summarization, and dialogue generation. It is the benchmark mem0's paper used for the "26% over OpenAI" claim, and also the one the newer benchmarks were built to replace — at 9K tokens, a modern long-context model can simply read the whole thing.

**LongMemEval** (ICLR 2025) is the more discriminating successor: 500 questions across seven types, testing five distinct abilities — information extraction, multi-session reasoning, knowledge updates, *temporal* reasoning, and abstention (refusing to answer when the information was never given). Abstention is the one most memory systems quietly fail; a system that confidently answers a question it has no basis for is worse than one that says it does not know.

**BEAM** (ICLR 2026, "Beyond a Million Tokens") is the stress test: 100 conversations and 2,000 validated questions built from dialogues up to 10M tokens. Its headline finding is the one that matters for this whole argument — even models with 1M-token context windows, with and without retrieval, degrade as conversations lengthen. Long context is not a substitute for memory architecture; it is a more expensive way to hit the same wall.

The numbers move every release, and they come from interested parties — mem0's April 2026 report puts its token-efficient algorithm at 92.5 on LoCoMo and 94.4 on LongMemEval while averaging under 7,000 tokens per query against the 25,000-plus a full-context approach burns; Supermemory claims to push LongMemEval to roughly 99% by abandoning vector search for an agent swarm. Take any single vendor's leaderboard with the appropriate salt — the methods are tuned to the test, the test sets are small, and the highest scores tend to come with the heaviest inference bills. The durable takeaway is not the ranking; it is that the field now has shared, adversarial benchmarks at all, and that the hardest ones (abstention, temporal reasoning, multi-hop across sessions) are exactly the failure modes the three-memory-types split was built to handle. A vector DB does not score well on abstention. A state machine does.

## Why most teams need none of it

Look at what an agent does on an average day. Run a turn loop. Read a few config blobs at startup. Write a row to SQLite per significant action. Occasionally recall a note from last week. Almost never recall a note from last quarter. Corpus growth runs at hundreds of items per day, not millions.

Now look at what a vector DB buys. Sub-second top-k over a million-vector corpus. ANN-tuned recall curves. Multi-tenant isolation. Replication. The feature set is calibrated for search workloads at a scale the average agent does not have and will not have for months.

The honest trade-off table:

| Need                          | Boring stack                       | Add vector?              |
|-------------------------------|------------------------------------|--------------------------|
| Identity, rules, active plan  | Files + git                        | No, never                |
| Recent N messages             | In-memory ring buffer              | No                       |
| Episodic events, structured   | SQLite + indexes                   | No                       |
| Episodic events, free-text    | SQLite FTS5 + BM25                 | Only past ~50K items     |
| Cross-session semantic recall | mem0 with ChromaDB (in-process)    | Only past ~50K items     |
| Multi-agent shared knowledge  | Postgres + pgvector                | Yes, this is the case    |
| Time-travel audit             | git for blocks, append-only SQLite | Graphiti if entities matter |

Five of seven rows do not need a vector database. The two that do can run sqlite-vec in the same process for months before they outgrow it.

The advice we keep landing on: do not start with a vector DB. Start with files. Add SQLite. Add FTS5 the day a `LIKE` query feels slow. By the time embeddings are genuinely needed, the team has enough production data to know which corpus and which query shape they belong to. Premature vector adoption is the new premature optimization.

## Anatomy of a long-running agent's memory

The picture that gets drawn in vector-DB tutorials is a single arrow from "user message" to "vector store" to "context." The picture that runs in production is messier and worth tracing in full, because every hop is a place teams skip a primitive that would have done the job.

A turn arrives. Before the model sees the user's text, the agent loop reads its structural state — the labelled markdown blocks for identity, rules, current focus, working notes, lessons. Those render into the system prompt every turn, no retrieval needed, no embedding involved. Letta calls this "core memory." Mastra calls it "working memory." Anthropic's memory tool reference describes the same shape. This is the memory the agent cannot afford to miss, and it lives in the part of the stack where missing is impossible.

The user's message lands. The agent loop appends a row to SQLite: timestamp, user text, conversation ID, any metadata that came with the request. If a tool fires, that's another row. If a decision is made, that's a third. The episodic log accumulates at the rate the agent runs, which for most production deployments is hundreds to low-thousands of rows per day. SQLite handles a million rows on a laptop without complaint and the schema is open enough to reshape later.

Then, and only then, semantic retrieval. If the agent decides it needs to recall something from beyond its turn buffer, it issues a query. The query first hits SQLite FTS5 with BM25 scoring. If the corpus is small or the query is structured, the right rows come back in microseconds. If the corpus has crossed the threshold and the query is genuinely semantic, the same SQLite database has a `vec0` virtual table sitting next to the FTS5 one, and a hybrid score combines the two. ChromaDB's `PersistentClient` does this same trick under the hood; you can either use it or skip the wrapper and write the SQL directly.

The cleanest implementations of this pattern fit in roughly 200 lines of code: five labelled blocks in markdown, a SQLite database with FTS5, and an `if vector_threshold_reached` branch that lazy-loads sqlite-vec. The instinct that holds up is that adding more is a lossy operation.

The result lands back in the agent loop, gets summarized into the prompt, and the model produces a response. The response gets written to disk. The block that needed editing (say the agent learned a new rule from this interaction) gets surgical-replaced and committed to git. The cycle ends with the entire memory state durable, auditable, and reproducible at any prior turn through `git checkout`.

What is conspicuously missing from this trace: a vector database. Not because vector search is wrong, but because the structural memory never goes through it, the episodic log doesn't need it, and the semantic layer can run inside the same SQLite file the agent already uses. The vector DB *as a separate piece of infrastructure* shows up only when the corpus crosses the threshold or the team is running a fleet that justifies the operational overhead.

## The "memory feature" trap

A growing fraction of the agent-memory category is selling a hosted "memory feature": a SaaS endpoint where you POST conversations and GET retrievals. mem0 has a hosted plan. Zep is a managed service. OpenAI has a memory feature for ChatGPT. Each is a reasonable product. Each is also a hard dependency on someone else's hot path for the part of your agent that has to be most reliable.

The trade is real. Hosted memory means you do not run the database. You also do not own the data layout, cannot replay an audit, cannot reproduce a turn without the vendor's API being up. For consumer products this is fine. For a trading agent, a healthcare agent, or anything that has to survive a vendor outage, it is the wrong place to add a network hop. The boring stack runs in-process. The hosted-memory stack does not. The right diagnostic before you adopt one is simple: when this endpoint is down, what does my agent forget? If the answer is "everything important," the agent is one vendor incident page from collapse.

This is not anti-SaaS. It is a recommendation that memory infrastructure follow the same rule as authentication infrastructure: the more durable and identity-shaping the data, the closer to your own process it should live. The fact that the cheapest place to put your agent's identity is a markdown file on disk and the most expensive place is a SaaS vector index should be telling you something about the shape of the problem.

## Memory as eval

The unloved third dimension of memory engineering is *evaluating* whether the memory is doing its job. Vector-DB tutorials assume that retrieval quality is a property of embedding choice and top-k tuning. In practice, the highest-impact interventions for long-running agents are upstream of retrieval entirely.

Three checks worth running on any memory system before scaling it:

- **Block-overwrite test.** Run a prompt-injection turn that tries to convince the agent to rewrite its identity block. The system should refuse, log the attempt, and survive. If the agent overwrites the block, your memory architecture is wrong before any retrieval question.
- **Replay test.** Pick a turn from a week ago. Replay it against the agent's state at the time. The output should match within a stochastic tolerance. If it doesn't, your structural memory is leaking turn-to-turn.
- **Recall-vs-precision audit.** For semantic recall, log every retrieval, the top-k results, and whether the chosen turn used them. The hit rate on retrieved-but-unused items is the truest measure of whether your similarity search is solving a real problem.

Most "memory bugs" are eval bugs. The agent is not forgetting; the team has no way to tell whether retrieval ever surfaced the right item. Teams that add retrieval logging routinely discover they can rip the vector DB out entirely without anyone noticing.

The teams that ship reliable long-running agents have memory eval as a first-class concern. The teams that have a memory bug at 3 a.m. and no logs are running production on a slide.

## What most teams should do

Default to files. Pick five labels for the agent's structural state: identity, rules, current focus, working notes, lessons. Put each in its own markdown file. Render all five into the system prompt every turn. Edit them with append and exact-match-replace tools. Never whole-file overwrites. Commit to git on every successful edit. Half a day of work, replaces what most teams spend a week building with a vector DB.

For episodic memory, write rows to SQLite. Add an FTS5 index when the corpus crosses a thousand items. Do not embed anything yet. Most "I need vector search" instincts go away once SQL queries with `WHERE` clauses are on the table.

Reach for vector when one of three things becomes true. The corpus crosses 50,000 items and is still growing. The queries you actually run are demonstrably semantic and miss with keyword search. Or you have measured a recall failure that cost you something. When the threshold trips, add mem0 with ChromaDB or sqlite-vec to the existing SQLite database. Same file, same backup story, same replication. No new server.

Skip Pinecone, Weaviate, standalone Qdrant unless you are running a fleet. Skip Graphiti unless entity relationships are the product. Skip the SaaS memory layers unless you are willing to make `api.vendor.ai` a hard dependency of your agent's hot path. Most of the agent-memory industry is shipping products against a problem the boring stack already solved.

The reason memory engines exist as a category is that the marketing department needed an answer to "how do you handle hallucination," and "we use files and git" does not raise a Series A. Teams shipping production agents have already had this conversation internally and quietly chosen the boring answer. The choice is yours. Make it for the right reason.

## Three failure modes you will hit anyway

![Three memory failure modes — context bloat (drowning in too many memory files), stale recall (confidently reciting outdated facts, in editorial red), and recency bias (forgetting older critical info while remembering trivial recent inputs).](/post-images/memory-engines-for-agents/failure-modes.jpg)

No matter which substrate you pick, three failure modes recur in production. Knowing the shapes is the difference between a 3 a.m. incident and a Slack message.

**The block-rewrite cascade.** The agent learns something new, decides the rule should be updated, and overwrites a structural block. The new content is wrong. Subsequent turns inherit the corrupted state. The fix is a block-edit policy that requires exact-match-or-fail edits (the Letta pattern), per-block size caps that surface unusual writes, and a git-revert tool the agent can call to undo its own mistakes when it notices them. The architectural mistake here is letting the agent overwrite blocks freely. Surgical edits are not a nicety; they are the bulwark against memory drift.

**The semantic miss the agent never knew about.** Vector retrieval surfaces top-k. The relevant item was rank 11. The agent never saw it, never knew it missed, and produced a confident answer that contradicted the unseen turn. This is the failure mode that vector-DB tutorials handle worst because there is no error to retry on. The right instrumentation is a "shadow query" path that periodically runs BM25 and vector retrieval against the same query and compares the result sets. When they disagree by more than a threshold, log it as an investigation candidate. Most teams discover their vector index is misconfigured this way.

**The audit trail that is fiction.** mem0 and similar LLM-extractor memory layers are non-deterministic by design. The same input can produce different `ADD/UPDATE/DELETE` decisions on different runs, which is fine for product but disastrous for compliance. The remediation is two-tiered logging: the raw turns (deterministic, append-only, the truth) go to one table; the extracted memory (interpreted, lossy, the *agent's view*) goes to another. When auditors ask "what did the agent know," you replay the raw turns. When the agent asks "what did I conclude," it reads the extracted view. Conflating the two is how you end up with audit trails that cannot be defended in front of a regulator.

The pattern across post-mortems is consistent. Most production agent incidents trace back to one of three things: the agent wiped a structural block, retrieval missed and nobody noticed, or the audit log was the agent's own summary instead of the raw turns. All three are architecture-level mistakes, not engine-level ones.

## A reference architecture

To make the boring stack concrete, here is the architecture we'd hand to a team starting today, with the file paths and the engines we'd reach for in each tier:

| Tier | Storage | Engine | When to upgrade |
|---|---|---|---|
| Identity, rules | `agent/blocks/identity.md`, `rules.md`, `plan.md` | Files + git | Never. This is the substrate. |
| Recent turns (last N) | In-memory ring buffer | Plain Python / TS / whatever | When N exceeds context window |
| Episodic log | `agent/state.db` (SQLite) | SQLite + indexes | When > 1M rows or queries slow |
| Free-text recall | Same `state.db`, FTS5 virtual table | SQLite FTS5 + BM25 | When BM25 misses items keyword search should not miss |
| Semantic recall (small) | Same `state.db`, vec0 virtual table | sqlite-vec | When > 500K vectors or recall degrades |
| Semantic recall (large) | Postgres or Chroma | mem0 + ChromaDB / pgvector | When fleet, multi-tenant, or > 1M vectors |
| Multi-tenant fleet | Dedicated DB | Qdrant / Weaviate / Pinecone | When ops team can run it as a service |
| Audit / replay | `agent/blocks/.git/` | git | Never. Free observability. |

The dirty secret of agent infrastructure is that 80% of the "memory layer" decisions disappear if you put SQLite at the center and treat everything else (FTS5, vec0, mem0, Chroma) as queries against the same file. The complexity is in the indexes, not the architecture.

A reframe worth holding next to all of this, from Haitham Bou Ammar:

> Hot take:
>
> ⛔️Long-context reasoning is not only a context-window problem.
>
> 🫢It is a control-flow problem.
>
> Standard recursive LLMs often ask the model to invent recursive code during inference.
>
> That means the model is not just reasoning.
> It is also writing the program that
>
> — [@hbouammar](https://x.com/hbouammar/status/2049862531506717157), Apr 30, 2026

## The thirty-second rule

If the only thing you take from this is one heuristic: before you reach for a vector database, ask whether you can describe the corpus, the query shape, and the failure cost of a missed retrieval out loud in thirty seconds. If you can't, you don't need a vector DB. You need to ship an agent against the boring stack, watch it run for a quarter, and let the corpus and the queries tell you which engine they want.

The quiet truth of agent infrastructure in 2026 is that the answer for most teams is a markdown directory, a SQLite file, and a git repo. The engines worth knowing are mostly there to be reached for *narrowly,* in the precise moment they earn their place. Reach for them then. Until then, the substrate is enough. It always was.

## Sources

- [GitHub — letta-ai/letta](https://github.com/letta-ai/letta)
- [GitHub — mem0ai/mem0](https://github.com/mem0ai/mem0)
- [GitHub — mastra-ai/mastra](https://github.com/mastra-ai/mastra)
- [GitHub — asg017/sqlite-vec](https://github.com/asg017/sqlite-vec)
- [GitHub — asg017/sqlite-vss](https://github.com/asg017/sqlite-vss)
- [GitHub — getzep/graphiti](https://github.com/getzep/graphiti)
- [GitHub — supermemoryai/smfs](https://github.com/supermemoryai/smfs)
- [GitHub — chroma-core/chroma](https://github.com/chroma-core/chroma)
- [GitHub — qdrant/qdrant](https://github.com/qdrant/qdrant)
- [MemGPT paper (arXiv)](https://arxiv.org/abs/2310.08560)
- [mem0 — Mem0 paper (arXiv)](https://arxiv.org/abs/2504.19413)
- [Mastra — Memory docs](https://mastra.ai/docs/memory/overview)
- [Anthropic — Memory tool reference](https://docs.claude.com/en/docs/agents/tool-use/memory)
- [LangChain — Memory architecture guide](https://blog.langchain.com/memory-for-agents/)
- [LangGraph — Persistence and memory](https://langchain-ai.github.io/langgraph/concepts/persistence/)
- [Zep — Graphiti temporal knowledge graph](https://help.getzep.com/graphiti/graphiti/overview)
- [pgvector — PostgreSQL vector similarity search](https://github.com/pgvector/pgvector)
- [SQLite FTS5 reference](https://www.sqlite.org/fts5.html)
- [OpenAI — Memory feature for ChatGPT](https://openai.com/index/memory-and-new-controls-for-chatgpt/)
- [Letta — Context Repositories: git-based memory for coding agents](https://www.letta.com/blog/context-repositories)
- [Letta — MemFS docs](https://docs.letta.com/letta-code/memfs)
- [Letta — Context Constitution](https://www.letta.com/blog/context-constitution)
- [mem0 — State of AI Agent Memory 2026](https://mem0.ai/blog/state-of-ai-agent-memory-2026)
- [GitHub — topoteretes/cognee](https://github.com/topoteretes/cognee)
- [Cognee — agent_memory decorator docs](https://docs.cognee.ai/core-concepts/further-concepts/agent-memory-decorator)
- [Anthropic — context editing and the memory tool](https://www.anthropic.com/news/context-management)
- [LongMemEval (ICLR 2025)](https://arxiv.org/abs/2410.10813)
- [LoCoMo benchmark](https://arxiv.org/abs/2402.17753)
- [BEAM — Beyond a Million Tokens (ICLR 2026)](https://arxiv.org/abs/2510.27246)
- [GitHub — supermemoryai/supermemory](https://github.com/supermemoryai/supermemory)
- [Supermemory — How Graph Memory Works](https://supermemory.ai/docs/concepts/graph-memory)

---

Canonical: https://www.thedeepfeed.ai/posts/2026-04-30-memory-engines-for-agents/
Site: https://www.thedeepfeed.ai
Full corpus: https://www.thedeepfeed.ai/llms-full.txt