Why stateful AI agents do not need a vector database

# Why stateful AI agents do not need a vector database

Open any "build an AI agent" tutorial from the last two years and you will see the same stack: an LLM, a vector database, and glue code. The vector DB became the default memory layer almost by accident — it was the first primitive that felt like it solved retrieval.

But for the majority of agentic workloads, a vector DB is the wrong tool. Here is why, and what to use instead.

What agents actually need to remember

Break "memory" down and it splits into four distinct things:

1. Working memory — the current conversation, the current task, the current plan. Lives for seconds to minutes.
2. Episodic memory — what happened in past sessions with this user. Lives for weeks.
3. Semantic memory — facts about the world, the user, the business. Lives forever.
4. Procedural memory — how to do things. Prompts, tool definitions, workflows.

A vector DB is only a good fit for one of these — semantic memory over unstructured text. For the other three, it is either overkill or actively wrong.

The four tools you actually want

Working memory → the event log

Every decision, tool call, and result is an event. Append them to a log keyed by task ID. The "memory" of the current task is just replaying the log. No embeddings, no similarity search, no dimensions to tune.

The agent asks "what did I just do?" and the runtime hands it back the last N events. That is it.

Episodic memory → a normal database

Users have IDs. Sessions have IDs. Store them in Postgres. Index them by (user_id, timestamp). When a user comes back, load their last 5 sessions in a single query.

"But what if the agent wants to search across sessions?" — 95% of the time it wants "the most recent thing about X", which is a WHERE clause with a LIKE, not a similarity search. The 5% that needs fuzzy search can use Postgres full-text search, which is free and already there.

Semantic memory → structured facts, not embeddings

Most of what an agent needs to know about a user or business is structured: name, plan, permissions, integrations, preferences. Store it as JSON in a context table, one row per user.

Unstructured knowledge — docs, tickets, transcripts — is a good vector DB use case. But for a typical customer-support or ops agent, this is maybe 10% of the memory it uses per turn. Do not build the whole architecture around the 10%.

Procedural memory → git

Prompts, tool schemas, workflow definitions — these belong in a repo, versioned, reviewed, and deployed like code. Not in a vector DB. Not in a "prompt management platform". In git, next to the code that runs them.

When a vector DB *is* right

Use one when:

• You have >100k documents of unstructured text that need semantic search.
• Users ask fuzzy questions in natural language ("that thing about pricing from last quarter").
• Keyword search demonstrably fails your users.

Do not use one because a tutorial told you to.

The runtime shift

Once you split memory into these four tools, the shape of your agent changes:

• The event log makes replays and debugging trivial.
• The relational DB makes queries fast and cheap.
• The structured context makes prompts small and predictable.
• Git makes changes reviewable and reversible.

You also delete a whole class of failure modes: no embedding drift, no re-indexing jobs, no similarity thresholds to tune, no "the agent hallucinated because retrieval was slightly off".

The one-liner

Vector DBs are a great tool for one job. Making them the memory layer for every agent is like using S3 as your primary database — it works until it does not, and by the time you notice, you have already built around it.

Start with a log, a table, some JSON, and a repo. Add a vector DB the day you actually need it — not before.

StreamFlow Pulse ships with the event log, structured context, and workflow-in-git model out of the box. See how it works →