Backpressure for AI agents: the pattern nobody teaches

# Backpressure for AI agents: the pattern nobody teaches

Backpressure is one of the oldest ideas in streaming systems. Producers produce, consumers consume, and when consumers slow down, producers slow down. Every well-behaved streaming platform implements some version of it.

AI agents break the standard playbook in two ways:

1. The cost per message is huge and variable. One event might trigger a $0.001 lookup. The next triggers a $0.30 chain-of-thought with 12 tool calls. Standard backpressure — "consumer is 90% behind, slow down producers" — is far too coarse.
2. Slowness is not a failure signal. An agent taking 45 seconds is normal. An agent taking 45 seconds while its budget said 5 is a runaway. The runtime has to tell those apart.

Here is the pattern we use.

Three signals, not one

Do not treat "lag" as one number. Track three:

1. Queue depth per agent type — how many events are waiting for the fraud agent vs the refund agent.
2. Time-in-flight per task — how long each currently running task has been running.
3. Budget burn rate — tokens spent per minute per agent type, vs the budget.

Each one tells you something different. Queue depth says "we are behind". Time-in-flight says "we are stuck". Budget burn says "we are about to be broke".

Three actions, not one

Map each signal to an action:

• Queue depth high → shed load. Drop low-priority events, or route them to a slower cheaper agent. Do not slow the producer if the producer is your users.
• Time-in-flight high → kill the task. Any task past 2× its budget is almost certainly stuck in a tool retry or a model loop. Kill it, log it, alert on it.
• Budget burn high → downgrade the plan. Switch from the expensive model to the cheap model. Drop the reflection stage. Cap tool budgets. Degrade gracefully instead of stopping entirely.

The goal is that the system never fully stops. It gets slower, or dumber, or more selective, but it stays alive.

Why "slow down the producer" is wrong for agents

In a classic Kafka pipeline, backpressure means "consumer is slow, so producers pause". This works because the producers are usually internal — a CDC job, an ETL, a batch load.

Agent workloads are triggered by real-world events: a payment happens, a sensor fires, a user asks a question. You cannot pause reality. Slowing the producer means dropping events entirely, which is worse than a slow reply.

Better: shed selectively, downgrade cheaply, and always give the caller some response — even if it is "we saw this, we will get back to you". That is what a real-time system looks like at the edge of its capacity.

The one-slide summary

| Signal | What it means | What to do | |---|---|---| | Queue depth high | Behind on volume | Shed low-priority events | | Time-in-flight high | A task is stuck | Kill it | | Budget burn high | Cost spiraling | Downgrade to cheaper plan |

Three signals, three actions, no one giant "pause" switch. That is backpressure for AI agents.

StreamFlow implements all three signals as first-class runtime metrics. See how →