The observe → decide → act loop: an architecture pattern for real-time AI agents

# The observe → decide → act loop: an architecture pattern for real-time AI agents

Most production AI agents fail for a boringly consistent reason: they conflate three responsibilities into one process. This post is the pattern we've watched teams converge on, why the separation matters, and how to actually build it.

The three responsibilities

An agent doing real work in production is always doing three things:

1. Observe — turn raw events (webhooks, database changes, IoT telemetry, log lines) into a normalized signal the agent can reason about.
2. Decide — take the signal plus historical context, ask an LLM (or a rule engine, or both) what to do.
3. Act — execute the decision through real-world tools: send emails, hit APIs, write to databases, page humans.

In demos, all three live in the same Python function. In production, they need to be separate stages with their own topics, their own retry logic, and their own scaling profile.

Why the separation matters

Observe is high-volume, low-CPU, latency-sensitive. It's mostly deserialization and schema validation. You want to scale it horizontally on the event rate and never let it call an LLM.

Decide is low-volume, high-CPU, expensive per call. Every LLM call is money and time. You want to batch, cache aggressively, and back off on failures. Scaling this like Observe would burn your budget.

Act is bursty, side-effect-heavy, needs strong retry semantics. Idempotency matters. A double-charge is worse than a missed charge. You want exactly-once semantics per action, plus a dead-letter queue for the ones that fail after N retries.

Bundle them together and you scale the whole thing to the peak needs of the most demanding stage. You also lose the ability to reason about failures.

What each stage looks like in YAML

Here's the fraud-detection pipeline distilled to its essence:

1pipeline: fraud-triage
2stages:
3  observe:
4    source: kafka://transactions
5    transform: |
6      { amount, currency, merchant, user_id, geo, device_fp,
7        _score: risk_features(user_id, amount, geo) }
8    output: signals.fraud
9
10  decide:
11    input: signals.fraud
12    condition: _score > 0.4
13    llm:
14      prompt: |
15        Given the transaction signal below, decide: allow, review, or block.
16        Return JSON: { action, reason, confidence }.
17      cache_key: hash(user_id, amount, merchant)
18    output: decisions.fraud
19
20  act:
21    input: decisions.fraud
22    tools:
23      - mcp://payments/hold_transaction
24      - mcp://alerts/notify_analyst
25    retry: exponential(max=5)
26    output: actions.done
27    dead_letter: actions.failed
28
29  audit:
30    input: actions.done
31    sink: postgres://audit_log

Note what's not in there: no orchestrator, no scheduler, no state machine. The topics are the state machine.

The mistakes we keep seeing

Skipping Observe. Teams point Decide directly at raw webhooks, then wonder why the LLM gets confused by 200 KB payloads containing 5 relevant fields. Observe is where you strip the noise.

Making Decide synchronous. If Decide calls Act inline and waits for the result, you've collapsed two stages back into one. Act asynchronously, always.

No dead-letter topic. Every real pipeline has actions that fail after every retry — a downstream API is down, a schema drifted, a tool got deprecated. Send them to a dead-letter topic and let a human triage.

Coupling to a specific LLM provider. Providers change pricing, deprecate models, rate-limit unpredictably. The Decide stage should treat the LLM as a swappable dependency.

Where the runtime fits in

You can build this pattern on Kafka + Kubernetes + custom workers. Many teams do. It works.

We built Pulse because we got tired of building the same pipeline scaffolding on every new project. Pulse is a runtime that takes the YAML above literally — you write the pipeline, it handles the topics, the consumer groups, the retry logic, the dead-lettering, and the MCP tool plumbing.

It's free, self-hosted, no signup. Bring your own LLM provider, install your own MCP tools, and you have a production-grade observe → decide → act loop running in about five minutes.

→ 5-minute quickstart → More pipeline examples → Architecture deep-dive