MCP tool calling in production: patterns that survive real traffic

# MCP tool calling in production: patterns that survive real traffic

MCP (Model Context Protocol) is the cleanest way we have to let an LLM reach into the outside world. In a notebook it feels magical. In production it breaks in ways that are painfully familiar to anyone who has run distributed systems.

This is a field guide to the four patterns we use inside StreamFlow Pulse to keep MCP calls boring.

1. Treat every tool call as a network call

The biggest mistake teams make is treating an MCP call like a function call. It is not. It is a network call to a process you do not control, over a transport that can and will hang.

Every MCP call in a production agent needs:

• A hard timeout. Not "reasonable" — hard. 5 seconds is a good default for read tools, 30 for write tools. If the model wants longer, make it justify it explicitly.
• A retry budget. Two retries with exponential backoff, then fail. Never let the model retry in a loop by "trying again" — that turns one flaky tool into a runaway bill.
• A circuit breaker per tool. If search_docs fails 10 times in 60 seconds, stop calling it for 5 minutes and route around it. The model does not need to know; the runtime just returns "tool unavailable" and lets it plan differently.

2. Separate the reasoning stage from the acting stage

Asking one LLM call to both decide what to do and do it is where most agents fall apart. The model gets one bad tool response, panics, and either loops or gives up.

The pattern that works:

• Decide — one call, tool_choice: "auto", temperature 0.2, no side effects. Output is a plan: which tools, in what order, with what arguments.
• Act — a deterministic runtime executes the plan, one tool at a time, with timeouts and retries.
• Reflect — a second LLM call sees the results and decides whether to continue, replan, or stop.

This is the observe→decide→act loop, and it is worth the extra token cost. Debugging becomes trivial because each stage is a separate event you can replay.

3. Make tool outputs idempotent-friendly

MCP does not give you idempotency out of the box. You have to build it in.

• Every write tool should accept an idempotency_key. Generate it deterministically from the plan (hash of the decide-stage output).
• Every read tool should be cache-safe. Wrap it with a 30-second in-memory cache keyed on the arguments. The same agent will ask the same question three times in a row — do not pay for it three times.
• Log every call with the key, arguments, response, and latency. This is the only way to debug an agent that "sometimes" double-books a meeting.

4. Budget everything

A production agent needs three budgets, enforced by the runtime, not the model:

1. Tool budget — max N tool calls per task. Default 10. When exceeded, the runtime forces a final answer.
2. Token budget — max input+output tokens per task. When exceeded, truncate the oldest tool responses first.
3. Wall-clock budget — max seconds per task. When exceeded, return the best partial answer.

All three should be visible to the model in its system prompt so it can plan accordingly, but enforced by the runtime so it cannot ignore them.

The payoff

With these four patterns, an agent that used to fail 15% of the time in production drops to under 1%. More importantly, when it does fail, you can look at a single event log and see exactly which tool, which call, and why.

That is the difference between an agent you can deploy and a demo you can only present.

Want to try these patterns without writing them yourself? Pulse ships all four out of the box — timeouts, retries, circuit breakers, idempotency keys, and budgets are runtime primitives, not code you write.