The Five Things an Agent Framework Won't Give You

I’ve shipped the same five things into every production agent I’ve built, regardless of which framework was underneath. They are never in the quickstart. They rarely have first-class support. And they are, without exception, where systems break when they finally meet a real workload.

None of this is an argument against frameworks. They’re good at what they do. It’s an argument for knowing exactly where their responsibility ends and yours begins, because the seam is not where the marketing puts it.

1. Durable state that survives a restart

A demo keeps its state in memory. A production run does not get that luxury: the process will be redeployed mid-task, the container will be evicted, the human will close the laptop. If your agent can’t resume from the last committed step, every long-running task is one deploy away from starting over.

The primitive you actually need is a checkpoint keyed by a stable run identifier, written transactionally with the side effect it describes:

# the smallest durable step that is safe to retry
async def execute_step(step, store):
    if store.already_done(step.id):
        return store.result_for(step.id)

    result = await step.run(idempotency_key=step.id)
    store.commit(step.id, result)   # state + effect, one transaction
    return result

Notice there’s nothing clever here. That’s the point. The cleverness is in deciding what counts as a step, not in the persistence.

2. Idempotency on every external effect

The moment an agent can retry, it can retry after it already charged the card. An idempotency_key on every outbound call is the difference between “the run recovered” and “the customer was billed twice.” Treat it as non-negotiable for anything that mutates the outside world.

An autonomous system you can’t safely retry is not autonomous. It’s a single point of failure with a chat interface.

3. A human-in-the-loop that is real

“Human-in-the-loop” is usually drawn as a box with a checkmark. In practice it is a suspension of the run: the agent reaches a gated action, persists its entire context, emits a request for approval, and stops, possibly for days, until a person responds. That is a distributed state machine, and the framework will hand you maybe half of it.

• The gate must be declarative: which actions require approval is policy, not code buried in a node.
• The suspended run must be fully serializable, including pending tool calls.
• Resuming must be idempotent, because approvals get clicked twice.

4. Observability built for non-determinism

You cannot debug an agent from request logs alone. You need the full trace: the prompt, the tools considered, the arguments chosen, the result, and the branch taken, captured even when nothing went wrong, because “nothing went wrong” is what you’ll be asked to prove. The config that earns its place looks boring on purpose:

# the orchestrator depends on the port, never the vendor
orchestrator:
  driver: engine-a        # swap target: engine-b
  checkpoint: postgres
  resume: by_run_id
  trace:
    capture: "always"     # not just on error
    sink: otel

5. Evaluation you own

The framework will not tell you whether your agent got better this week. That’s a regression suite (small, fast, run on every change), and it belongs to your team, not a public leaderboard. I’ll make that full argument in a later piece; for now, assume that if you can’t measure a change, you can’t ship one.

Where this leaves the framework

Use the framework for the graph, the tool-calling, the retries it does give you. Build the five primitives above against your own interfaces, and keep the framework on the far side of a seam you control. When the next orchestrator arrives (and it will), you’ll port an adapter, not rewrite a product.

That seam is the subject of the next essay.