Notes on building agents that touch the real world.

The smartest thing an anomaly detector can do is nothing, in the sense that it never touches a production account itself. It hands the finding to a table, and something else, under its own policy, decides whether to act.

For months I wrote about 'an agent framework' instead of naming one. That wasn't a style choice. It was cover for a decision I hadn't made: LangGraph, Google ADK, or PydanticAI, and which career each one actually bet on.

The design question that came out of a crashed orchestrator: how do you let an agent run unattended without ever giving it a decision to make? A closed decision table, four failed review rounds, and the line between orchestration and judgment.

Four fences is a lot to get right, and you do not get to get them right once. You get to get them right on every project, forever. So I stopped rebuilding the security and started inheriting it.

You can put three fences on the way in and still leak, because the agent legitimately fetched data it was allowed to and then said it to the wrong person. The breach is on the way out, and nothing inbound was ever going to catch it.

You can assume the agent is hostile and hand it only a small set of capabilities, and it will still try to read the wrong tenant's row. The fix is not a better prompt. It is three fences that sit below the agent, on the way in, none of which trust it.

For a while I answered every new jailbreak by hardening the prompt. That is a game the attacker moves last in. Capability security is how you leave it: make the agent's authority something it holds in a small, fixed amount, not something it can talk its way into.

I put a careful rule in the system prompt: never show one client's data to another. It felt like a wall. Then I remembered the agent reads untrusted text into the same context as my rules, and a prompt is the least trustworthy part of the system.

I planned to move my system off one agent framework and onto a newer one. I never finished, on purpose. Running both engines behind one seam, with a test that proves they match on every commit, turned a portability claim into a property I can check.

After an orchestrator crashed my machine, I came back to autonomous agents by running the whole loop myself, by hand. Two passes found seven guardrails I would never have specced from a whiteboard, starting with a static review that could not catch a runtime bug.

Every agent framework sells you the easy 80%: a graph, some tool-calling, a tidy demo. The remaining 20% is the part that decides whether your system survives contact with a real customer account. You are going to build all of it yourself.

I adopted Google's Agent Development Kit, shipped a feature on it, then pulled it out and replaced the agent with about 30 lines of plain Python. A field report on when an agent framework is the value and when it is just ceremony.

A hallucination is text a human can catch. A retry is an action that already happened, and it can happen twice. Here is the one seam that makes every external write idempotent, and why I chose to refuse a repeat rather than risk a duplicate.

Before my marketing-data platform could send a single useful alert, I deleted six attribution models, a competitive-intelligence feature, and a tracking audit. They were not wrong. I had built the brain before the warehouse.

My test suite was green. Every test passed. Then an adversarial review pass failed the feature anyway, twice, for bugs the suite physically could not see. Here is why a green suite lies to you about your database.

I built an agent that answers questions about marketing data in plain English. Before it could answer a single real one, it had to pass 30 it had never seen. The hard part was never the model. It was writing down what a correct answer even is.

My first autonomous-agent orchestrator wrote 549 GB to disk in one night, exhausted memory, and triggered a kernel panic. Then it kept dispatching into the dying machine. That night taught me where guardrails actually belong.