My Tests Passed. The Reviewer Failed Me Twice
My test suite was green. Every test passed. Then an adversarial review pass failed the feature anyway, twice, for bugs the suite physically could not see. Here is why a green suite lies to you about your database.
My tests passed. Every one of them. Then an independent reviewer failed the feature anyway. Twice. For bugs the green suite could not physically see.
Here is why your test suite is lying to you about your database.
The setup
The feature was done. Marked done, anyway. The full suite was green, the happy path worked, and if it had been up to me that afternoon I would have shipped it.
It was not up to me. Before anything ships, I run it through an adversarial review: a separate AI pass whose only job is to try to prove the change is unsafe. Not “does this look right.” “Where does this break.” It read the same diff my tests had just blessed, and it failed the feature.
Round one: three bugs the green suite could not see
Three findings. Not one of them was visible to a single passing test.
The double lies. I derived an id as PREFIX-{key}. The production column caps ids at 36 characters. A long upstream key produces an id that Postgres rejects outright, a hard 500. My tests never caught it, because they ran against an in-memory SQLite double, and SQLite accepted the oversized id without a word. The suite was green because it was testing a more forgiving database than the one that runs in production.
“At-most-once” that wasn’t. I had written the idempotency as read-then-insert: check whether the row exists, and if it does not, insert it. That is fine in one thread. Under two concurrent retries it is a race. Both reads miss the row, both inserts fire, and one of them dies on the unique constraint. My tests ran the happy path, single threaded, so it looked idempotent. It was idempotent right up until the moment idempotency would have mattered.
An impossible state, accepted. The write path took a handoff between two records, each with a start and an end, and it happily accepted an end that came before its start. A logically impossible ordering. Nothing in the code stopped it, and nothing in my tests thought to try.
Round two: it failed again
Here is the part that taught me something. I fixed all three, resubmitted, and the reviewer failed it again.
My round-one fix had bounded the id in the column I had noticed. Good. But the same raw, unbounded key was also being written into a second 36-character column a few lines down, and I had walked right past it. Same overflow. Same 500 sitting in production, waiting. I had fixed the bug I was looking at and left its twin untouched.
What it taught
Three things I now treat as load-bearing.
In-memory doubles lie about the datastore. SQLite is not Postgres. It accepts ids Postgres rejects, it shrugs at constraints Postgres enforces, and every gap between the two is a bug your green suite is actively hiding from you. If the test does not hit the real engine’s constraints, “passing” tells you nothing about production.
At-most-once written as read-then-insert is not idempotent. There is a gap between the check and the act, and concurrency lives in that gap. Make the claim the atomic primitive instead: insert the key first and let the unique constraint be the thing that arbitrates. Do not ask the database whether something exists and then act on a yes that was already stale.
A fix has to chase the whole input class, not the one column you happened to be looking at. If a value is dangerous in one place, go find every place it lands before you call it fixed. Better, put the check where the value is born instead of where it lands: one bounded constructor that every write path goes through, so an oversized id cannot reach any column, the one I saw or the one I didn’t.
This is not only a backend story. An agent doing its own retries and writes hits these exact gaps, the unbounded value and the read-then-insert race, faster and more often than I ever will by hand. Hardening the write path against concurrent retries stopped being database hygiene for me. It became the precondition for letting anything automated near real data.
I used to think a green suite meant “safe to ship.” Now I think it means “safe against the parts I remembered to doubt.” The reviewer’s job is to doubt the parts I didn’t.