It worked, it said so, and nothing happened

A contact form returned 201 and showed a green confirmation every time. Nobody was ever notified. We went looking for more of these and found about thirty, all with the same shape.

A contact form on our pricing page worked perfectly. It validated the input, wrote the inquiry to the database, returned 201, and showed the sender a green confirmation. It had done that several times. Nobody had ever been told.

The notification step read an environment variable that was never set on the host. When it came back empty the send was skipped, and the skip was reported as success. Every layer above it agreed the operation had completed, because by its own definition it had.

We went looking for more of these and found about thirty.

They all have the same shape: an operation succeeds, reports success, and the thing that gives it meaning never happens. A record is written and nothing ever reads it. An alert dispatches to zero destinations and returns ok. A deletion reports that it cannot be undone while removing the one row that made the undo possible. The failure is not that something broke. Nothing broke. The failure is that the success was real and hollow.

A few of the ones we found in our own code. An alert dispatcher returned a hardcoded ok regardless of whether any channel accepted the message, which neutralised the guard one layer up that existed specifically to catch a message reaching no destination. An account deletion told the user the action was irreversible while deleting the payments row holding the subscription identifier needed to cancel their billing. An audit export computed a verified flag by comparing each row against a hash recomputed from that same row, which is the row checked against itself. A health monitor computed which services had recovered and never sent it anywhere.

These survive because every local check passes. Tests do not catch them, because a test asserts what a function returns and the function returns the truth about itself. Monitoring does not catch them, because nothing errors. Code review does not catch them, because each individual line is correct — and the incorrect thing is a relationship between two files that no reviewer has both of open at once.

The question that finds them is not a code question. For each success message this system can produce, what would have to be true in the world for that message to be honest, and is anything actually checking it? A 201 on a contact form claims a human will see the inquiry. A green toast on a settings page claims the setting changed. An alert marked delivered claims something arrived somewhere. Each of those is a claim about the world, and the code that makes it usually has no idea whether it is true.

Four searches that will find most of them in any codebase. Look for literal True or ok values in return statements, especially in anything named send, dispatch, notify or deliver. Look for tables that are written and never selected from. Look for results whose error or reason field is computed and then discarded by the caller. And take one success message — pick the one a customer sees most often — and trace it end to end in production rather than in a test.

We now treat a claim in a response body the same way we treat a claim on the pricing page. If the system says it sent something, something should have been sent, and something should be able to prove it.

This article is about a shipped surface: All product surfaces. Integration details live in the docs.