We asked five people to read a scan report and say what they would do next. Four read it correctly. The fifth read a report that found nothing as confirmation that nothing was wrong.

The report was over a database where half the tables were empty. There was nothing to find because there was nothing there.

Why this is the worst possible failure

A wrong number is visible. Someone checks it, disagrees, and the disagreement surfaces. A report that produces false confidence generates no disagreement at all — the reader closes it satisfied and the problem stays where it was.

It is also the failure mode that a tool like this trends toward naturally. Fewer findings look like better news, and a clean-looking report is what everybody wants to see.

Change one: the report names its own shape

A report now opens by saying which of four situations it is in, and no two of those sentences can be read as each other: problems were found; nothing was found and coverage was good; nothing was found because there was almost nothing to look at; not enough could be read to say anything.

That verdict is printed at the top as well as the bottom. A conclusion only at the end is a conclusion for the readers who got to the end.

Change two: say the size, do not make them divide

The report used to print two numbers and leave the reader to compare them. Now it prints the comparison: 18 of the 36 tables — half of them — hold no rows.

With a rule about when to say it, because a clause that fires on every report is a clause that stops being read: state the magnitude when the magnitude is the problem, stay quiet below a quarter, and only use a fraction when the fraction is actually accurate. "About a third" for 34% is rounding toward the more comfortable sentence.


← All posts