Two different cost curves live on the same axis. What does it cost to ignore the 2.3% that slips through the threshold — and what does it cost to chase that last 2.3% and close it? Priced, both ways, in a real study.
A melanoma registry followed 9,063 patients across 28 years. 206 of them — exactly 2.27%, the “2.3%” — got a false-negative diagnosis: the melanoma was there, and the first read said it wasn’t. The threshold that decided “this one’s fine, send them home” let them through. Nobody labeled where that threshold sat. This lab is what it cost — on both sides.
● verified from source data Every number below is recomputed from the study’s released patient-level data (Kaplan–Meier survival + the years-of-life-lost model), not from a summary. Where a figure is the paper’s model-adjusted headline rather than a direct recompute, it’s marked ◐ paper GLM.
You’re at 97.7%. You declare victory. You don’t build the protocol for the edge cases; you raise the review threshold; the 2.3% flows through untouched. Here is the same disease, ten years later, split by whether the first read caught it.
● verified — Kaplan–Meier at 120 months reproduces the study exactly. Recurrence-free survival for the missed group is less than half the caught group: 32.9% vs 76.2%.
Now price the miss in time. The cost isn’t spread evenly across the 206. Most were eventually fine; the ones it killed lost decades. Press to tally it.
Averaged over everyone, the miss looks small — about 8 years. That’s the number a dashboard shows. But averages hide the shape: for the patients the miss actually kills, it’s ~23 years each, and the youngest lose the most. The system didn’t fail catastrophically. It failed quietly, at the margin, in the patients nobody was watching.
Flip the axis. Say you decide the 2.3% is unacceptable and you go get it. Now a different law bites: diminishing returns. Getting from 0% to 90% is cheap — you’re clearing gross errors. Getting from 97.7% to 99.9% can cost more than everything before it combined: second-read mandates, AI-assisted review layered over clinical judgment, biopsy-threshold recalibration (which mints its own false-positives and anxiety), regulatory overhead. Drag the target and watch the marginal cost.
Illustrative model — the shape, not the price. Marginal cost rises like 1/(1−p): each fraction of a point past the knee costs disproportionately more than the last. Same curve shows up in Six Sigma manufacturing, aviation safety audits, and hydrologic model calibration.
Put both axes on one chart and it forces a real decision instead of a reflex: we know what it costs to miss the 2.3%. We know what it costs to eliminate it. Where does the system sit right now — and who decided that?
That is the Legibility Problem: the data passed, the structure was misleading, and someone paid for it. It is domain-independent — the transferable skill isn’t reading a dermoscopy image or a flood model, it’s learning to ask where the threshold is, who set it, and what happens to the fraction that falls through. Once you see it in one domain, you can audit any system for it.
| Domain | The miss | The threshold problem | The cost metric |
|---|---|---|---|
| Medicine this lab | 2.3% false-negative melanoma | diagnostic threshold lets cases slip; nobody labeled where it sat | years of life lost |
| Hydrology Lester’s Lab | flood-map discrepancy at the margin | algorithm flags auto-closed, human-review bar raised | property, lives at risk |
| Catalysis on the Docket | a mechanism claim vs. the evidence for it | where does “proven” sit, and did the validation reach it? | credibility, wasted research |
The catalysis exhibit is parked on ⏳ The Docket — the holding room, now built, where a case waits before it’s sorted into Sledgehammer, Caliper, or dismissed. Its first exhibit, The Knot That Waits, is already up. The catalysis paper may turn out to be the “did-it-right” contrast rather than another miss — it gets read before it graduates.