Two different cost curves live on the same axis. What does it cost to ignore the 2.3% that slips through the threshold — and what does it cost to chase that last 2.3% and close it? Priced, both ways, in a real study.
A melanoma registry followed 9,063 patients across 28 years. 206 of them — exactly 2.27%, the “2.3%” — got a false-negative diagnosis: the melanoma was there, and the first read said it wasn’t. The threshold that decided “this one’s fine, send them home” let them through. What had never been priced is what letting them through costs in years — the authors write that “to our knowledge, there are no studies to date on the number of YLL after FND in melanoma.” This lab is what it cost — on both sides.
● verified from source data Every number below is recomputed from the study’s released patient-level data (Kaplan–Meier survival + the years-of-life-lost model), not from a summary. Where a figure is the paper’s model-adjusted headline rather than a direct recompute, it’s marked ◐ paper GLM.
You’re at 97.7%. You declare victory. You don’t build the protocol for the edge cases; you raise the review threshold; the 2.3% flows through untouched. Here is the same disease, ten years later, split by whether the first read caught it.
● verified — Kaplan–Meier at 120 months reproduces the study exactly. Recurrence-free survival for the missed group is less than half the caught group: 32.9% vs 76.2%.
Now price the miss in time. The cost isn’t spread evenly across the 206. Most were eventually fine; the ones it killed lost decades. Press to tally it.
Averaged over everyone, the miss looks small — about 8 years. That’s the number a dashboard shows. But averages hide the shape: the paper reports that the 82 patients who died of melanoma lost about 22.4 more years than the 28 who died of other causes, and the youngest lose the most. The system didn’t fail catastrophically. It failed quietly, at the margin, in the 2.3% the first read had already cleared.
Flip the axis. Say you decide the 2.3% is unacceptable and you go get it. Now a different law bites: diminishing returns. Getting from 0% to 90% is cheap — you’re clearing gross errors. Getting from 97.7% to 99.9% roughly doubles the total effort spent to reach 97.7% in the first place, and every fraction of a point costs far more than the one before it: second-read mandates, AI-assisted review layered over clinical judgment, biopsy-threshold recalibration (which mints its own false-positives and anxiety), regulatory overhead. Drag the target and watch the marginal cost.
Illustrative model — the shape, not the price. Marginal cost rises like 1/(1−p): each fraction of a point past the knee costs disproportionately more than the last. Same curve shows up in Six Sigma manufacturing, aviation safety audits, and hydrologic model calibration.
Put both axes on one chart and it forces a real decision instead of a reflex: we know what it costs to miss the 2.3%. We know what it costs to eliminate it. Where does the system sit right now — and who decided that?
That is the Legibility Problem: the data passed, the structure was misleading, and someone paid for it. It is domain-independent — the transferable skill isn’t reading a dermoscopy image or a flood model, it’s learning to ask where the threshold is, who set it, and what happens to the fraction that falls through. Once you see it in one domain, you can audit any system for it.
| Domain | The miss | The threshold problem | The cost metric |
|---|---|---|---|
| Medicine this lab | 2.3% false-negative melanoma | diagnostic threshold lets cases slip; nobody labeled where it sat | years of life lost |
| Hydrology Lester’s Lab | flood-map discrepancy at the margin | algorithm flags auto-closed, human-review bar raised | property, lives at risk |
| Catalysis on the Docket | a mechanism claim vs. the evidence for it | where does “proven” sit, and did the validation reach it? | credibility, wasted research |
The catalysis exhibit is parked on ⏳ The Docket — the holding room, now built, where a case waits before it’s sorted into Sledgehammer, Caliper, or dismissed. Its first exhibit, The Knot That Waits, is already up. The catalysis paper may turn out to be the “did-it-right” contrast rather than another miss — it gets read before it graduates.