The Lab · instrument 14/16 · interactive

Move the fairness threshold

Two groups scored by the identical process, differing only in base rate — enforce one fairness criterion and watch another break. The impossibility result, as a slider.

What's real here

Both groups are scored by exactly the same process; the only difference is how many positives each contains. That matters because it removes the usual explanation — no biased model, no biased labels, no missing feature — and leaves the arithmetic exposed. Every number in the table is computed from the points on screen: selection rate, true-positive rate, false-positive rate, and precision, per group, at whatever thresholds the policy picks.

Where to push

Start with one threshold for everyone — the intuitive definition of fair — and read the rows carefully. The true- and false-positive rates come out equal: with identical score distributions, one shared threshold buys equal opportunity for free. What differs is selection rate (about 51% against 34%) and precision (83% against 63%), and that is the whole conflict. Now switch to "equalize selection rate": B's threshold drops, the same fraction gets accepted, and B's precision falls from 63% to 47% against A's unchanged 83% — you are admitting more of B's low scorers to hit the quota. Switch to "equalize true-positive rate" and nothing moves, because that checkmark was already lit. Then try "equalize precision," the policy that swings hardest: B's acceptance rate collapses to around 19% against A's 51%. That pair — equal precision bought with unequal acceptance — is the exact shape of the recidivism dispute.

What it means outside the toy

No setting of the policy or threshold controls lights all three at once — though the base-rate slider is a control too, and dragging group B's base rate all the way up to group A's does clear every row, because the incompatibility is a function of that gap and nothing else. One caveat on the third row: what this tool equalizes is precision at a threshold — predictive parity. True calibration is a property of the score itself, and here it is broken by construction; no threshold repairs it. This is the shape of the argument in every real dispute about algorithmic fairness, including the famous recidivism one: two sides each citing a real, satisfied criterion and each correctly accusing the other of unfairness under a different one. The lesson isn't nihilism — it's that "make it fair" is not a specification. Someone has to choose which error the system may make more often, in the open, with the people who bear it — a choice you write down, not a metric you optimize. Note too that the move this tool makes so casually — a different cutoff per group — is prohibited outright in some regulated domains, US employment testing among them, which is why real remedies more often land on the proxy, the data, or the decision the score feeds.

Failure mode

Concluding that since nothing is perfectly fair, nothing matters. The impossibility is about satisfying all three criteria simultaneously; it says nothing about the enormous, measurable difference between a system whose disparities were chosen and documented and one whose disparities nobody ever looked at. Move the base rate slider back to a heavy imbalance, set one threshold for everyone, and read the gap: that is the default your product ships with when nobody decides. Deciding is still available.