← PROJECTSfalse-positive / story
SYSTEM: False Positive Simulator
LIVEWhy I Built This
Built this after watching an A/B test "win" on a metric that had nothing to do with the treatment. Run 20 comparisons, pick the one that hits p<0.05. Congratulations — you just proved nothing, statistically speaking. This tool makes that embarrassment interactive.
Background
In any data-heavy team, the pressure to show results is constant. A/B tests get run not to discover truth but to generate a green checkmark. Run enough tests, and one will hit p<0.05 by pure chance — and that one gets shipped. This is the multiple comparisons problem, and it is endemic.
The simulator makes this visceral: two identical distributions, no real effect. Watch the false positives accumulate. Watch your "significant" results dissolve when the alpha level actually matters.
Tech Stack
Next.jsTypeScriptRechartsStatistical Simulation