Statistical Thinking
Simpson's Paradox: When Aggregates Lie
Winning overall, losing everywhere
Treatment A cures 78% of patients; Treatment B cures 83%. B is better — obviously?
Split by kidney-stone size and the story flips:
- Small stones: A 93%, B 87% — A wins
- Large stones: A 73%, B 69% — A wins again
How does A win both groups but lose overall? Doctors assigned treatments differently: A took more of the hard cases (large stones), B took the easy ones. The aggregate compares different patient pools — the 'win' was a mirage.
This is Simpson's paradox: a trend in aggregated data that reverses inside every subgroup. Not a math trick — a warning that how you slice data changes the answer.
warning
UC Berkeley was once accused of gender bias: men were admitted at higher rates overall. Split by department, women were admitted at equal or higher rates nearly everywhere — women had applied to harder departments. The aggregate blamed the wrong culprit.