The Sample Lies

To learn about a whole population — every voter, every customer, every widget off the line — you measure a sample and hope it generalizes. The math works beautifully if the sample is random and representative.

When it isn't:

• 1936, Literary Digest: polled 2.4 million readers, predicted a Landon landslide — Roosevelt won 46 of 48 states. The sample was magazine subscribers: wealthier, more Republican
• Phone surveys that only reach people who answer unknown numbers
• Survivorship bias: studying planes that returned from battle to decide where to add armor — the holes you see are the ones planes survived

Sample size can't fix a biased method. A bigger biased sample is just a more precisely wrong answer.