Skip to content

Probability for ML

Distributions, Expectation & Variance

8 min+75 XPStep 1/11

Uncertainty has a shape

A distribution answers: over all possible outcomes, how is belief spread? That spread has a shape, and the shape is everything:

  • Uniform: every outcome equally likely — maximum shrugging, like a fair die
  • Normal (bell curve): mass piles at the center with rare extremes — heights, measurement noise, and anything that's a sum of many small effects
  • Long-tailed: rare events aren't that rare — wealth, word frequencies, city sizes. 'Once-in-a-century' events arrive yearly

ML depends on distributions everywhere: your training data is one, and the model quietly learns its shape.

warning

A model only knows the distribution it trained on. Deploy it where the world's distribution differs — new slang, new devices, new patients — and it fails. That's distribution shift, a core nemesis of applied ML.