ASR: The Whisper Recipe

Whisper's 2022 insight wasn't an architecture — a standard encoder-decoder transformer on mel spectrograms. It was the data philosophy: 680,000 hours of audio-transcript pairs harvested from the internet, lightly filtered, never hand-labeled. Where previous ASR trained on thousands of clean curated hours, Whisper trained on everything — accents, noise, overlapping chaos included.

Weak supervision trades label precision for coverage: transcripts are imperfect (punctuation quirks, caption conventions, occasional wrongness) but the sheer diversity teaches robustness that clean corpora never reach. The result was the first ASR that just worked in the wild.