Whisper: One Model, Many Languages

OpenAI's Whisper (2022) is architecturally ordinary — a standard encoder-decoder transformer, the same kind the Transformers track covers end to end.

The extraordinary part is the data: 680,000 hours of audio scraped from the web with transcripts, however imperfect. Trained on that, a single checkpoint transcribes 99 languages, translates any of them into English, and outputs timestamps — tasks that used to each need a specialist system. Whisper's lesson echoes GPT's: scale the data and a plain transformer becomes a generalist.