Self-Supervised Speech Features

Transcribed speech is expensive; raw audio is free. Self-supervised learning exploits that asymmetry — learn speech structure from unlabeled audio first, then fine-tune on tiny labeled sets.

Two recipes define the era:

• wav2vec 2.0 — mask spans of the encoded waveform, then contrastively identify the true quantized representation of masked positions against distractors. The model learns 'what sound belongs here' — phonetic structure without a single transcript.
• HuBERT — cluster features into pseudo-labels (k-means units), mask spans, predict the hidden units. BERT's recipe on discovered speech units — iterating clustering → training → re-clustering refines the units themselves.

Result: models fine-tuned on minutes of labels matched earlier systems trained on hundreds of hours.