Beyond Images
Audio Meets Language
Three audio problems wearing one name
'Audio understanding' is at least three different tasks: speech (transcribe the words), sound (classify the event — glass break, alarm, dog bark), and music (structure, genre, mood). They share a waveform and nothing else — different features, different labels, different failure modes.
The standard featurization bridges vision and audio: the spectrogram — frequency × time as a 2D image. Once audio is an image, the vision toolkit applies: ViT-style audio transformers, patch embeddings over time-frequency tiles. It's why audio models bootstrapped so fast — they inherited the image stack.
insight
The spectrogram is the great adapter: it converts a 1D waveform into the modality your vision stack already eats. Most 'audio transformers' are image transformers on a preprocessing trick.