Skip to content
Audio & Speech AI

Music & Audio Generation

Why Generating Audio Is Hard

8 min+80 XPStep 1/10

The resolution problem

Text generation is easy mode: a page of text is a few hundred tokens. Audio at 24 kHz is 24,000 values per second — generate it sample-by-sample and a ten-second clip becomes a quarter-million-step problem.

WaveNet did exactly this in 2016 and it worked — beautifully, at minutes per second of audio. Pure autoregression on raw waveforms was a dead end for production; every modern system attacks the resolution problem first.

insight

Per unit of meaning, audio is enormously redundant — neighboring samples are nearly identical. Almost every trick in audio generation is about not paying for that redundancy.