Speech Recognition & Synthesis
Voice Cloning & Its Ethics
8 min+75 XPStep 1/10
A voice as a vector
Classic TTS voices were baked in — one speaker per model. Modern systems condition the synthesizer on a speaker embedding: a vector distilled from a reference clip that captures that person's timbre.
The consequence is startling: zero-shot voice cloning — a few seconds of someone speaking is enough to make them say anything. The legitimate uses are real: giving ALS patients their own voice back, dubbing films in the actor's actual voice, personalized voices for people who can't speak.
insight
A speaker embedding is a voice fingerprint — like a word embedding, but for 'who' instead of 'what'. Swap the vector, swap the voice.