Speech & Audio · entry 03/05

Text to speech

Speech synthesis crossed the line from robotic to indistinguishable — the engineering is now about control, latency, and the awkward fact that ten seconds of your voice is enough to clone it.

From splices to samples

For decades, synthesis meant concatenation: record a voice actor for weeks, chop the audio into units, splice them back per sentence — intelligible, seam-ridden, and forever one voice. Neural synthesis replaced the splicing with generation: a model reads text, predicts a spectrogram with timing and pitch, and a vocoder renders it to waveform. Current systems — many built on diffusion or flow models, some as pure token-predicting transformers over discretized audio — produce speech that listeners rate at or near human, with breaths, hesitations, and emphasis in the right places.

Control is the product now

Once quality saturates, the differentiators are direction and delivery. Direction: prosody controls ("say it warmly," faster, with this emphasis), pronunciation overrides for names and jargon, and consistent character across a long audiobook rather than sentence-by-sentence beauty. Delivery: latency. A voice agent needs the first audible syllable in a few hundred milliseconds, which favors streaming synthesizers that start speaking before the sentence is finished — architecture, not model size, decides whether a conversation feels alive.

Cloning changed the stakes

The same architectures do voice cloning from a short sample — seconds of reference audio yield a workable copy of timbre and accent. Legitimate uses are real: dubbing an actor into eleven languages, restoring speech to people losing theirs, one narrator across your whole catalog. So is the abuse: the grandparent scam now calls in the grandchild's actual voice, and "voiceprint" phone authentication aged into a vulnerability overnight. Serious platforms verify consent and watermark output, and the provenance problem applies to audio with extra teeth, because a phone call carries no pixels to inspect.

Failure mode

Shipping the demo voice. Synthesis quality is judged on three polished sentences, then production feeds it edge cases: acronyms, part numbers, addresses, currency, a bilingual name in an English sentence. Text-to-speech fails on exactly the text you didn't normalize — "$1,024" read as "dollar one zero two four" in a production IVR is a classic for a reason. Budget for a text normalization layer, a pronunciation dictionary you own, and a listening pass over your real content — not the vendor's samples — before a voice meets customers.