Text-to-speech (TTS), also called speech synthesis, is the technology that converts written text into spoken audio. A TTS system analyzes the text, decides how it should be pronounced and phrased, and generates a waveform that sounds like a human voice reading it aloud.
In speech and language AI within artificial intelligence, text-to-speech is the output side of voice interfaces, complementing speech-to-text on the input side. Neural TTS has moved the field from robotic, concatenated sounds to voices that listeners often find hard to distinguish from recordings, which also raises new questions around voice cloning and consent.
How a text-to-speech system works
A classic neural TTS system has three main stages. A text front end normalizes and analyzes the input, an acoustic model predicts an intermediate representation such as a mel spectrogram, and a vocoder turns that representation into an audible waveform. Some modern systems merge these stages into one end-to-end model.
| Stage | Task | Example operations |
|---|---|---|
| Text normalization | Expand non-standard text | “Dr.” to doctor, “3:15” to three fifteen |
| Grapheme-to-phoneme | Convert spelling to sounds | Pronunciation lexicon plus learned model |
| Prosody prediction | Plan rhythm, stress, pitch | Duration and pitch contours |
| Acoustic model | Predict spectral frames | Transformer or flow-based networks |
| Vocoder | Generate the waveform | GAN-based or neural codec decoders |
Prosody, the melody and timing of speech, is the hardest part to get right. The same sentence can be a question, a statement or an emphasis on one word, and the model must infer this from punctuation, context and any explicit markup the application provides.
Generations of speech synthesis technology
Text-to-speech has passed through several distinct technical generations, and each still appears in some products today.
| Approach | How it produces speech | Strength | Weakness |
|---|---|---|---|
| Formant synthesis | Rule-based acoustic signal models | Tiny footprint, very fast | Clearly robotic sound |
| Concatenative synthesis | Joins recorded speech units | Natural units | Audible joins, rigid style |
| Statistical parametric | Models predict vocoder parameters | Flexible, compact | Muffled, buzzy quality |
| Neural two-stage | Acoustic model plus neural vocoder | High naturalness | More compute |
| Codec language models | Predict discrete audio tokens | Zero-shot voices, expressive style | Occasional skipped or repeated words |
The newest systems treat audio as sequences of discrete tokens from a neural audio codec and generate them with transformer language models. This enables zero-shot voice adaptation from a few seconds of reference audio, but it can also produce occasional hallucinated sounds that older pipelines did not.
Quality measures and technical parameters
TTS quality is judged on naturalness, intelligibility and speed. Naturalness is usually measured through mean opinion score (MOS) listening tests on a scale from 1 to 5, while intelligibility is often estimated by transcribing the output with speech recognition and computing the word error rate.
| Parameter | Typical values | What it affects |
|---|---|---|
| Mean opinion score | 1 to 5, top systems above 4 | Perceived naturalness |
| Sample rate | 16, 22.05, 24 or 48 kHz | Audio bandwidth and clarity |
| Real-time factor | Well below 1.0 for live use | Generation speed versus audio length |
| Time to first audio | Roughly 100 to 500 ms for conversation | Perceived responsiveness |
| Intelligibility WER | Lower is better | Correct words heard |
| Speaker similarity | Embedding cosine score | Match to target voice |
For conversational assistants, latency matters as much as naturalness. Streaming synthesis, which starts playing the first phrase while the rest is still being generated, is often required to keep pauses short enough to feel natural.
Text-to-speech compared with neighboring speech technologies
Text-to-speech is one part of a larger speech and language stack. The table clarifies how it differs from related Terms in the same field.
| Technology | Input | Output | Relation |
|---|---|---|---|
| Text-to-speech | Text | Speech audio | Speech generation |
| Speech-to-text | Speech audio | Text | Inverse direction |
| Voice cloning | Reference audio plus text | Speech in a target voice | TTS specialized to one speaker |
| Voice conversion | Speech audio | Same words, different voice | Changes timbre, not content |
| Speech translation | Speech in one language | Speech in another | Combines recognition, translation and TTS |
Applications control pronunciation and style through SSML (Speech Synthesis Markup Language), a W3C standard for pauses, emphasis, rate, pitch and phonetic spellings. Custom pronunciation lexicons handle brand names, abbreviations and domain terms that general models mispronounce.
Limitations, risks and practical pitfalls
Even high-quality TTS has predictable weak points, and some risks extend beyond audio quality into security and ethics.
- Normalization errors: dates, currencies, units and abbreviations are read wrongly without locale-aware rules.
- Homographs: words such as “read” or “lead” need context to choose the correct pronunciation.
- Flat prosody in long texts: paragraphs can sound monotonous without sentence-level style control.
- Code-switching: names and phrases from another language are often pronounced with the wrong phonetics.
- Voice misuse: synthetic voices enable impersonation fraud, so consent, watermarking and disclosure are essential.
- Accessibility regressions: screen reader users often prefer fast, predictable voices over expressive ones.
A practical safeguard is to test with real production text, including tables, numbers and names, rather than with clean sample sentences, and to keep a review loop for mispronounced terms.
The idea in brief
Text-to-speech is the technology that reads written words out loud. You give it a sentence, and it produces a voice saying that sentence, much like a narrator reading a book.
Modern systems learned to speak by studying many hours of recorded human speech, so they sound far more natural than older robotic voices. They power navigation directions, audiobooks, screen readers for people with visual impairments and voice assistants.
A practical scenario
A public transport operator wants spoken announcements in every train for delays, platform changes and connections, in two languages. Recording every possible sentence with voice actors is impossible because the messages change in real time.
The operator uses neural text-to-speech with a licensed voice, a custom pronunciation lexicon for 600 station names, and SSML rules that slow the rate slightly and insert pauses before platform numbers. Announcements are generated at 24 kHz and played through the existing speaker system.
Passenger feedback shows that station names are now pronounced consistently, and new messages can be issued within seconds of a disruption. The team reviews any newly added station names before they go live.