Speech technology

Automatic speech recognition

Automatic speech recognition (ASR), also called speech to text, is the technology that turns spoken audio into written text, and it is the first real step in almost every voice agent.

Barge-in

Barge-in is a voice agent stopping its own speech the instant a person starts talking over it, which is what makes the exchange feel like a conversation rather than a recording you have to sit through.

Custom vocabulary and keyword boosting

Custom vocabulary tells a speech recognition system which unusual words to expect, such as your product names and industry terms, and keyword boosting raises the chance the recognizer picks those words when the audio is ambiguous.

Diarization

Diarization is the process of working out who spoke when in a recording, separating a conversation into anonymous speaker turns without needing to know the identity of any speaker.

Echo cancellation

Echo cancellation stops a device from hearing its own voice, so that a voice agent speaking through a laptop speaker does not pick up its own audio through the microphone and mistake it for the visitor interrupting.

Endpointing

Endpointing is how a voice agent decides that the person has finished speaking and its own turn has begun, normally by timing the silence against a threshold set between 150 and 300 milliseconds.

Intonation

Intonation is the pattern of pitch across a phrase or sentence, the part of prosody that tells a listener whether they are hearing a question, a statement, or something not yet finished.

Neural TTS

Neural text to speech (neural TTS) uses neural networks to generate speech directly, replacing the older concatenative and parametric methods, and it is the reason synthetic voices stopped sounding robotic.

Noise suppression

Noise suppression removes background sound from captured audio so that speech is clearer, and it is what lets a voice agent work for someone in a cafe rather than only in a quiet room.

Phoneme

A phoneme is the smallest unit of sound that changes the meaning of a word in a language, so that "pat" and "bat" differ by exactly one, and it is the level at which a voice agent gets a name right or wrong.

Prosody

Prosody is the rhythm, stress, pace and pitch that run across a whole sentence rather than within a single sound, and it is most of what makes synthesized speech sound alive rather than flat.

Realtime API

A realtime API is an interface built for live, low-latency, bidirectional audio streaming rather than request-and-response calls, and it is what makes conversational voice possible without stitching separate recognition, reasoning and synthesis services together.

Sample rate

Sample rate is how many times per second audio is measured when it is digitized, and it sets a hard ceiling on which frequencies survive to reach the recognizer.

Speaker identification

Speaker identification works out who is speaking by matching a voice against a database of known voiceprints, answering who this is, which is a more legally sensitive question than who spoke when.

Speech-to-speech models

A speech-to-speech model is a single model that consumes audio and emits audio directly, with no transcription step in the middle, which buys latency and preserved tone at the cost of the text record a cascaded pipeline produces as a by-product.

SSML

SSML, the Speech Synthesis Markup Language, is a W3C standard for controlling how synthesized speech sounds by wrapping text in tags that set pauses, emphasis, pronunciation, speaking rate, and how numbers and codes are read.

Streaming against batch transcription

Streaming transcription emits words as they are spoken and revises them as more audio arrives, while batch transcription waits for the whole recording and returns a more accurate finished transcript.

Text to speech

Text to speech (TTS), also called speech synthesis, turns written text into spoken audio, and it is how a voice agent gets its voice.

Turn detection

Turn detection is working out whose turn it is to speak, and semantic turn detection does it by judging whether the sentence sounds finished rather than by timing the silence after it.

Voice activity detection

Voice activity detection (VAD) is the component at the very front of a voice agent that decides, frame by frame, whether the incoming audio contains speech or only background noise.

Voice cloning

Voice cloning builds a synthetic copy of one named person's voice out of a recording of them talking, and that copy can then be made to say things the person never said.

Wake word and hotword detection

A wake word is the phrase that activates a listening device, and hotword detection is the always-running, low-power on-device process that recognizes that one phrase and rejects everything else.