Keyword spotting

Also known as: KWS · wake word detection

On this page 8

Keyword spotting (KWS) is the task of detecting a small, predefined set of spoken words or phrases in a continuous audio stream, usually with a compact neural network that runs locally and continuously on the device. In Edge AI, keyword spotting is the classic always-on workload: a phone, smart speaker, earbud or appliance listens for a wake word such as a product name, and only after a confident detection does it start heavier speech recognition or send audio anywhere.

Keyword spotting is also called wake word detection when the vocabulary is a single trigger phrase. Unlike full automatic speech recognition, it does not transcribe speech; it answers one narrow question many times per second, namely whether the target phrase was just spoken, under strict limits on memory, power and latency.

How keyword spotting processes audio

A keyword spotting system converts raw audio into compact acoustic features, scores short windows with a small classifier and applies a decision rule over time. The microphone signal is typically sampled at 16 kHz, split into frames of about 25-40 ms with a 10-20 ms hop, and turned into log-mel filterbank energies or MFCCs, which summarize the spectrum in roughly 10 to 40 values per frame.

The classifier sees a sliding window of about one second of features and outputs a probability for each keyword plus classes for silence and unknown speech. Because the window advances every few tens of milliseconds, the device runs dozens of inferences per second, so per-inference cost dominates the energy budget. A posterior smoothing and threshold stage then averages scores across consecutive windows and fires only when the confidence stays high, which suppresses single-frame spikes.

The table below summarizes the stages of a typical on-device pipeline and the usual design values.

StageTypical settingPurpose
Audio capture16 kHz, 16-bit monoCovers the speech band at low cost
Framing30 ms window, 10-20 ms hopProduces short quasi-stationary segments
Feature extraction10-40 log-mel bands or MFCCsCompresses the spectrum for the model
Neural classifier20 KB to 1 MB of weightsScores keyword, silence and unknown classes
Posterior handlingSmoothing over 0.5-1 sTurns frame scores into stable decisions
Stages of a typical keyword spotting pipeline

Model architectures used for keyword spotting

Keyword spotting models are chosen for accuracy per operation, because the same network may run millions of times a day on a microcontroller or a low-power audio DSP. Early systems used hidden Markov models with keyword and filler paths; modern systems use small convolutional or recurrent networks trained end to end on labeled clips.

The following comparison shows the main architecture families and where each fits on constrained hardware.

ArchitectureTypical sizeStrengthWeakness
Fully connected DNN50-500 KBVery simple to deployIgnores local time-frequency structure
Standard CNN100-700 KBGood accuracy on spectrogramsHigh operation count per window
Depthwise separable CNN20-200 KBStrong accuracy for its costNeeds careful quantization
Temporal convolution network50-300 KBLong receptive field with few layersSensitive to input length choices
Small recurrent or attention model100 KB-1 MBModels longer phrases wellStreaming state adds complexity
Common keyword spotting model families

Depthwise separable designs borrowed from MobileNet are the most common choice on microcontrollers, often combined with INT8 quantization so that weights and activations fit in tens of kilobytes of SRAM. Benchmarks such as MLPerf Tiny include a keyword spotting task built on the Google Speech Commands dataset, which makes it a standard reference workload for TinyML hardware.

Keyword spotting differs from neighboring speech tasks mainly in vocabulary size, compute cost and duty cycle. It is deliberately narrow, which is exactly what allows it to stay on all the time, while broader tasks run only after it has fired.

TaskOutputTypical locationDuty cycle
Keyword spottingDetected phrase or nothingDevice, often a DSP or MCUAlways on
Voice activity detectionSpeech present or absentDevice, analog or digital front endAlways on
Speaker verificationAccept or reject a voiceDevice or cloudAfter a trigger
Automatic speech recognitionFull transcriptCloud, local LLM stack or phoneAfter a trigger
Keyword spotting versus related audio tasks

In practice these tasks form a cascade: voice activity detection gates keyword spotting, keyword spotting gates verification and recognition, and each stage wakes a more expensive one. This staging is a textbook case of always-on sensing and hybrid inference, since the final command may be processed locally or offloaded to the cloud.

Accuracy metrics and threshold tuning

Keyword spotting quality is measured by two competing error rates: the false rejection rate (FRR) and the false accepts per hour (FA/h). Plain classification accuracy on balanced test clips is misleading, because a deployed device hears mostly non-keyword audio and even a tiny per-window error rate produces many false triggers over a day.

Moving the detection threshold trades one error for the other, so vendors report a curve and pick an operating point. Consumer wake word products commonly aim for well under one false accept per hour in household noise while keeping false rejections in the low single-digit percent range for clear speech; the exact targets vary by product and are rarely published.

Threshold settingFalse rejectionsFalse acceptsTypical use
Low thresholdFewManyHands-busy settings with retries
Balanced thresholdModerateModerateGeneral consumer devices
High thresholdManyVery fewPrivacy-sensitive rooms and media playback
Two-stage thresholdFewVery fewOn-device check plus server verification
Effect of threshold choices on keyword spotting behavior

Deploying keyword spotting on edge hardware

A successful deployment fits the model, feature extractor and audio buffers into the memory and power envelope of the always-on domain. Typical targets are a Cortex-M class microcontroller or a dedicated audio DSP drawing on the order of a milliwatt or less, while the main application processor stays asleep.

  • Quantize the model with post-training quantization or quantization-aware training, and validate FRR and FA/h after conversion rather than only clip accuracy.
  • Match training features to device features exactly, including window, filterbank and scaling, because a small mismatch silently degrades accuracy.
  • Train with noise and reverberation augmentation and collect negative audio such as television, music and similar-sounding words.
  • Use a cascade in which a tiny first stage runs constantly and a larger second stage confirms candidates.
  • Plan for OTA model updates, since new accents, rooms and false-trigger patterns appear only after launch.

Limitations and common mistakes

The main limitation of keyword spotting is that it handles only a fixed, small vocabulary; adding a new phrase usually means collecting data and retraining, although few-shot and user-defined keyword methods based on embeddings are narrowing that gap. Performance also drops for far-field speech, overlapping talkers, strong accents and children’s voices when these are underrepresented in training data.

A frequent mistake is evaluating on clean, isolated clips and then discovering a high false trigger rate in a living room with a television. Another is assuming that local detection guarantees privacy: keyword spotting keeps idle audio on the device, but false accepts can still send unintended audio to later stages, so the threshold is also a privacy control.

In everyday language

Keyword spotting is the small listening skill that lets a device notice when you say its name. It does not understand sentences. It only checks, over and over, whether one particular word or phrase was just spoken.

Think of a person in a noisy cafe who ignores the chatter but turns around the moment someone says their name. Keyword spotting gives a gadget that same reflex. Because the job is so narrow, it can run on a tiny, low-power chip inside the device all day, and the bigger, more capable systems only switch on after the name is heard. The main takeaway is that the device listens for one word locally before anything else happens.

Seen in the real world

A team building battery-powered wireless earbuds wants hands-free voice commands without draining the battery. They train a depthwise separable CNN with about 60 KB of INT8 weights on 40 log-mel features per 20 ms frame, using thousands of recordings of the wake phrase mixed with street, cafe and wind noise, plus many hours of negative speech and music.

The model runs on the earbud’s always-on audio DSP, scoring a one-second window every 20 ms. Field tests across accents and noisy commutes show too many false triggers during podcasts, so the team raises the threshold and adds a second, larger verification model on the paired phone. The result is roughly one false wake per several hours of listening, few missed commands in normal use, and a power cost small enough that daily battery life barely changes.