Keyword spotting (KWS) is the task of detecting a small, predefined set of spoken words or phrases in a continuous audio stream, usually with a compact neural network that runs locally and continuously on the device. In Edge AI, keyword spotting is the classic always-on workload: a phone, smart speaker, earbud or appliance listens for a wake word such as a product name, and only after a confident detection does it start heavier speech recognition or send audio anywhere.
Keyword spotting is also called wake word detection when the vocabulary is a single trigger phrase. Unlike full automatic speech recognition, it does not transcribe speech; it answers one narrow question many times per second, namely whether the target phrase was just spoken, under strict limits on memory, power and latency.
How keyword spotting processes audio
A keyword spotting system converts raw audio into compact acoustic features, scores short windows with a small classifier and applies a decision rule over time. The microphone signal is typically sampled at 16 kHz, split into frames of about 25-40 ms with a 10-20 ms hop, and turned into log-mel filterbank energies or MFCCs, which summarize the spectrum in roughly 10 to 40 values per frame.
The classifier sees a sliding window of about one second of features and outputs a probability for each keyword plus classes for silence and unknown speech. Because the window advances every few tens of milliseconds, the device runs dozens of inferences per second, so per-inference cost dominates the energy budget. A posterior smoothing and threshold stage then averages scores across consecutive windows and fires only when the confidence stays high, which suppresses single-frame spikes.
The table below summarizes the stages of a typical on-device pipeline and the usual design values.
| Stage | Typical setting | Purpose |
|---|---|---|
| Audio capture | 16 kHz, 16-bit mono | Covers the speech band at low cost |
| Framing | 30 ms window, 10-20 ms hop | Produces short quasi-stationary segments |
| Feature extraction | 10-40 log-mel bands or MFCCs | Compresses the spectrum for the model |
| Neural classifier | 20 KB to 1 MB of weights | Scores keyword, silence and unknown classes |
| Posterior handling | Smoothing over 0.5-1 s | Turns frame scores into stable decisions |
Model architectures used for keyword spotting
Keyword spotting models are chosen for accuracy per operation, because the same network may run millions of times a day on a microcontroller or a low-power audio DSP. Early systems used hidden Markov models with keyword and filler paths; modern systems use small convolutional or recurrent networks trained end to end on labeled clips.
The following comparison shows the main architecture families and where each fits on constrained hardware.
| Architecture | Typical size | Strength | Weakness |
|---|---|---|---|
| Fully connected DNN | 50-500 KB | Very simple to deploy | Ignores local time-frequency structure |
| Standard CNN | 100-700 KB | Good accuracy on spectrograms | High operation count per window |
| Depthwise separable CNN | 20-200 KB | Strong accuracy for its cost | Needs careful quantization |
| Temporal convolution network | 50-300 KB | Long receptive field with few layers | Sensitive to input length choices |
| Small recurrent or attention model | 100 KB-1 MB | Models longer phrases well | Streaming state adds complexity |
Depthwise separable designs borrowed from MobileNet are the most common choice on microcontrollers, often combined with INT8 quantization so that weights and activations fit in tens of kilobytes of SRAM. Benchmarks such as MLPerf Tiny include a keyword spotting task built on the Google Speech Commands dataset, which makes it a standard reference workload for TinyML hardware.
Keyword spotting compared with related speech tasks
Keyword spotting differs from neighboring speech tasks mainly in vocabulary size, compute cost and duty cycle. It is deliberately narrow, which is exactly what allows it to stay on all the time, while broader tasks run only after it has fired.
| Task | Output | Typical location | Duty cycle |
|---|---|---|---|
| Keyword spotting | Detected phrase or nothing | Device, often a DSP or MCU | Always on |
| Voice activity detection | Speech present or absent | Device, analog or digital front end | Always on |
| Speaker verification | Accept or reject a voice | Device or cloud | After a trigger |
| Automatic speech recognition | Full transcript | Cloud, local LLM stack or phone | After a trigger |
In practice these tasks form a cascade: voice activity detection gates keyword spotting, keyword spotting gates verification and recognition, and each stage wakes a more expensive one. This staging is a textbook case of always-on sensing and hybrid inference, since the final command may be processed locally or offloaded to the cloud.
Accuracy metrics and threshold tuning
Keyword spotting quality is measured by two competing error rates: the false rejection rate (FRR) and the false accepts per hour (FA/h). Plain classification accuracy on balanced test clips is misleading, because a deployed device hears mostly non-keyword audio and even a tiny per-window error rate produces many false triggers over a day.
Moving the detection threshold trades one error for the other, so vendors report a curve and pick an operating point. Consumer wake word products commonly aim for well under one false accept per hour in household noise while keeping false rejections in the low single-digit percent range for clear speech; the exact targets vary by product and are rarely published.
| Threshold setting | False rejections | False accepts | Typical use |
|---|---|---|---|
| Low threshold | Few | Many | Hands-busy settings with retries |
| Balanced threshold | Moderate | Moderate | General consumer devices |
| High threshold | Many | Very few | Privacy-sensitive rooms and media playback |
| Two-stage threshold | Few | Very few | On-device check plus server verification |
Deploying keyword spotting on edge hardware
A successful deployment fits the model, feature extractor and audio buffers into the memory and power envelope of the always-on domain. Typical targets are a Cortex-M class microcontroller or a dedicated audio DSP drawing on the order of a milliwatt or less, while the main application processor stays asleep.
- Quantize the model with post-training quantization or quantization-aware training, and validate FRR and FA/h after conversion rather than only clip accuracy.
- Match training features to device features exactly, including window, filterbank and scaling, because a small mismatch silently degrades accuracy.
- Train with noise and reverberation augmentation and collect negative audio such as television, music and similar-sounding words.
- Use a cascade in which a tiny first stage runs constantly and a larger second stage confirms candidates.
- Plan for OTA model updates, since new accents, rooms and false-trigger patterns appear only after launch.
Limitations and common mistakes
The main limitation of keyword spotting is that it handles only a fixed, small vocabulary; adding a new phrase usually means collecting data and retraining, although few-shot and user-defined keyword methods based on embeddings are narrowing that gap. Performance also drops for far-field speech, overlapping talkers, strong accents and children’s voices when these are underrepresented in training data.
A frequent mistake is evaluating on clean, isolated clips and then discovering a high false trigger rate in a living room with a television. Another is assuming that local detection guarantees privacy: keyword spotting keeps idle audio on the device, but false accepts can still send unintended audio to later stages, so the threshold is also a privacy control.
In everyday language
Keyword spotting is the small listening skill that lets a device notice when you say its name. It does not understand sentences. It only checks, over and over, whether one particular word or phrase was just spoken.
Think of a person in a noisy cafe who ignores the chatter but turns around the moment someone says their name. Keyword spotting gives a gadget that same reflex. Because the job is so narrow, it can run on a tiny, low-power chip inside the device all day, and the bigger, more capable systems only switch on after the name is heard. The main takeaway is that the device listens for one word locally before anything else happens.
Seen in the real world
A team building battery-powered wireless earbuds wants hands-free voice commands without draining the battery. They train a depthwise separable CNN with about 60 KB of INT8 weights on 40 log-mel features per 20 ms frame, using thousands of recordings of the wake phrase mixed with street, cafe and wind noise, plus many hours of negative speech and music.
The model runs on the earbud’s always-on audio DSP, scoring a one-second window every 20 ms. Field tests across accents and noisy commutes show too many false triggers during podcasts, so the team raises the threshold and adds a second, larger verification model on the paired phone. The result is roughly one false wake per several hours of listening, few missed commands in normal use, and a power cost small enough that daily battery life barely changes.