TinyML, short for tiny machine learning, is the practice of running trained machine learning models directly on microcontrollers and other ultra-low-power hardware, typically with less than 1 MB of flash, a few hundred kilobytes of RAM and a power budget measured in milliwatts or less. Within Edge AI, it occupies the most constrained end of the spectrum: where phones and PCs run models with gigabytes of memory, TinyML devices must perform inference with resources roughly a thousand times smaller.
The goal is always-on intelligence at the sensor. A TinyML device listens, watches or measures continuously, interprets the signal locally and only wakes a larger processor or sends data to the cloud when something relevant happens. This makes artificial intelligence possible in battery-powered and energy-harvesting products where a radio link or a full application processor would be too costly in energy, latency or privacy.
How TinyML runs a model on a microcontroller
A TinyML system runs a pre-trained, heavily compressed model inside the firmware of a microcontroller unit (MCU). Training happens on a workstation or in the cloud; the device only performs inference, usually with 8-bit integer arithmetic because most MCUs lack a fast floating-point unit for neural workloads.
The typical pipeline has four stages. Raw sensor data is captured by an analog front end, converted into compact features such as a spectrogram of one second of audio, passed through a small neural network and turned into a decision. Because the model weights live in flash memory and the intermediate activations must fit in SRAM, the peak activation size often limits the design more than the parameter count.
Execution relies on a lightweight interpreter or compiled runtime. Interpreter-based runtimes such as TensorFlow Lite for Microcontrollers parse a serialized model and call optimized kernels, while ahead-of-time compilers generate plain C code for the specific network, removing interpreter overhead and saving memory. Vendor kernel libraries such as CMSIS-NN supply hand-optimized operators for Arm Cortex-M cores.
The table below summarizes the resource envelope that distinguishes TinyML hardware from other inference targets.
| Resource | Typical TinyML range | What it constrains |
|---|---|---|
| Flash memory | 128 KB to 2 MB | Model weights and firmware code |
| SRAM | 16 KB to 512 KB | Peak activations and input buffers |
| Clock speed | 32 MHz to 480 MHz | Inference latency |
| Active power | 1 mW to 100 mW | Battery life and duty cycle |
| Model size | 10 KB to 500 KB | Network depth and width |
Model compression techniques that make TinyML possible
TinyML depends on aggressive model compression, because even a small mobile network is too large for an MCU without it. Practitioners combine several techniques and measure the effect of each on accuracy, memory and latency.
Quantization is almost always applied first. Converting 32-bit floating-point weights and activations to INT8 cuts model size by about four times and enables integer-only kernels. Quantization-aware training recovers most of the accuracy lost by post-training quantization when a network is sensitive. Pruning, weight clustering and compact architectures designed through hardware-aware neural architecture search reduce the remaining footprint further.
The following table compares the main compression methods used for microcontroller targets.
| Technique | Typical saving | Main trade-off |
|---|---|---|
| INT8 quantization | About 4x smaller, 2-4x faster | Small accuracy loss, calibration needed |
| Structured pruning | 30-70% fewer operations | Requires fine-tuning to recover accuracy |
| Weight clustering | Better compression of stored weights | No speed gain without special kernels |
| Compact architectures | 10-100x fewer parameters than standard networks | Lower ceiling on task complexity |
| Binarized networks | Up to 32x smaller weights | Significant accuracy loss on hard tasks |
TinyML compared with other edge inference approaches
TinyML differs from general edge inference mainly in scale and in the operating model. A smartphone running a local language model has a neural processing unit, an operating system and gigabytes of memory; a TinyML device often has no operating system, a fixed firmware image and a single dedicated task.
The comparison below places TinyML among neighboring deployment patterns in the Edge AI landscape.
| Approach | Typical hardware | Model scale | Typical task |
|---|---|---|---|
| TinyML | Microcontroller, DSP, micro-NPU | Kilobytes | Keyword spotting, anomaly detection |
| Mobile edge inference | Phone or laptop SoC with NPU | Megabytes to gigabytes | Image enhancement, local LLM |
| Edge server inference | Gateway or GPU box on site | Gigabytes | Multi-camera video analytics |
| Cloud offloading | Remote data center | Unlimited in practice | Large generative models |
TinyML is often combined with these approaches in a cascade: the microcontroller filters the signal around the clock, and only confirmed events trigger a heavier model on a phone, a gateway or a cloud service. This pattern underlies AIoT products and many wake-word systems.
Typical TinyML applications and how to choose a target
TinyML suits problems where the input is a continuous sensor stream and the output is a short classification or alarm. Audio, vibration, motion and low-resolution vision dominate, because these signals compress well into small feature maps.
Choosing a target device starts from the latency, energy and accuracy requirements, not from the model. The decision table below summarizes common situations.
| Situation | Recommended setup | Reason |
|---|---|---|
| Wake word on a coin-cell device | Cortex-M4 or M33 with INT8 model | Sub-milliwatt always-on listening |
| Predictive maintenance on motors | MCU with accelerometer and small 1D CNN | Local vibration analysis avoids constant radio use |
| Person detection at 96×96 pixels | MCU with micro-NPU or Cortex-M55 | Vision needs vector acceleration |
| Gesture recognition in wearables | Low-power MCU with IMU features | Small time-series model, long battery life |
| Task needs open-ended language | Not TinyML; use phone or cloud | Model far exceeds MCU memory |
Benchmark suites such as MLPerf Tiny give comparable numbers for latency and energy per inference across hardware, which helps teams avoid relying on vendor claims measured under different conditions.
Limitations and common mistakes in TinyML projects
The central limitation of TinyML is model capacity. Networks with tens of thousands of parameters handle narrow tasks well but cannot generalize like large models, so dataset quality and domain match matter more than architecture tweaks. A keyword spotter trained on clean studio recordings often fails in a noisy factory.
- Ignoring peak RAM: a model that fits in flash can still fail because one layer’s activations exceed available SRAM.
- Mismatched preprocessing: feature extraction on the device must match training exactly, or accuracy collapses silently.
- Measuring only inference energy: sensor sampling, memory access and radio use often dominate the total power budget.
- Skipping update planning: without a secure over-the-air model update path, field errors cannot be corrected.
- Overlooking security: model files and firmware on unprotected MCUs can be extracted or altered.
On-device training remains limited on microcontrollers. Most TinyML products ship a frozen model, and personalization, when present, adjusts only a small final layer or a threshold.
Stripped to the essentials
TinyML puts a very small piece of artificial intelligence inside a cheap chip, the kind that runs a thermostat or a fitness band. The chip has little memory and almost no power to spare, so the model must be shrunk until it fits, much like packing for a long trip with only a small backpack.
Once it fits, the chip can listen, feel or watch on its own, all day, on a small battery. It does not need the internet to decide whether it heard a command or felt an unusual vibration. The key idea is simple: decide locally, send only what matters.
Where it is applied
A water utility installs battery-powered vibration sensors on pumps at remote pumping stations. Each sensor contains a Cortex-M4 microcontroller running a small 1D convolutional network of about 40 KB, quantized to INT8, which analyzes one second of accelerometer data every minute.
The model classifies the vibration pattern as normal operation, bearing wear or cavitation. Only when it detects an anomaly several times in a row does the device switch on its low-power radio and send a short alert with a compact summary of the signal. Raw data never leaves the sensor in normal operation.
Because the radio is idle most of the time, the sensors run for several years on two AA batteries. Maintenance crews receive targeted alerts days before failures, and unplanned pump outages fall noticeably compared with fixed inspection schedules.