TensorFlow Lite Micro (TFLM) is an open-source C++ inference runtime that executes trained neural networks on microcontrollers and digital signal processors with as little as tens of kilobytes of RAM and no operating system. Among AI frameworks & tools, it is the reference option for TinyML, the practice of running machine learning directly on battery-powered embedded devices.
TFLM was created by Google as the embedded branch of TensorFlow Lite, and after the 2024 rebranding of TensorFlow Lite as LiteRT it is also called LiteRT for Microcontrollers. It runs only inference, not training: models are trained elsewhere, converted to a compact file and compiled into the device firmware.
How TensorFlow Lite Micro runs a model
TensorFlow Lite Micro runs a model by interpreting a FlatBuffer model file that is stored in flash memory as a constant byte array. At startup, the application creates an interpreter, gives it the model, an operator resolver and a fixed block of RAM called the tensor arena, and then calls one function per inference.
The defining design choice is no dynamic memory allocation during operation. The interpreter plans all activation buffers inside the tensor arena once, during initialization, so memory use is deterministic and known at build time. This avoids heap fragmentation, which is a serious reliability risk on devices that must run for years without a restart.
The runtime pipeline has a few clearly separated components.
| Component | Role | Developer decision |
|---|---|---|
| Model array | FlatBuffer graph and weights in flash | Model size and quantization |
| Op resolver | Registers the kernels the model needs | Include only required operators |
| Tensor arena | Static RAM for activations and scratch data | Arena size, found by measurement |
| Interpreter | Plans memory and invokes kernels in order | Initialization and error handling |
| Optimized kernels | Hardware-specific operator code | Choice of vendor kernel library |
Hardware targets and resource budgets
TFLM targets devices whose resources are several orders of magnitude smaller than a phone. The core runtime fits in roughly 16 to 30 kilobytes of code on an Arm Cortex-M processor, and the remaining flash and RAM go to the model and the application.
| Device class | Typical memory | Realistic workloads |
|---|---|---|
| Arm Cortex-M0+ or M4 | 32 to 256 KB RAM, up to 1 MB flash | Keyword spotting, gesture and vibration detection |
| Arm Cortex-M7 or M55 | 512 KB to several MB RAM | Low-resolution vision, richer audio models |
| ESP32-S3 | 512 KB SRAM plus optional PSRAM | Wake words, simple person detection |
| DSP cores such as Xtensa | Hundreds of KB local memory | Always-on audio front ends |
| Microcontrollers with an NPU | Varies by vendor | Accelerated convolutional models |
Performance depends heavily on optimized kernel libraries. Arm’s CMSIS-NN, Espressif’s ESP-NN and Cadence’s Xtensa kernels replace the portable reference kernels and often make int8 convolutions several times faster, so the same model can be viable on one chip and too slow on another.
From trained model to firmware
Deploying a model with TensorFlow Lite Micro follows a fixed conversion workflow. The network is trained in TensorFlow or Keras, converted with the TensorFlow Lite converter, almost always with full integer quantization, and then turned into a C source array that is compiled into the firmware.
- Train a small model with an architecture suited to microcontrollers, such as a depthwise separable CNN.
- Quantize to int8 using a representative dataset to calibrate activation ranges.
- Convert to a .tflite file and check that every operator is supported by TFLM.
- Embed the model as a constant array, for example with the
xxd -itool. - Size the tensor arena by starting large and reading the memory actually used.
- Validate on the device, comparing outputs with the desktop model on the same inputs.
Full integer quantization is the most important step: it shrinks weights about four times compared with float32, removes the need for a floating-point unit and lets optimized integer kernels run. The cost is a small accuracy loss, which quantization-aware training can usually reduce.
TensorFlow Lite Micro compared with other edge runtimes
TensorFlow Lite Micro is the smallest-footprint member of the on-device runtime family, and the right choice depends mainly on whether the target runs a full operating system.
| Runtime | Typical target | Key difference |
|---|---|---|
| TensorFlow Lite Micro | Bare-metal microcontrollers | No OS, no heap, static memory plan |
| LiteRT | Android, iOS, Linux devices | Dynamic memory, GPU and NPU delegates |
| ExecuTorch | Mobile and embedded from PyTorch | Ahead-of-time compiled PyTorch programs |
| ONNX Runtime | Desktop, server, mobile | Framework-neutral ONNX model format |
| Core ML | Apple devices | Tight integration with Apple hardware |
Some vendors also offer compiler-based alternatives that turn a model directly into C code instead of interpreting it. These can save memory, but TFLM remains attractive because of its portable interpreter, broad operator coverage and a single model format shared with the wider TensorFlow ecosystem.
Limitations and common mistakes
The main limitation of TFLM is its restricted operator set: only a subset of TensorFlow operations has microcontroller kernels, so a model that converts cleanly for phones can still fail to load on a microcontroller. Most other problems come from memory sizing and from mismatched preprocessing.
| Symptom | Likely cause | Fix |
|---|---|---|
| AllocateTensors fails | Tensor arena too small | Enlarge the arena, then trim to measured use |
| Operator not found | Kernel missing from the op resolver | Register the op or redesign the layer |
| Accuracy drops on device | Different preprocessing than in training | Reuse identical feature extraction code |
| Inference too slow | Reference kernels instead of optimized ones | Build with CMSIS-NN or vendor kernels |
| Random wrong outputs | Wrong input scale or zero point | Apply the quantization parameters correctly |
Developers should also remember that TFLM provides no training, no automatic model updates and only limited profiling, so over-the-air updates and monitoring must be built into the surrounding firmware.
The short version
TensorFlow Lite Micro lets very small chips run artificial intelligence. These chips sit inside fitness bands, smart sensors and household devices. They have less memory than a single photo on a phone.
The model is trained on a powerful computer first. It is then shrunk and packed into the chip’s software. The chip can then recognize a spoken word or an unusual vibration on its own. The key takeaway: the device decides locally, without sending data to the cloud and while using very little battery.
Where it makes a difference
A maintenance team at a water utility wants early warning of pump bearing failure across hundreds of remote stations with weak cellular coverage. Engineers train a small anomaly detection network on vibration spectra, quantize it to int8 and deploy it with TensorFlow Lite Micro on an Arm Cortex-M4 sensor node with 256 KB of RAM.
The model uses a tensor arena of about 40 KB and runs one inference per second in a few milliseconds. Instead of streaming raw vibration data, each node sends only a short alert when the score crosses a threshold. Radio traffic falls sharply, batteries last for years rather than months, and technicians replace failing bearings before an unplanned shutdown occurs.