TensorFlow Lite Micro

Also known as: TFLM · LiteRT for Microcontrollers

On this page 7

TensorFlow Lite Micro (TFLM) is an open-source C++ inference runtime that executes trained neural networks on microcontrollers and digital signal processors with as little as tens of kilobytes of RAM and no operating system. Among AI frameworks & tools, it is the reference option for TinyML, the practice of running machine learning directly on battery-powered embedded devices.

TFLM was created by Google as the embedded branch of TensorFlow Lite, and after the 2024 rebranding of TensorFlow Lite as LiteRT it is also called LiteRT for Microcontrollers. It runs only inference, not training: models are trained elsewhere, converted to a compact file and compiled into the device firmware.

How TensorFlow Lite Micro runs a model

TensorFlow Lite Micro runs a model by interpreting a FlatBuffer model file that is stored in flash memory as a constant byte array. At startup, the application creates an interpreter, gives it the model, an operator resolver and a fixed block of RAM called the tensor arena, and then calls one function per inference.

The defining design choice is no dynamic memory allocation during operation. The interpreter plans all activation buffers inside the tensor arena once, during initialization, so memory use is deterministic and known at build time. This avoids heap fragmentation, which is a serious reliability risk on devices that must run for years without a restart.

The runtime pipeline has a few clearly separated components.

ComponentRoleDeveloper decision
Model arrayFlatBuffer graph and weights in flashModel size and quantization
Op resolverRegisters the kernels the model needsInclude only required operators
Tensor arenaStatic RAM for activations and scratch dataArena size, found by measurement
InterpreterPlans memory and invokes kernels in orderInitialization and error handling
Optimized kernelsHardware-specific operator codeChoice of vendor kernel library
Main runtime components

Hardware targets and resource budgets

TFLM targets devices whose resources are several orders of magnitude smaller than a phone. The core runtime fits in roughly 16 to 30 kilobytes of code on an Arm Cortex-M processor, and the remaining flash and RAM go to the model and the application.

Device classTypical memoryRealistic workloads
Arm Cortex-M0+ or M432 to 256 KB RAM, up to 1 MB flashKeyword spotting, gesture and vibration detection
Arm Cortex-M7 or M55512 KB to several MB RAMLow-resolution vision, richer audio models
ESP32-S3512 KB SRAM plus optional PSRAMWake words, simple person detection
DSP cores such as XtensaHundreds of KB local memoryAlways-on audio front ends
Microcontrollers with an NPUVaries by vendorAccelerated convolutional models
Typical targets and budgets

Performance depends heavily on optimized kernel libraries. Arm’s CMSIS-NN, Espressif’s ESP-NN and Cadence’s Xtensa kernels replace the portable reference kernels and often make int8 convolutions several times faster, so the same model can be viable on one chip and too slow on another.

From trained model to firmware

Deploying a model with TensorFlow Lite Micro follows a fixed conversion workflow. The network is trained in TensorFlow or Keras, converted with the TensorFlow Lite converter, almost always with full integer quantization, and then turned into a C source array that is compiled into the firmware.

  1. Train a small model with an architecture suited to microcontrollers, such as a depthwise separable CNN.
  2. Quantize to int8 using a representative dataset to calibrate activation ranges.
  3. Convert to a .tflite file and check that every operator is supported by TFLM.
  4. Embed the model as a constant array, for example with the xxd -i tool.
  5. Size the tensor arena by starting large and reading the memory actually used.
  6. Validate on the device, comparing outputs with the desktop model on the same inputs.

Full integer quantization is the most important step: it shrinks weights about four times compared with float32, removes the need for a floating-point unit and lets optimized integer kernels run. The cost is a small accuracy loss, which quantization-aware training can usually reduce.

TensorFlow Lite Micro compared with other edge runtimes

TensorFlow Lite Micro is the smallest-footprint member of the on-device runtime family, and the right choice depends mainly on whether the target runs a full operating system.

RuntimeTypical targetKey difference
TensorFlow Lite MicroBare-metal microcontrollersNo OS, no heap, static memory plan
LiteRTAndroid, iOS, Linux devicesDynamic memory, GPU and NPU delegates
ExecuTorchMobile and embedded from PyTorchAhead-of-time compiled PyTorch programs
ONNX RuntimeDesktop, server, mobileFramework-neutral ONNX model format
Core MLApple devicesTight integration with Apple hardware
TFLM and related on-device runtimes

Some vendors also offer compiler-based alternatives that turn a model directly into C code instead of interpreting it. These can save memory, but TFLM remains attractive because of its portable interpreter, broad operator coverage and a single model format shared with the wider TensorFlow ecosystem.

Limitations and common mistakes

The main limitation of TFLM is its restricted operator set: only a subset of TensorFlow operations has microcontroller kernels, so a model that converts cleanly for phones can still fail to load on a microcontroller. Most other problems come from memory sizing and from mismatched preprocessing.

SymptomLikely causeFix
AllocateTensors failsTensor arena too smallEnlarge the arena, then trim to measured use
Operator not foundKernel missing from the op resolverRegister the op or redesign the layer
Accuracy drops on deviceDifferent preprocessing than in trainingReuse identical feature extraction code
Inference too slowReference kernels instead of optimized onesBuild with CMSIS-NN or vendor kernels
Random wrong outputsWrong input scale or zero pointApply the quantization parameters correctly
Frequent problems and fixes

Developers should also remember that TFLM provides no training, no automatic model updates and only limited profiling, so over-the-air updates and monitoring must be built into the surrounding firmware.

The short version

TensorFlow Lite Micro lets very small chips run artificial intelligence. These chips sit inside fitness bands, smart sensors and household devices. They have less memory than a single photo on a phone.

The model is trained on a powerful computer first. It is then shrunk and packed into the chip’s software. The chip can then recognize a spoken word or an unusual vibration on its own. The key takeaway: the device decides locally, without sending data to the cloud and while using very little battery.

Where it makes a difference

A maintenance team at a water utility wants early warning of pump bearing failure across hundreds of remote stations with weak cellular coverage. Engineers train a small anomaly detection network on vibration spectra, quantize it to int8 and deploy it with TensorFlow Lite Micro on an Arm Cortex-M4 sensor node with 256 KB of RAM.

The model uses a tensor arena of about 40 KB and runs one inference per second in a few milliseconds. Instead of streaming raw vibration data, each node sends only a short alert when the score crosses a threshold. Radio traffic falls sharply, batteries last for years rather than months, and technicians replace failing bearings before an unplanned shutdown occurs.