NNAPI

Also known as: Neural Networks API · Android Neural Networks API

On this page 7

NNAPI, the Android Neural Networks API, is a low-level C interface in Android that lets machine learning frameworks run neural network inference on device hardware such as GPUs, digital signal processors and neural processing units. Within AI frameworks & tools, it served as the hardware abstraction layer for on-device AI on Android from 2017 until its deprecation.

Applications rarely called the Neural Networks API directly. Instead, frameworks such as TensorFlow Lite, now LiteRT, used it through a delegate that handed supported parts of a model to vendor drivers. Google deprecated NNAPI in Android 15, and new projects are directed to LiteRT delegates and vendor-specific runtimes.

How NNAPI executes a neural network

NNAPI works in three stages: model definition, compilation and execution. A framework first describes the network as a graph of operands and operations, then asks the system to compile it for the available hardware, and finally runs the compiled graph on input data as often as needed.

The table lists the core objects of the C API and the role each plays in that lifecycle.

ObjectRoleCreated when
ANeuralNetworksModelDefines operands, operations, inputs and outputsOnce per network
ANeuralNetworksCompilationMaps the model onto devices and driversOnce per model and preference
ANeuralNetworksExecutionBinds input and output buffers for one runFor every inference
ANeuralNetworksMemoryShares large buffers without copyingFor weights or reused tensors
ANeuralNetworksBurstKeeps a fast path open for repeated runsFor streaming workloads such as camera frames
Core NNAPI objects and their roles

During compilation, the NNAPI runtime partitions the graph. Each operation goes to a driver that reports support for it, and anything unsupported falls back to a CPU reference implementation. Drivers are supplied by chip vendors through a hardware abstraction layer (HAL), so the same app code can reach a Qualcomm, MediaTek, Samsung or Google Tensor accelerator.

NNAPI feature levels and Android versions

NNAPI evolved through numbered feature levels tied to Android API levels. Each level added operations, data types and runtime features, and an app could only rely on what the oldest supported device offered.

The table shows the main milestones and what each one added.

Android versionAPI levelMain additions
Android 8.127First release, basic convolution and quantized operations
Android 928More operations, padding and batch support
Android 1029About 90 new operations, burst mode, vendor extensions, device discovery
Android 1130Quality-of-service hints, control flow, signed quantization
Android 1231Runtime updatable through a Mainline module
Android 1535API marked deprecated
Major NNAPI milestones

Android 10 was the most important turning point. Device discovery let apps choose specific accelerators, and vendor extensions allowed chip makers to expose custom operations, which improved coverage but also increased fragmentation between devices.

How NNAPI relates to other on-device runtimes

NNAPI sits below the model frameworks and above the vendor drivers, which makes it easy to confuse with neighboring tools. The comparison clarifies where each one fits in an on-device stack.

RuntimeLayerPlatformRelationship to NNAPI
NNAPIOS hardware interfaceAndroid 8.1 and laterReference point
LiteRTInference frameworkAndroid, iOS, embeddedUsed NNAPI through a delegate
Core MLOS frameworkApple platformsApple equivalent with a higher-level API
ONNX RuntimeInference frameworkCross-platformOffers an NNAPI execution provider
ExecuTorchInference frameworkMobile and edgePrefers direct vendor backends
Windows MLOS frameworkWindowsComparable OS-level role on PCs
NNAPI compared with related runtimes

The key distinction is that NNAPI is an operating system interface, not a model format. A model is still trained in PyTorch or TensorFlow, converted to a deployable format, and only then mapped to NNAPI operations at load time.

Using NNAPI in practice

Most developers used NNAPI by enabling a delegate in their inference framework rather than writing C code. Good results depended on quantized models, usually 8-bit integers, because many early accelerators supported only integer arithmetic.

The compilation step accepts an execution preference that guides how drivers schedule work:

  • Low power favors energy efficiency for background tasks.
  • Fast single answer minimizes latency for one-off requests such as a single photo.
  • Sustained speed maximizes throughput for continuous streams such as video frames.

Developers were advised to benchmark on real devices, cache compilations where supported, and keep a CPU or GPU fallback. A model that runs fully on an NPU can be several times faster than on the CPU, while a model split into many small partitions can be slower than the CPU alone.

Limitations and the reasons for deprecation

NNAPI was deprecated mainly because driver quality varied widely between vendors and Android versions. The same model could be fast on one phone, fall back to the CPU on another and produce slightly different numerical results on a third.

The table summarizes the common problems practitioners met and the usual response.

SymptomCauseUsual response
Slower than CPUGraph split into many partitionsSimplify operations or use the GPU delegate
Silent CPU fallbackDriver lacks an operationCheck partitioning logs and replace the operation
Accuracy driftDifferent driver arithmeticValidate outputs on each target chipset
Crashes on old devicesBuggy vendor driversUse device allowlists or disable NNAPI
Missing transformer operationsOperation set frozen before large modelsMove to vendor runtimes or LiteRT
Typical NNAPI problems and responses

Existing apps continue to work on current Android releases, but no new features are planned. For new projects, Google recommends LiteRT with GPU or vendor NPU delegates, which receive updates independently of the operating system.

In plain words

Modern phones contain special chips for artificial intelligence, but every manufacturer builds them differently. NNAPI was a common doorway that Android offered so that apps could use those chips without knowing who made them.

It worked like a universal power adapter: the app plugged into one standard socket, and each phone maker supplied the part that fit its own chip. Because those parts were not always reliable, Google has retired the doorway and now points developers to newer tools.

A real-world example

A team building a document scanning app for field inspectors needs to detect page edges in the live camera view at 30 frames per second on mid-range Android phones from 2020 and 2021.

The developers convert their small segmentation model to an 8-bit quantized LiteRT model and enable the NNAPI delegate with the sustained speed preference. On phones with supported NPUs, inference drops from about 40 milliseconds on the CPU to under 10 milliseconds, and battery drain during long inspection sessions falls noticeably.

Testing on twelve devices reveals two models with faulty drivers that return distorted masks. The team adds a device allowlist with CPU fallback, and the app ships with smooth edge detection on most phones and correct results on all of them.