ExecuTorch

On this page 7

ExecuTorch is the on-device inference runtime of the PyTorch ecosystem, designed to run trained models on mobile phones, embedded systems, wearables and microcontrollers without a Python interpreter. Within AI frameworks & tools in artificial intelligence, it occupies the deployment end of the pipeline: developers keep authoring models in PyTorch, then export them ahead of time into a compact program that a lightweight C++ runtime executes on the target hardware.

Meta and the PyTorch community introduced ExecuTorch as part of the PyTorch Edge effort, released it in preview in 2023 and reached a stable 1.0 release in 2025. It replaced the older PyTorch Mobile path built on TorchScript, and it now powers on-device features in large consumer applications as well as local execution of compact language models.

How ExecuTorch turns a PyTorch model into a device program

ExecuTorch works through an ahead-of-time compilation flow that moves as much work as possible off the device. The model is captured once on a workstation, transformed in several stages and serialized into a .pte file, a FlatBuffer-based program that the runtime loads with minimal parsing.

The table below lists the stages of that flow and what each one produces.

StageMain APIResult
Graph capturetorch.exportExportedProgram in the ATen dialect
Edge loweringto_edgeGraph restricted to core ATen operators
Backend delegationto_backend with a partitionerSubgraphs handed to accelerator backends
Program finalizationto_executorchMemory-planned program ready to serialize
SerializationBuffer savePortable .pte file for the runtime
Stages of the export flow

Because graph capture uses torch.export, the model must be traceable into a static graph. Dynamic Python control flow has to be rewritten with explicit control-flow operators or resolved at export time, and dynamic shapes must be declared with upper bounds so that memory can be planned in advance.

Memory planning is a defining feature. The compiler assigns tensor lifetimes and offsets ahead of time, so the runtime can work from pre-allocated memory arenas instead of a general allocator. This matters on microcontrollers without a heap and on phones where allocation spikes cause jank.

Backends, delegates and supported hardware

ExecuTorch reaches hardware performance through delegates, which are backend-specific compilers that take ownership of supported subgraphs. Operators that no delegate claims run on portable or optimized CPU kernels compiled into the runtime.

The following table shows representative backends and where they fit.

BackendTarget hardwareTypical use
XNNPACKArm and x86 CPUsDefault fast path on phones and desktops
Core MLApple Neural Engine, GPU and CPUiOS and macOS applications
Qualcomm AI EngineSnapdragon Hexagon NPUAndroid flagships and XR headsets
VulkanMobile and desktop GPUsCross-vendor GPU acceleration
Arm Ethos-UMicrocontroller NPUsKeyword spotting and sensor models
MediaTek and Samsung backendsVendor NPUsDevice-specific acceleration
Representative delegates and their targets

Quantization is usually applied before delegation. The PT2E quantization flow and the torchao library produce 8-bit or 4-bit weights that delegates map onto integer hardware, which typically cuts model size by two to four times compared with 16-bit weights, with accuracy loss that depends on the model and calibration data.

Selective build keeps binaries small. Developers compile only the kernels a given model needs, and the core runtime itself can be tens of kilobytes before kernels and delegates are added, which suits constrained embedded targets.

ExecuTorch compared with other on-device runtimes

ExecuTorch competes and coexists with several runtimes that solve the same edge inference problem from different starting points. The comparison below helps place it among neighboring tools.

RuntimeNative model sourceStrengthMain limitation
ExecuTorchPyTorch via torch.exportNative PyTorch path, broad delegatesRequires export-friendly model code
LiteRTTensorFlow, JAX, converted PyTorchMature Android toolingConversion step for PyTorch models
Core MLConverted models via coremltoolsDeep Apple hardware integrationApple platforms only
ONNX RuntimeONNX graphs from many frameworksFramework-neutral deploymentExport to ONNX can lose operators
llama.cppGGUF language model filesHighly tuned local LLM inferenceFocused on transformer language models
Comparison with related runtimes

The main argument for ExecuTorch is staying inside PyTorch from training to deployment. Teams avoid a format conversion step, debug with familiar tools and keep operator semantics identical. Core ML can still be used underneath ExecuTorch as a delegate, so the two are complementary rather than mutually exclusive on Apple devices.

Practical adoption and configuration

Adopting ExecuTorch usually starts with one representative model and one target device. Engineers export the model, lower it with XNNPACK for a CPU baseline, measure latency and memory, and only then add an NPU delegate and quantization.

The decision table below summarizes common deployment choices.

SituationRecommended setupReason
First prototype on Android or iOSXNNPACK with 8-bit quantizationPortable and predictable baseline
Latency-critical camera or audio modelVendor NPU delegateLower power and higher throughput
Small language model on a phone4-bit weights with KV cache exportFits memory limits of mobile devices
Microcontroller sensor modelEthos-U or Cortex-M kernels with selective buildMinimal binary and static memory
Model with unsupported operatorsPartial delegation with CPU fallbackKeeps the model running end to end
Choosing a configuration

Applications integrate the runtime through C++, Java or Kotlin, and Swift or Objective-C bindings. Profiling tools such as ETDump and the developer inspector map runtime events back to the original PyTorch operators, which makes accuracy and latency debugging much more practical than with opaque compiled blobs.

Limitations and common pitfalls

ExecuTorch shifts complexity to export time, and most problems surface there. Teams that plan for the following issues usually avoid long debugging cycles.

  • Export failures: data-dependent Python branches and unsupported operations break torch.export and require model code changes.
  • Uneven delegate coverage: an NPU backend may accept only part of the graph, and frequent switches between delegate and CPU can erase the speed gain.
  • Quantization accuracy drops: poor calibration data or aggressive 4-bit settings can degrade results, so evaluation on the target device is essential.
  • Version coupling: a .pte file is tied to compatible runtime and delegate versions, so app updates must ship matching artifacts.
  • Benchmark drift: desktop emulation does not reflect thermal throttling and memory pressure on real phones.

In everyday language

ExecuTorch is a tool that lets an AI model run directly on your phone or gadget. It does not need to send data to a distant server. Developers build the model on a powerful computer. Then they pack it into a small, ready-to-run file.

Think of it like cooking a meal in advance and freezing it in portions. The hard work happens in the big kitchen. The phone only has to heat it up. As a result, apps can respond quickly, work offline and keep personal data on the device.

Seen in the real world

A health technology team builds a smartwatch app that detects irregular heart rhythms from a sensor signal. Sending raw readings to the cloud would drain the battery and raise privacy concerns, so the model must run on the watch itself.

The team trains a compact convolutional network in PyTorch, exports it with torch.export and applies 8-bit quantization. A selective build includes only the dozen kernels the network needs, and the program runs from a static memory arena. On the watch, inference takes a few milliseconds per window and the runtime adds only a small amount to the app size. The outcome is continuous on-device monitoring that works without a phone connection and never uploads raw heart data.