ExecuTorch is the on-device inference runtime of the PyTorch ecosystem, designed to run trained models on mobile phones, embedded systems, wearables and microcontrollers without a Python interpreter. Within AI frameworks & tools in artificial intelligence, it occupies the deployment end of the pipeline: developers keep authoring models in PyTorch, then export them ahead of time into a compact program that a lightweight C++ runtime executes on the target hardware.
Meta and the PyTorch community introduced ExecuTorch as part of the PyTorch Edge effort, released it in preview in 2023 and reached a stable 1.0 release in 2025. It replaced the older PyTorch Mobile path built on TorchScript, and it now powers on-device features in large consumer applications as well as local execution of compact language models.
How ExecuTorch turns a PyTorch model into a device program
ExecuTorch works through an ahead-of-time compilation flow that moves as much work as possible off the device. The model is captured once on a workstation, transformed in several stages and serialized into a .pte file, a FlatBuffer-based program that the runtime loads with minimal parsing.
The table below lists the stages of that flow and what each one produces.
| Stage | Main API | Result |
|---|---|---|
| Graph capture | torch.export | ExportedProgram in the ATen dialect |
| Edge lowering | to_edge | Graph restricted to core ATen operators |
| Backend delegation | to_backend with a partitioner | Subgraphs handed to accelerator backends |
| Program finalization | to_executorch | Memory-planned program ready to serialize |
| Serialization | Buffer save | Portable .pte file for the runtime |
Because graph capture uses torch.export, the model must be traceable into a static graph. Dynamic Python control flow has to be rewritten with explicit control-flow operators or resolved at export time, and dynamic shapes must be declared with upper bounds so that memory can be planned in advance.
Memory planning is a defining feature. The compiler assigns tensor lifetimes and offsets ahead of time, so the runtime can work from pre-allocated memory arenas instead of a general allocator. This matters on microcontrollers without a heap and on phones where allocation spikes cause jank.
Backends, delegates and supported hardware
ExecuTorch reaches hardware performance through delegates, which are backend-specific compilers that take ownership of supported subgraphs. Operators that no delegate claims run on portable or optimized CPU kernels compiled into the runtime.
The following table shows representative backends and where they fit.
| Backend | Target hardware | Typical use |
|---|---|---|
| XNNPACK | Arm and x86 CPUs | Default fast path on phones and desktops |
| Core ML | Apple Neural Engine, GPU and CPU | iOS and macOS applications |
| Qualcomm AI Engine | Snapdragon Hexagon NPU | Android flagships and XR headsets |
| Vulkan | Mobile and desktop GPUs | Cross-vendor GPU acceleration |
| Arm Ethos-U | Microcontroller NPUs | Keyword spotting and sensor models |
| MediaTek and Samsung backends | Vendor NPUs | Device-specific acceleration |
Quantization is usually applied before delegation. The PT2E quantization flow and the torchao library produce 8-bit or 4-bit weights that delegates map onto integer hardware, which typically cuts model size by two to four times compared with 16-bit weights, with accuracy loss that depends on the model and calibration data.
Selective build keeps binaries small. Developers compile only the kernels a given model needs, and the core runtime itself can be tens of kilobytes before kernels and delegates are added, which suits constrained embedded targets.
ExecuTorch compared with other on-device runtimes
ExecuTorch competes and coexists with several runtimes that solve the same edge inference problem from different starting points. The comparison below helps place it among neighboring tools.
| Runtime | Native model source | Strength | Main limitation |
|---|---|---|---|
| ExecuTorch | PyTorch via torch.export | Native PyTorch path, broad delegates | Requires export-friendly model code |
| LiteRT | TensorFlow, JAX, converted PyTorch | Mature Android tooling | Conversion step for PyTorch models |
| Core ML | Converted models via coremltools | Deep Apple hardware integration | Apple platforms only |
| ONNX Runtime | ONNX graphs from many frameworks | Framework-neutral deployment | Export to ONNX can lose operators |
| llama.cpp | GGUF language model files | Highly tuned local LLM inference | Focused on transformer language models |
The main argument for ExecuTorch is staying inside PyTorch from training to deployment. Teams avoid a format conversion step, debug with familiar tools and keep operator semantics identical. Core ML can still be used underneath ExecuTorch as a delegate, so the two are complementary rather than mutually exclusive on Apple devices.
Practical adoption and configuration
Adopting ExecuTorch usually starts with one representative model and one target device. Engineers export the model, lower it with XNNPACK for a CPU baseline, measure latency and memory, and only then add an NPU delegate and quantization.
The decision table below summarizes common deployment choices.
| Situation | Recommended setup | Reason |
|---|---|---|
| First prototype on Android or iOS | XNNPACK with 8-bit quantization | Portable and predictable baseline |
| Latency-critical camera or audio model | Vendor NPU delegate | Lower power and higher throughput |
| Small language model on a phone | 4-bit weights with KV cache export | Fits memory limits of mobile devices |
| Microcontroller sensor model | Ethos-U or Cortex-M kernels with selective build | Minimal binary and static memory |
| Model with unsupported operators | Partial delegation with CPU fallback | Keeps the model running end to end |
Applications integrate the runtime through C++, Java or Kotlin, and Swift or Objective-C bindings. Profiling tools such as ETDump and the developer inspector map runtime events back to the original PyTorch operators, which makes accuracy and latency debugging much more practical than with opaque compiled blobs.
Limitations and common pitfalls
ExecuTorch shifts complexity to export time, and most problems surface there. Teams that plan for the following issues usually avoid long debugging cycles.
- Export failures: data-dependent Python branches and unsupported operations break torch.export and require model code changes.
- Uneven delegate coverage: an NPU backend may accept only part of the graph, and frequent switches between delegate and CPU can erase the speed gain.
- Quantization accuracy drops: poor calibration data or aggressive 4-bit settings can degrade results, so evaluation on the target device is essential.
- Version coupling: a .pte file is tied to compatible runtime and delegate versions, so app updates must ship matching artifacts.
- Benchmark drift: desktop emulation does not reflect thermal throttling and memory pressure on real phones.
In everyday language
ExecuTorch is a tool that lets an AI model run directly on your phone or gadget. It does not need to send data to a distant server. Developers build the model on a powerful computer. Then they pack it into a small, ready-to-run file.
Think of it like cooking a meal in advance and freezing it in portions. The hard work happens in the big kitchen. The phone only has to heat it up. As a result, apps can respond quickly, work offline and keep personal data on the device.
Seen in the real world
A health technology team builds a smartwatch app that detects irregular heart rhythms from a sensor signal. Sending raw readings to the cloud would drain the battery and raise privacy concerns, so the model must run on the watch itself.
The team trains a compact convolutional network in PyTorch, exports it with torch.export and applies 8-bit quantization. A selective build includes only the dozen kernels the network needs, and the program runs from a static memory arena. On the watch, inference takes a few milliseconds per window and the runtime adds only a small amount to the app size. The outcome is continuous on-device monitoring that works without a phone connection and never uploads raw heart data.