Core ML is the Apple framework for running trained machine learning models on device, across iOS, iPadOS, macOS, watchOS, tvOS and visionOS. Among AI frameworks & tools, it occupies the deployment end of the pipeline: models are trained elsewhere, typically in PyTorch or TensorFlow, then converted into the Core ML format and executed locally on the CPU, GPU and Neural Engine of Apple hardware.
Introduced in 2017, Core ML gives applications low-latency, private and offline inference because input data never has to leave the device. It underpins many Apple features and third-party apps, from photo classification and text recognition to speech, image generation and compact language models.
How Core ML runs a model on device
Core ML works by loading a compiled model file and dispatching each layer to the most suitable processor at runtime. A developer adds a model to an Xcode project, and Xcode compiles it into an optimized .mlmodelc bundle and generates a typed Swift or Objective-C interface with named inputs and outputs. At runtime the app creates an MLModel instance and calls prediction with images, arrays or text features.
The framework performs automatic hardware scheduling: it partitions the model graph and places supported operations on the Apple Neural Engine, others on the GPU through Metal, and the remainder on the CPU. Developers can restrict this choice with the compute units setting, which is useful for debugging or for balancing battery use against speed. The table below summarizes the options.
| Setting | Processors used | Typical reason to choose it |
|---|---|---|
| All | CPU, GPU and Neural Engine | Default choice for best overall performance |
| CPU and Neural Engine | CPU plus Neural Engine | Low power use and keeping the GPU free for graphics |
| CPU and GPU | CPU plus GPU | Operations or precision not supported on the Neural Engine |
| CPU only | CPU | Debugging, determinism checks or background tasks |
Higher-level Apple frameworks build on Core ML. Vision handles image preprocessing and returns detections, Natural Language wraps text models, and Sound Analysis classifies audio, so many apps never call Core ML directly.
Model formats and conversion with Core ML Tools
Core ML models are produced mainly with Core ML Tools, the open-source Python package coremltools. It converts traced or exported PyTorch models and TensorFlow models into Core ML, and it also handles classic scikit-learn and XGBoost models. Conversion maps each source operation to a Core ML operation; unsupported operations must be rewritten, decomposed or added as custom layers.
Two model representations exist. The older neural network format stores a fixed layer list in a single .mlmodel file, while the newer ML Program format, introduced in 2021, stores a typed program and its weights in an .mlpackage directory and supports float16 and lower-precision weights natively.
| Format | Structure | Role |
|---|---|---|
| .mlmodel | Single protobuf file | Legacy neural network and classic models |
| .mlpackage | Directory with program and weights | Recommended ML Program format |
| .mlmodelc | Compiled bundle | What the app actually loads at runtime |
For a quick start without Python, Create ML trains image, text, sound, tabular and activity models in a Mac app and exports them directly as Core ML files.
Optimization for memory, speed and power
On-device deployment is constrained by memory, latency and energy, so Core ML Tools offers compression techniques applied before or after training. Weight quantization stores values in 8 bits or fewer, palettization replaces weights with indices into a small lookup table, and pruning sets small weights to zero for sparse storage. Since 2024 the framework also supports stateful models, which keep a key-value cache between calls and make transformer text generation practical on device.
| Technique | Typical size reduction | Main trade-off |
|---|---|---|
| Float16 precision | About 2x versus float32 | Negligible accuracy loss for most models |
| 8-bit quantization | About 4x versus float32 | Small accuracy loss, calibration may help |
| 4-bit palettization | About 8x versus float32 | Accuracy loss unless fine-tuned after compression |
| Pruning | Depends on sparsity level | Gains appear only at high sparsity |
The actual speedup depends on the device generation and on whether the compressed operations run on the Neural Engine, so profiling in the Xcode Core ML performance report and Instruments is essential before and after compression.
Core ML compared with other on-device runtimes
Core ML is the native inference layer for Apple platforms, while most alternatives are cross-platform runtimes that trade some hardware access for portability. It is the only runtime with full, officially supported access to the Neural Engine, which is often the decisive factor for energy-efficient inference on iPhone.
| Runtime | Platforms | Model source | Best fit |
|---|---|---|---|
| Core ML | Apple only | Converted via Core ML Tools | Native Apple apps needing Neural Engine speed |
| LiteRT | Android, iOS, embedded | TensorFlow, JAX, PyTorch | Shared mobile code across platforms |
| ExecuTorch | Mobile and edge | PyTorch export | PyTorch-centric teams on several platforms |
| ONNX Runtime | Desktop, server, mobile | ONNX models | One model file across many environments |
| Windows ML | Windows | ONNX models | Windows desktop applications |
Some cross-platform runtimes can delegate execution to Core ML on Apple devices, combining a single codebase with native acceleration. For large language models, Apple also offers the Apple Foundation Models framework and the MLX array library, which complement Core ML rather than replace it.
Limitations and common mistakes
Core ML is restricted to Apple operating systems, so teams shipping on Android need a second deployment path. The Neural Engine supports a narrower set of operations and shapes than the GPU, and an unsupported layer silently pushes part of the graph back to the CPU or GPU, often with a large performance penalty.
- Assuming the Neural Engine is used: verify placement in the performance report instead of guessing.
- Mismatched preprocessing: scaling, channel order and normalization must match the training pipeline exactly.
- Flexible shapes everywhere: fully dynamic input sizes can prevent Neural Engine execution; prefer enumerated shapes.
- Shipping uncompressed weights: large apps download slowly and load slowly; compress and consider on-demand model downloads.
- Skipping accuracy checks after conversion: compare outputs with the source model on a validation set.
Model files bundled in an app can be extracted, so sensitive models should use Core ML model encryption, which ties decryption to the developer team through Apple services.
In simple terms
Core ML is the part of an Apple device that lets apps use artificial intelligence without sending your data to the internet. A company teaches a model to do something, such as recognizing plants in photos, and then packs that model into its app.
When you use the app, your iPhone or Mac does the thinking itself, using a special chip built for this kind of work. The result is quick, works offline and keeps your data private, because photos or messages stay on your own device.
A practical example
A small team builds a hiking app that identifies about 1,200 alpine plant species from photos, often in mountain areas without mobile coverage. They fine-tune an image classification network in PyTorch, convert it with Core ML Tools to the ML Program format and apply 8-bit weight quantization, shrinking the model from about 90 MB to roughly 23 MB.
Using the Vision framework for cropping and scaling, the app runs predictions on the Neural Engine in about 10 milliseconds on a recent iPhone, with top-five accuracy within one percentage point of the original model. Hikers get instant, offline identification and no photo ever leaves the phone, which also removes server costs for the team.