Core ML

On this page 7

Core ML is the Apple framework for running trained machine learning models on device, across iOS, iPadOS, macOS, watchOS, tvOS and visionOS. Among AI frameworks & tools, it occupies the deployment end of the pipeline: models are trained elsewhere, typically in PyTorch or TensorFlow, then converted into the Core ML format and executed locally on the CPU, GPU and Neural Engine of Apple hardware.

Introduced in 2017, Core ML gives applications low-latency, private and offline inference because input data never has to leave the device. It underpins many Apple features and third-party apps, from photo classification and text recognition to speech, image generation and compact language models.

How Core ML runs a model on device

Core ML works by loading a compiled model file and dispatching each layer to the most suitable processor at runtime. A developer adds a model to an Xcode project, and Xcode compiles it into an optimized .mlmodelc bundle and generates a typed Swift or Objective-C interface with named inputs and outputs. At runtime the app creates an MLModel instance and calls prediction with images, arrays or text features.

The framework performs automatic hardware scheduling: it partitions the model graph and places supported operations on the Apple Neural Engine, others on the GPU through Metal, and the remainder on the CPU. Developers can restrict this choice with the compute units setting, which is useful for debugging or for balancing battery use against speed. The table below summarizes the options.

SettingProcessors usedTypical reason to choose it
AllCPU, GPU and Neural EngineDefault choice for best overall performance
CPU and Neural EngineCPU plus Neural EngineLow power use and keeping the GPU free for graphics
CPU and GPUCPU plus GPUOperations or precision not supported on the Neural Engine
CPU onlyCPUDebugging, determinism checks or background tasks
Compute unit options in Core ML

Higher-level Apple frameworks build on Core ML. Vision handles image preprocessing and returns detections, Natural Language wraps text models, and Sound Analysis classifies audio, so many apps never call Core ML directly.

Model formats and conversion with Core ML Tools

Core ML models are produced mainly with Core ML Tools, the open-source Python package coremltools. It converts traced or exported PyTorch models and TensorFlow models into Core ML, and it also handles classic scikit-learn and XGBoost models. Conversion maps each source operation to a Core ML operation; unsupported operations must be rewritten, decomposed or added as custom layers.

Two model representations exist. The older neural network format stores a fixed layer list in a single .mlmodel file, while the newer ML Program format, introduced in 2021, stores a typed program and its weights in an .mlpackage directory and supports float16 and lower-precision weights natively.

FormatStructureRole
.mlmodelSingle protobuf fileLegacy neural network and classic models
.mlpackageDirectory with program and weightsRecommended ML Program format
.mlmodelcCompiled bundleWhat the app actually loads at runtime
Core ML file formats and their roles

For a quick start without Python, Create ML trains image, text, sound, tabular and activity models in a Mac app and exports them directly as Core ML files.

Optimization for memory, speed and power

On-device deployment is constrained by memory, latency and energy, so Core ML Tools offers compression techniques applied before or after training. Weight quantization stores values in 8 bits or fewer, palettization replaces weights with indices into a small lookup table, and pruning sets small weights to zero for sparse storage. Since 2024 the framework also supports stateful models, which keep a key-value cache between calls and make transformer text generation practical on device.

TechniqueTypical size reductionMain trade-off
Float16 precisionAbout 2x versus float32Negligible accuracy loss for most models
8-bit quantizationAbout 4x versus float32Small accuracy loss, calibration may help
4-bit palettizationAbout 8x versus float32Accuracy loss unless fine-tuned after compression
PruningDepends on sparsity levelGains appear only at high sparsity
Compression techniques and their trade-offs

The actual speedup depends on the device generation and on whether the compressed operations run on the Neural Engine, so profiling in the Xcode Core ML performance report and Instruments is essential before and after compression.

Core ML compared with other on-device runtimes

Core ML is the native inference layer for Apple platforms, while most alternatives are cross-platform runtimes that trade some hardware access for portability. It is the only runtime with full, officially supported access to the Neural Engine, which is often the decisive factor for energy-efficient inference on iPhone.

RuntimePlatformsModel sourceBest fit
Core MLApple onlyConverted via Core ML ToolsNative Apple apps needing Neural Engine speed
LiteRTAndroid, iOS, embeddedTensorFlow, JAX, PyTorchShared mobile code across platforms
ExecuTorchMobile and edgePyTorch exportPyTorch-centric teams on several platforms
ONNX RuntimeDesktop, server, mobileONNX modelsOne model file across many environments
Windows MLWindowsONNX modelsWindows desktop applications
Core ML and related inference runtimes

Some cross-platform runtimes can delegate execution to Core ML on Apple devices, combining a single codebase with native acceleration. For large language models, Apple also offers the Apple Foundation Models framework and the MLX array library, which complement Core ML rather than replace it.

Limitations and common mistakes

Core ML is restricted to Apple operating systems, so teams shipping on Android need a second deployment path. The Neural Engine supports a narrower set of operations and shapes than the GPU, and an unsupported layer silently pushes part of the graph back to the CPU or GPU, often with a large performance penalty.

  • Assuming the Neural Engine is used: verify placement in the performance report instead of guessing.
  • Mismatched preprocessing: scaling, channel order and normalization must match the training pipeline exactly.
  • Flexible shapes everywhere: fully dynamic input sizes can prevent Neural Engine execution; prefer enumerated shapes.
  • Shipping uncompressed weights: large apps download slowly and load slowly; compress and consider on-demand model downloads.
  • Skipping accuracy checks after conversion: compare outputs with the source model on a validation set.

Model files bundled in an app can be extracted, so sensitive models should use Core ML model encryption, which ties decryption to the developer team through Apple services.

In simple terms

Core ML is the part of an Apple device that lets apps use artificial intelligence without sending your data to the internet. A company teaches a model to do something, such as recognizing plants in photos, and then packs that model into its app.

When you use the app, your iPhone or Mac does the thinking itself, using a special chip built for this kind of work. The result is quick, works offline and keeps your data private, because photos or messages stay on your own device.

A practical example

A small team builds a hiking app that identifies about 1,200 alpine plant species from photos, often in mountain areas without mobile coverage. They fine-tune an image classification network in PyTorch, convert it with Core ML Tools to the ML Program format and apply 8-bit weight quantization, shrinking the model from about 90 MB to roughly 23 MB.

Using the Vision framework for cropping and scaling, the app runs predictions on the Neural Engine in about 10 milliseconds on a recent iPhone, with top-five accuracy within one percentage point of the original model. Hikers get instant, offline identification and no photo ever leaves the phone, which also removes server costs for the team.