Windows ML (Windows Machine Learning) is the on-device inference runtime built into Windows that lets desktop applications load trained models in the ONNX format and run them locally on the CPU, GPU or neural processing unit. Among AI frameworks & tools in artificial intelligence, it occupies the deployment end of the pipeline: models are trained elsewhere, for example in PyTorch or TensorFlow, then exported to ONNX and executed by Windows ML on the end user’s machine.
The runtime exists so that developers do not need to ship and maintain a separate inference engine for every hardware vendor. Windows ML abstracts the silicon, selects or accepts an execution target, and handles memory binding, tensor conversion and scheduling, which keeps the application code largely identical across very different PCs.
How Windows ML executes an ONNX model
Windows ML executes a model by loading an ONNX graph, binding inputs and outputs to tensors, and dispatching each operator to a hardware backend. The original API, introduced with Windows 10 version 1809 in 2018, exposed the WinRT namespace Windows.AI.MachineLearning with three central objects: LearningModel, LearningModelSession and LearningModelBinding.
The typical flow is short. The application loads the model file once, creates a session on a chosen device, binds input features such as an image or a float tensor, and calls Evaluate. The session caches the compiled graph and device resources, so the first call is markedly slower than later calls; this warm-up cost matters for interactive software.
The following table summarizes the core objects of the classic programming model and their responsibilities.
| Object | Role | Lifetime |
|---|---|---|
| LearningModel | Holds the parsed ONNX graph and its metadata | Loaded once per model |
| LearningModelDevice | Selects CPU, default GPU or a specific adapter | Chosen at session creation |
| LearningModelSession | Compiles the graph for the device and runs it | Reused across many evaluations |
| LearningModelBinding | Maps named inputs and outputs to tensors or images | Created per evaluation or reused |
In 2025 Microsoft redesigned the runtime. The current Windows ML is built on ONNX Runtime, ships through the Windows App SDK, and obtains vendor-specific execution providers (EPs) from Windows itself, which downloads and updates them for the hardware present in the device.
Classic and current Windows ML generations
The two generations share a name and the ONNX format but differ in architecture and API surface. Developers who read older samples should know which generation a sample targets, because the classic API relied on DirectML for GPU acceleration, while the current one delegates to ONNX Runtime and its providers.
| Aspect | Classic Windows ML | Current Windows ML |
|---|---|---|
| Introduced | Windows 10 version 1809, 2018 | Announced 2025, generally available late 2025 |
| Engine | Inbox WinRT runtime | ONNX Runtime |
| Accelerators | CPU and GPU via DirectML | CPU, GPU and NPU via execution providers |
| Distribution | Part of Windows or a NuGet package | Windows App SDK with system-managed providers |
| Main API style | WinRT objects such as LearningModelSession | ONNX Runtime sessions plus provider catalog |
| Operator coverage | Fixed by the shipped opset version | Follows the ONNX Runtime release |
The practical consequence is that NPU support belongs to the current generation. Neural processing units in Copilot+ PCs from Qualcomm, Intel and AMD require vendor-specific compilers, and Windows ML hides them behind a common provider registration step instead of asking each application to bundle them.
Execution providers and hardware targets
An execution provider is the plug-in that translates ONNX operators into calls for one class of hardware. Windows ML can register the providers available on the device, after which ONNX Runtime partitions the graph: supported subgraphs run on the accelerator, and unsupported operators fall back to the CPU.
The table below lists typical targets; real performance depends on the model, numeric precision and drivers.
| Target | Strength | Typical precision | Common use |
|---|---|---|---|
| CPU | Universal availability, full operator coverage | FP32, INT8 | Small models and fallback |
| GPU | High parallel throughput | FP16, FP32 | Image, video and larger networks |
| NPU | Low power for sustained workloads | INT8, INT4, FP16 | Background and always-on features |
Applications can let the runtime apply a device selection policy, such as preferring lowest power or highest performance, or they can pin a specific provider. Explicit selection gives predictable behavior during testing, while policy-based selection adapts better to the wide range of Windows hardware.
Windows ML compared with related runtimes
Windows ML is best understood as a Windows-specific distribution layer over ONNX inference, not as a training framework. It competes with, and partly builds on, neighboring tools in the same space.
| Tool | Scope | Relationship to Windows ML |
|---|---|---|
| ONNX Runtime | Cross-platform inference engine | Engine underneath the current generation |
| DirectML | DirectX 12 machine learning library | GPU backend of the classic generation |
| TensorRT | NVIDIA GPU optimizer and runtime | Can serve as an NVIDIA execution provider |
| Core ML | Apple on-device runtime | Equivalent role on Apple platforms |
| LiteRT | Google on-device runtime | Equivalent role on Android and embedded devices |
The decisive difference from a plain ONNX Runtime installation is provider management by the operating system: instead of each application packaging vendor providers, Windows supplies shared, updated providers to all applications.
Preparing and deploying models for Windows ML
Deployment starts with a clean ONNX export. Teams usually export from PyTorch with a fixed or explicitly declared dynamic shape, verify the graph, and then apply quantization or graph optimization to fit the target accelerator.
- Export the trained model to ONNX with a supported opset version.
- Validate outputs against the original framework on a reference dataset.
- Quantize to INT8 or FP16 where the target provider benefits, and re-check accuracy.
- Compile or cache the model for NPU targets to avoid long first-run delays.
- Ship the model with the application and register execution providers at startup.
The optional AI Toolkit for Visual Studio Code helps convert, quantize and profile models, shortening the trial-and-error cycle of finding a variant that runs fully on the accelerator.
Limitations and common mistakes
Most problems with Windows ML come from operator coverage, precision and first-run latency, not from the API itself. The table pairs frequent symptoms with their usual causes.
| Symptom | Likely cause | Remedy |
|---|---|---|
| Model fails to load | Opset newer than the runtime supports | Re-export with an older opset |
| Accelerator shows little speedup | Many operators fall back to the CPU | Simplify the graph or replace operators |
| Accuracy drops after deployment | Aggressive INT8 or INT4 quantization | Use calibration data or mixed precision |
| First inference takes seconds | Graph compilation on session creation | Cache compiled models and warm up early |
| Results differ across PCs | Different providers and numeric paths | Set tolerances and test on each class |
Windows ML is inference only: it does not train or fine-tune models. It is also limited to Windows, so cross-platform products typically call ONNX Runtime directly on other systems and use Windows ML only where its provider management pays off.
A non-technical view
Windows ML is the part of Windows that lets an app use an artificial intelligence model directly on your own computer, without sending data to the internet. A company trains the model elsewhere and packs it into a common file format. Windows ML then reads that file and runs it.
Think of it as a universal adapter. Computers contain very different chips, and each chip speaks its own language. Windows ML translates the model for whatever chip is present, so the same app works on many machines and uses the fastest or most power-efficient part available.
A concrete example
A small software company builds a photo management application for Windows that needs to remove backgrounds and tag images offline. Its data scientists train a segmentation network and a classifier in PyTorch, export both to ONNX with opset 17, and quantize the classifier to INT8 after checking that top-1 accuracy falls by less than one percentage point.
At startup, the application uses Windows ML to register the available execution providers and asks for a low-power policy for background tagging and a performance policy for interactive background removal. On a laptop with an NPU, tagging of a 10,000-photo library runs on the NPU while the user keeps working; on an older desktop, the same code runs on the GPU or CPU.
The result is one codebase and one installer for all customers. The team caches compiled models after the first launch, which reduces the initial delay on later starts, and no customer photo ever leaves the device.