An XR SDK is a software development kit for building extended reality applications: a package of libraries, runtime interfaces, engine plug-ins, sample projects and tools that lets developers access a headset’s or phone’s tracking, rendering, input and environment-sensing features. In XR development, it is the layer between an application and the device platform, so developers do not have to program sensors, displays and controllers directly.
XR SDKs range from open standards implementations such as OpenXR runtimes to platform kits such as ARKit, ARCore and RealityKit, vendor kits for a specific headset family and engine toolkits that add ready-made interaction systems. Most real projects combine several of them, and choosing the right combination largely decides how easily an app can reach more than one device.
What an XR SDK contains
An XR SDK provides programming interfaces for the core XR functions that every immersive app needs. The most basic functions are reading the pose of the head and hands in real time, rendering a stereo image with correct projection for each eye, and submitting frames to the device compositor on time. Beyond that, SDKs expose input, environment understanding and platform services.
Most SDKs also ship developer tools: device simulators, performance profilers, frame timing overlays, sample scenes and documentation of design guidelines. These tools often matter as much as the libraries, because performance problems in XR show up as dropped frames and discomfort rather than slow menus.
| Component | What it provides | Example function |
|---|---|---|
| Tracking interface | Head, controller and hand poses | Predicted pose for the next frame |
| Rendering interface | Stereo views and frame submission | Swapchain images per eye |
| Input system | Buttons, triggers, gestures and gaze | Action bound to a trigger press |
| Scene understanding | Planes, meshes and room layout | Detected floor and walls |
| Spatial anchors | Persistent world positions | Object saved to a fixed spot |
| Platform services | Accounts, store, multiplayer | Entitlement check at startup |
Types of XR SDKs
XR SDKs fall into four main types, distinguished by who defines them and how many devices they serve. A typical project uses a cross-vendor standard for the core, a platform or vendor kit for special features and an engine toolkit for interaction.
| Type | Defined by | Device reach | Examples |
|---|---|---|---|
| Open standard runtime | Industry standards group | Many headsets | OpenXR, WebXR |
| Mobile AR platform kit | Phone operating system vendor | One mobile platform | ARKit, ARCore |
| Headset vendor kit | Device manufacturer | One device family | Meta XR All-in-One SDK, Meta Spatial SDK |
| Engine interaction toolkit | Game engine maker | Depends on the backend used | Unity XR Interaction Toolkit |
Some platforms combine several roles. On visionOS, developers mainly use RealityKit and ARKit together with the system’s native interface frameworks, while Android XR builds on OpenXR and Android tools. Understanding which layer each kit occupies prevents duplicate functionality and conflicts between plug-ins.
XR SDKs compared with standards and engines
An XR SDK is often confused with an XR standard and with a game engine, but each plays a different role. A standard such as OpenXR defines a common interface; an SDK is a concrete package of code and tools, which may implement a standard or extend it; an engine supplies the general rendering, physics and editor environment in which the SDK runs.
| Layer | Main role | Portability |
|---|---|---|
| XR standard | Defines a common interface and behavior | Highest, by design |
| XR SDK | Provides code, tools and device features | Depends on vendor extensions used |
| Game engine | Rendering, physics, editor and build system | High, through plug-ins |
| Device runtime | Implements the interface on the hardware | Tied to one device family |
The practical consequence is that vendor-specific features reduce portability. Hand tracking, eye tracking or passthrough effects may be offered through standard extensions on some devices and through proprietary calls on others, so careful projects isolate vendor code behind their own interfaces.
Choosing an XR SDK for a project
The choice of SDK starts with the target devices and the required features, not with the most popular toolkit. A decision table helps match common project goals to sensible starting points.
| Project goal | Recommended starting point | Reason |
|---|---|---|
| Reach many headsets | OpenXR through an engine | One code path for most devices |
| Run in a web browser | WebXR with a web 3D library | No app store installation |
| Phone-based AR | ARKit and ARCore, often through an engine layer | Best tracking on each platform |
| Use advanced headset features | Vendor SDK on top of OpenXR | Access to the newest capabilities |
| Native Apple headset app | RealityKit with ARKit | Deep integration with the system |
Teams should also check SDK maturity and update cadence. XR platforms change quickly, and an SDK that falls behind new firmware or engine versions can block a release. Locking SDK versions per release and testing upgrades on a separate branch is common practice.
Common pitfalls when working with XR SDKs
Most problems with XR SDKs come from mixing layers carelessly or from ignoring performance budgets that the SDK cannot hide.
- Duplicate input systems: combining two interaction toolkits often causes conflicting grabs and double events.
- Vendor lock-in: building core logic on proprietary calls makes later ports slow and expensive.
- Ignoring frame timing: missing the device’s frame budget, for example about 11 milliseconds at 90 hertz, causes reprojection artifacts and discomfort.
- Testing only in the simulator: tracking, lighting and performance behave differently on real hardware.
- Outdated samples: copying old sample code may rely on deprecated interfaces removed in newer releases.
In plain words
An XR SDK is a toolbox for building virtual and augmented reality apps. It already knows how to talk to the headset, the cameras and the controllers. Developers can focus on the app itself instead of the hardware details.
The key takeaway is that the toolbox decides which devices an app can run on. Some toolboxes work on many headsets. Others unlock special features of only one.
A real-world example
A small company builds a VR safety training app for industrial clients who use headsets from two different manufacturers. The team uses a game engine with OpenXR as the core interface and the engine’s interaction toolkit for grabbing tools, pressing buttons and teleporting.
One client wants hand tracking without controllers. The team adds the vendor SDK for that client’s headsets but wraps every vendor call in a small internal module, so the rest of the app only asks whether hands are available and where the fingers are.
The result is one code base for both device families, with the vendor feature enabled only where it exists. When a third client later chooses another headset that supports OpenXR, the port takes days of testing rather than months of rewriting.