MR, short for mixed reality, is the class of immersive experiences in which virtual objects are anchored in the physical environment and respond to it, so that real and digital elements coexist and interact in real time. Among XR fundamentals, MR occupies the middle of the reality–virtuality continuum, between a fully physical view and a fully virtual one.
The defining feature of mixed reality is spatial awareness: the system understands walls, tables and hands well enough that a virtual ball can bounce off a real floor or hide behind a real sofa. This distinguishes MR from simple overlays that float in front of the eyes regardless of what lies behind them.
How mixed reality works
Mixed reality works by continuously sensing the environment, tracking the user’s head and rendering aligned virtual content that matches the real scene. Cameras and depth sensors build a model of the room, inside-out tracking estimates the headset’s position in six degrees of freedom, and the renderer draws virtual objects from exactly the user’s viewpoint.
- Environment sensing: cameras, depth sensors or LiDAR capture geometry and lighting.
- Scene understanding: the system labels surfaces such as floors, walls and furniture and builds a spatial mesh.
- Tracking: 6DoF head tracking, often combined with hand and eye tracking, keeps alignment stable.
- Occlusion: real objects hide virtual ones that are behind them, which makes content feel physically present.
- Spatial anchors: virtual content stays fixed to a real location across sessions.
The result must be updated with very low delay. If motion-to-photon latency grows much beyond about 20 milliseconds, virtual objects appear to drift or swim against the real background, which breaks the illusion and can cause discomfort.
Optical see-through and video see-through
Mixed reality devices show the real world in one of two ways: optical see-through, where the user looks through transparent optics, or video see-through, where cameras capture the surroundings and displays reproduce them as pass-through video. The choice shapes almost every characteristic of the device.
| Property | Optical see-through | Video see-through |
|---|---|---|
| View of the real world | Direct, natural, no delay | Camera image with small latency |
| Virtual content | Additive light, cannot render true black | Full control, including dark and opaque objects |
| Field of view | Usually narrow for virtual content | Wide, similar to VR headsets |
| Occlusion of reality | Limited | Complete |
| Typical devices | Waveguide headsets, smart glasses | Pass-through headsets |
Early enterprise MR relied on optical see-through waveguide headsets. Recent consumer devices have shifted toward color video pass-through, because it allows convincing occlusion and lets one headset switch smoothly between mixed reality and full virtual reality.
MR compared with AR, VR and XR
MR is distinguished from its neighbors by how much of the view is real and how deeply virtual content interacts with it. The boundaries are not standardized, and vendors use the terms inconsistently, so the comparison below reflects common technical usage.
| Term | Share of the view that is real | Interaction with the real world |
|---|---|---|
| AR (augmented reality) | Mostly real | Overlay, often minimal environment awareness |
| MR (mixed reality) | Mostly real or blended | Content anchored to and occluded by real objects |
| Augmented virtuality | Mostly virtual | Real elements inserted into a virtual scene |
| VR (virtual reality) | None | Real world replaced entirely |
| XR (extended reality) | Any | Umbrella term covering all of the above |
In the academic model of the reality–virtuality continuum, mixed reality refers to everything between the purely real and purely virtual ends, which includes both AR and augmented virtuality. In product language, MR usually means advanced AR with spatial understanding, and the term spatial computing is increasingly used for similar experiences.
Typical uses of mixed reality
Mixed reality is most valuable where digital information must line up with physical objects. Tasks that benefit from hands-free guidance in a real location gain the most.
| Field | Typical use | Why MR fits |
|---|---|---|
| Industrial maintenance | Step-by-step instructions on a machine | Guidance appears on the exact component |
| Remote assistance | Expert annotates the technician’s view | Shared spatial markers reduce errors |
| Architecture and design | Full-scale models placed on site | Scale and fit judged in real space |
| Medical training | Anatomy overlaid on manikins | Hands stay free for procedures |
| Entertainment | Games that use the living room | Furniture becomes part of play |
Limitations and common misconceptions
Mixed reality remains technically demanding, and its main limitations come from sensing accuracy, display optics and comfort. Users should expect compromises that depend on the device class.
- Registration errors: small tracking errors shift virtual objects by centimeters, which matters in precision work.
- Pass-through quality: camera resolution, distortion and latency make the real world look less sharp than with the naked eye.
- Lighting: virtual objects rarely match real lighting perfectly, which reduces realism.
- Privacy: continuous room scanning captures detailed data about homes and workplaces.
- Misconception: a heads-up display that ignores the environment is not mixed reality, even if marketed as such.
Battery life and weight also limit sessions. Many standalone devices offer roughly two to three hours of active use per charge, which is enough for training sessions but short for a full work shift.
Content design adds its own constraints. Virtual objects placed at arm’s length or closer strain the eyes because of the vergence-accommodation conflict, so comfortable interfaces usually sit about one to two meters away. Developers also need fallbacks for rooms the sensors cannot map well, such as dark spaces, mirrors and glass walls, where scene understanding often fails.
The idea in brief
Mixed reality lets you see your real room with digital objects placed inside it, and those objects behave as if they really were there. A virtual pet can walk around your actual chair, and a virtual screen can stay attached to your real wall even when you turn away.
Ordinary augmented reality can simply float information in front of you. Mixed reality goes further by understanding the space around you, so the digital and the physical parts fit together.
A practical scenario
A wind turbine technician must replace a gearbox sensor, a task she performs only a few times per year. Instead of reading a paper manual balanced on the nacelle floor, she wears an MR headset that recognizes the gearbox housing and places numbered arrows directly on the bolts to loosen, in the correct order.
When a connector looks unfamiliar, she starts a remote assistance session: an engineer hundreds of kilometers away sees her view and draws a circle that stays anchored to the correct port. The outcome is a repair completed without a second site visit, with both hands free for tools and fewer errors than with printed instructions.