Reality–virtuality continuum

On this page 7

The reality–virtuality continuum is a conceptual scale that places every display environment between two poles: the unmodified real environment at one end and a fully computer-generated virtual environment at the other. Among the XR fundamentals of extended reality, it is the reference model that explains why augmented reality, mixed reality and virtual reality are points on one spectrum rather than unrelated technologies.

The model was introduced by Paul Milgram and Fumio Kishino in 1994 as part of a taxonomy of mixed reality visual displays. Its central claim is that the meaningful variable is how much of what the user perceives is modeled by the computer and how much comes directly from the physical world.

How the continuum orders real and virtual content

The continuum orders environments by the proportion and dominance of synthetic content in the user’s perception. Everything between the two endpoints is what Milgram and Kishino called mixed reality, a broad region in which real and virtual elements are presented together in one display.

Inside that region, the original paper distinguished two sub-zones. In augmented reality, the primary experience is the real world and virtual objects are added to it. In augmented virtuality, the primary experience is a virtual world into which real elements, such as a live video of the user’s hands or a scanned room, are inserted. The boundary between the two is deliberately blurry, because the scale is continuous.

The following table summarizes the main positions on the scale and what dominates the user’s view at each one.

PositionDominant contentTypical example
Real environmentPhysical world onlyLooking through a clear window
Augmented realityReal world with virtual overlaysNavigation arrows projected on a street view
Augmented virtualityVirtual world with real insertsVirtual cockpit showing the pilot’s real hands
Virtual environmentSynthetic world onlyFully rendered VR game level
Positions along the reality–virtuality continuum

The three dimensions behind the original taxonomy

The continuum is only the most cited part of a richer framework. Milgram and Kishino argued that a single axis could not describe a display completely, so they proposed three supporting dimensions that explain where on the scale a system can realistically sit.

Extent of world knowledge describes how much the system knows about the objects and geometry of the real scene. Reproduction fidelity describes how realistically virtual content is rendered. Extent of presence metaphor describes how strongly the interface makes the user feel located inside the displayed scene, from a flat monitor to a fully immersive head-mounted display.

DimensionLow endHigh endWhy it matters
Extent of world knowledgeUnmodeled sceneEvery object modeledDetermines occlusion and anchoring accuracy
Reproduction fidelityWireframe graphicsPhotorealistic renderingAffects how convincingly virtual objects blend
Extent of presence metaphorMonitor-based windowReal-time immersive viewShapes immersion and sense of presence
Supporting dimensions of the 1994 taxonomy

These dimensions explain a practical point: placing virtual furniture convincingly in a real room requires high world knowledge, because the system must understand floors, walls and lighting. Modern spatial mapping and scene understanding in headsets are, in effect, engineering answers to that dimension.

How AR, MR, VR and XR map onto the scale

Current industry vocabulary maps onto the continuum, but not always in the way the original authors intended. In the 1994 paper, mixed reality meant the entire middle region. Many vendors now use MR to mean a specific class of experiences in which virtual objects are spatially anchored and interact with real surfaces, which narrows the original meaning.

Extended reality (XR) is an umbrella label covering the whole continuum, including its virtual endpoint. Spatial computing describes a computing paradigm rather than a position on the scale, although it usually relies on mixed reality capabilities.

TermCoverage on the continuumTypical display approach
ARNear the real endOptical see-through glasses or phone camera view
MRMiddle region with anchored contentVideo see-through or optical see-through headset
Augmented virtualityNear the virtual endVR headset with real-world inserts
VRVirtual endpointOpaque head-mounted display
XREntire continuumAny of the above
Common terms compared with the continuum

Display technology influences position but does not fix it. An optical see-through device shows the real world directly through transparent optics, while a video see-through device captures the world with cameras and redisplays it. A video see-through headset can therefore move freely along the scale, from passthrough-heavy AR to full VR, simply by changing how much camera imagery it shows.

Using the continuum to design and evaluate experiences

Designers use the continuum as a planning tool for choosing the right blend of real and virtual content for a task. The key question is whether the user needs continuous awareness of the physical environment or benefits from full isolation from it.

SituationRecommended positionReason
Maintenance on real machineryAugmented realityTechnician must see and touch the real equipment
Collaborative design review in an officeMixed realityVirtual model shares space with colleagues and tables
Flight simulator with physical controlsAugmented virtualityVirtual scenery with real hands and switches visible
Exposure therapy or immersive trainingVirtual realityFull control over every stimulus
Choosing a position for a given task

The model also helps evaluate devices. A headset marketed as mixed reality can be judged by how far along the scale it can travel and how well it handles the hard middle region, where errors in tracking, latency or lighting estimation become most visible.

Limitations and common misconceptions

The continuum is a conceptual model, not a measurement scale. It has no units, and two experts can place the same application at different points. It is best read as an ordering principle rather than a precise metric.

  • It is visual by origin. The 1994 framework focused on displays; audio, haptics and smell can also be mixed, and later researchers proposed multisensory extensions.
  • The endpoints are idealized. Even the most immersive VR still involves the real body, real gravity and real floor contact, so a truly pure virtual environment does not exist in practice.
  • MR is not a separate device category by definition. Treating MR as a product class obscures the original idea that it spans most of the scale.
  • Position is not quality. A point closer to the virtual end is not more advanced; it simply serves different tasks.

Some researchers also criticize the model for ignoring interaction and user agency, since it describes what is displayed rather than what users can do. Frameworks for presence and immersion are therefore used alongside it, not replaced by it.

In simple terms

Picture a slider. At one end you see the world exactly as it is. At the other end everything you see is made by a computer. The reality–virtuality continuum is that slider.

Phone apps that put a digital sticker on your table sit close to the real end. A headset game that replaces the whole room sits at the virtual end. Everything in between mixes the two. The key idea is that AR, MR and VR are not rivals but different settings on the same slider.

A practical example

An aircraft maintenance training center plans a new program for engine inspection. The team uses the continuum to decide where each module belongs. For the first lessons, trainees wear a fully immersive VR headset and explore a virtual engine, because mistakes cost nothing and instructors can trigger rare faults on demand.

Later modules shift toward the real end. Trainees stand next to a real engine and wear a video see-through headset that overlays torque values, part labels and step-by-step arrows onto the physical components. The final assessment uses no overlays at all.

By moving students deliberately along the scale, the center shortens time to hands-on readiness while keeping access to the real engine, its most expensive resource, for the stages where it matters most.