AR (augmented reality) is a technology that overlays computer-generated content on a live view of the physical world, registered to real places and objects and updated in real time as the user moves. Among XR fundamentals in extended reality, AR sits near the real end of the reality-virtuality continuum: the real environment stays the main frame of reference, and digital elements are added to it rather than replacing it.
A widely used definition sets three requirements for AR: it combines real and virtual content, it is interactive in real time, and the virtual content is registered in three dimensions to the real world. A label floating at a fixed spot on a screen does not meet the last condition; an arrow that stays anchored to a specific doorway while the user walks does.
How AR aligns digital content with the real world
AR works as a continuous loop of sensing, tracking, rendering and display. Cameras and inertial sensors capture the environment and the device’s motion; tracking algorithms estimate the device’s position and orientation; a renderer draws virtual objects from that viewpoint; and the display combines them with the real view. The loop repeats for every frame, typically at 60 to 90 frames per second or more on head-worn devices.
The core of tracking is visual-inertial odometry and SLAM (simultaneous localization and mapping). The system detects feature points in the camera image, fuses them with accelerometer and gyroscope data and builds a map of surfaces. That map lets the software place a virtual object on a real table and keep it there, a property called registration. Depth sensors such as lidar or time-of-flight cameras improve surface detection and allow occlusion, where real objects correctly hide virtual ones.
The table below summarizes the main components of an AR system and the role of each.
| Component | Function | Typical technology |
|---|---|---|
| Sensors | Capture the scene and device motion | Cameras, IMU, depth sensors |
| Tracking | Estimates position and orientation | Visual-inertial SLAM |
| Scene understanding | Finds planes, meshes and objects | Computer vision and machine learning |
| Rendering | Draws virtual content from the user’s viewpoint | Real-time 3D engine |
| Display | Combines real and virtual imagery | Phone screen, optical or video see-through |
Types of AR devices and display methods
AR is delivered through three main device classes, and the display method determines how real and virtual light are combined. Handheld AR uses the phone camera feed; head-worn AR uses either transparent optics or cameras that pass the world through to internal screens.
| Device class | How the real world is seen | Strengths | Limitations |
|---|---|---|---|
| Handheld AR | Camera image on a phone or tablet | Huge installed base | Small view, hands occupied |
| Optical see-through headset | Directly through transparent optics | Real world at full clarity | Narrow field of view, weak in sunlight |
| Video see-through headset | Cameras feed internal displays | Solid, opaque virtual objects | Latency and camera image quality |
| Smart glasses | Directly through lenses with a small display | Light and wearable | Limited graphics and tracking |
| Projection AR | Images projected onto real surfaces | No device worn | Needs controlled lighting |
Optical see-through designs use waveguides or birdbath optics and typically offer a diagonal field of view of a few tens of degrees, far less than human vision. Video see-through headsets can show a wider and more convincing overlay, but every real object is seen through cameras, so pass-through quality and latency become critical.
AR compared with VR, MR and other XR categories
AR differs from VR (virtual reality) in its relationship to the environment: VR replaces the real world with a fully synthetic one, while AR keeps the real world visible and adds to it. MR (mixed reality) is used inconsistently; it often describes AR in which virtual objects interact convincingly with real geometry, such as bouncing off walls or hiding behind furniture.
| Term | Real world visible | Virtual content | Typical example |
|---|---|---|---|
| AR | Yes, as the main frame | Added on top, registered | Navigation arrows on a phone |
| MR | Yes | Interacts with real geometry | A virtual ball rolling under a real table |
| Augmented virtuality | Partly, inside a virtual scene | Main environment | Real hands shown in a virtual room |
| VR | No | Replaces reality | A fully virtual game world |
| XR | Varies | Umbrella term | Any of the above |
Where AR is used and how to choose an approach
AR is most valuable when information belongs to a specific place or object and the user needs to keep seeing the real world. That is why it is common in maintenance, logistics, medicine, retail and navigation, and less suited to experiences that benefit from full immersion.
| Need | Recommended approach | Reason |
|---|---|---|
| Reach a broad consumer audience | Phone-based AR | No extra hardware |
| Hands-free work instructions | Head-worn optical see-through | Keeps hands and view free |
| Convincing virtual objects in a room | Video see-through headset | Opaque rendering and occlusion |
| Quick product previews on the web | WebXR or browser AR | No app installation |
Developers build mobile AR mainly with ARKit on Apple devices and ARCore on Android, while headset content is commonly created in engines such as Unity or Unreal through the OpenXR standard or platform-specific SDKs.
Limitations and common misconceptions
The biggest technical challenge in AR is stable registration. Small tracking errors or latency make virtual objects drift or swim, which breaks the illusion and can cause discomfort. Lighting, reflective surfaces and featureless walls all degrade tracking.
- AR is not the same as a heads-up display: a fixed overlay without registration is not AR in the strict sense.
- Holograms in AR are not true holograms: most devices show stereoscopic images, not holographic light fields.
- Optical see-through cannot show black: transparent displays only add light, so dark content looks faint.
- Privacy concerns arise because AR devices continuously capture their surroundings.
- Battery and heat limits constrain how long and how intensely AR can run on mobile hardware.
A non-technical view
Augmented reality is like a smart transparent sticker layer placed over the world you already see. You still look at your real room, street or machine, but helpful pictures, labels or 3D objects appear in exactly the right places.
If you walk around a virtual sofa placed in your living room, it stays where you put it, just like a real one would. That is the key difference from a normal screen overlay: the digital things behave as if they belong to the real space around you.
A concrete example
A field technician servicing an industrial air handling unit wears an optical see-through AR headset. When she looks at the unit, the system recognizes it from a stored 3D model and overlays numbered arrows on the exact panels she must open, in the correct order.
Live sensor values from the unit appear next to each component, and a remote expert can draw annotations that stay anchored to the real parts as she moves around. Her hands remain free for tools throughout the job.
The outcome is a repair completed without a printed manual and with fewer return visits, because the instructions were attached directly to the equipment instead of to a separate document.