Eye tracking in extended reality is the technology that measures where each of the user’s eyes is looking, in real time, using small cameras and infrared illumination built into the headset. In the area of tracking & interaction, eye tracking serves as both an input method, letting the user select objects by looking at them, and a system signal that enables foveated rendering, automatic display adjustment and expressive avatars.
Unlike head tracking, which tells the system where the face is pointing, eye tracking reveals the actual point of attention. That difference makes interfaces faster and rendering more efficient, but it also makes gaze data among the most sensitive personal data an XR device collects.
How eye tracking works in a headset
Most XR eye tracking uses video-based pupil and corneal reflection tracking. Infrared LEDs arranged around each lens illuminate the eye with light the user cannot see, and a small infrared camera records images at a high frame rate. Software locates the pupil center and the glints, the bright reflections of the LEDs on the cornea.
Because the glints stay relatively fixed while the pupil moves as the eye rotates, the vector between them indicates gaze direction. A geometric or machine-learning model converts this into a gaze ray for each eye. Where the two rays converge gives an estimate of fixation depth, although depth estimates are much less precise than direction.
A short calibration routine, in which the user looks at a few dots, adapts the model to individual eye shape and headset fit. Some systems recalibrate continuously in the background, since the headset shifts slightly on the face during use.
Key performance parameters
Eye tracking quality is described by accuracy, precision, latency and sampling rate, and each matters for a different use. Foveated rendering needs low latency, while gaze-based selection needs good accuracy.
| Parameter | Typical range | What it affects |
|---|---|---|
| Accuracy | About 0.5 to 1.5 degrees | How closely gaze matches the true target |
| Precision | Below about 0.5 degrees | Stability of gaze readings over time |
| Sampling rate | About 60 to 250 Hz | Ability to follow fast eye movements |
| End-to-end latency | Tens of milliseconds | Whether foveation keeps up with the eye |
| Trackable area | Central field of view | Accuracy drops toward the lens edges |
Accuracy of one degree sounds small, but at a distance of one meter it corresponds to roughly 1.7 centimeters. Interface targets must therefore be large enough to tolerate this error, which is why gaze-driven interfaces use generous button sizes and spacing.
What eye tracking enables
Eye tracking enables several distinct functions, from performance optimization to more natural interaction. The table summarizes the most important ones.
| Function | How gaze data is used | Benefit |
|---|---|---|
| Dynamic foveated rendering | Full detail only where the user looks | Large savings in GPU workload |
| Gaze input | Look to target, pinch or click to confirm | Fast, low-effort selection |
| Avatar expression | Eye movement and blinks shown on avatars | More natural social presence |
| IPD and fit adjustment | Measures eye position relative to lenses | Sharper image, less strain |
| User research | Records attention in training or design studies | Insight into what users notice |
Dynamic foveated rendering exploits the fact that human vision is sharp only in the fovea, a region covering roughly two degrees of the visual field. Rendering the periphery at reduced resolution can save a substantial share of GPU work, which matters most on standalone headsets with limited processing power.
Eye tracking compared with other input methods
Eye tracking is fastest for pointing, but it is poor at confirming an action, so it is usually paired with another input. The well-known Midas touch problem describes the risk: if looking alone triggered actions, everything the user glanced at would activate.
| Input method | Strength | Weakness |
|---|---|---|
| Eye tracking | Fastest pointing, no arm movement | Cannot confirm intent alone |
| Hand tracking | Natural gestures, no controllers | Arm fatigue, occlusion of fingers |
| Motion controller | Precise, buttons and haptics | Extra hardware to carry |
| Head gaze | Works on any headset | Slow, causes neck strain |
| Voice input | Hands-free commands | Noise, privacy, social awkwardness |
The common pattern of look and pinch combines eye tracking for targeting with a small hand gesture for confirmation. It keeps the hands relaxed in the lap and avoids the fatigue of holding arms up, a problem often called gorilla arm.
Limitations, privacy and common pitfalls
Eye tracking works well for most users but not for everyone, and it raises privacy questions that other tracking data does not. Designers should plan for both.
- Glasses and lenses: some spectacles, strong prescriptions or contact lenses with reflective coatings reduce tracking quality.
- Physiological variation: droopy eyelids, long eyelashes or certain eye conditions can block the camera’s view.
- Headset slippage: if the headset moves after calibration, accuracy degrades until it is refit or recalibrated.
- Sensitive data: gaze patterns can reveal interests, reading ability, fatigue and possibly health conditions.
- Missing fallbacks: interfaces that require gaze exclude users the tracker cannot follow.
Platforms increasingly treat gaze as protected data. Some systems never expose raw gaze coordinates to apps and report only which element was selected, while others require explicit permission before an app can read eye data. Developers should request gaze access only when it is essential and process it on the device whenever possible.
Understanding it simply
Eye tracking lets a headset know exactly where you are looking. Tiny cameras inside watch your eyes, much like a person who follows your gaze across a room, but far faster and more precisely.
This means you can choose a button just by looking at it and tapping two fingers together. The headset can also save effort by drawing only what you are looking at in full detail. Because your gaze reveals much about you, good systems keep that information private.
Applied in practice
An airline uses VR cockpit procedure trainers for new first officers. The instructors want to know not only whether trainees press the right switches during an engine failure scenario, but also whether they check the correct instruments before acting.
Headsets with eye tracking record fixation sequences during each session. The review shows that several trainees glance at the engine display only after reaching for the controls. Instructors adjust the briefing and repeat the scenario, and gaze recordings confirm the corrected scan pattern. At the same time, dynamic foveated rendering keeps the detailed cockpit running smoothly at 90 Hz. The outcome is training feedback based on measured attention, with gaze data stored only for the duration of the course.