Remote assistance is an extended reality application in which an off-site expert sees a field worker’s live first-person view and guides the task by voice and by annotations anchored to real objects in that worker’s surroundings. Among XR applications it is one of the most widely deployed uses of augmented reality headsets and smartphones, because it moves expertise instead of moving people.
The technician wears a headset or holds a phone or tablet, the expert works from a desktop, laptop or another headset, and both share a spatially aware video session. Arrows, circles, sketches and documents placed by the expert stay attached to the machine part they refer to even when the worker moves, which is the key difference from an ordinary video call.
How remote assistance works in extended reality
A remote assistance session combines real-time video, two-way audio and spatial tracking. The device on site streams its camera image while its tracking system builds a map of the surroundings through simultaneous localization and mapping, usually called SLAM. That map lets the software convert a mark drawn on a flat video frame into a 3D anchor in the room.
When the expert circles a valve on the screen, the system casts a ray from that pixel into the reconstructed scene, finds the surface it hits and places the annotation there. The worker then sees the circle locked to the valve, from any angle. Many platforms also offer a freeze-frame mode, in which the expert pauses a still image, annotates it precisely and sends it back, which helps when the network is slow.
The following table lists the main components of a typical session and what each contributes.
| Component | Role in the session | Typical implementation |
|---|---|---|
| On-site device | Captures the worker’s view and displays guidance | AR headset, assisted reality headset, phone or tablet |
| Expert client | Shows the live view and annotation tools | Desktop application, browser or second headset |
| Tracking and anchoring | Keeps annotations fixed to real objects | SLAM, plane detection, depth sensing |
| Media transport | Carries video, audio and annotation data | WebRTC or a similar low-latency protocol |
| Session services | Handle calls, recording and access control | Cloud or on-premises server |
Device types and their trade-offs
The choice of device decides whether the worker has both hands free and how convincingly guidance blends with the real world. Three device classes dominate, and each fits a different working environment.
| Device class | Hands-free | Spatial overlay | Best fit |
|---|---|---|---|
| Optical see-through AR headset | Yes | Full, anchored holograms | Complex assembly and inspection |
| Assisted reality headset | Yes | Small monocular screen, no true anchoring | Noisy, rugged or hazardous sites |
| Smartphone or tablet | No | Anchored overlays on the screen | Occasional support, customers, low budget |
Assisted reality devices place a small display just below the eye and show the expert’s video and documents without trying to overlay graphics on the world. They are often voice-controlled, ruggedized and certified for demanding sites, which explains their popularity in heavy industry. Full AR headsets deliver richer guidance but are heavier, have a limited field of view and can struggle in bright sunlight.
Smartphones are the lowest barrier to entry: a customer can join a session through a browser link without installing anything. The price is that one hand holds the device, so phone-based support suits diagnosis and simple fixes better than two-handed repairs.
Remote assistance compared with related XR uses
Remote assistance is live, one-to-one or one-to-few help for a task happening now. Neighboring XR applications share hardware and software but answer different needs.
| Application | Main purpose | Timing | Who guides |
|---|---|---|---|
| Remote assistance | Solve a specific problem on site | Live | A human expert |
| Industrial AR work instructions | Guide standard procedures step by step | Pre-authored | Software content |
| Virtual training | Build skills before real work | Scheduled | Scenario or instructor |
| XR collaboration | Let teams review designs or data together | Live | Shared among participants |
In practice these uses often merge. A recorded remote session can become reusable training material, and a step-by-step instruction app may offer a call an expert button when a worker gets stuck.
Network, latency and image quality requirements
Session quality depends mostly on the uplink from the site, because the worker’s device sends the video. A usable session typically needs a stable uplink of roughly 1 to 4 Mbit/s for 720p to 1080p video; less bandwidth forces lower resolution or frame rate, which can make small labels and gauges unreadable.
| Parameter | Typical working range | What it affects |
|---|---|---|
| Uplink bandwidth | About 1-4 Mbit/s per stream | Resolution and sharpness of the expert’s view |
| Round-trip latency | Below about 200-300 ms preferred | Natural conversation and timely pointing |
| Video resolution | 720p to 1080p | Legibility of labels, serial numbers and gauges |
| Frame rate | 15-30 frames per second | Smoothness when the worker moves |
| Battery runtime | Roughly 2-8 hours depending on device | Length of shifts without recharging |
Industrial sites often have weak coverage inside metal structures, basements or ships. Good platforms degrade gracefully by dropping to lower resolution, audio with still images, or freeze-frame annotation instead of failing completely.
Deploying remote assistance well
Successful deployments treat remote assistance as a change to a support process, not as a device purchase. The key decisions concern who the experts are, how calls are routed and how the knowledge gained is kept.
- Define the use cases first, such as first-time fix of equipment, quality inspection or customer self-service, and measure the outcome that matters, for example avoided travel or downtime.
- Match the device to the environment: hearing protection, gloves, helmets and safety glasses all limit which headsets are practical.
- Plan expert availability, including time zones and escalation rules, because an unanswered call is worse than no system.
- Address security and privacy: live video may show confidential equipment or bystanders, so encryption, access control and recording policies need approval.
- Capture sessions as knowledge, tagging recordings and annotated snapshots so that recurring problems turn into instructions.
Common mistakes include rolling out headsets without trained experts, ignoring network surveys on site and expecting workers to wear a headset for a full shift. Comfort, weight and heat decide acceptance as much as software features do.
The simple explanation
Remote assistance is like having an experienced colleague looking over your shoulder, even when that colleague is in another country. The person doing the job wears smart glasses or points a phone at the problem, and the expert sees exactly what they see.
The expert can talk, and can also draw arrows and circles that appear on the real object. If the expert marks the third bolt from the left, that mark stays on that bolt as the worker moves around. The result is that problems get solved without anyone traveling.
How it is used in practice
A packaging line at a food plant stops because a sealing unit keeps misaligning. The only specialist for that machine works for the manufacturer several hundred kilometers away, and a site visit would take at least a day and a half.
The plant technician puts on an assisted reality headset and calls the specialist. Over a 40-minute session the specialist reads the error codes through the camera, asks the technician to open a side panel and marks the sensor bracket that needs adjusting, then sends a wiring diagram to the small display.
The technician corrects the bracket position and the line restarts the same morning. The session recording and the annotated snapshots are filed with the machine record, so the next occurrence can be fixed from the documented steps without another call.