A neural radiance field (NeRF) is a neural network that represents a three-dimensional scene as a continuous function from position and viewing direction to color and volume density, learned from a set of posed photographs. In computer vision, NeRF is a leading method for novel view synthesis: once trained, it renders photorealistic images of the scene from camera positions that were never photographed.
Introduced in 2020, the approach replaced explicit meshes and point clouds with an implicit neural representation optimized through differentiable volume rendering, and it triggered a large family of faster and more robust variants.
How a neural radiance field represents a scene
The core of a NeRF is a multilayer perceptron that takes a 3D point (x, y, z) and a viewing direction and returns an RGB color and a density value sigma. Density depends only on position, which keeps geometry consistent across views, while color also depends on direction, which lets the model reproduce view-dependent effects such as specular highlights and reflections.
Raw coordinates are passed through positional encoding, a set of sine and cosine functions at increasing frequencies, before entering the network. Without this encoding, the network is biased toward smooth functions and renders blurry images; with it, a network of roughly eight layers and 256 units can capture fine texture. The table summarizes the main inputs, outputs and components.
| Component | Role | Typical setting |
|---|---|---|
| Input position | 3D sample point along a camera ray | Encoded with about 10 frequency bands |
| Input direction | Viewing direction of the ray | Encoded with about 4 frequency bands |
| Density output | How much a point blocks light | Non-negative scalar, view independent |
| Color output | Emitted RGB radiance | View dependent, values in 0 to 1 |
| Sampling scheme | Points evaluated per ray | Coarse and fine passes, about 64 plus 128 |
Volume rendering and training
A NeRF produces an image by casting one ray per pixel through the scene and sampling points along it. Each sample’s color is weighted by its density and by the transmittance, the probability that light reaches that point without being absorbed earlier. Summing these weighted colors gives the pixel color, a numerical approximation of the classical volume rendering integral.
Because every step is differentiable, the network is trained by minimizing the photometric error between rendered pixels and the real photographs. Training requires accurate camera poses, which are usually estimated beforehand with structure from motion tools; a typical capture uses 50 to 200 images. Hierarchical sampling concentrates the fine samples where the coarse pass found high density, which saves computation on empty space.
The following table lists the inputs a NeRF pipeline needs and how each one affects quality.
| Input | Requirement | Effect when poor |
|---|---|---|
| Photographs | Sharp, overlapping, consistent exposure | Blur and floating artifacts |
| Camera poses | Accurate intrinsics and extrinsics | Ghosting and doubled geometry |
| View coverage | Many angles around the subject | Distortion in unseen regions |
| Static scene | No moving objects or lighting changes | Transient blobs and flicker |
| Scene bounds | Known near and far ray limits | Wasted samples and lower detail |
Variants and faster successors
The original NeRF needed many hours to a day of GPU training per scene and seconds per rendered frame. Later work attacked speed, scale and robustness, often by replacing the large network with explicit data structures such as voxel grids or hash tables that store learned features.
| Variant | Main idea | Benefit |
|---|---|---|
| Instant neural graphics primitives | Multiresolution hash encoding with a tiny network | Training in seconds to minutes |
| Mip-NeRF | Renders cones instead of rays | Reduces aliasing across scales |
| NeRF in the Wild | Per-image appearance and transient embeddings | Handles tourist photos with varied lighting |
| Plenoxels | Sparse voxel grid without a neural network | Fast optimization, simple structure |
| Dynamic NeRFs | Adds time or deformation fields | Models moving scenes |
Since 2023, 3D Gaussian splatting has become a strong alternative. It represents a scene with millions of explicit anisotropic Gaussians and rasterizes them, reaching real-time rendering with comparable quality. Many practitioners now treat NeRF and Gaussian splatting as related members of the broader family of radiance field methods.
How NeRF compares with other 3D reconstruction methods
NeRF differs from classical 3D reconstruction because it optimizes for image appearance rather than explicit surfaces. Multi-view stereo and photogrammetry produce point clouds and meshes that are easy to edit and measure, whereas a NeRF excels at reproducing thin structures, transparency and reflections that meshes handle poorly.
| Technique | Scene representation | Strength | Weakness |
|---|---|---|---|
| NeRF | Implicit neural field | Photorealistic novel views | Slow rendering, hard to edit |
| Photogrammetry | Textured mesh | Measurable, editable geometry | Struggles with shiny or thin objects |
| Gaussian splatting | Explicit 3D Gaussians | Real-time rendering | Large files, less clean geometry |
| Depth estimation | Per-image depth map | Single image input | No consistent full scene model |
Limitations and practical considerations
A standard NeRF is one network per scene: it does not generalize to new scenes without retraining, although generalizable variants condition on input images. Geometry extracted from the density field is often noisy, so applications needing precise surfaces usually add signed distance function methods or convert results with marching cubes and accept some loss of detail.
- Floaters: semi-transparent blobs appear in poorly observed space; regularization and better coverage reduce them.
- Baked lighting: the model stores appearance under capture lighting, so relighting requires separate inverse rendering approaches.
- Pose sensitivity: small calibration errors cause blur; joint pose refinement can partly compensate.
- Sparse views: with fewer than about 10 images, quality drops sharply unless priors from pretrained models are added.
In simple terms
A neural radiance field is a way to turn a set of photos into a 3D scene you can fly through. You walk around an object taking pictures, and a computer learns what color and how solid every point in space is.
Once it has learned this, it can draw the scene from any new angle, including angles nobody photographed. It is like a painter who has studied a room from many corners and can now paint it convincingly from a spot where they never stood.
A practical example
A real estate visualization studio needs an interactive walkthrough of a furnished apartment with glass tables and polished floors that its photogrammetry pipeline renders badly. A technician captures about 180 photographs with a mirrorless camera at fixed exposure, moving through each room at two heights.
Camera poses are computed with structure from motion, and the scene is trained with a hash-encoded NeRF in about 20 minutes on a single workstation GPU. The result reproduces reflections on the floor and the transparency of the glass tables, and the studio exports smooth video flythroughs at 4K resolution. Rooms with a moving curtain show some ghosting, so the team recaptures them with windows closed, which removes the artifact.