SIFT, the scale-invariant feature transform, is a classic computer vision algorithm that detects distinctive keypoints in an image and describes each one with a 128-dimensional vector that stays stable under changes in scale, rotation and moderate lighting or viewpoint. Introduced by David Lowe in 1999 and refined in 2004, it became the reference method for local feature matching in computer vision.
SIFT answers a practical question: how can two photographs of the same scene, taken from different distances and angles, be linked point by point? Its keypoints and descriptors power image stitching, structure from motion, visual localization and object recognition, and they remain a common baseline against which learned features are measured.
How SIFT finds keypoints in scale space
SIFT finds keypoints by searching a Gaussian scale space for points that stand out at a characteristic size. The image is blurred repeatedly with Gaussian kernels of increasing sigma, grouped into octaves where each octave halves the resolution. Adjacent blurred images are subtracted to form a difference of Gaussians, an efficient approximation of the scale-normalized Laplacian.
A candidate keypoint is a pixel that is a local extremum among its 26 neighbors: eight in its own level and nine in each level above and below. Candidates are then refined to subpixel and subscale accuracy by fitting a quadratic function, and weak or unstable points are discarded.
The pipeline therefore runs in four stages, summarized in the following table.
| Stage | What it does | Result |
|---|---|---|
| Scale-space extrema detection | Searches difference-of-Gaussian levels | Candidate locations and scales |
| Keypoint localization | Refines position, removes weak and edge points | Stable keypoints |
| Orientation assignment | Builds a gradient orientation histogram | Rotation-invariant reference frame |
| Descriptor computation | Summarizes local gradients in a grid | 128-dimensional feature vector |
Orientation and the 128-dimensional descriptor
SIFT achieves rotation invariance by assigning each keypoint a dominant gradient orientation. Gradient directions around the keypoint are collected into a 36-bin histogram weighted by magnitude and a Gaussian window. The highest peak sets the orientation, and any other peak above 80 percent of the maximum creates an additional keypoint at the same location.
The descriptor is computed in a patch rotated to that orientation and scaled to the keypoint size. The patch is divided into a 4 by 4 grid of cells, and each cell holds an 8-bin orientation histogram, giving 4 x 4 x 8 = 128 values. The vector is normalized to unit length, values are clipped at 0.2 to reduce the influence of strong illumination edges, and the vector is normalized again.
The default parameters from the original paper, which most libraries still follow, are listed below.
| Parameter | Typical value | What it controls |
|---|---|---|
| Levels per octave | 3 | Scale sampling density |
| Initial sigma | 1.6 | Base blur of the first level |
| Contrast threshold | About 0.03 to 0.04 | Rejection of low-contrast points |
| Edge threshold | 10 (principal curvature ratio) | Rejection of edge responses |
| Descriptor size | 128 values | Distinctiveness and memory cost |
| Ratio test threshold | 0.7 to 0.8 | Acceptance of ambiguous matches |
Matching SIFT features between images
SIFT features are matched by nearest-neighbor search in descriptor space, usually with Euclidean distance. Because exhaustive search becomes slow with tens of thousands of features, practical systems use approximate nearest-neighbor structures such as k-d tree forests or hierarchical clustering indexes.
The decisive filtering step is Lowe’s ratio test. A match is accepted only if the distance to the closest descriptor is clearly smaller than the distance to the second closest, typically by a ratio below 0.8. This rejects ambiguous matches on repetitive textures such as windows or tiles. Surviving matches are then checked geometrically, usually with RANSAC estimating a homography or fundamental matrix, which removes the remaining outliers.
A widely used refinement called RootSIFT applies L1 normalization and an element-wise square root to the descriptor, which makes Euclidean comparison behave like the Hellinger kernel and often improves retrieval at almost no cost.
SIFT compared with other feature methods
SIFT trades speed for robustness and distinctiveness compared with later handcrafted detectors, while learned features now often outperform it under extreme conditions. The table compares it with common alternatives in the same Topic neighborhood.
| Method | Descriptor | Speed | Typical strength |
|---|---|---|---|
| SIFT | 128 floating-point values | Moderate | Robust scale and rotation invariance |
| SURF | 64 floating-point values | Faster | Similar invariance with box filters |
| ORB | 256-bit binary string | Very fast | Real-time use on limited hardware |
| Histogram of oriented gradients | Dense grid histogram | Fast | Whole-window object detection |
| Learned local features | Network-produced vectors | Needs a GPU for speed | Large viewpoint and lighting changes |
The difference from histogram of oriented gradients is often misunderstood. Both use gradient orientation histograms, but HOG describes a dense, fixed window for detection, while SIFT describes sparse, scale-adapted keypoints for correspondence.
Strengths, limitations and practical choices
SIFT is a strong choice when accuracy of correspondences matters more than frame rate. It handles scale changes of several times, full in-plane rotation and moderate viewpoint changes of roughly 30 to 50 degrees, but it is not affine invariant and degrades under strong perspective, motion blur and textureless surfaces.
The following decision table shows where SIFT fits in practice.
| Situation | Recommended choice | Reason |
|---|---|---|
| Offline photogrammetry | SIFT or RootSIFT | High-quality matches, time is not critical |
| Real-time mobile tracking | ORB or similar binary features | SIFT is too slow on a CPU |
| Extreme day-night changes | Learned features | Gradients change too much |
| Teaching and baselines | SIFT | Well documented and reproducible |
Licensing once limited adoption. SIFT was protected by a United States patent that expired in March 2020, after which OpenCV moved it into its main module in version 4.4. It is now freely usable in commercial software, which has renewed its role as a dependable default in structure from motion pipelines.
The short version
SIFT is a method that lets a computer find the same distinctive spots in two different photos. It works even if one photo is zoomed in or turned sideways. It picks out small, recognizable details, such as the corner of a window. Then it writes a short numeric fingerprint for each one.
When two fingerprints are very similar, both photos show the same real point. That simple idea makes it possible to stitch panoramas and build 3D models from ordinary pictures.
Where it makes a difference
A heritage conservation team wants a 3D model of a stone church facade before restoration work begins. A drone captures about 600 overlapping photographs from different heights and angles, with distances to the wall ranging from 5 to 30 meters.
The photogrammetry software extracts roughly 10,000 SIFT keypoints per image, matches them between overlapping photos with a ratio test threshold of 0.8 and filters the matches with RANSAC. Because SIFT tolerates the sixfold change in scale between close-up and distant shots, the software links images that a faster binary detector had failed to connect.
Structure from motion then recovers every camera position and a dense point cloud of the facade. The outcome is a measurable 3D model with centimeter-level detail, which the restorers use to document cracks and plan the work before any scaffolding goes up.