SIFT

Also known as: Scale-invariant feature transform

On this page 7

SIFT, the scale-invariant feature transform, is a classic computer vision algorithm that detects distinctive keypoints in an image and describes each one with a 128-dimensional vector that stays stable under changes in scale, rotation and moderate lighting or viewpoint. Introduced by David Lowe in 1999 and refined in 2004, it became the reference method for local feature matching in computer vision.

SIFT answers a practical question: how can two photographs of the same scene, taken from different distances and angles, be linked point by point? Its keypoints and descriptors power image stitching, structure from motion, visual localization and object recognition, and they remain a common baseline against which learned features are measured.

How SIFT finds keypoints in scale space

SIFT finds keypoints by searching a Gaussian scale space for points that stand out at a characteristic size. The image is blurred repeatedly with Gaussian kernels of increasing sigma, grouped into octaves where each octave halves the resolution. Adjacent blurred images are subtracted to form a difference of Gaussians, an efficient approximation of the scale-normalized Laplacian.

A candidate keypoint is a pixel that is a local extremum among its 26 neighbors: eight in its own level and nine in each level above and below. Candidates are then refined to subpixel and subscale accuracy by fitting a quadratic function, and weak or unstable points are discarded.

The pipeline therefore runs in four stages, summarized in the following table.

StageWhat it doesResult
Scale-space extrema detectionSearches difference-of-Gaussian levelsCandidate locations and scales
Keypoint localizationRefines position, removes weak and edge pointsStable keypoints
Orientation assignmentBuilds a gradient orientation histogramRotation-invariant reference frame
Descriptor computationSummarizes local gradients in a grid128-dimensional feature vector
The four stages of the SIFT pipeline

Orientation and the 128-dimensional descriptor

SIFT achieves rotation invariance by assigning each keypoint a dominant gradient orientation. Gradient directions around the keypoint are collected into a 36-bin histogram weighted by magnitude and a Gaussian window. The highest peak sets the orientation, and any other peak above 80 percent of the maximum creates an additional keypoint at the same location.

The descriptor is computed in a patch rotated to that orientation and scaled to the keypoint size. The patch is divided into a 4 by 4 grid of cells, and each cell holds an 8-bin orientation histogram, giving 4 x 4 x 8 = 128 values. The vector is normalized to unit length, values are clipped at 0.2 to reduce the influence of strong illumination edges, and the vector is normalized again.

The default parameters from the original paper, which most libraries still follow, are listed below.

ParameterTypical valueWhat it controls
Levels per octave3Scale sampling density
Initial sigma1.6Base blur of the first level
Contrast thresholdAbout 0.03 to 0.04Rejection of low-contrast points
Edge threshold10 (principal curvature ratio)Rejection of edge responses
Descriptor size128 valuesDistinctiveness and memory cost
Ratio test threshold0.7 to 0.8Acceptance of ambiguous matches
Typical SIFT parameters

Matching SIFT features between images

SIFT features are matched by nearest-neighbor search in descriptor space, usually with Euclidean distance. Because exhaustive search becomes slow with tens of thousands of features, practical systems use approximate nearest-neighbor structures such as k-d tree forests or hierarchical clustering indexes.

The decisive filtering step is Lowe’s ratio test. A match is accepted only if the distance to the closest descriptor is clearly smaller than the distance to the second closest, typically by a ratio below 0.8. This rejects ambiguous matches on repetitive textures such as windows or tiles. Surviving matches are then checked geometrically, usually with RANSAC estimating a homography or fundamental matrix, which removes the remaining outliers.

A widely used refinement called RootSIFT applies L1 normalization and an element-wise square root to the descriptor, which makes Euclidean comparison behave like the Hellinger kernel and often improves retrieval at almost no cost.

SIFT compared with other feature methods

SIFT trades speed for robustness and distinctiveness compared with later handcrafted detectors, while learned features now often outperform it under extreme conditions. The table compares it with common alternatives in the same Topic neighborhood.

MethodDescriptorSpeedTypical strength
SIFT128 floating-point valuesModerateRobust scale and rotation invariance
SURF64 floating-point valuesFasterSimilar invariance with box filters
ORB256-bit binary stringVery fastReal-time use on limited hardware
Histogram of oriented gradientsDense grid histogramFastWhole-window object detection
Learned local featuresNetwork-produced vectorsNeeds a GPU for speedLarge viewpoint and lighting changes
SIFT and related feature methods

The difference from histogram of oriented gradients is often misunderstood. Both use gradient orientation histograms, but HOG describes a dense, fixed window for detection, while SIFT describes sparse, scale-adapted keypoints for correspondence.

Strengths, limitations and practical choices

SIFT is a strong choice when accuracy of correspondences matters more than frame rate. It handles scale changes of several times, full in-plane rotation and moderate viewpoint changes of roughly 30 to 50 degrees, but it is not affine invariant and degrades under strong perspective, motion blur and textureless surfaces.

The following decision table shows where SIFT fits in practice.

SituationRecommended choiceReason
Offline photogrammetrySIFT or RootSIFTHigh-quality matches, time is not critical
Real-time mobile trackingORB or similar binary featuresSIFT is too slow on a CPU
Extreme day-night changesLearned featuresGradients change too much
Teaching and baselinesSIFTWell documented and reproducible
When to choose SIFT

Licensing once limited adoption. SIFT was protected by a United States patent that expired in March 2020, after which OpenCV moved it into its main module in version 4.4. It is now freely usable in commercial software, which has renewed its role as a dependable default in structure from motion pipelines.

The short version

SIFT is a method that lets a computer find the same distinctive spots in two different photos. It works even if one photo is zoomed in or turned sideways. It picks out small, recognizable details, such as the corner of a window. Then it writes a short numeric fingerprint for each one.

When two fingerprints are very similar, both photos show the same real point. That simple idea makes it possible to stitch panoramas and build 3D models from ordinary pictures.

Where it makes a difference

A heritage conservation team wants a 3D model of a stone church facade before restoration work begins. A drone captures about 600 overlapping photographs from different heights and angles, with distances to the wall ranging from 5 to 30 meters.

The photogrammetry software extracts roughly 10,000 SIFT keypoints per image, matches them between overlapping photos with a ratio test threshold of 0.8 and filters the matches with RANSAC. Because SIFT tolerates the sixfold change in scale between close-up and distant shots, the software links images that a faster binary detector had failed to connect.

Structure from motion then recovers every camera position and a dense point cloud of the facade. The outcome is a measurable 3D model with centimeter-level detail, which the restorers use to document cracks and plan the work before any scaffolding goes up.