Homography

On this page 7

A homography is a projective transformation between two planes, represented by an invertible 3×3 matrix that maps points in one image to corresponding points in another. In computer vision, a homography exactly relates two views of the same planar surface, or two views taken by a camera that only rotates about its optical center.

Homographies underpin image stitching, document rectification, augmented reality marker tracking and camera calibration. Because the matrix is defined only up to scale, it has eight degrees of freedom and can be estimated from as few as four point correspondences.

How a homography maps points between planes

A homography maps points by working in homogeneous coordinates, in which a 2D point (x, y) is written as the vector (x, y, 1). Multiplying this vector by the 3×3 matrix H gives a new vector (x’, y’, w’), and the mapped image point is obtained by dividing by w’. This division is what makes the transformation projective rather than linear.

The projective division explains the characteristic effects of perspective. Straight lines remain straight, but parallel lines may converge to a vanishing point, and distances, angles and area ratios are generally not preserved. What a homography does preserve is collinearity and the cross-ratio of four points on a line.

The table below shows how the hierarchy of planar transformations relates to the homography, which is the most general member.

TransformationDegrees of freedomWhat it preserves
Translation2Orientation, lengths, angles
Euclidean (rigid)3Lengths and angles
Similarity4Angles and length ratios
Affine6Parallelism and area ratios
Projective (homography)8Collinearity and cross-ratio
Planar transformations ordered by generality

When a homography exactly describes two views

A homography exactly relates two images only under two geometric conditions. The first is that all observed points lie on a single physical plane, such as a wall, a page, a floor or a football pitch. The second is that the camera rotates without translating, so both views share the same center of projection, regardless of scene depth.

In the planar case, the matrix can be written as H = K'(R – t n^T / d) K^-1, where K and K’ are the camera intrinsic matrices, R and t the relative rotation and translation, n the plane normal and d the plane distance. In the pure rotation case, the translation term vanishes and H = K’ R K^-1, which is why panoramas taken from a tripod align well.

When neither condition holds, points at different depths shift by different amounts as the camera moves, an effect called parallax. A single homography cannot model parallax, so the correct relation between general views is the fundamental matrix or, with calibrated cameras, the essential matrix.

Scene and camera motionCorrect modelPoints needed
Planar scene, any motionHomography4
Any scene, pure rotationHomography4
General 3D scene, translationFundamental matrix7 or 8
General 3D scene, calibrated cameraEssential matrix5
Distant scene, small baselineHomography as an approximation4
Choosing the correct two-view model

Estimating a homography from correspondences

A homography is estimated from pairs of corresponding points found by keypoint detection and feature matching, for example with SIFT or ORB descriptors. Each correspondence contributes two linear equations, so four pairs in general position, with no three collinear, determine the eight unknowns.

The standard algorithm is the Direct Linear Transform (DLT). It stacks the equations into a matrix A and solves Ah = 0 by taking the singular vector of A with the smallest singular value. Normalizing the coordinates first, by translating points to their centroid and scaling them to an average distance of the square root of two, is essential for numerical stability.

Real matches always include errors, so estimation is wrapped in a robust estimator. RANSAC repeatedly fits a homography to four random correspondences, counts how many others agree within a reprojection threshold of typically 1 to 5 pixels, and keeps the model with the most inliers. The final matrix is usually refined with nonlinear least squares on the inliers, minimizing geometric rather than algebraic error.

MethodStrengthWeakness
Normalized DLTFast closed-form solutionSensitive to outliers
RANSAC with DLTTolerates many wrong matchesRandom; needs a threshold
MAGSAC and USAC variantsLess sensitive to threshold choiceMore complex to tune
Levenberg-Marquardt refinementMinimizes geometric errorNeeds a good initial estimate
Deep homography networksWork on low-texture imagesNeed training data; less precise
Estimation methods and their characteristics

Practical applications in computer vision systems

Homographies appear wherever a vision system must relate a plane seen from different viewpoints. The decision table below summarizes typical uses and why the homography model fits each one.

ApplicationPlane or motion involvedResult
Panorama stitchingCamera rotating on a tripodImages warped onto a common canvas
Document rectificationPaper page or whiteboardFronto-parallel scan from a phone photo
Sports analyticsPlaying fieldPlayer positions in field coordinates
Augmented reality markersPrinted fiducial markerVirtual content anchored to the marker
Camera calibrationPlanar checkerboardIntrinsic parameters from several views
Bird’s-eye viewRoad surfaceTop-down map for driver assistance
Typical homography applications

Calibration deserves special mention. Zhang’s method estimates one homography per checkerboard image and derives the camera intrinsics from constraints that these homographies impose, which is why a flat printed pattern is enough to calibrate most cameras. A homography can also be decomposed into rotation, translation and plane normal, typically with up to four candidate solutions that must be disambiguated.

Limitations and common mistakes

Most homography failures come from violating the planar or pure-rotation assumption or from poor correspondences. Recognizing the symptom usually points directly to the cause.

  • Ghosting in panoramas indicates parallax from moving the camera; rotate around the lens entrance pupil instead.
  • Wildly distorted warps often result from nearly collinear or clustered matches; spread correspondences across the image.
  • Unstable estimates usually mean coordinates were not normalized before the DLT.
  • Lens distortion bends straight lines, so images should be undistorted before fitting a homography.
  • A too-strict RANSAC threshold rejects valid matches, while a too-loose one admits outliers.

A further subtlety is degeneracy: if the scene is planar, the fundamental matrix becomes ill-defined, and if it is not, a homography gives misleading inliers on the dominant plane. Robust pipelines therefore test both models and select one with a criterion such as the geometric robust information criterion.

The simple explanation

A homography is a mathematical recipe for re-projecting a flat surface as it would look from another viewpoint. Imagine photographing a poster at an angle, so it looks like a slanted trapezoid. A homography tells the computer exactly how to stretch and tilt that trapezoid so the poster looks as if it were photographed straight on.

The recipe needs only four matching points, such as the four corners of the poster. The key limitation is that it works perfectly only for flat things or for photos taken by turning the camera without moving it.

How it is used in practice

A sports analytics team wants player positions in meters for a football match filmed by a single broadcast camera. An analyst identifies four or more pitch landmarks in a frame, such as penalty box corners and the center spot, whose real coordinates on a 105 by 68 meter pitch are known.

The software estimates a homography from these image-to-pitch correspondences with RANSAC, then maps the foot position of every detected player through the matrix. Because the camera pans and zooms, the system re-estimates the homography every frame from automatically tracked pitch lines.

The result is a top-down tactical map with position errors typically under one meter near the camera, rising toward the far touchline where pixels cover more ground. The team uses these coordinates to compute distances covered, pressing intensity and defensive line height.