Camera calibration is the process of estimating the parameters that describe how a camera maps points in the 3D world onto pixels in its image. In robotics perception it is the foundation for any task that measures geometry from images, including stereo vision, visual odometry, SLAM, object pose estimation and visual servoing.
Calibration produces three groups of values: intrinsic parameters such as focal length and principal point, lens distortion coefficients, and extrinsic parameters that place the camera relative to the robot or other sensors. An uncalibrated camera still produces pictures, but it cannot produce trustworthy measurements.
The camera model behind calibration
Most robotics software uses the pinhole camera model: a 3D point is transformed into the camera frame, projected through a single center onto an image plane and then converted to pixel coordinates. Real lenses deviate from this ideal, so a distortion model is added between projection and pixel conversion.
The intrinsic parameters form the camera matrix, usually written as K. It contains the focal lengths fx and fy expressed in pixels, the principal point cx and cy where the optical axis meets the sensor, and occasionally a skew term that is effectively zero for modern sensors.
| Parameter group | Symbols | What it describes |
|---|---|---|
| Focal length | fx, fy | Scaling from angle to pixels |
| Principal point | cx, cy | Image location of the optical axis |
| Radial distortion | k1, k2, k3 | Barrel or pincushion bending of lines |
| Tangential distortion | p1, p2 | Lens not parallel to the sensor |
| Extrinsics | Rotation R, translation t | Camera pose relative to a reference frame |
How camera calibration is performed
The standard method observes a calibration target with precisely known geometry, such as a checkerboard, a circle grid or a ChArUco board, from many viewpoints. Software detects the target corners with subpixel accuracy and solves a nonlinear least-squares problem that finds the parameters which best reproject the known 3D points onto the detected pixels.
- Print or buy a flat, rigid target and measure its square size accurately.
- Capture 15 to 50 images covering the whole image area, including corners, at varied tilts and distances.
- Detect target features in each image and reject blurred or partial detections.
- Optimize intrinsics and distortion jointly with the pose of the target in each image.
- Check the reprojection error and inspect residuals for systematic patterns.
The quality measure is the root-mean-square reprojection error in pixels. Values below about 0.5 pixel usually indicate a good calibration for a well-focused camera, while values above one pixel suggest poor target detection, motion blur or a model that does not fit the lens. The residuals should also look random across the image; a systematic pattern near the borders points to an unsuitable distortion model.
Types of calibration in a robot system
A robot needs more than intrinsic calibration. Every camera must also be placed precisely in the robot coordinate frames, and multi-sensor systems need the relationships between cameras, LiDAR and IMU.
| Calibration type | What it estimates | Typical use |
|---|---|---|
| Intrinsic calibration | Camera matrix and distortion | Every measuring camera |
| Stereo calibration | Pose between two cameras | Depth from stereo vision |
| Hand-eye calibration | Camera pose relative to robot flange or base | Visual servoing, bin picking |
| Camera-LiDAR calibration | Pose between camera and LiDAR | Sensor fusion, colored point clouds |
| Camera-IMU calibration | Pose and time offset to the IMU | Visual-inertial odometry |
Multi-sensor calibration often has a temporal component as well. Even a perfect spatial calibration fails if camera timestamps lag the IMU by a few milliseconds, because a fast-moving robot then fuses measurements taken at different moments.
Choosing a lens model and target
The right distortion model depends on the lens field of view. A standard lens is well described by a few radial terms, while fisheye and omnidirectional lenses require dedicated models, because the polynomial radial model breaks down at wide angles.
| Lens type | Approximate field of view | Recommended model |
|---|---|---|
| Standard lens | Up to about 90 degrees | Pinhole with radial and tangential terms |
| Wide-angle lens | About 90 to 120 degrees | Pinhole with extra radial terms |
| Fisheye lens | About 120 to 200 degrees | Equidistant or Kannala-Brandt model |
| Omnidirectional or catadioptric | Above 180 degrees | Unified or double-sphere model |
Target choice matters as much. Checkerboards are accurate but must be fully visible; ChArUco boards encode marker identities, so partial views in the image corners are still usable, which helps estimate distortion where it is strongest.
Common mistakes and how calibration degrades
Most poor calibrations come from insufficient or biased data, not from the algorithm. A low reprojection error can even hide a bad result when all images were taken from similar viewpoints, because the optimizer then fits that narrow set perfectly.
| Symptom | Likely cause | Fix |
|---|---|---|
| Straight lines curve near edges | Target never placed in image corners | Add views covering the full frame |
| Wrong depth scale | Incorrect square size entered | Measure the printed target precisely |
| Unstable focal length | Too few tilted views | Include strong target tilts |
| Drift after weeks of use | Lens focus or mount shifted | Lock focus, recalibrate on schedule |
| Fusion misalignment | Unsynchronized timestamps | Calibrate time offset, use hardware triggering |
Calibration is valid only for the exact optical configuration used. Changing focus, zoom, aperture or image resolution, or bumping the camera mount, invalidates part or all of the parameters, so production robots often include self-check routines that compare known features against expected positions.
A plain-language summary
Every camera lens bends the picture a little, and every camera sits somewhere slightly different on a robot. Camera calibration is measuring those quirks precisely, by photographing a pattern with known sizes, such as a checkerboard, from many angles.
Once the robot knows how its camera distorts the world, it can undo the distortion and turn what it sees into real distances. It is like learning exactly how a pair of glasses changes your view so that you can still judge distances correctly while wearing them.
An example from practice
A logistics company deploys a bin-picking robot that uses a camera above the cell to locate parcels and guide a suction gripper. In early trials the gripper missed the center of small parcels by about one centimeter near the edges of the bin, although picks in the middle worked well.
The integrator found that the original intrinsic calibration had been made with a checkerboard held only in the center of the image, so radial distortion near the edges was poorly estimated. They recalibrated with a ChArUco board covering the full frame, then repeated the hand-eye calibration between camera and robot base. The reprojection error fell to about 0.3 pixel, and edge picks became as reliable as center picks, which removed a frequent cause of line stoppages.