SSIM (Structural Similarity Index Measure) is a full-reference image and video quality metric that estimates how similar a distorted frame is to its original by comparing local luminance, contrast and structure. In video quality assessment, it is used to evaluate codecs, encoder settings and processing chains, producing a score where 1.0 means identical and lower values mean more visible degradation.
SSIM was introduced in 2004 as an alternative to PSNR, based on the observation that human vision is highly sensitive to structural information in a scene. It became one of the most widely used objective metrics in compression research and encoder tuning, although newer learned metrics such as VMAF now dominate streaming decisions.
How SSIM measures structural similarity
SSIM compares a reference and a test image inside a small sliding window, classically an 11×11 Gaussian-weighted window, and computes three components for each position. The local mean stands for luminance, the standard deviation for contrast, and the correlation between the two windows for structure.
The three terms are multiplied into a local SSIM value, with small stabilizing constants that prevent division by near-zero values in flat areas. The local values form an SSIM map, and their average gives the mean SSIM for the frame; a video score is usually the average over all frames.
The following table summarizes what each component captures.
| Component | Statistic used | Distortion it detects |
|---|---|---|
| Luminance | Local mean | Brightness shifts, gamma errors |
| Contrast | Local standard deviation | Loss of texture, blur, flattening |
| Structure | Normalized cross-correlation | Blocking, ringing, noise patterns |
In most video work, SSIM is computed on the luma (Y) channel only, because luma carries most perceived detail. Some tools also report chroma SSIM or a weighted combination of Y, Cb and Cr, so scores from different tools are not always comparable.
Interpreting SSIM scores
SSIM scores cluster close to 1.0 for typical encodes, so small numerical differences can matter. Many engineers convert SSIM to a decibel scale, calculated as minus ten times the base-10 logarithm of one minus SSIM, which spreads the high end and makes comparisons easier.
| SSIM (luma) | SSIM in dB | Typical visual impression |
|---|---|---|
| Above 0.99 | Above 20 dB | Visually lossless for most content |
| 0.97 to 0.99 | About 15 to 20 dB | Good, minor artifacts on close viewing |
| 0.93 to 0.97 | About 12 to 15 dB | Noticeable softness or blocking |
| Below 0.90 | Below 10 dB | Clearly degraded |
These ranges are only rough guides: the same SSIM value can look different on grainy film footage and on clean animation. Scores should be compared within the same content, the same resolution and the same tool.
SSIM compared with PSNR and VMAF
SSIM sits between the purely mathematical PSNR and the perceptually trained VMAF. It correlates better with human judgment than PSNR for many distortions, but it was not trained on viewer scores the way VMAF was.
| Metric | Principle | Strength | Weakness |
|---|---|---|---|
| PSNR | Pixel error energy | Simple and fast | Ignores how errors look |
| SSIM | Local structure statistics | Captures blur and blocking well | Insensitive to some color and temporal errors |
| MS-SSIM | SSIM at several scales | Accounts for viewing distance | More computation |
| VMAF | Machine-learned fusion of features | Best match to viewer ratings | Model depends on viewing conditions |
Multi-scale SSIM (MS-SSIM) evaluates the image at several downsampled resolutions and usually tracks perception more closely than single-scale SSIM. Encoders such as x264 and x265 offer a tune option for SSIM, which shifts bits toward structure preservation.
Using SSIM in practice
SSIM is most useful when the reference and test videos are perfectly aligned in resolution, frame rate and timing. Any spatial shift or dropped frame lowers the score dramatically even when the picture looks fine.
| Task | Suitability of SSIM | Reason |
|---|---|---|
| Comparing encoder presets on one clip | Good | Same content, aligned frames |
| Tuning in-loop filters during development | Good | Fast and sensitive to blur and blocking |
| Ranking quality across different titles | Weak | Scores depend strongly on content |
| Building a streaming bitrate ladder | Supplementary | VMAF usually preferred |
| Checking color or HDR accuracy | Weak | Luma-only SSIM ignores most color errors |
Limitations of SSIM
- No temporal model: frame-by-frame SSIM misses flicker and judder that viewers notice immediately.
- Weak on color: luma-only scores ignore chroma subsampling errors and color shifts.
- Banding is underestimated: smooth gradients with contouring can still score high because local structure changes little.
- Sensitive to misalignment: a one-pixel shift can cut the score sharply without any visible quality loss.
- Pooling hides local damage: averaging the SSIM map can conceal a small but very visible artifact.
Stripped to the essentials
SSIM is a score that tells how close a compressed video looks to the original. Instead of counting how many pixels changed, it checks whether shapes, edges and textures are still intact, which is closer to what people notice.
A score of 1.0 means the two pictures are identical. The closer the score stays to 1.0, the better the copy, although engineers still watch the video themselves because no single number catches every problem.
Where it is applied
A video engineer is choosing between two HEVC encoder presets for a sports channel. She encodes a demanding 10-second clip with fast motion and crowd detail at 8 Mbit/s with both presets and runs a frame-by-frame SSIM comparison against the uncompressed source.
The slower preset averages 0.982, the faster one 0.971, and the per-frame graph shows the faster preset dropping to 0.94 during camera pans. She confirms the difference by watching the pans side by side, sees visible blocking on the crowd, and selects the slower preset, adding a VMAF check before final approval.