Image colorization

Also known as: photo colorization

On this page 7

Image colorization is the process of adding plausible color to a grayscale image by predicting, for every pixel, the chrominance that the original scene most likely had. Among AI applications, it is one of the most visible examples of generative computer vision, used in archives, film restoration, journalism, family photo services and scientific imaging.

Modern photo colorization does not recover lost color. A black-and-white photograph stores only luminance, so the color information is simply absent. A neural network instead infers a likely color from context such as object shape, texture and scene type, which means the result is an informed estimate rather than a measurement.

How neural networks predict missing color

AI image colorization works by learning a mapping from luminance to chrominance across millions of color photographs. Training data is created cheaply: color images are converted to grayscale, and the model learns to reproduce the original color channels from the gray version.

Most systems operate in the CIELAB color space, where the L channel holds lightness and the a and b channels hold color. The model keeps the input L channel unchanged and predicts only a and b, which preserves the original detail and tonal structure. The predicted channels are then combined with L and converted back to RGB for display.

Colorization is an ill-posed problem: a gray shirt could have been red, blue or green with the same lightness. Early regression models averaged these possibilities and produced desaturated, brownish results. Later methods treat color as a probability distribution over color bins, or use adversarial and diffusion training, so the model commits to one vivid and coherent choice instead of a muddy average.

The following table summarizes the main stages of a typical colorization pipeline.

StageWhat happensWhy it matters
PreprocessingScan cleanup, resizing, conversion to the L channelDamage and noise mislead color prediction
Semantic encodingA deep encoder recognizes objects and scene typeGrass, sky and skin need different colors
Chrominance predictionThe network outputs a and b values per pixelDetermines hue and saturation
Upsampling and fusionLow-resolution color is merged with full-resolution LKeeps fine detail sharp
PostprocessingSaturation tuning, artifact removal, manual touch-upImproves realism and consistency
Stages of an AI colorization pipeline

Main approaches to automatic colorization

Colorization methods differ in how much human guidance they use and in the model family behind the prediction. The choice affects realism, controllability and processing cost.

ApproachHow it worksTypical strengthTypical weakness
Fully automatic CNNConvolutional network predicts color binsFast and simple to deployConservative or faded colors
GAN-basedGenerator is trained against a discriminatorVivid, natural-looking outputOccasional artifacts and color bleeding
Diffusion-basedIterative denoising conditioned on the gray imageHigh realism and diversitySlower and more compute-intensive
User-guided (scribble)Artist places color hints on regionsPrecise control over key areasRequires manual effort
Exemplar-basedColor is transferred from a reference photoMatches a known palette or periodDepends on a suitable reference
Text-guidedA prompt such as a color description steers outputFast intent controlPrompt may be ignored in small regions
Families of colorization methods

For video, temporal consistency is a separate challenge. Colorizing each frame independently causes flicker, because the model may choose a different hue for the same jacket in consecutive frames. Video systems propagate color along motion estimates or use reference frames to keep objects stable over time.

How image colorization relates to neighboring tasks

Image colorization is often bundled with restoration tools, but it solves a different problem. AI photo restoration repairs damage that exists in the image, while colorization adds information that never existed in the source.

TaskInputOutputInformation added
Image colorizationGrayscale imageColor imageInvented chrominance
AI photo restorationDamaged photoRepaired photoReconstructed scratches and tears
Object removalImage with unwanted objectImage with filled backgroundSynthesized background pixels
Super-resolutionLow-resolution imageHigher-resolution imageEstimated fine detail
Color correctionColor image with a castBalanced color imageAdjusted existing color
Colorization compared with related image tasks

In practice, a professional workflow usually runs restoration first and colorization second. Dust, scratches and fading confuse the encoder, which may interpret a crack as an edge and assign it a separate color.

Using colorization in professional work

Colorization is most useful when the goal is engagement and accessibility rather than documentary precision. Archives and broadcasters use it to make historical material feel immediate, while researchers sometimes use it to make grayscale microscopy or satellite data easier to read.

The decision table below shows which method suits common situations.

SituationRecommended approachReason
Large family photo collectionFully automatic batch processingLow cost per image, acceptable accuracy
Museum exhibition printUser-guided with historian reviewKnown uniform and object colors matter
Documentary film footageVideo model with reference framesPrevents flicker across frames
Recreating a specific era paletteExemplar-based transferMatches period film stock or dyes
Scientific grayscale dataRule-based false color, not AIColor must encode measured values
Choosing a colorization method

A sound workflow keeps the original grayscale file alongside the colorized version, records the method and settings used, and labels the output as colorized. Where accuracy matters, specialists check colors against historical evidence such as uniform regulations, paint catalogs or written descriptions.

Limitations, bias and ethical concerns

The central limitation is that every predicted color is a guess. Models learn from modern photographs, so they tend to apply contemporary color statistics to historical scenes, for example modern car paints or present-day clothing dyes.

  • Skin tone bias: models trained on unbalanced data can lighten or homogenize skin tones, which misrepresents people in historical images.
  • Color bleeding: color spills across object boundaries when edges are weak or the image is blurred.
  • Default palettes: ambiguous objects often receive safe colors such as brown, beige or muted blue.
  • Small objects: flags, badges and signs are frequently colored incorrectly because they occupy few pixels.
  • False authenticity: unlabeled colorized images can be mistaken for original color records.

These issues make transparency and disclosure essential when colorized images appear in journalism, education or exhibitions. Responsible publishers state that color was added by software and, where possible, describe the level of human verification applied.

In everyday language

A black-and-white photo records how bright each part of a scene was, but not its color. Image colorization software looks at the picture, recognizes things such as sky, trees, faces and clothes, and paints in the colors those things usually have in the many color photos it studied.

Think of an experienced painter coloring an old print from memory. The result can look convincing, but it is still an educated guess, not a record of what was really there. That is why careful users keep the original and say clearly that color was added.

Seen in the real world

A regional history museum prepares an exhibition about its town in the 1920s using about 300 scanned glass-plate negatives. Staff first clean the scans with restoration software to remove dust and cracks, then run an automatic colorization model in batch mode to produce draft color versions within a single afternoon.

A curator reviews the drafts and finds that tram cars, shop signs and military uniforms received modern-looking colors. Using archival paint records and uniform descriptions, a retoucher adds color hints for about 40 key images and reruns them in user-guided mode. The final prints hang next to the original grayscale versions, each labeled as digitally colorized, and the museum reports longer visitor dwell times at the colorized panels while avoiding any claim that the colors are documented fact.