Image-to-image

Also known as: img2img

On this page 7

Image-to-image (img2img) is a generative AI technique that takes an existing image as the starting point and produces a new image that keeps chosen properties of the original, such as composition, pose or color layout, while changing others according to a text prompt or another condition. In generative AI, it is the counterpart of text-to-image generation, which starts from pure noise and a prompt alone.

The term most often refers to the diffusion-based img2img workflow popularized by Stable Diffusion, but it also covers earlier paired and unpaired translation networks based on GANs. Designers, photographers, game artists and product teams use it to restyle sketches, create variations, upscale drafts and explore ideas without starting from a blank canvas.

How image-to-image generation works in diffusion models

In a diffusion model, image-to-image works by partially noising the input image and then denoising it under the guidance of a prompt. The input is first encoded by a variational autoencoder into the latent space, where a controlled amount of Gaussian noise is added. The model then runs only the remaining denoising steps, so the output inherits the coarse structure of the source.

The single most important control is denoising strength, sometimes called strength or noise level, expressed from 0 to 1. A value of 0.3 adds little noise, so the result stays close to the original; a value of 0.8 destroys most detail, so the result follows the prompt far more than the source. With 30 sampling steps and a strength of 0.6, the sampler executes roughly the last 18 steps only.

The following table shows how denoising strength typically affects the result. Exact thresholds vary by model, sampler and resolution, so the ranges are approximate.

Strength rangeWhat is preservedTypical use
0.1 to 0.3Composition, edges and most fine detailSubtle color or texture refinement
0.3 to 0.5Layout, shapes and lighting directionRestyling while keeping the design
0.5 to 0.7Overall composition and color massesTurning sketches into finished renders
0.7 to 0.9Rough placement of main forms onlyLoose reinterpretation and ideation
Close to 1.0Almost nothing from the sourceEffectively text-to-image
Typical effect of denoising strength in img2img

Other parameters interact with strength. Classifier-free guidance scale controls how strongly the prompt is followed, usually between 5 and 12, and a fixed seed makes runs reproducible so that one variable can be changed at a time.

Types of image-to-image methods

Image-to-image is an umbrella for several technical approaches that share the same goal of mapping one image to another. They differ in training data requirements, controllability and typical output quality.

MethodCore mechanismTraining data needed
Diffusion img2imgPartial noising and prompt-guided denoisingNone beyond a pretrained diffusion model
Conditional GAN translationGenerator learns a mapping from paired imagesAligned input and output pairs
Unpaired GAN translationCycle-consistency between two image domainsTwo unpaired image collections
Structure-conditioned diffusionExtra network reads edges, depth or pose mapsPretrained control module such as ControlNet
Instruction-based editingModel follows written edit commandsLarge sets of edit examples
Main families of image-to-image methods

Older GAN-based translation models, such as those that convert maps to aerial photographs or summer scenes to winter, are narrow but fast once trained for one domain. Diffusion-based img2img is far more general, because a single pretrained model can handle almost any style or subject described in a prompt, at the cost of slower, multi-step sampling.

Image-to-image sits between pure generation and precise editing, and it is often confused with neighboring techniques that also accept an image as input. The table clarifies the differences.

TechniqueInputScope of change
Text-to-imagePrompt and random noiseEntire image created from scratch
Image-to-imageSource image and promptWhole image changed by a chosen degree
InpaintingSource image, mask and promptOnly the masked region is regenerated
Style transferContent image and style referenceTexture and palette change, content stays
ControlNetPrompt plus a structure mapExact pose, edges or depth are enforced
How img2img relates to neighboring techniques

The practical difference is how structure is preserved. Plain img2img preserves structure only implicitly through the noise level, so high strength values lose geometry. Structure conditioning with edge or depth maps keeps geometry fixed even when strength is high, which is why professional pipelines often combine img2img with a control module.

Practical workflows and configuration

Effective image-to-image work starts with a clear decision about what must stay fixed. Once that is known, strength, prompt and conditioning can be chosen to protect those properties while freeing the rest.

GoalRecommended setupReason
Sketch to finished renderStrength 0.55 to 0.7 with a descriptive promptKeeps layout but adds detail and lighting
Product photo restylingStrength 0.3 to 0.45 plus edge conditioningProtects product shape and labels
Generating variationsSame prompt, strength about 0.5, new seedsExplores alternatives around one design
Upscaling with added detailResize first, then strength 0.2 to 0.35Adds texture without changing content
Changing one object onlyUse inpainting insteadPrevents unwanted changes elsewhere
Choosing settings for common img2img goals

Input resolution and aspect ratio should match what the model was trained on, for example around one megapixel for many current models, because mismatched sizes produce duplicated subjects or stretched anatomy. Iterating in small strength increments of about 0.05 with a fixed seed is the most efficient way to find the right balance.

Limitations, risks and common mistakes

The central limitation of image-to-image is the trade-off between fidelity and creativity: one parameter controls both, so gaining stronger stylistic change almost always costs structural accuracy. Fine details such as text, hands and logos are especially fragile and often degrade even at moderate strength values.

  • Strength set too high makes the source disappear and wastes the reference image.
  • Prompt contradicting the source, such as asking for a night scene from a bright noon photo, yields muddy compromises.
  • Color drift appears after repeated passes, because each round shifts the palette slightly.
  • Rights and consent issues arise when the source image belongs to someone else or depicts a real person.
  • Misleading edits of documentary photos can spread as authentic, so provenance labeling matters.

Responsible use includes checking ownership of source images, avoiding realistic manipulation of identifiable people without consent and attaching provenance metadata, for example C2PA content credentials, when outputs are published. These steps matter more for img2img than for text-to-image because the output is anchored to a real picture.

The short version

Image-to-image means giving an AI a picture and asking it to redraw that picture in a new way. It is similar to handing an illustrator a rough sketch and saying, keep the layout, but make it look like a watercolor painting.

A single slider decides how much of the original survives. Low settings make small touch-ups, while high settings let the AI reinvent almost everything. The art lies in finding the point where the new style appears but the important parts of the original remain.

Where it makes a difference

An interior design studio prepares concepts for a client who wants to renovate a living room. The designer photographs the empty room, draws simple furniture outlines over the photo and feeds the result into an img2img workflow with the prompt describing a warm Scandinavian style with oak floors and linen textiles.

At a strength of 0.6 with depth conditioning, the model keeps the window positions, ceiling height and furniture placement while adding realistic materials and daylight. The designer generates twelve variations with different seeds in about ten minutes, then refines the client’s favorite at a lower strength of 0.3 to sharpen textures.

The client sees photorealistic options of their own room in the first meeting instead of a week later. The studio still produces final technical drawings by hand, because the generated images may contain inaccurate dimensions and furniture that does not exist in any catalog.