Generative model

On this page 7

A generative model is a machine learning model that learns the probability distribution of its training data so that it can produce new samples resembling that data, such as images, text, audio or molecules. In generative AI, it is the component that actually creates content, whether it is a diffusion model drawing an image or a language model writing a paragraph.

Generative models are usually contrasted with discriminative models, which learn only the boundary between classes. A discriminative classifier answers “is this a cat?”, while a generative model learns what cat images look like well enough to draw a new one.

What a generative model learns

A generative model learns an approximation of the data distribution p(x), or a conditional version p(x given c) when generation is guided by a prompt, label or other condition. Real data such as photographs occupy a thin, highly structured region of an enormous space of possible pixel arrays, and the model must learn where that region lies and how probability is spread across it.

Most modern models do not store the distribution as an explicit formula. They learn a sampling procedure that maps random noise into realistic outputs, often passing through a compressed latent space where similar concepts sit close together. Some families also provide an exact or approximate likelihood, which is useful for evaluation and anomaly detection.

The table below contrasts generative and discriminative modeling on the points that matter most in practice.

AspectGenerative modelDiscriminative model
What is learnedDistribution of the data itselfDecision boundary or label probability
Main outputNew samples or likelihoodsClass labels or scores
Typical examplesDiffusion models, GANs, GPTLogistic regression, image classifiers
Data demandsHigh; must model every detailLower; models only what separates classes
EvaluationHard; quality and diversity both matterStraightforward accuracy metrics
Generative and discriminative models compared

Main families of generative models

Generative models fall into several families that differ in how they represent and sample the distribution. Each family makes a distinct trade-off among sample quality, sampling speed and training stability, and modern systems often combine more than one approach, as latent diffusion models do with an autoencoder and a diffusion process.

FamilyCore mechanismMain strengthMain weakness
Autoregressive modelsPredict each element from earlier onesExact likelihood, strong for textSlow sequential sampling
Variational autoencodersEncode to a latent, decode backFast sampling, smooth latent spaceBlurry outputs without help
GANsGenerator competes with a discriminatorSharp images in one stepUnstable training, mode collapse
Normalizing flowsInvertible transforms of simple noiseExact likelihoodArchitectural constraints
Diffusion modelsLearn to reverse gradual noisingHigh quality and diversityMany sampling steps
Flow matchingLearn a velocity field from noise to dataStable training, fewer stepsStill iterative sampling
Principal generative model families

Since about 2021, diffusion and flow-based models have dominated image, video and audio generation, while autoregressive transformers dominate text. The boundary is blurring: diffusion transformers use the transformer architecture, and autoregressive image generation over discrete visual tokens has become competitive again.

How a generative model is trained and sampled

Training a generative model means adjusting its parameters so that generated samples become statistically indistinguishable from real data. The training objective depends on the family: maximum likelihood for autoregressive models and flows, an evidence lower bound for variational autoencoders, an adversarial game for GANs and a denoising or score-matching loss for diffusion models.

Sampling is a separate stage with its own controls. A random seed fixes the initial noise, so the same seed and settings reproduce the same output. Conditioning techniques such as classifier-free guidance strengthen adherence to a prompt, typically at guidance scales between about 3 and 12 for image models, at some cost to diversity.

The following table lists the sampling controls that most often change results.

ControlApplies toEffect of increasing it
Sampling stepsDiffusion and flow modelsFiner detail, up to a plateau, slower
Guidance scaleConditional diffusion modelsCloser prompt match, less variety
TemperatureAutoregressive modelsMore diverse and less predictable output
Denoising strengthImage-to-image editingLarger departure from the source image
Common sampling controls and their effects

Evaluating generative model quality

Evaluating a generative model is difficult because a good model must achieve fidelity, diversity and controllability at the same time. A model that reproduces a few perfect images scores well on fidelity but fails on diversity, while a model that memorizes training data may look excellent yet raise privacy and copyright concerns.

MeasureWhat it capturesKnown limitation
Fréchet Inception DistanceDistance between real and generated feature statisticsDepends on a specific image network
Precision and recallFidelity and coverage of the data distributionSensitive to feature choice
Likelihood or perplexityHow well the model explains held-out dataWeakly linked to perceived quality
CLIP scoreAgreement between image and text promptInherits biases of the scoring model
Human preference studiesOverall perceived qualityCostly and hard to reproduce
Widely used evaluation measures

In practice, teams combine automatic metrics with blind human comparisons, because no single metric tracks perceived quality reliably across domains.

Limitations, risks and common mistakes

Generative models inherit the gaps and biases of their training data, and they can only interpolate within what they have learned. Several failure modes recur across families:

  • Mode collapse: a GAN covers only a few types of output and ignores the rest of the distribution.
  • Memorization: rare or duplicated training items can be reproduced almost exactly.
  • Model collapse: training repeatedly on AI-generated content gradually narrows and degrades the learned distribution.
  • Weak consistency: characters, objects or frames drift across a series of images or a video.
  • Plausible but wrong output: fluent text or realistic images that contain factual or physical errors.

A common mistake is to treat a generative model as a database of facts or images. It stores statistical regularities, not records, so outputs must be verified when accuracy matters. Responsible deployment also requires provenance labeling, content filtering and clear rules on training data rights.

In plain words

A generative model is a computer program that studies a huge collection of examples, such as photos or texts. It learns what typical examples look like. It can then make brand-new examples that fit the same pattern. Think of a cook who has tasted thousands of soups and can invent a new one that still tastes like soup.

The model does not copy a specific item from its collection. It creates something new from learned patterns. That is why its results can be impressive. It is also why its output can look right while being wrong.

A real-world example

A furniture retailer needs lifestyle images for 600 sofa variants, and photographing each one in several room settings would take months. The design team fine-tunes a latent diffusion model on about 40 studio photos per product family, then uses inpainting to place each sofa into generated living rooms while keeping the real product pixels unchanged.

They generate images at 30 sampling steps with a guidance scale of 7, fixing the random seed for each room style so the series stays consistent. A reviewer checks every image for distorted legs, wrong fabric textures and impossible shadows, and rejects roughly one image in five. The catalog is completed in about three weeks, and every published image carries an internal label recording that it was generated.