Gaussian process

Also known as: Gaussian process regression

On this page 7

A Gaussian process is a collection of random variables, any finite subset of which has a joint Gaussian distribution, used in machine learning as a prior over functions for regression and classification. Instead of fitting a fixed set of weights, the model describes which functions are plausible and updates that belief with data, so every prediction comes with a mean and a variance.

In practice the method is usually called Gaussian process regression. It is a nonparametric Bayesian model, prized for small and medium data sets where calibrated uncertainty matters, such as surrogate modeling, spatial statistics and Bayesian optimization.

How a Gaussian process defines a distribution over functions

A Gaussian process is fully specified by a mean function and a covariance function, also called the kernel. The mean function gives the expected value at each input and is often set to zero after centering the targets. The kernel states how strongly the outputs at two inputs are correlated, which encodes assumptions about smoothness, scale and periodicity.

Given training inputs, the kernel produces a covariance matrix over the observed points. Because all values are jointly Gaussian, conditioning on observed targets yields a closed-form posterior: the predictive mean is a weighted combination of training targets, and the predictive variance shrinks near observed data and grows in regions without data.

Observation noise is handled by adding a noise variance term to the diagonal of the covariance matrix. This term also stabilizes the numerical inversion and prevents the model from interpolating noisy measurements exactly.

Kernels and their hyperparameters

The choice of kernel is the central modeling decision in a Gaussian process, because it determines which functions the prior considers likely. The table below compares widely used kernels and the behavior each one assumes.

KernelFunctions it favorsTypical use
Squared exponential (RBF)Infinitely smooth curvesDefault for smooth physical responses
Matérn 3/2 and 5/2Rougher, finitely differentiable curvesEngineering data and Bayesian optimization
PeriodicRepeating patterns with fixed periodSeasonal or cyclic signals
LinearStraight lines through the originTrends; equals Bayesian linear regression
Rational quadraticMixture of length scalesData with variation at several scales
White noiseIndependent random noiseAdded to model measurement error
Common covariance functions

Kernels can be added and multiplied to build composite structures, for example a linear trend plus a periodic component plus noise. Each kernel has hyperparameters: the length scale sets how far apart inputs must be before their outputs become nearly independent, and the signal variance sets the vertical amplitude of the functions.

HyperparameterSmall valueLarge value
Length scaleWiggly fits, risk of overfittingSmooth fits, risk of underfitting
Signal varianceNarrow range of output valuesWide range of output values
Noise varianceFit passes close to every pointFit treats variation as noise
PeriodFast repetitionSlow repetition
Key hyperparameters and their effects

Hyperparameters are usually learned by maximizing the log marginal likelihood, which automatically balances data fit against model complexity. This built-in Occam’s razor is a major advantage, although the likelihood surface can have several local optima, so multiple random restarts are common.

A Gaussian process shares ideas with several classic machine learning algorithms but differs in how it represents uncertainty and complexity. The comparison below places it among neighboring methods.

MethodUncertainty estimateScaling with dataRelationship to Gaussian processes
Kernel ridge regressionNoneCubic in sample sizeSame predictive mean as a GP
Support vector machineNone by defaultRoughly quadratic to cubicUses kernels but optimizes a margin
Bayesian linear regressionFull posteriorLinear in sample sizeA GP with a linear kernel
Random forestHeuristic spread across treesNear linearNo explicit function prior
Neural networkNone without extra methodsLinear per epochInfinitely wide networks converge to GPs
Gaussian process regression and related models

The link to the kernel trick is direct: a Gaussian process works implicitly in the feature space a kernel defines. The name sometimes causes confusion with the Gaussian mixture model, which is a clustering method for densities and is unrelated to function priors.

Where Gaussian processes are used

Gaussian processes excel when data are expensive to collect and decisions depend on knowing how certain a prediction is. They are a standard surrogate model in Bayesian optimization, where the predictive variance drives acquisition functions such as expected improvement for tuning hyperparameters or experimental settings.

  • Geostatistics, where Gaussian process regression is known as kriging and interpolates mineral grades or pollution levels.
  • Computer experiments, where a GP emulates a slow physical simulation from a few hundred runs.
  • Time series with irregular sampling, using periodic and trend kernels.
  • Active learning, choosing the next measurement where uncertainty is highest.
  • Robotics and control, modeling system dynamics with safe exploration.

For classification, the Gaussian likelihood no longer applies, so approximate inference such as the Laplace approximation or expectation propagation is needed. These approximations make GP classification less common than GP regression.

Computational limits and common pitfalls

The main limitation of an exact Gaussian process is cost: inverting or factorizing the covariance matrix takes cubic time in the number of points and quadratic memory. Exact inference becomes impractical beyond roughly ten thousand points on ordinary hardware, although GPU methods push this boundary further.

ProblemCauseRemedy
Slow trainingCubic cost of matrix factorizationSparse inducing points or structured kernels
Numerical instabilityNearly singular covariance matrixAdd a small jitter to the diagonal
Poor fit in high dimensionsDistance-based kernels lose meaningAutomatic relevance determination or dimensionality reduction
Overconfident predictionsWrong kernel or fixed noise levelLearn noise and compare kernels
Unscaled inputsOne feature dominates distancesApply feature scaling before fitting
Problems and remedies in Gaussian process modeling

The curse of dimensionality affects Gaussian processes strongly: with more than about twenty informative input dimensions, standard stationary kernels struggle. A further pitfall is treating predictive variance as ground truth; it is only as reliable as the kernel assumptions behind it.

A plain-language summary

Imagine drawing a smooth curve through a few dots on a page. Many curves could pass through them. A Gaussian process keeps all reasonable curves in mind at once and reports both the most likely value and how unsure it is.

Near the dots, the curves agree, so the model is confident. Far from any dot, the curves spread apart, so the model says honestly that it does not know much there. That honesty is the main reason people use it: it tells you where to trust the prediction and where to collect more data.

An example from practice

A materials engineering team wants the strongest blend of a polymer, controlled by three variables: curing temperature, curing time and filler percentage. Each test sample takes two days to prepare and break, so the lab budget allows only about forty experiments.

The team runs twelve initial experiments spread across the range and fits Gaussian process regression with a Matérn 5/2 kernel and learned length scales. The model predicts tensile strength everywhere in the design space with error bars. An expected improvement rule then selects each next experiment, balancing regions with high predicted strength against regions with high uncertainty.

After twenty-eight experiments, the team finds a blend about 15 percent stronger than its previous best recipe. The learned length scales also show that curing time matters far less than temperature, so later work fixes time and saves further testing.