Mean squared error

Also known as: MSE

On this page 8

Mean squared error (MSE) is the average of the squared differences between predicted values and observed values. In model evaluation for artificial intelligence and statistics, it is the standard measure of how far a regression model’s numerical predictions fall from the truth, and it also serves as the most common loss function for training regression models.

Because each error is squared, MSE is always non-negative, equals zero only for perfect predictions and penalizes large errors much more heavily than small ones. That property makes it informative and mathematically convenient, but it also makes the metric sensitive to outliers and expressed in squared units that are hard to interpret directly.

How mean squared error is calculated

MSE is computed by taking each residual, the difference between the true value and the prediction, squaring it, summing the squares and dividing by the number of observations n. Written compactly, it is MSE = (1/n) * sum((y_i - yhat_i)^2). The result is in the square of the target unit, so a model predicting prices in dollars yields an MSE in squared dollars.

The small worked calculation below shows how one large miss dominates the total. Four predictions are compared with their true values.

ObservationTrue valuePredictionSquared error
Case 110111
Case 220184
Case 330311
Case 44050100
Worked calculation of MSE for four predictions

The squared errors sum to 106, so the MSE is 26.5. A single error of 10 contributes 100 of the 106 points, which illustrates the quadratic penalty: doubling an error quadruples its contribution.

What MSE measures statistically

MSE measures the expected squared distance between an estimator and the target, and it decomposes into interpretable parts. For a predictor evaluated over many possible training sets, expected MSE equals squared bias plus variance plus irreducible noise. This bias-variance decomposition explains why both underfitting and overfitting raise test MSE.

Minimizing MSE leads a model to predict the conditional mean of the target given the inputs. This is the right goal when errors are roughly symmetric and normally distributed: under a Gaussian noise assumption, minimizing MSE is equivalent to maximum likelihood estimation. When the distribution is skewed or heavy-tailed, the mean may be a poor summary, and other losses can serve better.

MSE is also the basis of the coefficient of determination, R squared, which equals one minus the ratio of model MSE to the variance of the target. That ratio puts MSE on a scale-free footing and allows comparison across datasets with different units.

Mean squared error compared with other regression metrics

MSE belongs to a family of error metrics that weigh mistakes differently. The table compares it with the closest neighbors used in regression evaluation.

MetricUnitSensitivity to outliersBest suited for
Mean squared errorSquared target unitHighTraining loss, penalizing large misses
Root mean squared errorTarget unitHighReporting error in readable units
Mean absolute errorTarget unitModerateRobust typical-error reporting
Huber lossMixedTunableTraining with occasional outliers
R squaredUnitlessHighExplained variance across datasets
Mean absolute percentage errorPercentUnstable near zero targetsBusiness forecasting with positive values
MSE and related regression metrics

Root mean squared error is simply the square root of MSE, so the two always rank models identically; RMSE is preferred for reporting because it shares the unit of the target. Mean absolute error grows linearly with each miss and corresponds to predicting the conditional median rather than the mean.

MSE as a loss function in training

As a training objective, MSE is attractive because it is smooth and differentiable everywhere. Its gradient with respect to a prediction is proportional to the residual itself, so large errors produce large corrective updates and small errors produce gentle ones. For linear regression, minimizing MSE has a closed-form solution known as ordinary least squares.

In deep learning, MSE appears in regression heads, autoencoder reconstruction, value functions in reinforcement learning and the noise-prediction objective of diffusion models. For classification, however, cross-entropy is generally preferred: pairing MSE with sigmoid or softmax outputs produces weak gradients for confident wrong predictions, which slows learning.

Interpreting and using MSE correctly

An MSE value has no meaning in isolation; it must be compared with a baseline on the same data and scale. The decision table lists typical situations and the appropriate way to use or replace the metric.

SituationRecommended approachReason
Large errors are costlyUse MSE or RMSEQuadratic penalty matches the cost
Data contain gross outliersPrefer MAE or Huber lossMSE would be dominated by a few points
Comparing across datasetsReport R squared or normalized RMSERaw MSE depends on target scale
Targets span orders of magnitudeCompute MSE on log-transformed targetsRelative errors become comparable
Communicating to stakeholdersReport RMSE in original unitsSquared units are hard to interpret
When to rely on MSE and when to adjust

A useful baseline is the MSE of a model that always predicts the training mean, which equals the variance of the target. Any useful regression model should score clearly below that value on a held-out validation set, ideally checked with cross-validation.

Limitations and common pitfalls

MSE is simple, but several recurring mistakes distort conclusions drawn from it.

  • Ignoring scale: an MSE of 0.04 may be excellent or terrible depending on whether targets range from 0 to 1 or from 0 to 0.1.
  • Evaluating on training data: low training MSE says little about generalization and can hide overfitting.
  • Comparing MSE and RMSE values directly: they live on different scales, so one must be converted first.
  • Overlooking outlier influence: a handful of extreme points can make a good model look poor; inspect the residual distribution as part of error analysis.
  • Assuming symmetric costs: MSE treats over-prediction and under-prediction equally, which is wrong for problems such as inventory planning.

The idea in brief

Mean squared error tells you how wrong a model’s number guesses are on average. For every guess, you measure the miss, multiply it by itself and then take the average of all those results.

Squaring makes big misses count much more than small ones. Missing by 1 adds 1, but missing by 10 adds 100. A lower score means better guesses, and zero means every guess was exactly right.

A practical scenario

A regional energy utility builds a model that forecasts next-day electricity demand for each hour, in megawatts. On a held-out month of 720 hourly forecasts, the team computes an MSE of 400 squared megawatts, which corresponds to an RMSE of 20 megawatts against a typical load of about 1,000 megawatts.

A naive baseline that repeats yesterday’s demand scores an MSE of 1,600, so the model cuts squared error by 75%. Error analysis then shows that most of the remaining MSE comes from a few heat-wave hours with misses above 80 megawatts. Because large shortfalls are expensive for grid balancing, the team keeps MSE as the training loss and adds temperature extremes as features, reducing test MSE to 290 in the next iteration.