Gradient boosting

Also known as: gradient boosted trees · GBM

On this page 7

Gradient boosting is a supervised machine learning technique that builds a strong predictive model as a sequential sum of weak learners, usually shallow decision trees, where each new learner is fitted to the negative gradient of the loss function with respect to the current predictions. In practice this means every tree corrects the residual errors left by the trees before it.

Within machine learning, gradient boosting belongs to the family of ensemble methods alongside bagging, random forests and stacking. Implementations known as gradient boosted trees or GBM remain the default choice for tabular data, where they frequently outperform deep neural networks on structured business, scientific and financial datasets.

How gradient boosting fits trees to residual errors

Gradient boosting performs gradient descent in function space rather than in parameter space. The algorithm starts from a constant prediction, such as the mean target value for regression or the log-odds of the positive class for binary classification, and then adds trees one at a time.

At each iteration the algorithm computes pseudo-residuals, the negative gradient of the loss for every training example. For squared error these equal the ordinary residuals; for log loss they equal the difference between the label and the predicted probability. A new regression tree is fitted to those pseudo-residuals, its leaf values are optimized for the chosen loss, and its output is added to the ensemble after multiplication by a learning rate (also called shrinkage).

The process can be summarized in a short sequence of steps, each of which is repeated for a fixed number of rounds or until a validation metric stops improving.

  1. Initialize the model with a constant that minimizes the loss.
  2. Compute the pseudo-residuals for every training example.
  3. Fit a shallow regression tree to the pseudo-residuals.
  4. Compute optimal leaf values for the loss function.
  5. Add the scaled tree to the ensemble and repeat.

Because the gradient is defined for any differentiable loss, the same framework handles regression, classification, ranking and survival analysis. The following table shows how common loss functions translate into the residuals that each tree learns.

Loss functionTypical taskPseudo-residualMain property
Squared errorRegressiony minus predictionSensitive to outliers
Absolute errorRobust regressionSign of y minus predictionResistant to outliers
Huber lossRegression with noiseClipped residualCompromise of both
Log lossBinary classificationy minus predicted probabilityOutputs calibrated scores
Softmax cross-entropyMulti-class classificationOne residual per classOne tree per class per round
LambdaRank objectiveLearning to rankPairwise ranking gradientOptimizes ranking metrics
Loss functions and the pseudo-residuals they produce

Key hyperparameters of gradient boosted trees

The behavior of gradient boosting is governed by a small set of interacting hyperparameters. The most important relationship is between the learning rate and the number of trees: a smaller learning rate requires more trees but usually generalizes better, a pattern that practitioners exploit by fixing a low rate and letting early stopping choose the tree count.

The table below lists the parameters that matter most in hyperparameter tuning, with ranges that work as reasonable starting points for medium-sized datasets.

HyperparameterTypical rangeEffect when increased
Learning rate0.01 to 0.3Faster fitting, higher overfitting risk
Number of trees100 to 10,000Lower bias, higher variance without early stopping
Maximum depth3 to 10Captures higher-order feature interactions
Minimum samples per leaf1 to 100Smoother, more conservative trees
Row subsampling0.5 to 1.0Less randomness and less regularization
Column subsampling0.5 to 1.0Less decorrelation between trees
L1 and L2 leaf penalties0 to 10Shrinks leaf weights toward zero
Core hyperparameters, typical ranges and their effects

Using row subsampling turns the method into stochastic gradient boosting, which reduces variance and speeds up training. Tree depth controls the order of feature interactions the model can express: depth 1 trees (stumps) produce an additive model, while depth 6 trees can represent interactions among up to six features.

Gradient boosting differs from bagging and random forests mainly in how trees are combined. Bagging trains independent deep trees in parallel and averages them to reduce variance, whereas boosting trains dependent shallow trees in sequence to reduce bias. AdaBoost, the earliest practical boosting algorithm, can be understood as a special case of gradient boosting with an exponential loss.

MethodTree trainingPrimary error reducedTypical tree depth
Gradient boostingSequential, fitted to gradientsBiasShallow, 3 to 8 levels
Random forestParallel, bootstrapped samplesVarianceDeep, often unpruned
AdaBoostSequential, reweighted samplesBiasVery shallow, often stumps
BaggingParallel, bootstrapped samplesVarianceDeep
StackingSeparate base models plus a meta-modelModel-specific errorsDepends on base models
How gradient boosting differs from neighboring ensemble techniques

Modern libraries extend the original algorithm in important ways. XGBoost adds a second-order Taylor approximation of the loss and explicit regularization of leaf weights; LightGBM uses histogram-based splits and leaf-wise tree growth for speed; CatBoost introduces ordered boosting and native handling of categorical features to limit target leakage.

When to choose gradient boosting

Gradient boosting is the strongest general baseline for structured data with mixed numeric and categorical columns, missing values and nonlinear relationships. It needs no feature scaling, handles missing values natively in most libraries and produces feature importance scores that help with feature selection.

SituationRecommended choiceReason
Tabular data, thousands to millions of rowsGradient boostingHighest accuracy with moderate tuning
Very small dataset, under a few hundred rowsRegularized linear modelLower variance and easier interpretation
Images, audio or free textDeep neural networkLearns representations from raw signals
Need for quick, robust baselineRandom forestFew hyperparameters, hard to overfit
Strict monotonic business rulesGradient boosting with constraintsMonotonic constraints enforce known directions
Decision guide for selecting a modeling approach

Limitations and common mistakes

The main weakness of gradient boosting is its tendency to overfit when the learning rate is high, trees are deep or training continues too long. Because each tree fits the remaining errors, the ensemble eventually starts memorizing noise in the training data, which is why a held-out validation set and early stopping are essential.

  • Sequential training limits parallelism across trees, although split finding within a tree is parallelized.
  • Tree ensembles cannot extrapolate beyond the range of target values seen during training.
  • Split-based feature importance is biased toward high-cardinality features; permutation or SHAP values are more reliable.
  • Encoding categorical variables with target statistics computed on the full training set causes target leakage.
  • Under concept drift, a model tuned on historical data degrades silently unless it is monitored and retrained.

Put simply

Gradient boosting works like a team of students correcting a shared answer. The first student makes a rough guess. The second student looks only at what the first got wrong and tries to fix those mistakes. The third fixes what is still wrong, and so on.

Each student is not very clever alone, but hundreds of small corrections add up to a very accurate result. The careful part is knowing when to stop, because after a while the team starts fixing random noise instead of real mistakes. Gradient boosting turns many weak guesses into one strong prediction.

In practice

A regional bank wants to predict which credit card customers will default within twelve months. The data team has about 800,000 customer records with 120 columns, including income, payment history, credit utilization and account age, many of which contain missing values.

The team trains a gradient boosted tree model with log loss, a learning rate of 0.05, maximum depth 6, 80 percent row and column subsampling, and early stopping on a time-based validation split. Training stops after roughly 1,400 trees. Monotonic constraints ensure that higher utilization never lowers predicted risk, which satisfies the model risk review.

Compared with the previous logistic regression scorecard, the new model raises the area under the ROC curve by several points and lets the bank flag high-risk accounts earlier, while SHAP explanations give credit officers a reason for every individual decision.