Hyperparameter tuning

Also known as: hyperparameter optimization

On this page 7

Hyperparameter tuning, also called hyperparameter optimization, is the systematic search for the configuration values that a learning algorithm does not learn from data, such as regularization strength, tree depth or learning rate, with the goal of maximizing performance on unseen data. In machine learning, and in artificial intelligence more broadly, it sits between model selection and final training: the algorithm fixes its parameters during fitting, while the practitioner or an automated search fixes the hyperparameters around it.

The distinction matters because hyperparameters control model capacity and therefore the bias-variance tradeoff. A well-tuned gradient boosting model or support vector machine can differ from its default configuration by several points of accuracy, while a badly designed search can produce optimistic scores that do not hold in production.

How hyperparameter tuning works

Hyperparameter tuning works as an outer optimization loop wrapped around ordinary training. Each candidate configuration is trained on part of the data and scored on held-out data, and the search keeps the configuration with the best validation score. The objective is usually noisy, expensive to evaluate and has no usable gradient, which is why tuning relies on black-box search strategies rather than calculus.

Every tuning job has four components: a search space that defines the allowed range and scale of each hyperparameter, a search strategy that proposes candidates, an evaluation protocol such as k-fold cross-validation, and a budget measured in trials, compute hours or wall-clock time. The table below separates the concepts that are most often confused.

AspectModel parametersHyperparameters
How they are setLearned by the training algorithmChosen before training starts
Typical examplesRegression weights, split thresholdsPenalty strength, number of trees, k
Optimized againstTraining lossValidation metric
QuantityThousands to millionsUsually 2 to 20
Search methodGradient descent or closed formGrid, random or Bayesian search
Parameters versus hyperparameters

Search spaces need care. Hyperparameters that span orders of magnitude, such as the regularization coefficient in ridge regression or the C and gamma values of a kernel support vector machine, should be sampled on a logarithmic scale. Integer and categorical choices, such as the split criterion of a decision tree, require strategies that handle mixed variable types.

Search strategies and their trade-offs

The search strategy decides how efficiently the budget is spent. Grid search is exhaustive but scales exponentially with the number of hyperparameters, while random search covers the important dimensions far better for the same number of trials because only a few hyperparameters usually dominate performance. Model-based methods learn from earlier trials, and early-stopping methods abandon weak candidates before they consume the full budget.

StrategyHow it proposes candidatesStrengthWeakness
Grid searchEvery combination of listed valuesSimple and reproducibleCost grows exponentially
Random searchIndependent random samplesEfficient in high dimensionsIgnores earlier results
Bayesian optimizationSurrogate model and acquisition functionFew trials neededSequential and harder to parallelize
Successive halvingMany cheap trials, survivors get more budgetFast pruning of weak settingsEarly scores may mislead
Evolutionary searchMutation and selection of a populationHandles odd search spacesNeeds many evaluations
Common search strategies compared

Bayesian optimization, which has its own entry in this topic, typically uses a Gaussian process or a tree-structured Parzen estimator as the surrogate. Hyperband combines successive halving with random sampling and is popular when a model can be trained on a partial budget, such as fewer boosting rounds or a data subsample. AutoML systems automate these strategies together with feature preprocessing and model selection.

Key hyperparameters in classic algorithms

Each algorithm family has a small set of influential hyperparameters, and tuning effort should concentrate there. Most of them adjust the strength of regularization in some form, either by penalizing coefficients, limiting tree growth or smoothing decision boundaries. The table lists the usual starting points and typical ranges.

AlgorithmKey hyperparametersTypical rangeEffect of increasing
Ridge and lasso regressionPenalty strength alpha0.0001 to 100, log scaleSimpler model, higher bias
Random forestNumber of features per split, minimum leaf sizesqrt(p) to p; 1 to 20Less correlated or smoother trees
Gradient boostingLearning rate, number of trees, maximum depth0.01 to 0.3; 100 to 5000; 3 to 10Slower learning, more capacity, deeper interactions
Support vector machineC, kernel gamma0.01 to 1000; 0.0001 to 10Tighter fit, more local boundary
k-nearest neighborsk, distance metric1 to 50Smoother boundary, higher bias
Influential hyperparameters by algorithm family

Some hyperparameters interact strongly. In gradient boosting, a lower learning rate needs more trees, so the two should be tuned jointly or the number of trees should be set by early stopping on a validation set. Ranges in the table are common starting points, not universal limits; the right range depends on data size, feature scaling and noise level.

Validation design and data leakage

Validation design determines whether the tuned score is trustworthy or inflated. Because the search selects the best of many noisy scores, the winning score is biased upward, an effect known as overfitting the validation set. The risk grows with the number of trials and shrinks with the size of the validation data.

The standard remedy is a three-way separation: tune with cross-validation on the training data, then report performance once on an untouched test set. When data is scarce, nested cross-validation runs the whole tuning procedure inside each outer fold and gives an unbiased estimate of the full pipeline. For time-ordered data, folds must respect chronology, otherwise future information leaks into training.

SchemeWhen to use itMain caveat
Single holdout splitLarge datasets, expensive modelsHigh variance on small data
k-fold cross-validationMost tabular problemsk times the training cost
Stratified k-foldClass imbalanceOnly preserves label proportions
Nested cross-validationSmall data, honest reportingVery high compute cost
Time-series splitForecasting, concept driftFewer training samples in early folds
Validation schemes for tuning

Leakage also enters through preprocessing. Feature scaling, imputation, feature selection and resampling methods such as SMOTE must be fitted inside each fold, normally by placing them in a pipeline that is tuned as a whole. Fitting them on the full dataset first lets validation folds influence training and silently inflates the score.

Limitations and common mistakes

Hyperparameter tuning cannot compensate for weak features or mislabeled data, and returns diminish quickly once the main hyperparameters are near a good region. Experienced practitioners treat it as the last refinement step, after data cleaning, feature engineering and a sensible baseline.

  • Tuning on the test set: reusing the test set for decisions turns it into a validation set and removes the honest estimate.
  • Linear ranges for scale parameters: sampling alpha or C linearly wastes most trials on large values.
  • Ignoring score variance: differences smaller than the standard deviation across folds are usually noise.
  • Optimizing the wrong metric: accuracy on imbalanced classes rewards trivial models; choose the metric the business decision depends on.
  • Unbounded budgets: thousands of trials raise compute cost and the risk of validation overfitting without reliable gains.

Reproducibility is a further concern. Recording random seeds, search spaces and every trial result allows a tuned model to be rebuilt and audited, which matters in regulated settings where model choices must be explained.

Stripped to the essentials

A learning algorithm has two kinds of settings. Some it figures out by itself from the data. Others must be chosen by a person before learning starts, such as how complex the model may become. Hyperparameter tuning is the organized trial of different choices for those second settings.

Think of tuning a radio. You turn the dial, listen and keep the position where the signal is clearest. Tuning a model works the same way, except the clarity is measured on data the model has not seen. The key point is to keep a final set of data aside and check it only once at the end, so the result is honest.

Where it is applied

A credit risk team at a mid-sized lender builds a gradient boosting model to predict loan defaults from 200,000 historical applications with a default rate of about 4 percent. The default configuration reaches an area under the ROC curve of 0.78 on five-fold stratified cross-validation, and the team sets aside the most recent quarter of applications as a test set.

The team runs 150 trials of Bayesian optimization over learning rate, maximum depth, minimum child weight and subsampling ratios, with early stopping deciding the number of trees. Scaling and category encoding sit inside the pipeline, so every fold is processed independently. The best configuration uses a learning rate of 0.03 and a depth of 5 and reaches 0.81 in cross-validation.

On the untouched recent-quarter test set, the tuned model scores 0.80, close to the validation result. The small gap confirms that the search did not overfit, and the gain over the default lets the lender approve more good applicants at the same accepted risk level.