Decision boundary

On this page 8

A decision boundary is the set of points in feature space where a classifier is indifferent between two or more classes, so that crossing it changes the predicted label. In machine learning, every classification model, from logistic regression to a random forest, implicitly divides the input space into decision regions, and the boundaries between those regions describe exactly how the model separates one class from another.

The shape of the boundary is determined by the model family, the features it receives and the amount of regularization applied during training. Studying it reveals whether a model is too rigid, too flexible or well matched to the data, which makes the decision boundary one of the most practical diagnostic ideas in classical machine learning and artificial intelligence.

How a classifier forms a decision boundary

A decision boundary arises wherever the model’s scores for competing classes are equal. For a binary probabilistic classifier, it is the level set where the predicted probability equals the threshold, usually 0.5; for a model that outputs a score f(x), it is the set where f(x) = 0. In multi-class problems, each pair of neighboring regions shares a boundary on which the two highest class scores tie.

The boundary is a property of the trained function, not of the data. Two models trained on identical samples can draw very different boundaries that both classify the training points correctly, and they will disagree mainly on new inputs that fall in sparse regions. This is why boundary geometry is closely tied to generalization: it shows how the model fills the empty space between observed examples.

The following table summarizes how the boundary is defined for common output types.

Model outputBoundary conditionTypical example
Probability for one classp(y = 1 | x) equals the thresholdLogistic regression
Signed margin scorew · x + b = 0Linear support vector machine
Class scores per labelTop two scores are equalSoftmax or one-vs-rest models
Majority voteVote counts tie between classesk-nearest neighbors, bagging
Leaf assignmentSplit thresholds on single featuresDecision tree
Where the boundary lies for different model outputs

Linear and nonlinear decision boundaries

A linear decision boundary is a hyperplane: a line in two dimensions, a plane in three and a flat surface of dimension d − 1 in d dimensions. Logistic regression, the linear support vector machine and linear discriminant analysis all produce hyperplanes in the original feature space. They work well when classes are roughly linearly separable and fail when the true separation curves or wraps around.

A nonlinear decision boundary can bend, fold or form closed regions. Models obtain this flexibility in different ways: polynomial features expand the input, the kernel trick maps data implicitly into a higher-dimensional space where a hyperplane suffices, trees build axis-aligned staircases, and k-nearest neighbors traces a piecewise boundary made of Voronoi cell edges. Quadratic discriminant analysis yields quadratic surfaces because each class keeps its own covariance matrix.

The table compares the boundary shapes typical of the main model families.

Model familyBoundary shapeMain control of flexibility
Logistic regressionSingle hyperplaneFeature set, regularization strength
Linear discriminant analysisHyperplane from shared covarianceCovariance estimate, shrinkage
Support vector machine with kernelSmooth curved surfaceKernel type, C and gamma
Decision treeAxis-aligned rectangular cellsMaximum depth, minimum leaf size
Random forest and gradient boostingFine-grained, near-smooth staircaseNumber of trees, depth, learning rate
k-nearest neighborsJagged piecewise-linear edgesValue of k, distance metric
Naive Bayes classifierLinear or quadratic, depending on likelihoodDistribution assumptions, smoothing
Boundary shapes by model family

Decision boundaries and the bias-variance tradeoff

The complexity of a decision boundary is a visual expression of the bias-variance tradeoff. A boundary that is too simple, such as a straight line through concentric rings, underfits and shows high bias. A boundary that snakes around individual training points overfits, shows high variance and usually performs worse on held-out data.

Hyperparameters move the boundary along this spectrum in predictable directions. Increasing regularization strength in ridge or lasso logistic regression shrinks weights and smooths the boundary; a small k in nearest neighbors produces islands around single points; a large gamma in an RBF kernel lets the boundary curve tightly around each sample. The table lists the most common levers.

HyperparameterChangeEffect on the boundary
Regularization strength (λ)IncreaseSmoother, closer to linear
SVM penalty CIncreaseTighter fit, fewer margin violations
RBF kernel gammaIncreaseMore local, more wiggly
k in k-nearest neighborsIncreaseSmoother, fewer isolated islands
Tree maximum depthIncreaseMore cells, finer staircase
Polynomial degreeIncreaseMore curvature, higher overfitting risk
Hyperparameters that change boundary complexity

The right complexity is chosen by cross-validated performance, not by how plausible the boundary looks in a plot. Hyperparameter tuning and Bayesian optimization search this space systematically.

Margins, thresholds and probability calibration

The margin is the distance between the decision boundary and the closest training points. Support vector machines explicitly maximize this margin, on the reasoning that a wide gap between classes leaves more room for noise in new data. Points lying on the margin edge are the support vectors, and they alone determine where the boundary sits.

For probabilistic classifiers, the boundary location depends on the decision threshold. Moving the threshold from 0.5 to 0.2 shifts the boundary toward the negative class, raising recall at the cost of precision. With class imbalance, the default threshold often places the boundary badly, so practitioners tune it on validation data, adjust class weights or rebalance training data with methods such as SMOTE. A boundary built on poorly calibrated probabilities may separate classes well yet produce misleading confidence values near it.

Visualizing and diagnosing the boundary

In two dimensions, the boundary is plotted by evaluating the model on a dense grid and coloring each grid cell by its predicted class or probability. This technique is invaluable for teaching and for small diagnostic studies, but it does not scale directly to dozens of features.

For high-dimensional data, practitioners project points onto two components with principal component analysis, t-SNE or UMAP, or they plot partial slices in which two features vary while the others stay fixed. Both views are approximations: a projection can make well-separated classes appear mixed, and a slice shows only one cross-section of a far more complex surface. Inspecting the points with predicted probabilities near the threshold is often more informative than any picture.

Limitations and common misconceptions

Several recurring mistakes arise when practitioners reason about decision boundaries. The table pairs each symptom with its usual cause and remedy.

SymptomLikely causeRemedy
Boundary dominated by one featureMissing feature scalingStandardize inputs before distance-based models
Perfect training accuracy, poor test accuracyOverly complex boundaryIncrease regularization or reduce depth
Minority class almost never predictedThreshold set for balanced classesTune threshold or reweight classes
Accuracy drops months after deploymentConcept drift moved the true boundaryMonitor and retrain on recent data
Confident predictions far from training dataExtrapolation beyond observed regionAdd out-of-distribution checks
Typical boundary problems and remedies
  • A linear boundary in a transformed space is not linear in the original space; kernels and polynomial features rely on exactly this effect.
  • Distance from the boundary is not the same as calibrated confidence, especially for SVM scores.
  • A clean boundary on a two-dimensional plot does not prove good generalization in the full feature space.

Explained without jargon

Imagine sorting fruit on a table by drawing a chalk line: apples go on one side, oranges on the other. A decision boundary is that chalk line drawn by a computer model. Whenever a new item lands on the table, the model checks which side of the line it falls on and gives its answer.

Some models can only draw straight lines, while others can draw curves or many small boxes. A line that is too simple misses obvious patterns, and a line that twists around every single fruit memorizes the table instead of learning the rule. The goal is a line that will still sort the next basket correctly.

A typical use case

A regional bank builds a model to flag possibly fraudulent card transactions using two strong features, transaction amount and distance from the cardholder’s usual location, plus about 30 weaker signals. An initial logistic regression draws a straight boundary and misses fraud involving small amounts far from home, because the true separation curves around that corner of feature space.

The analytics team switches to gradient boosting, plots the boundary on a two-feature slice and sees it now captures the curved region. Training accuracy climbs to 99.8 percent, but validation recall falls, revealing that the boundary has wrapped around individual training cases. After limiting tree depth to four and lowering the decision threshold from 0.5 to 0.3 to account for the rare fraud class, the team reaches a stable recall of about 85 percent with an acceptable false-alarm rate, and it schedules monthly retraining because fraud patterns shift the boundary over time.