One-hot encoding

Also known as: dummy variables

On this page 7

One-hot encoding is a feature engineering technique in machine learning that represents a categorical variable as a set of binary indicator columns, one column per category, where exactly one column holds the value 1 and all others hold 0. The resulting columns are often called dummy variables, especially in statistics and regression analysis.

Most learning algorithms operate on numbers and assume that numeric values carry order and magnitude. A column that stores colors as 1, 2 and 3 would falsely suggest that blue lies between red and green. One-hot encoding removes that false ordinal relationship by giving every category its own independent dimension.

How one-hot encoding transforms a categorical feature

One-hot encoding works in two steps: the encoder first learns the vocabulary of distinct categories from the training data, then maps each value to a vector of length equal to that vocabulary. A feature with k categories becomes k columns, and each row has a single 1 in the position of its category.

The table below shows how a small color feature with three categories is expanded into three indicator columns.

Original valueRed columnGreen columnBlue column
Red100
Green010
Blue001
Green010
One-hot encoding of a three-category color feature

Geometrically, every category becomes a corner of a unit hypercube, and all categories sit at the same Euclidean distance of the square root of 2 from each other. This equal spacing is exactly what a nominal variable needs: no category is closer to another unless the data later teaches the model so.

Because most entries are zero, encoded matrices are usually stored in a sparse matrix format such as compressed sparse row. Sparse storage keeps memory proportional to the number of rows rather than to rows multiplied by categories, which matters once a feature has thousands of levels.

Full-rank encoding and the dummy variable trap

The choice between k columns and k minus 1 columns depends on whether the model includes an intercept and how it is regularized. With all k columns and an intercept, the indicator columns always sum to 1, which creates perfect multicollinearity, known as the dummy variable trap.

In ordinary least squares and unregularized logistic regression this makes the design matrix singular, so coefficients are not uniquely identifiable. Dropping one reference category fixes the problem: each remaining coefficient then measures the difference from the reference level.

The following table summarizes which variant suits common model families.

Model familyRecommended variantReason
Unregularized linear modelsk minus 1 columnsAvoids a singular design matrix
Regularized linear modelsk columnsPenalty resolves collinearity and treats levels symmetrically
Tree-based modelsk columnsSplits are unaffected by collinearity
Neural networksk columns or embeddingsRedundancy is harmless to gradient training
Distance-based modelsk columnsKeeps all categories equidistant
Choosing between k and k minus 1 indicator columns

With ridge regression, lasso regression or elastic net, keeping all k columns is usually preferable, because dropping a column makes the penalty depend on an arbitrary reference choice.

How one-hot encoding compares with other categorical encodings

One-hot encoding is the default choice for nominal features with low to moderate cardinality, but it is one of several encoding strategies. Each alternative trades interpretability, dimensionality and leakage risk differently.

EncodingOutput sizeBest suited forMain drawback
One-hot encodingk columnsNominal features, up to a few hundred levelsDimensionality grows with cardinality
Ordinal encoding1 columnOrdered categories and tree modelsImposes order on nominal data
Target encoding1 columnHigh-cardinality featuresTarget leakage without cross-fitting
Feature hashingFixed m columnsVery large or open vocabulariesHash collisions merge categories
Learned embeddingsd dense columnsDeep learning with many levelsRequires training data and tuning
Common categorical encodings compared

Ordinal encoding is appropriate only when categories have a genuine natural order, such as small, medium and large. Target encoding replaces each category with a smoothed mean of the target and must be computed out of fold to avoid leakage. Feature hashing bounds dimensionality but sacrifices the ability to map columns back to categories.

Applying one-hot encoding in a production pipeline

In practice, one-hot encoding must be fitted on training data only and then applied unchanged to validation, test and live data. Fitting on the full dataset leaks information about category frequencies and can hide problems that appear only in production.

  • Handle unseen categories explicitly, for example by mapping them to an all-zero vector or to a dedicated “other” column.
  • Group rare categories below a frequency threshold, often 0.5 to 1 percent of rows, into one bucket to reduce noise and dimensionality.
  • Treat missing values as a category when missingness itself carries signal.
  • Keep the column order fixed and versioned with the model so that scoring code produces identical features.
  • Store output as sparse matrices whenever cardinality exceeds a few dozen levels.

Feature scaling is usually unnecessary for indicator columns, because they already lie in the range 0 to 1. Mixing them with standardized continuous features is acceptable, although some practitioners scale indicators as well when using strong L1 or L2 penalties so that every feature faces a comparable penalty.

Limitations and common mistakes

The main limitation of one-hot encoding is that it scales poorly with cardinality. A feature such as postal code or product identifier can produce tens of thousands of sparse columns, which slows training and worsens the curse of dimensionality for distance-based methods such as k-nearest neighbors.

The table below lists typical symptoms, their causes and the usual remedies.

SymptomLikely causeFix
Scoring fails on new dataUnseen category at inferenceConfigure the encoder to ignore unknown values
Unstable or huge coefficientsDummy variable trapDrop a reference level or add regularization
Memory errors during trainingDense output for high cardinalityUse sparse output or hashing
Feature importance split across columnsOne feature spread over many indicatorsAggregate importance per original feature
Train and test columns differEncoding fitted separately per splitFit once on training data and reuse
Frequent one-hot encoding problems and fixes

Tree ensembles such as random forest and gradient boosting can also suffer, because one-hot columns force many shallow, unbalanced splits. Some gradient boosting libraries therefore offer native categorical handling, which often trains faster and generalizes better than wide indicator matrices.

Stripped to the essentials

Computers learn from numbers, not words. If a dataset says a car is red, green or blue, the model needs those colors as numbers. Giving them the numbers 1, 2 and 3 would wrongly hint that blue is bigger than red.

One-hot encoding solves this with a row of switches. Each color gets its own switch, and only one switch is on at a time. A red car turns on the red switch and leaves the others off. Every category is treated as equal and separate, which is exactly how labels like colors or city names should be treated.

Where it is applied

A data science team at an online retailer builds a model that predicts whether a customer will return a purchased item. The dataset includes the payment method with five values, the device type with three values and the product category with 40 values, alongside numeric features such as price and delivery time.

The team one-hot encodes payment method and device type into eight indicator columns. For product category, it groups the 12 rarest categories into an “other” bucket, which leaves 29 columns. The encoder is fitted on the training split, configured to ignore unknown categories, and saved with the model.

When a new product category is introduced three months later, scoring continues without errors, because the unseen value maps to an all-zero vector. A regularized logistic regression trained on these features gives the merchandising team readable coefficients per category, showing which product groups carry the highest return risk.