CatBoost is an open-source gradient boosting library for decision trees that trains accurate models on tabular data and is known for its native handling of categorical features. Among AI frameworks & tools, it belongs to the classical machine learning family alongside XGBoost and LightGBM rather than to deep learning frameworks such as PyTorch or TensorFlow.
Yandex released CatBoost in 2017 under the Apache 2.0 license. Its name combines “category” and “boosting”, which reflects its two signature ideas: ordered target statistics for encoding categories and ordered boosting for reducing a subtle bias called prediction shift. It offers Python, R, command-line and Java interfaces, supports CPU and GPU training, and often produces strong results with default hyperparameters.
How CatBoost builds a gradient boosted ensemble
CatBoost builds an ensemble by adding one tree at a time, where each new tree is fitted to the gradients of the loss function computed from the predictions of all previous trees. The final prediction is the sum of all tree outputs scaled by the learning rate, so every tree corrects a small part of the remaining error.
Unlike most boosting libraries, CatBoost grows oblivious trees, also called symmetric trees. In an oblivious tree, every node at the same depth uses the same split feature and threshold. A tree of depth 6 therefore contains only six split conditions and 64 leaves, and the leaf index can be computed with fast bit operations. This structure acts as a built-in regularizer and makes inference very fast, at the cost of some flexibility per tree.
The following table summarizes the main components of the training process and their purpose.
| Component | What it does | Effect on the model |
|---|---|---|
| Oblivious trees | Use one split rule per depth level | Fast inference, lower variance |
| Ordered target statistics | Encode categories from earlier rows only | No target leakage in encoding |
| Ordered boosting | Compute residuals with models that never saw the row | Less prediction shift |
| Feature combinations | Join categorical features greedily during tree growth | Captures interactions between categories |
| Quantization | Bins numeric features into borders | Faster split search, less memory |
Categorical features and ordered target statistics
CatBoost encodes categorical features internally, so users pass raw string columns through the cat_features parameter instead of preparing one-hot or label encodings themselves. For low-cardinality columns, it applies one-hot encoding up to the `one_hot_max_size` threshold; for the rest, it computes target statistics, which replace each category with a smoothed average of the target.
A naive target encoding uses the target value of the same row, which causes target leakage and inflated training accuracy. CatBoost avoids this by drawing a random permutation of the training rows and computing each row’s statistic only from rows that appear earlier in that permutation. A prior value smooths rare categories, so a category seen once does not receive an extreme estimate. Several permutations are used across the trees to reduce the variance this procedure introduces.
The table below compares the encoding strategies a practitioner might otherwise apply by hand.
| Encoding | Leakage risk | Suitable cardinality | Main drawback |
|---|---|---|---|
| One-hot encoding | None | Low, under about 10 levels | Very wide, sparse matrices |
| Label encoding | None | Any | Imposes a false order |
| Naive target encoding | High | Medium to high | Overfits on rare categories |
| Ordered target statistics | Low | Medium to very high | Slower training, randomness |
Ordered boosting and prediction shift
Ordered boosting is the CatBoost technique that computes the gradient for each training row with a model that was not trained on that row. In standard gradient boosting, the same rows serve both to fit a tree and to estimate the residuals for the next tree, which makes the residuals on training data systematically smaller than on new data. This bias is called prediction shift, and it is a quiet source of overfitting.
CatBoost addresses it with the same permutation idea used for categories: supporting models are maintained so that each row’s residual depends only on preceding rows. Because this is expensive, the library offers two boosting_type modes. Ordered mode is most valuable on small and noisy datasets, while Plain mode is the faster default on large datasets, where prediction shift matters less.
Key hyperparameters and typical values
CatBoost is known for robust defaults, but a small set of parameters controls most of its behavior. The table lists the parameters that practitioners tune first, with their defaults and usual ranges.
| Parameter | Default | Typical range | What it controls |
|---|---|---|---|
iterations | 1000 | 500 to 10000 | Number of trees |
learning_rate | Chosen automatically | 0.01 to 0.3 | Step size per tree |
depth | 6 | 4 to 10 | Tree depth and interaction order |
l2_leaf_reg | 3 | 1 to 10 | L2 penalty on leaf values |
border_count | 254 on CPU | 32 to 255 | Bins per numeric feature |
one_hot_max_size | 2 on CPU | 2 to 255 | Cardinality limit for one-hot |
The most effective tuning step is usually to lower the learning rate and raise the iteration count while using the overfitting detector with early_stopping_rounds on a validation set. Depth beyond 10 rarely helps, because oblivious trees grow exponentially in leaf count with each level.
CatBoost compared with XGBoost and LightGBM
CatBoost, XGBoost and LightGBM are the three dominant gradient boosting libraries, and on many tabular benchmarks their tuned accuracy is close. They differ mainly in tree structure, categorical support and speed profile, as the following table shows.
| Aspect | CatBoost | XGBoost | LightGBM |
|---|---|---|---|
| Tree growth | Symmetric, level by level | Depth-wise by default | Leaf-wise |
| Categorical features | Native ordered statistics | Native support in newer versions | Native split grouping |
| Default quality | Strong without tuning | Needs moderate tuning | Needs moderate tuning |
| Training speed on CPU | Moderate | Moderate | Usually fastest |
| Inference speed | Very fast | Fast | Fast |
In practice, CatBoost is a natural first choice for data dominated by categorical columns, such as user IDs, product codes or regions. LightGBM is often preferred for very large numeric datasets where training time dominates, and scikit-learn remains the reference for simpler pipelines.
Practical use, deployment and limitations
CatBoost supports classification, regression and ranking, with loss functions such as Logloss, RMSE, MultiClass and YetiRank. It can also process text features and embedding features, provides built-in SHAP values for explanation, and exports models to formats including ONNX, Core ML, JSON, C++ and Python code. The Pool class bundles data, categorical column indices and weights for efficient reuse.
- Training cost: ordered mode and many categorical combinations can make training several times slower than LightGBM.
- Memory use: high-cardinality categories with feature combinations increase memory consumption, especially on GPU.
- Not for raw unstructured data: images, audio and long text are better served by deep learning frameworks.
- Export limits: ONNX and Core ML export do not cover every model with categorical features, so the deployment path must be tested early.
- Validation discipline: target statistics reduce leakage, but they do not replace a proper time-based split for temporal data.
The idea in brief
CatBoost is a tool that learns patterns from tables of data. Think of spreadsheets of customers or orders. It builds hundreds of small decision trees. Each tree fixes a few mistakes of the trees before it.
Its special talent is text labels like city names. People do not have to turn them into numbers by hand. It also avoids “peeking” at the answers while it learns. That helps it perform well on new data, not only on the data it studied.
A practical scenario
A data team at a regional insurance company wants to predict which car insurance customers will file a claim in the next year. Their table has 400,000 rows and 35 columns, and more than half of the columns are categorical: vehicle model, postal district, occupation, sales channel and policy type. Several of these columns have thousands of distinct values.
Instead of building a one-hot pipeline with tens of thousands of sparse columns, the team passes the raw columns to CatBoost through cat_features, sets the loss to Logloss, uses a learning rate of 0.05 with up to 3,000 iterations and enables early stopping on a time-based validation set. Training on a single GPU takes under 15 minutes.
The resulting model improves the area under the ROC curve by several points over the previous logistic regression. Built-in SHAP values show that postal district and vehicle age drive most predictions, which the actuaries can review before the model is deployed for pricing decisions.