A Naive Bayes classifier is a probabilistic classification model that applies Bayes’ theorem to predict the most likely class of an example, under the simplifying assumption that all features are conditionally independent given the class. In machine learning, naive Bayes is one of the oldest and fastest supervised methods, and it remains a standard baseline for text classification, spam filtering and other problems with many sparse features.
The word naive refers to the independence assumption, which is almost never strictly true. Despite this, the classifier often ranks classes correctly, because a correct decision only requires the highest posterior score, not an accurate probability estimate.
How the naive Bayes classifier computes a prediction
The naive Bayes classifier predicts the class with the largest posterior probability, computed from a class prior and a product of per-feature likelihoods. Bayes’ theorem states that the posterior P(class | features) is proportional to P(class) multiplied by P(features | class). The independence assumption replaces the joint likelihood with a product of one-dimensional likelihoods, one for each feature.
This factorization is what makes the model practical. Estimating a full joint distribution over hundreds of features would require an enormous amount of data, whereas each per-feature distribution can be estimated from simple counts or means. Training therefore reduces to a single pass over the data, and prediction costs time proportional to the number of features multiplied by the number of classes.
In practice, implementations sum log probabilities instead of multiplying raw probabilities. Multiplying thousands of small numbers causes floating-point underflow, while the logarithm turns the product into a stable sum and does not change which class wins. The table below summarizes the components of the computation.
| Component | Meaning | How it is estimated |
|---|---|---|
| Class prior | Probability of a class before seeing features | Share of training examples in that class |
| Feature likelihood | Probability of a feature value given the class | Counts, frequencies or a fitted distribution |
| Evidence | Probability of the observed features | Ignored, because it is equal for all classes |
| Posterior | Probability of the class given the features | Prior times likelihoods, then normalized |
Variants of naive Bayes and their feature models
Naive Bayes variants differ in the probability distribution assumed for each feature, and the right choice depends on the data type. Choosing a variant that does not match the features is a more common source of poor results than the independence assumption itself.
| Variant | Feature type | Typical use |
|---|---|---|
| Gaussian naive Bayes | Continuous values, normal per class | Sensor readings, measurements |
| Multinomial naive Bayes | Counts or term frequencies | Document and topic classification |
| Bernoulli naive Bayes | Binary presence or absence | Short texts, binary indicators |
| Complement naive Bayes | Counts with skewed classes | Imbalanced text corpora |
| Categorical naive Bayes | Discrete categories | Survey answers, coded attributes |
Multinomial and Bernoulli models need smoothing to handle feature values that never appeared with a class during training. Without it, a single unseen word produces a zero likelihood and eliminates the class regardless of all other evidence. Laplace smoothing adds one to every count, and the more general additive smoothing uses a tunable constant alpha, commonly searched between 0.01 and 1.0.
Naive Bayes compared with related classifiers
Naive Bayes is a generative classifier, because it models how features are produced within each class, whereas logistic regression is a discriminative classifier that models the class boundary directly. The two form a well-known pair: with enough data, logistic regression usually reaches a lower error, while naive Bayes tends to approach its own best accuracy with far fewer training examples.
| Method | Model type | Strength | Weakness |
|---|---|---|---|
| Naive Bayes | Generative, linear in log space | Very fast, works on small data | Ignores feature interactions |
| Logistic regression | Discriminative, linear | Well-calibrated probabilities | Needs more data and tuning |
| Support vector machine | Discriminative, margin-based | Strong on high-dimensional text | Slower training, no native probabilities |
| Bayesian network | Generative, explicit dependencies | Models feature relationships | Structure learning is costly |
| Decision tree | Discriminative, rule-based | Captures interactions | Overfits without pruning |
A Bayesian network can be seen as the generalization of naive Bayes: naive Bayes is the special network in which the class is the only parent of every feature. Adding edges between features relaxes the independence assumption at the price of more parameters and slower learning.
When to use a naive Bayes classifier
Naive Bayes is the right choice when speed, small data or very many features dominate the requirements. It trains in seconds on millions of documents, supports online learning by simply updating counts, and needs almost no hyperparameter tuning.
| Situation | Recommendation | Reason |
|---|---|---|
| Text with large vocabulary | Use multinomial naive Bayes | Sparse counts fit the model well |
| Few labeled examples | Use naive Bayes as baseline | Low variance on small samples |
| Streaming data | Use incremental updates | Counts update without retraining |
| Strongly correlated features | Prefer logistic regression or trees | Correlations distort the posterior |
| Calibrated risk scores needed | Calibrate or choose another model | Raw probabilities are overconfident |
Feature preparation matters more than tuning. For text, TF-IDF weighting, removal of very rare terms and n-grams can raise accuracy noticeably. For continuous features that are far from normal, discretizing them into bins and using a categorical model is often more robust than Gaussian naive Bayes.
Limitations and common mistakes with naive Bayes
The main limitation is double counting of correlated evidence. If two features carry the same information, for example a word and its plural, the model treats them as independent confirmations and pushes the posterior toward an extreme value. This is why naive Bayes probabilities are typically poorly calibrated, clustering near 0 and 1.
- Zero-frequency problem: forgetting smoothing lets one unseen value veto a class.
- Misleading probabilities: using raw outputs as risk estimates without Platt scaling or isotonic calibration.
- Wrong variant: applying Gaussian naive Bayes to word counts or Bernoulli to long documents.
- Skewed priors: strong class imbalance biases predictions toward the majority class unless priors are adjusted.
- No interactions: the model cannot learn that two features matter only in combination, as in an XOR pattern.
Stripped to the essentials
A naive Bayes classifier sorts things into groups by weighing clues one at a time. Imagine a mail sorter who has learned how often words like free or invoice appear in junk mail and in normal mail. For a new message, the sorter checks each word, adds up how strongly each one points toward junk or toward normal, and picks the group with the stronger total.
It is called naive because it treats every clue as unrelated to the others, which is a simplification. Even so, it is fast, cheap and surprisingly accurate, which is why it is often the first model people try.
Where it is applied
A municipal help desk receives about 4,000 citizen emails per week and must route each one to one of nine departments, such as waste collection, parking or tax. The team has only 6,000 labeled historical emails, too few for a large language model fine-tuned in-house, and the routing must run on a modest server.
The data team trains a multinomial naive Bayes model on word unigrams and bigrams with TF-IDF weighting and an additive smoothing constant of 0.1, chosen by cross-validation. Training takes under two seconds. On a held-out test set the model routes about 88 percent of emails correctly, and messages whose top posterior falls below a confidence threshold go to a human dispatcher.
The result is that manual routing work drops by roughly two thirds, and the model is retrained every night with the corrections made by dispatchers, at negligible computing cost.