Logistic regression is a supervised learning method that models the probability of a categorical outcome by passing a weighted sum of input features through the logistic (sigmoid) function. In machine learning and the wider field of artificial intelligence, it is the standard baseline for binary classification: it is fast, interpretable and usually well calibrated, which is why practitioners fit it first and keep it whenever more complex models fail to beat it by a meaningful margin.
Despite its name, logistic regression is a classification model, not a regression model for continuous targets. Statisticians also call it the logit model, because it assumes that the log-odds of the positive class change linearly with the inputs.
How logistic regression turns a linear score into a probability
Logistic regression first computes a linear score z = w·x + b, where x is the feature vector, w the learned weights and b the intercept. The sigmoid function σ(z) = 1 / (1 + e^(-z)) then maps any real score into the open interval between 0 and 1, which is read as the probability that the example belongs to the positive class.
Inverting the sigmoid gives the logit, log(p / (1 − p)), so the model is linear in the log-odds, not in the probability itself. This is the key to interpretation: increasing a feature by one unit adds its weight to the log-odds, which multiplies the odds by e raised to that weight.
The weights are estimated by maximum likelihood estimation, which is equivalent to minimizing the binary cross-entropy (log loss). The loss is convex, so gradient descent, Newton-type methods such as L-BFGS or iteratively reweighted least squares reach the global optimum without the local-minimum problems of neural networks. The table below summarizes the components of the model.
| Component | Formula or role | What it controls |
|---|---|---|
| Linear score | z = w·x + b | Position relative to the decision boundary |
| Sigmoid link | σ(z) = 1 / (1 + e^(-z)) | Conversion of the score into a probability |
| Log loss | −[y log p + (1 − y) log(1 − p)] | Penalty for confident wrong predictions |
| Decision threshold | Default 0.5, tunable | Trade-off between precision and recall |
| Regularization term | λ‖w‖² or λ‖w‖₁ | Weight size and overfitting |
Variants of logistic regression
The basic model handles two classes, but several extensions of the same principle cover other target types. They share the linear score and the likelihood objective and differ in the link function and in how the classes relate to each other.
| Variant | Target type | Typical use |
|---|---|---|
| Binary logistic regression | Two classes | Churn, fraud, click prediction |
| Multinomial (softmax) regression | Three or more unordered classes | Document topic, product category |
| One-vs-rest logistic regression | Multiple classes or labels | Multi-label tagging, simple multi-class setups |
| Ordinal logistic regression | Ordered categories | Ratings, severity grades |
| Regularized (L1, L2, elastic net) | Any of the above | Many or correlated features |
Regularization deserves special attention. An L2 penalty, as in ridge regression, shrinks all weights smoothly and stabilizes the model when features are correlated. An L1 penalty, as in lasso regression, drives some weights to exactly zero and therefore performs embedded feature selection. Elastic net combines both. In most libraries the strength is set by a parameter C, the inverse of the regularization strength, so a smaller C means a simpler model.
How it compares with other classifiers
Logistic regression draws a linear decision boundary in feature space, which is both its strength and its main limitation. The comparison below places it among other classic algorithms from the same toolbox.
| Model | Decision boundary | Probability output | Interpretability |
|---|---|---|---|
| Logistic regression | Linear in features | Well calibrated by design | High, via odds ratios |
| Naive Bayes classifier | Linear or quadratic | Often overconfident | Moderate |
| Linear support vector machine | Linear, margin-based | Needs separate calibration | Moderate |
| Decision tree | Axis-aligned steps | Coarse, leaf-based | High for shallow trees |
| Gradient boosting | Highly nonlinear | Usually needs calibration | Low without explanation tools |
Linear discriminant analysis also yields a linear boundary but assumes Gaussian class distributions with shared covariance; logistic regression makes no assumption about how the features are distributed and is therefore more robust when that assumption fails. Compared with tree ensembles, it usually loses accuracy on tabular data with strong interactions, but it trains in seconds, scales to millions of sparse features and is far easier to audit.
Using logistic regression well in practice
Good results depend more on feature preparation than on the solver. Numeric inputs should be standardized through feature scaling, because regularization penalizes all weights equally and unscaled features distort that penalty. Categorical inputs are expanded with one-hot encoding, and nonlinear effects are captured by adding splines, bins or interaction terms explicitly.
The model outputs a probability, and the decision threshold is a separate business choice. The default of 0.5 is rarely optimal under class imbalance; the threshold should be chosen from a precision-recall curve or from the real cost of each error type.
| Situation | Recommended choice | Reason |
|---|---|---|
| Many correlated features | L2 penalty, tuned C | Stable, shrunken coefficients |
| Need a sparse model | L1 or elastic net | Zero weights remove features |
| Rare positive class | Class weights and threshold tuning | Prevents a trivial majority prediction |
| Coefficients must be explained | Standardized inputs, odds ratios | Comparable effect sizes |
| Millions of sparse features | SGD or liblinear-type solver | Low memory, fast convergence |
Limitations and common mistakes
The most frequent errors come from misreading the model rather than from fitting it. Experienced practitioners watch for the following problems.
- Perfect separation: when a feature splits the classes completely, unregularized weights grow without bound and the solver fails to converge; a penalty fixes this.
- Multicollinearity: correlated inputs make individual coefficients unstable even when predictions remain accurate, so coefficients are not causal effects.
- Missed nonlinearity: the model cannot learn curved boundaries or interactions unless they are engineered into the features.
- Probability scale confusion: a weight describes a change in log-odds, not a fixed change in probability, which depends on the starting point.
- Data leakage: fitting the scaler or encoder on the full dataset before splitting inflates validation scores.
Calibration is a real advantage, but it holds only when the training distribution matches production. Under concept drift or resampling such as SMOTE, predicted probabilities shift and should be recalibrated on representative data.
In simple terms
Logistic regression is a way for a computer to answer yes-or-no questions with a percentage instead of a flat yes or no. It looks at several facts, gives each one a weight, adds them up and turns the total into a chance between 0 and 100 percent.
Think of a doctor estimating the risk of a condition from age, blood pressure and smoking. Each factor pushes the risk up or down by a known amount. The model learns those amounts from past cases, and because each weight is visible, people can see exactly why it reached its answer.
A practical example
A subscription software company wants to know which customers are likely to cancel in the next 30 days. Its data team trains logistic regression on about 200,000 past accounts, using standardized features such as login frequency, open support tickets, seat usage and contract length, with an L2 penalty tuned by cross-validation.
The model reaches a ROC AUC of roughly 0.82, only slightly below a gradient boosting model, but its coefficients show that a drop in weekly logins roughly doubles the odds of cancellation. Because only about 6 percent of customers churn, the team lowers the threshold to 0.25 so that the retention staff can contact more at-risk accounts. The result is a ranked, explainable call list that account managers trust and use every week.