Classic machine learning algorithms and methods, from regression and decision trees to clustering, ensembles and feature engineering.
56 terms
AdaBoost is a boosting algorithm that combines many weak classifiers into one strong model by reweighting the examples that earlier learners misclassified.
Association rule learning is an unsupervised method that finds if-then patterns between items that frequently occur together in large transaction datasets.
AutoML, or automated machine learning, automates model selection, feature preprocessing and hyperparameter tuning to build predictive models faster.
Bagging, or bootstrap aggregating, is an ensemble method in machine learning that trains models on resampled data and averages them to cut variance.
A Bayesian network is a probabilistic graphical model that encodes variables and their conditional dependencies as a directed acyclic graph.
Bayesian optimization is a sample-efficient method that finds good settings for expensive black-box functions, such as model hyperparameters.
The bias-variance tradeoff describes how model complexity balances systematic error against sensitivity to training data, shaping generalization error.
Binary classification is the machine learning task of assigning each input to one of exactly two classes, such as spam or not spam.
Boosting is an ensemble method in machine learning that trains weak models in sequence, each one correcting the errors left by the models before it.
Class imbalance in machine learning is a skewed label distribution in which one class vastly outnumbers another, biasing models toward the majority.
Concept drift is a change over time in the relationship between input features and the target, which makes a trained machine learning model less accurate.
The curse of dimensionality describes how data becomes sparse and distances lose meaning as features grow, degrading many machine learning methods.
DBSCAN is a density-based clustering algorithm in machine learning that groups dense regions of points and labels sparse points as noise.
A decision boundary is the surface in feature space where a machine learning classifier switches its predicted class from one label to another.
A decision tree is a machine learning model that predicts a label or value by routing each input through a sequence of simple feature tests.
Dimensionality reduction in machine learning maps data with many features into fewer dimensions while keeping the structure that matters for the task.
Elastic net is a regularized regression method in machine learning that combines L1 and L2 penalties to shrink coefficients and select correlated features.
The elbow method is a heuristic in machine learning for choosing the number of clusters by finding where adding clusters stops reducing error sharply.
Ensemble learning combines the predictions of several machine learning models so that the group is more accurate and stable than any single member.
The expectation-maximization algorithm estimates model parameters from data with hidden variables by alternating expectation and maximization steps.
Feature extraction in machine learning transforms raw data into a compact set of informative numeric features that models can learn from.
Feature importance in machine learning is a score that estimates how much each input variable contributes to a model's predictions or its accuracy.
Feature scaling in machine learning transforms numeric input features onto comparable ranges so that distance-based and gradient-based models train reliably.
Feature selection in machine learning chooses the subset of input variables that best predicts the target, cutting noise, cost and overfitting.
A Gaussian mixture model (GMM) represents data as a weighted blend of normal distributions, enabling soft clustering and flexible density estimation.
A Gaussian process is a probabilistic machine learning model that places a distribution over functions and predicts values with calibrated uncertainty.
Gini impurity measures how mixed the class labels in a decision tree node are and guides which split a tree-learning algorithm chooses.
Gradient boosting is a machine learning ensemble method that builds shallow decision trees one after another, each correcting the errors of the previous ones.
A hidden Markov model (HMM) is a probabilistic sequence model in which unobserved states follow a Markov chain and emit the observations a system records.
Hierarchical clustering is an unsupervised method that groups data into a nested tree of clusters, shown as a dendrogram that can be cut at any level.
Hyperparameter tuning is the search for model settings that are fixed before training, chosen by validated performance rather than by fitting to the data.
Information gain measures how much a split on a feature reduces entropy in the target labels, guiding decision tree learning in machine learning.
Isolation forest is an unsupervised anomaly detection algorithm that flags outliers by how few random tree splits are needed to isolate each data point.
k-means clustering is an unsupervised machine learning algorithm that splits data into k groups by minimizing distances to each group's mean.
k-nearest neighbors (k-NN) is a machine learning method that predicts a label or value from the k most similar stored training examples.
The kernel trick lets machine learning algorithms learn nonlinear patterns by computing inner products in a feature space without building that space.
Lasso regression is a linear model in machine learning that adds an L1 penalty, shrinking coefficients and setting weak ones exactly to zero.
Linear discriminant analysis is a supervised machine learning method that finds linear projections which best separate known classes of data.
Logistic regression is a machine learning classifier that models the log-odds of an outcome as a linear function and returns calibrated class probabilities.
Multi-class classification is the supervised learning task of assigning each input to exactly one of three or more mutually exclusive classes.
Multi-label classification is a machine learning task in which a model assigns each example any number of labels from a fixed set at the same time.
A Naive Bayes classifier is a probabilistic machine learning model that applies Bayes' theorem with an assumption of independent features to assign classes.
One-hot encoding turns a categorical feature into binary indicator columns, one per category, so machine learning models can use labels as numbers.
Online learning in machine learning updates a model one example or small batch at a time as data arrives, instead of retraining on a fixed dataset.
Polynomial regression fits a curved relationship by adding powers of input features to a linear model, a classic machine learning method for nonlinear data.
Principal component analysis (PCA) is a linear method in machine learning that projects data onto orthogonal directions of maximum variance.
A random forest is a machine learning ensemble that averages many decorrelated decision trees to deliver accurate, robust classification and regression.
Regularization in machine learning adds a penalty or constraint to model training so that a model generalizes to new data instead of memorizing noise.
Ridge regression is a linear model in machine learning that adds an L2 penalty on coefficient size to reduce variance and tame multicollinearity.
Singular value decomposition (SVD) factors any matrix into rotations and scalings, the basis of PCA, low-rank approximation and latent factor models.