Artificial intelligence

Machine learning

Classic machine learning algorithms and methods, from regression and decision trees to clustering, ensembles and feature engineering.

Browse A–Z

Terms

56 terms

  • AdaBoost

    AdaBoost is a boosting algorithm that combines many weak classifiers into one strong model by reweighting the examples that earlier learners misclassified.

  • Association rule learning

    Association rule learning is an unsupervised method that finds if-then patterns between items that frequently occur together in large transaction datasets.

  • AutoML

    AutoML, or automated machine learning, automates model selection, feature preprocessing and hyperparameter tuning to build predictive models faster.

  • Bagging

    Bagging, or bootstrap aggregating, is an ensemble method in machine learning that trains models on resampled data and averages them to cut variance.

  • Bayesian network

    A Bayesian network is a probabilistic graphical model that encodes variables and their conditional dependencies as a directed acyclic graph.

  • Bayesian optimization

    Bayesian optimization is a sample-efficient method that finds good settings for expensive black-box functions, such as model hyperparameters.

  • Bias-variance tradeoff

    The bias-variance tradeoff describes how model complexity balances systematic error against sensitivity to training data, shaping generalization error.

  • Binary classification

    Binary classification is the machine learning task of assigning each input to one of exactly two classes, such as spam or not spam.

  • Boosting

    Boosting is an ensemble method in machine learning that trains weak models in sequence, each one correcting the errors left by the models before it.

  • Class imbalance

    Class imbalance in machine learning is a skewed label distribution in which one class vastly outnumbers another, biasing models toward the majority.

  • Concept drift

    Concept drift is a change over time in the relationship between input features and the target, which makes a trained machine learning model less accurate.

  • Curse of dimensionality

    The curse of dimensionality describes how data becomes sparse and distances lose meaning as features grow, degrading many machine learning methods.

  • DBSCAN

    DBSCAN is a density-based clustering algorithm in machine learning that groups dense regions of points and labels sparse points as noise.

  • Decision boundary

    A decision boundary is the surface in feature space where a machine learning classifier switches its predicted class from one label to another.

  • Decision tree

    A decision tree is a machine learning model that predicts a label or value by routing each input through a sequence of simple feature tests.

  • Dimensionality reduction

    Dimensionality reduction in machine learning maps data with many features into fewer dimensions while keeping the structure that matters for the task.

  • Elastic net

    Elastic net is a regularized regression method in machine learning that combines L1 and L2 penalties to shrink coefficients and select correlated features.

  • Elbow method

    The elbow method is a heuristic in machine learning for choosing the number of clusters by finding where adding clusters stops reducing error sharply.

  • Ensemble learning

    Ensemble learning combines the predictions of several machine learning models so that the group is more accurate and stable than any single member.

  • Expectation-maximization algorithm

    The expectation-maximization algorithm estimates model parameters from data with hidden variables by alternating expectation and maximization steps.

  • Feature extraction

    Feature extraction in machine learning transforms raw data into a compact set of informative numeric features that models can learn from.

  • Feature importance

    Feature importance in machine learning is a score that estimates how much each input variable contributes to a model's predictions or its accuracy.

  • Feature scaling

    Feature scaling in machine learning transforms numeric input features onto comparable ranges so that distance-based and gradient-based models train reliably.

  • Feature selection

    Feature selection in machine learning chooses the subset of input variables that best predicts the target, cutting noise, cost and overfitting.

  • Gaussian mixture model GMM

    A Gaussian mixture model (GMM) represents data as a weighted blend of normal distributions, enabling soft clustering and flexible density estimation.

  • Gaussian process

    A Gaussian process is a probabilistic machine learning model that places a distribution over functions and predicts values with calibrated uncertainty.

  • Gini impurity

    Gini impurity measures how mixed the class labels in a decision tree node are and guides which split a tree-learning algorithm chooses.

  • Gradient boosting

    Gradient boosting is a machine learning ensemble method that builds shallow decision trees one after another, each correcting the errors of the previous ones.

  • Hidden Markov model HMM

    A hidden Markov model (HMM) is a probabilistic sequence model in which unobserved states follow a Markov chain and emit the observations a system records.

  • Hierarchical clustering

    Hierarchical clustering is an unsupervised method that groups data into a nested tree of clusters, shown as a dendrogram that can be cut at any level.

  • Hyperparameter tuning

    Hyperparameter tuning is the search for model settings that are fixed before training, chosen by validated performance rather than by fitting to the data.

  • Information gain

    Information gain measures how much a split on a feature reduces entropy in the target labels, guiding decision tree learning in machine learning.

  • Isolation forest

    Isolation forest is an unsupervised anomaly detection algorithm that flags outliers by how few random tree splits are needed to isolate each data point.

  • k-means clustering

    k-means clustering is an unsupervised machine learning algorithm that splits data into k groups by minimizing distances to each group's mean.

  • k-nearest neighbors k-NN

    k-nearest neighbors (k-NN) is a machine learning method that predicts a label or value from the k most similar stored training examples.

  • Kernel trick

    The kernel trick lets machine learning algorithms learn nonlinear patterns by computing inner products in a feature space without building that space.

  • Lasso regression LASSO

    Lasso regression is a linear model in machine learning that adds an L1 penalty, shrinking coefficients and setting weak ones exactly to zero.

  • Linear discriminant analysis

    Linear discriminant analysis is a supervised machine learning method that finds linear projections which best separate known classes of data.

  • Logistic regression

    Logistic regression is a machine learning classifier that models the log-odds of an outcome as a linear function and returns calibrated class probabilities.

  • Multi-class classification

    Multi-class classification is the supervised learning task of assigning each input to exactly one of three or more mutually exclusive classes.

  • Multi-label classification

    Multi-label classification is a machine learning task in which a model assigns each example any number of labels from a fixed set at the same time.

  • Naive Bayes classifier

    A Naive Bayes classifier is a probabilistic machine learning model that applies Bayes' theorem with an assumption of independent features to assign classes.

  • One-hot encoding

    One-hot encoding turns a categorical feature into binary indicator columns, one per category, so machine learning models can use labels as numbers.

  • Online learning · machine learning

    Online learning in machine learning updates a model one example or small batch at a time as data arrives, instead of retraining on a fixed dataset.

  • Polynomial regression

    Polynomial regression fits a curved relationship by adding powers of input features to a linear model, a classic machine learning method for nonlinear data.

  • Principal component analysis PCA

    Principal component analysis (PCA) is a linear method in machine learning that projects data onto orthogonal directions of maximum variance.

  • Random forest

    A random forest is a machine learning ensemble that averages many decorrelated decision trees to deliver accurate, robust classification and regression.

  • Regularization

    Regularization in machine learning adds a penalty or constraint to model training so that a model generalizes to new data instead of memorizing noise.

  • Ridge regression

    Ridge regression is a linear model in machine learning that adds an L2 penalty on coefficient size to reduce variance and tame multicollinearity.

  • Singular value decomposition SVD

    Singular value decomposition (SVD) factors any matrix into rotations and scalings, the basis of PCA, low-rank approximation and latent factor models.