Artificial intelligence

Reinforcement learning

How agents learn by trial and error from rewards: environments, policies, value functions and the main algorithms.

Browse A–Z

Terms

55 terms

  • A3C

    A3C, asynchronous advantage actor-critic, is a reinforcement learning algorithm in which parallel workers update a shared policy and value network.

  • Action space

    The action space in reinforcement learning is the set of all actions an agent may choose from in a given state, either discrete or continuous.

  • Actor-critic

    Actor-critic is a reinforcement learning architecture in which a policy network acts while a value network evaluates actions to reduce gradient variance.

  • Advantage function

    The advantage function in reinforcement learning measures how much better one action is than the policy's average behavior in a given state.

  • Bellman equation

    The Bellman equation expresses the value of a state in reinforcement learning as immediate reward plus the discounted value of the states that follow.

  • Contextual bandit

    A contextual bandit is a reinforcement learning setting in which an agent sees context, picks one action and learns only from that action's reward.

  • Credit assignment problem

    The credit assignment problem in reinforcement learning is deciding which earlier actions deserve credit or blame for a reward that arrives later.

  • Cumulative reward

    Cumulative reward is the total, often discounted, sum of rewards an agent collects over time, the quantity that reinforcement learning tries to maximize.

  • Deep deterministic policy gradient DDPG

    Deep deterministic policy gradient (DDPG) is an off-policy actor-critic algorithm that learns deterministic policies for continuous control tasks.

  • Deep Q-network DQN

    A Deep Q-network (DQN) is a neural network that approximates the Q-function, letting a reinforcement learning agent choose actions from raw, complex data.

  • Deep reinforcement learning

    Deep reinforcement learning combines trial-and-error reward learning with deep neural networks so agents can act well in large, complex environments.

  • Discount factor

    The discount factor in reinforcement learning is a number between 0 and 1 that sets how much an agent values future rewards compared with immediate ones.

  • Double DQN

    Double DQN is a deep reinforcement learning algorithm that reduces Q-value overestimation by splitting action selection from action evaluation.

  • Episode · reinforcement learning

    An episode in reinforcement learning is one complete run of agent-environment interaction, from an initial state to a terminal state or a time limit.

  • Epsilon-greedy

    Epsilon-greedy is an exploration strategy in reinforcement learning that picks a random action with probability ε and the best-known action otherwise.

  • Experience replay

    Experience replay is a reinforcement learning technique that stores past transitions in a buffer and reuses random samples of them to train an agent.

  • Exploration-exploitation tradeoff

    The exploration-exploitation tradeoff is the reinforcement learning dilemma of choosing between trying new actions and repeating the best known ones.

  • Generalized advantage estimation GAE

    Generalized advantage estimation (GAE) computes policy-gradient advantages as a weighted sum of TD errors, trading bias against variance via lambda.

  • Group relative policy optimization GRPO

    Group relative policy optimization (GRPO) is a reinforcement learning method that scores groups of sampled answers against each other instead of using a critic.

  • Hierarchical reinforcement learning HRL

    Hierarchical reinforcement learning splits a long task into subgoals and temporally extended skills, so an agent plans at several levels of abstraction.

  • Intrinsic motivation · reinforcement learning

    Intrinsic motivation in reinforcement learning is a self-generated reward, such as novelty or prediction error, that drives an agent to explore.

  • Inverse reinforcement learning IRL

    Inverse reinforcement learning (IRL) infers the reward function an expert is optimizing from observed behavior, so an agent can learn the underlying goal.

  • Markov decision process MDP

    A Markov decision process (MDP) is the formal model of sequential decisions under uncertainty that underlies nearly every reinforcement learning algorithm.

  • Model-based reinforcement learning

    Model-based reinforcement learning trains an agent that learns or uses a model of its environment to plan actions and improve its policy with less real data.

  • Model-free reinforcement learning

    Model-free reinforcement learning trains an agent to act from sampled rewards alone, without learning a model of environment dynamics.

  • Monte Carlo methods · reinforcement learning

    Monte Carlo methods in reinforcement learning estimate value functions by averaging the actual returns observed over complete episodes of experience.

  • Monte Carlo tree search MCTS

    Monte Carlo tree search (MCTS) is a planning algorithm that grows a search tree from simulated outcomes to choose strong actions in sequential decisions.

  • Multi-agent reinforcement learning MARL

    Multi-agent reinforcement learning (MARL) studies how several agents learn policies by trial and error in a shared environment where their actions interact.

  • Off-policy learning

    Off-policy learning lets a reinforcement learning agent evaluate or improve one policy using experience generated by a different behavior policy.

  • Offline reinforcement learning

    Offline reinforcement learning trains a decision policy entirely from a fixed dataset of logged interactions, without collecting any new experience.

  • On-policy learning

    On-policy learning is a reinforcement learning approach in which an agent improves the same policy that it uses to collect its experience.

  • Partially observable Markov decision process POMDP

    A partially observable Markov decision process (POMDP) models sequential decisions when an agent cannot see the true state and must act on noisy observations.

  • Policy · reinforcement learning

    A policy in reinforcement learning is the rule that maps each state an agent observes to an action or a probability distribution over actions.

  • Policy gradient

    A policy gradient method in reinforcement learning improves a parameterized policy by gradient ascent on expected return, estimated from sampled episodes.

  • Policy iteration

    Policy iteration is a reinforcement learning algorithm that alternates policy evaluation and greedy improvement until it reaches an optimal policy.

  • Proximal policy optimization PPO

    Proximal policy optimization (PPO) is a reinforcement learning algorithm that improves a policy with clipped, conservative updates for stable training.

  • Q-function

    The Q-function in reinforcement learning gives the expected discounted return of taking an action in a state and then following a given policy.

  • Q-learning

    Q-learning is a model-free reinforcement learning algorithm that learns the value of each action in each state and derives an optimal policy from it.

  • REINFORCE

    REINFORCE is the Monte Carlo policy gradient algorithm in reinforcement learning that raises the probability of actions followed by high returns.

  • Reward function

    A reward function in reinforcement learning maps states and actions to scalar feedback that defines the goal an agent learns to maximize over time.

  • Reward shaping

    Reward shaping adds extra reward signals to a reinforcement learning task so an agent learns faster without changing which behavior is optimal.

  • RL environment

    An RL environment is the simulated or real world a reinforcement learning agent acts in, returning observations, rewards and episode signals.

  • Safe reinforcement learning

    Safe reinforcement learning trains agents to maximize reward while respecting explicit safety constraints during both learning and deployment.

  • SARSA

    SARSA is an on-policy temporal difference algorithm in reinforcement learning that learns action values from the actions an agent actually takes.

  • Self-play

    Self-play is a reinforcement learning method in which an agent improves by competing against copies of itself instead of fixed opponents or human data.

  • Soft actor-critic SAC

    Soft actor-critic (SAC) is an off-policy reinforcement learning algorithm that maximizes reward plus policy entropy for stable continuous control.

  • Sparse reward

    A sparse reward is a reinforcement learning signal that is zero for most steps and nonzero only at rare events such as reaching a goal.

  • State space · reinforcement learning

    In reinforcement learning, the state space is the set of all situations an agent can encounter, which shapes how its policy and values are learned.

  • TD3

    TD3 is an off-policy reinforcement learning algorithm for continuous control that fixes the overestimation bias of DDPG with twin critics and delayed updates.

  • Temporal difference learning

    Temporal difference learning is a reinforcement learning method that updates value estimates from the gap between successive predictions.