Reinforcement learning (RL) is a branch of machine learning in which an agent learns to make sequences of decisions by interacting with an environment and receiving numerical rewards, with the goal of maximizing the total reward it collects over time. Among the fundamentals of artificial intelligence, it is the paradigm for learning from consequences rather than from labeled examples.
Unlike supervised learning, no one tells the agent the correct action for each situation. The agent must discover good behavior through trial and error, balancing actions it already knows work with new actions that might work better, and it must assign credit to decisions whose effects appear only much later.
How reinforcement learning works
Reinforcement learning works through a repeated interaction loop. At each step the agent observes a state, chooses an action, and the environment returns a reward and a new state. Over many episodes, the agent adjusts its policy, the rule that maps states to actions, to increase expected cumulative reward.
The problem is usually formalized as a Markov decision process (MDP), defined by states, actions, transition probabilities, a reward function and a discount factor. The discount factor, typically between 0.9 and 0.999, determines how much the agent values future rewards relative to immediate ones.
| Element | Meaning | Example in a warehouse robot |
|---|---|---|
| Agent | The learner and decision maker | The robot controller |
| Environment | Everything the agent interacts with | Warehouse floor, shelves, other robots |
| State | Information available for a decision | Position, battery level, current order |
| Action | A choice the agent can make | Move, turn, pick, wait |
| Reward | Scalar feedback after an action | Positive for delivery, negative for collisions |
| Policy | Mapping from states to actions | The learned driving strategy |
| Value function | Expected future reward from a state | How promising a location is |
Families of reinforcement learning algorithms
Reinforcement learning algorithms differ in what they learn and how they use experience. Value-based methods estimate how good actions are, policy-based methods optimize the policy directly, and model-based methods learn or use a model of the environment to plan ahead.
| Family | What is learned | Representative methods | Typical strength |
|---|---|---|---|
| Value-based | Action values | Q-learning, deep Q-networks | Discrete actions, sample reuse |
| Policy gradient | The policy directly | REINFORCE, PPO | Continuous actions, stable updates |
| Actor-critic | Policy plus value estimate | A2C, SAC, TD3 | Lower variance learning |
| Model-based | Environment dynamics | Planning with learned models | Fewer real interactions |
| Offline RL | Policy from fixed logs | Conservative Q-learning | No live experimentation |
Deep reinforcement learning combines these algorithms with neural networks that approximate value functions or policies, which allows learning from high-dimensional inputs such as images. It also brings instability and sensitivity to hyperparameters, so results often vary noticeably between random seeds.
A further distinction is on-policy versus off-policy learning. On-policy methods such as PPO learn only from data generated by the current policy, which is stable but discards old experience. Off-policy methods such as Q-learning and SAC can reuse a replay buffer of past transitions, which improves sample efficiency at the cost of more delicate training.
Reinforcement learning compared with other learning paradigms
Reinforcement learning differs from supervised and unsupervised learning in the type of feedback and in the fact that the agent’s actions change the data it sees. That feedback loop makes RL powerful for control and sequential decisions, and also harder to train and evaluate.
| Paradigm | Feedback | Data source | Typical task |
|---|---|---|---|
| Supervised learning | Correct label per example | Fixed labeled dataset | Classification, regression |
| Unsupervised learning | No explicit feedback | Fixed unlabeled dataset | Clustering, representation |
| Reinforcement learning | Delayed scalar reward | Generated by the agent’s actions | Control, sequential decisions |
| Contextual bandits | Immediate reward per choice | Interaction without state transitions | Recommendations, ad selection |
Reinforcement learning also plays a central role in modern language models. In reinforcement learning from human feedback (RLHF), a reward model trained on human preference comparisons scores model responses, and a policy optimization method adjusts the model toward responses people prefer.
Key challenges in reinforcement learning
The central challenge is the exploration-exploitation trade-off: an agent that only exploits known actions may miss better ones, while an agent that explores too much wastes time and may take harmful actions. Simple strategies such as epsilon-greedy exploration choose a random action with a small probability, often decaying from about 1.0 to 0.01 during training.
| Challenge | Why it occurs | Common mitigation |
|---|---|---|
| Sample inefficiency | Millions of interactions may be needed | Simulation, model-based methods, offline data |
| Sparse rewards | Useful feedback arrives rarely | Reward shaping, curriculum learning |
| Credit assignment | Effects appear many steps later | Discounting, value functions, eligibility traces |
| Reward hacking | Agent exploits flaws in the reward | Careful reward design, constraints, audits |
| Sim-to-real gap | Simulators differ from reality | Domain randomization, real-world fine-tuning |
| Training instability | Nonstationary targets and data | Target networks, clipped updates, multiple seeds |
Reward hacking deserves particular attention. An agent optimizes exactly what is rewarded, not what the designer intended, so a cleaning robot rewarded for not seeing mess may learn to avoid looking rather than to clean.
When reinforcement learning is the right choice
Reinforcement learning is appropriate when a problem involves sequential decisions with delayed consequences and when the agent can interact with a real or simulated environment safely and cheaply. Games, robotics, resource scheduling, industrial control and some recommendation problems fit this profile.
- Prefer supervised learning when correct answers are known for each case.
- Prefer contextual bandits when each decision is independent and rewards are immediate.
- Prefer classical optimization or control when an accurate model of the system already exists.
- Use RL when a reliable simulator or large logged interaction data is available.
Understanding it simply
Reinforcement learning resembles teaching a dog a new trick with treats. Nobody explains the trick in words. The dog tries different things, gets a treat when it does something right and gradually repeats what earns rewards.
A computer agent learns the same way, except it may try millions of times in a simulation. The key idea is learning from results rather than instructions, including the patience to accept a small cost now for a bigger reward later.
Applied in practice
A large data center operator wants to reduce the energy used by its cooling system. Engineers build a simulator from historical sensor data covering temperatures, pump speeds, fan settings and outside weather, and define the reward as negative energy consumption with heavy penalties for exceeding safe server temperatures.
An actor-critic agent trains in the simulator for millions of steps, then runs in the real facility first in recommendation mode, where operators approve each suggested setting, and later with automatic control inside strict safety limits. Over several months, the agent finds combinations of setpoints that human operators had not tried, and cooling energy falls measurably while temperatures stay within limits. The safety layer, which overrides any action outside approved ranges, remains in place permanently.