Gymnasium is an open-source Python library that defines a standard interface between reinforcement learning agents and environments and ships a collection of reference environments for training and benchmarking. Among AI frameworks & tools, it plays the role of a common contract: any algorithm written against its API can run on any environment that implements it, from a balancing pole to a simulated humanoid robot.
Gymnasium is the maintained successor of OpenAI Gym, forked in 2022 and developed by the Farama Foundation after the original project stopped receiving updates. The name OpenAI Gym still appears in older papers, tutorials and code, but new work in artificial intelligence research and teaching targets Gymnasium, which reached its 1.0 release in 2024.
How the Gymnasium environment loop works
Gymnasium models reinforcement learning as a loop of observations, actions and rewards. An environment is created with gymnasium.make("CartPole-v1"), started with env.reset(seed=42), which returns an initial observation and an info dictionary, and advanced with env.step(action). Each step returns five values: the next observation, a scalar reward, a terminated flag, a truncated flag and an info dictionary.
The split between terminated and truncated is the most important design decision in the modern API. Termination means the task reached a true terminal state, such as a pole falling or a goal being reached, so the value of the next state is zero. Truncation means the episode was cut off externally, usually by a time limit, so the agent should still bootstrap from the next state. The older Gym API merged both into a single done flag, which silently biased value estimates in many published implementations.
The table below lists the core methods and attributes that every environment exposes.
| Element | Purpose | Typical value or return |
|---|---|---|
| reset(seed, options) | Starts a new episode | Observation, info |
| step(action) | Applies one action | Observation, reward, terminated, truncated, info |
| render() | Produces a visual output | Window, RGB array or text |
| close() | Releases simulator resources | None |
| observation_space | Describes valid observations | A space object such as Box |
| action_space | Describes valid actions | A space object such as Discrete |
Spaces, wrappers and vectorized environments
Spaces describe the shape, type and bounds of observations and actions, which lets an algorithm build the right network inputs and outputs without knowing the environment in advance. Spaces can be sampled, checked with contains() and flattened, which makes them the backbone of environment-agnostic code.
| Space | Represents | Example |
|---|---|---|
| Discrete | One of n integer choices | Push cart left or right |
| Box | Bounded continuous array | Joint torques or pixel frames |
| MultiDiscrete | Several discrete choices at once | Game controller buttons and directions |
| MultiBinary | Vector of on and off flags | Set of switches |
| Dict and Tuple | Composite structures | Camera image plus sensor readings |
Wrappers modify an environment without editing its source code. Observation, action and reward wrappers transform the corresponding signals, while utility wrappers add a time limit, record video, normalize observations or stack consecutive frames. Wrappers compose like layers, so a standard Atari preprocessing pipeline is simply a chain of four or five wrappers around the raw emulator.
Vector environments run several copies of an environment in parallel and return batched results. SyncVectorEnv steps copies sequentially in one process, which suits cheap simulations, while AsyncVectorEnv uses separate worker processes and pays off when each step is expensive. Batching is essential for on-policy methods such as Proximal Policy Optimization, which need many transitions per update.
Built-in environment families
Gymnasium bundles several environment families that span difficulty from teaching examples to research benchmarks. Heavier families require optional dependencies installed as extras, such as the MuJoCo physics engine or the Arcade Learning Environment for Atari games.
| Family | Examples | Typical use |
|---|---|---|
| Toy Text | FrozenLake, Taxi, Blackjack | Tabular methods and teaching |
| Classic Control | CartPole, MountainCar, Pendulum | Quick sanity checks of algorithms |
| Box2D | LunarLander, BipedalWalker, CarRacing | Intermediate continuous control |
| MuJoCo | Hopper, HalfCheetah, Humanoid | Continuous control research benchmarks |
| Atari | Breakout, Pong, Space Invaders | Learning from pixels |
Environment identifiers carry a version suffix such as -v1 or -v5. A version changes whenever dynamics, rewards or observations change, so results are only comparable at the same version, a convention that protects reproducibility across papers.
Gymnasium compared with related tools
Gymnasium defines environments, not learning algorithms, so it is usually paired with a separate agent library. The comparison below places it among neighboring tools in the reinforcement learning stack.
| Project | Role | Relationship to Gymnasium |
|---|---|---|
| OpenAI Gym | Original environment API | Unmaintained predecessor |
| Stable-Baselines3 | Reference algorithm implementations | Trains agents on Gymnasium environments |
| PettingZoo | Multi-agent environment API | Sister project for several agents |
| Shimmy | Compatibility layer | Converts other simulators to the API |
| Minari | Offline datasets | Stores recorded episodes for offline learning |
Distributed frameworks such as Ray RLlib also accept Gymnasium environments, which makes the API a de facto interchange format. Deep learning libraries such as PyTorch or JAX supply the networks, while Gymnasium supplies the world the network acts in.
Writing custom environments and common pitfalls
A custom environment subclasses gymnasium.Env, declares its observation and action spaces, and implements reset() and step(). Registering it with gymnasium.register() allows creation through make(), which automatically applies wrappers such as the time limit. The bundled environment checker detects many contract violations before training starts.
- Ignoring truncation and treating time-limit endings as terminal states, which distorts value targets.
- Not seeding reset(), which makes experiments impossible to reproduce.
- Returning observations outside the declared space, for example wrong dtype or unclipped values.
- Poorly scaled rewards, which destabilize gradient-based learning far more than most people expect.
- Mixing old Gym code that expects four return values from step() with the five-value API.
Version 1.0 also changed autoreset behavior in vector environments: by default, a finished sub-environment resets on the following step rather than inside the same call. Code ported from older versions must account for this, or it may record a spurious transition between episodes.
The idea in brief
Gymnasium is like a standard set of practice arenas for learning software. One arena holds a cart balancing a pole. Another holds a lander touching down on the moon. All of them follow the same simple rules. The arena shows the program a situation. The program picks a move. The arena answers with a score and a new situation.
Every arena speaks the same language. So a learning program written once can practice in any of them. The key benefit is a shared plug. Researchers can compare methods fairly, and students can move from simple puzzles to robot simulations without rewriting code.
A practical scenario
A logistics company wants to test whether reinforcement learning can control a small warehouse robot that picks items from shelves. The engineering team first wraps its existing physics simulator as a custom Gymnasium environment, with a Box observation space holding 24 sensor values and a Discrete action space of six movement commands. A time limit of 500 steps produces truncation, while a collision or a completed pick produces termination.
The team runs the environment checker, fixes one observation returned as 64-bit floats instead of the declared 32-bit type, and then trains a Proximal Policy Optimization agent from Stable-Baselines3 on 16 parallel copies through an asynchronous vector environment. After about two million steps, the agent completes roughly nine of ten picks in simulation. Because the interface is standard, the team later swaps in a second algorithm for comparison without changing a single line of environment code.