AlphaZero

On this page 7

AlphaZero is a game-playing system developed by DeepMind that learned to play chess, shogi and Go at superhuman level using only the rules of each game, self-play reinforcement learning and a neural-network-guided tree search. First described in a preprint in December 2017 and published in full in 2018, it belongs among the landmark AI models & assistants because it showed that one general algorithm could master several classic board games without human game data or handcrafted evaluation knowledge.

AlphaZero generalized the approach of AlphaGo Zero, which had been specific to Go, and it became the conceptual parent of later systems such as MuZero and open-source engines such as Leela Chess Zero. Within artificial intelligence, it is a reference point for the combination of learning and planning.

AlphaZero chooses moves by running a Monte Carlo tree search (MCTS) that is guided by a single deep residual neural network. The network takes a board position as input and produces two outputs: a policy, which is a probability distribution over legal moves, and a value, which estimates the expected game outcome from that position.

During search, the policy tells the algorithm which moves deserve attention, and the value replaces the random playouts of classical MCTS. A selection rule known as PUCT balances exploiting moves with high estimated value against exploring moves the policy considers promising but that have few visits so far.

ComponentFunctionLearned or fixed
Policy headRanks candidate moves by prior probabilityLearned
Value headPredicts win, draw or loss from a positionLearned
MCTS with PUCTExpands the most promising lines of playFixed algorithm
Game rulesGenerate legal moves and terminal resultsFixed and supplied by designers
Self-play loopProduces training data from its own gamesFixed procedure
Main components of AlphaZero

Because the network concentrates the search, AlphaZero examined roughly tens of thousands of positions per second in chess, about three orders of magnitude fewer than the alpha-beta engine Stockfish searched in the published match setup. It compensated with far more selective evaluation.

Training through self-play reinforcement learning

AlphaZero learned entirely by playing games against itself, starting from random weights and no opening books, endgame tables or human games. Each self-play move used several hundred search simulations, typically 800 simulations per move, and the visit counts of the search became the training target for the policy.

After each game, every position was labeled with the final result, which trained the value head. The training loss combined value error, policy cross-entropy and weight regularization. Dirichlet noise added to the root of the search kept exploration alive, so the system kept trying unusual moves instead of collapsing into one repertoire.

Training required large-scale specialized hardware: thousands of first-generation tensor processing units generated self-play games, while a smaller pool of second-generation units updated the network. The reported training times are summarized below.

GameApproximate training timeMain reference opponent
ChessAbout 9 hoursStockfish
ShogiAbout 12 hoursElmo
GoAbout 13 daysAlphaGo Zero
Reported training scale and opponents by game

These hours describe wall-clock time on a very large hardware budget, not modest compute. Reproducing the results on commodity hardware took the volunteer Leela Chess Zero project many months of distributed training.

Results against established engines

AlphaZero defeated the strongest available engines in all three games under the match conditions chosen by its authors. In the 2018 chess match against Stockfish 8, it won 155 games and lost 6 out of 1,000, with the rest drawn.

The results attracted methodological criticism, particularly for the preliminary 2017 match, where Stockfish ran with a fixed time per move, a limited hash table and no opening book. The later full match addressed several of these points, yet hardware comparisons between TPUs and CPUs remain inherently imperfect. Chess experts were equally struck by its style: long-term piece activity, willingness to sacrifice material and pressure on king safety.

AlphaZero is one step in a lineage of game-playing systems, and each step removed a dependency on human knowledge or on known rules. The table below places it among its closest relatives.

SystemGamesHuman game dataNeeds known rules
AlphaGoGoYes, for initial trainingYes
AlphaGo ZeroGoNoYes
AlphaZeroChess, shogi and GoNoYes
MuZeroBoard games and AtariNoNo, learns a dynamics model
AlphaZero compared with its predecessors and successor

Compared with AlphaGo Zero, AlphaZero dropped Go-specific assumptions such as board symmetry augmentation and handled games that can end in draws. It also updated a single continuously trained network instead of gating new versions through evaluation matches.

Limitations and lasting influence

AlphaZero works only in domains with perfect information, known rules and a cheap simulator, which makes it unsuitable for most real-world decision problems without major changes. It was never released as a product or as open model weights, so independent verification relied on papers and later reproductions.

AspectStrengthConstraint
Knowledge sourceNo human data neededRequires an exact rule simulator
GeneralityOne algorithm for three gamesSeparate network trained per game
Search efficiencyFew, well-chosen positionsExpensive network evaluation per node
ComputeFast wall-clock learningVery high training cost
Strengths and constraints of the AlphaZero approach

Its influence is nevertheless broad. The policy-value network plus search pattern shaped later work on planning, including MuZero, algorithm discovery systems and research on test-time search for language models. In chess, it encouraged the shift toward neural evaluation in engines and changed opening preparation at the top level.

Explained without jargon

AlphaZero is a computer program that taught itself chess, shogi and Go by playing millions of games against itself. Nobody showed it human games or told it which positions are good. It only knew the rules.

Picture a student locked in a room with a chessboard who plays both sides, notes which ideas lead to wins and slowly develops intuition. AlphaZero does the same at enormous speed. When it plays, it combines that learned intuition with careful thinking ahead, checking only the most promising moves instead of every possibility. The key lesson is that a general learning method can discover expert-level strategy on its own.

A typical use case

A professional chess player preparing for a tournament studies the games AlphaZero played against Stockfish, which DeepMind published together with its research. The player focuses on games in which AlphaZero gave up a pawn early to gain long-term activity for its pieces, a choice that traditional engines of that time often rated as slightly worse.

Because AlphaZero itself is not publicly available, the player then uses Leela Chess Zero, an open-source engine built on the same self-play principle, to analyze those positions further. The engine confirms that several of the pawn sacrifices give lasting positional compensation rather than short-term tactics.

The outcome is a new opening line in the player’s repertoire, built around piece activity instead of material balance, and a better understanding of when long-term pressure justifies a material deficit.