AlphaGo

On this page 7

AlphaGo is a computer program developed by DeepMind that plays the board game Go by combining deep neural networks with Monte Carlo tree search. In March 2016 it defeated Lee Sedol, one of the strongest professionals of the previous decade, by four games to one, a result many researchers had expected to be at least ten years away. In the history of artificial intelligence, AlphaGo marks the point at which learned intuition plus search overcame a domain long considered too complex for brute-force methods.

AlphaGo was a research system rather than a product or assistant. Its importance lies in the methods it validated: policy and value networks, self-play reinforcement learning and the tight integration of learning with planning. Those ideas shaped its successors AlphaGo Zero and AlphaZero and influenced later work on reasoning and search in other AI models.

Why Go was a hard problem for computers

Go was difficult because of its enormous search space and the difficulty of evaluating positions. A 19 by 19 board allows roughly 10 to the power of 170 legal positions, and a typical position offers around 250 legal moves, compared with about 35 in chess. Exhaustive look-ahead of the kind that powered chess engines becomes hopeless at that branching factor.

The second obstacle was evaluation. In chess, material counts and handcrafted features give a useful score. In Go, the value of a position depends on subtle territorial and influence judgments that strong players describe as intuition, and no simple formula captured them. Before AlphaGo, the best programs used Monte Carlo tree search with random playouts and reached strong amateur level, but not professional strength on the full board.

PropertyChessGo
Board size8 by 819 by 19
Typical legal moves per turnAbout 35About 250
Typical game lengthAbout 80 pliesAbout 150 plies or more
Position evaluationHandcrafted features work wellHard to express in rules
Pre-2015 computer strengthAbove world championStrong amateur
Go compared with chess as a search problem

How AlphaGo works

AlphaGo works by using neural networks to narrow and evaluate the search, while Monte Carlo tree search decides the move. Instead of examining every option, the program explores only moves a learned policy considers promising, and it judges positions with a learned value estimate instead of playing every line to the end.

The original system trained its components in stages. A supervised policy network first learned to predict expert moves from about 30 million positions taken from online games between strong human players, reaching roughly 57 percent move-prediction accuracy. That network was then improved by reinforcement learning through self-play, and a separate value network learned to predict the winner from positions generated by those self-play games.

ComponentRoleHow it was trained
SL policy networkSuggests likely strong movesSupervised learning on expert games
RL policy networkPlays stronger moves than the SL networkSelf-play reinforcement learning
Value networkEstimates the probability of winningRegression on self-play outcomes
Fast rollout policyPlays quick simulated games to the endSimple patterns from expert games
Monte Carlo tree searchCombines all signals to choose a moveNot trained; a search algorithm
Components of the original AlphaGo system

During play, each simulation descends the tree using the policy network as a prior, evaluates the leaf with a mix of the value network and a fast rollout, and backs the result up the tree. The move visited most often is played. This design spends computation where the networks indicate it matters, which is why AlphaGo searched far fewer positions than chess engines while playing at a higher relative level.

Versions and milestones

AlphaGo evolved through several versions within about two years, each stronger and more efficient than the last. The progression shows a clear trend: less reliance on human data and on handcrafted elements, with more reliance on self-play and specialized hardware.

VersionPeriodNotable resultKey change
AlphaGo FanOctober 2015Beat European champion Fan Hui 5 to 0First win over a professional on a full board
AlphaGo LeeMarch 2016Beat Lee Sedol 4 to 1 in SeoulStronger networks, run on TPUs
AlphaGo MasterEarly to mid 2017Won 60 online games and beat Ke Jie 3 to 0Much stronger with less compute
AlphaGo ZeroOctober 2017Beat AlphaGo Lee 100 to 0Learned from self-play only, no human games
Main AlphaGo versions and their results

The Lee Sedol match produced two famous moments. Move 37 in game two, a shoulder hit on the fifth line, looked like a mistake to commentators but proved strong and is often cited as an example of machine creativity. In game four, Lee Sedol’s Move 78 exposed a weakness in AlphaGo’s reading and gave the only human win of the match.

How AlphaGo relates to later DeepMind systems

AlphaGo is the first member of a family of DeepMind systems built on learning plus search. Its successors simplified the design and generalized it, while related projects carried the research approach into science.

SystemDomainHuman data usedNetwork design
AlphaGoGoYes, for initial trainingSeparate policy and value networks
AlphaGo ZeroGoNo, rules onlySingle network with policy and value heads
AlphaZeroGo, chess and shogiNo, rules onlySame algorithm across games
MuZeroBoard games and AtariNo, and no given rulesLearns its own model of the environment
AlphaGo compared with its successors

AlphaGo should not be confused with AlphaFold, which predicts protein structures and uses a different architecture. The shared prefix reflects the laboratory’s naming habit and a common research culture, not a shared model. The influence of AlphaGo on modern language models is conceptual rather than direct: the idea of spending extra computation on search and self-improvement at inference time reappears in reasoning-focused systems.

Limitations and common misconceptions

AlphaGo was a narrow system that played only Go under fixed rules on a fixed board size. It could not explain its moves, transfer its skill to other games, or handle tasks outside perfect-information board play. Its training also required substantial computing resources; the distributed version used for the Fan Hui match ran on over a thousand CPUs and well over a hundred GPUs.

  • Not general intelligence: success in Go does not imply broad reasoning ability.
  • Not a pure neural network: tree search was essential to its strength.
  • Not flawless: game four showed blind spots in tactical reading.
  • Not the end of human Go: professionals now study AI moves to improve, and opening theory has changed noticeably.

Later research found that even superhuman Go programs descended from these methods can be beaten by adversarial strategies that exploit specific blind spots. Such results show that superhuman average play does not guarantee robustness, a lesson relevant to deploying any learned system.

The idea in brief

AlphaGo is a computer program that learned to play Go. Go is an ancient board game with a huge number of possible positions. No computer could ever check them all one by one. So AlphaGo learned a sense of which moves look good and who is likely winning. A strong human player builds the same kind of feel for the board.

It then used that feel to think ahead only along the most promising paths. This mix of trained intuition and focused planning worked. In 2016 it beat one of the best human players in the world, years earlier than most experts expected.

A practical scenario

A university course on machine learning uses AlphaGo as a teaching case for combining learning with search. Students first implement a small Monte Carlo tree search for a 9 by 9 Go board with random playouts and find that it plays weakly and needs many simulations per move.

They then add a compact policy network trained on a few thousand recorded games and a value network trained on self-play results, following the AlphaGo design at a much smaller scale. With the same budget of 800 simulations per move, the guided version wins most of its games against the random-playout version, because it spends its search on plausible moves.

The exercise gives students a concrete result they can measure, and the instructor uses it to explain why the original system needed both components. The outcome illustrates the central insight: learned evaluation makes search efficient, and search makes learned evaluation more reliable.