AlphaGo.pdf

AlphaGo uses Monte-Carlo Tree Search (MCTS) with neural networks to play Go. MCTS in AlphaGo annotates search nodes with utility estimates, visit counters, and prior probabilities from a supervised learning (SL) policy network. The tree policy selects successors based on utility estimates and prior probabilities, while the simulation stage combines the results of a simulation with a default policy from a rollout policy network and a heuristic value from a value network. AlphaGo computes four neural networks: SL policy network, rollout policy network, reinforcement learning (RL) policy network, and value network. The SL policy network uses convolutional layers to approximate a function that predicts human moves, with a prediction accuracy of 57%. The rollout policy network is faster but less accurate, with a prediction accuracy of 24.2%.