Stories
Slash Boxes
Comments

SoylentNews is people

Submission Preview

Link to Story

AlphaGo Zero Makes AlphaGo Obsolete

Accepted submission by takyon at 2017-10-19 01:39:25
Software

Google DeepMind researchers have made their old AlphaGo program obsolete [theregister.co.uk]:

The old AlphaGo relied on a computationally intensive Monte Carlo tree search to play through Go scenarios. The nodes and branches created a much larger tree than AlphaGo practically needed to play. A combination of reinforcement learning and human-supervised learning was used to build "value" and "policy" neural networks that used the search tree to execute gameplay strategies. The software learned from 30 million moves played in human-on-human games, and benefited from various bodges and tricks to learn to win. For instance, it was trained from master-level human players, rather than picking it up from scratch.

AlphaGo Zero did start from scratch with no experts guiding it. And it is much more efficient: it only uses a single computer and four of Google's custom TPU1 [theregister.co.uk] chips to play matches, compared to AlphaGo's several machines and 48 TPUs. Since Zero didn't rely on human gameplay, and a smaller number of matches, its Monte Carlo tree search is smaller. The self-play algorithm also combined both the value and policy neural networks into one, and was trained on 64 GPUs and 19 CPUs over a few days by playing nearly five million games against itself. In comparison, AlphaGo needed months of training and used 1,920 CPUs and 280 GPUs to beat Lee Sedol.

Though self-play AlphaGo Zero even discovered for itself, without human intervention, classic moves in the theory of Go, such as fuseki [wikipedia.org] opening tactics, and what's called life and death [wikipedia.org]. More details can be found in Nature [nature.com], or from the paper directly here [deepmind.com]. Stanford computer science academic Bharath Ramsundar has a summary of the more technical points, here [twitter.com].

Go [wikipedia.org] is an abstract strategy board game for two players, in which the aim is to surround more territory than the opponent.

Previously: Google's New TPUs are Now Much Faster -- will be Made Available to Researchers [soylentnews.org]
Google's AlphaGo Wins Again and Retires From Competition [soylentnews.org]


Original Submission