Stories
Slash Boxes
Comments

SoylentNews is people

posted by martyb on Thursday October 19 2017, @02:39PM   Printer-friendly
from the Zeroing-in-on-AI dept.

Google DeepMind researchers have made their old AlphaGo program obsolete:

The old AlphaGo relied on a computationally intensive Monte Carlo tree search to play through Go scenarios. The nodes and branches created a much larger tree than AlphaGo practically needed to play. A combination of reinforcement learning and human-supervised learning was used to build "value" and "policy" neural networks that used the search tree to execute gameplay strategies. The software learned from 30 million moves played in human-on-human games, and benefited from various bodges and tricks to learn to win. For instance, it was trained from master-level human players, rather than picking it up from scratch.

AlphaGo Zero did start from scratch with no experts guiding it. And it is much more efficient: it only uses a single computer and four of Google's custom TPU1 chips to play matches, compared to AlphaGo's several machines and 48 TPUs. Since Zero didn't rely on human gameplay, and a smaller number of matches, its Monte Carlo tree search is smaller. The self-play algorithm also combined both the value and policy neural networks into one, and was trained on 64 GPUs and 19 CPUs over a few days by playing nearly five million games against itself. In comparison, AlphaGo needed months of training and used 1,920 CPUs and 280 GPUs to beat Lee Sedol.

Though self-play AlphaGo Zero even discovered for itself, without human intervention, classic moves in the theory of Go, such as fuseki opening tactics, and what's called life and death. More details can be found in Nature, or from the paper directly here. Stanford computer science academic Bharath Ramsundar has a summary of the more technical points, here.

Go is an abstract strategy board game for two players, in which the aim is to surround more territory than the opponent.

Previously: Google's New TPUs are Now Much Faster -- will be Made Available to Researchers
Google's AlphaGo Wins Again and Retires From Competition


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 3, Interesting) by looorg on Thursday October 19 2017, @03:21PM (17 children)

    by looorg (578) on Thursday October 19 2017, @03:21PM (#584582)

    Though self-play AlphaGo Zero even discovered for itself, without human intervention, classic moves in the theory of Go ...

    OK so it breaks their Tabula Rasa condition but it would still have been somewhat more impressive, and useful, if it had discovered things we didn't already know. Seems they could just have skipped these steps if the program was just taught what we already knew before it started instead of "rediscovering" what we already know.

    There doesn't seem to be a lot of deepmind or actual deep thinking going on here at all, from the human inventors sure but from the program? Not really. So DeepMind can come back when they actually do something we, as humans, have not done before, just doing it faster isn't really a very impressive feature when it comes to software, after all everything will usually by default just become faster as there are hardware improvements. Overall this is just like reading that version 2.0 is a superior product to version 1.0, which is normally how things go.

    it is possible to train to superhuman level, without human examples or guidance, given no knowledge of the domain beyond basic rules. Furthermore, a pure reinforcement learning approach requires just a few more hours to train

    In our evaluation, all programs were allowed 5 s of thinking time per move;

    How many games in a few hours? The speed of which the computer/program doesn't is really an interesting and comparative factor since it will always be faster then a human (5 seconds is probably barely enough for a human to think thru their options -- not including moving actual pieces on to the board). Previously it's been discussed how many hours it takes for a human to become a master at some task, 10k or whatever -- it's probably not even true. A Go game is how long? Just for simplicities sake lets say it lasts half an hour, so two games per hour so about 20k games. A computer computes that quite a lot faster then 10k hours. So they are just faster, something that is not in doubt.

    In the space of a few days, starting tabula rasa, AlphaGo Zero was able to rediscover much of this Go knowledge, as well as novel strategies that provide new insights into the oldest of games.

    Novel strategies ... Sounds like they won't be very useful. Have they previously been discovered by man and dismissed due to their level of novelty?

    Starting Score:    1  point
    Moderation   +1  
       Interesting=1, Total=1
    Extra 'Interesting' Modifier   0  
    Karma-Bonus Modifier   +1  

    Total Score:   3  
  • (Score: 0, Insightful) by Anonymous Coward on Thursday October 19 2017, @03:30PM (2 children)

    by Anonymous Coward on Thursday October 19 2017, @03:30PM (#584588)

    I swear. You people try to take a shit on everything.

    • (Score: -1, Troll) by Anonymous Coward on Thursday October 19 2017, @04:54PM (1 child)

      by Anonymous Coward on Thursday October 19 2017, @04:54PM (#584660)

      What do you mean "you people"??? Are you assuming my political identity? You should realize by now that Libertarian != alt right nazi, no matter how hard some dimwits on this site wrongly call themselves.

      • (Score: 0) by Anonymous Coward on Thursday October 19 2017, @06:50PM

        by Anonymous Coward on Thursday October 19 2017, @06:50PM (#584736)

        I think AC meant "assholes" and not any political affiliation. I wasn't sure at first but your post solidified my conclusion.

  • (Score: 2) by vux984 on Thursday October 19 2017, @03:46PM (3 children)

    by vux984 (5045) on Thursday October 19 2017, @03:46PM (#584607)

    it would still have been somewhat more impressive, and useful, if it had discovered things we didn't already know

    "In the space of a few days, starting tabula rasa, AlphaGo Zero was able to [...discover...] novel strategies that provide new insights into the oldest of games."

    Did you find that somewhat more impressive and useful?

    Novel strategies ... Sounds like they won't be very useful.

    Close minded much?

    • (Score: 2) by looorg on Thursday October 19 2017, @04:08PM (2 children)

      by looorg (578) on Thursday October 19 2017, @04:08PM (#584629)

      Since, as far as I can tell, they don't go into what these novel or new strategies are it's somewhat hard to go into it. They found something, they claim to be new or novel, that doesn't mean it's actually going to be useful for human beings playing Go. Novel as far as I know doesn't mean useful or great, just new and not like anything seen before. Might just be so utterly useless it has been written off by humans or it's something that requires a form of thinking humans can't or won't do, certainly not within their allotted 5-second rule. So it might be interesting, it might be great. But it could also be utterly useless unless you have X TPU:s hardwired into your brain to make use of them. I'm sure it was great for their software but for the game? And for humans? Probably not so much.
      This is in general the issue or problem, I have, with AlphaGo. It hasn't really done anything, except to play games. But it can't apply any of that knowledge to anything that is remotely useful, or anything that isn't Go, as of yet - things will hopefully change eventually. But the current AlphaGo ego stroke is really not all that awesomesauce they claim it to be.

      • (Score: 4, Insightful) by vux984 on Thursday October 19 2017, @06:31PM (1 child)

        by vux984 (5045) on Thursday October 19 2017, @06:31PM (#584712)

        Novel as far as I know doesn't mean useful or great

        The fact that the strategies were discovered and reinforced as valid by repeated play does tell us the AI found them useful.

        Might just be so utterly useless it has been written off by humans or it's something that requires a form of thinking humans can't or won't do

        Now we're just moving the goal posts. "So, it found a new strategy that had never been formally recognized that it is actively using to help it win, well... I'll only be impressed if humans can use it!"
        Frankly, take it as a small compliment to the human race that that it didn't find 'one weird trick that always wins' that we'd somehow missed for a few thousand years. Seriously what did you EXPECT?

        I'm sure it was great for their software but for the game? And for humans? Probably not so much.

        Are the goalposts even on the field anymore? They set out to beat humans at go with a machine, something that was only recently projected to be something still a long ways away. And they succeeded, decisively. That's impressive. Now the new generation requires only a fraction of the hardware and resources the previous generation needed, and not only still cleans up humans, but also cleans up the previous system. That's impressive.

        This is in general the issue or problem, I have, with AlphaGo. It hasn't really done anything, except to play games.

        That's precisely the task for which it was made.

        But the current AlphaGo ego stroke is really not all that awesomesauce they claim it to be.

        What claim did they make that you are so offended by?
        Go was considered something that couldn't be defeated by machine until very recently. It was considered that the immense number of possible moves, and the difficulty that even humans had at quantifying the strength of a move made it a difficult problem. I'm very impressed they solved it. I don't think for a second that this means we're on the brink of a sentient AI, but I'm still impressed.

        • (Score: 2) by rylyeh on Thursday October 19 2017, @10:34PM

          by rylyeh (6726) Subscriber Badge <{kadath} {at} {gmail.com}> on Thursday October 19 2017, @10:34PM (#584953)

          People who play Go know that there are few, if any, truly novel openings that have not been analyzed long ago.
          For AG to discover Any new openings that work is, itself, amazing. Remember the Bobby Fischer opening for Chess? [URL:https://en.wikipedia.org/wiki/Bobby_Fischer/]

          --
          don’t tell nobody, but I swar ter Gawd thet picter begun ta make me hungry fer victuals I couldn’t raise nor buy—
  • (Score: 2, Touché) by Anonymous Coward on Thursday October 19 2017, @03:47PM (3 children)

    by Anonymous Coward on Thursday October 19 2017, @03:47PM (#584610)

    OK so it breaks their Tabula Rasa condition but it would still have been somewhat more impressive, and useful, if it had discovered things we didn't already know.
    [...]
    Novel strategies ... Sounds like they won't be very useful. Have they previously been discovered by man and dismissed due to their level of novelty?

    First you say novel strategies would be useful, then that they "won't be very useful". Which is it?

    • (Score: 2) by looorg on Thursday October 19 2017, @04:11PM (2 children)

      by looorg (578) on Thursday October 19 2017, @04:11PM (#584631)

      It clearly depends on the strategy. It would have been awesome if they had actually mentioned what it was. For all we know their new strategy is completely worthless for human players. Then what is it good for? When AlphaGo-1 tries to play AlphaGo-2 and they try and trick eachother?

      • (Score: 2) by HiThere on Thursday October 19 2017, @07:13PM (1 child)

        by HiThere (866) Subscriber Badge on Thursday October 19 2017, @07:13PM (#584770)

        Why should you expect it to be useful for human players? Perhaps it's a strategy that's only useful when you're playing against something better than any human player. Not that I expect this to be true, but your basic criterion seems to need justification.

        --
        Put not your faith in princes.
  • (Score: -1, Redundant) by Anonymous Coward on Thursday October 19 2017, @03:59PM

    by Anonymous Coward on Thursday October 19 2017, @03:59PM (#584624)

    I swear. You people try to take a shit on everything.

  • (Score: -1, Redundant) by Anonymous Coward on Thursday October 19 2017, @04:56PM (2 children)

    by Anonymous Coward on Thursday October 19 2017, @04:56PM (#584661)

    I swear. You people try to take a shit on everything.

    • (Score: 2) by DeathMonkey on Thursday October 19 2017, @05:31PM (1 child)

      by DeathMonkey (1380) on Thursday October 19 2017, @05:31PM (#584683) Journal

      I was considering modding the first instance up but then you had to go and take a shit on your own comment.

      • (Score: 0) by Anonymous Coward on Thursday October 19 2017, @05:45PM

        by Anonymous Coward on Thursday October 19 2017, @05:45PM (#584689)

        ... I don't depend on your mod points.

  • (Score: 2) by DannyB on Thursday October 19 2017, @06:59PM

    by DannyB (5839) Subscriber Badge on Thursday October 19 2017, @06:59PM (#584752)

    it would still have been somewhat more impressive, and useful, if it had discovered things we didn't already know.

    If it did already, would we be able to recognize it?

    The reason we know it discovered various game playing tragedies that we have names for, is because we recognize those particular game tragedies. How would we recognize that it discovered a hence unknown and unnamed game tragedy?

    . . . as well as novel strategies that provide new insights into the oldest of games.

    Novel strategies ... Sounds like they won't be very useful. Have they previously been discovered by man and dismissed due to their level of novelty?

    Did you catch the part about providing new insights?

    Maybe these novel tragedies the machine discovered, which provide new insights, are indeed useful and represent an advancement in human knowledge.

  • (Score: 2, Interesting) by Meepy on Friday October 20 2017, @02:26PM

    by Meepy (2099) on Friday October 20 2017, @02:26PM (#585238)

    As someone who plays and watches go regularly, I can tell you that the alphago discoveries have had a massive influence on the game. You almost can't find a review of professional games where they don't comment "This is a move alphago would make." If you're interested, there are several english language series exploring the new ideas found. I expect this new version will make even more contributions