Submitted via IRC for BoyceMagooglyMonkey
AI agents continue to rack up wins in the video game world. Last week, OpenAI's bots were playing Dota 2; this week, it's Quake III, with a team of researchers from Google's DeepMind subsidiary successfully training agents that can beat humans at a game of capture the flag.
As we've seen with previous examples of AI playing video games, the challenge here is training an agent that can navigate a complex 3D environment with imperfect information. DeepMind's researchers used a method of AI training that's also becoming standard: reinforcement learning, which is basically training by trial and error at a huge scale.
Agents are given no instructions on how to play the game, but simply compete against themselves until they work out the strategies needed to win. Usually this means one version of the AI agent playing against an identical clone. DeepMind gave extra depth to this formula by training a whole cohort of 30 agents to introduce a "diversity" of play styles. How many games does it take to train an AI this way? Nearly half a million, each lasting five minutes.
Source: https://www.theverge.com/2018/7/4/17533898/deepmind-ai-agent-video-game-quake-iii-capture-the-flag
Related Stories
DeepMind's AI agents conquer human pros at Starcraft II
AI agents developed by Google's DeepMind subsidiary have beaten human pros at Starcraft II — a first in the world of artificial intelligence. In a series of matches streamed on YouTube and Twitch, AI players beat the humans 10 games in a row. In the final match, pro player Grzegorz "MaNa" Komincz was able to snatch a single victory for humanity.
[...] Beating humans at video games might seem like a sideshow in AI development, but it's a significant research challenge. Games like Starcraft II are harder for computers to play than board games like chess or Go. In video games, AI agents can't watch the movement of every piece to calculate their next move, and they have to react in real time.
These factors didn't seem like much of an impediment to DeepMind's AI system, dubbed AlphaStar. First, it beat pro player Dario "TLO" Wünsch, before moving to take on MaNa. The games were originally played in December last year at DeepMind's London HQ, but a final match against MaNa was streamed live today, providing humans with their single victory.
Professional Starcraft commentators described AlphaStar's play as "phenomenal" and "superhuman." In Starcraft II, players start on different sides of the same map before building up a base, training an army, and invading the enemy's territory. AlphaStar was particularly good at what's called "micro," short for micromanagement, referring to the ability to control troops quickly and decisively on the battlefield.
[...] Experts have already begun to dissect the games and argue over whether AlphaStar had any unfair advantages. The AI agent was hobbled in some ways. For example, it was restricted from performing more clicks per minute than a human. But unlike human players, it was able to view the whole map at once, rather than navigating it manually.
Previously: Google DeepMind to Take on Starcraft II
Google's AI Declares Galactic War on Starcraft
Related: DeepMind's AI Agents Exceed Human-Level Gameplay in Quake III
Move Over AlphaGo: AlphaZero Taught Itself to Play Three Different Games
(Score: 4, Insightful) by SomeGuy on Monday July 09 2018, @10:59PM
Aimbot!
Aimbot!
Aimbot!
* You were kicked from the server for spamming. Plus you suck, noob.
(Of course a computer could do better, duh)
(Score: 3, Informative) by The Mighty Buzzard on Monday July 09 2018, @11:25PM (5 children)
Well no shit a bot can do a better job than humans at a FPS. They don't have the limits of human reflexes. I'd be amazed if a bot couldn't beat a human at any twitch game. It doesn't even have to be an AI bot, a handcoded one will do just fine.
My rights don't end where your fear begins.
(Score: 5, Funny) by bob_super on Monday July 09 2018, @11:30PM (3 children)
I think the point was that after the AI takes your job, it will keep making your life miserable when you're playing to forget your troubles.
(Score: 0) by Anonymous Coward on Monday July 09 2018, @11:45PM (1 child)
Hell, if this means that we might get opponents in single player and cooperative games that aren't dumb as a box of rocks, I say go for it.
(Score: 3, Funny) by c0lo on Tuesday July 10 2018, @12:55AM
FTFY
Why should I believe that you, an AC, are better?
https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
(Score: 2) by Gaaark on Tuesday July 10 2018, @01:58AM
The Mighty Bot?
--- Please remind me if I haven't been civil to you: I'm channeling MDC. ---Gaaark 2.0 ---
(Score: 0) by Anonymous Coward on Monday July 09 2018, @11:42PM
But can a Twitch bot show bobs and vegana?
(Score: 4, Insightful) by looorg on Monday July 09 2018, @11:26PM (3 children)
Doesn't that sort of make it hard to compare to humans. After all if you can never learn the map, which is a fairly big part of the games, it would be a very different game. A large explanation that humans get better and better is that the map is the same, almost all is known. It never really changes. Walls, doors whatever are always in the same spots. Power ups and weapons are always in the same known spots. The only thing that changes is where players are, but even that is probably not completely true since for various reasons they do tend to appear at about the same places over and over again (good sniping spot, power ups, spawnzones etc), there is a reason you shoot missiles into some rooms just on "random" and get kills. So it's not like it's completely random and they are all fumbling about in the dark.
So with that in mind is watching this like watching a group of aimbots playing? Do they actually miss when they fire or? Normal players tend to miss sometimes. Perhaps the most important question is; Can the AI rocket-jump yet?
(Score: 3, Funny) by Gaaark on Tuesday July 10 2018, @02:03AM (1 child)
And will the AI teabag you and make Stephen Hawking sex talk?
"Yes.
Yes
youwere
naughty.
Take my
tea bag Yes."
--- Please remind me if I haven't been civil to you: I'm channeling MDC. ---Gaaark 2.0 ---
(Score: 2) by looorg on Tuesday July 10 2018, @02:49PM
Teabagging should probably be classified as a form of psychological warfare, it's clearly an important part of the online playing toolkit. It is more of a close combat skill tho since if you snipe you wont run to the other side of the map to do it, or get there in time.
(Score: 2) by FatPhil on Tuesday July 10 2018, @03:07PM
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 0) by Anonymous Coward on Monday July 09 2018, @11:27PM
Ok if the bots are going to win all the time...
> that can beat humans at a game of capture the flag.
...then I'm just going to play capture the flag at my local university campus, with real humans. That ought to last for a few years, until someone from Boston Robotics (etc) combines the bot with a robot. They've got a ways to go, all those clever fast robots were really noisy with an IC engine running to supply power.
(Score: 2) by Mykl on Monday July 09 2018, @11:55PM (2 children)
I would be more impressed if these AIs were hooked up to a set of mechanical arms with 5 fingers that controlled the game via keyboard-and-mouse. As it is, the AI doesn't have to worry about moving the mouse, hitting the wrong keys etc. As mentioned above too, they have no reflex time to worry about.
(Score: 2) by takyon on Tuesday July 10 2018, @12:02AM (1 child)
It would be trivial to program in delays or APM limits that match average or pro gamer humans. A straight APM limit or even simulated mouse movement, and tens or hundreds of milliseconds of added delay to match humans. TFA doesn't say whether they did that but another article might, and Google did/is doing the same thing with Starcraft [soylentnews.org].
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 2) by looorg on Tuesday July 10 2018, @12:14AM
There are also probably other issues, such as consistency of play. No human would ever be able to play 450k games of QuakeIII. The graph in the article is a bit odd since it seems to indicate that humans would be all the same all the time on their level. They are after all not machines so it's highly unlikely.
(Score: 2) by acid andy on Tuesday July 10 2018, @09:45AM (3 children)
Uhh, that sounds less like intelligence and more like good, old-fashioned, brute forcing to me. Who cares if it involves a neural net if you have to throw that much time at it?
If a cat has kittens, does a rat have rittens, a bat bittens and a mat mittens?
(Score: 2) by looorg on Tuesday July 10 2018, @11:26AM (1 child)
That is a fairly valid point. Also 450k games a 5 minutes each is 2.250.000 minutes or about four years and change. I wonder how good a player would be if they played that many minutes of the game, I'm not sure there are any players like that alive today. The previous numbers assume you are playing around the clock 24h per day everyday which no human can sustain. Even if one was to play for just 12h per day that would be 8 or 9 years of playing QuakeIII (or whichever game one picks) every single day for 12h non-stop.
I'm sure one could argue that humans learn in other ways all the time, life experience bleeds into the gaming experience etc such as we know how to open doors etc. But I still doubt it's comparable. Perhaps it goes back to that idea of how long it takes to master something; the previous idea of it taking 10.000 hours was deemed to be bullshit years ago, which would even if it was true only be about 600.000 minutes which is still a lot less then the FTW-AI got to play the game.
(Score: 2) by FatPhil on Tuesday July 10 2018, @03:19PM
It's 4 months and change for a small group of mates who then go on to nominate their champion - a totally feasible endeavor.
However, it's 4 hours and change for players of a reasonably popular online game who then go on to chose their champion. That's the better comparison for the work factor involved.
Great minds discuss ideas; average minds discuss events; small minds discuss people; the smallest discuss themselves
(Score: 2, Insightful) by Kalas on Tuesday July 10 2018, @09:52PM
Please consider that given a game using entirely AI players, there's no reason the devs can't just set the game to run 10 or 100 times faster than realtime.
Processing power is pretty much the only limitation for how fast these simulations can be ran. You could get all those thousands of 5 minute (subjective time) games done in weeks or maybe days given enough cores to throw at it.
(Score: 2) by PinkyGigglebrain on Tuesday July 10 2018, @05:34PM (1 child)
When one of these "AI" bots can get to level 70 in Final Fantasy XIV on it's own, from first quest to final boss, then I'll be impressed.
Till then it is nothing more than an classic aim bot with "AI" stuck into it's description.
"Beware those who would deny you Knowledge, For in their hearts they dream themselves your Master."
(Score: 0) by Anonymous Coward on Thursday July 12 2018, @05:41PM
Can dominate EVE Online without triggering the anti-bot mechanisms, then we will know AI Agents have reached the minimum acceptable level of capability.
Since very little in-game in EVE is bounded, it would make it a lot harder for such an AI to decide what is optimizable win/lose behavior and would require a much larger corpus of knowledge as well as submodels bounding different areas of interestm whether economic, combat, or specific sectors of space with unusual physics or gameplay features.
Once it is capable of learning all of that, becoming successful both economically and militarily, AND being able to intuit new changes to the game (like the new rift space or whatever it is called) without human intervention or hundreds to thousands of failures first, then it will be a leading world-class gaming AI.