This week DeepMind Technologies, UK company owned by Alphabet (previously known as Google), announced they they’ve reached a new milestone with their AlphaStar AI. DeepMind has been training their most sophisticated AI software “agent” to play Blizzard Entertainment’s real time strategy title StarCraft II and AlphaStar is now Grandmaster-ranked “player,” making it statistically capable of beating 99.8% of the game’s player base. DeepMind is planning to publish these new developments as a research paper in Nature, a multidisciplinary scientific journal, and AlphaStar is available for competitive matches at this year’s BlizzCon.
“And just a quick note that if you try this. You will not win,” Brack said during BlizzCon 2019’s Opening Ceremony about playing the AI-driven agents. “Three of you will win. I won’t win.”
According to DeepMind, the team wanted to make sure that AlphaStar didn’t go into competitive play with too many inhuman advantages when it entered it into public competitive matches last summer. First, DeepMind “taught” AlphaStar to compete with all three playable races in StarCraft II: Terrans, Zerg, and Protoss to keep the competition varied and complex as AlphaStar claimed the ladder in Ranked play. DeepMind also allows AlphaStar to only “see” the portions of a map that a real human player could see, as well as restrict AlphaStar to 22 non-duplicated actions to bring their agent in line with a human player’s limited actions per minute, or APM.
Even with these restrictions, DeepMind’s AlphaStar was able to advance to the Grandmaster level of StarCraft II‘s competitive community, a rank achieved by less than 2% of players, making it the first artificial intelligence “agent” to reach this RTS milestone. For DeepMind, this means there’s a strong case to be made for reinforced general purpose learning, a type of machine learning that allows AI agents to prioritize actins that lead to net positive rewards. DeepMind hopes that agents like AlphaStar and its StarCraft II training can eventually be used to advance self-learning robots, cars, and image and object recognition software.
“Many real-world applications require artificial agents to compete and coordinate with other agents in complex environments. As a stepping stone to this goal, the domain of StarCraft has emerged as an important challenge for artificial intelligence research, owing to its iconic and enduring status among the most difficult professional esports and its relevance to the real world in terms of its raw complexity and multi-agent challenges,” reads the abstract for DeepMind’s research paper for Nature.
Back in January, DeepMind announced that their AlphaStar could beat some of the world’s top professional StarCraft II players in prerecorded first to ten sets, but ultimately lost to Grzegorz “MaNa” Komincz in a live-streamed match. Since then DeepMind has been hard at work on the task of improving AlphaStar, and it appears to be paying off with these newly debuted advancements.
What’s it like to play against DeepMind’s AlphaStar anyway? “AlphaStar is an intriguing and unorthodox player – one with the reflexes and speed of the best pros but strategies and a style that are entirely its own. The way AlphaStar was trained, with agents competing against each other in a league, has resulted in gameplay that’s unimaginably unusual; it really makes you question how much of StarCraft’s diverse possibilities pro players have really explored,” said pro StarCraft II player Diego “Kelazhur” Schwimer on the DeepMind’s AlphaStar blog.
BlizzCon 2019 also revealed the next co-op commander, Arcturus Mengsk, along with new announcements for Overwatch 2, new entry in the Diablo series, Heroes of the Storm, Hearthstone, and World of Warcraft. Blizzard president J. Allen Brack also made special statement addressing the recent controversy surrounding professional Hearthstone player Chung “Blitzchung” Ng Wai.