At the BlizzCon gaming convention last weekend, players of the strategy game StarCraft II stood in line to get walloped by DeepMind’s AI. After training for the better part of a year, the bot has become one of the world’s top players.
What’s new: DeepMind, the AI research division of Alphabet, announced that its AlphaStar model had achieved StarCraft II Grandmaster status, able to beat 99.8 percent of active players, under restrictions that mimic those affecting human players.
How it works: StarCraft II boils down to simple goals — mining resources, raising armies, and annihilating opponents — but gameplay is complex, offering 1026 possible actions at each time step. AlphaStar was designed not only to defeat human rivals but to play like humans do. Indeed, the model doesn’t invent its own strategies. It adopts human strategies and hones them with experience, according to the project’s latest paper.
- Agents were initialized via supervised learning.
- The developers used imitation learning to train an initial strategic policy capable of beating roughly 84 percent of human players.
- Then they set up a simulated league where AlphaStar used reinforcement learning while playing against itself the way human StarCraft II players do — not to win at all costs but to improve its skills by exposing and learning to defend against strategic flaws. To this end, the developers added agents whose sole purpose was to exploit weaknesses discovered in previous matches.
- After sharpening its skills in the virtual league, AlphaStar faced human opponents on Battle.net, an online gaming network, where it attained Grandmaster status.
Humanizing the bot: DeepMind announced AlphaStar in January and showcased its ability to beat professional human players in a series of matches. That version had features that gave it a clear advantage over humans. For instance, it could see the entire field of play rather than a limited view and could perform any number of actions per minute. The Grandmaster version was revamped to put it on equal footing with human players.
Why it matters: DeepMind has invested heavily in game-playing AI since its early days with classic Atari titles. Its accomplishments have generated lots of excitement around the technology. Its AlphaGo system is credited with motivating many countries to invest more in AI. AlphaStar keeps the momentum going.
We’re thinking: AI has beaten humans at a succession of games: Othello, Checkers, Chess, Go, Hold’em poker, and now StarCraft II. Still, there remains a significant gap between mastering even a very complex video game and practical, real-world applications.