June 26, 2018 weblog
Dota 2 challenging bots turn hard to beat after being taught cooperative mode
Oh boy. Another computer game where a crushing victory can be claimed by AI. In short, the OpenAI team, of five neural networks, aka OpenAI Five, won amateur human teams at Dota 2.
For the uninitiated, the latter is a multiplayers battle game. No two games are the same. The game gets an update about once every two weeks, changing the environment semantics. Any hero can fill multiple roles.
What is the game about? Painfully short description, but at least a clue: Kill the enemy teams. Take buildings.
While the war cry is fundamental, don't take Dota for granted—its rules are very complex. Dota has been developed for over 10 years. Game logic, said the OpenAI blog, was implemented in "hundreds of thousands of lines of code."
Christy Dennison, machine learning engineer at OpenAI, thinks it's one of the most challenging and competitive video games ever.
Dota 2, said Tom Simonite in Wired, is one of the internet's most-watched videogames.
Here, let Simonite explain what both AI and human players are up against when they begin. "Complex videogames also require players to make more decisions, more quickly. A chess player has, on average, about 35 possible moves at any time, and a Go player 250. OpenAI says each of its team's bots must choose between an average of 1,000 valid actions every eighth of a second."
The team spent the last few months training a new AI system. The system uses reinforcement at a much larger scale, to train its bots to play together as a team.
Simonite in Wired commented on the training part too. "OpenAI Five learned how to play Dota 2 by playing against clones of itself millions of times."
Meanwhile, back at the blog. In a Monday posting with video, Dennison talked about the team journey in preparing for wins.
They decided to test the skills against amateur human teams. She said they were surprised: They have won their first games. While they were pleased, they still went to an expert to take a look at their work.
William Lee, better known as Blitz in the player community, is that person, regarded as quite good at the game. He said it took him eight years to learn some of the strategies that the bot was intuitively doing. He said the team fight bot aspect was excellent. Didn't mess up. (Simonite mentioned that "Each day, OpenAI Five played the equivalent of 180 years of Dota 2." He reflected on that. "No human has 180 years to learn a videogame.")
So how did they do it? Dennison said the bots start out completely selfish but now they care about their team mates. They learned how to play as a unit. What's next? The world championships will take place in August.
Dennison: "While the best players in the world are getting ready to compete, we' re also working on the next version of our bot."
To see how far they have come, they will host a live match in July. This time they will play a team of top players. The blog carried a July 28 date.
What else excites them in addition to hopes for victories: They hope their journey gives them more insight about how AI can solve complex problems—of any kind.
The manner in which these bots cooperate with mates cannot be taken lightly. Will Knight in MIT Technology Review: "This is an important and novel direction for AI, since algorithms typically operate independently. Approaches that help algorithms cooperate with each other could prove important for commercial uses of the technology. AI algorithms could, for instance, team up to outmaneuver opponents in online trading or ad bidding. Collaborative algorithms might also cooperate with humans."
© 2018 Tech Xplore