The games people play smarten up AI

(Tech Xplore)—A team of researchers from Germany and the UK (RWTH Aachen University and Microsoft Research) have figured out how to give artificial intelligence a boost toward learning games at a faster pace. Just as we learn from our mistakes, they are—well, learning from our mistakes too.

How so? A database of Atari Gameplay was unleashed to show AI what's what with game play. Tech sites are talking about this Atari Grand Challenge dataset—worthy of attention as the dataset, said Dan Robitzski in Inverse, gives AI systems access to new ways of learning and honing skills over time.

The researchers stated in their paper that "We collect and describe a large dataset of human Atari 2600 replays – the largest and most diverse such data set publicly released to date."

Jordan Pearson, Motherboard: "Computer scientists from RWTH Aachen University in Germany and Microsoft Research have released the largest-ever database of human playthroughs for some of the most popular games for the Atari 2600."

The games involved in their research were Ms. Pacman, Space Invaders, Video Pinball, Q*bert and Montezuma's Revenge.

The researchers invited people to play those five games on their website and then collected data on how well the gamers were doing. "Each time they entered a command in the game, it was recorded along with their current score and the in-game environment at that moment," said Dan Robitzski in Inverse.

How they made use of that information: "By recording each digital command, the current score, and the in- environment, scientists applied that information to an AI system for the process of reinforcement learning," said

And this is how AI was trained, through these replays, to play the same games.

Jordan Pearson, Motherboard, wrote about about their study method. He said the team collected the database with a "browser-based Atari 2600 emulator paired with an app that captures the player's input."

It's useful to observe how their approach was special regarding reinforcement learning. The distinction lies in the human element. "In a typical experiment, computers learn to perform a task by repeatedly trying and failing in a digital environment until they finally refine their behavior," said Robitzski.

So we are looking at AI learning from our mistakes. "Artificial agents using deep learning techniques will be able to pull patterns out of these playthroughs and learn from them."

Their paper, "The Atari Grand Challenge Dataset" is on arXiv. Authors are Vitaly Kurin, Sebastian Nowozin, Katja Hofmann, Lucas Beyer, Bastian Leibe.

Kurin, Beyer and Liebe are from RWTH Aachen University in Germany. Nowozin and Hofmann are from Microsoft Research, Cambridge, UK.

What's next? They are still into their research.

"The Atari Grand Challenge website is still on-line and people keep playing. We plan to update the dataset in the future as more data becomes available. The most important development of the is to collect more data of "professional players" who achieve higher scores. As we have shown, the data quality affects the final performance dramatically, it will be a good improvement, when we do that."

Motherboard found their focus worthwhile.

"The payoff for research using the database could be huge. Machine learning using human demonstrations has already shown itself to be an impressive technique for quick learning. Researchers from the OpenAI institute recently unveiled an AI system that uses a single human demonstration in virtual reality to teach itself how to stack toy blocks."

Explore further: DeepMind researchers boost AI learning speed with UNREAL agent


— The Atari Grand Challenge Dataset, arXiv:1705.10998 [cs.AI]