Google's GameNGen simulates parts of video game Doom

A team of researchers from Google Research, Google Deep Mind and Tel Aviv University reports that it is possible to use machine learning applications to recreate and simulate parts or all of an existing video game.

The group has posted a paper on the arXiv preprint server describing their modification of an existing learning application, which they call GameNGen, and used it to reproduce and simulate short sections of the video game Doom.

Efforts to recreate video games with generative AI comprise two types of work: recreating the imagery and recreating the action. The process is called "neural rendering," and it is being investigated by several interested groups.

Like other AI applications, the science is based on the use of diffusion models, which are generative systems that allow a computer to create new data from old using special algorithms. In this new study, the research team wanted to determine whether they could faithfully simulate the game Doom by recreating it via machine learning.

The team started with Stable Diffusion 1.4, a diffusion model researchers at Google have been working on over several iterations. Its purpose is to create new images using machine learning. After tweaking the model, the researchers gave it the ability to learn from video games alone, instead of from everything on the internet, and named it GameNGen.

They trained it with video from internet sources showing screens of the game of Doom in progress while a human played. That data was used to teach the new system how the Doom world is supposed to look and how gameplay is supposed to occur. They then let it run and found that it could generate realistic new frames of gameplay at more than 20 frames per second using just a single TPU.

The researchers showed Doom snippets created by GameNGen to human raters and found that they were unable to tell the difference between the snippets and real game action more than half the time.

© 2024 Science X Network