VRKitchen: An interactive virtual environment to train and test AI agents

**VRKitchen: an interactive virtual environment to train and test AI agents
A sample sequence of an agent making a sandwich in VRKitchen. Credit: Gao et al.

A team of researchers at the University of California Los Angeles (UCLA), under the supervision of Prof. Song-Chun Zhu, has developed an interactive 3-D virtual environment for training and evaluating AI systems. Their platform, called VRKitchen, was recently presented in a paper pre-published on arXiv.

"Training robots in the real world is expensive, slow and hardly reproducible," Tianmin Shu, a Ph.D. student at UCLA who carried out the study, told TechXplore. "Because of this, in recent years, we have witnessed great progress in building physics-enabled 3-D virtual platforms (e.g. AI2-THOR, House3D) where embodied AI agents can walk around and find a target object."

Despite their many advantages, most existing virtual platforms for training AI systems do not enable with physical objects, which are very common in real-world situations. In addition, many of these platforms do not allow humans to take part in the training of agents within the virtual .

"In this work, we aim to build a highly interactive and realistic virtual platform, allowing researchers to train and evaluate their AI agents in a large range of real world tasks involving complex object manipulations (e.g. cooking a meal)," Shu said. "A further goal of our research is to build VR interfaces that allow humans to teach AI agents by giving demonstrations in the same environment."

VRKitchen, the virtual environment created by Shu and his colleagues, is based on Unreal Engine 4, a suite of tools generally used by game developers to design games or simulations. Compared to existing virtual environments, VRKitchen has two key advantages.

First, in VRKitchen, an AI agent can perform a wide range of cooking tasks with rich object state changes and compositional goals. For instance, if an agent is learning to make pizza, the virtual environment simulates visual effects of the cheese being grated, tomatoes being cut and the ingredients being placed onto the dough.

"We really hoped that the ability to simulate these complex cooking tasks would make VRKitchen stand out from other platforms," Xiaofeng Gao, Ph.D. student at UCLA who carried out the study, told TechXplore.

A second advantage of VRKitchen is that the embodied agent navigating the virtual environment can be controlled by both AI algorithms and human users. This allows humans to give demonstrations to AI agents, so that the latter can acquire knowledge about how to complete a task via observation, rather than having to learn the independently.

"We built three major modules in VRKitchen," Gao explained. "(1) the UE4 (Unreal Engine 4)-based physics engine and photorealistic rendering module with several humanoid agents, and a set of customizable kitchen scenes consisting of various furniture, kitchen appliances, food ingredients, containers, and tools for cooking activities; (2) a user interface module that allows users or algorithms to perform tasks by virtual reality device or Python API; (3) a Python-UE4 bridge, which transfers high-level commands to motor control signals and sends them to the agent."

The developed by Shu, Gao and their colleagues could soon be used to train agents powered by a broad range of machine learning techniques on complex tasks that involve fine-grained object manipulation. Along with VRKitchen, the researchers presented new standardized evaluation benchmarks and data collection tools, which are designed to facilitate the use of their platform in research settings.

"Popular approaches in recent literature that work well in simpler virtual environments fail to achieve decent performance on the cooking tasks implemented in our platform, imposing a new challenge for AI research and highlighting the importance of building more realistic and complex virtual environments," "Shu said. "We now plan to open source our and host standardized challenges (VR Chef Challenge) for the AI community. We are also working on building AI agents that can efficiently acquire complex skills by exploring the virtual world themselves as well as by learning from human demonstrations."

More information: VRKItchen: an interactive 3-D virtual environment for task-oriented learning. arXiv:1903.05757 [cs.HC]. arxiv.org/abs/1903.05757


© 2019 Science X Network

Citation: VRKitchen: An interactive virtual environment to train and test AI agents (2019, March 27) retrieved 20 February 2024 from https://techxplore.com/news/2019-03-vrkitchen-interactive-virtual-environment-ai.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

When little robot will go through your rooms to find the orange purse


Feedback to editors