An IKEA furniture assembly environment to train robots on complex manipulation tasks

An IKEA furniture assembly environment to train robots on complex manipulation tasks
Screenshots of the IKEA furniture assembly environment. Credit: Lee et al.

In order to complete complex everyday tasks such as using equipment, cooking or building furniture, robots should be able to plan their actions and manipulate objects in their surroundings. So far, however, teaching robots to complete complicated tasks, such as those that require planning over a considerable period of time, has proved to be rather challenging, also due to a lack of reliable simulated environments to test them in.

With this in mind, researchers at the University of Southern Carolina have recently developed the IKEA assembly environment, a simulation platform where researchers can test artificial intelligence (AI) agents on complex manipulation tasks. In the environment they developed, presented in a paper prepublished on arXiv, agents can be evaluated on a variety of manipulation tasks that involve building and manipulating different items of furniture.

"The environment is designed to advance reinforcement learning from simple toy tasks to complex tasks requiring both and sophisticated low-level control," the researchers wrote in their paper.

Assembling furniture can be a challenging even for humans, as it typically requires both long-term planning and sophisticated manipulation skills. The environment developed by the researchers, which is fairly easy to use, has several interesting functions. As it generates a vast amount of synthetic labeled data, it can be used to train on a wide range of tasks, including object pose estimation, scene understanding, and many more, without requiring human-annotated data.

In addition, the environment could serve as a benchmark for machine learning methods designed for furniture assembly or other long-horizon manipulation tasks, improving their control and planning capabilities. Interestingly, the visual and interactive data produced by the platform can also be used to acquire domain-specific knowledge for other applications, such as intuitive physics models.

An IKEA furniture assembly environment to train robots on complex manipulation tasks
The three robots currently supported by the platform. Credit: Lee et al.

The IKEA furniture assembly environment supports over 80 models of furniture and can be customized with background images, lighting and textures. It could ultimately be seen as a testbed for machine learning techniques designed to give robots advanced object manipulation skills.

So far, the environment can be used to train or test three robots of different shapes and sizes, called Cursor, Sawyer and Baxter. In the next update, however, the researchers also plan to add support for Fetch, UR, Jaco and other popular robots.

In the future, this customizable could open up new possibilities for training and evaluating numerous machine learning techniques for robotics applications. In the meantime, the researchers plan to update the platform and improve some of its functions.

For instance, they would like to add support for 3-D motion devices, allowing users to operate robots remotely and create demonstration videos using a VR controller or 3-D mouse. These demonstration videos could then be used to train machine learning models, using a strategy known as imitation learning.

In future versions of the platform, users could also be allowed to guide robots via spoken instructions, and might even be able to train multiple agents simultaneously. Moreover, robots could eventually be trained on how to use specific tools, such as screwdrivers and hammers.

More information: IKEA furniture assembly environment for long-horizon complex manipulation tasks. arXiv:1911.07246 [cs.RO].

© 2019 Science X Network

Citation: An IKEA furniture assembly environment to train robots on complex manipulation tasks (2019, December 13) retrieved 4 December 2022 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Using imitation and reinforcement learning to tackle long-horizon robotic tasks


Feedback to editors