Real2sim2real: A self-supervised learning technique applied to planar robot casting

Real2sim2real: A self-supervised learning technique applied to planar robot casting
In Planar Robot Casting (PRC), a single planar motion of a robot wrist holding one end of a cable causes the other end of the cable to slide across the plane and stop at a desired target point, which may lie outside the robot workspace. (A) shows a side view of a UR5 robot with cable and planar workspace, (B) illustrates test performance on Cable 1 for 16 targets. The gold inner sector represents the robot workspace, while the grey outer sector represents the reachable workspace of the cable. (C,D) show multiple superimposed overhead views of the robot and cable with associated target points after PRC actions with the learned policy, in (C) an example with low error and in (D) an example with high error, in endpoint position. Credit: Lim et al.

Over the next few decades, robots could be introduced into human environments, including homes, offices and retail spaces. Among other things, robotic systems could be used to tidy up spaces and make them safer for humans.

While robots have achieved highly promising results so far, they cannot yet reliably manipulate deformable structures, such as power cables, ropes and hoses. Moreover, most robotic systems also cannot effectively manipulate two-dimensional deformable objects, such as clothes, napkins and bedding, or three-dimensional deformable objects, such as pillows, products, food items and bags.

Researchers at University of California Berkeley's AUTOLAB are working with researchers at the Toyota Research Institute (TRI) to enhance the ability of robots to manipulate deformable objects; specifically, to untangle cables and handle fabrics. In a recent paper pre-published on arXiv, they introduced a new self-supervised deep learning technique for planar casting, a task that involves the manipulation of cables on planar surfaces.

"While our prior work focuses more on (quasi-)static manipulation, this project explores the efficiency and effectiveness of dynamic motions for deformable object manipulation," Ken Goldberg, one of the researchers who carried out the study, told TechXplore. "In a previous paper, we focused on dynamic actions, such as "vaulting" to manipulate a cable with one endpoint fixed to a wall. Our new paper focuses on manipulating a free-end cable on a planar surface."

One of the key objectives of the recent study is to identify a new dynamic cable manipulation task, which they refer to as 'planar robot casting." To address it, they developed a self-supervised learning framework that can acquire control policies with little human intervention.

In their paper, they presented a specific pipeline for solving this task, dubbed 'real2sim2real." They also evaluated different simulation environments where the planar robot casting task could be simulated.

"Our self-supervised learning technique, real2sim2real, can speed up training and improve performance," Raven Huang and Vincent Lim explained. "To speed up learning, we want to use a realistic physics simulation. However, physics simulation in this context is not representative of reality. To address this, the robot first collects some real data in a self-supervised way, utilizing a pre-recorded reset motion so as not to acquire human intervention."

The real2sim2real pipeline uses the real data collected by Huang and Lim and their colleagues to tune the simulation of a planar robot casting task, so that it matches reality as closely as possible. Subsequently, it computes a much larger amount of simulated data, more safely, and at a far faster rate than it would be able to gather in the real world.

To learn control policies for robotic systems, the model uses a combination of real and simulated data. This dataset containing both real and simulated data is then also used to address discrepancies between performance in real and simulated environments, further enhancing its ability to tackle planar robot casting tasks.

"Compared to other approaches, which learn using purely simulated data or purely real data, our method balances the need to make the simulation as accurate as possible (what we call reducing the sim2real gap) and to learn as efficiently as possible," Lawrence Chen said. "Our framework also allows the robot to autonomously collect data for long periods of time."

Real2sim2real: A self-supervised learning technique applied to planar robot casting
The Real2Sim2Real pipeline for PRC. We collect a physical dataset in a self-supervised manner. We subsample the physical dataset to generate a tuning dataset, and use it to tune simulation parameters so that its trajectories match real trajectories using Differential Evolution (B), then use the tuned simulator to generate a large simulation dataset (C). We use a weighted combination of simulation dataset and physical dataset to train the policy (D) and evaluate the policy in real (E). Credit: Lim et al.

Goldberg and his team evaluated the self-supervised tool they developed in a series of tests. Remarkably, real2sim2real outperformed all the baseline methods they compared it to, as well as techniques trained only on simulated data or only on real data.

"We were surprised by the efficiency of the real2sim2real pipeline," Mike Laskey of TRI said. "We demonstrated its ability to learn a dynamic manipulation policy for deformable objects efficiently and achieve relatively high accuracy. The hybrid approach of combining simulated and real data both significantly improves performance and data efficiency."

The new method requires 96 percent less real data than other approaches. The rest of the data is created by the researchers' model in a reliably tuned simulation environment. This could ultimately facilitate the use of for managing cables in homes, boats, factories and other environments. In the future, the team hopes to apply the real2sim2real framework to other robot manipulation tasks.

"One of our future research directions will be to extend our method to more complex deformable objects, such as fabrics or bags," Goldberg said. "For such objects, we have an even higher dimension with more complex dynamic properties motivating us to combine the dynamic motion with (quasi-)static motions."

More information: Vincent Lim et al, Planar robot casting with real2sim2real self-supervised learning. arXiv:2111.04814v1 [cs.RO],

© 2021 Science X Network

Citation: Real2sim2real: A self-supervised learning technique applied to planar robot casting (2021, December 2) retrieved 12 April 2024 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

A system to transfer robotic dexterous manipulation skills from simulations to real robots


Feedback to editors