A model to improve robots' ability to hand over objects to humans

For decades, researchers worldwide have been trying to develop robots that can efficiently assist humans and work alongside them as they tackle a variety of everyday tasks. To do this effectively, however, the robots should be able to interact naturally with humans, including handing them and receiving objects from them.

Researchers at NVIDIA have recently developed a model that could be used to enhance the ability of robots to naturally pass and receive objects from human agents. This approach, introduced in a paper set to be presented at ICRA 2022, is based on a framework called STORM, which they presented in one of their previous works.

"In this work, we focus specifically on making sure that robots can take any object from a human user," Dieter Fox, one of the researchers who carried out the study, told TechXplore. "We came up with a solution that integrates a learned approach for grasping with our predictive control approach, in order to ensure these handovers are predictable, natural, and fast, so that people see the robot as a safe, helpful assistant."

Fox and his colleagues at NVIDIA have been working on improving the manipulation and interaction skills of robots for several years now. The structure of the model presented in their new paper, however, differs from the approaches presented in their previous works.

More specifically, their newly presented approach initially uses a learned model to predict several ways in which the robot could take an object from a human agent's hand. Then, instead of using a simple rule-based algorithm to determine which of these ways is more effective, it identifies the optimal approach using STORM, the model predictive control (MPC) framework previously created by the team.

"Our MPC approach, dubbed STORM, works by sampling many different possible motions the robot could make from the current location to where the object is," Fox explained. "It leverages GPU computations to evaluate these trajectories, allowing us to quickly check 500 of them in parallel."

The model created by Fox and his colleagues updates the robot's control decisions several times per second. This ultimately allows it to quickly adapt the robot's planned trajectory and its decisions about where to grasp objects based on the movements performed by the user it is collaborating with.

"This work shows how we can combine learned object grasping and human tracking techniques with efficient motion planning to generate robust, reliable, and natural robot behavior," Fox said. "It provides a foundation to build all kinds of human-robot collaborative behavior."

Fox and his colleagues evaluated their model's performance in a series of experiments where robots handed over and received a variety of objects while interacting with four human users. In these tests, their approach achieved very promising results, as almost all the participants felt that the robot was better at collaborating with them when powered by the researchers MPC framework than when it was based on a baseline approach.

In the future, the new model introduced by this team at NVIDIA could be used to improve the performance of both existing and newly developed robots on tasks that involve the manipulation of objects in close collaboration with human agents. Meanwhile, the team plans to develop other tools to enhance human-robot collaboration.

"More broadly, approaches like this that combine the power of deep learning with planning-based reasoning will be useful for many applications," Fox added. "In our next studies, we would like to explore more general human-robot collaborative systems to enable the robot to work with humans efficiently, while also exploring the possibility of training these robot systems virtually in simulated environments and deploying the trained models on real-world robots."