Shared control allows a robot to use two hands working together to complete tasks

A team of researchers from the University of Wisconsin and the Naval Research Laboratory has designed and built a robotic system that allows for bimanual robot manipulation through shared control. In their paper published in the journal Science Robotics, the group explains the ideas behind their work and how well they worked in practice.

As the researchers note, using two hands working together to complete a task is very complicated—it entails far more than just two individual hands working independently on the same task at the same time. When a person opens a jar, for example, the brain has to serve as a mediator of sorts, directing the action as it receives and sends signals to both hands. Such simple tasks are so complicated that robots are not able to perform them—this is why virtually all robots work with just one hand. In this new effort, the researchers have taken one small step in the direction of teaching robots how to use two hands to complete a task.

The researchers note that many semi-robotic applications allow robots to serve an augmentation role, rather than do something on their own. A surgeon directing a robot hand by using his own hand is one example. But they also note that simple mimicry would not work for two-handed augmented robotic systems—such movements are too complex. The solution, they found, was to combine mimicry with a deep learning network. The result was a technique that enabled a robot to carry out bimanual tasks by sharing control with a human being.

The work started by equipping a robot with two arms and hands. They added hardware to allow the robot arms to communicate with a deep learning network and sensors placed on a human being. A robot was told which task was going to be attempted, and then the human carried it out. As the human did so, the robot did its best to mimic the action at the same time. Repeating the procedure many times allowed the robot to learn about the many tiny tasks that were involved in carrying out the main task, which led to some intuition on its part.

Over time, as a robot worked in conjunction with a human, it added its own commands to achieve a better result. The robot did not progress to the point of performing the task on its own—instead, it learned to serve as a more fully capable augmented assistant. The researchers note that such a robot could possibly serve as an assistant for partially disabled people. Also, it would seem the Navy could use such a robot for remote underwater operations.