A new approach to reproduce human and animal movements in robots

In recent years, developers have created a wide range of sophisticated robots that can operate in specific environments in increasingly efficient ways. The body structure of many among these systems is inspired by nature, animals, and humans.

Although many existing robots have bodies that resemble those of humans or other animal species, programming them so that they also move like the animal they are inspired by is not always an easy task. Doing this typically entails the development of advanced locomotion controllers, which can require considerable resources and development efforts.

Researchers at DeepMind have recently created a new technique that can be used to efficiently train robots to replicate the movements of humans or animals. This new tool, introduced in a paper pre-published on arXiv, is inspired from previous work that leveraged data representing real-world human and animal movements, collected using motion capture technology.

"We investigate the use of prior knowledge of human and animal movement to learn reusable locomotion skills for real legged robots," the team at DeepMind wrote in their paper. "Our approach builds upon previous work on imitating human or dog Motion Capture (MoCap) data to learn a movement skill module. Once learned, this skill module can be reused for complex downstream tasks."

A large part of the robot locomotion controllers developed in the past have modular designs, in which a system is divided into different parts (i.e., modules), which interact with each other. While some of these controllers have achieved promising results, developing them often requires significant engineering efforts. In addition, modular designs are typically task-specific, thus they do not generalize well across different tasks, situations, and environments.

As an alternative to these controllers, some researchers have proposed a method called "trajectory optimization," which combines a motion planner with a tracking controller. These approaches require less engineering than modular controllers, yet they often need to perform extensive computations and thus can be too slow to be applied in real-time.

In their paper, Steven Bohez and his colleagues at DeepMind introduced an alternative approach for training humanoid and legged robots to move in ways that resemble the locomotion styles of humans and animals. Their technique summarizes the motor skills of humans and animals from data collected with motion capture technology, then uses this data to train real-world robots.

When developing their approach, the team completed four main stages. Firstly, they re-targeted motion capture data to real-world robots. Subsequently, they trained a policy to imitate desired motion trajectories in the motion capture data within a simulated environment.

"This policy has a hierarchical structure in which a tracking policy encodes the desired reference trajectory into a latent action that subsequentially instructs a proprioception-conditioned low-level controller," the researchers wrote in their paper.

After they trained this policy to imitate reference trajectories, the researchers were able to reuse the low-level controller, which has fixed parameters, by training a new task policy to output latent actions. This allows their controllers to replicate complex human or animal movements in robots, such as dribbling a ball. Finally, Bohez and his colleagues transferred the controllers they developed from simulations to real hardware.

"Importantly, due to the prior imposed by the MoCap data, our approach does not require extensive reward engineering to produce sensible and natural looking behavior at the time of reuse," the researchers wrote in their paper. "This makes it easy to create well-regularized, task-oriented controllers that are suitable for deployment on real robots."

So far, the team at DeepMind evaluated their approach in a series of experiments, both in simulation and real-world environments. In these tests, they successfully used their technique to train the controller to replicate two main behaviors, namely walking and ball dribbling. Subsequently, they evaluated the quality of the movements achieved using their approach on two real-world robots: the ANYmal quadruped and OP3 humanoid robots.

The results collected by Bohez and his colleagues are very promising, suggesting that their approach could help to develop robots that emulate humans and animals more realistically. In their next studies, they would like to train their policies on new animal and human behaviors, to then try and replicate them in robots.

"We want to extend our datasets with a larger variety of behaviors and further explore the range of downstream tasks that the skill module enables," the researchers wrote in their paper.