Using a simulation framework to study spine behaviors of quadruped robots

Researchers at the Robert Bosch center for cyber physical systems in Bangalore, India, have recently proposed a simulation framework to systematically study the effects of spinal joint actuation on the locomotion performance of quadruped robots. In their study, outlined in a paper pre-published on arXiv, they used this framework to investigate the spine behaviors of a quadruped robot called Stoch 2 and their effects on its bounding performance.

"This research came about while exploring the basis of fast locomotion in quadruped animals and robots, Shounak Bhattacharya, one of the researchers who carried out the study, told TechXplore. "It has been well known for some time now that small and medium-sized quadruped animals use the flexibility of their spine to improve their locomotion speed and energy efficiency. This use of flexibility inspired roboticists to explore the concept in more detail to find a solution to this problem using the mathematical tools available."

Contrarily to previous researchers, Bhattacharya and his colleagues set out to investigate the spine behaviors of quadruped robots using machine learning techniques. The key objective of their study was to use deep reinforcement learning (D-RL) to realize the complex coordination between spine and legs in robots.

"In a D-RL framework, learning from scratch can lead to better and robust policies than incorporating human guidance," Bhattacharya said. "However, learning from scratch involves multiple initial failures before the generation of working policy. In this work, over 2 million steps were required before an active and usable policy was obtained."

Carrying out many trials directly on a robot's hardware requires a vast amount of time and resources. The researchers thus decided to simulate the robot and its environment in PyBullet, a python module that enhances the Bullet physics engine with robotics and machine learning capabilities. In their study, they used PyBullet to evaluate the effects of spinal joint actuation on the bounding performance of Stoch 2, a 16-DOF quadruped spined robot.

"We trained the robot model in this environment and observed that the learning algorithm executed one complete training in 4 hours, when it is executed for a maximum of 10 million steps, on a PC powered by an Intel Core i7 at 3.5Ghz with 12 cores, and 32 GB RAM," Bhattacharya said. "The use of a simulation framework reduced the time per training and removed the necessity to experiment on the hardware."

Using PyBullet as a learning framework for their robot, the researchers attained very promising results. The findings they gathered during simulations suggest that the active use of the spine does, in fact, improve a robot's stride length and cost of transport, while also reducing the natural frequency to more realistic values. Ultimately, Stoch 2 achieved a bounding speed of 2.1 m/s, with a maximum Froude number of 2.

This study, supervised by four faculty members at the Indian Institute of Science: Shishir Kolathaya, Ashitava Ghosal, Bharadwaj Amrutur and Shalabh Bhatnagar, is part of a broader project called Walking Robot. In the future, it could inspire other researchers to use the same simulation framework to enhance their robots' spine behaviors and the consequent locomotion performance.

"We obtained the coordination of the spine and leg through a D-RL based framework, which increased energy efficiency and speed of the quadruped," Bhattacharya said. "It must be noted that all these behaviors were obtained from scratch without any understanding of the mechanics of the spine. In our future work, we plan to deploy the neural network on the hardware and execute the learned policy directly."

© 2019 Science X Network