A reinforcement learning framework to improve the soccer shooting skills of quadruped robots

Researchers University of California, Berkeley (UC Berkeley), Université de Montréal and Mila have recently developed a hierarchical reinforcement learning framework to improve the precision of quadrupedal robots in soccer shooting. This framework, introduced in a paper pre-published on arXiv, was deployed on a Unitree A1, a quadruped robot developed by UnitreeRobotics.

"Human legs are not only for locomotion but can also be used for manipulation like playing soccer, and we want to enable quadrupedal robots to also attain this ability," Zhongyu Li, one of the researchers who carried out the study, told TechXplore. "There is a notable league in the robotics community called 'RoboCup' (Robot World Cup), that has been inviting researchers to train their robots to play soccer games for decades."

Recent advancements have enabled the creation of more reliable hardware and advanced control algorithms for robots. As a result, robots are now more agile and could potentially tackle more complex tasks, including playing soccer alongside humans. The framework developed by Li and his colleagues could help to improve the ability of quadruped robots to shoot the ball during soccer games.

The new framework has two key components: a motion control policy and a motion planning policy. The motion control component allows the robot to track an arbitrary trajectory for the toe on its kicking leg. The motion planning policy, on the other hand, selects an optimal toe trajectory to shoot a nearby soccer ball (detected by an external camera) to a target location (e.g., the goalpost).

"Our design allows us to decouple the challenge of the precise soccer shooing task into the two subtasks: control and planning," Li said. "We can first focus on training a robust control policy that can work on the hardware, then re-use such a controller to learn the planning strategy. To precisely shoot the ball to targets in the real world, the planner is trained using the real-world data when the robot is shooting the real soccer ball."

Li and his colleagues tested their framework in a series of real-world tests, using an A1 quadruped robot. They found that it allowed the robot to shoot a deformable soccer ball to random targets with high precision. This is a very complex task to tackle, as the robot needs to swing its kicking leg quickly and gain momentum without losing its balance.

"The soccer ball introduces more challenges because the robot needs to deal with not only the hard-to-model soft contact with the deformable ball but also the uncertainties of the rolling friction between the ball and the ground," Li said. "The methodology we developed to address such problems could be potentially useful for the tasks where dynamic robots, such as legged robots, have to interact with soft objects, such as ball, ropes, leash, clothes, etc."

• 

• 

• 

In the future, the framework created by this team of researchers could be used to improve the performance of robots in soccer tournaments, particularly Robocup. Meanwhile, Li and his colleagues plan to devise other frameworks and machine learning models to improve the performance of robots in other elements of soccer playing.

"Our long-term goal is to develop quadrupedal robotic soccer players that could one day compete with humans," Li added. "We are developing more complex soccer skills using quadrupedal robots and hopefully, in the near future, we can start a fully autonomous soccer game using quadrupedal robots."