A reinforcement learning framework to improve the soccer shooting skills of quadruped robots

A reinforcement learning framework to improve the soccer shooting skills of quadruped robots
Credit: Ji et al.

Researchers University of California, Berkeley (UC Berkeley), Université de Montréal and Mila have recently developed a hierarchical reinforcement learning framework to improve the precision of quadrupedal robots in soccer shooting. This framework, introduced in a paper pre-published on arXiv, was deployed on a Unitree A1, a quadruped robot developed by UnitreeRobotics.

"Human legs are not only for locomotion but can also be used for manipulation like playing , and we want to enable quadrupedal robots to also attain this ability," Zhongyu Li, one of the researchers who carried out the study, told TechXplore. "There is a notable league in the robotics community called 'RoboCup' (Robot World Cup), that has been inviting researchers to train their robots to play soccer games for decades."

Recent advancements have enabled the creation of more reliable hardware and advanced control algorithms for robots. As a result, robots are now more agile and could potentially tackle more , including playing soccer alongside humans. The developed by Li and his colleagues could help to improve the ability of quadruped robots to shoot the ball during soccer games.

The new framework has two key components: a motion control policy and a motion planning policy. The motion control component allows the to track an arbitrary trajectory for the toe on its kicking leg. The motion planning policy, on the other hand, selects an optimal toe trajectory to shoot a nearby (detected by an external camera) to a target location (e.g., the goalpost).

A reinforcement learning framework to improve the soccer shooting skills of quadruped robots
Credit: Ji et al.

"Our design allows us to decouple the challenge of the precise soccer shooing task into the two subtasks: control and planning," Li said. "We can first focus on training a robust control policy that can work on the hardware, then re-use such a controller to learn the planning strategy. To precisely shoot the ball to targets in the real world, the planner is trained using the real-world data when the robot is shooting the real soccer ball."

Li and his colleagues tested their framework in a series of tests, using an A1 quadruped robot. They found that it allowed the robot to shoot a deformable soccer ball to random targets with high precision. This is a very complex task to tackle, as the robot needs to swing its kicking leg quickly and gain momentum without losing its balance.

"The soccer ball introduces more challenges because the robot needs to deal with not only the hard-to-model soft contact with the deformable ball but also the uncertainties of the rolling friction between the ball and the ground," Li said. "The methodology we developed to address such problems could be potentially useful for the tasks where dynamic robots, such as legged robots, have to interact with soft objects, such as ball, ropes, leash, clothes, etc."

  • A reinforcement learning framework to improve the soccer shooting skills of quadruped robots
    Credit: Ji et al.
  • A reinforcement learning framework to improve the soccer shooting skills of quadruped robots
    Credit: Ji et al.
  • A reinforcement learning framework to improve the soccer shooting skills of quadruped robots
    Credit: Ji et al.

In the future, the framework created by this team of researchers could be used to improve the performance of robots in soccer tournaments, particularly Robocup. Meanwhile, Li and his colleagues plan to devise other frameworks and machine learning models to improve the performance of robots in other elements of soccer playing.

"Our long-term goal is to develop quadrupedal robotic soccer players that could one day compete with humans," Li added. "We are developing more complex soccer skills using quadrupedal robots and hopefully, in the near future, we can start a fully autonomous soccer game using quadrupedal robots."

More information: Yandong Ji et al, Hierarchical reinforcement learning for precise soccer shooting skills using a quadrupedal robot. arXiv:2208.01160v1 [cs.RO], arxiv.org/abs/2208.01160

© 2022 Science X Network

Citation: A reinforcement learning framework to improve the soccer shooting skills of quadruped robots (2022, August 22) retrieved 2 December 2022 from https://techxplore.com/news/2022-08-framework-soccer-skills-quadruped-robots.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

A Q-learning algorithm to generate shots for walking robots in soccer simulations

73 shares

Feedback to editors