This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:



trusted source


Barkour benchmark measures quadruped robot agility

Barkour benchmark measures quadruped robot agility
Overview of the Barkour benchmark’s obstacle course setup, which consists of weave poles, an A-frame, a broad jump, and pause tables. The intuitive scoring mechanism, inspired by dog agility competitions, balances speed, agility and performance and can be easily modified to incorporate other types of obstacles or course configurations. Credit: Google

The accelerated pace of robotics development has given us a veritable zoo filled with creatures sometimes indistinguishable from the real deal.

We've got RoboFish, Bionic Kangaroo, Robot Snake, and even RoboTuna, Bionic Ants and RoboSalamander.

Of particular interest to developers, however, are quadruped creations. They provide invaluable opportunities to study , stability and ruggedness over varying terrains that will help develop robotic solutions for disaster response, medical treatment, environmental monitoring and surveillance.

Spot, from Boston Dynamics, for example, has grabbed headlines as a law enforcement tool in Honolulu, a monitor in Singapore enforcing social distancing during the COVID-19 pandemic, a thermal-camera sentry at Haneda Airport in Tokyo and an aide at construction sites in Denmark.

There are other four-legged beasts in this race: AIbo, ANYmal, WildCat and Hyq.

One challenge facing researchers and potential customers of robotic quadrupeds is how to measure the differences in capabilities of a growing menagerie of AI-based critters.

The team invited a handful of dooglers to try the obstacle course to ensure that our agility objectives were realistic and challenging. Small dogs complete the obstacle course in approximately 10s, whereas our robot’s typical performance hovers around 20s. Credit: Google

After all, there are tests for seemingly everything digital: Dhrystone measures performance of integers operations on a CPU; N-Body Simulation checks performance of simulating the motion of particles in a ; Whetstone is used for floating point performance on a CPU; Pro Agility Shuttle for lateral quickness and change of direction; and T-Test for shuffling, sprinting and backpedaling. Scores of additional tests assess countless other attributes of digital machines.

Research scientists at Google say it's time to zero in on a sound measurement tool specifically for robotic agility.

"[W]hile researchers have enabled robots to hike or jump over some obstacles, there is still no generally accepted benchmark that comprehensively measures agility or mobility," the Google DeepMind team reported in a paper posted on the pre-print server arXiv last week. "In contrast, benchmarks are driving forces behind the development of machine learning, such as ImageNet for computer vision, and OpenAI Gym for reinforcement learning."

The Google researchers designed an that includes weave poles, A-frames and a jump board. A rewards tasks completed successfully within a specified period of time, penalizing entrants who fail or skip challenges. The project is called Barkour.

"It is essential to establish metrics that can accurately measure robot agility and to define a standard set of tasks that can serve as a common evaluation framework," the team said in their paper, "Barkour: Benchmarking Animal-level Agility with Quadruped Robots."

The course was tested on real dogs first—"dooglers," in this instance, who were pets owned by Google staff members.

The team then trained robotic quadrupeds to navigate the varying terrains while efficiently accommodating balance and speed demands. The challenges included walking, slope climbing and jumping.

While the team achieved a practical set of measurements to test agility, they reported that live untrained dogs were able to successfully tackle the obstacle course in about half the time it took their custom-built quadruped robot to do so.

"There is still big gap in agility between robots and their animal counterparts," the team acknowledged. "There is still notable room to push towards dog-level by improving speed and robustness."

But they maintained that "Barkour will serve the robotics community as an important testbed for different learning and control methods and different hardware designs" in the future.

More information: Ken Caluwaerts et al, Barkour: Benchmarking Animal-level Agility with Quadruped Robots, arXiv (2023). DOI: 10.48550/arxiv.2305.14654

Journal information: arXiv

© 2023 Science X Network

Citation: Barkour benchmark measures quadruped robot agility (2023, June 1) retrieved 1 October 2023 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Team designs four-legged robotic system that can walk a balance beam


Feedback to editors