A robot vision system that diminishes occlusions using mirror reflections

A robot vision system that diminishes occlusions using mirror reflections
The concept of our tilt-based reflection sensing system. Credit: Yoshioka et al.

To navigate their surroundings safely and most effectively, robots should be supported by a highly performing computer vision system. One of the greatest challenges when developing computer vision systems for robots is to limit occlusions so that the robot can sense most of its surroundings.

Researchers at Toshiba Corporation have recently developed a system that could reduce occlusions in robot vision systems. This system, presented in a paper pre-published on arXiv, uses tilting mirrors to increase the space covered by a robot's .

"Occlusion stand as a large challenge in robot vision systems," Kentaro Yoshioka, one of the researchers who carried out the study, told Tech Xplore. "Occlusions may cause errors in the robot's perception, causing overlooks or worse, even damage the robot or objects."

So far, researchers have devised two main approaches for reducing occlusions in robot vision systems. The first is based on the installation of more sensors or cameras in the environments a robot is exploring and the second on the integration of sensors directly onto mobile robots. While these approaches can diminish occlusions, they both have significant limitations.

"One way to reduce occlusions in robot vision systems is to install multiple sensors in the workplace," Yoshioka said. "While a straightforward approach, it can efficiently reduce occlusions without affecting the robot. However, this approach significantly increases the sensor cost. Alternatively, we can also install sensors onto the robot itself and reduce occlusions by manipulating the robot to sense from multiple viewpoints. Although the sensor cost can be reduced, this harms the robot's work-efficiency."

Video showing the robot in action. Credit: Yoshioka et al.

To eliminate occlusions to robot vision systems, Yoshioka and his colleagues proposed a new mechanism that tilts sensors to face a mirror and then retrieves images of the environment reflected on the mirror. Although other studies experimented with 3D sensing via mirror reflections, this team of researchers was among the first to exploit this technique to reduce or eliminate occlusions in robot vision systems.

"Our system works by sensing from the usual zenith sensor (calling it direct sensing) and detecting the occlusion from the direct sensing results," Yoshioka explained. "We set tilt the sensor with an optimal tilt angle so that we can eliminate the occlusions efficiently with reflection sensing."

The new robot vision approach proposed by Yoshioka and his colleagues has several advantages over existing systems. Firstly, it can be realized using cheap mirrors and tilt-units, hence it saves the cost of purchasing multiple expensive 3D sensors. In addition, the tilt motion performed by the sensors is a simple single-axis motion that can be performed much faster than a robot with integrated sensors would move its limbs to sense other parts of its surroundings.

"Our system detects the occlusion in the scene from the direct sensing results on-the-fly, and using the area coordinates, the optimal tilt angle is calculated," Yoshioka said. "Thus, even if the robot in the scene moves around, our system does not need reprogramming and it can adaptively reduce occlusions."

A robot vision system that diminishes occlusions using mirror reflections
Sensing and detection results are shown with several sensing setups. The researchers’ system (Direct+Reflection) can significantly reduce occlusions and misdetections. The sensing results are similar to those achieved when using two sensors in the scene. Credit: Yoshioka et al.

Yoshioka and his colleagues evaluated their system in a series of experiments using COBOTTA, a robotic arm developed by DENSO, a Japanese company owned by Toyota. They found that their system led to a significant improvement in accuracy compared to direct sensing systems. Remarkably, the team was able to achieve the same detection accuracy as that achieved by robot sensing systems that employ two 3D sensors.

In the future, the new approach presented in this recent paper could be used to improve the performance, mobility and navigation capabilities of both existing and emerging robotic systems. Meanwhile, Yoshioka and his colleagues plan to explore ways to improve their system's performance further.

"So far, our system only used a single mirror," Yoshioka said. "Extending this to multi-mirror setups will be our direction for future research. By adding more mirrors, we can adapt the system to more challenging case scenarios and environments (e.g., multi-robot setups)."

Yoshioka has recently left Toshiba. All opinions expressed here are his own and do not express the views or opinions of the company.

More information: Kentaro Yoshioka et al, Through the looking glass: diminishing occlusions in robot vision systems with mirror reflections (2021). arXiv:2108.13599v1 [cs.RO], arxiv.org/abs/2108.13599

© 2021 Science X Network

Citation: A robot vision system that diminishes occlusions using mirror reflections (2021, September 21) retrieved 30 May 2024 from https://techxplore.com/news/2021-09-robot-vision-diminishes-occlusions-mirror.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

A framework for robot path finding in unstructured environments


Feedback to editors