Extra 'eye' movements are the key to better self-driving cars

self-driving car
Credit: Pixabay/CC0 Public Domain

Andrea Benucci and colleagues at the RIKEN Center for Brain Science has developed a way to create artificial neural networks that learn to recognize objects faster and more accurately. The study, recently published in the scientific journal PLOS Computational Biology, focuses on all the unnoticed eye movements that we make, and shows that they serve a vital purpose in allowing us to stably recognize objects. These findings can be applied to machine vision, for example, making it easier for self-driving cars to learn how to recognize important features on the road.

Despite making constant head and throughout the day, objects in the world do not blur or become unrecognizable, even though the physical information hitting our retinas changes constantly. What likely make this perceptual stability possible are neural copies of the movement commands. These copies are sent throughout the brain each time we move and are thought to allow the brain to account for our own movements and keep our perception stable.

In addition to stable perception, evidence suggests that eye movements, and their motor copies, might also help us to stably recognize objects in the world, but how this happens remains a mystery. Benucci developed a (CNN) that offers a solution to this problem. The CNN was designed to optimize the classification of objects in a visual scene while the eyes are moving.

First, the network was trained to classify 60,000 black and white images into 10 categories. Although it performed well on these images, when tested with shifted images that mimicked naturally altered visual input that would occur when the eyes move, performance dropped drastically to chance level. However, classification improved significantly after training the network with shifted images, as long as the direction and size of the eye movements that resulted in the shift were also included.

In particular, adding the eye movements and their motor copies to the network model allowed the system to better cope with visual noise in the images. "This advancement will help avoid dangerous mistakes in ," says Benucci. "With more efficient and robust machine vision, it is less likely that pixel alterations—also known as 'adversarial attacks'—will cause, for example, self-driving cars to label a stop sign as a light pole, or military drones to misclassify a hospital building as an enemy target."

Bringing these results to real world machine vision is not as difficult as it seems. As Benucci explains, "the benefits of mimicking eye movements and their efferent copies implies that 'forcing' a machine-vision sensor to have controlled types of movements, while informing the vision network in charge of processing the associated images about the self-generated movements, would make machine vision more robust, and akin to what is experienced in ."

The next step in this research will involve collaboration with colleagues working with neuromorphic technologies. The idea is to implement actual silicon-based circuits based on the principles highlighted in this study and test whether they improve machine-vision capabilities in real world applications.

More information: Andrea Benucci et al, Motor-related signals support localization invariance for stable visual perception, PLOS Computational Biology (2022). DOI: 10.1371/journal.pcbi.1009928

Journal information: PLoS Computational Biology
Provided by RIKEN
Citation: Extra 'eye' movements are the key to better self-driving cars (2022, June 28) retrieved 19 May 2024 from https://techxplore.com/news/2022-06-extra-eye-movements-key-self-driving.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

The benefits of peripheral vision for machines


Feedback to editors