Researchers find a way to fool deep neural networks into 'recognizing' images that aren't there

Researchers find a way to fool deep neural networks into 'recognizing' images that aren’t there
Evolved images that are unrecognizable to humans, but that state-of-the-art DNNs trained on ImageNet believe with ≥ 99.6% certainty to be a familiar object. This result highlights differences between how DNNs and humans recognize objects. Images are either directly (top) or indirectly (bottom) encoded. Credit: arXiv:1412.1897

A trio of researchers in the U.S. has found that deep neural networks (DNNs) can be tricked into "believing" an image it is analyzing is of something recognizable to humans when in fact it isn't. They have written a paper about what they have discovered and uploaded it to the preprint server arXiv.

As time marches on, we humans are becoming more accustomed to computers being able to recognize things around us (faces on our smartphones, for example) and to do something with that information (pick out the face of a wanted person from a crowd). As part of that process we've come to believe that such systems are as good as they seem. But, as the trio working on this new effort has found, that assessment may be incorrect.

DNNs "learn" to recognize images by being exposed to many of those of the same type (thousands or millions of faces, for example)—they use learning algorithms that spot commonalities between parts of information in the images to map out different aspects of different objects. Once the learning has progressed to a certain level, the DNN is able to very accurately predict what object appears in an image, except, apparently, under certain circumstances. To find this rare circumstance, the researchers hooked a well known and respected DNN called AlexNet to a system that also included algorithms developed to evolve pictures using basic elements. The team expected the output to be exceptionally clear images of objects that most any person would instantly recognize. Instead, in many case, the result was a garbled mess, which the researchers described as static. More interesting, AlexNet offered confidence ratings up to 99 percent of the false images—the computer was certain the images were of things like lions, yet to the humans, they looked like static on an old TV set.

The reason for this apparent weakness in the DNN goes back to the way that they learn—all those parts that are supposed to add up to a discernible whole. If the algorithms creating the add all the basic parts the DNN is looking for, but not in a logical way, then the result can look like static to people looking at them while appearing to be what the DNN learned from its early training, because it's able to find those basic image parts.

This discovery is more than just passing interest, DNNs are used in applications such as by cars that drive themselves—if someone with ill intent were bent on harm, it might not be too difficult to imagine placing something on a highway that to us humans looked like fog or smoke, but to the car's computer, was identified as an object or perhaps a pedestrian, causing an accident to occur as the car sought to prevent a collision with the nonexistent object.


Explore further

On Welsh corgis, computer vision, and the power of deep learning

More information: Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images, arXiv:1412.1897 [cs.CV] arxiv.org/abs/1412.1897

Abstract
Deep neural networks (DNNs) have recently been achieving state-of-the-art performance on a variety of pattern-recognition tasks, most notably visual classification problems. Given that DNNs are now able to classify objects in images with near-human-level performance, questions naturally arise as to what differences remain between computer and human vision. A recent study revealed that changing an image (e.g. of a lion) in a way imperceptible to humans can cause a DNN to label the image as something else entirely (e.g. mislabeling a lion a library). Here we show a related result: it is easy to produce images that are completely unrecognizable to humans, but that state-of-the-art DNNs believe to be recognizable objects with 99.99% confidence (e.g. labeling with certainty that white noise static is a lion). Specifically, we take convolutional neural networks trained to perform well on either the ImageNet or MNIST datasets and then find images with evolutionary algorithms or gradient ascent that DNNs label with high confidence as belonging to each dataset class. It is possible to produce images totally unrecognizable to human eyes that DNNs believe with near certainty are familiar objects. Our results shed light on interesting differences between human vision and current DNNs, and raise questions about the generality of DNN computer vision.

© 2014 Tech Xplore

Citation: Researchers find a way to fool deep neural networks into 'recognizing' images that aren't there (2014, December 12) retrieved 23 October 2018 from https://techxplore.com/news/2014-12-deep-neural-networks-images.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.
1240 shares

Feedback to editors

User comments

Dec 12, 2014
That's the reason why you don't have neural networks in medical diagnostics (yet): You can't predit when they will fail.

Neural networks are awesome, but they sometimes latch onto incomplete or side channel information (which is perfectly OK if the training set and the test set have the same side channel behavior)

On the plus side this will lead to new research into recognizing 'sensible' images.

On a more sinister side this is a guideline to spoofing neural networks by 'weaponized images'. Think of something like this being posted at the side of a road when an autonomous vehicle passes by. The passenger would recognize nothing threatening (as it looks like static), but the DNN might recognize a curve or might swerve to evade a (non-present) obstacle.

Dec 12, 2014
So computerised image recognition is prone to false positives, just as humans are. Is this really a surprise?!!

Dec 12, 2014
Interesting article! Couldn't this problem be overcome by implementing a filter which looks for noise or clearly repeating patterns before the picture is handed to the DNN?

Dec 12, 2014
"So computerised image recognition is prone to false positives, just as humans are. Is this really a surprise?!!"

That you can fool image recognition isn't much of a surprise (CAPTCHA systems have been actively trying to do that for years). That you can fool it with something that looks totally unlike the thing (to humans) it's supposed to recognize is.

It's like fooling a person into seeing a lion where there is a house.

Dec 12, 2014
So the DNN weren't trained on gross outliers. The real question is if they had them in their learning itinerary, whether that would have knocked down their final visual recognition scores. Cheap trick visual hack on work-in-progress or crippling intrinsic limitation that sinks the whole program?

Dec 12, 2014

On a more sinister side this is a guideline to spoofing neural networks by 'weaponized images'. Think of something like this being posted at the side of a road when an autonomous vehicle passes by. The passenger would recognize nothing threatening (as it looks like static), but the DNN might recognize a curve or might swerve to evade a (non-present) obstacle.
-- Antialias

Um, yeah, how about that. You came up with an idea that is almost a precise summary of the final paragraph of this article. Thank you for that contribution. Maybe read the whole thing next time? sigh...

Dec 13, 2014
DNN has nothing to do with where and how information is stored in the mind and brain.
As long as this fundamental difference is ignore you will continue to read comments
comparing biological intelligence and perception with "A.I. versions of illusion".


Please sign in to add a comment. Registration is free, and takes less than a minute. Read more