When all the world's a toaster, according to tricked AI

When all the world's a toaster, according to tricked AI

Image recognition technology can be duped by psychedelic stickers created by a Google team.

The stickers made the tech to "see" things that were not there.

Emma Sims in Alphr: "machine learning systems can be distracted with highly localised psychedelic stickers, causing an oversight in broader computer vision."

The lowly toaster decked out with computer generated patterns took center-stage as a research example, with the toaster-inspired patterns distracting image software. In short, Google researchers figured out how to trick into thinking something not a toaster was a toaster.

"AI loves a psychedelic aesthetic." That was a subhead in Alphr and that was exactly the problem that pushed a machine learning system off the rails.

Think about that. Sims mused, "In other words, something that would sell like hotcakes for £8.99 in Urban Outfitters has the capacity to deceive highly advanced machine learning systems."

As described in BBC News: "When the patterns were put next to another item, such as a banana, many neural networks saw a toaster instead."

Sims in Alphr explained what made the stickers so intoxicating. "AI uses cognitive shortcuts, as humans do, to visually apprehend images." The Google team came up with mesmerising visuals on which AI involuntarily fixated; the "funky psychedelic stickers" lured the AI "away from what it should be focusing on."

Google researcher Tom Brown said, "Our adversarial patch is more effective than a picture of a real toaster, even when the patch is significantly smaller than the toaster."

BBC News quoted them as saying, "These adversarial patches can be printed, added to any scene, photographed, and presented to image classifiers."

The team wrote a paper discussing their work, titled "Adversarial Patch." The patch is described in detail. Authors are Tom Brown, Dandelion Mané, Aurko Roy, Martín Abadi and Justin Gilmer; the paper is on arXiv.

When a photo of a tabletop with a banana and a notebook is passed through VGG16, the team said in their paper, the network reports class 'banana' with 97% confidence. If they place a targeted to the class "toaster" on the table, the photograph is classified as a toaster with 99% confidence.

What is VGG16? This is a convolutional neural network architecture named after the Visual Geometry Group from Oxford, who developed it.

The BBC report noted that the pattern consistently tricked image recognition software when it took up at least 10% of a scene.

Because this patch is scene-independent, it allows attackers to create a physical-world attack without prior knowledge of the lighting conditions, camera angle, type of classifier being attacked, or even the other items within the scene, the authors said.

Thomas Claburn in The Register raised the point that "The attack differs from other approaches in that it doesn't rely on altering an image with graphic artifacts. Rather, it involves adding the adversarial to the scene being captured by ."

Google's researchers, as part of their continued exploration into artificial intelligence, are always interested in learning how AI might be tricked. "The team said the method could be used to 'attack' image recognition systems," said the BBC.

"Practical applications for the discovery include the bypassing of security systems at airports or prisons, allowing contraband material to elude recognition," said Sims.


Explore further

When is a baseball espresso? Neural network tricked and it is no joke

More information: Adversarial Patch, arXiv:1712.09665 [cs.CV] arxiv.org/abs/1712.09665

Abstract
We present a method to create universal, robust, targeted adversarial image patches in the real world. The patches are universal because they can be used to attack any scene, robust because they work under a wide variety of transformations, and targeted because they can cause a classifier to output any target class. These adversarial patches can be printed, added to any scene, photographed, and presented to image classifiers; even when the patches are small, they cause the classifiers to ignore the other items in the scene and report a chosen target class.

© 2018 Tech Xplore

Citation: When all the world's a toaster, according to tricked AI (2018, January 5) retrieved 13 December 2018 from https://techxplore.com/news/2018-01-world-toaster-ai.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.
46 shares

Feedback to editors

User comments

Jan 05, 2018
"machine learning systems can be distracted with highly localised psychedelic stickers, causing an oversight in broader computer vision."

After I saw this being reported a couple years ago I was already wondering if autonomous vehicles can be tricked the same way (or the other way around: I find it absolutely essential that such systems don't rely on just one kind of sensor)

Of course, as an attacker, you can always let an AI of your own learn what works best against another AI. Just like in real life: who learns last, wins.


Jan 05, 2018
"I find it absolutely essential that such systems don't rely on just one kind of sensor"


Cameras and other passive sensors are the most robust sensors you can have. Any active sensor, like a radar or a sonar, is going to run into trouble with other sensors in other cars - they're less dependable to return useful data under a variety of conditions, than simply looking out the window.

And with multiple sensors, sensory fusion requires judgement over which sensor to trust in which cases, which the AI systems cannot do for the moment. When it gets conflicting information, the computer has to solve the conflict to figure out which sensor is still reliable, but it doesn't have the smarts to do that.

Same with people. When your inner ear gets confused by the bumps and turns in the road, you feel dizzy and lose your "butt feel" of the car, so you have to rely on what you see out the window. Vision is the main fallback.

Jan 05, 2018
So basically, the approach of stacking multiple sensors in a self-driving car in hopes that some of them will work, is just a hack patch on the bigger problem that the AI is not sufficiently powerful to drive the car in the first place.

It's covering the issue that the AI cannot actually make good judgements or interpretations about what it percieves - the AI is dumb, deaf, and blind - it has to be spoonfed with pre-filtered and pre-digested data about its environment to simplify the processing requirements.

The problem then becomes, since the AI has no access to the raw data or the ability to process it, it cannot double-guess whether the information is correct or real - it simply has to trust that the sensors are returning good data. This is a serious limitation, which makes these types of systems non-reliable in the real world.

It's not until the AI can get over these "fooled by a sticker" issues that you can put one to drive a car.


Jan 06, 2018
I cannot post a link to this article on Facebook. Facebook informs me that techxplore.com is an unsafe website. Why does Facebook think techxplore.com is an unsafe website? And, yes, I used https://

Please sign in to add a comment. Registration is free, and takes less than a minute. Read more