Sony unveils first built-in AI image sensors

Sony is bringing machine intelligence to its image sensors. The electronics and entertainment giant announced this week a sensor that applies artificial intelligence while processing imagery without the need for extrema hardware or assistance.

Players in the photography industry have been focusing on increasingly greater numbers of pixels for ever-sharper enlargement and reproduction capabilities and on increasingly compressed devices for lighter weight and greater portability.

But Sony's new IMX500 sensor is the first to improve efficiency though AI implementation.

Sony says the new 12.3 megapixel sensor bypasses the commonly used practices of sending video images to the cloud or tacking on additional hardware for processing and analysis. Instead, the sensors will act as mini-computers capable of virtually instantaneous image recognition.

The chips will speed up analysis and eliminate reliance on cloud resources. Video monitoring systems tracking human behavior transmit trillions of bits of data daily. By processing that data within the chip rather than having to send it to cloud processing units, time-consuming transfer times are slashed. There is also greater security for users since often sensitive video data remains on board, not in the cloud.

Considering the notion of using machine learning to increase sensor intelligence, Sony vice president of business and innovation Mark Hanson recently observed, "If we can miniaturize that capability and put it on the backside of a chip we can do all sorts of interesting things."

According to Sony, the IMX500 (and companion IMX501 designed for smart board installation) "are the first image sensors in the world to be equipped with AI processing functionality."

For purposes of comparison, the IMX500 can apply a standard image recognition algorithm to an individual video frame in 3.1 milliseconds. Google's Movidius sensors, on the other hand, require up to a few seconds to process the same data.

"Including AI processing functionality on the image sensor itself enables high-speed edge AI processing and extraction of only the necessary data, which, when using cloud services, reduces data transmission latency, addresses privacy concerns, and reduces power consumption and communication costs," the report said.

Sony offers the example of in-store cameras that are being introduced globally to track shopper behavior and conduct cashless purchases. The faster, cheaper AI sensors will improve efficiency of cameras tracking numerous customers in stores, and create heat maps to monitor behavior, offer crowd control or even determine—in this age of social distancing—who is violating distancing norms or failing to wear a mask. They can also be used for product inventory tracking.

The sensors can handle up to 4K 60 fps video streaming.

Sony is a major supplier of image sensors for Apple iPhones and Nikon cameras.

The first sensors are being shipped to commercial establishments for evaluation. Consumer rollout is expected early next year.

© 2020 Science X Network