This article has been reviewed according to Science X's editorial process and policies. Editors have highlighted the following attributes while ensuring the content's credibility:


peer-reviewed publication


Convolutional optical neural networks herald a new era for AI imaging

Convolutional optical neural networks herald a new era for AI imaging
Schematic of scattering through multiple diffusers. Each diffuser can be modeled as a thin scattering layer, with N representing the number of diffusers and d representing the spacing between diffusers. Credit: Science Advances (2024). DOI: 10.1126/sciadv.adn2205

Convolutional neural networks (CNNs), with their exceptional image recognition capabilities, have performed outstandingly in the field of AI and notably within platforms like ChatGPT. Recently, a team of Chinese researchers from University of Shanghai for Science and Technology have successfully introduced the concept of CNNs into the field of optics and realized convolutional all-optical neural network, bringing revolutionary progress to AI imaging technology.

Led by Prof. Min Gu and Prof. Qiming Zhang from School of Artificial Intelligence Science and Technology (SAIST) at the University of Shanghai for Science and Technology (USST), the research team has developed an ultrafast convolutional optical neural network (ONN), which achieves efficient and clear imaging of objects behind scattering media without relying on the optical memory effect.

This finding was published in the journal Science Advances, in a paper titled "Memory-less scattering imaging with ultrafast convolutional optical neural networks."

Dr. Yuchao Zhang, a researcher at SAIST, is the first author. Prof. Min Gu and Prof. Qiming Zhang serve as the corresponding authors.

The core of CNNs, convolutional operations, extracts local features from images and constructs more complex and abstract feature representations layer by layer, greatly advancing the fields of image processing and pattern recognition. However, applying the concept of convolution network operations to the field of optics faces the challenge of converting electronic signals into optical signals.

The research team ingeniously designed an all-optical solution, performing convolution network operations directly in the optical domain, eliminating the cumbersome signal conversion process, and achieving true optical computing speeds.

The key to this research is the construction of a multi-stage convolutional ONN, composed of multiple parallel cores capable of operating at the speed of light, directly extracting features from for rapid image reconstruction.

This process not only greatly improves imaging speed but also significantly enhances , making imaging in complex scattering environments possible. Moreover, the convolutional ONN's computational speed reaches 1.57 peta operations per second (POPS), providing robust support for real-time dynamic imaging.

Another highlight of this research is its multitasking capability. By simply adjusting the network structure, the same convolutional ONN can perform a variety of different image processing tasks, such as classification and reconstruction, concurrently—a first in the field of optical artificial intelligence.

Prof Qiming Zhang noted, "This combination of flexibility and efficiency not only underscores the importance of convolutional networks in artificial intelligence but also opens up new avenues for optical imaging ."

The advent of this research is not only a successful transplantation of into the optical field but also a significant boost to AI imaging technology.

Prof Min Gu stated, "In the near future, convolutional optical neural networks will play an increasingly vital role in autonomous driving, robotic vision, and medical imaging."

More information: Yuchao Zhang et al, Memory-less scattering imaging with ultrafast convolutional optical neural networks, Science Advances (2024). DOI: 10.1126/sciadv.adn2205

Journal information: Science Advances
Provided by University of Shanghai for Science and Technology
Citation: Convolutional optical neural networks herald a new era for AI imaging (2024, June 26) retrieved 24 July 2024 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Classical optical neural network exhibits 'quantum speedup'


Feedback to editors