Teaching AI to accurately colorize marine plankton images
Plankton are fundamental to the marine ecosystem and an indispensable means of modern marine ecological management. Since previous studies have shown that zooplankton are insensitive to long wavelength (i.e., red) light, ...
In response to this problem, a research team led by Dr. Li Jianping from the Shenzhen Institute of Advanced Technology (SIAT) of the Chinese Academy of Sciences (CAS) recently developed an automatic plankton colorization algorithm, the IsPlanktonCLR network, based on deep convolutional neural networks.
The algorithm is trained to convert gray images acquired under red lighting into high-fidelity color images. It can automatically and truthfully "dye" marine plankton grayscale images shot underwater into their natural colors, and the colorization effect is very close to human eye perception.
The study was released in the European Conference on Computer Vision (ECCV), which was held in Israel from October 23 to 27.
Researchers from Xiamen University and the Harbin Institute of Technology (Shenzhen) were also involved in the research.
The IsPlanktonCLR network uses a two-path structure with a self-guidance function, a customized color palette and a stepwise focusing loss function to achieve automatic colorization of plankton grayscale images. It shows excellent accuracy and fidelity in color restoration for rare species and key parts of common species.
Fig. 1 Overview of IsPlanktonCLR: (a) palette customization; (b) reference module; (c) colorization module. Credit: Li Jianping
Fig. 3 (a) Comparison of visual perception and CDSIM evaluations of marine plankton images produced by various colorization models. (b) Comparison of visual perception and numerical metrics-based evaluations of artificially colorized natural scene images. Credit: Li Jianping