Teaching AI to accurately colorize marine plankton images

Teaching AI to Accurately Colorize Marine Plankton Images
Fig. 1 Overview of IsPlanktonCLR: (a) palette customization; (b) reference module; (c) colorization module. Credit: Li Jianping

Plankton are fundamental to the marine ecosystem and an indispensable means of modern marine ecological management. Since previous studies have shown that zooplankton are insensitive to long wavelength (i.e., red) light, such light is often used to make images of plankton as part of the ecological management process. Unfortunately, red light only produces grayscale images of plankton, thus losing information about their true color.

In response to this problem, a research team led by Dr. Li Jianping from the Shenzhen Institute of Advanced Technology (SIAT) of the Chinese Academy of Sciences (CAS) recently developed an automatic plankton colorization , the IsPlanktonCLR network, based on deep convolutional neural networks.

The algorithm is trained to convert gray images acquired under red lighting into high-fidelity color images. It can automatically and truthfully "dye" marine plankton grayscale images shot underwater into their natural colors, and the colorization effect is very close to human eye perception.

The study was released in the European Conference on Computer Vision (ECCV), which was held in Israel from October 23 to 27.

Researchers from Xiamen University and the Harbin Institute of Technology (Shenzhen) were also involved in the research.

The IsPlanktonCLR network uses a two-path structure with a self-guidance function, a customized color palette and a stepwise focusing loss function to achieve automatic colorization of plankton grayscale images. It shows excellent accuracy and fidelity in color restoration for rare species and key parts of common species.

  • Teaching AI to Accurately Colorize Marine Plankton Images
    Fig. 2 Image examples of IsPlanktonCLR dataset. Credit: Li Jianping
  • Teaching AI to Accurately Colorize Marine Plankton Images
    Fig. 3 (a) Comparison of visual perception and CDSIM evaluations of marine plankton images produced by various colorization models. (b) Comparison of visual perception and numerical metrics-based evaluations of artificially colorized natural scene images. Credit: Li Jianping

In order to train and develop the IsPlanktonCLR algorithm, the team constructed a dataset containing thousands of color-gray plankton image pairs. Using this dataset, the team not only trained the colorization algorithm, but also compared its performance with state-of-the-art colorization algorithms. The experimental results showed that the IsPlanktonCLR algorithm offered the best results not only in terms of human visual perception but also in terms of peak signal to noise ratio (PSNR), structural similarity (SSIM), Fréchet inception distance (FID) and other classical metrics frequently used in machine vision.

In addition, existing colorization algorithms generally lack objective and quantitative evaluation metrics. Existing evaluation metrics not only have difficulty simulating the color perception ability of human vision but also have difficulty objectively evaluating the color similarity between the colored image and the ground-truth image.

Therefore, the researchers proposed a color similarity evaluation index, Color Dissimilarity (CDSIM), which combines a color histogram, color aggregation vector, color correlogram and gradient. Through the tests on marine and natural scene images, the researchers verified that CDSIM is not only effective in evaluating the effect of colorization algorithms but is also more suitable for image colorization evaluation in the field of scientific imaging than other existing metrics.

"The development of IsPlanktonCLR provides a new artificial intelligence solution for ocean imaging instruments to obtain accurate and real observation results," said Dr. Li. "This provides new technical means for us understanding the ocean."

More information: Guannan Guo et al, Colorization for in situ Marine Plankton Images, Computer Vision—ECCV 2022 (2022). DOI: 10.1007/978-3-031-19839-7_13

Citation: Teaching AI to accurately colorize marine plankton images (2022, October 27) retrieved 20 April 2024 from https://techxplore.com/news/2022-10-ai-accurately-marine-plankton-images.html
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Rapid full-color Fourier ptychographic microscopy provides new solution for digital pathology


Feedback to editors