Decoupled style structure in Fourier domain method improves raw to sRGB mapping

A team of researchers led by Professor Xie Chengjun and Associate Professor Zhang Jie at Hefei Institutes of Physical Science (HFIPS), Chinese Academy of Sciences (CAS), unveiled Fourier-ISP, a novel deep-learning based framework for RAW-to-sRGB image conversion.

This approach was accepted for publication in the 2024 proceedings of the Association for the Advancement of Artificial Intelligence (AAAI).

Converting RAW images to sRGB images enhances the visual appeal and usability of smartphone photography. However, current methods struggle with color and spatial structure accuracy, especially with resolution and image type variations. Combining color mapping and spatial structure produces suboptimal results due to the complex interplay between style and structure within the images.

To overcome these challenges, the team has developed a novel framework called Fourier-ISP. Inspired by the Image Signal Processing pipeline, this approach separates the style and structure of the image within the frequency domain.

"It enabled independent optimization," said Zhang Jie, member of the team.

Fourier-ISP consists of three subnetworks: one for refining the structural details, another for learning accurate colors, and a third for blending these elements seamlessly. This decoupling of style and structure enables enhanced performance in image conversion, producing sharper and more accurate color and structural details.

Extensive evaluations across varied datasets confirm that Fourier-ISP realizes state-of-the-art results in qualitative and quantitative assessments, surpassing existing methods in precision and detail reproduction. It demonstrates robust transferability and effectiveness in handling both structural and style information, ensuring enhanced color reproduction and texture preservation. Notably, Fourier-ISP achieved an impressive PSNR improvement of 0.17dB in the ZRR dataset.

This framework introduces a novel insight into the field of image processing, showcasing the potential of style-structure decoupling in achieving high-fidelity image conversion, particularly in mobile photography, according to the team.

The paper is available on the arXiv preprint server.