AI method to upscale low-resolution images to high-resolution

EnhanceNet-PAT is capable of upsampling a low-resolution image (left) to a high definition version (middle). The result is indistinguishable from the original image (right). Credit: Max Planck Institute for Intelligent Systems

Scientists at the Max Planck Institute for Intelligent Systems in Tübingen have used artificial intelligence to create a high-definition version of a low resolution image. While not pixel-perfect, the system produces a better result.

Technology to create a large-sized image from a low-resolution image is known as single-image super-resolution (SISR) technology. SISR has been studied for decades, but with limited results. Software adds extra pixels and averages them with the surrounding pixels, but the result is blurriness. Researchers at the Max Planck Institute of Intelligent Systems propose a new approach to give images a realistic texture when magnified from small to large using machine learning. Artificial intelligence is applied, and an adaptive algorithm for upsampling the image learns from experience to improve the result.

The learning process is much like that of a human. "The algorithm is given the task of upsampling millions of to a high-resolution version, and is then shown the original. Notice the difference? OK, then learn from your mistake," says Mehdi M.S. Sajjadi, who together with Dr. Michael Hirsch and Prof. Dr. Bernhard Schölkopf,, developed the EnhanceNet-PAT technology. Once EnhanceNet-PAT is trained, it no longer needs the original photos.

According to the researchers, the technology is more efficient than any other SISR technology currently on the market. In contrast to existing algorithms, EnhanceNet-PATdoes not attempt pixel-perfect reconstruction, but rather aims for faithful texture synthesis. By detecting and generating patterns in a low-resolution image and applying these patterns in the upsampling process, EnhanceNet-PAT adds extra pixels to the low-resolution image accordingly. For most viewers, the result is very much like the original photo.

Explore further: Automatic image retouching on your phone

More information: EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis. arXiv.

Provided by Max Planck Institute for Intelligent Systems Tübingen