Those meh photos that are too grainy can be given a new lease on digital life through a method worked up by researchers who found a way to cut the noise and artifacts. Noise in this context refers to visual distortion, as Cole's Classroom put it—specks that get in the way of enjoying your picture, the tiny colored pixels, sometimes looking like "grain" in film photography.
The team discuss their work in their paper, "Noise2Noise: Learning Image Restoration without Clean Data." The paper is on arXiv. The team includes affiliations with NVIDIA, Aalto University and MIT.
(Aalto U is a university in Finland and was founded in 2010 from the merger of Helsinki University of Technology, the Helsinki School of Economics and the University of Art and Design Helsinki.)
"This deep learning-based approach has learned to fix photos by simply looking at examples of corrupted photos only," said a posting on the NVIDIA Developer News Center.
"A noise-free photograph requires a long exposure ...In this work, we observe that under suitable, common circumstances, we can learn to reconstruct signals from only corrupted examples, without ever observing clean signals, and often do this just as well as if we were using clean examples."
Their paper was presented at ICML, which stands for International Conference on Machine Learning, in Sweden.
Katyanna Quach explained what is special about their work: "Computer vision algorithms are already automatically used to improve snaps taken on smartphones like the Pixel 2 or the iPhone X, but this takes things further," she wrote in The Register. "Instead of feeding neural networks a pair of images, where one is high quality and the other one is blurry, this latest model – nicknamed noise2noise – can learn how to clean images without needing to see high-resolution examples."
Method and approach: They used NVIDIA Tesla P100 GPUs with the cuDNN-accelerated TensorFlow deep learning framework. They trained the system on 50,000 images in the ImageNet validation set.
Quach: "The team trained their noise2noise model on 50,000 images taken from the ImageNet dataset and added a random distribution of noise to each image. The system has to estimate the magnitude of the noise in the photo and remove it."
The authors said, "Our proof-of-concept demonstrations point the way to significant potential benefits in these applications by removing the need for potentially strenuous collection of clean data. Of course, there is no free lunch – we cannot learn to pick up features that are not there in the input data – but this applies equally to training with clean targets."
The method could be used to enhance MRI images, too. This caught the attention of Brandon Hill in HotHardware. "Not only have NVIDIA and its academic partners used Noise2Noise to help restore grainy photos, but they are also using it for Magnetic Resonance Image (MRI) scans, which can be extremely beneficial in the medical sector."