Two researchers at Facebook, Brian Dolhansky and Cristian Canton Ferrer, have posted a paper on the social network giant's site detailing a new AI application they are working on. The goal of the app, they report, is to open eyes that appear closed in a photograph.
Many people have difficulty keeping their eyes open when someone is taking their picture, resulting in a photograph that looks as if they are napping. In response, engineers working on photo editing software have added routines that help users open them. And others working on AI apps to solve the problem have used large datasets of photos as learning material for deep learning networks used to paint new eyes on a target. But to date, the pair at Facebook note, none of these approaches has produced very good results. They hope to improve on such efforts by using other photos of the same person that have also been posted to Facebook as learning material.
The technology is called "Eye in-painting with Exemplar Generative Adversarial Networks," or ExGans. Eye in-painting refers to painting over parts of an existing image with new material to create a desired effect. And GANs are a specific type of deep learning neural network. The app that Dolhansky and Ferrer are working on has several parts to help achieve their objective. It has to look for and find other photos of the same person, make sure that the photos used are applicable, and then generate eyes based on those found in other photos, taking into account lighting and other factors in the original photo. It also has to analyze the results to test itself on quality. Finally, it has to paint the eyes it created onto the original photo in a way that looks right to people looking at it. Using such an approach, they note, overcomes problems of incorrect eye shape or eyes that look like they belong to someone else.
The researchers report that thus far, they have met with a lot of success—many of the photos they tested have come out looking better than the results of other methods. On the other hand, they also encountered some glitches, such as misshapen or poorly colored eyes. Most such problems, they further report, were due to poor original picture quality, poor angles, or obstructions such as hair.