Teaching AI to identify colors in the dark

A team of researchers at the University of California used deep learning to enable limited color vision in the dark. The group published a paper describing their work on the open-access site PLOS ONE.

As the researchers note, humans are able to see light in the spectrum 400 to 700 nm—objects appear to have certain colors depending on the wavelength of light that bounces off of them and enters the eye. Humans have developed a means for extending the range of sight by creating night-vison technology, shining infrared light onto objects and capturing the signals that are bounced back, which are then converted to visible light in the form of monochromatic images or video. And while infrared technology has proven useful in a variety of applications, scientists have been working toward technology that could provide full-color images or videos using cameras in places with no light. In this new effort, the researchers have taken one small step in that direction.

Prior research has shown that deep learning networks can be useful in adding color to monochrome data. Such systems can be taught what colors should be in a scene, for example, by showing them multiple examples of a given scene during times when it is well lit. But such applications are limited to scenes that have already been studied. In this new effort, the researchers extended the means by which a deep learning network can be used to "teach" an AI system how to figure out what colors something should be when there is no visible light present.

The work involved using more than one wavelength of infrared light when teaching an AI system, along with data from light in the visible spectrum. To that end, they used a monochrome camera that was capable of responding to light in both the visible and infrared spectrum. They then used it to take multiple pictures of people's faces as a way to teach the AI system about the colors involved in human faces. They then used the system to take similar photographs in the dark. In so doing, they found the system capable of making accurate guesses to color the pictures which were displayed on a computer screen.

The researchers acknowledge that their application is limited to photographs of human faces, and that the results are best guesses rather than true representations of a subject—but they also note that systems such as theirs could evolve over time into technology to colorize night-vision imagery.