Researchers explore decoding faces from neural activity

(Tech Xplore)—What can technology accomplish in reconstructing a face? Scientists have come up with surprising results. Yale Daily News in 2014 reported on a study about how, with a brain scan, a face is reconstructed—neuroscience research providing us "with a way to see the world through other people's eyes."

Fast-forward to now, and an interesting article has been published in The Journal of Neuroscience. "Reconstructing Perceived and Retrieved Faces from Activity Patterns in Lateral Parietal Cortex," by Hongmi Lee and Brice A. Kuhl. The two authors showed that, in their words, "visually perceived faces were reliably reconstructed from activity patterns in occipitotemporal cortex and several lateral parietal subregions, including ANG." The angular gyrus (ANG) is a subregion of the posterior lateral parietal cortex.

Discover headlined its report of the study with "Decoding faces from the brain." Discover found the paper "fascinating" in how the work explored decoding faces from neural activity. "Armed with a brain scanner, they can reconstruct which face a participant has in mind. It's a cool technique that really seems to fit the description of 'mind reading' – although the method's accuracy is only modest."

They used an fMRI (functional magnetic resonance imaging) scan, during which participants were shown images and neural responses were recorded. "The set of faces was then decomposed into 300 eigenfaces using the technique of principal component analysis," said Discover.

"As well as reconstructing faces from neural activity during perception of the faces (which has been done before), Lee and Kuhl examined neural activity when faces weren't actually on the screen at all. Participants were given a face recall task, in which they had to hold a face in memory."

Commenting on their work, the authors reviewed their approach and its significance. They said, "we used an innovative form of fMRI pattern analysis to test whether lateral parietal cortex actively represents the contents of memory. Using a large set of human face images, we first extracted latent face components (eigenfaces). We then used machine learning algorithms to predict face components from fMRI activity patterns and, ultimately, to reconstruct images of individual faces."

They showed that activity patterns in a subregion of the lateral parietal cortex (angular gyrus) supported successful reconstruction of perceived and remembered faces.

The Yale Daily News in its 2014 report described the term "eigenfaces" as derivatives of normal faces that are manipulated to highlight basic components of the original face.

ZME Science in reporting the present study said that "these eigenfaces act like statistical patterns that can be used as anchors to match a request."

ZME Science also commented that "Researchers are still working in rough, uncharted terrain. By analyzing more neural patterns, say from thousands of participants, and fine tweaking their reconstruction algorithms, it's foreseeable that these reconstructions could become a lot better."

In 1991, Matthew Turk and Alex Pentland authored a paper "Eigenfaces for recognition " where they said that "We have developed a near-real-time computer system that can locate and track a subject's head, and then recognize the person by comparing characteristics of the face to those of known individuals. The computational approach taken in this system is motivated by both physiology and information theory, as well as by the practical requirements of near-real-time performance and accuracy." They also said that "Some particular advantages of our approach are that it provides for the ability to learn and later recognize new faces in an unsupervised manner, and that it is easy to implement using a neural network architecture."

© 2016 Tech Xplore