Researchers at the University of Tartu's Computational Neuroscience Lab, in Estonia, have discovered that activations of deep convolutional neural networks are aligned with the gamma band activity of the human visual cortex. Their study, published in Communications Biology, further highlights the potential of artificial intelligence (AI) to broaden understanding of the human brain.
The human ability to visually recognize objects is mediated by a hierarchy of complex feature representations along the ventral stream.Past research has found that these are similar to the hierarchy of transformations learned by deep convolutional neural networks (DCNN) trained on images.
"From previous research we knew that there is a correspondence between the hierarchical architecture of the human visual system and the layered architecture of DCNNs," Jaan Aru, Raul Vicente, and Ilya Kuzovkin, three of the researchers who carried out the study, told TechXplore. "However, this research relied on neuroimaging techniques such as fMRI and MEG, each of which has its own limitations."
MEG imaging only captures the average activity of large populations of neurons at once, while fMRI does not capture temporal information. Hence, the researchers decided to collect their dataset using intracranial electrodes implanted directly into the brains of their test subjects. This technique can identify when brain activity happens, its anatomical location, and how it changes over time.
"This allowed us to explore in more detail the activity that governs visual processing in the human brain and characterize more precisely what kind of activity bears similarities with the activity of DCNNs," the researchers said.
DCNNs are a type of machine learning algorithm for computer vision, which perform especially well on object recognition tasks. Their key feature is that they acquire rules to classify objects automatically, without human engineers outlining them.
During training, DCNNs are trained on thousands of images of objects (e.g. cats, trees, cars, etc.), learning to distinguish visual features of each of these categories. The algorithms can then correctly identify objects in images they have never encountered before, with up to 95 percent accuracy.
"A DCNN consists of layers of artificial neurons, with each layer performing certain operations on the image and then sending information to the next layer," the researchers said. "During training, the algorithm forms rules about what information is to be sent to the upper layers and when."
Recent studies have investigated the exact patterns and features learned by a DCNN. They found that as one dives deeper into its layers, the visual patterns represented by its neurons become increasingly complex.
"The first layer is responsible for detecting straight lines, changes in brightness and other simple visual features," the researchers explain. "This information is passed onto the second layer, which combines simple features to build detectors that can identify simple shapes. And so it progresses, becoming more and more abstract with every layer, with the higher layer neurons representing whole objects, such as cats, dogs and so on. We knew that a very similar phenomenon is observed in the human visual cortex, so the obvious question was: How similar are these two systems, and what are their similarities?"
When measuring electrical responses from the brain, researchers observe complex patterns of activity. These patterns are grouped according to their frequency: alpha (eight to 14 times per second), beta (15 to 30 Hz), gamma (from 30 to ~70 Hz), high gamma (more than 70 Hz), and other bands. These frequency bands have been found to depend on different types of activity. For instance, alpha is stronger when humans are relaxed, while beta and gamma increase during active engagement in a task.
"We found that activity in low gamma (30 to 70 Hz) and high gamma (70 to 150 Hz) are best aligned with the activity that happens in DCNNs, indicating that what happens in the brain at those frequencies is most similar to what DCNNs are doing," the researchers said.
These findings are aligned with previous research highlighting the importance of gamma activity for object recognition. In the future, they could help to better understand the exact computations that are reflected by gamma frequency signals during visual processing.
"The ultimate quest of neuroscience is to understand how the brain codes, stores and transmits information and how firings of billions of neurons lead to complex mental processes, such as understanding a text or communicating it to a friend," the researchers said. "This work provides yet another piece of this huge puzzle, and highlights the important role that AI algorithms can play in understanding human brain."
The Computational Neuroscience Lab at the University of Tartu studies biological and artificial learning systems side by side, as comparing them could lead to fascinating biological discoveries. The researchers are now working on two further projects, which will form the core of Kuzovkin's PhD thesis.
"In one of the projects, we are going to look into the inner workings of an algorithm that we trained to make sense of human brain data; exploring what brain activity it deems useful for the end task and which one it discards. This will provide a tool to go through large volumes of activity and filter out parts that are relevant to a particular mental task."