Scientists show how shallow learning mechanism used by the brain can compete with deep learning

Neural network learning techniques stem from the dynamics of the brain. However, these two scenarios, brain learning and deep learning, are intrinsically different. One of the most prominent differences is the number of layers each one possesses.

Deep learning architectures typically consist of numerous layers that can be increased to hundreds, enabling efficient learning of complex classification tasks. Contrastingly, the brain consists of very few layers, yet despite its shallow architecture and noisy and slow dynamics, it can efficiently perform complex classification tasks.

The critical question driving new research is the possible mechanism underlying the brain's efficient shallow learning, enabling it to perform classification tasks with the same accuracy as deep learning. In an article published in Physica A: Statistical Mechanics and its Applications, researchers from Bar-Ilan University in Israel show how such shallow learning mechanisms can compete with deep learning.

"Instead of a deep architecture, like a skyscraper, the brain consists of a wide shallow architecture, more like a very wide building with only very few floors," said Prof. Ido Kanter of Bar-Ilan's Department of Physics and Gonda (Goldschmied) Multidisciplinary Brain Research Center, who led the research.

"The capability to correctly classify objects increases where the architecture becomes deeper, with more layers. In contrast, the brain's shallow mechanism indicates that a wider network better classifies objects," said Ronit Gross, an undergraduate student and one of the key contributors to this work. "Wider and higher architectures represent two complementary mechanisms," she added.

Nevertheless, the realization of very wide shallow architectures, imitating the brain's dynamics, requires a shift in the properties of advanced GPU technology, which is capable of accelerating deep architecture, but fails in the implementation of wide shallow ones.