Mesoscale neural plasticity helps in AI learning

Mesoscale neural plasticity helps in AI learning
Fig. 1. Introducing biological SBP into SNNs.(A) Schematic diagram depicting the BP of potentiation (“+”) or depression (“−”) from the synapses at the output layer to those at the hidden layer in a three-layer network. The propagated synaptic modification had the same sign, consistent with the biological discovery of SBP (27). Similar configurations of fixed gradient mapping between neighborhood layers exist in artificial feedback alignment (48) and direct target propagation (49). (B) For a three-layer SNN, the induction of potentiation (+) or depression (−) occurred at the synapse Wj, k on the output neuron by STDP, based on the timing of presynaptic spikes (in the hidden neuron) relative to the postsynaptic spikes in the output neuron, after updating by mean square error (MSE) of network-generated (Out2) and teaching spike trains (Teaching2). Wi, j and Wj, k represent synaptic weights of connections onto hidden and output neurons, respectively. The + and − signals created at synapses of hidden layer neuron (pink) onto an output neuron (blue) were allowed to spread to a percentage factor λp (λp∈ [10%,100%]) of input synapses with a fraction factor λf (λf∈ [0.1,1]) of the signals generated by the STDP (orange). Vj and Vk are membrane potentials at hidden and output layers, respectively. (C) The three-layer architecture of SNN, in which SBP and local plasticity (STP, STDP, and homeostatic V adjustment) were introduced at synapses at hidden and output layers, and the teaching spike train was given to the output LIF neurons. The diagram illustrates an output neuron inducing STDP (blue), a hidden neuron with the output synapse inducing STDP (pink), and input neurons with synapses receiving SBP (yellow). Credit: DOI: 10.1126/sciadv.abh0146

A joint research team led by Xu Bo from the Institute of Automation and Mu-Ming Poo from the Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, have discovered that self-backpropagation, a form of mesoscale synaptic plasticity rule in natural neural networks, can elevate the accuracy and reduce the computational cost of spiking neural networks (SNNs) and artificial neural networks (ANNs).

Their findings were published in Science Advances on Oct. 20.

Previous studies proved that self-backpropagation (SBP) is caused by the rapid retrograde axonal transport of molecular signals. It is considered to be the key for efficient and flexible learning of biological neural networks.

The backpropagation (BP) algorithm in artificial uses a global strategy for optimization, which can achieve excellent performance but, at the same time, take too much computational cost during learning.

The researchers introduced biologically plausible SBP into a three-layer SNN. They found an elevated accuracy of network performance in three standard benchmark tasks, MNIST, NETtalk, and DvsGesture.

"The computational cost in terms of the product of mean epochs and algorithmic complexity per epoch was markedly reduced," said Xu. Similar results were obtained by further applying SBP on Restricted Boltzmann Machine.

According to the study, SBP is a special mesoscale biological plasticity mechanism, indicating a similar important role of SBP in SNNs compared to its counterpart BP in ANNs.

This will attract attention in the field of machine learning because training SNNs with pure biologically plausible algorithms (e.g., ) is difficult, in which the information is spatio-temporal and carried by discontinuous spikes.

The study has paved a way towards biologically plausible effective learning on both SNNs and ANNs, with high accuracy and low computational cost for learning different tasks.

Explore further

Brain-on-a-chip would need little training

More information: Tielin Zhang et al, Self-backpropagation of synaptic modifications elevates the efficiency of spiking and artificial neural networks, Science Advances (2021). DOI: 10.1126/sciadv.abh0146
Journal information: Science Advances

Citation: Mesoscale neural plasticity helps in AI learning (2021, October 22) retrieved 29 November 2021 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Feedback to editors