Biologically plausible spatiotemporal adjustment helps train deep spiking neural networks

Biologically plausible spatiotemporal adjustment helps train deep spiking neural networks
The forward and backward process of spiking neural networks. The dotted lines of different colors indicate the impact on the network at different time steps. The earlier spiking node will have more influence on the parameter update. Credit: Patterns (2022). DOI: 10.1016/j.patter.2022.100522

Spiking neural networks (SNNs) capture the most important aspects of brain information processing. They are considered a promising approach for next-generation artificial intelligence. However, the biggest problem restricting the development of SNNs is the training algorithm.

To solve this problem, a research team led by Prof. Zeng Yi from the Institute of Automation of the Chinese Academy of Sciences has proposed backpropagation (BP) with biologically plausible spatiotemporal adjustment for training deep spiking .

The associated study was published in Patterns on June 2.

Backpropagation-based training has extended SNNs to more complex network structures and tasks. However, the traditional design of BP ignores the dynamic characteristics of SNNs and is not biologically plausible.

Inspired by neural mechanisms in the brain, the researchers proposed a biologically plausible spatiotemporal adjustment to replace the traditional artificial design.

"After rethinking the relationship between membrane potential and spikes, we proposed the biologically plausible spatial adjustment of gradients to different time steps. It precisely controls the backpropagation of the error along the spatial dimension," said Prof. Zeng Yi, corresponding author of the study.

To overcome the problem of temporal dependency of traditional spiking neurons within a single spike period, the researchers proposed a biologically plausible temporal adjustment to make the error propagate across the spikes in the temporal dimension, according to Shen Guobin, first author of the study.

The adjustment improves the performance of the SNNs and reduces and latency. Compared with other surrogate gradient algorithms, the proposed in this study achieves 4.34% and 6.36% improvement in accuracy with only about half the energy consumption on DVS-Gesture and DVS-CIFAR10, typical datasets for temporal-sequential information processing.

"In theory, compared with of the same structure, the proposed algorithm uses only about 3% of the energy to achieve competitive classification accuracy," said Assistant Professor Zhao Dongcheng.

This study is part of the Brain-inspired Cognitive Intelligence Engine (BrainCog) project initiated by Prof. Zeng Yi's team, an on-going scientific exploration of the infrastructure of brain-inspired artificial intelligence.

More information: Yi Zenga, Backpropagation with Biologically Plausible Spatio-Temporal Adjustment For Training Deep Spiking Neural Networks, Patterns (2022). DOI: 10.1016/j.patter.2022.100522. … 2666-3899(22)00119-2

Citation: Biologically plausible spatiotemporal adjustment helps train deep spiking neural networks (2022, June 2) retrieved 13 June 2024 from
This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.

Explore further

Mesoscale neural plasticity helps in AI learning


Feedback to editors