Biologically plausible spatiotemporal adjustment helps train deep spiking neural networks

The forward and backward process of spiking neural networks. The dotted lines of different colors indicate the impact on the network at different time steps. The earlier spiking node will have more influence on the parameter update. Credit: Patterns (2022). DOI: 10.1016/j.patter.2022.100522

Spiking neural networks (SNNs) capture the most important aspects of brain information processing. They are considered a promising approach for next-generation artificial intelligence. However, the biggest problem restricting the development of SNNs is the training algorithm.

To solve this problem, a research team led by Prof. Zeng Yi from the Institute of Automation of the Chinese Academy of Sciences has proposed backpropagation (BP) with biologically plausible spatiotemporal adjustment for training deep spiking neural networks.

The associated study was published in Patterns on June 2.

Backpropagation-based training has extended SNNs to more complex network structures and tasks. However, the traditional design of BP ignores the dynamic characteristics of SNNs and is not biologically plausible.

Inspired by neural mechanisms in the brain, the researchers proposed a biologically plausible spatiotemporal adjustment to replace the traditional artificial design.

“After rethinking the relationship between membrane potential and spikes, we proposed the biologically plausible spatial adjustment of gradients to different time steps. It precisely controls the backpropagation of the error along the spatial dimension,” said Prof. Zeng Yi, corresponding author of the study.

To overcome the problem of temporal dependency of traditional spiking neurons within a single spike period, the researchers proposed a biologically plausible temporal adjustment to make the error propagate across the spikes in the temporal dimension, according to Shen Guobin, first author of the study.

The adjustment improves the performance of the SNNs and reduces energy consumption and latency. Compared with other surrogate gradient algorithms, the algorithm proposed in this study achieves 4.34% and 6.36% improvement in accuracy with only about half the energy consumption on DVS-Gesture and DVS-CIFAR10, typical datasets for temporal-sequential information processing.

“In theory, compared with artificial neural networks of the same structure, the proposed algorithm uses only about 3% of the energy to achieve competitive classification accuracy,” said Assistant Professor Zhao Dongcheng.

This study is part of the Brain-inspired Cognitive Intelligence Engine (BrainCog) project initiated by Prof. Zeng Yi’s team, an on-going scientific exploration of the infrastructure of brain-inspired artificial intelligence.

Mesoscale neural plasticity helps in AI learning

More information:
Yi Zenga, Backpropagation with Biologically Plausible Spatio-Temporal Adjustment For Training Deep Spiking Neural Networks, Patterns (2022). DOI: 10.1016/j.patter.2022.100522. … 2666-3899(22)00119-2

Provided by
Chinese Academy of Sciences

Biologically plausible spatiotemporal adjustment helps train deep spiking neural networks (2022, June 2)
retrieved 2 June 2022

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.

For all the latest Technology News Click Here 

 For the latest news and updates, follow us on Google News

Read original article here

Denial of responsibility! is an automatic aggregator around the global media. All the content are available free on Internet. We have just arranged it in one platform for educational purpose only. In each content, the hyperlink to the primary source is specified. All trademarks belong to their rightful owners, all materials to their authors. If you are the owner of the content and do not want us to publish your materials on our website, please contact us by email – [email protected]. The content will be deleted within 24 hours.