[R] Learning Long Sequences in Spiking Neural Networks

**Paper**: [https://arxiv.org/abs/2401.00955](https://arxiv.org/abs/2401.00955)

**Abstract**:

>Spiking neural networks (SNNs) take inspiration from the brain to enable energy-efficient computations. Since the advent of Transformers, SNNs have struggled to compete with artificial networks on modern sequential tasks, as they inherit limitations from recurrent neural networks (RNNs), with the added challenge of training with non-differentiable binary spiking activations. However, a recent renewed interest in efficient alternatives to Transformers has given rise to state-of-the-art recurrent architectures named state space models (SSMs). This work systematically investigates, for …

abstract artificial binary brain challenge differentiable energy inspiration limitations machinelearning modern networks neural networks recurrent neural networks spiking neural networks tasks training transformers