The fine line between dead neurons and sparsity in binarized spiking neural networks. (arXiv:2201.11915v1 [cs.NE])

Web: http://arxiv.org/abs/2201.11915

Spiking neural networks can compensate for quantization error by encoding
information either in the temporal domain, or by processing discretized
quantities in hidden states of higher precision. In theory, a wide dynamic
range state-space enables multiple binarized inputs to be accumulated together,
thus improving the representational capacity of individual neurons. This may be
achieved by increasing the firing threshold, but make it too high and sparse
spike activity turns into no spike emission. In this paper, we propose the use …

arxiv line networks neural neural networks neurons sparsity