A Brain-inspired Algorithm for Training Highly Sparse Neural Networks. (arXiv:1903.07138v3 [cs.NE] UPDATED)

Sparse neural networks attract increasing interest as they exhibit comparable
performance to their dense counterparts while being computationally efficient.
Pruning the dense neural networks is among the most widely used methods to
obtain a sparse neural network. Driven by the high training cost of such
methods that can be unaffordable for a low-resource device, training sparse
neural networks sparsely from scratch has recently gained attention. However,
existing sparse training algorithms suffer from various issues, including poor
performance in high sparsity …

algorithm arxiv brain brain-inspired networks neural networks training