Graph Expansion in Pruned Recurrent Neural Network Layers Preserve Performance

arXiv:2403.11100v1 Announce Type: new
Abstract: Expansion property of a graph refers to its strong connectivity as well as sparseness. It has been reported that deep neural networks can be pruned to a high degree of sparsity while maintaining their performance. Such pruning is essential for performing real time sequence learning tasks using recurrent neural networks in resource constrained platforms. We prune recurrent networks such as RNNs and LSTMs, maintaining a large spectral gap of the underlying graphs and ensuring their …

abstract arxiv connectivity cs.cv cs.lg cs.ne expansion graph network networks neural network neural networks performance property pruning recurrent neural network sparsity type