Rethinking the Relationship between Recurrent and Non-Recurrent Neural Networks: A Study in Sparsity

arXiv:2404.00880v1 Announce Type: new
Abstract: Neural networks (NN) can be divided into two broad categories, recurrent and non-recurrent. Both types of neural networks are popular and extensively studied, but they are often treated as distinct families of machine learning algorithms. In this position paper, we argue that there is a closer relationship between these two types of neural networks than is normally appreciated. We show that many common neural network models, such as Recurrent Neural Networks (RNN), Multi-Layer Perceptrons (MLP), …

abstract algorithms arxiv cs.lg families machine machine learning machine learning algorithms networks neural networks paper popular recurrent neural networks relationship sparsity study type types