NEPENTHE: Entropy-Based Pruning as a Neural Network Depth's Reducer

arXiv:2404.16890v1 Announce Type: new
Abstract: While deep neural networks are highly effective at solving complex tasks, their computational demands can hinder their usefulness in real-time applications and with limited-resources systems. Besides, for many tasks it is known that these models are over-parametrized: neoteric works have broadly focused on reducing the width of these networks, rather than their depth. In this paper, we aim to reduce the depth of over-parametrized deep neural networks: we propose an eNtropy-basEd Pruning as a nEural …

abstract applications arxiv computational cs.ai cs.lg entropy hinder network networks neural network neural networks pruning real-time real-time applications resources systems tasks type while