Structure-Preserving Network Compression Via Low-Rank Induced Training Through Linear Layers Composition

arXiv:2405.03089v1 Announce Type: new
Abstract: Deep Neural Networks (DNNs) have achieved remarkable success in addressing many previously unsolvable tasks. However, the storage and computational requirements associated with DNNs pose a challenge for deploying these trained models on resource-limited devices. Therefore, a plethora of compression and pruning techniques have been proposed in recent years. Low-rank decomposition techniques are among the approaches most utilized to address this problem. Compared to post-training compression, compression-promoted training is still under-explored. In this paper, we present …

abstract arxiv challenge compression computational cs.lg devices however linear low network networks neural networks pruning requirements storage success tasks through training type via