LNPT: Label-free Network Pruning and Training

arXiv:2403.12690v1 Announce Type: new
Abstract: Pruning before training enables the deployment of neural networks on smart devices. By retaining weights conducive to generalization, pruned networks can be accommodated on resource-constrained smart devices. It is commonly held that the distance on weight norms between the initialized and the fully-trained networks correlates with generalization performance. However, as we have uncovered, inconsistency between this metric and generalization during training processes, which poses an obstacle to determine the pruned structures on smart devices in …

abstract arxiv cs.lg deployment devices free however network networks neural networks performance pruning smart training type