Partial Search in a Frozen Network is Enough to Find a Strong Lottery Ticket

arXiv:2402.14029v1 Announce Type: new
Abstract: Randomly initialized dense networks contain subnetworks that achieve high accuracy without weight learning -- strong lottery tickets (SLTs). Recently, Gadhikar et al. (2023) demonstrated theoretically and experimentally that SLTs can also be found within a randomly pruned source network, thus reducing the SLT search space. However, this limits the search to SLTs that are even sparser than the source, leading to worse accuracy due to unintentionally high sparsity. This paper proposes a method that reduces …

abstract accuracy arxiv cs.ai cs.lg found network networks search tickets type