Dual Lottery Ticket Hypothesis. (arXiv:2203.04248v1 [cs.LG])

Fully exploiting the learning capacity of neural networks requires
overparameterized dense networks. On the other side, directly training sparse
neural networks typically results in unsatisfactory performance. Lottery Ticket
Hypothesis (LTH) provides a novel view to investigate sparse network training
and maintain its capacity. Concretely, it claims there exist winning tickets
from a randomly initialized network found by iterative magnitude pruning and
preserving promising trainability (or we say being in trainable condition). In
this work, we regard the winning ticket from …

arxiv hypothesis lottery ticket hypothesis