Variational Nested Dropout. (arXiv:2101.11353v2 [cs.LG] UPDATED)

Web: http://arxiv.org/abs/2101.11353

Nested dropout is a variant of dropout operation that is able to order
network parameters or features based on the pre-defined importance during
training. It has been explored for: I. Constructing nested nets: the nested
nets are neural networks whose architectures can be adjusted instantly during
testing time, e.g., based on computational constraints. The nested dropout
implicitly ranks the network parameters, generating a set of sub-networks such
that any smaller sub-network forms the basis of a larger one. II. Learning …

arxiv dropout lg