[D] How many NN optimization techniques are actually incorporated?

Hey guys, I’m an undergraduate student just getting into this field so forgive me if this question is dumb.

I noticed that a lot of papers at NIPS/ICML/ICLR are concerned with NN optimization; for example, this paper https://arxiv.org/pdf/2010.14501.pdf from ICLR 2021 claims to reduce memory requirement by 3x for various PyTorch models, which ostensibly would be massive. However, it was also “only” cited 8 times, which indicates that it isn’t being used that often.

I know that a lot of …

machinelearning optimization