Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques. (arXiv:2112.15131v1 [cs.LG])

Deep learning is pervasive in our daily life, including self-driving cars,
virtual assistants, social network services, healthcare services, face
recognition, etc. However, deep neural networks demand substantial compute
resources during training and inference. The machine learning community has
mainly focused on model-level optimizations such as architectural compression
of deep learning models, while the system community has focused on
implementation-level optimization. In between, various arithmetic-level
optimization techniques have been proposed in the arithmetic community. This
article provides a survey on resource-efficient …

arxiv deep learning implementation learning survey