Look-ups are not (yet) all you need for deep learning inference. (arXiv:2207.05808v1 [cs.LG])

Fast approximations to matrix multiplication have the potential to
dramatically reduce the cost of neural network inference. Recent work on
approximate matrix multiplication proposed to replace costly multiplications
with table-lookups by fitting a fast hash function from training data. In this
work, we propose improvements to this previous work, targeted to the deep
learning inference setting, where one has access to both training data and
fixed (already learned) model weight matrices. We further propose a fine-tuning
procedure for accelerating entire …

arxiv deep learning deep learning inference inference learning lg ups