OLLIE: Derivation-based Tensor Program Optimizer. (arXiv:2208.02025v1 [cs.LG])

Boosting the runtime performance of deep neural networks (DNNs) is critical
due to their wide adoption in real-world tasks. Existing approaches to
optimizing the tensor algebra expression of a DNN only consider expressions
representable by a fixed set of predefined operators, missing possible
optimization opportunities between general expressions. We propose OLLIE, the
first derivation-based tensor program optimizer. OLLIE optimizes tensor
programs by leveraging transformations between general tensor algebra
expressions, enabling a significantly larger expression search space that
includes those supported …

arxiv derivation lg tensor