[R] Large Language Models for Compiler Optimization - MetaAi 2023 - Autotuner needs 949 CPU-days to achive nearly the same as this approach in 1shot!

Paper: [https://arxiv.org/abs/2309.07062](https://arxiv.org/abs/2309.07062)

Abstract:

>We explore the novel application of Large Language Models to code optimization. We present a 7B-parameter transformer model trained from scratch to optimize LLVM assembly for code size. The model takes as input unoptimized assembly and outputs a list of compiler options to best optimize the program. Crucially, during training, we ask the model to predict the instruction counts before and after optimization, and the optimized code itself. These auxiliary learning tasks significantly improve the optimization performance …

abstract application assembly code compiler explore language language models large language large language models list machinelearning novel optimization training transformer transformer model