Microsoft’s LLMA Accelerates LLM Generations via an ‘Inference-With-Reference’ Decoding Approach

In the new paper Inference with Reference: Lossless Acceleration of Large Language Models, a Microsoft research team proposes LLMA, an inference-with-reference decoding mechanism that achieves up to 2x lossless speed-ups with identical generation results by exploiting the overlaps between LLM outputs and references.

ai artificial intelligence decoding deep-neural-networks inference language language models large language model large language models llm machine learning machine learning & data science microsoft microsoft research ml nature language tech paper reference research research team speed team technology ups