Microsoft's Orca 2 LLM Outperforms Models That Are 10x Larger

Microsoft Research released its Orca 2 LLM, a fine-tuned version of Llama 2 that performs as well as or better than models that contain 10x the number of parameters. Orca 2 uses a synthetic training dataset and a new technique called Prompt Erasure to achieve this performance.

ai anthony dataset deep learning generative-ai large language models llama llama 2 llm microsoft microsoft research ml & data engineering neural networks orca orca 2 parameters performance prompt research synthetic training