Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

arXiv:2404.14219v1 Announce Type: new
Abstract: We introduce phi-3-mini, a 3.8 billion parameter language model trained on 3.3 trillion tokens, whose overall performance, as measured by both academic benchmarks and internal testing, rivals that of models such as Mixtral 8x7B and GPT-3.5 (e.g., phi-3-mini achieves 69% on MMLU and 8.38 on MT-bench), despite being small enough to be deployed on a phone. The innovation lies entirely in our dataset for training, a scaled-up version of the one used for phi-2, composed …

abstract academic arxiv benchmarks billion cs.ai cs.cl gpt gpt-3 gpt-3.5 language language model mixtral mixtral 8x7b mmlu performance phi phone report technical testing tokens type