MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases

arXiv:2402.14905v1 Announce Type: new
Abstract: This paper addresses the growing need for efficient large language models (LLMs) on mobile devices, driven by increasing cloud costs and latency concerns. We focus on designing top-quality LLMs with fewer than a billion parameters, a practical choice for mobile deployment. Contrary to prevailing belief emphasizing the pivotal role of data and parameter quantity in determining model quality, our investigation underscores the significance of model architecture for sub-billion scale LLMs. Leveraging deep and thin architectures, …

abstract arxiv belief billion cases cloud concerns costs cs.ai cs.cl cs.lg deployment designing devices focus language language models large language large language models latency llms mobile mobile devices paper parameters practical quality type use cases