Porting Large Language Models to Mobile Devices for Question Answering

arXiv:2404.15851v1 Announce Type: new
Abstract: Deploying Large Language Models (LLMs) on mobile devices makes all the capabilities of natural language processing available on the device. An important use case of LLMs is question answering, which can provide accurate and contextually relevant answers to a wide array of user queries. We describe how we managed to port state of the art LLMs to mobile devices, enabling them to operate natively on the device. We employ the llama.cpp framework, a flexible and …

abstract array arxiv capabilities case cs.cv devices language language models language processing large language large language models llms mobile mobile devices natural natural language natural language processing processing queries question question answering type