Merino: Entropy-driven Design for Generative Language Models on IoT Devices

arXiv:2403.07921v1 Announce Type: new
Abstract: Generative Large Language Models (LLMs) stand as a revolutionary advancement in the modern era of artificial intelligence (AI). However, directly deploying LLMs in resource-constrained hardware, such as Internet-of-Things (IoT) devices, is difficult due to their high computational cost. In this paper, we propose a novel information-entropy framework for designing mobile-friendly generative language models. Our key design paradigm is to maximize the entropy of transformer decoders within the given computational budgets. The whole design procedure involves …

abstract advancement artificial artificial intelligence arxiv computational cost cs.ai cs.cl cs.lg design devices entropy generative hardware however intelligence internet iot language language models large language large language models llms modern novel paper type