Transformer-Lite: High-efficiency Deployment of Large Language Models on Mobile Phone GPUs

arXiv:2403.20041v1 Announce Type: new
Abstract: The Large Language Model (LLM) is widely employed for tasks such as intelligent assistants, text summarization, translation, and multi-modality on mobile phones. However, the current methods for on-device LLM deployment maintain slow inference speed, which causes poor user experience. To facilitate high-efficiency LLM deployment on device GPUs, we propose four optimization techniques: (a) a symbolic expression-based approach to support dynamic shape model inference; (b) operator optimizations and execution priority setting to enhance inference speed and …

abstract arxiv assistants cs.cl current deployment efficiency experience gpus however inference intelligent language language model language models large language large language model large language models llm mobile mobile phones phone phones speed summarization tasks text text summarization transformer translation type