SMART: Automatically Scaling Down Language Models with Accuracy Guarantees for Reduced Processing Fees

arXiv:2403.13835v1 Announce Type: new
Abstract: The advancement of Large Language Models (LLMs) has significantly boosted performance in natural language processing (NLP) tasks. However, the deployment of high-performance LLMs incurs substantial costs, primarily due to the increased number of parameters aimed at enhancing model performance. This has made the use of state-of-the-art LLMs more expensive for end-users. AI service providers, such as OpenAI and Anthropic, often offer multiple versions of LLMs with varying prices and performance. However, end-users still face challenges …

abstract accuracy advancement arxiv costs cs.ai cs.cl cs.db cs.lg deployment however language language models language processing large language large language models llms natural natural language natural language processing nlp parameters performance processing scaling smart tasks type