TREACLE: Thrifty Reasoning via Context-Aware LLM and Prompt Selection

arXiv:2404.13082v1 Announce Type: cross
Abstract: Recent successes in natural language processing have led to the proliferation of large language models (LLMs) by multiple providers. Each LLM offering has different inference accuracy, monetary cost, and latency, and their accuracy further depends on the exact wording of the question (i.e., the specific prompt). At the same time, users often have a limit on monetary budget and latency to answer all their questions, and they do not know which LLMs to choose for …

abstract accuracy arxiv context cost cs.ai cs.cl cs.lg inference language language models language processing large language large language models latency llm llms multiple natural natural language natural language processing processing prompt question reasoning type via