OrchestraLLM: Efficient Orchestration of Language Models for Dialogue State Tracking

arXiv:2311.09758v2 Announce Type: replace
Abstract: Large language models (LLMs) have revolutionized the landscape of Natural Language Processing systems, but are computationally expensive. To reduce the cost without sacrificing performance, previous studies have explored various approaches to harness the potential of Small Language Models (SLMs) as cost-effective alternatives to their larger counterparts. Driven by findings that SLMs and LLMs exhibit complementary strengths in a structured knowledge extraction task, this work presents a novel SLM/LLM routing framework designed to improve computational efficiency …

abstract arxiv cost cs.cl dialogue harness landscape language language models language processing large language large language models llms natural natural language natural language processing orchestration performance processing reduce slms small small language models state studies systems tracking type