ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Models

arXiv:2403.20262v1 Announce Type: cross
Abstract: Research on Large Language Models (LLMs) has recently witnessed an increasing interest in extending models' context size to better capture dependencies within long documents. While benchmarks have been proposed to assess long-range abilities, existing efforts primarily considered generic tasks that are not necessarily aligned with real-world applications. In contrast, our work proposes a new benchmark for long-context LLMs focused on a practical meeting assistant scenario. In this scenario, the long contexts consist of transcripts obtained …

abstract arxiv assistant benchmark benchmarks context cs.ai cs.cl cs.lg dependencies documents language language models large language large language models llms research tasks type world