Ada-LEval: Evaluating long-context LLMs with length-adaptable benchmarks

arXiv:2404.06480v1 Announce Type: new
Abstract: Recently, the large language model (LLM) community has shown increasing interest in enhancing LLMs' capability to handle extremely long documents. As various long-text techniques and model architectures emerge, the precise and detailed evaluation of models' long-text capabilities has become increasingly important. Existing long-text evaluation benchmarks, such as L-Eval and LongBench, construct long-text test sets based on open-source datasets, focusing mainly on QA and summarization tasks. These datasets include test samples of varying lengths (from 2k …

ada arxiv benchmarks context cs.ai cs.cl llms type