CriticBench: Benchmarking LLMs for Critique-Correct Reasoning

arXiv:2402.14809v1 Announce Type: cross
Abstract: The ability of Large Language Models (LLMs) to critique and refine their reasoning is crucial for their application in evaluation, feedback provision, and self-improvement. This paper introduces CriticBench, a comprehensive benchmark designed to assess LLMs' abilities to critique and rectify their reasoning across a variety of tasks. CriticBench encompasses five reasoning domains: mathematical, commonsense, symbolic, coding, and algorithmic. It compiles 15 datasets and incorporates responses from three LLM families. Utilizing CriticBench, we evaluate and dissect …

abstract application arxiv benchmark benchmarking critique cs.ai cs.cl cs.lg evaluation feedback improvement language language models large language large language models llms paper reasoning refine self-improvement tasks type