RankPrompt: Step-by-Step Comparisons Make Language Models Better Reasoners

arXiv:2403.12373v1 Announce Type: new
Abstract: Large Language Models (LLMs) have achieved impressive performance across various reasoning tasks. However, even state-of-the-art LLMs such as ChatGPT are prone to logical errors during their reasoning processes. Existing solutions, which include deploying task-specific verifiers or voting over multiple reasoning paths, either require extensive human annotations or fail in scenarios with inconsistent responses. To address these challenges, we introduce RankPrompt, a new prompting method that enables LLMs to self-rank their responses without additional resources. RankPrompt …

abstract annotations art arxiv chatgpt cs.cl errors however human language language models large language large language models llms multiple performance processes reasoning solutions state step-by-step tasks type voting