Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruning

arXiv:2312.08901v3 Announce Type: replace
Abstract: Large Language Models (LLMs) have shown impressive capabilities, yet they still struggle with math reasoning. In this work, we propose CoT-Influx, a novel approach that pushes the boundary of few-shot Chain-of-Thoughts (CoT) learning to improve LLM mathematical reasoning. Motivated by the observation that adding more concise CoT examples in the prompt can improve LLM reasoning performance, CoT-Influx employs a coarse-to-fine pruner to maximize the input of effective and concise CoT examples. The pruner first selects …

abstract arxiv boosting capabilities context cs.ai cs.cl few-shot language language models large language large language models llm llm reasoning llms math mathematical reasoning novel observation pruning reasoning struggle thoughts type work