GLoRe: When, Where, and How to Improve LLM Reasoning via Global and Local Refinements

arXiv:2402.10963v1 Announce Type: cross
Abstract: State-of-the-art language models can exhibit impressive reasoning refinement capabilities on math, science or coding tasks. However, recent work demonstrates that even the best models struggle to identify \textit{when and where to refine} without access to external feedback. Outcome-based Reward Models (\textbf{ORMs}), trained to predict correctness of the final answer indicating when to refine, offer one convenient solution for deciding when to refine. Process Based Reward Models (\textbf{PRMs}), trained to predict correctness of intermediate steps, can …

abstract art arxiv capabilities coding cs.cl cs.lg feedback global identify language language models llm llm reasoning math reasoning refine science state struggle tasks type via work