Multi-step Problem Solving Through a Verifier: An Empirical Analysis on Model-induced Process Supervision

Process supervision, using a trained verifier to evaluate the intermediate steps generated by reasoner, has demonstrated significant improvements in multi-step problem solving. In this paper, to avoid expensive human annotation effort on the verifier training data, we introduce Model-induced Process Supervision (MiPS), a novel method for automating data curation. MiPS annotates an intermediate step by sampling completions of this solution through the reasoning model, and obtaining an accuracy defined as the proportion of correct completions. Errors in the reasoner would …

analysis annotation cs.ai cs.cl cs.lg data generated human improvements intermediate novel paper process process supervision supervision through training training data