DMoERM: Recipes of Mixture-of-Experts for Effective Reward Modeling

arXiv:2403.01197v1 Announce Type: new
Abstract: The performance of the reward model (RM) is a critical factor in improving the effectiveness of the large language model (LLM) during alignment fine-tuning. There remain two challenges in RM training: 1) training the same RM using various categories of data may cause its generalization performance to suffer from multi-task disturbance, and 2) the human annotation consistency rate is generally only $60\%$ to $75\%$, causing training data to contain a lot of noise. To tackle …

arxiv cs.cl experts modeling recipes type