CURATRON: Complete Robust Preference Data for Robust Alignment of Large Language Models

arXiv:2403.02745v1 Announce Type: cross
Abstract: This paper addresses the challenges of aligning large language models (LLMs) with human values via preference learning (PL), with a focus on the issues of incomplete and corrupted data in preference datasets. We propose a novel method for robustly and completely recalibrating values within these datasets to enhance LLMs resilience against the issues. In particular, we devise a guaranteed polynomial time ranking algorithm that robustifies several existing models, such as the classic Bradley--Terry--Luce (BTL) (Bradley …

abstract alignment arxiv challenges corrupted data cs.ai cs.cl data datasets focus human language language models large language large language models llms novel paper robust type values via