Bayesian Preference Elicitation with Language Models

arXiv:2403.05534v1 Announce Type: new
Abstract: Aligning AI systems to users' interests requires understanding and incorporating humans' complex values and preferences. Recently, language models (LMs) have been used to gather information about the preferences of human users. This preference data can be used to fine-tune or guide other LMs and/or AI systems. However, LMs have been shown to struggle with crucial aspects of preference learning: quantifying uncertainty, modeling human mental states, and asking informative questions. These challenges have been addressed in …

abstract ai systems arxiv bayesian cs.cl data gather guide however human humans information language language models lms systems type understanding values