Optimal Design for Human Feedback

arXiv:2404.13895v1 Announce Type: new
Abstract: Learning of preference models from human feedback has been central to recent advances in artificial intelligence. Motivated by this progress, and the cost of obtaining high-quality human annotations, we study the problem of data collection for learning preference models. The key idea in our work is to generalize optimal designs, a tool for computing efficient data logging policies, to ranked lists. To show the generality of our ideas, we study both absolute and relative feedback …

abstract advances annotations artificial artificial intelligence arxiv collection cost cs.lg data data collection design designs feedback human human feedback intelligence key progress quality study the key type work