Large language models can accurately predict searcher preferences

arXiv:2309.10621v2 Announce Type: replace-cross
Abstract: Relevance labels, which indicate whether a search result is valuable to a searcher, are key to evaluating and optimising search systems. The best way to capture the true preferences of users is to ask them for their careful feedback on which results would be useful, but this approach does not scale to produce a large number of labels. Getting relevance labels at scale is usually done with third-party labellers, who judge on behalf of the …

abstract arxiv cs.ai cs.cl cs.ir cs.lg feedback key labels language language models large language large language models results search search result systems them true type