Towards Personalized Evaluation of Large Language Models with An Anonymous Crowd-Sourcing Platform

arXiv:2403.08305v1 Announce Type: new
Abstract: Large language model evaluation plays a pivotal role in the enhancement of its capacity. Previously, numerous methods for evaluating large language models have been proposed in this area. Despite their effectiveness, these existing works mainly focus on assessing objective questions, overlooking the capability to evaluate subjective questions which is extremely common for large language models. Additionally, these methods predominantly utilize centralized datasets for evaluation, with question banks concentrated within the evaluation platforms themselves. Moreover, the …

anonymous arxiv cs.cl evaluation language language models large language large language models personalized platform type