PrOnto: Language Model Evaluations for 859 Languages

arXiv:2305.12612v2 Announce Type: replace
Abstract: Evaluation datasets are critical resources for measuring the quality of pretrained language models. However, due to the high cost of dataset annotation, these resources are scarce for most languages other than English, making it difficult to assess the quality of language models. In this work, we present a new method for evaluation dataset construction which enables any language with a New Testament translation to receive a suite of evaluation datasets suitable for pretrained language model …

abstract annotation arxiv cost cs.cl dataset datasets english evaluation however language language model language models languages making measuring quality resources type work