all AI news
ElitePLM: An Empirical Study on General Language Ability Evaluation of Pretrained Language Models. (arXiv:2205.01523v1 [cs.CL])
May 4, 2022, 1:11 a.m. | Junyi Li, Tianyi Tang, Zheng Gong, Lixin Yang, Zhuohao Yu, Zhipeng Chen, Jingyuan Wang, Wayne Xin Zhao, Ji-Rong Wen
cs.CL updates on arXiv.org arxiv.org
Nowadays, pretrained language models (PLMs) have dominated the majority of
NLP tasks. While, little research has been conducted on systematically
evaluating the language abilities of PLMs. In this paper, we present a
large-scale empirical study on general language ability evaluation of PLMs
(ElitePLM). In our study, we design four evaluation dimensions, i.e. memory,
comprehension, reasoning, and composition, to measure ten widely-used PLMs
within five categories. Our empirical results demonstrate that: (1) PLMs with
varying training objectives and strategies are good …
More from arxiv.org / cs.CL updates on arXiv.org
Jobs in AI, ML, Big Data
Data Scientist (m/f/x/d)
@ Symanto Research GmbH & Co. KG | Spain, Germany
(Fluent Ukrainian) ML Engineer
@ Outstaff Your Team | Warsaw, Masovian Voivodeship, Poland - Remote
Senior Back-end Engineer (Cargo Models)
@ Kpler | London
Senior Data Science Manager, Marketplace Foundations
@ Reddit | Remote - United States
Intermediate Data Engineer
@ JUMO | South Africa
Data Engineer ( remote )
@ AssistRx | Orlando, Florida, United States - Remote