Has Your Pretrained Model Improved? A Multi-head Posterior Based Approach

arXiv:2401.02987v4 Announce Type: replace
Abstract: The emergence of pre-trained models has significantly impacted Natural Language Processing (NLP) and Computer Vision to relational datasets. Traditionally, these models are assessed through fine-tuned downstream tasks. However, this raises the question of how to evaluate these models more efficiently and more effectively. In this study, we explore a novel approach where we leverage the meta-features associated with each entity as a source of worldly knowledge and employ entity representations from the models. We propose …

abstract arxiv computer computer vision cs.ai cs.cl datasets emergence head language language processing multi-head natural natural language natural language processing nlp posterior pre-trained models processing question raises relational tasks through type vision