AI on AI: Exploring the Utility of GPT as an Expert Annotator of AI Publications

arXiv:2403.09097v1 Announce Type: new
Abstract: Identifying scientific publications that are within a dynamic field of research often requires costly annotation by subject-matter experts. Resources like widely-accepted classification criteria or field taxonomies are unavailable for a domain like artificial intelligence (AI), which spans emerging topics and technologies. We address these challenges by inferring a functional definition of AI research from existing expert labels, and then evaluating state-of-the-art chatbot models on the task of expert data annotation. Using the arXiv publication database …

abstract annotation artificial artificial intelligence arxiv classification cs.cl domain dynamic expert experts gpt intelligence matter publications research resources taxonomies topics type utility