Astro-NER -- Astronomy Named Entity Recognition: Is GPT a Good Domain Expert Annotator?

arXiv:2405.02602v1 Announce Type: new
Abstract: In this study, we address one of the challenges of developing NER models for scholarly domains, namely the scarcity of suitable labeled data. We experiment with an approach using predictions from a fine-tuned LLM model to aid non-domain experts in annotating scientific entities within astronomy literature, with the goal of uncovering whether such a collaborative process can approximate domain expertise. Our results reveal moderate agreement between a domain expert and the LLM-assisted non-experts, as well …

abstract arxiv astro astronomy challenges cs.ai cs.cl cs.it data domain domain expert domain experts domains experiment expert experts good gpt llm math.it ner predictions recognition scientific study type