Improving Pre-trained Language Model Sensitivity via Mask Specific losses: A case study on Biomedical NER

arXiv:2403.18025v1 Announce Type: cross
Abstract: Adapting language models (LMs) to novel domains is often achieved through fine-tuning a pre-trained LM (PLM) on domain-specific data. Fine-tuning introduces new knowledge into an LM, enabling it to comprehend and efficiently perform a target domain task. Fine-tuning can however be inadvertently insensitive if it ignores the wide array of disparities (e.g in word meaning) between source and target domains. For instance, words such as chronic and pressure may be treated lightly in social conversations, …

abstract arxiv biomedical case case study cs.ai cs.cl cs.ir cs.lg data domain domains enabling fine-tuning however improving knowledge language language model language models lms losses ner novel sensitivity study through type via