Unsupervised LLM Adaptation for Question Answering

arXiv:2402.12170v1 Announce Type: new
Abstract: Large language models (LLM) learn diverse knowledge present in the large-scale training dataset via self-supervised training. Followed by instruction-tuning, LLM acquires the ability to return correct information for diverse questions. However, adapting these pre-trained LLMs to new target domains, such as different organizations or periods, for the question-answering (QA) task incurs a substantial annotation cost. To tackle this challenge, we propose a novel task, unsupervised LLM adaptation for question answering. In this task, we leverage …

abstract arxiv cs.ai cs.cl dataset diverse domains information knowledge language language models large language large language models learn llm llms organizations question question answering questions scale supervised training training type unsupervised via