Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answering

arXiv:2311.06503v2 Announce Type: replace
Abstract: Deploying large language models (LLMs) to real scenarios for domain-specific question answering (QA) is a key thrust for LLM applications, which poses numerous challenges, especially in ensuring that responses are both accommodating to user requirements and appropriately leveraging domain-specific knowledge bases. They are the two major difficulties for LLM application as vanilla fine-tuning falls short of addressing. Combining these requirements, we conceive of them as the requirement for the model's preference to be harmoniously aligned …

abstract alignment applications arxiv challenges cs.ai cs.cl domain key knowledge language language models large language large language models llm llm applications llms major question question answering requirements responses type