Role Prompting Guided Domain Adaptation with General Capability Preserve for Large Language Models

arXiv:2403.02756v1 Announce Type: new
Abstract: The growing interest in Large Language Models (LLMs) for specialized applications has revealed a significant challenge: when tailored to specific domains, LLMs tend to experience catastrophic forgetting, compromising their general capabilities and leading to a suboptimal user experience. Additionally, crafting a versatile model for multiple domains simultaneously often results in a decline in overall performance due to confusion between domains. In response to these issues, we present the RolE Prompting Guided Multi-Domain Adaptation (REGA) strategy. …

abstract applications arxiv capabilities capability catastrophic forgetting challenge cs.cl domain domain adaptation domains experience general language language models large language large language models llms prompting role type