Mixture of insighTful Experts (MoTE): The Synergy of Thought Chains and Expert Mixtures in Self-Alignment

arXiv:2405.00557v1 Announce Type: new
Abstract: As the capabilities of large language models (LLMs) have expanded dramatically, aligning these models with human values presents a significant challenge, posing potential risks during deployment. Traditional alignment strategies rely heavily on human intervention, such as Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF), or on the self-alignment capacities of LLMs, which usually require a strong LLM's emergent ability to improve its original bad answer. To address these challenges, we propose a novel …

abstract alignment arxiv capabilities challenge cs.ai cs.cl deployment expert experts fine-tuning human human intervention language language models large language large language models llms risks sft strategies supervised fine-tuning synergy thought type values