The Real, the Better: Aligning Large Language Models with Online Human Behaviors

arXiv:2405.00578v1 Announce Type: new
Abstract: Large language model alignment is widely used and studied to avoid LLM producing unhelpful and harmful responses. However, the lengthy training process and predefined preference bias hinder adaptation to online diverse human preferences. To this end, this paper proposes an alignment framework, called Reinforcement Learning with Human Behavior (RLHB), to align LLMs by directly leveraging real online human behaviors. By taking the generative adversarial framework, the generator is trained to respond following expected human behavior; …

abstract alignment arxiv bias cs.ai cs.cl diverse framework hinder however human language language model language models large language large language model large language models llm paper process responses training type