TeaMs-RL: Teaching LLMs to Teach Themselves Better Instructions via Reinforcement Learning

arXiv:2403.08694v1 Announce Type: new
Abstract: The development of Large Language Models (LLMs) often confronts challenges stemming from the heavy reliance on human annotators in the reinforcement learning with human feedback (RLHF) framework, or the frequent and costly external queries tied to the self-instruct paradigm. In this work, we pivot to Reinforcement Learning (RL) -- but with a twist. Diverging from the typical RLHF, which refines LLMs following instruction data training, we use RL to directly generate the foundational instruction dataset …

abstract arxiv challenges cs.cl development feedback framework human human feedback language language models large language large language models llms paradigm pivot queries reinforcement reinforcement learning reliance rlhf stemming teaching teams type via work