Balancing Exploration and Exploitation in LLM using Soft RLLF for Enhanced Negation Understanding

arXiv:2403.01185v1 Announce Type: new
Abstract: Finetuning approaches in NLP often focus on exploitation rather than exploration, which may lead to suboptimal models. Given the vast search space of natural language, this limited exploration can restrict their performance in complex, high-stakes domains, where accurate negation understanding and logical reasoning abilities are crucial. To address this issue, we leverage Reinforcement Learning from Logical Feedback (RLLF) to create an effective balance between exploration and exploitation in LLMs. Our approach employs an appropriate benchmark …

abstract arxiv cs.ai cs.cl domains exploitation exploration finetuning focus language llm natural natural language nlp performance search space type understanding vast