LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback

arXiv:2311.09336v2 Announce Type: replace
Abstract: Recent large language models (LLM) are leveraging human feedback to improve their generation quality. However, human feedback is costly to obtain, especially during inference. In this work, we propose LLMRefine, an inference time optimization method to refine LLM's output. The core idea is to use a learned fine-grained feedback model to pinpoint defects and guide LLM to refine them iteratively. Using original LLM as a proposal of edits, LLMRefine searches for defect-less text via simulated …

abstract arxiv core cs.cl feedback fine-grained however human human feedback inference language language models large language large language models llm optimization quality refine type via work