Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agents

arXiv:2402.11651v1 Announce Type: new
Abstract: Large language models (LLMs) have achieved success in acting as agents, which interact with environments through tools like search engines. However, LLMs are not optimized specifically for tool use during training or alignment, limiting their effectiveness as agents. To resolve this problem, previous work has collected interaction trajectories between GPT-4 and environments, and fine-tuned smaller models with them. As part of this, the standard approach has been to simply discard trajectories that do not finish …

abstract acting agents alignment arxiv cs.cl environments examples failure fine-tuning language language models large language large language models llms negative search success through tool tools training type