Aligning LLM Agents by Learning Latent Preference from User Edits

arXiv:2404.15269v1 Announce Type: cross
Abstract: We study interactive learning of language agents based on user edits made to the agent's output. In a typical setting such as writing assistants, the user interacts with a language agent to generate a response given a context, and may optionally edit the agent response to personalize it based on their latent preference, in addition to improving the correctness. The edit feedback is naturally generated, making it a suitable candidate for improving the agent's alignment …

abstract agent agents arxiv assistants context cs.ai cs.cl cs.ir cs.lg edit generate interactive language llm study type writing