Improving Code Generation by Training with Natural Language Feedback

arXiv:2303.16749v2 Announce Type: replace-cross
Abstract: The potential for pre-trained large language models (LLMs) to use natural language feedback at inference time has been an exciting recent development. We build upon this observation by formalizing an algorithm for learning from natural language feedback at training time instead, which we call Imitation learning from Language Feedback (ILF). ILF requires only a small amount of human-written feedback during training and does not require the same feedback at test time, making it both user-friendly …

abstract algorithm arxiv build call code code generation cs.ai cs.cl cs.lg cs.se development feedback imitation learning inference language language models large language large language models llms natural natural language observation training type