Self-Alignment with Instruction Backtranslation

arXiv:2308.06259v3 Announce Type: replace
Abstract: We present a scalable method to build a high quality instruction following language model by automatically labelling human-written text with corresponding instructions. Our approach, named instruction backtranslation, starts with a language model finetuned on a small amount of seed data, and a given web corpus. The seed model is used to construct training examples by generating instruction prompts for web documents (self-augmentation), and then selecting high quality examples from among these candidates (self-curation). This data …

abstract alignment arxiv build cs.cl data human labelling language language model quality scalable seed small text type web