OpenBezoar: Small, Cost-Effective and Open Models Trained on Mixes of Instruction Data

arXiv:2404.12195v1 Announce Type: cross
Abstract: Instruction fine-tuning pretrained LLMs for diverse downstream tasks has demonstrated remarkable success and has captured the interest of both academics and practitioners. To ensure such fine-tuned LLMs align with human preferences, techniques such as RLHF and DPO have emerged. At the same time, there is increasing interest in smaller parameter counts for models. In this work, using OpenLLaMA 3Bv2 as a base model, we describe the recipe used to fine-tune the OpenBezoar family of models. …

arxiv cost cs.cl cs.lg data open models small type