Self-AMPLIFY: Improving Small Language Models with Self Post Hoc Explanations

arXiv:2402.12038v1 Announce Type: new
Abstract: Incorporating natural language rationales in the prompt and In-Context Learning (ICL) has led to a significant improvement of Large Language Models (LLMs) performance. However, rationales currently require human-annotation or the use of auxiliary proxy models to target promising samples or generate high-quality rationales. In this work, we propose Self-AMPLIFY to generate automatically rationales from post hoc explanation methods applied to Small Language Models (SLMs) to improve their own performance. Self-AMPLIFY is a 3-step method that …

abstract amplify annotation arxiv context cs.cl cs.lg generate human improvement in-context learning language language models large language large language models llms natural natural language performance prompt quality samples small small language models the prompt type