Instruction Matters, a Simple yet Effective Task Selection Approach in Instruction Tuning for Specific Tasks

arXiv:2404.16418v1 Announce Type: new
Abstract: Instruction tuning has shown its ability to not only enhance zero-shot generalization across various tasks but also its effectiveness in improving the performance of specific tasks. A crucial aspect in instruction tuning for a particular task is a strategic selection of related tasks that offer meaningful supervision, thereby enhancing efficiency and preventing performance degradation from irrelevant tasks. Our research reveals that leveraging instruction information \textit{alone} enables the identification of pertinent tasks for instruction tuning. This …

abstract arxiv cs.cl improving performance simple specific tasks tasks type zero-shot