DEM: Distribution Edited Model for Training with Mixed Data Distributions

arXiv:2406.15570v1 Announce Type: new
Abstract: Training with mixed data distributions is a common and important part of creating multi-task and instruction-following models. The diversity of the data distributions and cost of joint training makes the optimization procedure extremely challenging. Data mixing methods partially address this problem, albeit having a sub-optimal performance across data sources and require multiple expensive training runs. In this paper, we propose a simple and efficient alternative for better optimization of the data sources by combining models …

abstract arxiv cost cs.cl cs.lg data distribution diversity important mixed multi optimization part problem training type