Induced Model Matching: How Restricted Models Can Help Larger Ones

arXiv:2402.12513v1 Announce Type: new
Abstract: We consider scenarios where a very accurate predictive model using restricted features is available at the time of training of a larger, full-featured, model. This restricted model may be thought of as "side-information", derived either from an auxiliary exhaustive dataset or on the same dataset, by forcing the restriction. How can the restricted model be useful to the full model? We propose an approach for transferring the knowledge of the restricted model to the full …

abstract arxiv cs.lg dataset featured features information predictive thought training type