IMO: Greedy Layer-Wise Sparse Representation Learning for Out-of-Distribution Text Classification with Pre-trained Models

arXiv:2404.13504v1 Announce Type: new
Abstract: Machine learning models have made incredible progress, but they still struggle when applied to examples from unseen domains. This study focuses on a specific problem of domain generalization, where a model is trained on one source domain and tested on multiple target domains that are unseen during training. We propose IMO: Invariant features Masks for Out-of-Distribution text classification, to achieve OOD generalization by learning invariant features. During training, IMO would learn sparse mask layers to …

abstract arxiv classification cs.cl distribution domain domains examples layer machine machine learning machine learning models multiple pre-trained models progress representation representation learning struggle study text text classification type wise