Label Alignment Regularization for Distribution Shift

arXiv:2211.14960v3 Announce Type: replace
Abstract: Recent work has highlighted the label alignment property (LAP) in supervised learning, where the vector of all labels in the dataset is mostly in the span of the top few singular vectors of the data matrix. Drawing inspiration from this observation, we propose a regularization method for unsupervised domain adaptation that encourages alignment between the predictions in the target domain and its top singular vectors. Unlike conventional domain adaptation approaches that focus on regularizing representations, …

abstract alignment arxiv cs.lg data dataset distribution domain inspiration labels matrix observation property regularization shift singular stat.ml supervised learning type unsupervised vector vectors work