Complexity Matters: Dynamics of Feature Learning in the Presence of Spurious Correlations

arXiv:2403.03375v1 Announce Type: new
Abstract: Existing research often posits spurious features as "easier" to learn than core features in neural network optimization, but the impact of their relative simplicity remains under-explored. Moreover they mainly focus on the end performance intead of the learning dynamics of feature learning. In this paper, we propose a theoretical framework and associated synthetic dataset grounded in boolean function analysis which allows for fine-grained control on the relative complexity (compared to core features) and correlation strength …

abstract arxiv complexity core correlations cs.lg dynamics feature features focus impact learn network neural network optimization performance research simplicity the end type