Knowing Your Nonlinearities: Shapley Interactions Reveal the Underlying Structure of Data

arXiv:2403.13106v1 Announce Type: new
Abstract: Measuring nonlinear feature interaction is an established approach to understanding complex patterns of attribution in many models. In this paper, we use Shapley Taylor interaction indices (STII) to analyze the impact of underlying data structure on model representations in a variety of modalities, tasks, and architectures. Considering linguistic structure in masked and auto-regressive language models (MLMs and ALMs), we find that STII increases within idiomatic expressions and that MLMs scale STII with syntactic distance, relying …

abstract analyze arxiv attribution cs.ai cs.cl cs.cv cs.lg data feature impact interactions measuring paper patterns tasks taylor type understanding