What makes Models Compositional? A Theoretical View: With Supplement

arXiv:2405.02350v1 Announce Type: new
Abstract: Compositionality is thought to be a key component of language, and various compositional benchmarks have been developed to empirically probe the compositional generalization of existing sequence processing models. These benchmarks often highlight failures of existing models, but it is not clear why these models fail in this way. In this paper, we seek to theoretically understand the role the compositional structure of the models plays in these failures and how this structure relates to their …

abstract arxiv benchmarks clear cs.ai cs.lg highlight key language probe processing thought type view