Discovering modular solutions that generalize compositionally

arXiv:2312.15001v2 Announce Type: replace
Abstract: Many complex tasks can be decomposed into simpler, independent parts. Discovering such underlying compositional structure has the potential to enable compositional generalization. Despite progress, our most powerful systems struggle to compose flexibly. It therefore seems natural to make models more modular to help capture the compositional nature of many tasks. However, it is unclear under which circumstances modular systems can discover hidden compositional structure. To shed light on this question, we study a teacher-student setting …

abstract arxiv cs.lg cs.ne independent modular natural nature progress solutions struggle systems tasks type