Understanding and Improving Compositional Generalization | AI2

Understanding and Improving Compositional Generalization
Ben Bogin

Pre-trained language models perform well on a large variety of question-answering tasks, but they still often fail in the compositional generalization setup, where models are tested on unseen compositions of reasoning skills.

In this talk, I will go over three research directions that address this challenge. In the first part, I will show how we can improve generalization and interpretability with a compositional model architecture that recursively computes outputs and representations over grounded …

ai2 architecture challenge interpretability language language models part reasoning research setup skills talk trees understanding