Transformer Module Networks for Systematic Generalization in Visual Question Answering. (arXiv:2201.11316v1 [cs.CV])

Web: http://arxiv.org/abs/2201.11316

Transformer-based models achieve great performance on Visual Question
Answering (VQA). However, when we evaluate them on systematic generalization,
i.e., handling novel combinations of known concepts, their performance
degrades. Neural Module Networks (NMNs) are a promising approach for systematic
generalization that consists on composing modules, i.e., neural networks that
tackle a sub-task. Inspired by Transformers and NMNs, we propose Transformer
Module Network (TMN), a novel Transformer-based model for VQA that dynamically
composes modules into a question-specific Transformer network. TMNs achieve
state-of-the-art …

arxiv cv networks question answering transformer