Declaration-based Prompt Tuning for Visual Question Answering. (arXiv:2205.02456v1 [cs.CV])

Web: http://arxiv.org/abs/2205.02456

In recent years, the pre-training-then-fine-tuning paradigm has yielded
immense success on a wide spectrum of cross-modal tasks, such as visual
question answering (VQA), in which a visual-language (VL) model is first
optimized via self-supervised task objectives, e.g., masked language modeling
(MLM) and image-text matching (ITM), and then fine-tuned to adapt to downstream
task (e.g., VQA) via a brand-new objective function, e.g., answer prediction.
The inconsistency of the objective forms not only severely limits the
generalization of pre-trained VL models to …

arxiv cv question answering