GRAM: Global Reasoning for Multi-Page VQA

arXiv:2401.03411v2 Announce Type: replace-cross
Abstract: The increasing use of transformer-based large language models brings forward the challenge of processing long sequences. In document visual question answering (DocVQA), leading methods focus on the single-page setting, while documents can span hundreds of pages. We present GRAM, a method that seamlessly extends pre-trained single-page models to the multi-page setting, without requiring computationally-heavy pretraining. To do so, we leverage a single-page encoder for local page-level understanding, and enhance it with document-level designated layers and …

abstract arxiv challenge cs.cl cs.cv document documents focus global language language models large language large language models page processing question question answering reasoning transformer type visual vqa