[R] Vision Grid Transformers for Document Layout Analysis

AlibabaResearch recently (Oct 2023) published a new model for Document Layout Analysis which sets a new benchmark in the task of Document Layout Analysis.


Introduction - To fully leverage multi-modal information and exploit pre-training techniques to learn better representation for DLA, in this paper, we present VGT, a two-stream Vision Grid Transformer, in which Grid Transformer (GiT) is proposed and pre-trained for 2D token-level and segment-level semantic understanding
[https://arxiv.org/abs/2308.14978](https://arxiv.org/abs/2308.14978)

​

Effect on LLM usage - VGT can dissect the page …

analysis benchmark dla document exploit grid information introduction learn machinelearning multi-modal paper pre-training representation training transformer transformers vision