CREPE: Coordinate-Aware End-to-End Document Parser

arXiv:2405.00260v1 Announce Type: new
Abstract: In this study, we formulate an OCR-free sequence generation model for visual document understanding (VDU). Our model not only parses text from document images but also extracts the spatial coordinates of the text based on the multi-head architecture. Named as Coordinate-aware End-to-end Document Parser (CREPE), our method uniquely integrates these capabilities by introducing a special token for OCR text, and token-triggered coordinate decoding. We also proposed a weakly-supervised framework for cost-efficient training, requiring only parsing …

abstract architecture arxiv cs.cv document document understanding free head images multi-head ocr spatial study text type understanding visual