M3-VRD: Multimodal Multi-task Multi-teacher Visually-Rich Form Document Understanding

arXiv:2402.17983v1 Announce Type: cross
Abstract: This paper presents a groundbreaking multimodal, multi-task, multi-teacher joint-grained knowledge distillation model for visually-rich form document understanding. The model is designed to leverage insights from both fine-grained and coarse-grained levels by facilitating a nuanced correlation between token and entity representations, addressing the complexities inherent in form documents. Additionally, we introduce new inter-grained and cross-grained loss functions to further refine diverse multi-teacher knowledge distillation transfer process, presenting distribution gaps and a harmonised understanding of form documents. …

abstract arxiv complexities correlation cs.cl cs.cv distillation document documents document understanding fine-grained form groundbreaking insights knowledge multimodal paper token type understanding