all AI news
Knowing Where and What: Unified Word Block Pretraining for Document Understanding. (arXiv:2207.13979v2 [cs.CL] UPDATED)
Aug. 1, 2022, 1:11 a.m. | Song Tao, Zijian Wang, Tiantian Fan, Canjie Luo, Can Huang
cs.CL updates on arXiv.org arxiv.org
Due to the complex layouts of documents, it is challenging to extract
information for documents. Most previous studies develop multimodal pre-trained
models in a self-supervised way. In this paper, we focus on the embedding
learning of word blocks containing text and layout information, and propose
UTel, a language model with Unified TExt and Layout pre-training. Specifically,
we propose two pre-training tasks: Surrounding Word Prediction (SWP) for the
layout learning, and Contrastive learning of Word Embeddings (CWE) for
identifying different word …
More from arxiv.org / cs.CL updates on arXiv.org
Jobs in AI, ML, Big Data
Senior Machine Learning Engineer
@ GPTZero | Toronto, Canada
ML/AI Engineer / NLP Expert - Custom LLM Development (x/f/m)
@ HelloBetter | Remote
Doctoral Researcher (m/f/div) in Automated Processing of Bioimages
@ Leibniz Institute for Natural Product Research and Infection Biology (Leibniz-HKI) | Jena
Seeking Developers and Engineers for AI T-Shirt Generator Project
@ Chevon Hicks | Remote
Technical Program Manager, Expert AI Trainer Acquisition & Engagement
@ OpenAI | San Francisco, CA
Director, Data Engineering
@ PatientPoint | Cincinnati, Ohio, United States