OCR-Free Document Data Extraction with Transformers (1/2)

Donut versus Pix2Struct on custom data

Donut and Pix2Struct are image-to-text models that combine the simplicity of pure pixel inputs with visual language understanding tasks. Simply put: an image goes in and extracted indexes come out as JSON.

Recently I released a Donut model finetuned on invoices. Ever so often I get the question how to train with a custom dataset. Also, a similar model was released: Pix2Struct, it claims to be significantly better. …

author data data extraction data science dataset donut extraction free image image-to-text json language language understanding machine learning ocr pixel simplicity text transformers understanding