CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Model

arXiv:2305.14014v3 Announce Type: replace
Abstract: Pre-trained vision-language models~(VLMs) are the de-facto foundation models for various downstream tasks. However, scene text recognition methods still prefer backbones pre-trained on a single modality, namely, the visual modality, despite the potential of VLMs to serve as powerful scene text readers. For example, CLIP can robustly identify regular (horizontal) and irregular (rotated, curved, blurred, or occluded) text in images. With such merits, we transform CLIP into a scene text reader and introduce CLIP4STR, a simple …

abstract arxiv cs.cv example foundation however language language model language models readers recognition serve simple tasks text type vision vision-language vision-language models visual vlms