LocCa: Visual Pretraining with Location-aware Captioners

arXiv:2403.19596v1 Announce Type: new
Abstract: Image captioning has been shown as an effective pretraining method similar to contrastive pretraining. However, the incorporation of location-aware information into visual pretraining remains an area with limited research. In this paper, we propose a simple visual pretraining method with location-aware captioners (LocCa). LocCa uses a simple image captioner task interface, to teach a model to read out rich information, i.e. bounding box coordinates, and captions, conditioned on the image pixel input. Thanks to the …

abstract arxiv captioning cs.cv however image information location paper pretraining research simple type visual