SyncMask: Synchronized Attentional Masking for Fashion-centric Vision-Language Pretraining

arXiv:2404.01156v1 Announce Type: new
Abstract: Vision-language models (VLMs) have made significant strides in cross-modal understanding through large-scale paired datasets. However, in fashion domain, datasets often exhibit a disparity between the information conveyed in image and text. This issue stems from datasets containing multiple images of a single fashion item all paired with one text, leading to cases where some textual details are not visible in individual images. This mismatch, particularly when non-co-occurring elements are masked, undermines the training of conventional …

abstract arxiv cs.ai cs.cv datasets domain fashion however image images information issue language language models masking modal multiple pretraining scale text the information through type understanding vision vision-language models vlms