Open Vocabulary Semantic Scene Sketch Understanding

arXiv:2312.12463v2 Announce Type: replace
Abstract: We study the underexplored but fundamental vision problem of machine understanding of abstract freehand scene sketches. We introduce a sketch encoder that results in semantically-aware feature space, which we evaluate by testing its performance on a semantic sketch segmentation task. To train our model we rely only on the availability of bitmap sketches with their brief captions and do not require any pixel-level annotations. To obtain generalization to a large set of sketches and categories, …

abstract arxiv cs.cv encoder feature machine performance results segmentation semantic sketches space study testing train type understanding vision