Telling Left from Right: Identifying Geometry-Aware Semantic Correspondence

arXiv:2311.17034v2 Announce Type: replace
Abstract: While pre-trained large-scale vision models have shown significant promise for semantic correspondence, their features often struggle to grasp the geometry and orientation of instances. This paper identifies the importance of being geometry-aware for semantic correspondence and reveals a limitation of the features of current foundation models under simple post-processing. We show that incorporating this information can markedly enhance semantic correspondence performance with simple but effective solutions in both zero-shot and supervised settings. We also construct …

arxiv cs.cv geometry semantic type