SD4Match: Learning to Prompt Stable Diffusion Model for Semantic Matching

arXiv:2310.17569v2 Announce Type: replace-cross
Abstract: In this paper, we address the challenge of matching semantically similar keypoints across image pairs. Existing research indicates that the intermediate output of the UNet within the Stable Diffusion (SD) can serve as robust image feature maps for such a matching task. We demonstrate that by employing a basic prompt tuning technique, the inherent potential of Stable Diffusion can be harnessed, resulting in a significant enhancement in accuracy over previous approaches. We further introduce a …

abstract arxiv challenge cs.cv cs.lg diffusion diffusion model feature image intermediate maps paper prompt research robust semantic serve stable diffusion type unet