Key-point Guided Deformable Image Manipulation Using Diffusion Model

arXiv:2401.08178v2 Announce Type: replace
Abstract: In this paper, we introduce a Key-point-guided Diffusion probabilistic Model (KDM) that gains precise control over images by manipulating the object's key-point. We propose a two-stage generative model incorporating an optical flow map as an intermediate output. By doing so, a dense pixel-wise understanding of the semantic relation between the image and sparse key point is configured, leading to more realistic image generation. Additionally, the integration of optical flow helps regulate the inter-frame variance of …

abstract arxiv control cs.cv diffusion diffusion model flow generative image images intermediate key manipulation map object optical optical flow paper pixel probabilistic model semantic stage type understanding wise