ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertion

arXiv:2403.18818v1 Announce Type: new
Abstract: Diffusion models have revolutionized image editing but often generate images that violate physical laws, particularly the effects of objects on the scene, e.g., occlusions, shadows, and reflections. By analyzing the limitations of self-supervised approaches, we propose a practical solution centered on a \q{counterfactual} dataset. Our method involves capturing a scene before and after removing a single object, while minimizing other changes. By fine-tuning a diffusion model on this dataset, we are able to not only …

abstract arxiv bootstrapping counterfactual cs.cv dataset diffusion diffusion models editing effects generate image images laws limitations object objects photorealistic practical reflections solution type