What Does Stable Diffusion Know about the 3D Scene?

arXiv:2310.06836v2 Announce Type: replace
Abstract: Recent advances in generative models like Stable Diffusion enable the generation of highly photo-realistic images. Our objective in this paper is to probe the diffusion network to determine to what extent it 'understands' different properties of the 3D scene depicted in an image. To this end, we make the following contributions: (i) We introduce a protocol to evaluate whether features of an off-the-shelf diffusion model encode a number of physical 'properties' of the 3D scene, …

abstract advances arxiv cs.cv diffusion generative generative models image images network paper photo probe stable diffusion type