LightIt: Illumination Modeling and Control for Diffusion Models

arXiv:2403.10615v1 Announce Type: cross
Abstract: We introduce LightIt, a method for explicit illumination control for image generation. Recent generative methods lack lighting control, which is crucial to numerous artistic aspects of image generation such as setting the overall mood or cinematic appearance. To overcome these limitations, we propose to condition the generation on shading and normal maps. We model the lighting with single bounce shading, which includes cast shadows. We first train a shading estimation module to generate a dataset …

abstract arxiv control cs.cv cs.gr cs.lg diffusion diffusion models generative image image generation lighting limitations modeling mood type