Multimodal Semantic-Aware Automatic Colorization with Diffusion Prior

arXiv:2404.16678v1 Announce Type: new
Abstract: Colorizing grayscale images offers an engaging visual experience. Existing automatic colorization methods often fail to generate satisfactory results due to incorrect semantic colors and unsaturated colors. In this work, we propose an automatic colorization pipeline to overcome these challenges. We leverage the extraordinary generative ability of the diffusion prior to synthesize color with plausible semantics. To overcome the artifacts introduced by the diffusion prior, we apply the luminance conditional guidance. Moreover, we adopt multimodal high-level …

abstract arxiv challenges colorization colors cs.cv diffusion experience generate generative images multimodal pipeline prior results semantic type visual work