U-Sketch: An Efficient Approach for Sketch to Image Diffusion Models

arXiv:2403.18425v1 Announce Type: cross
Abstract: Diffusion models have demonstrated remarkable performance in text-to-image synthesis, producing realistic and high resolution images that faithfully adhere to the corresponding text-prompts. Despite their great success, they still fall behind in sketch-to-image synthesis tasks, where in addition to text-prompts, the spatial layout of the generated images has to closely follow the outlines of certain reference sketches. Employing an MLP latent edge predictor to guide the spatial layout of the synthesized image by predicting edge maps …

abstract arxiv cs.ai cs.cv cs.lg diffusion diffusion models generated image image diffusion images performance prompts resolution spatial success synthesis tasks text text-to-image type