[P] Stable-diffusion's implementation of Paint-with-words : method from NVIDIA that generates images from text-labeled segmentation map.

Hi. Very recently researchers from NVIDIA released their recent work on text-to-image diffusion models, eDiffi ([https://deepimagination.cc/eDiffi/](https://deepimagination.cc/eDiffi/)) . In their paper they proposed various methods, including paint-with-words.

Paint-with-words let you generate image from arbitrary text-labeled segmentation map. Checkout their paper and method for more details.

Unfortunately, their code + eDiffi models were not available. However, Stable Diffusion can do just the same, as they both have cross attention module.

I've tried to make it work with stable diffusion, and it worked! …

diffusion images implementation machinelearning map nvidia segmentation text words