Semantic Image Synthesis with Unconditional Generator

arXiv:2402.14395v1 Announce Type: new
Abstract: Semantic image synthesis (SIS) aims to generate realistic images that match given semantic masks. Despite recent advances allowing high-quality results and precise spatial control, they require a massive semantic segmentation dataset for training the models. Instead, we propose to employ a pre-trained unconditional generator and rearrange its feature maps according to proxy masks. The proxy masks are prepared from the feature maps of random samples in the generator by simple clustering. The feature rearranger learns …

abstract advances arxiv control cs.cv dataset feature generate generator image images maps masks massive match quality segmentation semantic spatial synthesis training type