Controlled Training Data Generation with Diffusion Models

arXiv:2403.15309v1 Announce Type: cross
Abstract: In this work, we present a method to control a text-to-image generative model to produce training data specifically "useful" for supervised learning. Unlike previous works that employ an open-loop approach and pre-define prompts to generate new data using either a language model or human expertise, we develop an automated closed-loop system which involves two feedback mechanisms. The first mechanism uses feedback from a given supervised model and finds adversarial prompts that result in image generations …

abstract arxiv control cs.cl cs.cv cs.lg data diffusion diffusion models expertise generate generative human image language language model loop prompts supervised learning text text-to-image training training data type work