Mind the Gap Between Synthetic and Real: Utilizing Transfer Learning to Probe the Boundaries of Stable Diffusion Generated Data

arXiv:2405.03243v1 Announce Type: new
Abstract: Generative foundation models like Stable Diffusion comprise a diverse spectrum of knowledge in computer vision with the potential for transfer learning, e.g., via generating data to train student models for downstream tasks. This could circumvent the necessity of collecting labeled real-world data, thereby presenting a form of data-free knowledge distillation. However, the resultant student models show a significant drop in accuracy compared to models trained on real data. We investigate possible causes for this drop …

abstract arxiv computer computer vision cs.cv data diffusion diverse foundation gap generated generative knowledge mind probe spectrum stable diffusion synthetic tasks train transfer transfer learning type via vision