[D] Why don't diffusion models overfit?

DMs learn to map noise to the data distribution. I'm struggling to understand why in the limit of enough training epochs and a big enough/powerful enough model they wouldn't just (over)fit (to) the empirical data distribution, in which case samples would just be copies from the training dataset. There isn't anything like the KL term in a VAE regularizing things.

big case data dataset diffusion diffusion models distribution isn learn machinelearning map noise samples training