[D] What makes PPO reinforcement learning and not just having a fancy loss function?

I was looking at training a diffusion model using RLHF, and was looking at this paper [kvablack/ddpo-pytorch: DDPO for finetuning diffusion models, implemented in PyTorch with LoRA support (github.com)](https://github.com/kvablack/ddpo-pytorch/tree/main), but the code itself just seems to be backpropagating the unet based on a fancy(and differentiable at first glance!) loss function. What distinguishes reinforcement learning from just normal model training? Are the two the same and is it merely a matter of terminology?

Copying the relevant code here?

for i, sample …

accelerator code config false list machinelearning negative prompts sample torch tqdm train training unet