Distilling Conditional Diffusion Models for Offline Reinforcement Learning through Trajectory Stitching

Deep generative models have recently emerged as an effective approach to offline reinforcement learning. However, their large model size poses challenges in computation. We address this issue by proposing a knowledge distillation method based on data augmentation. In particular, high-return trajectories are generated from a conditional diffusion model, and they are blended with the original trajectories through a novel stitching algorithm that leverages a new reward generator. Applying the resulting dataset to behavioral cloning, the learned shallow policy whose size …

augmentation challenges computation cs.lg data deep generative models diffusion diffusion model diffusion models distillation generated generative generative models issue knowledge offline reinforcement reinforcement learning stitching through trajectory