Action-conditioned video data improves predictability

arXiv:2404.05439v1 Announce Type: new
Abstract: Long-term video generation and prediction remain challenging tasks in computer vision, particularly in partially observable scenarios where cameras are mounted on moving platforms. The interaction between observed image frames and the motion of the recording agent introduces additional complexities. To address these issues, we introduce the Action-Conditioned Video Generation (ACVG) framework, a novel approach that investigates the relationship between actions and generated image frames through a deep dual Generator-Actor architecture. ACVG generates video sequences conditioned …

abstract agent arxiv cameras complexities computer computer vision cs.cv data image long-term moving observable platforms prediction recording tasks type video video data video generation vision