[R] Transframer: Arbitrary Frame Prediction with Generative Models - DeepMind 2022 - Can generate 30 Second Videos from a single frame while also being able to do 8 different Vision tasks including depth estimation, object detection and instance segm

Paper: [https://arxiv.org/abs/2203.09494](https://arxiv.org/abs/2203.09494)

Abstract:

>We present a general-purpose framework for image modelling and vision tasks based on probabilistic frame prediction. Our approach unifies a broad range of tasks, from image segmentation, to novel view synthesis and video interpolation. We pair this framework with an architecture we term Transframer, which uses U-Net and Transformer components to condition on annotated context frames, and outputs sequences of sparse, compressed image features. Transframer is the state-of-the-art on a variety of video generation benchmarks, is competitive …

deepmind detection machinelearning prediction videos vision