Genie: Generative Interactive Environments

arXiv:2402.15391v1 Announce Type: new
Abstract: We introduce Genie, the first generative interactive environment trained in an unsupervised manner from unlabelled Internet videos. The model can be prompted to generate an endless variety of action-controllable virtual worlds described through text, synthetic images, photographs, and even sketches. At 11B parameters, Genie can be considered a foundation world model. It is comprised of a spatiotemporal video tokenizer, an autoregressive dynamics model, and a simple and scalable latent action model. Genie enables users to …

abstract arxiv cs.ai cs.cv cs.lg environment environments foundation generate generative genie images interactive internet parameters photographs synthetic text through type unsupervised videos virtual virtual worlds world