[R] Scaling Instructable Agents Across Many Simulated Worlds - DeepMind 2024 - SIMA - A generalist AI agent for 3D virtual environments. Plays AAA games like No Mans Sky and Valheim!

Blog: [https://deepmind.google/discover/blog/sima-generalist-ai-agent-for-3d-virtual-environments/](https://deepmind.google/discover/blog/sima-generalist-ai-agent-for-3d-virtual-environments/)

Paper: [https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/sima-generalist-ai-agent-for-3d-virtual-environments/Scaling%20Instructable%20Agents%20Across%20Many%20Simulated%20Worlds.pdf](https://storage.googleapis.com/deepmind-media/DeepMind.com/Blog/sima-generalist-ai-agent-for-3d-virtual-environments/Scaling%20Instructable%20Agents%20Across%20Many%20Simulated%20Worlds.pdf)

Abstract:

>Building **embodied AI systems that can follow arbitrary language instructions in any 3D environment is a key challenge for creating general AI.** Accomplishing this goal requires learning to ground language in perception and embodied actions, in order to accomplish complex tasks. The **Scalable, Instructable, Multiworld Agent (SIMA) project** tackles this by training agents to follow free-form instructions across a diverse range of virtual 3D environments, including curated research environments as well as **open-ended, commercial video …

abstract agent agents ai systems building challenge embodied embodied ai environment form free general general ai key language machinelearning perception project scalable sima systems tasks training