Learning to Act without Actions

arXiv:2312.10812v2 Announce Type: replace
Abstract: Pre-training large models on vast amounts of web data has proven to be an effective approach for obtaining powerful, general models in domains such as language and vision. However, this paradigm has not yet taken hold in reinforcement learning. This is because videos, the most abundant form of embodied behavioral data on the web, lack the action labels required by existing methods for imitating behavior from demonstrations. We introduce Latent Action Policies (LAPO), a method …

abstract act arxiv cs.ai cs.lg data domains embodied form general however language large models paradigm pre-training reinforcement reinforcement learning training type vast videos vision web