MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulation

arXiv:2309.14236v2 Announce Type: replace-cross
Abstract: Robotic systems that aspire to operate in uninstrumented real-world environments must perceive the world directly via onboard sensing. Vision-based learning systems aim to eliminate the need for environment instrumentation by building an implicit understanding of the world based on raw pixels, but navigating the contact-rich high-dimensional search space from solely sparse visual reward signals significantly exacerbates the challenge of exploration. The applicability of such systems is thus typically restricted to simulated or heavily engineered environments …

arxiv cs.ai cs.cv cs.lg cs.ro manipulation replace robot robot manipulation type world world models