ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulation

arXiv:2403.08321v1 Announce Type: cross
Abstract: Performing language-conditioned robotic manipulation tasks in unstructured environments is highly demanded for general intelligent robots. Conventional robotic manipulation methods usually learn semantic representation of the observation for action prediction, which ignores the scene-level spatiotemporal dynamics for human goal completion. In this paper, we propose a dynamic Gaussian Splatting method named ManiGaussian for multi-task robotic manipulation, which mines scene dynamics via future scene reconstruction. Specifically, we first formulate the dynamic Gaussian Splatting framework that infers the …

abstract arxiv cs.cv cs.ro dynamic dynamics environments general human intelligent language learn manipulation observation paper prediction representation robotic robotic manipulation robots semantic tasks type unstructured