RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation

arXiv:2402.14623v1 Announce Type: cross
Abstract: Rapid progress in high-level task planning and code generation for open-world robot manipulation has been witnessed in Embodied AI. However, previous studies put much effort into general common sense reasoning and task planning capabilities of large-scale language or multi-modal models, relatively little effort on ensuring the deployability of generated code on real robots, and other fundamental components of autonomous robot systems including robot perception, motion planning, and control. To bridge this ``ideal-to-real'' gap, this paper …

abstract arxiv capabilities code code generation common sense cs.ai cs.cl cs.cv cs.ro embodied embodied ai form free general language manipulation modal multi-modal open-world planning progress reasoning robot robot manipulation scale sense simulation studies tasks type world