Grounding Language Plans in Demonstrations Through Counterfactual Perturbations

arXiv:2403.17124v1 Announce Type: cross
Abstract: Grounding the common-sense reasoning of Large Language Models in physical domains remains a pivotal yet unsolved problem for embodied AI. Whereas prior works have focused on leveraging LLMs directly for planning in symbolic spaces, this work uses LLMs to guide the search of task structures and constraints implicit in multi-step demonstrations. Specifically, we borrow from manipulation planning literature the concept of mode families, which group robot configurations by specific motion constraints, to serve as an …

arxiv counterfactual cs.ai cs.cl cs.lg cs.ro language through type