Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representations

arXiv:2404.17521v1 Announce Type: cross
Abstract: Autonomous robotic systems capable of learning novel manipulation tasks are poised to transform industries from manufacturing to service automation. However, modern methods (e.g., VIP and R3M) still face significant hurdles, notably the domain gap among robotic embodiments and the sparsity of successful task executions within specific action spaces, resulting in misaligned and ambiguous task representations. We introduce Ag2Manip (Agent-Agnostic representations for Manipulation), a framework aimed at surmounting these challenges through two key innovations: a novel …

abstract agent arxiv automation autonomous cs.cv cs.ro domain face gap however industries manipulation manufacturing modern novel robotic service skills sparsity systems tasks type visual