Text-driven Affordance Learning from Egocentric Vision

arXiv:2404.02523v1 Announce Type: new
Abstract: Visual affordance learning is a key component for robots to understand how to interact with objects. Conventional approaches in this field rely on pre-defined objects and actions, falling short of capturing diverse interactions in realworld scenarios. The key idea of our approach is employing textual instruction, targeting various affordances for a wide range of objects. This approach covers both hand-object and tool-object interactions. We introduce text-driven affordance learning, aiming to learn contact points and manipulation …

abstract arxiv cs.ai cs.cv diverse interactions key objects robots targeting text textual the key type vision visual