What does CLIP know about peeling a banana?

arXiv:2404.12015v1 Announce Type: new
Abstract: Humans show an innate capability to identify tools to support specific actions. The association between objects parts and the actions they facilitate is usually named affordance. Being able to segment objects parts depending on the tasks they afford is crucial to enable intelligent robots to use objects of daily living. Traditional supervised learning methods for affordance segmentation require costly pixel-level annotations, while weakly supervised approaches, though less demanding, still rely on object-interaction examples and support …

abstract arxiv association capability clip cs.cv humans identify intelligent objects robots segment show support tasks tools type