Exploring Interactive Semantic Alignment for Efficient HOI Detection with Vision-language Model

arXiv:2404.12678v1 Announce Type: new
Abstract: Human-Object Interaction (HOI) detection aims to localize human-object pairs and comprehend their interactions. Recently, two-stage transformer-based methods have demonstrated competitive performance. However, these methods frequently focus on object appearance features and ignore global contextual information. Besides, vision-language model CLIP which effectively aligns visual and text embeddings has shown great potential in zero-shot HOI detection. Based on the former facts, We introduce a novel HOI detector named ISA-HOI, which extensively leverages knowledge from CLIP, aligning interactive …

abstract alignment arxiv clip cs.cv detection features focus global however human information interactions interactive language language model object performance semantic stage transformer type vision vision-language visual