EventBind: Learning a Unified Representation to Bind Them All for Event-based Open-world Understanding

arXiv:2308.03135v3 Announce Type: replace
Abstract: In this paper, we propose EventBind, a novel and effective framework that unleashes the potential of vision-language models (VLMs) for event-based recognition to compensate for the lack of large-scale event-based datasets. In particular, due to the distinct modality gap with the image-text data and the lack of large-scale datasets, learning a common representation space for images, texts, and events is non-trivial.Intuitively, we need to address two key challenges: 1) how to generalize CLIP's visual encoder …

abstract arxiv cs.cv datasets event framework gap image language language models novel open-world paper recognition representation scale text them type understanding vision vision-language models vlms world