DENOISER: Rethinking the Robustness for Open-Vocabulary Action Recognition

arXiv:2404.14890v1 Announce Type: new
Abstract: As one of the fundamental video tasks in computer vision, Open-Vocabulary Action Recognition (OVAR) recently gains increasing attention, with the development of vision-language pre-trainings. To enable generalization of arbitrary classes, existing methods treat class labels as text descriptions, then formulate OVAR as evaluating embedding similarity between visual samples and textual classes. However, one crucial issue is completely ignored: the class descriptions given by users may be noisy, e.g., misspellings and typos, limiting the real-world practicality …

abstract action recognition arxiv attention class computer computer vision cs.cv development embedding fundamental labels language recognition robustness tasks text type video vision vision-language visual