OpenESS: Event-based Semantic Scene Understanding with Open Vocabularies

arXiv:2405.05259v1 Announce Type: new
Abstract: Event-based semantic segmentation (ESS) is a fundamental yet challenging task for event camera sensing. The difficulties in interpreting and annotating event data limit its scalability. While domain adaptation from images to event data can help to mitigate this issue, there exist data representational differences that require additional effort to resolve. In this work, for the first time, we synergize information from image, text, and event-data domains and introduce OpenESS to enable scalable ESS in an …

abstract arxiv cs.cv cs.ro data differences domain domain adaptation event fundamental images issue scalability segmentation semantic sensing type understanding while