OW-VISCap: Open-World Video Instance Segmentation and Captioning

arXiv:2404.03657v1 Announce Type: new
Abstract: Open-world video instance segmentation is an important video understanding task. Yet most methods either operate in a closed-world setting, require an additional user-input, or use classic region-based proposals to identify never before seen objects. Further, these methods only assign a one-word label to detected objects, and don't generate rich object-centric descriptions. They also often suffer from highly overlapping predictions. To address these issues, we propose Open-World Video Instance Segmentation and Captioning (OW-VISCap), an approach to …

abstract arxiv captioning cs.ai cs.cv generate identify instance objects open-world proposals segmentation type understanding video video instance segmentation video understanding word world