Towards Temporally Consistent Referring Video Object Segmentation

arXiv:2403.19407v1 Announce Type: new
Abstract: Referring Video Object Segmentation (R-VOS) methods face challenges in maintaining consistent object segmentation due to temporal context variability and the presence of other visually similar objects. We propose an end-to-end R-VOS paradigm that explicitly models temporal instance consistency alongside the referring segmentation. Specifically, we introduce a novel hybrid memory that facilitates inter-frame collaboration for robust spatio-temporal matching and propagation. Features of frames with automatically generated high-quality reference masks are propagated to segment the remaining frames …

abstract arxiv challenges consistent context cs.cv face hybrid instance novel object objects paradigm segmentation temporal type video