Semantics Meets Temporal Correspondence: Self-supervised Object-centric Learning in Videos

arXiv:2308.09951v2 Announce Type: replace
Abstract: Self-supervised methods have shown remarkable progress in learning high-level semantics and low-level temporal correspondence. Building on these results, we take one step further and explore the possibility of integrating these two features to enhance object-centric representations. Our preliminary experiments indicate that query slot attention can extract different semantic components from the RGB feature map, while random sampling based slot attention can exploit temporal correspondence cues between frames to assist instance identification. Motivated by this, we …

arxiv cs.cv object semantics temporal type videos