UnSAMFlow: Unsupervised Optical Flow Guided by Segment Anything Model

arXiv:2405.02608v1 Announce Type: new
Abstract: Traditional unsupervised optical flow methods are vulnerable to occlusions and motion boundaries due to lack of object-level information. Therefore, we propose UnSAMFlow, an unsupervised flow network that also leverages object information from the latest foundation model Segment Anything Model (SAM). We first include a self-supervised semantic augmentation module tailored to SAM masks. We also analyze the poor gradient landscapes of traditional smoothness losses and propose a new smoothness definition based on homography instead. A simple …

abstract arxiv cs.ai cs.cv cs.ro flow foundation foundation model information latest network object optical optical flow sam segment segment anything segment anything model semantic type unsupervised vulnerable