DINO-Tracker: Taming DINO for Self-Supervised Point Tracking in a Single Video

arXiv:2403.14548v1 Announce Type: new
Abstract: We present DINO-Tracker -- a new framework for long-term dense tracking in video. The pillar of our approach is combining test-time training on a single video, with the powerful localized semantic features learned by a pre-trained DINO-ViT model. Specifically, our framework simultaneously adopts DINO's features to fit to the motion observations of the test video, while training a tracker that directly leverages the refined features. The entire framework is trained end-to-end using a combination of …

abstract arxiv cs.cv features framework long-term semantic test tracking training type video vit