Context-Aware Integration of Language and Visual References for Natural Language Tracking

arXiv:2403.19975v1 Announce Type: new
Abstract: Tracking by natural language specification (TNL) aims to consistently localize a target in a video sequence given a linguistic description in the initial frame. Existing methodologies perform language-based and template-based matching for target reasoning separately and merge the matching results from two sources, which suffer from tracking drift when language and visual templates miss-align with the dynamic target state and ambiguity in the later merging stage. To tackle the issues, we propose a joint multi-modal …

abstract arxiv context cs.cv integration language merge natural natural language reasoning results template tracking type video visual