Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondence

arXiv:2403.11120v1 Announce Type: new
Abstract: This paper introduces a Transformer-based integrative feature and cost aggregation network designed for dense matching tasks. In the context of dense matching, many works benefit from one of two forms of aggregation: feature aggregation, which pertains to the alignment of similar features, or cost aggregation, a procedure aimed at instilling coherence in the flow estimates across neighboring pixels. In this work, we first show that feature aggregation and cost aggregation exhibit distinct characteristics and reveal …

abstract aggregation alignment arxiv benefit context cost cs.cv feature features forms network paper semantic tasks transformer transformers type visual