Enriching Phrases with Coupled Pixel and Object Contexts for Panoptic Narrative Grounding

arXiv:2311.01091v2 Announce Type: replace
Abstract: Panoptic narrative grounding (PNG) aims to segment things and stuff objects in an image described by noun phrases of a narrative caption. As a multimodal task, an essential aspect of PNG is the visual-linguistic interaction between image and caption. The previous two-stage method aggregates visual contexts from offline-generated mask proposals to phrase features, which tend to be noisy and fragmentary. The recent one-stage method aggregates only pixel contexts from image features to phrase features, which …

abstract arxiv cs.cv image multimodal narrative object objects pixel png segment stage type visual