$\text{R}^2$-Bench: Benchmarking the Robustness of Referring Perception Models under Perturbations

arXiv:2403.04924v1 Announce Type: new
Abstract: Referring perception, which aims at grounding visual objects with multimodal referring guidance, is essential for bridging the gap between humans, who provide instructions, and the environment where intelligent systems perceive. Despite progress in this field, the robustness of referring perception models (RPMs) against disruptive perturbations is not well explored. This work thoroughly assesses the resilience of RPMs against various perturbations in both general and specific contexts. Recognizing the complex nature of referring perception tasks, we …

abstract arxiv benchmarking cs.cv environment gap guidance humans intelligent intelligent systems multimodal objects perception progress robustness systems text the environment type visual