On the Robustness of Language Guidance for Low-Level Vision Tasks: Findings from Depth Estimation

arXiv:2404.08540v1 Announce Type: new
Abstract: Recent advances in monocular depth estimation have been made by incorporating natural language as additional guidance. Although yielding impressive results, the impact of the language prior, particularly in terms of generalization and robustness, remains unexplored. In this paper, we address this gap by quantifying the impact of this prior and introduce methods to benchmark its effectiveness across various settings. We generate "low-level" sentences that convey object-centric, three-dimensional spatial relationships, incorporate them as additional language priors …

abstract advances arxiv cs.cv guidance impact language low natural natural language paper prior results robustness tasks terms type vision