Beyond Literal Descriptions: Understanding and Locating Open-World Objects Aligned with Human Intentions

arXiv:2402.11265v1 Announce Type: new
Abstract: Visual grounding (VG) aims at locating the foreground entities that match the given natural language expression. Previous datasets and methods for classic VG task mainly rely on the prior assumption that the given expression must literally refer to the target object, which greatly impedes the practical deployment of agents in real-world scenarios. Since users usually prefer to provide the intention-based expressions for the desired object instead of covering all the details, it is necessary for …

abstract arxiv beyond cs.cv datasets human language match natural natural language objects open-world prior type understanding visual world