Hierarchical Spatial Proximity Reasoning for Vision-and-Language Navigation

arXiv:2403.11541v1 Announce Type: new
Abstract: Most Vision-and-Language Navigation (VLN) algorithms tend to make decision errors, primarily due to a lack of visual common sense and insufficient reasoning capabilities. To address this issue, this paper proposes a Hierarchical Spatial Proximity Reasoning (HSPR) model. Firstly, we design a Scene Understanding Auxiliary Task (SUAT) to assist the agent in constructing a knowledge base of hierarchical spatial proximity for reasoning navigation. Specifically, this task utilizes panoramic views and object features to identify regions in …

abstract algorithms arxiv capabilities common sense cs.cv decision design errors hierarchical issue language navigation paper reasoning sense spatial type understanding vision vision-and-language visual