Towards Deviation-Robust Agent Navigation via Perturbation-Aware Contrastive Learning

arXiv:2403.05770v1 Announce Type: new
Abstract: Vision-and-language navigation (VLN) asks an agent to follow a given language instruction to navigate through a real 3D environment. Despite significant advances, conventional VLN agents are trained typically under disturbance-free environments and may easily fail in real-world scenarios, since they are unaware of how to deal with various possible disturbances, such as sudden obstacles or human interruptions, which widely exist and may usually cause an unexpected route deviation. In this paper, we present a model-agnostic …

abstract advances agent agents arxiv cs.ai cs.cv cs.ro deal deviation environment environments free language navigation robust through type via vision vision-and-language world