TINA: Think, Interaction, and Action Framework for Zero-Shot Vision Language Navigation

arXiv:2403.08833v1 Announce Type: new
Abstract: Zero-shot navigation is a critical challenge in Vision-Language Navigation (VLN) tasks, where the ability to adapt to unfamiliar instructions and to act in unknown environments is essential. Existing supervised learning-based models, trained using annotated data through reinforcement learning, exhibit limitations in generalization capabilities. Large Language Models (LLMs), with their extensive knowledge and emergent reasoning abilities, present a potential pathway for achieving zero-shot navigation. This paper presents a VLN agent based on LLMs, exploring approaches to …

abstract act adapt annotated data arxiv capabilities challenge cs.ai cs.cv data environments framework language limitations navigation reinforcement reinforcement learning supervised learning tasks think through type vision zero-shot