Temporal-Spatial Object Relations Modeling for Vision-and-Language Navigation

arXiv:2403.15691v1 Announce Type: new
Abstract: Vision-and-Language Navigation (VLN) is a challenging task where an agent is required to navigate to a natural language described location via vision observations. The navigation abilities of the agent can be enhanced by the relations between objects, which are usually learned using internal objects or external datasets. The relationships between internal objects are modeled employing graph convolutional network (GCN) in traditional studies. However, GCN tends to be shallow, limiting its modeling ability. To address this …

abstract agent arxiv cs.cv datasets language location modeling natural natural language navigation object objects relations spatial temporal type via vision vision-and-language