Exploring Unseen Environments with Robots using Large Language and Vision Models through a Procedurally Generated 3D Scene Representation

arXiv:2404.00318v1 Announce Type: cross
Abstract: Recent advancements in Generative Artificial Intelligence, particularly in the realm of Large Language Models (LLMs) and Large Vision Language Models (LVLMs), have enabled the prospect of leveraging cognitive planners within robotic systems. This work focuses on solving the object goal navigation problem by mimicking human cognition to attend, perceive and store task specific information and generate plans with the same. We introduce a comprehensive framework capable of exploring an unfamiliar environment in search of an …

abstract artificial artificial intelligence arxiv cognitive cs.cv cs.ro environments generated generative generative artificial intelligence intelligence language language models large language large language models llms realm representation robotic robots systems through type vision vision models work