Constrained Robotic Navigation on Preferred Terrains Using LLMs and Speech Instruction: Exploiting the Power of Adverbs

arXiv:2404.02294v1 Announce Type: cross
Abstract: This paper explores leveraging large language models for map-free off-road navigation using generative AI, reducing the need for traditional data collection and annotation. We propose a method where a robot receives verbal instructions, converted to text through Whisper, and a large language model (LLM) model extracts landmarks, preferred terrains, and crucial adverbs translated into speed settings for constrained navigation. A language-driven semantic segmentation model generates text-based masks for identifying landmarks and terrain types in images. …

abstract annotation arxiv collection cs.lg cs.ro data data collection free generative language language models large language large language models llms map navigation paper power robot robotic speech text through type verbal whisper