ChatHuman: Language-driven 3D Human Understanding with Retrieval-Augmented Tool Reasoning

arXiv:2405.04533v1 Announce Type: cross
Abstract: Numerous methods have been proposed to detect, estimate, and analyze properties of people in images, including the estimation of 3D pose, shape, contact, human-object interaction, emotion, and more. Each of these methods works in isolation instead of synergistically. Here we address this problem and build a language-driven human understanding system -- ChatHuman, which combines and integrates the skills of many different methods. To do so, we finetune a Large Language Model (LLM) to select and …

abstract analyze arxiv build cs.cv cs.lg emotion human images language object people reasoning retrieval retrieval-augmented tool type understanding