Pose Priors from Language Models

arXiv:2405.03689v1 Announce Type: new
Abstract: We present a zero-shot pose optimization method that enforces accurate physical contact constraints when estimating the 3D pose of humans. Our central insight is that since language is often used to describe physical interaction, large pretrained text-based models can act as priors on pose estimation.
We can thus leverage this insight to improve pose estimation by converting natural language descriptors, generated by a large multimodal model (LMM), into tractable losses to constrain the 3D pose …

abstract act arxiv constraints cs.cl cs.cv humans insight language language models optimization text type zero-shot