Semantics-aware Motion Retargeting with Vision-Language Models

arXiv:2312.01964v3 Announce Type: replace
Abstract: Capturing and preserving motion semantics is essential to motion retargeting between animation characters. However, most of the previous works neglect the semantic information or rely on human-designed joint-level representations. Here, we present a novel Semantics-aware Motion reTargeting (SMT) method with the advantage of vision-language models to extract and maintain meaningful motion semantics. We utilize a differentiable module to render 3D motions. Then the high-level motion semantics are incorporated into the motion retargeting process by feeding …

abstract animation arxiv characters cs.cv cs.gr extract however human information language language models novel semantic semantics type vision vision-language models