HOI-Diff: Text-Driven Synthesis of 3D Human-Object Interactions using Diffusion Models

arXiv:2312.06553v2 Announce Type: replace
Abstract: We address the problem of generating realistic 3D human-object interactions (HOIs) driven by textual prompts. To this end, we take a modular design and decompose the complex task into simpler sub-tasks. We first develop a dual-branch diffusion model (HOI-DM) to generate both human and object motions conditioned on the input text, and encourage coherent motions by a cross-attention communication module between the human and object motion generation branches. We also develop an affordance prediction diffusion …

abstract arxiv cs.cv design diff diffusion diffusion model diffusion models generate human interactions modular object prompts synthesis tasks text textual type