DiffH2O: Diffusion-Based Synthesis of Hand-Object Interactions from Textual Descriptions

arXiv:2403.17827v1 Announce Type: cross
Abstract: Generating natural hand-object interactions in 3D is challenging as the resulting hand and object motions are expected to be physically plausible and semantically meaningful. Furthermore, generalization to unseen objects is hindered by the limited scale of available hand-object interaction datasets. We propose DiffH2O, a novel method to synthesize realistic, one or two-handed object interactions from provided text prompts and geometry of the object. The method introduces three techniques that enable effective learning from limited data. …

abstract arxiv cs.ai cs.cv cs.gr cs.lg datasets diffusion interactions natural novel object objects scale synthesis textual type