Frankenstein: Generating Semantic-Compositional 3D Scenes in One Tri-Plane

arXiv:2403.16210v1 Announce Type: new
Abstract: We present Frankenstein, a diffusion-based framework that can generate semantic-compositional 3D scenes in a single pass. Unlike existing methods that output a single, unified 3D shape, Frankenstein simultaneously generates multiple separated shapes, each corresponding to a semantically meaningful part. The 3D scene information is encoded in one single tri-plane tensor, from which multiple Singed Distance Function (SDF) fields can be decoded to represent the compositional shapes. During training, an auto-encoder compresses tri-planes into a latent …

3d scenes abstract arxiv cs.ai cs.cv cs.gr diffusion framework generate information multiple part plane semantic type