MaterialSeg3D: Segmenting Dense Materials from 2D Priors for 3D Assets

arXiv:2404.13923v1 Announce Type: new
Abstract: Driven by powerful image diffusion models, recent research has achieved the automatic creation of 3D objects from textual or visual guidance. By performing score distillation sampling (SDS) iteratively across different views, these methods succeed in lifting 2D generative prior to the 3D space. However, such a 2D generative image prior bakes the effect of illumination and shadow into the texture. As a result, material maps optimized by SDS inevitably involve spurious correlated components. The absence …

3d objects abstract arxiv cs.cv diffusion diffusion models distillation generative guidance however image image diffusion materials objects prior research sampling space textual type visual