REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignment

Traditional image-to-3D models often struggle with scenes containing multipleobjects due to biases and occlusion complexities. To address this challenge, wepresent REPARO, a novel approach for compositional 3D asset generation fromsingle images. REPARO employs a two-step process: first, it extracts individualobjects from the scene and reconstructs their 3D meshes using off-the-shelf imageto-3D models; then, it optimizes the layout of these meshes through differentiablerendering techniques, ensuring coherent scene composition. By integrating optimaltransport-based long-range appearance loss term and high-level semantic loss termin the differentiable rendering, REPARO can effectively recover the layout of 3Dassets. The proposed method can significantly enhance object independence, detailaccuracy, and overall scene coherence. Extensive evaluation of multi-object scenesdemonstrates that our REPARO offers a comprehensive approach to address thecomplexities of multi-object 3D scene generation from single images.