Robust 3D-Masked Part-level Editing in 3D Gaussian Splatting with Regularized Score Distillation Sampling

Due to limited 3D data, recent prior arts in 3D editing rely mainly on the Score Distillation Sampling (SDS) loss that edits and segments in 2D rendered views using pre-trained diffusion priors and then projects back onto 3D space to update the model. While these approaches are effective for 3D instance-level editing, they struggle with 3D part-level editing especially for Gaussian splatting due to inconsistent multi-view 2D part segmentations and inherently ambiguous SDS loss with localized nature of Gaussians. To address these limitations, we propose RoMaP, a novel local 3D Gaussian editing that enables drastic part-level changes. Firstly, we propose 3D-geometry aware label prediction (3D-GALP) exploiting the uncertainty in soft-label segmentations. Secondly, we propose a regularized SDS loss with masks that consists of a usual SDS loss with the predicted 3D mask and an L1 regularizer as an anchor loss for high-quality part-edited 2D images using our proposed scheduled latent mixing and part editing (SLaMP) method. Our SDS loss improves flexibility in local editing by removing 3D masked regions, allowing changes beyond existing context. SLaMP uses the projected 2D mask of the predicted 3D mask to confine modifications to the target region while preserving contextual coherence. Experimental results demonstrate that our RoMaP achieves state-of-the-art performance in local 3D editing for reconstructed and generated 3D Gaussian scenes and objects qualitatively and quantitatively, making it possible for more robust 3D-masked part-level editing in 3D Gaussian splatting.