Zero-shot Inexact CAD Model Alignment from a Single Image

One practical approach to infer 3D scene structure from a single image is to retrieve a closely matching 3D model from a database and align it with the object in the image. Existing retrieve-and-alignmethods rely on supervised training with images and pose annotations, which limits them to a narrow set of object categories. To address this, we propose an unsupervised 9-DoF alignment method for inexact 3D models that requires no pose annotations and generalizes to unseen categories. Our approach derives a novel feature space based on foundation features that ensure multi-view consistency and overcome symmetry ambiguities inherent in foundation features using a self-supervised triplet loss.Additionally, we introduce a texture-invariant pose refinement technique that performs dense alignment in normalized object coordinates, estimated through the enhanced feature space.We conduct extensive evaluations on the real-world ScanNet25k dataset, where our method outperforms SOTA unsupervised baselines by +4.3% mean alignment accuracy and is the only unsupervised approach to surpass the supervised ROCA by +2.7%.To assess generalization, we introduce SUN2CAD, a real-world test set with 20 novel object categories, where our method achieves SOTA results without prior training on them.