PacGDC: Label-Efficient Generalizable Depth Completion with Projection Ambiguity and Consistency

Generalizable depth completion enables the acquisition of dense metric depth maps for unseen environments, offering robust perception capabilities for various downstream tasks. However, training such models typically requires large-scale datasets with metric depth labels, which are often labor-intensive to collect. This paper presents PacGDC, a label-efficient technique that enhances geometry diversity with minimal annotation effort for generalizable depth completion. PacGDC builds on insights into inherent 2D-to-3D projection ambiguities and consistencies in object shapes and positions, allowing the synthesis of numerous pseudo geometries for the same visual scene. This process greatly broadens data coverage by manipulating scene scales of the corresponding depth maps. To leverage this property, we propose a novel data synthesis pipeline built upon multiple depth foundation models. These models robustly provide pseudo depth labels with varied scene scales in both local objects and global layouts, while ensuring projection consistency that contributes to generalization. To further diversify geometries, we introduce interpolation and relocation strategies, as well as unlabeled images, extending the coverage beyond the individual use of foundation models. Extensive experiments show that PacGDC achieves remarkable generalizability across multiple benchmarks, excelling in diverse scene semantics/scales and depth sparsity/patterns under both zero-shot and few-shot settings.