Generalized Few-Shot Point Cloud Segmentation via LLM-Assisted Hyper-Relation Matching

Generalized few-shot point cloud segmentation (GFS-3DSeg) aims to segment objects of both base and novel classes using abundant base class samples and limited novel class samples. Existing GFS-3DSeg methods encounter bottlenecks due to the scarcity of novel class data and inter-class confusion. In this paper, we propose the LLM-Assisted Hyper-Relation Matching (LARM) framework, which leverages the wealth of prior knowledge in LLM to enrich novel category prototypes and introduces a hyper-relation matching strategy to mitigate false matches between point features and category prototypes caused by inter-class confusion. The proposed LARM enjoys several merits. First, the vast knowledge embedded in LLM can be an effective complement to vanilla category prototypes, enabling them to exhibit greater robustness. Second, the hyper-relation matching strategy harnesses the structure information implicit in the inter-class relationships, making it more robust than comparing individually.Extensive experiments on two benchmarks demonstrate that LARM outperforms previous state-of-the-art methods by large margins. The code will be open-sourced for further research.