Skip to yearly menu bar Skip to main content


Poster

CMAD: Correlation-Aware and Modalities-Aware Distillation for Multimodal Sentiment Analysis with Missing Modalities

Yan Zhuang · Minhao Liu · Wei Bai · Yanru Zhang · Xiaoyue Zhang · Jiawen Deng · Fuji Ren


Abstract:

Multimodal Sentiment Analysis (MSA) enhances emotion recognition by integrating information from multiple modalities. However, multimodal learning with missing modalities suffers from representation inconsistency and optimization instability, leading to suboptimal performance. In this paper, we introduce Correlation-Aware and Modalities-Aware Distillation (CMAD), a unified framework designed for MSA under varying missing-modality conditions. Specifically, CMAD comprises two key components: (1) Correlation-Aware Feature Distillation (CAFD), which enforces multi-level representation alignment by preserving both feature similarities and high-order correlation structures between teacher and student models, and (2) Modality-Aware Regularization (MAR) employs an adaptive weighting strategy guided by modality difficulty, enabling a curriculum learning paradigm to stabilize the training process. Extensive evaluations on five datasets show that CMAD consistently outperforms existing methods, achieving average performance improvements of 1.0\% on MOSEI, 4.4\% on IEMOCAP, 1.9\% on MUStARD, 0.5\% on UR-FUNNY and 1.9\% on CHERMA.

Live content is unavailable. Log in and register to view live content