Skip to yearly menu bar Skip to main content


Poster

Forensic-MoE: Exploring Comprehensive Synthetic Image Detection Traces with Mixture of Experts

Mingqi Fang · Ziguang Li · Lingyun Yu · Quanwei Yang · Hongtao Xie · Yongdong Zhang


Abstract:

Recently, synthetic images have evolved incredibly realistic with the development of generative techniques.To avoid the spread of misinformation and identify synthetic content, research on synthetic image detection becomes urgent. Unfortunately, limited to the singular forensic perspective, existing methods struggle to explore sufficient traces encountered with diverse synthetic techniques. In response to this, we argue that different synthetic images encompass a variety of forensic traces, and utilizing multiple experts to explore traces from diverse perspectives will be beneficial. Accordingly, a novel detector with the Mixture of multiple forensic Experts is proposed, named Forensic-MoE. To integrate multiple experts and enhance the knowledge interaction, Forensic-MoE follows an adapter-backbone architecture. Specifically, multiple adapters trained on different synthetic images serve as the trace exploration experts, and they are uniformly integrated into a pretrained backbone model to learn the detection prior and encourage the expert interaction. By guiding multiple experts to align with each other and collaborate together, Forensic-MoE can integrate comprehensive and discriminative detection traces from multiple perspectives. Moreover, for the discrimination improvement of each expert, a multi-stage structure is proposed for efficient trace perception, and a patch decentralization strategy is applied to encourage the model's attention on every local region. Extensive experiments demonstrate the superiority of our method, reflected in a 7.86% mean Acc advantage in comparison.

Live content is unavailable. Log in and register to view live content