Abstract: We propose VIBE, a model-agnostic framework that trains classifiers resilient to backdoor attacks.The key concept behind our approachis to treat malicious inputs and corrupted labels from the training dataset as observed random variables,while the actual clean labelsare latent.VIBE then recovers the corresponding latent clean label posteriorthrough variational inference. The resulting training procedure follows the expectation-maximization (EM) algorithm.The E-step infers the clean pseudolabels by solvingan entropy-regularized optimal transport problem,while the M-step updates the classifier parameters via gradient descent.Being modular,VIBE can seamlessly integratewith recent advancements in self-supervised representation learning,which enhance its ability to resist backdoor attacks.We experimentally validate the method effectiveness against contemporary backdoor attacks on standard datasets, a large-scale setup with 1$k$ classes,and a dataset poisoned with multiple attacks.VIBE consistently outperforms previous defenses across all tested scenarios.