Semi-supervised Deep Transfer for Regression without Domain Alignment

Deep learning models are seldom deployed widely for real-world applications (e.g., medicine), because source models do not generalize well to \``domain-shifted'' target data. Many successful domain adaptation approaches require full access to source data and reliably labeled target data. Yet, such requirements are unrealistic in scenarios where source data cannot be shared either because of privacy concerns or are too large, and incur prohibitive storage or computation costs. Moreover, resource constraints may limit the availability of labeled targets. We illustrate this challenge in a neuroscience setting where source data are unavailable, labeled target data are meager, and predictions involve continuous-valued outputs. We build upon Contradistinguisher (CUDA), an efficient framework that learns a shared model across the labeled source and unlabeled target samples, without intermediate alignment of representations. Yet, CUDA was designed for unsupervised DA, with full access to source data and for classification tasks. We develop CRAFT -- a CUDA-based Regularization Approach for Flexible Training -- for source-free (SF), semi-supervised transfer of pretrained models in regression tasks. We showcase the efficacy of CRAFT in two important neuroscience settings: gaze prediction with electroencephalography (EEG) data and ``brain age'' prediction with structural MRI data. For both datasets, CRAFT yielded up to $9\\%$ improvement in root-mean-squared error (RMSE) over finetuned models when labeled training examples were scarce. CRAFT leveraged unlabeled target data and outperformed four competing state-of-the-art source-free domain adaptation models by up to $4\\%$. We propose CRAFT as an efficient approach for source-free, semi-supervised deep transfer for regression that is ubiquitous in biology and medicine.