We introduce ViCTr (Vital Consistency Transfer), a framework for advancing medical image synthesis through a principled integration with Rectified Flow trajectories. Unlike traditional approaches, we modify the Tweedie formulation to accommodate linear trajectories within the Rectified Flow framework, enabling more accurate initial state approximation and consistent trajectory paths. ViCTr’s design allows for precise control over anatomical accuracy and pathological attributes across CT and MRI modalities via a two-stage architecture. In Stage 1, it performs anatomical learning on the ATLAS-8k dataset using Elastic Weight Consolidation (EWC) to selectively train model weights tailored for medical data. In Stage 2, an adversarial fine-tuning strategy is applied: the base model from Stage 1 remains frozen while a LoRA adapter is exclusively applied to the weights tuned in Stage 1, allowing targeted adaptation for downstream tasks while preserving the core medical data properties learned during pretraining. ViCTr achieves notable improvements by utilizing segmentation maps and textual prompts to enable refined control over CT and MRI synthesis. Extensive experiments on benchmark datasets, including BTCV, AMOS, and CirrMRI600+, demonstrate ViCTr’s superiority, showing significant enhancements in quantitative metrics and clinical detail, such as liver surface nodularity in cirrhosis synthesis. These results establish ViCTr as a major advancement in medical image synthesis with impactful applications in data augmentation and clinical training.