Pseudo-labeling is a key technique of semi-supervised and cross-domian semantic segmentation, yet its efficacy is often hampered by the intrinsic noise of pseudo-labels. This study introduces Pseudo-SD, a novel framework that redefines the utilization of pseudo-label knowledge through Stable Diffusion (SD). Our Pseudo-SD innovatively combines pseudo-labels and its text prompts to fine-tune SD models, facilitating the generation of high-quality, diverse synthetic images that closely mimic target data characteristics. Within this framework, two novel mechanisms, \textit{i.e.}, partial attention manipulation, and structured pseudo-labeling, are proposed to effectively spread text-to-image corresponding during SD fine-tuning process and to ensure controllable high-quality image synthesis respectively. Extensive results demonstrate that Pseudo-SD significantly improves the performance on semi-supervised and cross-domain segmentation scenarios. Moreover, our method is versatile and model-agnostic, which can complement existing methods. By injecting our Pseudo-SD into current methods, we establish new state of the arts in different datasets, offering a new way for the exploration of effective pseudo-label utilization.