Toward Better Out-painting: Improving the Image Composition with Initialization Policy Model

With its extensive applications, Foreground Conditioned Out-painting (FCO) has attracted considerable attention in the research field. Through the utilization of text-driven FCO, users are enabled to generate diverse backgrounds for a given foreground by adjusting the text prompt, which considerably enhances the efficiency in fields like e-commerce. Since the foreground is fixed in FCO, a key concern is whether the generated background can match the foreground well to achieve a coherent composition. However, most existing methods are lacking in this regard. Artifacts and incorrect interactions are common defects in synthesized images. This issue is linked to the influence of the initial noise in the sampling process. As the initial noise is sampled independently, it's highly likely that the implied image composition will conflict with the given foreground. In this paper, a novel Initialization Policy Model (IPM) is proposed to address this problem. Its function is to replace the early denoising steps and directly predict the intermediate state that is conducive to the reasonable image composition. Since the IPM is designed to take only the foreground image and the text prompt as inputs, it isolates the impact of the initial noise. The subsequently proposed training paradigm that combines inversion-derived label supervision and diffusion reward supervision further ensures the efficient training of the IPM. The evaluation is conducted using the task-specific OpenImage-FCO dataset developed by us. The results verify that the introduction of the IPM can significantly improve the composition of the synthesized images and achieve advanced performance in the FCO task.