NuiScene: Exploring Efficient Generation of Unbounded Outdoor Scenes

In this paper, we explore the task of generating expansive outdoor scenes, ranging from city skyscrapers to medieval castles and houses. Unlike indoor scene generation, which has been a primary focus of prior work, outdoor scene generation presents unique challenges, including the wide variation in scene heights and the need for an efficient approach capable of rapidly producing large landscapes. To address this, we introduce an efficient representation that encodes scene chunks as homogeneous vector sets, offering better compression than spatially structured latents used in prior methods. Furthermore, we train an outpainting model under four conditional patterns to generate scene chunks in a zig-zag manner, enabling more coherent generation compared to prior work that relies on inpainting methods. This provides richer context and speeds up generation by eliminating extra diffusion steps. Finally, to facilitate this task, we curate NuiScene43, a small but high-quality set of scenes and preprocess them for joint training. Interestingly, when trained on scenes of varying styles, our model can blend vastly different scenes, such as rural houses and city skyscrapers, within the same scene.