Skip to yearly menu bar Skip to main content


Poster

Long Context Tuning for Video Generation

Yuwei Guo · Ceyuan Yang · Ziyan Yang · Zhibei Ma · Zhijie Lin · Zhenheng Yang · Dahua Lin · Lu Jiang


Abstract:

Recent advances in video generation can produce realistic, minute-long single-shot videos with scalable diffusion transformers. However, real-world narrative videos require multi-shot scenes with visual and semantic consistency across shots. In this work, we introduce Long Context Tuning (LCT), a training paradigm that extends the context window of pre-trained single-shot video diffusion models to learn scene-level consistency directly from data. Our method expands full attention mechanisms from individual shots to encompass all shots within a scene, incorporating interleaved 3D position embedding and an asynchronous noise strategy, enabling both joint and auto-regressive shot generation without additional parameters. Models with bidirectional attention after LCT can further be fine-tuned with context-causal attention, facilitating auto-regressive generation with efficient KV-cache. Experiments demonstrate single-shot models after LCT can produce coherent multi-shot scenes and exhibit emerging capabilities, including composable generation and interactive shot extension, paving the way for more practical visual content creation.

Live content is unavailable. Log in and register to view live content