Ishan VaishandClaude Sonnet 5
7f7431779e
Wired up FastVideo/LingBot-World-Fast-Diffusers, the old LingBot-World repo's distilled causal I2V checkpoint that was never ported. Reused the existing LingBot World 2 causal-fast DiT unchanged (verified all 1421 parameter names and shapes against the released checkpoint) and added the arch/pipeline configs, registry entries, preset, pipeline subclass, example, and SSIM test needed to run it. Fixed a model-family detector collision where the LingBot World (Cam) detector would have wrongly claimed this checkpoint's path and model_index name, since it only excluded the "causal-fast" spelling. Fixed WanCrossAttention in the shared LingBot World 2 causal-fast DiT to route through the attention() dispatcher instead of calling flash_attention() directly, so the already-advertised TORCH_SDPA backend actually works when flash-attn isn't installed. Self-attention already used this dispatcher; cross-attention did not. Verified end to end: weights load (18.54B params, 16 shards) and generation produces a coherent video with correct prompt-following camera motion. Not yet verified against the original implementation's numerics (activation-trace parity) or via the SSIM regression test, both of which need hardware unavailable in this environment. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Basic Video Generation Tutorial
The VideoGenerator class provides the primary Python interface for doing offline video generation, which is interacting with a diffusion pipeline without using a separate inference api server.
Requirements
- At least a single NVIDIA GPU with CUDA 12.4.
- Python 3.10-3.12
Installation
If you have not installed FastVideo, please following these instructions first.
Usage
The first script in this example shows the most basic usage of FastVideo. If you are new to Python and FastVideo, you should start here.
# if you have not cloned the directory:
git clone https://github.com/hao-ai-lab/FastVideo.git && cd FastVideo
python examples/inference/basic/basic.py
For an example on Apple silicon:
python examples/inference/basic/basic_mps.py
For an example running DMD+VSA inference:
python examples/inference/basic/basic_dmd.py
For the typed config/request path added during the inference API refactor:
python examples/inference/basic/basic_dmd_new_api.py
Basic Walkthrough
All you need to generate videos using multi-gpus from state-of-the-art diffusion pipelines is the following few lines!
from fastvideo import VideoGenerator
def main():
generator = VideoGenerator.from_pretrained(
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
num_gpus=1,
)
prompt = ("A curious raccoon peers through a vibrant field of yellow sunflowers, its eyes "
"wide with interest. The playful yet serene atmosphere is complemented by soft "
"natural light filtering through the petals. Mid-shot, warm and cheerful tones.")
video = generator.generate_video(prompt)
if __name__ == "__main__":
main()