Three cumulative dtype-boundary divergences caused FV parity tests to fail after the Wave 14 channel-major fix exposed real-signal processing (vs the prior garbage-in-garbage-out kernel cancellation): 1. MagiAttention forward: hardcode bf16 for SDPA q/k/v inputs to match upstream flash_attn_with_cp's hardcoded bf16 cast at dit_module.py:508. Upcast attention output to fp32 before the per-head gating multiply to match upstream's bf16*fp32 promotion at dit_module.py:649. Cast back to bf16 only for linear_proj. This is an intentional exception to Wave 10's dtype-agnostic pattern: upstream is not dtype-agnostic at this boundary. 2. MagiHumanDiT forward: drop the x.to(linear_qkv.weight.dtype) cast that ran the residual stream in bf16 across all 40 layers. Upstream casts to params_dtype which defaults to fp32, keeping the cross-layer accumulator fp32 with bf16 internal compute. The bf16 residual cast was compounding ~6-7 bits of mantissa loss per layer and was visible in pipeline parity diff. 3. Parity test _build_fastvideo_schedulers: switch to single-shift construction (default shift=1 in __init__, then set_timesteps with the real shift) to match production migration in Wave 11. The stale double-shift was applying a non-trivial shift twice versus upstream's single-shift, leaving FV with a different timestep schedule. Result: all 7 magi-human local parity tests pass with bit-exact or near-bit-exact (VAE 8e-4 max) outputs vs upstream: test_magi_human_dit_parity diff=0.0 test_magi_human_t5gemma_parity diff=0.0 test_magi_human_sa_audio_parity diff=0.0 test_magi_human_sa_audio_official_parity diff=0.0 test_magi_human_vae_parity max=8e-4 test_magi_human_pipeline_smoke passes test_magi_human_pipeline_latent_parity diff=0.0 Production E2E re-validated: examples/inference/basic/basic_magi_human.py still produces coherent video at the standard 480x256 / 32-step / seed-42 prompt, with unchanged 23.5s runtime. Closes OQ-6 (full resolution including dtype boundaries).
| Documentation | Quick Start | Weekly Dev Meeting | 🟣💬 Slack | 🟣💬 WeChat |
FastVideo is a unified post-training and real-time inference framework for accelerated video generation.
NEWS
2026/03/17: Release Live demo: Into the Dreamverse: Vibe Directing in FastVideo, check out the Blog.2026/03/13: Release Live demo: Create a 5s 1080p Video in 4.5s with FastVideo on a Single GPU, check out the Blog.2025/11/19: Release CausalWan2.2 I2V A14B Preview models, Blog and Inference Code!.2025/08/04: Release FastWan models and Sparse-Distillation.
More News
2025/06/14: Release finetuning and inference code for VSA.2025/04/24: FastVideo V1 is released!2025/02/18: Release the inference code for Sliding Tile Attention.
Key Features
FastVideo has the following features:
- End-to-end post-training support for bidirectional and autoregressive models:
- Support full finetuning and LoRA finetuning for state-of-the-art open video DiTs
- Data preprocessing pipeline for video, image, and text data
- Distribution Matching Distillation (DMD2) stepwise distillation.
- Sparse attention with Video Sparse Attention
- Sparse distillation to achieve >50x denoising speedup
- Scalable training with FSDP2, sequence parallelism, and selective activation checkpointing.
- Causal distillation through Self-Forcing
- See this page for full list of supported models and recipes.
- State-of-the-art performance optimizations for inference
- Sequence Parallelism for distributed inference
- Multiple state-of-the-art attention backends
- User-friendly CLI and Python API
- See this page for full list of supported optimizations.
- Diverse hardware and OS support
- Support H100, A100, 4090
- Support Linux, Windows, MacOS
- See this page for full list of supported models, hardware assumptions, and optimization compatibility.
Getting Started
We recommend using uv to create a clean environment. If you previously used Conda, switching to uv generally gives faster and more stable installs.
# Create and activate a new uv environment
uv venv --python 3.12 --seed
source .venv/bin/activate
# Install FastVideo
uv pip install fastvideo
Please see our docs for more detailed installation instructions.
Sparse Distillation
For our sparse distillation techniques, please see our distillation docs and check out our blog.
See below for recipes and datasets:
| Model | Sparse Distillation | Dataset |
|---|---|---|
| FastWan2.1-T2V-1.3B | Recipe | FastVideo Synthetic Wan2.1 480P |
| FastWan2.2-TI2V-5B | Recipe | FastVideo Synthetic Wan2.2 720P |
Inference
Generating Your First Video
Here's a minimal example to generate a video using the default settings. Make sure VSA kernels are installed. Create a file called example.py with the following code:
import os
from fastvideo import VideoGenerator
def main():
os.environ["FASTVIDEO_ATTENTION_BACKEND"] = "VIDEO_SPARSE_ATTN"
# Create a video generator with a pre-trained model
generator = VideoGenerator.from_pretrained(
"FastVideo/FastWan2.1-T2V-1.3B-Diffusers",
num_gpus=1, # Adjust based on your hardware
)
# Define a prompt for your video
prompt = "A curious raccoon peers through a vibrant field of yellow sunflowers, its eyes wide with interest."
# Generate the video
video = generator.generate_video(
prompt,
output_path="my_videos/", # Controls where videos are saved
save_video=True
)
if __name__ == '__main__':
main()
Run the script with:
python example.py
For a more detailed guide, please see our inference quick start.
More Guides
Awesome work using FastVideo or our research projects
- SGLang: SGLang's diffusion inference functionality is based on a fork of FastVideo on Sept. 24, 2025.
- DanceGRPO: A unified framework to adapt Group Relative Policy Optimization (GRPO) to visual generation paradigms. Code based on FastVideo.
- SRPO: A method to directly align the full diffusion trajectory with fine-grained human preference. Code based on FastVideo.
- DCM: Dual-expert consistency model for efficient and high-quality video generation. Code based on FastVideo.
- HY-WorldPlay: An action-conditioned world model model trained using FastVideo framework.
- Hunyuan Video 1.5: A leading lightweight video generation model, where they proposed SSTA based on Sliding Tile Attention.
- Kandinsky-5.0: A family of diffusion models for video & image generation, where their NABLA attention includes a Sliding Tile Attention branch.
- LongCat Video: A foundational video generation model with 13.6B parameters with block-sparse attention similar to Video Sparse Attention.
🤝 Contributing
We welcome all contributions. Please check out our guide here. See details in development roadmap.
Acknowledgement
We learned the design and reused code from the following projects: Wan-Video, ThunderKittens, DMD2, diffusers, xDiT, vLLM, SGLang. We thank MBZUAI, Anyscale, and GMI Cloud for their support throughout this project.
Citation
If you find FastVideo useful, please consider citing our research work:
@article{zhang2025vsa,
title={Vsa: Faster video diffusion with trainable sparse attention},
author={Zhang, Peiyuan and Chen, Yongqi and Huang, Haofeng and Lin, Will and Liu, Zhengzhong and Stoica, Ion and Xing, Eric and Zhang, Hao},
journal={arXiv preprint arXiv:2505.13389},
year={2025}
}
@article{zhang2025fast,
title={Fast video generation with sliding tile attention},
author={Zhang, Peiyuan and Chen, Yongqi and Su, Runlong and Ding, Hangliang and Stoica, Ion and Liu, Zhengzhong and Zhang, Hao},
journal={arXiv preprint arXiv:2502.04507},
year={2025}
}