lint

2025-08-04 11:53:02 -07:00
31 changed files with 38 additions and 93 deletions
@@ -7,7 +7,7 @@
 FastVideo features an end-to-end unified pipeline for accelerating diffusion models, starting from data preprocessing to model training, finetuning, distillation, and inference. FastVideo is designed to be modular and extensible, allowing users to easily add new optimizations and techniques. Whether it is training-free optimizations or post-training optimizations, FastVideo has you covered.

 <p align="center">
-    | <a href="https://hao-ai-lab.github.io/FastVideo"><b>Documentation</b></a> | <a href="https://hao-ai-lab.github.io/FastVideo/inference/inference_quick_start.html"><b> Quick Start</b></a> | 🤗 <a href="https://huggingface.co/FastVideo/FastWan2.1-T2V-1.3B-Diffusers"  target="_blank"><b>FastWan2.1</b></a>  | 🤗 <a href="https://huggingface.co/FastVideo/FastWan2.2-TI2V-5B-Diffusers" target="_blank"><b>FastWan2.2</b></a> | 🟣💬 <a href="https://join.slack.com/t/fastvideo/shared_invite/zt-38u6p1jqe-yDI1QJOCEnbtkLoaI5bjZQ" target="_blank"> <b>Slack</b> </a> |  🟣💬 <a href="https://ibb.co/qqPzbrw" target="_blank"> <b> WeChat </b> </a> |
+    | <a href="https://hao-ai-lab.github.io/FastVideo"><b>Documentation</b></a> | <a href="https://hao-ai-lab.github.io/FastVideo/inference/inference_quick_start.html"><b> Quick Start</b></a> | 🤗 <a href="https://huggingface.co/FastVideo/FastWan2.1-T2V-1.3B-Diffusers"  target="_blank"><b>FastWan2.1</b></a>  | 🤗 <a href="https://huggingface.co/FastVideo/FastWan2.2-TI2V-5B-Diffusers" target="_blank"><b>FastWan2.2</b></a> | 🟣💬 <a href="https://join.slack.com/t/fastvideo/shared_invite/zt-38u6p1jqe-yDI1QJOCEnbtkLoaI5bjZQ" target="_blank"> <b>Slack</b> </a> |
 </p>

 <div align="center">
@@ -151,10 +151,11 @@ If you find FastVideo useful, please considering citing our work:
  year         = {2024},
 }

-@article{zhang2025vsa,
-  title={VSA: Faster Video Diffusion with Trainable Sparse Attention},
-  author={Zhang, Peiyuan and Huang, Haofeng and Chen, Yongqi and Lin, Will and Liu, Zhengzhong and Stoica, Ion and Xing, Eric and Zhang, Hao},
-  journal={arXiv preprint arXiv:2505.13389},
+@article{zhang2025faster,
+  title={Faster video diffusion with trainable sparse attention},
+  author={Zhang, Peiyuan and Huang, Haofeng and Chen, Yongqi and Lin, Will and Liu, Zhengzhong and Stoica, Ion and Xing, Eric P and Zhang, Hao},
+  journal={arXiv e-prints},
+  pages={arXiv--2505},
  year={2025}
 }

@@ -6,9 +6,8 @@ We introduce a new finetuning strategy - **Sparse-distill**, which jointly integ

 We provide two distilled models:

- **[FastWan2.1-T2V-1.3B-Diffusers](https://huggingface.co/FastVideo/FastWan2.1-T2V-1.3B-Diffusers)**: 3-step inference, up to **16 FPS** on H100 GPU
- **[FastWan2.1-T2V-14B-480P-Diffusers](https://huggingface.co/FastVideo/FastWan2.1-T2V-14B-480P-Diffusers)**: 3-step inference, up to **60x speed up** at 480P, **90x speed up** at 720P for denoising loop
- **[FastWan2.2-TI2V-5B-FullAttn-Diffusers](https://huggingface.co/FastVideo/FastWan2.2-TI2V-5B-FullAttn-Diffusers)**: 3-step inference, up to **50x speed up** at 720P for denoising loop
+- **[FastWan2.1-T2V-1.3B-Diffusers](https://huggingface.co/FastVideo/FastWan2.1-T2V-1.3B-Diffusers)**: 3-step inference, up to **20 FPS** on H100 GPU
+- **[FastWan2.1-T2V-14B-480P-Diffusers](https://huggingface.co/FastVideo/FastWan2.1-T2V-14B-480P-Diffusers)**: 3-step inference, up to **50x speed up** at 480P, **70x speed up** at 720P for denoising loop

 Both models are trained on **61×448×832** resolution but support generating videos with **any resolution** (1.3B  model mainly support 480P, 14B model support 480P and 720P, quality may degrade for different resolutions).

@@ -35,7 +34,7 @@ python scripts/huggingface/download_hf.py \

 ## 🚀 Training Scripts

-### Wan2.1 1.3B Model Sparse-Distill
+### 1.3B Model Sparse-Distill

 For the 1.3B model, we use **4 nodes with 32 H200 GPUs** (8 GPUs per node):

@@ -51,7 +50,7 @@ sbatch examples/distill/Wan2.1-T2V/Wan-Syn-Data-480P/distill_dmd_VSA_t2v_1.3B.sl
 - VSA attention sparsity: 0.8
 - Training steps: 4000 (~12 hours)

-### Wan2.1 14B Model Sparse-Distill
+### 14B Model Sparse-Distill

 For the 14B model, we use **8 nodes with 64 H200 GPUs** (8 GPUs per node):

@@ -68,20 +67,3 @@ sbatch examples/distill/Wan2.1-T2V/Wan-Syn-Data-480P/distill_dmd_VSA_t2v_14B.slu
 - VSA attention sparsity: 0.9
 - Training steps: 3000 (~52 hours)
 - HSDP shard dim: 8
-
-### Wan2.2 5B Model Sparse-Distill
-
-For the 5B model, we use **8 nodes with 64 H200 GPUs** (8 GPUs per node):
-
-```bash
-# Multi-node training (8 nodes, 64 GPUs total)
-sbatch examples/distill/Wan2.2-TI2V-5B-Diffusers/Data-free/distill_dmd_t2v_5B.sh 
-```
-
-**Key Configuration:**
- Global batch size: 64
- Sequence parallel size: 1
- Gradient accumulation steps: 1
- Learning rate: 2e-5
- Training steps: 3000 (~12 hours)
- HSDP shard dim: 1
@@ -9,7 +9,7 @@

 :::{raw} html
 <p style="text-align:center">
-<strong>FastVideo is a unified inference and post-training framework for accelerated video generation.
+<strong>FastVideo is An unified inference and post-training framework for accelerated video generation.
 </strong>
 </p>

@@ -53,9 +53,3 @@ out = sliding_tile_attention(q, k, v, window_size, text_length)
 out = sliding_tile_attention(q, k, v, window_size, 0, False)

 ```
-
-# 🚀Inference
-
-```bash
-bash scripts/inference/v1_inference_wan_STA.sh
-```
@@ -59,9 +59,3 @@ from vsa import video_sparse_attn
 output = video_sparse_attn(q, k, v, variable_block_sizes, topk, block_size, compress_attn_weight)

 ``` 
-
-# 🚀Inference
-
-```bash
-bash scripts/inference/v1_inference_wan_VSA.sh
-```
@@ -102,6 +102,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.0
  --dit_precision "fp32"
@@ -102,6 +102,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.0
  --dit_precision "fp32"
@@ -102,6 +102,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.0
  --dit_precision "fp32"
@@ -1,13 +0,0 @@
-# Wan2.2-5B Distill Example
-These are end-to-end example scripts for distilling Wan2.2 TI2V 5B model DMD+VSA methods.
-
-### 0. Make sure you have installed VSA
-
-```bash
-cd csrc/attn
-git submodule update --init --recursive
-python setup_vsa.py install
-```
-
-### Data-free Distillation
-When `--simulate_generator_forward` is enabled, distillation becomes data-free by simulating intermediate steps through forward inference of the generator. This helps avoid training–inference mismatch. See Section 4.5 of [DMD2](https://arxiv.org/pdf/2405.14867) for details.
@@ -108,6 +108,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.0
  --dit_precision "fp32"
@@ -108,6 +108,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.0
  --dit_precision "fp32"
@@ -9,14 +9,4 @@ git submodule update --init --recursive
 python setup_vsa.py install
 ```

-### 1. Download dataset:
-```bash
-bash examples/distill/Wan2.2-TI2V-5B-Diffusers/crush_smol/download_dataset.sh
-```
-
-### 2. Configure and run distillation:
-
-#### For DMD-only distillation:
-```bash
-bash examples/distill/Wan2.2-TI2V-5B-Diffusers/crush_smol/examples/distill/Wan2.2-TI2V-5B-Diffusers/crush_smol/
-```
+### TODO
@@ -77,6 +77,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.0
  --dit_precision "fp32"
@@ -69,6 +69,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.1
  --multi_phased_distill_schedule "4000-1"
@@ -103,6 +103,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.1
  --multi_phased_distill_schedule "4000-1"
@@ -101,6 +101,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.1
  --dit_precision "fp32"
@@ -101,6 +101,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.1
  --dit_precision "fp32"
@@ -69,6 +69,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.1
  --multi_phased_distill_schedule "4000-1"
@@ -103,6 +103,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.1
  --multi_phased_distill_schedule "4000-1"
@@ -69,6 +69,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.1
  --multi_phased_distill_schedule "4000-1"
@@ -69,6 +69,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.1
  --multi_phased_distill_schedule "4000-1"
@@ -100,6 +100,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.1
  --multi_phased_distill_schedule "4000-1"
@@ -69,6 +69,7 @@ optimizer_args=(
 # Miscellaneous arguments
 miscellaneous_args=(
  --inference_mode False
+  --allow_tf32
  --checkpoints_total_limit 3
  --training_cfg_rate 0.1
  --multi_phased_distill_schedule "4000-1"
@@ -20,7 +20,7 @@ class SamplingParam:

    # Text inputs
    prompt: str | list[str] | None = None
-    negative_prompt: str = "Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured, misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards"
+    negative_prompt: str | None = None
    prompt_path: str | None = None
    output_path: str = "outputs/"
    output_video_name: str | None = None
@@ -1 +1 @@
-__version__ = "0.1.5"
+__version__ = "0.1.4"
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

 [project]
 name = "fastvideo"
-version = "0.1.5"
+version = "0.1.4"
 description = "FastVideo"
 readme = "README.md"
 requires-python = ">=3.10"
@@ -35,6 +35,7 @@ torchrun --nnodes 1 --nproc_per_node $NUM_GPUS \
    --validation_sampling_steps "3" \
    --log_validation \
    --checkpoints_total_limit 3 \
+    --allow_tf32 \
    --ema_start_step 0 \
    --training_cfg_rate 0.0 \
    --output_dir "outputs_dmd/wan_finetune" \
@@ -36,6 +36,7 @@ torchrun --nnodes 1 --nproc_per_node $NUM_GPUS \
    --validation_sampling_steps "3" \
    --log_validation \
    --checkpoints_total_limit 3 \
+    --allow_tf32 \
    --ema_start_step 0 \
    --training_cfg_rate 0.0 \
    --output_dir "outputs_dmd/wan_finetune" \
@@ -35,6 +35,7 @@ torchrun --nnodes 1 --nproc_per_node $NUM_GPUS\
    --validation_sampling_steps "2,4,8" \
    --log_validation \
    --checkpoints_total_limit 3\
+    --allow_tf32\
    --ema_start_step 0\
    --training_cfg_rate 0.0\
    --output_dir="$DATA_DIR/outputs/wan_finetune"\
@@ -41,6 +41,7 @@ torchrun --nnodes 1 --nproc_per_node $NUM_GPUS \
    --validation_sampling_steps "50" \
    --log_validation \
    --checkpoints_total_limit 3 \
+    --allow_tf32 \
    --ema_start_step 0 \
    --training_cfg_rate 0.0 \
    --output_dir "$DATA_DIR/outputs/wan_finetune" \
@@ -11,9 +11,9 @@ fastvideo generate \
    --sp-size $num_gpus \
    --tp-size 1 \
    --num-gpus $num_gpus \
-    --height 480 \
+    --height 448 \
    --width 832 \
-    --num-frames 81 \
+    --num-frames 61 \
    --num-inference-steps 3 \
    --dit-cpu-offload False \
    --vae-cpu-offload False \
@@ -42,7 +42,7 @@ fastvideo generate \
    --tp-size 1 \
    --num-gpus $num_gpus \
    --height 480 \
-    --width 832 \
+    --width 848 \
    --num-frames 81 \
    --num-inference-steps 3 \
    --fps 16 \
@@ -78,27 +78,4 @@ fastvideo generate \
    --output-path outputs_video_dmd_14B_720P/ \
    --VSA-sparsity 0.9 \
    --dmd-denoising-steps "1000,757,522" \
-    --enable_torch_compile 
-
-# 720P 5B
-num_gpus=1
-export FASTVIDEO_ATTENTION_BACKEND=FLASH_ATTN
-export MODEL_BASE=FastVideo/FastWan2.2-TI2V-5B-FullAttn-Diffusers
-# export MODEL_BASE=hunyuanvideo-community/HunyuanVideo
-# You can either use --prompt or --prompt-txt, but not both.
-fastvideo generate \
-    --model-path $MODEL_BASE \
-    --sp-size $num_gpus \
-    --tp-size 1 \
-    --num-gpus $num_gpus \
-    --height 704 \
-    --width 1280 \
-    --num-frames 104 \
-    --num-inference-steps 3 \
-    --fps 24 \
-    --prompt-txt assets/prompt.txt \
-    --negative-prompt "Bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured, misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards" \
-    --seed 1024 \
-    --output-path outputs_video_dmd_5B_720P/ \
-    --dmd-denoising-steps "1000,757,522" \
    --enable_torch_compile