hao-ai-lab-FastVideo/examples/inference/basic/README.md

Basic Video Generation Tutorial

The VideoGenerator class provides the primary Python interface for doing offline video generation, which is interacting with a diffusion pipeline without using a separate inference api server.

Requirements

• At least a single NVIDIA GPU with CUDA 12.4.
• Python 3.10-3.12

Installation

If you have not installed FastVideo, please following these instructions first.

Usage

The first script in this example shows the most basic usage of FastVideo. If you are new to Python and FastVideo, you should start here.

# if you have not cloned the directory:
git clone https://github.com/hao-ai-lab/FastVideo.git && cd FastVideo

python examples/inference/basic/basic.py

Apple Silicon (FastMetal-QAD)

Use the MLX runtime with FastMetal-QAD. See the MLX install guide.

hf download FastVideo/FastMetal-1.3B-QAD --local-dir ./FastMetal-1.3B-QAD

python examples/inference/basic/mlx_wan_prompt_to_video.py \
  --model-root ./FastMetal-1.3B-QAD \
  --mlx-checkpoint ./FastMetal-1.3B-QAD \
  --prompt "A bird's-eye view of a misty forest valley at dawn."

5B uses mlx_wan22_generate.py with FastVideo/FastMetal-5B-QAD.

examples/inference/basic/basic_mps.py is the older PyTorch MPS demo.

FastH3 V1 T2VA also runs through the native MLX runtime. Convert the DiT to INT8, INT6, or INT4 first, then run:

python examples/inference/basic/mlx_fasth3.py \
  --model-root ./FastH3-Preview-v0.2 \
  --mlx-checkpoint ./FastH3-MLX/int6 \
  --prompt "(S1) A presenter says <d>[English] Fast H3 is amazing.</d>" \
  --height 480 --width 832 --num-frames 124 \
  --output-path ./outputs/fasth3_int6.mp4

FastH3 V2 is a separate checkpoint and script. Convert with --include-vsa and run mlx_fasth3_8step.py. Do not reuse a V1 DiT.

Pass --fast for temporal RIFE fast mode and --fast-spatial for spatial fast mode (reduced-canvas denoise + pixel-space upsample); the two compose. V1 VSA is opt-in: convert with --include-vsa and pass --vsa. V2 turns VSA on by default. See the MLX install guide. This MLX entrypoint supports T2VA only; FL2VA, Ref2VA, and two-pass refinement remain follow-up work. INT6/INT8/INT4 are weight-only; VSA attention activations stay BF16. Dense-only V1 checkpoints keep working for dense inference.

The complete setup and conversion commands are in the MLX install guide.

For an example running DMD+VSA inference:

python examples/inference/basic/basic_dmd.py

For the typed config/request path added during the inference API refactor:

python examples/inference/basic/basic_dmd_new_api.py

FastH3 Preview

The verified basic FastH3 example runs the few-step (4-forward, DMD2-distilled) MiniMax-H3 preview, generating synchronized video and audio with its trained block-sparse VSA attention:

UV_TORCH_BACKEND=cu130 uv pip install -e ".[fasth3]"

This installs the pinned FA4 CuTe package and FastVideo kernel release used by the measured GB200 profile. Then run:

python examples/inference/basic/basic_fasth3.py --prompt "your prompt"

The default checkpoint, FastH3 Preview v0.2, is public on the Hub under the MiniMax H3 Community License. Review its model card and license before use or redistribution.

The default all profile is the fastest measured four-GPU Preview recipe on GB200. It selects VSA sparsity 0.9 with 64-token tiles and the sm_100a sparse kernel, enables FA4 for eligible non-VSA paths, regionally compiles and replicates the sparse DiT, compiles and temporally parallelizes the video VAE with the gather strategy, and pins CPU-offloaded component memory. It also pins the benchmark protocol: five sigma-grid points (exactly four DiT forwards), one excluded seed-999 warmup, then three timed seed-1000 requests with distinct output paths.

The equivalent explicit command is:

python examples/inference/basic/basic_fasth3.py \
  --prompt "your prompt" \
  --profile all \
  --num-gpus 4 \
  --steps 5 \
  --vsa-sparsity 0.9 \
  --vsa-tile-size 64 \
  --vsa-kernel sm100a \
  --compile-vae \
  --parallel-vae \
  --replicated-dit \
  --pin-cpu-memory \
  --fa4 \
  --no-torch-compile \
  --inference-torch-compile \
  --ulysses-a2a off \
  --warmup \
  --repeats 3 \
  --seed 1000 \
  --warmup-seed 999

all enables the inference-only H3 fusions and regional compile. Both can change floating-point operation order, so this is a report-only performance profile rather than an exact-parity route. Use --profile strict to disable the H3 fusions while preserving regional compile, or --profile strict --no-inference-torch-compile for the eager strict route. Individual --no-* switches are available for portability and attribution; in particular, use --vsa-kernel triton --no-fa4 if the Blackwell kernels are unavailable. --h3-sequential-load / --no-h3-sequential-load override the auto split that releases Qwen3-VL before DiT/VAE load (on by default on GB10, off on discrete GPUs). The script preserves the warmup and each measured video under distinct paths, then prints per-request wall time plus a warmup-excluded median.

One script covers each validated duration; regional compile is the fastest measured DiT route for all three:

# 5 s
python examples/inference/basic/basic_fasth3.py \
  --prompt "your prompt" --output outputs/fasth3_5s
# 10 s
python examples/inference/basic/basic_fasth3.py \
  --prompt "your prompt" --num-frames 243 --output outputs/fasth3_10s
# 15 s
python examples/inference/basic/basic_fasth3.py \
  --prompt "your prompt" --num-frames 345 --output outputs/fasth3_15s

Pass --no-inference-torch-compile to recover the eager sparse-DiT route.

FastH3 Preview LoRAs

The LoRA release runs on top of MiniMaxAI/MiniMax-H3 with the same default compile, fusion, FA4, VSA, and parallel-VAE profile as the full FastH3 example:

bash examples/inference/basic/run_fasth3_lora_preview_vsa_datafree.sh \
  --prompt "your prompt"

The four release launchers are:

• run_fasth3_lora_preview_vsa_datafree.sh
• run_fasth3_lora_preview_vsa_synthetic_step1300.sh
• run_fasth3_lora_preview_vsa_synthetic_step1900.sh
• run_fasth3_lora_preview_dense_datafree.sh

Each downloads its exact private adapter file from FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA; authenticate with hf auth login first. Pass --lora-strength 0.5 to interpolate every adapter payload at half strength. Strength 1 applies the published rank-64 adapter at its trained scale and approximates the full student; 0 removes its weight deltas. VSA launchers still use sparse attention at strength 0 and require FastVideo's tile-64 VSA kernel; the dense launcher selects FA4. Each launcher writes to its own variant directory by default so comparison outputs do not collide.

Basic Walkthrough

All you need to generate videos using multi-gpus from state-of-the-art diffusion pipelines is the following few lines!

from fastvideo import VideoGenerator

def main():
    generator = VideoGenerator.from_pretrained(
        "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
        {"engine": {"num_gpus": 1}},
    )

    prompt = ("A curious raccoon peers through a vibrant field of yellow sunflowers, its eyes "
             "wide with interest. The playful yet serene atmosphere is complemented by soft "
             "natural light filtering through the petals. Mid-shot, warm and cheerful tones.")
    video = generator.generate({"prompt": prompt})

if __name__ == "__main__":
    main()