Basic Video Generation Tutorial
The VideoGenerator class provides the primary Python interface for doing offline video generation, which is interacting with a diffusion pipeline without using a separate inference api server.
Requirements
- At least a single NVIDIA GPU with CUDA 12.4.
- Python 3.10-3.12
Installation
If you have not installed FastVideo, please following these instructions first.
Usage
The first script in this example shows the most basic usage of FastVideo. If you are new to Python and FastVideo, you should start here.
# if you have not cloned the directory:
git clone https://github.com/hao-ai-lab/FastVideo.git && cd FastVideo
python examples/inference/basic/basic.py
Apple Silicon (FastMetal-QAD)
Use the MLX runtime with FastMetal-QAD. See the MLX install guide.
hf download FastVideo/FastMetal-1.3B-QAD --local-dir ./FastMetal-1.3B-QAD
python examples/inference/basic/mlx_wan_prompt_to_video.py \
--model-root ./FastMetal-1.3B-QAD \
--mlx-checkpoint ./FastMetal-1.3B-QAD \
--prompt "A bird's-eye view of a misty forest valley at dawn."
5B uses
mlx_wan22_generate.py
with
FastVideo/FastMetal-5B-QAD.
examples/inference/basic/basic_mps.py is the older PyTorch MPS demo.
FastH3 V1 T2VA also runs through the native MLX runtime. Convert the DiT to INT8, INT6, or INT4 first, then run:
python examples/inference/basic/mlx_fasth3.py \
--model-root ./FastH3-Preview-v0.2 \
--mlx-checkpoint ./FastH3-MLX/int6 \
--prompt "(S1) A presenter says <d>[English] Fast H3 is amazing.</d>" \
--height 480 --width 832 --num-frames 124 \
--output-path ./outputs/fasth3_int6.mp4
FastH3 V2 is a separate checkpoint and script. Convert with
--include-vsa and run mlx_fasth3_8step.py. Do not reuse a V1 DiT.
Pass --fast for temporal RIFE fast mode and --fast-spatial for spatial
fast mode (reduced-canvas denoise + pixel-space upsample); the two compose.
V1 VSA is opt-in: convert with --include-vsa and pass --vsa. V2
turns VSA on by default. See the
MLX install guide.
This MLX entrypoint supports T2VA only; FL2VA, Ref2VA, and
two-pass refinement remain follow-up work. INT6/INT8/INT4 are
weight-only; VSA attention activations stay BF16. Dense-only V1
checkpoints keep working for dense inference.
The complete setup and conversion commands are in the MLX install guide.
For an example running DMD+VSA inference:
python examples/inference/basic/basic_dmd.py
For the typed config/request path added during the inference API refactor:
python examples/inference/basic/basic_dmd_new_api.py
FastH3 Preview
The verified basic FastH3 example runs the few-step (4-forward, DMD2-distilled) MiniMax-H3 preview, generating synchronized video and audio with its trained block-sparse VSA attention:
UV_TORCH_BACKEND=cu130 uv pip install -e ".[fasth3]"
This installs the pinned FA4 CuTe package and FastVideo kernel release used by the measured GB200 profile. Then run:
python examples/inference/basic/basic_fasth3.py --prompt "your prompt"
The default checkpoint, FastH3 Preview v0.2, is public on the Hub under the MiniMax H3 Community License. Review its model card and license before use or redistribution.
The default all profile is the fastest measured four-GPU Preview recipe on GB200. It selects VSA sparsity 0.9 with 64-token tiles and the ptx_blackwell sparse kernel, enables FA4 for eligible non-VSA paths, regionally compiles and replicates the sparse DiT, compiles and temporally parallelizes the video VAE with the gather strategy, and pins CPU-offloaded component memory. It also pins the benchmark protocol: five sigma-grid points (exactly four DiT forwards), one excluded seed-999 warmup, then three timed seed-1000 requests with distinct output paths.
The equivalent explicit command is:
python examples/inference/basic/basic_fasth3.py \
--prompt "your prompt" \
--profile all \
--num-gpus 4 \
--steps 5 \
--vsa-sparsity 0.9 \
--vsa-tile-size 64 \
--vsa-kernel ptx_blackwell \
--compile-vae \
--parallel-vae \
--replicated-dit \
--pin-cpu-memory \
--fa4 \
--no-torch-compile \
--inference-torch-compile \
--ulysses-a2a off \
--warmup \
--repeats 3 \
--seed 1000 \
--warmup-seed 999
all enables the inference-only H3 fusions and regional compile. Both can change floating-point operation order, so this is a report-only performance profile rather than an exact-parity route. Use --profile strict to disable the H3 fusions while preserving regional compile, or --profile strict --no-inference-torch-compile for the eager strict route. Individual --no-* switches are available for portability and attribution; in particular, use --vsa-kernel triton --no-fa4 if the Blackwell kernels are unavailable. --h3-sequential-load / --no-h3-sequential-load override the auto split that releases Qwen3-VL before DiT/VAE load (on by default on GB10, off on discrete GPUs). The script preserves the warmup and each measured video under distinct paths, then prints per-request wall time, the worker's peak memory, and a warmup-excluded median. Peak memory is the worker's allocator high-water mark: it is never reset between generations, so it includes model load and every prior run, and only rank 0's value is reported. It is populated by the mp backend only: the Ray backend leaves peak_memory_mb unset, so --execution-backend ray prints no Peak memory line.
One script covers each validated duration; regional compile is the fastest measured DiT route for all three:
# 5 s
python examples/inference/basic/basic_fasth3.py \
--prompt "your prompt" --output outputs/fasth3_5s
# 10 s
python examples/inference/basic/basic_fasth3.py \
--prompt "your prompt" --num-frames 243 --output outputs/fasth3_10s
# 15 s
python examples/inference/basic/basic_fasth3.py \
--prompt "your prompt" --num-frames 345 --output outputs/fasth3_15s
Pass --no-inference-torch-compile to recover the eager sparse-DiT route.
Hopper (H100 80 GB)
Hopper has no FA4 and no ptx_blackwell VSA kernel, so the GB200 profile above does
not apply there. The measured Hopper route keeps the same checkpoint, four
forwards and 90% VSA sparsity and changes three things: the 64-token VSA path
runs on the ThunderKittens sm_90a kernel, which fastvideo-kernel picks on sm_90
whenever it is built with
cd fastvideo-kernel && CMAKE_ARGS='-DFASTVIDEO_KERNEL_BUILD_TK=ON' ./build.sh
(--vsa-kernel tk keeps the ptx_blackwell route off and fails early when that
extension is missing),
the DiT is FSDP-sharded across the GPUs instead of replicated
(--no-replicated-dit; a replicated 66 GB BF16 DiT does not fit 80 GB), and
the Qwen3-VL text encoder stays resident as block-scaled FP8 (35.5 GB instead
of 66.7 GB) rather than being offloaded on every request:
python scripts/checkpoint_conversion/quantize_minimax_h3_text_encoder_fp8.py \
--source ./FastH3-Preview-v1/text_encoder --output ./FastH3-TextEncoder-FP8
python examples/inference/basic/basic_fasth3.py \
--prompt "your prompt" \
--num-gpus 4 --no-replicated-dit \
--vsa-kernel tk --no-fa4 \
--text-encoder-weights ./FastH3-TextEncoder-FP8 \
--no-offload-text-encoder --no-offload-vae \
--height 544 --width 960 --num-frames 345
examples/inference/basic/basic_fasth3_h100.yaml is the same profile for
fastvideo generate --config. Measured points on this route (345 frames,
24 FPS, stereo audio, warm, loading and compilation excluded):
| GPUs | Resolution | Playback | Generation | Source |
|---|---|---|---|---|
| 8x H100 80 GB | 1344x768 | 14.375 s | 13.506 s (13.503 / 13.506 / 13.554) | hlander-ai/minimax-h3 on FastVideo b2db0c0 with these patches, Triton VSA |
| 4x H100 80 GB, NVLink | 960x544 | 14.375 s | 12.98-14.02 s over 8 consecutive clips | Windflow streaming worker on FastVideo b2db0c0 with these patches, TK VSA |
The 4x H100 row was measured through a downstream streaming worker rather than
this script; basic_fasth3.py numbers for that shape will follow in a later
PR. The FP8 text encoder is a precision change relative to the stock BF16
encoder; the other two changes are lossless.
FastH3 Preview LoRAs
The LoRA release runs on top of MiniMaxAI/MiniMax-H3 with the same default
compile, fusion, FA4, VSA, and parallel-VAE profile as the full FastH3 example:
bash examples/inference/basic/run_fasth3_lora_preview_vsa_datafree.sh \
--prompt "your prompt"
The four release launchers are:
run_fasth3_lora_preview_vsa_datafree.shrun_fasth3_lora_preview_vsa_synthetic_step1300.shrun_fasth3_lora_preview_vsa_synthetic_step1900.shrun_fasth3_lora_preview_dense_datafree.sh
Each downloads its exact private adapter file from
FastVideo/FastVideo-FastH3-4-step-Preview-v1-LoRA; authenticate with hf auth login first. Pass --lora-strength 0.5 to interpolate every adapter payload at
half strength. Strength 1 applies the published rank-64 adapter at its trained
scale and approximates the full student; 0 removes its weight deltas. VSA
launchers still use sparse attention at strength 0 and require FastVideo's
tile-64 VSA kernel; the dense launcher selects FA4. Each launcher writes to its
own variant directory by default so comparison outputs do not collide.
FastH3 OmniRef PDD (Ref2VA)
basic_fasth3_omniref_pdd.py
runs a Parallel Decoding Distillation Ref2VA student in eight transformer
forwards. The export carries only its transformer_ref, scheduler configs,
and fastvideo_inference.json; the script links them with the base
MiniMax-H3 components into one local model directory. Its 128-token VSA tiles
need the ptx_blackwell kernel:
python examples/inference/basic/basic_fasth3_omniref_pdd.py \
--model-path <local export directory or Hugging Face repo id> \
--video reference.mp4 --image character.png --prompt "your prompt"
See FastH3 distilled checkpoint schedules
for the contract and the reference-video sparsity policy. For the consumer NVFP4
stack (RTX 5090, RTX PRO 6000, DGX Spark; FP8 on the RTX 4090) add
--profile <name>, and see
FastH3 OmniRef inference.
Basic Walkthrough
All you need to generate videos using multi-gpus from state-of-the-art diffusion pipelines is the following few lines!
from fastvideo import VideoGenerator
def main():
generator = VideoGenerator.from_pretrained(
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
num_gpus=1,
)
prompt = ("A curious raccoon peers through a vibrant field of yellow sunflowers, its eyes "
"wide with interest. The playful yet serene atmosphere is complemented by soft "
"natural light filtering through the petals. Mid-shot, warm and cheerful tones.")
video = generator.generate_video(prompt)
if __name__ == "__main__":
main()