hao-ai-lab-FastVideo/docs/inference/inference_quick_start.md

Inference Quick Start

This page contains step-by-step instructions to get you quickly started with video generation using FastVideo.

Requirements

• OS: Linux (tested on Ubuntu 22.04+), or macOS on Apple silicon via the MLX install guide
• Python: 3.10-3.12
• CUDA: 12.6 or 13.0 (NVIDIA GPUs)
• GPU: At least one NVIDIA GPU, or an Apple silicon chip with the MLX runtime

Installation

If you previously used Conda, we recommend using uv instead for a faster and more stable environment setup:

# Create and activate a new uv environment
uv venv --python 3.12 --seed
source .venv/bin/activate

# Install FastVideo
UV_TORCH_BACKEND=cu126 uv pip install fastvideo

Use UV_TORCH_BACKEND=cu130 instead on CUDA 13.

For advanced installation options, see the Installation Guide.

Generating Your First Video

Here's a minimal example to generate a video using the default settings. Create a file called example.py with the following code:

from fastvideo import VideoGenerator

def main():
    # Create a video generator with a pre-trained model
    generator = VideoGenerator.from_pretrained(
        "Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
        num_gpus=1,  # Adjust based on your hardware
    )

    # Define a prompt for your video
    prompt = "A curious raccoon peers through a vibrant field of yellow sunflowers, its eyes wide with interest."

    # Generate the video
    video = generator.generate_video(
        prompt,
        output_path="my_videos/",  # Controls where videos are saved
        save_video=True
    )

if __name__ == '__main__':
    main()

num_gpus=1 uses the in-process executor (no worker subprocess). On Colab/Kaggle free-tier hosts (~16GB system RAM), keep num_gpus=1; extra T4s do not add host memory, so num_gpus>1 is likely to OOM.

Run the script with:

python example.py

The generated video will be saved in the current directory under my_videos/

More inference scripts and recipes can be found in examples/inference/ and scripts/inference/.

Available Models

Please see the support matrix for the list of supported models and their available optimizations.

Image-to-Video Generation

You can generate a video starting from an initial image:

from fastvideo import VideoGenerator, SamplingParam

def main():
    # Create the generator
    model_name = "Wan-AI/Wan2.1-I2V-14B-480P-Diffusers"
    generator = VideoGenerator.from_pretrained(model_name, num_gpus=1)

    # Set up parameters with an initial image
    sampling_param = SamplingParam.from_pretrained(model_name)
    sampling_param.image_path = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/astronaut.jpg"
    sampling_param.num_frames = 107

    # Generate video based on the image
    prompt = "A photograph coming to life with gentle movement"
    generator.generate_video(prompt, sampling_param=sampling_param,
                             output_path="my_videos/",
                             save_video=True)

if __name__ == '__main__':
    main()

Troubleshooting

Common issues and their solutions:

Out of Memory Errors

Host RAM and GPU memory are separate. The default num_gpus=1 path now loads weights once in the current process. Free-tier Colab/Kaggle notebooks (~16GB host RAM) should not set num_gpus>1.

If you encounter CUDA out of memory errors:

• Reduce num_frames or video resolution
• Enable FastVideo offloading options such as dit_layerwise_offload=True (single GPU) or use_fsdp_inference=True (multi-GPU)
• Try a smaller model or use distilled versions
• Use num_gpus > 1 if multiple GPUs are available
• Try enabling FSDP inference with use_fsdp_inference=True (may slow down generation)
• Try enabling DiT layerwise offload with dit_layerwise_offload=True (now only a few models support this, but may introduce less overhead than FSDP)

Slow Generation

To speed up generation:

• Reduce num_inference_steps (20-30 is usually sufficient)
• Use half precision (fp16) for the VAE
• Use multiple GPUs if available

Unexpected Results

If the generated video doesn't match your prompt:

• Try increasing guidance_scale (7.0-9.0 works well)
• Make your prompt more detailed and specific
• Experiment with different random seeds
• Try a different model

Next Steps

• Learn about Advanced Inference Configurations
• Learn about using Optimizations
• See Examples for more usage scenarios
• Join our Community Slack.