nodes/hao-ai-lab-FastVideo
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

[Docs] More docs update (#394)

Browse Source
This commit is contained in:
William Lin
2025-05-11 16:19:47 -07:00
committed by GitHub
parent 51956369a5
commit 0cf001986a
11 changed files with 363 additions and 333 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/source/inference/inference_quick_start.md
+2 -139
View File
@@ -3,15 +3,10 @@
This page contains step-by-step instructions to get you quickly started with video generation using FastVideo.
## Table of Contents
- [Software Requirements](#software-requirements)
- [Installation](#installation)
- [Generating Your First Video](#generating-your-first-video)
- [Customizing Generation](#customizing-generation)
- [Available Models](#available-models)
- [Multi-GPU Setup](#multi-gpu-setup)
- [Image-to-Video Generation](#image-to-video-generation)
- [Output Formats and Options](#output-formats-and-options)
- [Performance Optimization](#performance-optimization)
- [Troubleshooting](#troubleshooting)
- [Advanced Configuration](#advanced-configuration)
- [Next Steps](#next-steps)
@@ -27,7 +22,7 @@ We recommend using an environment manager such as `Conda` to create a clean envi
```bash
# Create and activate a new conda environment
conda create -n fastvideo python=3.10
conda create -n fastvideo python=3.12
conda activate fastvideo
# Install FastVideo
@@ -72,72 +67,10 @@ python example.py
The generated video will be saved in the current directory under `my_videos/`.
## Customizing Generation
You can customize various parameters when generating videos using the `SamplingParam` class:
```python
from fastvideo import VideoGenerator, SamplingParam
def main():
# Create the generator
generator = VideoGenerator.from_pretrained(
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
num_gpus=1,
)
# Create and customize sampling parameters
sampling_param = SamplingParam.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers")
# How many frames to generate
sampling_param.num_frames = 45
# Video resolution (width, height)
sampling_param.width = 1024
sampling_param.height = 576
# How many steps we denoise the video (higher = better quality, slower generation)
sampling_param.num_inference_steps = 30
# How strongly the video conforms to the prompt (higher = more faithful to prompt)
sampling_param.guidance_scale = 7.5
# Random seed for reproducibility
sampling_param.seed = 42 # Optional, leave unset for random results
# Generate video with custom parameters
prompt = "A beautiful sunset over a calm ocean, with gentle waves."
video = generator.generate_video(
prompt,
sampling_param=sampling_param,
output_path="my_videos/", # Controls where videos are saved
return_frames=True, # Also return frames from this call (defaults to False)
save_video=True
)
# If return_frames=True, video contains the generated frames as a NumPy array
print(f"Generated {len(video)} frames")
if __name__ == '__main__':
main()
```
## Available Models
Please see the [support matrix](#support-matrix) for the list of supported models and their available optimizations.
## Multi-GPU Setup
FastVideo automatically distributes the generation process when multiple GPUs are specified:
```python
# Will use 4 GPUs in parallel for faster generation
generator = VideoGenerator.from_pretrained(
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
num_gpus=4,
)
```
## Image-to-Video Generation
You can generate a video starting from an initial image:
@@ -159,53 +92,6 @@ prompt = "A photograph coming to life with gentle movement"
video = generator.generate_video(prompt, sampling_param=sampling_param)
```
## Output Formats and Options
FastVideo provides several options for saving and manipulating the generated videos:
```python
from fastvideo import VideoGenerator, SamplingParam, VideoFormat
# Create the generator
generator = VideoGenerator.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers")
# Generate with custom output options
video = generator.generate_video(
"Drone footage of a tropical rainforest",
output_path="videos/rainforest/", # Custom save location
filename="rainforest_flyover", # Custom filename (no extension needed)
format=VideoFormat.MP4, # Output format: MP4, GIF, or WEBM
fps=30, # Frames per second for the output video
loop=False, # Whether to loop the video (for GIF)
save_frames=True, # Also save individual frames as images
)
```
## Performance Optimization
Optimize FastVideo for your hardware:
```python
from fastvideo import VideoGenerator, PipelineConfig
# Create an optimized configuration
config = PipelineConfig.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers")
# Memory optimization
config.enable_vae_slicing = True # Process VAE in slices to reduce memory
config.enable_model_cpu_offload = True # Offload models to CPU when not in use
# Speed optimization
config.vae_config.precision = "fp16" # Use half precision for VAE
config.enable_xformers = True # Use xformers for attention computation
# Create generator with optimized config
generator = VideoGenerator.from_pretrained(
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
pipeline_config=config,
)
```
## Troubleshooting
Common issues and their solutions:
@@ -232,30 +118,7 @@ If the generated video doesn't match your prompt:
## Advanced Configuration
For advanced customization, use the `PipelineConfig` class:
```python
from fastvideo import VideoGenerator, PipelineConfig
# Load the default configuration for a model
config = PipelineConfig.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers")
# Modify configuration settings
config.vae_config.scale_factor = 16
config.vae_config.precision = "fp16"
config.scheduler_config.beta_schedule = "linear"
config.attention_config.attention_type = "xformers"
# Create generator with custom config
generator = VideoGenerator.from_pretrained(
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
pipeline_config=config,
)
# Generate video
prompt = "A futuristic cityscape with flying cars and neon signs."
video = generator.generate_video(prompt)
```
## Optimizations
## Next Steps