[Docs] More docs update (#394)
This commit is contained in:
William Lin
@@ -11,11 +11,52 @@ FastVideo currently only supports Linux and NVIDIA CUDA GPUs.
|
||||
- **CUDA 12.4**
|
||||
- **At least 1 NVIDIA GPU**
|
||||
|
||||
## Quick start
|
||||
We recommend using an environment manager such as [Conda](#miniconda-recommended)
|
||||
## Set up using Python
|
||||
### Create a new Python environment
|
||||
|
||||
#### Conda
|
||||
You can create a new python environment using [Conda](https://docs.conda.io/projects/conda/en/stable/user-guide/getting-started.html)
|
||||
##### 1. Install Miniconda (if not already installed)
|
||||
|
||||
```bash
|
||||
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
|
||||
bash Miniconda3-latest-Linux-x86_64.sh
|
||||
source ~/.bashrc
|
||||
```
|
||||
|
||||
##### 2. Create and activate a Conda environment for FastVideo
|
||||
|
||||
```bash
|
||||
# (Recommended) Create a new conda environment.
|
||||
conda create -n fastvideo python=3.12 -y
|
||||
conda activate fastvideo
|
||||
```
|
||||
|
||||
:::{note}
|
||||
[PyTorch has deprecated the conda release channel](https://github.com/pytorch/pytorch/issues/138506). If you use `conda`, please only use it to create Python environment rather than installing packages.
|
||||
:::
|
||||
|
||||
#### uv
|
||||
|
||||
:::{tip}
|
||||
We highly recommend using `uv` to install FastVideo. In our experience, `uv` speeds up installation by at least 3x.
|
||||
:::
|
||||
|
||||
Or you can create a new Python environment using [uv](https://docs.astral.sh/uv/), a very fast Python environment manager. Please follow the [documentation](https://docs.astral.sh/uv/#getting-started) to install `uv`. After installing `uv`, you can create a new Python environment using the following command:
|
||||
|
||||
```console
|
||||
# (Recommended) Create a new uv environment. Use `--seed` to install `pip` and `setuptools` in the environment.
|
||||
uv venv --python 3.12 --seed
|
||||
source .venv/bin/activate
|
||||
```
|
||||
|
||||
### Installation
|
||||
|
||||
```bash
|
||||
pip install fastvideo
|
||||
|
||||
# or if you are using uv
|
||||
uv pip install fastvideo
|
||||
```
|
||||
|
||||
Also optionally install flash-attn:
|
||||
@@ -24,30 +65,9 @@ Also optionally install flash-attn:
|
||||
pip install flash-attn==2.7.4.post1 --no-build-isolation
|
||||
```
|
||||
|
||||
[Click here to install from source](#installation-from-source)
|
||||
### Installation from Source
|
||||
|
||||
## Environment Setup
|
||||
|
||||
We highly recommend using a `conda` or `virtualenv` environment:
|
||||
### Miniconda (Recommended)
|
||||
#### 1. Install Miniconda (if not already installed)
|
||||
|
||||
```bash
|
||||
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
|
||||
bash Miniconda3-latest-Linux-x86_64.sh
|
||||
source ~/.bashrc
|
||||
```
|
||||
|
||||
#### 2. Create and activate a Conda environment for FastVideo
|
||||
|
||||
```bash
|
||||
conda create -n fastvideo python=3.12 -y
|
||||
conda activate fastvideo
|
||||
```
|
||||
|
||||
## Installation from Source
|
||||
|
||||
### 1. Clone the FastVideo repository
|
||||
#### 1. Clone the FastVideo repository
|
||||
|
||||
```bash
|
||||
git clone https://github.com/hao-ai-lab/FastVideo.git && cd FastVideo
|
||||
@@ -59,23 +79,23 @@ Basic installation:
|
||||
|
||||
```bash
|
||||
pip install -e .
|
||||
|
||||
# or if you are using uv
|
||||
uv pip install -e .
|
||||
```
|
||||
|
||||
## Optional Dependencies
|
||||
### Optional Dependencies
|
||||
|
||||
### Flash Attention
|
||||
#### Flash Attention
|
||||
|
||||
```bash
|
||||
pip install flash-attn==2.7.4.post1 --no-build-isolation
|
||||
```
|
||||
|
||||
### Sliding Tile Attention (STA) (Requires CUDA 12.4 and H100)
|
||||
|
||||
To try Sliding Tile Attention (optional), please follow the instructions in [csrc/sliding_tile_attention/README.md](#sta-installation) to install STA.
|
||||
|
||||
## Docker Images
|
||||
## Set up using Docker
|
||||
We also have prebuilt docker images with FastVideo dependencies pre-installed:
|
||||
[Docker Images](#docker)
|
||||
|
||||
## Development Environment Setup
|
||||
|
||||
If you're planning to contribute to FastVideo please see the following page:
|
||||
@@ -84,7 +104,7 @@ If you're planning to contribute to FastVideo please see the following page:
|
||||
## Hardware Requirements
|
||||
|
||||
### For Basic Inference
|
||||
- NVIDIA GPU with CUDA 12.4support
|
||||
- NVIDIA GPU with CUDA 12.4 support
|
||||
|
||||
### For Lora Finetuning
|
||||
- 40GB GPU memory each for 2 GPUs with lora
|
||||
|
||||
@@ -10,9 +10,74 @@ This class will be the primary Python API for generating videos and images.
|
||||
fastvideo.VideoGenerator
|
||||
```
|
||||
|
||||
VideoGenerator.from_pretrained() should be the primary way of creating a new video generator.
|
||||
`````{py:class} VideoGenerator(fastvideo_args: fastvideo.v1.fastvideo_args.FastVideoArgs, executor_class: type[fastvideo.v1.worker.executor.Executor], log_stats: bool)
|
||||
:canonical: fastvideo.v1.entrypoints.video_generator.VideoGenerator
|
||||
|
||||
````{py:method} from_pretrained(model_path: str, device: typing.optional[str] = none, torch_dtype: typing.optional[torch.dtype] = none, pipeline_config: typing.optional[typing.union[str | fastvideo.v1.configs.pipelines.pipelineconfig]] = none, **kwargs) -> v1.entrypoints.video_generator.videogenerator
|
||||
:canonical: v1.entrypoints.video_generator.videogenerator.from_pretrained
|
||||
```{autodoc2-docstring} fastvideo.v1.entrypoints.video_generator.VideoGenerator
|
||||
:parser: docs.source.autodoc2_docstring_parser
|
||||
```
|
||||
|
||||
`VideoGenerator.from_pretrained()` should be the primary way of creating a new video generator.
|
||||
|
||||
````{py:method} from_pretrained(model_path: str, device: typing.Optional[str] = None, torch_dtype: typing.Optional[torch.dtype] = None, pipeline_config: typing.Optional[typing.Union[str | fastvideo.v1.configs.pipelines.PipelineConfig]] = None, **kwargs) -> fastvideo.v1.entrypoints.video_generator.VideoGenerator
|
||||
:canonical: fastvideo.v1.entrypoints.video_generator.VideoGenerator.from_pretrained
|
||||
:classmethod:
|
||||
|
||||
```{autodoc2-docstring} fastvideo.v1.entrypoints.video_generator.VideoGenerator.from_pretrained
|
||||
:parser: docs.source.autodoc2_docstring_parser
|
||||
```
|
||||
|
||||
|
||||
## Configuring FastVideo
|
||||
|
||||
The follow two classes `PipelineConfig` and `SamplingParam` are used to configure initialization and sampling parameters, respectively.
|
||||
|
||||
### PipelineConfig
|
||||
```{autodoc2-summary}
|
||||
fastvideo.PipelineConfig
|
||||
```
|
||||
|
||||
`````{py:class} PipelineConfig
|
||||
:canonical: fastvideo.v1.configs.pipelines.base.PipelineConfig
|
||||
|
||||
```{autodoc2-docstring} fastvideo.v1.configs.pipelines.base.PipelineConfig
|
||||
:parser: docs.source.autodoc2_docstring_parser
|
||||
```
|
||||
|
||||
````{py:method} from_pretrained(model_path: str) -> fastvideo.v1.configs.pipelines.base.PipelineConfig
|
||||
:canonical: fastvideo.v1.configs.pipelines.base.PipelineConfig.from_pretrained
|
||||
:classmethod:
|
||||
|
||||
```{autodoc2-docstring} fastvideo.v1.configs.pipelines.base.PipelineConfig.from_pretrained
|
||||
:parser: docs.source.autodoc2_docstring_parser
|
||||
```
|
||||
|
||||
|
||||
````{py:method} dump_to_json(file_path: str)
|
||||
:canonical: fastvideo.v1.configs.pipelines.base.PipelineConfig.dump_to_json
|
||||
|
||||
```{autodoc2-docstring} fastvideo.v1.configs.pipelines.base.PipelineConfig.dump_to_json
|
||||
:parser: docs.source.autodoc2_docstring_parser
|
||||
```
|
||||
|
||||
|
||||
### SamplingParam
|
||||
|
||||
```{autodoc2-summary}
|
||||
fastvideo.SamplingParam
|
||||
```
|
||||
|
||||
`````{py:class} SamplingParam
|
||||
:canonical: fastvideo.v1.configs.sample.base.SamplingParam
|
||||
|
||||
```{autodoc2-docstring} fastvideo.v1.configs.sample.base.SamplingParam
|
||||
:parser: docs.source.autodoc2_docstring_parser
|
||||
```
|
||||
|
||||
````{py:method} from_pretrained(model_path: str) -> fastvideo.v1.configs.sample.base.SamplingParam
|
||||
:canonical: fastvideo.v1.configs.sample.base.SamplingParam.from_pretrained
|
||||
:classmethod:
|
||||
|
||||
```{autodoc2-docstring} fastvideo.v1.configs.sample.base.SamplingParam.from_pretrained
|
||||
:parser: docs.source.autodoc2_docstring_parser
|
||||
```
|
||||
|
||||
@@ -51,7 +51,7 @@ Dev in progress and highly experimental.
|
||||
:maxdepth: 1
|
||||
|
||||
getting_started/installation
|
||||
getting_started/v1_api
|
||||
<!-- getting_started/v1_api -->
|
||||
:::
|
||||
|
||||
:::{toctree}
|
||||
@@ -60,10 +60,10 @@ getting_started/v1_api
|
||||
|
||||
inference/inference_quick_start
|
||||
inference/configuration
|
||||
inference/optimizations
|
||||
inference/support_matrix
|
||||
inference/examples/basic
|
||||
inference/examples/gradio
|
||||
inference/examples/optimizations
|
||||
inference/examples/examples_inference_index
|
||||
inference/add_pipeline
|
||||
inference/v0_inference
|
||||
:::
|
||||
|
||||
@@ -98,14 +98,13 @@ design/overview
|
||||
|
||||
contributing/overview
|
||||
contributing/developer_env/index
|
||||
contributing/add_pipeline
|
||||
:::
|
||||
|
||||
:::{toctree}
|
||||
:caption: API Reference
|
||||
:maxdepth: 2
|
||||
|
||||
api/summary
|
||||
<!-- api/summary -->
|
||||
api/fastvideo/fastvideo
|
||||
:::
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
(add-pipeline)=
|
||||
|
||||
# 🏗️ Adding a New Diffusion Pipeline
|
||||
# 🏗️ Adding a New Pipeline
|
||||
|
||||
This guide explains how to implement a custom diffusion pipeline in FastVideo, leveraging the framework's modular architecture for high-performance video generation.
|
||||
|
||||
@@ -3,15 +3,10 @@
|
||||
This page contains step-by-step instructions to get you quickly started with video generation using FastVideo.
|
||||
|
||||
## Table of Contents
|
||||
- [Software Requirements](#software-requirements)
|
||||
- [Installation](#installation)
|
||||
- [Generating Your First Video](#generating-your-first-video)
|
||||
- [Customizing Generation](#customizing-generation)
|
||||
- [Available Models](#available-models)
|
||||
- [Multi-GPU Setup](#multi-gpu-setup)
|
||||
- [Image-to-Video Generation](#image-to-video-generation)
|
||||
- [Output Formats and Options](#output-formats-and-options)
|
||||
- [Performance Optimization](#performance-optimization)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
- [Advanced Configuration](#advanced-configuration)
|
||||
- [Next Steps](#next-steps)
|
||||
@@ -27,7 +22,7 @@ We recommend using an environment manager such as `Conda` to create a clean envi
|
||||
|
||||
```bash
|
||||
# Create and activate a new conda environment
|
||||
conda create -n fastvideo python=3.10
|
||||
conda create -n fastvideo python=3.12
|
||||
conda activate fastvideo
|
||||
|
||||
# Install FastVideo
|
||||
@@ -72,72 +67,10 @@ python example.py
|
||||
|
||||
The generated video will be saved in the current directory under `my_videos/`.
|
||||
|
||||
## Customizing Generation
|
||||
|
||||
You can customize various parameters when generating videos using the `SamplingParam` class:
|
||||
|
||||
```python
|
||||
from fastvideo import VideoGenerator, SamplingParam
|
||||
|
||||
def main():
|
||||
# Create the generator
|
||||
generator = VideoGenerator.from_pretrained(
|
||||
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
|
||||
num_gpus=1,
|
||||
)
|
||||
|
||||
# Create and customize sampling parameters
|
||||
sampling_param = SamplingParam.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers")
|
||||
|
||||
# How many frames to generate
|
||||
sampling_param.num_frames = 45
|
||||
|
||||
# Video resolution (width, height)
|
||||
sampling_param.width = 1024
|
||||
sampling_param.height = 576
|
||||
|
||||
# How many steps we denoise the video (higher = better quality, slower generation)
|
||||
sampling_param.num_inference_steps = 30
|
||||
|
||||
# How strongly the video conforms to the prompt (higher = more faithful to prompt)
|
||||
sampling_param.guidance_scale = 7.5
|
||||
|
||||
# Random seed for reproducibility
|
||||
sampling_param.seed = 42 # Optional, leave unset for random results
|
||||
|
||||
# Generate video with custom parameters
|
||||
prompt = "A beautiful sunset over a calm ocean, with gentle waves."
|
||||
video = generator.generate_video(
|
||||
prompt,
|
||||
sampling_param=sampling_param,
|
||||
output_path="my_videos/", # Controls where videos are saved
|
||||
return_frames=True, # Also return frames from this call (defaults to False)
|
||||
save_video=True
|
||||
)
|
||||
|
||||
# If return_frames=True, video contains the generated frames as a NumPy array
|
||||
print(f"Generated {len(video)} frames")
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
```
|
||||
|
||||
## Available Models
|
||||
|
||||
Please see the [support matrix](#support-matrix) for the list of supported models and their available optimizations.
|
||||
|
||||
## Multi-GPU Setup
|
||||
|
||||
FastVideo automatically distributes the generation process when multiple GPUs are specified:
|
||||
|
||||
```python
|
||||
# Will use 4 GPUs in parallel for faster generation
|
||||
generator = VideoGenerator.from_pretrained(
|
||||
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
|
||||
num_gpus=4,
|
||||
)
|
||||
```
|
||||
|
||||
## Image-to-Video Generation
|
||||
|
||||
You can generate a video starting from an initial image:
|
||||
@@ -159,53 +92,6 @@ prompt = "A photograph coming to life with gentle movement"
|
||||
video = generator.generate_video(prompt, sampling_param=sampling_param)
|
||||
```
|
||||
|
||||
## Output Formats and Options
|
||||
|
||||
FastVideo provides several options for saving and manipulating the generated videos:
|
||||
|
||||
```python
|
||||
from fastvideo import VideoGenerator, SamplingParam, VideoFormat
|
||||
|
||||
# Create the generator
|
||||
generator = VideoGenerator.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers")
|
||||
|
||||
# Generate with custom output options
|
||||
video = generator.generate_video(
|
||||
"Drone footage of a tropical rainforest",
|
||||
output_path="videos/rainforest/", # Custom save location
|
||||
filename="rainforest_flyover", # Custom filename (no extension needed)
|
||||
format=VideoFormat.MP4, # Output format: MP4, GIF, or WEBM
|
||||
fps=30, # Frames per second for the output video
|
||||
loop=False, # Whether to loop the video (for GIF)
|
||||
save_frames=True, # Also save individual frames as images
|
||||
)
|
||||
```
|
||||
|
||||
## Performance Optimization
|
||||
|
||||
Optimize FastVideo for your hardware:
|
||||
|
||||
```python
|
||||
from fastvideo import VideoGenerator, PipelineConfig
|
||||
|
||||
# Create an optimized configuration
|
||||
config = PipelineConfig.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers")
|
||||
|
||||
# Memory optimization
|
||||
config.enable_vae_slicing = True # Process VAE in slices to reduce memory
|
||||
config.enable_model_cpu_offload = True # Offload models to CPU when not in use
|
||||
|
||||
# Speed optimization
|
||||
config.vae_config.precision = "fp16" # Use half precision for VAE
|
||||
config.enable_xformers = True # Use xformers for attention computation
|
||||
|
||||
# Create generator with optimized config
|
||||
generator = VideoGenerator.from_pretrained(
|
||||
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
|
||||
pipeline_config=config,
|
||||
)
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
Common issues and their solutions:
|
||||
@@ -232,30 +118,7 @@ If the generated video doesn't match your prompt:
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
For advanced customization, use the `PipelineConfig` class:
|
||||
|
||||
```python
|
||||
from fastvideo import VideoGenerator, PipelineConfig
|
||||
|
||||
# Load the default configuration for a model
|
||||
config = PipelineConfig.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers")
|
||||
|
||||
# Modify configuration settings
|
||||
config.vae_config.scale_factor = 16
|
||||
config.vae_config.precision = "fp16"
|
||||
config.scheduler_config.beta_schedule = "linear"
|
||||
config.attention_config.attention_type = "xformers"
|
||||
|
||||
# Create generator with custom config
|
||||
generator = VideoGenerator.from_pretrained(
|
||||
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
|
||||
pipeline_config=config,
|
||||
)
|
||||
|
||||
# Generate video
|
||||
prompt = "A futuristic cityscape with flying cars and neon signs."
|
||||
video = generator.generate_video(prompt)
|
||||
```
|
||||
## Optimizations
|
||||
|
||||
## Next Steps
|
||||
|
||||
|
||||
@@ -0,0 +1,147 @@
|
||||
# Optimizations
|
||||
|
||||
This page describes the various options for speeding up generation times in FastVideo.
|
||||
|
||||
## Table of Contents
|
||||
- Optimized Attention Backends
|
||||
- [Flash Attention](#optimizations-flash)
|
||||
- [Sliding Tile Attention](#optimizations-sta)
|
||||
- [Sage Attention](#optimizations-sage)
|
||||
|
||||
- Caching Techniques
|
||||
- [TeaCache](#optimizations-teacache)
|
||||
|
||||
(optimizations-backends)=
|
||||
## Attention Backends
|
||||
|
||||
### Available Backends
|
||||
- Torch SDPA: `FASTVIDEO_ATTENTION_BACKEND=TORCH_SDPA`
|
||||
- Flash Attention 2 and 3: `FASTVIDEO_ATTENTION_BACKEND=FLASH_ATTN`
|
||||
- Sliding Tile Attention: `FASTVIDEO_ATTENTION_BACKEND=SLIDING_TILE_ATTN`
|
||||
- Sage Attention: `FASTVIDEO_ATTENTION_BACKEND=SAGE_ATTN`
|
||||
|
||||
### Configuring Backends
|
||||
|
||||
There are two ways to configure the attention backend in FastVideo.
|
||||
|
||||
#### 1. In Python
|
||||
In python, set the `FASTVIDEO_ATTENTION_BACKEND` environment variable before instantiating `VideoGenerator` like this:
|
||||
|
||||
```python
|
||||
os.environ["FASTVIDEO_ATTENTION_BACKEND"] = "SLIDING_TILE_ATTN"
|
||||
```
|
||||
|
||||
#### 2. In CLI
|
||||
You can also set the environment variable on the command line:
|
||||
|
||||
```bash
|
||||
FASTVIDEO_ATTENTION_BACKEND=SAGE_ATTN python example.py
|
||||
```
|
||||
|
||||
(optimizations-flash)=
|
||||
### Flash Attention
|
||||
|
||||
**`FLASH_ATTN`**
|
||||
|
||||
We recommend always installing [Flash Attention 2](https://github.com/Dao-AILab/flash-attention):
|
||||
|
||||
```bash
|
||||
pip install flash-attn==2.7.4.post1 --no-build-isolation
|
||||
```
|
||||
|
||||
And if using a Hopper+ GPU (ie H100), installing [Flash Attention 3](https://github.com/Dao-AILab/flash-attention?tab=readme-ov-file#flashattention-3-beta-release) by compiling it from source (takes about 10 minutes for me):
|
||||
|
||||
```bash
|
||||
git clone https://github.com/Dao-AILab/flash-attention.git && cd flash-attention
|
||||
|
||||
cd hopper
|
||||
pip install ninja
|
||||
python setup.py install
|
||||
```
|
||||
|
||||
:::{note}
|
||||
FastVideo will automatically detect and use `FA3` if it is installed when using `FLASH_ATTN` backend.
|
||||
:::
|
||||
|
||||
(optimizations-sta)=
|
||||
### Sliding Tile Attention
|
||||
**`SLIDING_TILE_ATTN`**
|
||||
|
||||
```bash
|
||||
pip install st_attn==0.0.4
|
||||
```
|
||||
|
||||
Please see [this page](#sta-installation) for more installation instructions.
|
||||
|
||||
(optimizations-sage)=
|
||||
### Sage Attention
|
||||
**`SAGE_ATTN`**
|
||||
|
||||
To use [SageAttention](https://github.com/thu-ml/SageAttention) 2.1.1, please compile from source:
|
||||
|
||||
```bash
|
||||
git clone https://github.com/thu-ml/SageAttention.git
|
||||
cd sageattention
|
||||
python setup.py install # or pip install -e .
|
||||
```
|
||||
|
||||
(optimizations-teacache)=
|
||||
## Teacache
|
||||
TeaCache is an optimization technique supported in FastVideo that can significantly speed up video generation by skipping redundant calculations across diffusion steps. This guide explains how to enable and configure TeaCache for optimal performance in FastVideo.
|
||||
|
||||
### What is TeaCache?
|
||||
|
||||
See the official [TeaCache](https://github.com/ali-vilab/TeaCache) repo and their paper for more details.
|
||||
|
||||
### How to Enable TeaCache
|
||||
|
||||
Enabling TeaCache is straightforward - simply add the `enable_teacache=True` parameter to your `generate_video()` call:
|
||||
|
||||
```python
|
||||
# ... previous code
|
||||
generator.generate_video(
|
||||
prompt="Your prompt here",
|
||||
sampling_param=params,
|
||||
enable_teacache=True
|
||||
)
|
||||
# more code ...
|
||||
```
|
||||
|
||||
### Complete Example
|
||||
|
||||
At the bottom is a complete example of using TeaCache for faster video generation. You can run it using the following command:
|
||||
|
||||
```bash
|
||||
python examples/inference/optimizations/teacache_example.py
|
||||
```
|
||||
|
||||
### Advanced Configuration
|
||||
|
||||
While TeaCache works well with default settings, you can fine-tune its behavior by adjusting the threshold value:
|
||||
|
||||
1. Lower threshold values (e.g., 0.1) will result in more skipped calculations and faster generation with slightly more potential for quality degradation
|
||||
2. Higher threshold values (e.g., 0.15-0.23) will skip fewer calculations but maintain quality closer to the original
|
||||
|
||||
Note that the optimal threshold depends on your specific model and content.
|
||||
|
||||
## Benchmarking different optimizations
|
||||
|
||||
To benchmark the performance improvement, try generating the same video with and without TeaCache enabled and compare the generation times:
|
||||
|
||||
```python
|
||||
# Without TeaCache
|
||||
start_time = time.perf_counter()
|
||||
generator.generate_video(prompt="Your prompt", enable_teacache=False)
|
||||
standard_time = time.perf_counter() - start_time
|
||||
|
||||
# With TeaCache
|
||||
start_time = time.perf_counter()
|
||||
generator.generate_video(prompt="Your prompt", enable_teacache=True)
|
||||
teacache_time = time.perf_counter() - start_time
|
||||
|
||||
print(f"Standard generation: {standard_time:.2f} seconds")
|
||||
print(f"TeaCache generation: {teacache_time:.2f} seconds")
|
||||
print(f"Speedup: {standard_time/teacache_time:.2f}x")
|
||||
```
|
||||
|
||||
Note: If you want to benchmark different attention backends, you'll need to reinstantiate `VideoGenerator`.
|
||||
@@ -1,15 +1,87 @@
|
||||
(support-matrix)=
|
||||
# Support Matrix
|
||||
# Compatibility Matrix
|
||||
The table below shows every supported model and optimizations supported for them.
|
||||
|
||||
| Model Name | HuggingFace Model ID | Resolutions | TeaCache| Sliding Tile Attn | Sage Attn |
|
||||
|------------|----------------------|-------------|-------------|--------------------------|--|
|
||||
| HunyuanVideo | hunyuanvideo-community/HunyuanVideo | | | ✅ | ✅ |
|
||||
| FastHunyuan | FastVideo/FastHunyuan-diffusers | | | ✅| ✅|
|
||||
| Wan T2V 1.4B | Wan-AI/Wan2.1-T2V-1.3B-Diffusers | 480P | ✅ | ✅ | ✅ |
|
||||
| Wan T2V 14B | Wan-AI/Wan2.1-T2V-1.3B-Diffusers | 480P, 720P | ✅| ✅| ✅|
|
||||
| Wan I2V 480P | Wan-AI/Wan2.1-I2V-14B-480P-Diffusers | 480P | ✅| ✅| ✅|
|
||||
| Wan T2V 720P | Wan-AI/Wan2.1-T2V-14B-Diffusers | 720P | ✅| ✅| ✅|
|
||||
| StepVideo T2V | Coming soon! | 768px768px204f, 544px992px204f, 544px992px136f | | | |
|
||||
The symbols used have the following meanings:
|
||||
|
||||
- ✅ = Full compatibility
|
||||
- ❌ = No compatibility
|
||||
|
||||
## Models x Optimization
|
||||
The `HuggingFace Model ID` can be directly pass to `from_pretrained()` methods and FastVideo will use the optimal default parameters when initializing and generating videos.
|
||||
|
||||
:::{raw} html
|
||||
<style>
|
||||
/* Make smaller to try to improve readability */
|
||||
td {
|
||||
font-size: 0.9rem;
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
th {
|
||||
text-align: center;
|
||||
font-size: 0.9rem;
|
||||
}
|
||||
</style>
|
||||
:::
|
||||
|
||||
:::{list-table}
|
||||
:header-rows: 1
|
||||
:stub-columns: 3
|
||||
:widths: auto
|
||||
:class: vertical-table-header
|
||||
|
||||
- * Model Name
|
||||
* HuggingFace Model ID
|
||||
* Resolutions
|
||||
* TeaCache
|
||||
* Sliding Tile Attn
|
||||
* Sage Attn
|
||||
- * HunyuanVideo
|
||||
* `hunyuanvideo-community/HunyuanVideo`
|
||||
* 720px1280p<br>544px960p
|
||||
* ❌
|
||||
* ✅
|
||||
* ✅
|
||||
- * FastHunyuan
|
||||
* `FastVideo/FastHunyuan-diffusers`
|
||||
* 720px1280p<br>544px960p
|
||||
* ❌
|
||||
* ✅
|
||||
* ✅
|
||||
- * Wan T2V 1.4B
|
||||
* `Wan-AI/Wan2.1-T2V-1.3B-Diffusers`
|
||||
* 480P
|
||||
* ✅
|
||||
* ✅*
|
||||
* ✅
|
||||
- * Wan T2V 14B
|
||||
* `Wan-AI/Wan2.1-T2V-1.3B-Diffusers`
|
||||
* 480P, 720P
|
||||
* ✅
|
||||
* ✅*
|
||||
* ✅
|
||||
- * Wan I2V 480P
|
||||
* `Wan-AI/Wan2.1-I2V-14B-480P-Diffusers`
|
||||
* 480P
|
||||
* ✅
|
||||
* ✅*
|
||||
* ✅
|
||||
- * Wan T2V 720P
|
||||
* `Wan-AI/Wan2.1-T2V-14B-Diffusers`
|
||||
* 720P
|
||||
* ✅
|
||||
* ✅*
|
||||
* ✅
|
||||
- * StepVideo T2V
|
||||
* Coming soon!
|
||||
* 768px768px204f<br>544px992px204f<br>544px992px136f
|
||||
*
|
||||
*
|
||||
*
|
||||
:::
|
||||
|
||||
**Note**: there are some known quality issues with Wan2.1 + Sliding Tile Attn. We are working on fixing this issue.
|
||||
|
||||
## Special requirements
|
||||
|
||||
|
||||
Reference in New Issue
Block a user