[V1] Docs Update (#402)
This commit is contained in:
@@ -19,11 +19,11 @@ With FastVideo's optimizations, you can achieve more than 3x inference improveme
|
||||
|
||||
FastVideo has the following features:
|
||||
- State-of-the-art performance optimizations for inference
|
||||
- [Sliding Tile Attention](https://hao-ai-lab.github.io/blogs/sta/).
|
||||
- TeaCache
|
||||
- Sage Attention
|
||||
- [Sliding Tile Attention](https://arxiv.org/pdf/2502.04507)
|
||||
- [TeaCache](https://arxiv.org/pdf/2411.19108)
|
||||
- [Sage Attention](https://arxiv.org/abs/2410.02367)
|
||||
- Cutting edge models
|
||||
- Wan2.1 T2V, I2v
|
||||
- Wan2.1 T2V, I2V
|
||||
- HunyuanVideo
|
||||
- FastHunyuan: consistency distilled video diffusion models for 8x inference speedup.
|
||||
- StepVideo T2V
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 311 KiB After Width: | Height: | Size: 303 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 311 KiB After Width: | Height: | Size: 303 KiB |
@@ -32,11 +32,11 @@ With FastVideo's optimizations, you can achieve more than 3x inference improveme
|
||||
|
||||
FastVideo has the following features:
|
||||
- State-of-the-art performance optimizations for inference
|
||||
- [Sliding Tile Attention](https://hao-ai-lab.github.io/blogs/sta/).
|
||||
- TeaCache
|
||||
- Sage Attention
|
||||
- [Sliding Tile Attention](https://arxiv.org/pdf/2502.04507)
|
||||
- [TeaCache](https://arxiv.org/pdf/2411.19108)
|
||||
- [Sage Attention](https://arxiv.org/abs/2410.02367)
|
||||
- Cutting edge models
|
||||
- Wan2.1 T2V, I2v
|
||||
- Wan2.1 T2V, I2V
|
||||
- HunyuanVideo
|
||||
- FastHunyuan: consistency distilled video diffusion models for 8x inference speedup.
|
||||
- StepVideo T2V
|
||||
|
||||
@@ -0,0 +1,77 @@
|
||||
(inference-configuration)=
|
||||
# Configuration
|
||||
|
||||
## Multi-GPU Setup
|
||||
|
||||
FastVideo automatically distributes the generation process when multiple GPUs are specified:
|
||||
|
||||
```python
|
||||
# Will use 4 GPUs in parallel for faster generation
|
||||
generator = VideoGenerator.from_pretrained(
|
||||
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
|
||||
num_gpus=4,
|
||||
)
|
||||
```
|
||||
|
||||
## Customizing Generation
|
||||
|
||||
- `PipelineConfig`: Initialization time parameters
|
||||
- `SamplingParam`: Generation time parameters
|
||||
|
||||
You can customize various parameters when generating videos using the `PipelineConfig` and `SamplingParam` class:
|
||||
|
||||
```python
|
||||
from fastvideo import VideoGenerator, SamplingParam, PipelineConfig
|
||||
|
||||
def main():
|
||||
model_name = "Wan-AI/Wan2.1-T2V-1.3B-Diffusers"
|
||||
config = PipelineConfig.from_pretrained(model_name)
|
||||
config.vae_precision = "fp16"
|
||||
config.use_cpu_offload = True
|
||||
|
||||
# Create the generator
|
||||
generator = VideoGenerator.from_pretrained(
|
||||
model_name,
|
||||
num_gpus=1,
|
||||
pipeline_config=config
|
||||
)
|
||||
|
||||
# Create and customize sampling parameters
|
||||
sampling_param = SamplingParam.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers")
|
||||
|
||||
# How many frames to generate
|
||||
sampling_param.num_frames = 45
|
||||
|
||||
# Video resolution (width, height)
|
||||
sampling_param.width = 1024
|
||||
sampling_param.height = 576
|
||||
|
||||
# How many steps we denoise the video (higher = better quality, slower generation)
|
||||
sampling_param.num_inference_steps = 30
|
||||
|
||||
# How strongly the video conforms to the prompt (higher = more faithful to prompt)
|
||||
sampling_param.guidance_scale = 7.5
|
||||
|
||||
# Random seed for reproducibility
|
||||
sampling_param.seed = 42 # Optional, leave unset for random results
|
||||
|
||||
# Generate video with custom parameters
|
||||
prompt = "A beautiful sunset over a calm ocean, with gentle waves."
|
||||
video = generator.generate_video(
|
||||
prompt,
|
||||
sampling_param=sampling_param,
|
||||
output_path="my_videos/", # Controls where videos are saved
|
||||
return_frames=True, # Also return frames from this call (defaults to False)
|
||||
save_video=True
|
||||
)
|
||||
|
||||
# If return_frames=True, video contains the generated frames as a NumPy array
|
||||
print(f"Generated {len(video)} frames")
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
```
|
||||
|
||||
## Performance Optimization
|
||||
|
||||
For configuring optimizations, please see our [optimizations guide](#inference-optimizations)
|
||||
@@ -2,19 +2,11 @@
|
||||
|
||||
This page contains step-by-step instructions to get you quickly started with video generation using FastVideo.
|
||||
|
||||
## Table of Contents
|
||||
- [Generating Your First Video](#generating-your-first-video)
|
||||
- [Customizing Generation](#customizing-generation)
|
||||
- [Available Models](#available-models)
|
||||
- [Image-to-Video Generation](#image-to-video-generation)
|
||||
- [Troubleshooting](#troubleshooting)
|
||||
- [Advanced Configuration](#advanced-configuration)
|
||||
- [Next Steps](#next-steps)
|
||||
|
||||
## Software Requirements
|
||||
## Requirements
|
||||
- **OS**: Linux (Tested on Ubuntu 22.04+)
|
||||
- **Python**: 3.10-3.12
|
||||
- **CUDA**: 12.4
|
||||
- **GPU**: At least one NVIDIA GPU
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -78,18 +70,25 @@ You can generate a video starting from an initial image:
|
||||
```python
|
||||
from fastvideo import VideoGenerator, SamplingParam
|
||||
|
||||
# Create the generator
|
||||
generator = VideoGenerator.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers")
|
||||
def main():
|
||||
# Create the generator
|
||||
model_name = "Wan-AI/Wan2.1-I2V-14B-480P-Diffusers"
|
||||
generator = VideoGenerator.from_pretrained(model_name, num_gpus=1)
|
||||
|
||||
# Set up parameters with an initial image
|
||||
sampling_param = SamplingParam.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers")
|
||||
sampling_param.image_path = "path/to/your/image.jpg"
|
||||
sampling_param.num_frames = 24
|
||||
sampling_param.image_strength = 0.8 # How much to preserve the original image (0-1)
|
||||
# Set up parameters with an initial image
|
||||
sampling_param = SamplingParam.from_pretrained(model_name)
|
||||
sampling_param.image_path = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/astronaut.jpg"
|
||||
sampling_param.num_frames = 107
|
||||
sampling_param.image_strength = 0.8 # How much to preserve the original image (0-1)
|
||||
|
||||
# Generate video based on the image
|
||||
prompt = "A photograph coming to life with gentle movement"
|
||||
video = generator.generate_video(prompt, sampling_param=sampling_param)
|
||||
# Generate video based on the image
|
||||
prompt = "A photograph coming to life with gentle movement"
|
||||
generator.generate_video(prompt, sampling_param=sampling_param,
|
||||
output_path="my_videos/",
|
||||
save_video=True)
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
@@ -100,7 +99,7 @@ Common issues and their solutions:
|
||||
If you encounter CUDA out of memory errors:
|
||||
- Reduce `num_frames` or video resolution
|
||||
- Enable memory optimization with `enable_model_cpu_offload`
|
||||
- Try a smaller model or use quantized versions
|
||||
- Try a smaller model or use distilled versions
|
||||
- Use `num_gpus` > 1 if multiple GPUs are available
|
||||
|
||||
### Slow Generation
|
||||
@@ -116,14 +115,10 @@ If the generated video doesn't match your prompt:
|
||||
- Experiment with different random seeds
|
||||
- Try a different model
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
## Optimizations
|
||||
|
||||
## Next Steps
|
||||
|
||||
- Explore the [API Reference](../api/index.md) for detailed documentation
|
||||
- Learn about [Advanced Inference Options](../inference/overview_back.md)
|
||||
- See [Examples](../examples/index.md) for more usage scenarios
|
||||
- Check out the [Model Training](../training/overview.md) guide to fine-tune models
|
||||
- Join our [Community Discord](https://discord.gg/fastvideo) for support and sharing
|
||||
- Learn about [Advanced Inference Configurations](#inference-configuration)
|
||||
- Learn about using [Optimizations](#inference-optimizations)
|
||||
- See [Examples](../examples/examples_inference_index.md) for more usage scenarios
|
||||
- Join our [Community Discord](https://discord.gg/JA7cksDz86).
|
||||
- Join our [Community Slack](https://join.slack.com/t/fastvideo/shared_invite/zt-2zf6ru791-sRwI9lPIUJQq1mIeB_yjJg).
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
(inference-optimizations)=
|
||||
# Optimizations
|
||||
|
||||
This page describes the various options for speeding up generation times in FastVideo.
|
||||
|
||||
Reference in New Issue
Block a user