8530fdc550
Co-authored-by: Will Lin <willlin@nvidia.com> Co-authored-by: William Lin <8941107+SolitaryThinker@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
148 lines
4.7 KiB
Markdown
148 lines
4.7 KiB
Markdown
# Configuration
|
|
|
|
## Multi-GPU Setup
|
|
|
|
FastVideo automatically distributes the generation process when multiple GPUs are specified:
|
|
|
|
```python
|
|
# Will use 4 GPUs in parallel for faster generation
|
|
generator = VideoGenerator.from_pretrained(
|
|
"Wan-AI/Wan2.1-T2V-1.3B-Diffusers",
|
|
num_gpus=4,
|
|
)
|
|
```
|
|
|
|
One node uses the multiprocessing executor (`execution_backend: mp`, the
|
|
default). With `num_gpus=1`, `mp` automatically selects the in-process (`uni`)
|
|
executor so weights load in the current process. Set `execution_backend: uni`
|
|
to force that path. Two machines — for example two DGX Sparks, one GPU each —
|
|
need Ray:
|
|
|
|
```yaml
|
|
generator:
|
|
engine:
|
|
num_gpus: 2
|
|
execution_backend: ray
|
|
parallelism:
|
|
sp_size: 2
|
|
```
|
|
|
|
Set `RAY_ADDRESS` and `FASTVIDEO_HOST_IP` to the interconnect IPs, not Wi-Fi.
|
|
The FastH3 example selects Ray automatically when `RAY_ADDRESS` is set. Full
|
|
bring-up: [Pair two NVIDIA DGX Sparks](../getting_started/installation/spark_pair.md).
|
|
|
|
## Customizing Generation
|
|
|
|
- `PipelineConfig`: Initialization time parameters
|
|
- `SamplingParam`: Generation time parameters
|
|
|
|
You can customize generation behavior using `PipelineConfig` and
|
|
`SamplingParam`:
|
|
|
|
```python
|
|
from fastvideo import VideoGenerator, SamplingParam, PipelineConfig
|
|
|
|
def main():
|
|
model_name = "Wan-AI/Wan2.1-T2V-1.3B-Diffusers"
|
|
config = PipelineConfig.from_pretrained(model_name)
|
|
config.vae_precision = "fp16"
|
|
|
|
# Create the generator
|
|
generator = VideoGenerator.from_pretrained(
|
|
model_name,
|
|
num_gpus=1,
|
|
dit_layerwise_offload=True, # FastVideoArgs option
|
|
pipeline_config=config
|
|
)
|
|
|
|
# Create and customize sampling parameters
|
|
sampling_param = SamplingParam.from_pretrained("Wan-AI/Wan2.1-T2V-1.3B-Diffusers")
|
|
|
|
# How many frames to generate
|
|
sampling_param.num_frames = 45
|
|
|
|
# Video resolution (width, height)
|
|
sampling_param.width = 1024
|
|
sampling_param.height = 576
|
|
|
|
# How many steps we denoise the video (higher = better quality, slower generation)
|
|
sampling_param.num_inference_steps = 30
|
|
|
|
# How strongly the video conforms to the prompt (higher = more faithful to prompt)
|
|
sampling_param.guidance_scale = 7.5
|
|
|
|
# Random seed for reproducibility
|
|
sampling_param.seed = 42 # Optional, leave unset for random results
|
|
|
|
# Generate video with custom parameters
|
|
prompt = "A beautiful sunset over a calm ocean, with gentle waves."
|
|
video = generator.generate_video(
|
|
prompt,
|
|
sampling_param=sampling_param,
|
|
output_path="my_videos/", # Controls where videos are saved
|
|
save_video=True
|
|
)
|
|
|
|
# If return_frames=True, frames are available in video["frames"]
|
|
print(f"Generated {len(video['frames'])} frames")
|
|
|
|
if __name__ == '__main__':
|
|
main()
|
|
```
|
|
|
|
## JSON/YAML Config Files (CLI)
|
|
|
|
The inference CLI is config-first. Use an explicit subcommand with `--config`,
|
|
then apply optional dotted overrides on top, matching the training CLI style.
|
|
By default, CLI generation uses `return_frames=false` unless you set
|
|
`request.output.return_frames: true` in config or via a dotted override.
|
|
|
|
```bash
|
|
fastvideo generate --config config.yaml
|
|
```
|
|
|
|
Example nested config:
|
|
|
|
```yaml
|
|
generator:
|
|
model_path: FastVideo/FastHunyuan-diffusers
|
|
engine:
|
|
num_gpus: 2
|
|
parallelism:
|
|
sp_size: 2
|
|
request:
|
|
prompt: A capybara relaxing in a hammock
|
|
sampling:
|
|
num_frames: 45
|
|
height: 720
|
|
width: 1280
|
|
num_inference_steps: 6
|
|
seed: 1024
|
|
output:
|
|
output_path: outputs/
|
|
```
|
|
|
|
`generator.engine.execution_backend` accepts `mp`, `uni`, `ray`, or
|
|
`external_launcher`. The last option is for synchronized offline generation
|
|
under `torchrun` or `srun`; it is not supported by `fastvideo serve`. Set
|
|
`generator.engine.num_gpus` and the relevant parallelism sizes to the total
|
|
launcher world size. See the [distributed inference contract](architecture.md#distributed-inference)
|
|
and the [MiniMax-H3 run config](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/minimax_h3_external_launcher.yaml).
|
|
|
|
`generator.engine.parallelism.dist_timeout` is an optional positive integer
|
|
in **seconds**. FastVideo applies it to the default process group when
|
|
FastVideo initializes that group, and to every FastVideo-created device and
|
|
Gloo control group. If it is omitted, PyTorch's backend defaults apply. This
|
|
timeout bounds a missing collective participant; external launchers should
|
|
still supervise the full worker group and terminate peers when one rank exits.
|
|
|
|
Override individual values from the CLI with dotted paths:
|
|
|
|
```bash
|
|
fastvideo generate --config config.yaml --request.sampling.seed 42
|
|
```
|
|
|
|
## Performance Optimization
|
|
|
|
For configuring optimizations, please see our [optimizations guide](optimizations.md)
|