From 989b3dc4c304085a7283c4cb1a737b2a4d75d210 Mon Sep 17 00:00:00 2001 From: John Pollock Date: Tue, 30 Sep 2025 18:10:51 -0500 Subject: [PATCH] additional standard loaders documentation --- ...eckpointLoaderAdvancedDisTorch2MultiGPU.md | 55 +++++++++++++++++++ web/docs/CheckpointLoaderAdvancedMultiGPU.md | 22 ++++++++ 2 files changed, 77 insertions(+) create mode 100644 web/docs/CheckpointLoaderAdvancedDisTorch2MultiGPU.md create mode 100644 web/docs/CheckpointLoaderAdvancedMultiGPU.md diff --git a/web/docs/CheckpointLoaderAdvancedDisTorch2MultiGPU.md b/web/docs/CheckpointLoaderAdvancedDisTorch2MultiGPU.md new file mode 100644 index 0000000..faae6d0 --- /dev/null +++ b/web/docs/CheckpointLoaderAdvancedDisTorch2MultiGPU.md @@ -0,0 +1,55 @@ +# CheckpointLoaderAdvancedDisTorch2MultiGPU + +The `CheckpointLoaderAdvancedDisTorch2MultiGPU` node is used to load checkpoint models with advanced DisTorch2 distributed tensor allocation, providing granular control over UNet, CLIP, and VAE component allocation across multiple devices with independent virtual VRAM management. + +This node automatically detects models located in the `ComfyUI/models/checkpoints` folder, and it will also read models from additional paths configured in the `extra_model_paths.yaml` file. Sometimes, you may need to **refresh the ComfyUI interface** to allow it to read the model files from the corresponding folder. + +## Inputs + +| Parameter | Data Type | Description | +| --- | --- | --- | +| `ckpt_name` | `STRING` | The name of the checkpoint model to load. | +| `unet_compute_device` | `STRING` | Target compute device for UNet distributed allocation (e.g., 'cuda:0', 'cuda:1', 'cpu'). | +| `unet_virtual_vram_gb` | `FLOAT` | Amount of virtual VRAM in gigabytes for UNet component distributed allocation (default: 4.0, range: 0.0-128.0). | +| `unet_donor_device` | `STRING` | Device to donate VRAM from when allocating UNet virtual memory (default: 'cpu'). | +| `clip_compute_device` | `STRING` | Target compute device for CLIP distributed allocation (default: 'cpu'). | +| `clip_virtual_vram_gb` | `FLOAT` | Amount of virtual VRAM in gigabytes for CLIP component distributed allocation (default: 2.0, range: 0.0-128.0). | +| `clip_donor_device` | `STRING` | Device to donate VRAM from when allocating CLIP virtual memory (default: 'cpu'). | +| `vae_device` | `STRING` | Target device for the VAE component (e.g., 'cuda:0', 'cuda:1', 'cpu'). | +| `unet_expert_mode_allocations` | `STRING` | Advanced UNet allocation string for expert device/ratio distributions. | +| `clip_expert_mode_allocations` | `STRING` | Advanced CLIP allocation string for expert device/ratio distributions. | +| `high_precision_loras` | `BOOLEAN` | Whether to use high-precision LoRA patches (default: true). | + +## Outputs + +| Output Name | Data Type | Description | +| --- | --- | --- | +| `MODEL` | `MODEL` | The loaded UNet diffusion model with DisTorch2 distributed allocation. | +| `CLIP` | `CLIP` | The loaded CLIP text encoder model with DisTorch2 distributed allocation. | +| `VAE` | `VAE` | The loaded VAE decoder/encoder model. | + +## DisTorch2 Distributed Loading + +DisTorch2 is an advanced memory management system that enables loading and running large diffusion models across multiple GPUs by intelligently distributing tensor allocations. Instead of loading an entire model on a single device, DisTorch2 splits the model's layers across available devices while maintaining computational efficiency. + +This advanced checkpoint loader provides independent DisTorch2 allocation control for UNet and CLIP components, while using standard device placement for VAE. + +### Key Concepts + +**Individual Component Control**: Each model component (UNet, CLIP, VAE) can have its own allocation strategy. + +**Virtual VRAM Allocation**: Artificially increases the available VRAM on compute devices by borrowing memory capacity from donor devices through intelligent tensor distribution. + +**Expert Mode Allocations**: Advanced users can manually specify exactly how much of each component should be placed on each device using ratio or byte-based allocation strings. + +### Allocation Examples + +**Basic Virtual VRAM Mode**: +- `unet_compute_device`: `cuda:0`, `unet_virtual_vram_gb`: `8.0`, `unet_donor_device`: `cuda:1` +- `clip_compute_device`: `cuda:1`, `clip_virtual_vram_gb`: `2.0`, `clip_donor_device`: `cpu` +- Result: UNet loads as if cuda:0 has 8GB more VRAM, CLIP loads with cuda:1 having 2GB more capacity. + +**Expert Ratio Allocation**: +- `unet_expert_mode_allocations`: `cuda:0,70%;cuda:1,30%` +- `clip_expert_mode_allocations`: `cuda:1,50%;cpu,50%` +- Distributes UNet with 70% on GPU 0, 30% on GPU 1, and CLIP with 50% on GPU 1, 50% on CPU. diff --git a/web/docs/CheckpointLoaderAdvancedMultiGPU.md b/web/docs/CheckpointLoaderAdvancedMultiGPU.md new file mode 100644 index 0000000..2673dcb --- /dev/null +++ b/web/docs/CheckpointLoaderAdvancedMultiGPU.md @@ -0,0 +1,22 @@ +# CheckpointLoaderAdvancedMultiGPU + +The `CheckpointLoaderAdvancedMultiGPU` node is used to load checkpoint models (complete diffusion models containing UNet, CLIP, and VAE components) with granular device control, allowing individual placement of each model component on different GPUs or devices. + +This node automatically detects models located in the `ComfyUI/models/checkpoints` folder, and it will also read models from additional paths configured in the `extra_model_paths.yaml` file. Sometimes, you may need to **refresh the ComfyUI interface** to allow it to read the model files from the corresponding folder. + +## Inputs + +| Parameter | Data Type | Description | +| --- | --- | --- | +| `ckpt_name` | `STRING` | The name of the checkpoint model to load. | +| `unet_device` | `STRING` | Target device for the UNet diffusion model component (e.g., 'cuda:0', 'cuda:1', 'cpu'). | +| `clip_device` | `STRING` | Target device for the CLIP text encoder component (e.g., 'cuda:0', 'cuda:1', 'cpu'). | +| `vae_device` | `STRING` | Target device for the VAE decoder/encoder component (e.g., 'cuda:0', 'cuda:1', 'cpu'). | + +## Outputs + +| Output Name | Data Type | Description | +| --- | --- | --- | +| `MODEL` | `MODEL` | The loaded UNet diffusion model. | +| `CLIP` | `CLIP` | The loaded CLIP text encoder model. | +| `VAE` | `VAE` | The loaded VAE decoder/encoder model. |