This commit is contained in:
John Pollock
2025-01-28 06:26:25 -06:00
2 changed files with 10 additions and 4 deletions
+9 -3
View File
@@ -1,10 +1,16 @@
# ComfyUI-MultiGPU
## Advanced user nodes for using multiple GPUs as well as offloading model components to the CPU in a single ComfyUI workflow
## Device selection and model offloading tools for ComfyUI workloads that exceed single GPU capacity
This extension adds device selection capabilities to model loading nodes in ComfyUI. It monkey patches the memory management of ComfyUI in a hacky way and is neither a comprehensive solution is nor is it well-tested on any edge-case CUDA/CPU solutions. **Use at your own risk.**
This extension empowers users to spread model components across multiple GPUs or offload to CPU in a single ComfyUI workflow. Aimed at complex workloads that would normally require sequential model loading/unloading:
*Note: This does not add parallelism. The workflow steps are still executed sequentially just with model components loaded on different GPUs or offloaded to the CPU where allowed. Any potential speedup comes from not having to constantly load and unload models from VRAM.*
- Full control over model placement
- Support for all major model loader nodes
- NEW! Hunyuan Video UNet model splitting across two devices using DiffSynth block-swapping
*EXPERIMENTAL: This extension modifies ComfyUI's memory management behavior. While functional for standard GPU and CPU configurations, edge cases may exist. Use at your own risk.*
**Note:** This enhances memory management, not parallelism. Workflow steps execute sequentially but with components loaded across your specified devices. Performance gains come from avoiding repeated model loading/unloading when VRAM is constrained.
# NEW: DisTorch - Advanced GGUF-Quantized Model Layer Distribution
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "comfyui-multigpu"
description = "This extension adds CUDA/CPU device selection to supported loader nodes in ComfyUI. By monkey-patching ComfyUI’s memory management, each model component (like UNet, Clip, or VAE) can be loaded on a specific GPU. Examples included are multi-GPU workflows for SDXL, FLUX, LTXVideo, and Hunyuan Video for both standard and GGUF loader nodes."
description = "This extension adds CUDA/CPU device selection to supported loader nodes in ComfyUI. Using these nodes, each model component (like UNet, Clip, or VAE) can be loaded on a specific GPU. Examples included are multi-GPU workflows for SDXL, FLUX, LTXVideo, and Hunyuan Video for both standard and GGUF loader nodes."
version = "1.4.0"
license = {file = "LICENSE"}