diff --git a/README.md b/README.md
index bc72e1c..06e8834 100644
--- a/README.md
+++ b/README.md
@@ -1,5 +1,12 @@
# ComfyUI-MultiGPU
+
+ 
+
+ Add Virtual VRAM and unleash the power of all of your latent space
+
+
+
## Device selection and model offloading tools for ComfyUI workloads that exceed single GPU capacity
This extension empowers users to spread model components across multiple GPUs or offload to CPU in a single ComfyUI workflow. Aimed at complex workloads that would normally require sequential model loading/unloading:
@@ -12,16 +19,63 @@ This extension empowers users to spread model components across multiple GPUs or
**Note:** This enhances memory management, not parallelism. Workflow steps execute sequentially but with components, or in the case of GGUF files `GGML` layers, loaded across your specified devices. *Performance gains* come from avoiding repeated model loading/unloading when VRAM is constrained. *Capability gains* come from offloading as much of the model (VAE/CLIP/UNet) off of your main `compute` device, allowing you to maximize the amount of latent space available for `compute`
-# NEW: DisTorch - Advanced GGUF-Quantized Model Layer Distribution
+**NEW** DisTorch 2.0: Virtual VRAM Made Simple
-DisTorch nodes are now available, allowing fine-grained control over model layer distribution across multiple devices for GGUF quantized models. Using a simple allocation string (e.g., "cuda:0,0.025;cuda:1,0.05;cpu,0.10"), you can precisely specify how much memory each device should contribute to hosting model layers. This enables sophisticated memory management strategies like:
+
+ 
+
+ DisTorch 2.0 in Action
+
-- Splitting large models across multiple GPUs with different VRAM capacities
-- Utilizing CPU memory alongside GPU VRAM for handling memory-intensive models
-- Optimizing layer placement based on your specific hardware configuration
-Check out the updated examples `hunyuan_ip2v_distorch_gguf.json`, `hunyuan_gguf_distorch.json`, and `flux1dev_gguf_distorch.json` to see DisTorch in action, demonstrating advanced layer distribution across multiple devices including adapting kijai's `HunyuanVideo TextImageEncode (IP2V)` node for the native HunyuanVideo sampler in Comfy Core - meaning HunyuanVideo with IP2V has come to GGUFs!
+## What's New?
+DisTorch now features simple Virtual VRAM control that lets you offload model layers from your GPU with zero configuration. Just set how much VRAM you want to free up, and DisTorch handles the rest.
+
+## How It Works
+- **Virtual VRAM**: Defaults to 4GB - just adjust it based on your needs
+- **Two Modes**:
+ - **Default**: Offloads to system RAM
+ - **Multi-GPU**: Distributes across other GPUs (optional)
+
+## 🎯 Key Benefits
+- Free up GPU VRAM instantly without complex settings
+- Run larger models by offloading layers to other system RAM
+- Use all your main GPU's VRAM for actual `compute` / latent processing
+- Seamlessly distribute GGML layers across multiple GPUs if available
+- Allows **you** to easily shift from ___on-device speed___ to ___open-device capability___ with a simple one-number change
+
+
+ 
+
+ DisTorch 2.0 Node with one simple number to tune its Vitual VRAM to your needs
+
+
+## đź’ˇ Quick Start
+1. Load any GGUF model using a DisTorch node
+2. Set your Virtual VRAM amount (default: 4GB, in this example we chose 8GB)
+3. Toggle "Use Other VRAM" if you have multiple GPUs[^1]
+4. That's it!
+[^1]: DisTorch's Virtual VRAM aims to span as few a devices as possible. I recommend users try both ways (VRAM/DRAM) and see which works best for you.
+
+## 🔄 Real-World Example
+With a 12GB GPU running an 8GB model:
+- Set Virtual VRAM to 4GB
+- DisTorch moves 4GB of model layers to RAM
+- Your GPU now has extra VRAM for larger batches, higher resolutions, or longer video
+
+## 🚀 Compatibility
+Works with all GGUF-quantized ComfyUI/ComfyUI-GGUF-supported UNet/CLIP models.
+
+⚙️ Expert users: For those of you who were here for the 1.0 release of DisTorch, manual allocation strings still available for advanced configurations. Each log will contain the allocation string for the run so it can be easily recreated and/or manipulated for more sophisticated setups.
+
+
+
+
+ 
+
+ The new Virtual VRAM even lets you offload ALL of the model and still run compute on your CUDA device!
+
## Installation
Installation via [ComfyUI-Manager](https://github.com/ltdrdata/ComfyUI-Manager) is preferred. Simply search for `ComfyUI-MultiGPU` in the list of nodes and follow installation instructions.
@@ -85,11 +139,6 @@ This workflow attaches a HunyuanVideo GGUF-quantized model on `cuda:0` for compu
- [examples/flux1dev_gguf_distorch.json](https://github.com/pollockjj/ComfyUI-MultiGPU/blob/main/examples/flux1dev_gguf_distorch.json)
This workflow loads a FLUX.1-dev model on `cuda:0` for compute and distrubutes its UNet across multiple CUDA devices using new DisTorch distributed-load methodology. While the text encoders and VAE are loaded on GPU 1 and use `cuda:1` for compute.
-### Split Hunyuan Video UNet across two devices and use DiffSynth Just-in-Time loading
-
-- [examples/hunyuanvideowrapper_diffsynth.json](https://github.com/pollockjj/ComfyUI-MultiGPU/blob/main/examples/hunyuanvideowrapper_diffsynth.json)
-This workflow demonstrates DiffSynth's memory optimization strategy enabled in kijai's `ComfyUI-HunyuanVideoWrapper` UNet loader, splitting the UNet model across two CUDA devices using block-swapping. The main device handles active computations while blocks are swapped to and from the offload device as needed. As written, the CLIP loads on cuda:0 and then offloads, and the VAE is loaded after the UNet model has been cleared from memory after generation. This approach enables processing of higher resolution or longer duration videos that would exceed a single GPU's memory capacity, though at the cost of additional processing time. Note that an initial OOM error is expected as the workflow calibrates its memory management strategy - simply run the generation again with the same parameters.
-
### Split Hunyuan Video generation across multiple resources
- [examples/hunyuanvideowrapper_native_vae.json](https://github.com/pollockjj/ComfyUI-MultiGPU/blob/main/examples/hunyuanvideowrapper_native_vae.json)
diff --git a/assets/distorch2_0.gif b/assets/distorch2_0.gif
new file mode 100755
index 0000000..ddf27c5
Binary files /dev/null and b/assets/distorch2_0.gif differ
diff --git a/assets/distorch2_0.png b/assets/distorch2_0.png
new file mode 100755
index 0000000..31ce659
Binary files /dev/null and b/assets/distorch2_0.png differ
diff --git a/assets/distorch_average.png b/assets/distorch_average.png
new file mode 100755
index 0000000..d5467e2
Binary files /dev/null and b/assets/distorch_average.png differ
diff --git a/assets/distorch_node.png b/assets/distorch_node.png
new file mode 100755
index 0000000..8c403e9
Binary files /dev/null and b/assets/distorch_node.png differ
diff --git a/examples/flux1dev_gguf_distorch.json b/examples/flux1dev_gguf_distorch.json
old mode 100644
new mode 100755
index 5bb0145..bec502f
--- a/examples/flux1dev_gguf_distorch.json
+++ b/examples/flux1dev_gguf_distorch.json
@@ -14,7 +14,7 @@
46
],
"flags": {},
- "order": 10,
+ "order": 8,
"mode": 0,
"inputs": [
{
@@ -90,7 +90,7 @@
387.28668212890625
],
"flags": {},
- "order": 11,
+ "order": 9,
"mode": 0,
"inputs": [
{
@@ -158,7 +158,7 @@
692.112548828125
],
"flags": {},
- "order": 13,
+ "order": 11,
"mode": 0,
"inputs": [
{
@@ -187,7 +187,7 @@
200
],
"flags": {},
- "order": 8,
+ "order": 7,
"mode": 0,
"inputs": [
{
@@ -213,79 +213,6 @@
"A towering technological monolith in a cyberpunk cityscape at night, with \"DisTorch\" emblazoned across its surface in massive neon red letters that illuminate the surrounding buildings. The text occupies the central third of the frame, crafted from glowing plasma tubes and crackling energy. Rain-slicked streets below reflect the brilliant signage, while holographic advertisements and flying vehicles populate the background. Moody atmospheric lighting, heavy contrast, photorealistic textures, cinematic color grading."
]
},
- {
- "id": 35,
- "type": "DualCLIPLoaderGGUFMultiGPU",
- "pos": [
- -74.29531860351562,
- 272.3004455566406
- ],
- "size": [
- 435.3804016113281,
- 130
- ],
- "flags": {},
- "order": 1,
- "mode": 0,
- "inputs": [],
- "outputs": [
- {
- "name": "CLIP",
- "type": "CLIP",
- "links": [
- 99
- ],
- "slot_index": 0
- }
- ],
- "properties": {
- "Node name for S&R": "DualCLIPLoaderGGUFMultiGPU"
- },
- "widgets_values": [
- "t5-v1_1-xxl-encoder-Q4_K_M.gguf",
- "clip_l.safetensors",
- "flux",
- "cuda:1"
- ],
- "color": "#233",
- "bgcolor": "#355"
- },
- {
- "id": 33,
- "type": "VAELoaderMultiGPU",
- "pos": [
- -62.57144546508789,
- 460.42584228515625
- ],
- "size": [
- 450.23980712890625,
- 82
- ],
- "flags": {},
- "order": 2,
- "mode": 0,
- "inputs": [],
- "outputs": [
- {
- "name": "VAE",
- "type": "VAE",
- "links": [
- 84
- ],
- "slot_index": 0,
- "shape": 3
- }
- ],
- "properties": {
- "Node name for S&R": "VAELoaderMultiGPU"
- },
- "widgets_values": [
- "ae.safetensors",
- "cuda:1"
- ],
- "color": "#233",
- "bgcolor": "#355"
- },
{
"id": 17,
"type": "BasicScheduler",
@@ -298,7 +225,7 @@
106
],
"flags": {},
- "order": 9,
+ "order": 6,
"mode": 0,
"inputs": [
{
@@ -339,7 +266,7 @@
46
],
"flags": {},
- "order": 12,
+ "order": 10,
"mode": 0,
"inputs": [
{
@@ -368,43 +295,6 @@
},
"widgets_values": []
},
- {
- "id": 37,
- "type": "UnetLoaderGGUFDisTorchMultiGPU",
- "pos": [
- -71.38706970214844,
- 90.67648315429688
- ],
- "size": [
- 417.6407775878906,
- 124
- ],
- "flags": {},
- "order": 3,
- "mode": 0,
- "inputs": [],
- "outputs": [
- {
- "name": "MODEL",
- "type": "MODEL",
- "links": [
- 102,
- 103
- ],
- "slot_index": 0
- }
- ],
- "properties": {
- "Node name for S&R": "UnetLoaderGGUFDisTorchMultiGPU"
- },
- "widgets_values": [
- "flux1-dev-Q8_0.gguf",
- "cuda:0",
- "cuda:0,0.4608;cuda:1,0.0271;cpu,0.0136"
- ],
- "color": "#233",
- "bgcolor": "#355"
- },
{
"id": 39,
"type": "EmptySD3LatentImage",
@@ -417,7 +307,7 @@
106
],
"flags": {},
- "order": 4,
+ "order": 1,
"mode": 0,
"inputs": [],
"outputs": [
@@ -439,30 +329,6 @@
1
]
},
- {
- "id": 52,
- "type": "Note",
- "pos": [
- -577.845458984375,
- 64.50007629394531
- ],
- "size": [
- 495.7899169921875,
- 484.21075439453125
- ],
- "flags": {},
- "order": 5,
- "mode": 0,
- "inputs": [],
- "outputs": [],
- "title": "How the DisTorch allocation string works",
- "properties": {},
- "widgets_values": [
- "The allocation string uses a semicolon (;) to separate device allocations, and each device allocation has two parts separated by a comma:\n\nDevice name (e.g., \"cuda:0\", \"cuda:1\", \"cpu\")\nMemory fraction as a decimal (e.g., 0.025 = 2.5%, 0.05 = 5%, 0.10 = 10%)\n\n\nFor my defaults:\n\nCUDA GPU 0: Will allocate 2.5% of its total available memory\nCUDA GPU 1: Will allocate 5% of its total available memory\nCPU: Will allocate 10% of available system memory\n\n\nDisTorch then:\n\nCalculates how much actual memory this means for each device\nDistributes model layers across devices proportionally based on these allocations\nAssigns layers to minimize memory usage while maintaining model functionality\n\n\n\nSo if you have a model with 100 layers and your GPUs have the following memory:\n\nCUDA:0 has 24GB\nCUDA:1 has 24GB\nSystem has 64GB RAM\n\nThe allocation would be:\n\nCUDA:0: 600MB (2.5% of 24GB)\nCUDA:1: 1.2GB (5% of 24GB)\nCPU: 6.4GB (10% of 64GB)\n\nThis would help manage memory usage across your devices while keeping the model functional. The semicolon-separated format makes it easy to specify different allocations for different devices in a single string."
- ],
- "color": "#233",
- "bgcolor": "#355"
- },
{
"id": 25,
"type": "RandomNoise",
@@ -475,7 +341,7 @@
88.84210968017578
],
"flags": {},
- "order": 6,
+ "order": 2,
"mode": 0,
"inputs": [],
"outputs": [
@@ -497,34 +363,113 @@
]
},
{
- "id": 34,
- "type": "UnetLoaderGGUFMultiGPU",
+ "id": 37,
+ "type": "UnetLoaderGGUFDisTorchMultiGPU",
"pos": [
- -79.20733642578125,
- -116.61892700195312
+ -71.38706970214844,
+ 90.67648315429688
],
"size": [
- 417.5072937011719,
- 97.88888549804688
+ 417.6407775878906,
+ 154
],
"flags": {},
- "order": 7,
- "mode": 4,
+ "order": 3,
+ "mode": 0,
"inputs": [],
"outputs": [
{
"name": "MODEL",
"type": "MODEL",
- "links": [],
+ "links": [
+ 102,
+ 103
+ ],
"slot_index": 0
}
],
"properties": {
- "Node name for S&R": "UnetLoaderGGUFMultiGPU"
+ "Node name for S&R": "UnetLoaderGGUFDisTorchMultiGPU"
},
"widgets_values": [
"flux1-dev-Q8_0.gguf",
- "cuda:0"
+ "cuda:0",
+ 6,
+ false,
+ ""
+ ],
+ "color": "#233",
+ "bgcolor": "#355"
+ },
+ {
+ "id": 33,
+ "type": "VAELoaderMultiGPU",
+ "pos": [
+ -66.57144165039062,
+ 479.7591552734375
+ ],
+ "size": [
+ 450.23980712890625,
+ 82
+ ],
+ "flags": {},
+ "order": 4,
+ "mode": 0,
+ "inputs": [],
+ "outputs": [
+ {
+ "name": "VAE",
+ "type": "VAE",
+ "links": [
+ 84
+ ],
+ "slot_index": 0,
+ "shape": 3
+ }
+ ],
+ "properties": {
+ "Node name for S&R": "VAELoaderMultiGPU"
+ },
+ "widgets_values": [
+ "ae.safetensors",
+ "cuda:1"
+ ],
+ "color": "#233",
+ "bgcolor": "#355"
+ },
+ {
+ "id": 35,
+ "type": "DualCLIPLoaderGGUFMultiGPU",
+ "pos": [
+ -69.62869262695312,
+ 285.6338195800781
+ ],
+ "size": [
+ 435.3804016113281,
+ 130
+ ],
+ "flags": {},
+ "order": 5,
+ "mode": 0,
+ "inputs": [],
+ "outputs": [
+ {
+ "name": "CLIP",
+ "type": "CLIP",
+ "links": [
+ 99
+ ],
+ "slot_index": 0
+ }
+ ],
+ "properties": {
+ "Node name for S&R": "DualCLIPLoaderGGUFMultiGPU"
+ },
+ "widgets_values": [
+ "t5-v1_1-xxl-encoder-Q4_K_M.gguf",
+ "clip_l.safetensors",
+ "flux",
+ "cuda:1"
],
"color": "#233",
"bgcolor": "#355"
@@ -632,10 +577,10 @@
"config": {},
"extra": {
"ds": {
- "scale": 0.8769226950001245,
+ "scale": 1,
"offset": [
- 955.3126283315189,
- 498.64012539888034
+ 574.0015045555798,
+ 267.12195585438565
]
},
"ue_links": [],
@@ -644,7 +589,9 @@
"node_versions": {
"comfy-core": "0.3.12",
"ComfyUI-MultiGPU": "3dbfcc713552b0560a471c10d9b29520e8b1981e"
- }
+ },
+ "VHS_MetadataImage": true,
+ "VHS_KeepIntermediate": true
},
"version": 0.4
}
\ No newline at end of file
diff --git a/examples/hunyuan_gguf_distorch.json b/examples/hunyuan_gguf_distorch.json
old mode 100644
new mode 100755
index 048c9ce..68f6bd1
--- a/examples/hunyuan_gguf_distorch.json
+++ b/examples/hunyuan_gguf_distorch.json
@@ -1,6 +1,6 @@
{
- "last_node_id": 119,
- "last_link_id": 155,
+ "last_node_id": 120,
+ "last_link_id": 156,
"nodes": [
{
"id": 72,
@@ -14,7 +14,7 @@
58
],
"flags": {},
- "order": 11,
+ "order": 10,
"mode": 0,
"inputs": [
{
@@ -55,7 +55,7 @@
58
],
"flags": {},
- "order": 9,
+ "order": 7,
"mode": 0,
"inputs": [
{
@@ -93,7 +93,7 @@
150
],
"flags": {},
- "order": 14,
+ "order": 13,
"mode": 0,
"inputs": [
{
@@ -127,76 +127,6 @@
8
]
},
- {
- "id": 75,
- "type": "SamplerCustomAdvanced",
- "pos": [
- 3488.497314453125,
- -150.8239288330078
- ],
- "size": [
- 292.4319763183594,
- 479.03521728515625
- ],
- "flags": {
- "collapsed": false
- },
- "order": 13,
- "mode": 0,
- "inputs": [
- {
- "name": "noise",
- "type": "NOISE",
- "link": 87,
- "slot_index": 0
- },
- {
- "name": "guider",
- "type": "GUIDER",
- "link": 88,
- "slot_index": 1
- },
- {
- "name": "sampler",
- "type": "SAMPLER",
- "link": 89,
- "slot_index": 2
- },
- {
- "name": "sigmas",
- "type": "SIGMAS",
- "link": 90,
- "slot_index": 3
- },
- {
- "name": "latent_image",
- "type": "LATENT",
- "link": 155,
- "slot_index": 4
- }
- ],
- "outputs": [
- {
- "name": "output",
- "type": "LATENT",
- "links": [
- 85
- ],
- "slot_index": 0,
- "shape": 3
- },
- {
- "name": "denoised_output",
- "type": "LATENT",
- "links": null,
- "shape": 3
- }
- ],
- "properties": {
- "Node name for S&R": "SamplerCustomAdvanced"
- },
- "widgets_values": []
- },
{
"id": 66,
"type": "BasicScheduler",
@@ -209,7 +139,7 @@
109.8011474609375
],
"flags": {},
- "order": 10,
+ "order": 8,
"mode": 0,
"inputs": [
{
@@ -270,41 +200,6 @@
"euler"
]
},
- {
- "id": 76,
- "type": "VAELoaderMultiGPU",
- "pos": [
- 2629.39599609375,
- 275.80596923828125
- ],
- "size": [
- 341.07464599609375,
- 87.14057922363281
- ],
- "flags": {},
- "order": 1,
- "mode": 0,
- "inputs": [],
- "outputs": [
- {
- "name": "VAE",
- "type": "VAE",
- "links": [
- 99
- ],
- "slot_index": 0
- }
- ],
- "properties": {
- "Node name for S&R": "VAELoaderMultiGPU"
- },
- "widgets_values": [
- "hunyuan_video_vae_bf16.safetensors",
- "cuda:1"
- ],
- "color": "#233",
- "bgcolor": "#355"
- },
{
"id": 115,
"type": "EmptyHunyuanLatentVideo",
@@ -319,7 +214,7 @@
"flags": {
"collapsed": false
},
- "order": 2,
+ "order": 1,
"mode": 0,
"inputs": [],
"outputs": [
@@ -354,7 +249,7 @@
190.77996826171875
],
"flags": {},
- "order": 3,
+ "order": 2,
"mode": 0,
"inputs": [],
"outputs": [],
@@ -378,7 +273,7 @@
82
],
"flags": {},
- "order": 4,
+ "order": 3,
"mode": 0,
"inputs": [],
"outputs": [
@@ -413,7 +308,7 @@
902.6787109375
],
"flags": {},
- "order": 15,
+ "order": 14,
"mode": 0,
"inputs": [
{
@@ -465,13 +360,13 @@
"hidden": false,
"paused": false,
"params": {
- "filename": "HunyuanVideo_00315.mp4",
+ "filename": "HunyuanVideo_00336.mp4",
"subfolder": "",
"type": "output",
"format": "video/h264-mp4",
"frame_rate": 24,
- "workflow": "HunyuanVideo_00315.png",
- "fullpath": "/home/johnj/ComfyUI/output/HunyuanVideo_00315.mp4"
+ "workflow": "HunyuanVideo_00336.png",
+ "fullpath": "/home/johnj/ComfyUI/output/HunyuanVideo_00336.mp4"
},
"muted": false
}
@@ -491,7 +386,7 @@
"flags": {
"collapsed": true
},
- "order": 12,
+ "order": 11,
"mode": 0,
"inputs": [
{
@@ -523,30 +418,6 @@
},
"widgets_values": []
},
- {
- "id": 119,
- "type": "Note",
- "pos": [
- 2050.9873046875,
- -145.64498901367188
- ],
- "size": [
- 549.7151489257812,
- 528.065185546875
- ],
- "flags": {},
- "order": 5,
- "mode": 0,
- "inputs": [],
- "outputs": [],
- "title": "How the DisTorch allocation string works",
- "properties": {},
- "widgets_values": [
- "The allocation string uses a semicolon (;) to separate device allocations, and each device allocation has two parts separated by a comma:\n\nDevice name (e.g., \"cuda:0\", \"cuda:1\", \"cpu\")\nMemory fraction as a decimal (e.g., 0.025 = 2.5%, 0.05 = 5%, 0.10 = 10%)\n\n\nFor my defaults:\n\nCUDA GPU 0: Will allocate 2.5% of its total available memory\nCUDA GPU 1: Will allocate 5% of its total available memory\nCPU: Will allocate 10% of available system memory\n\n\nDisTorch then:\n\nCalculates how much actual memory this means for each device\nDistributes model layers across devices proportionally based on these allocations\nAssigns layers to minimize memory usage while maintaining model functionality\n\n\n\nSo if you have a model with 100 layers and your GPUs have the following memory:\n\nCUDA:0 has 24GB\nCUDA:1 has 24GB\nSystem has 64GB RAM\n\nThe allocation would be:\n\nCUDA:0: 600MB (2.5% of 24GB)\nCUDA:1: 1.2GB (5% of 24GB)\nCPU: 6.4GB (10% of 64GB)\n\nThis would help manage memory usage across your devices while keeping the model functional. The semicolon-separated format makes it easy to specify different allocations for different devices in a single string."
- ],
- "color": "#233",
- "bgcolor": "#355"
- },
{
"id": 67,
"type": "CLIPTextEncode",
@@ -559,13 +430,13 @@
164.31304931640625
],
"flags": {},
- "order": 8,
+ "order": 9,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
- "link": 154
+ "link": 156
}
],
"outputs": [
@@ -589,56 +460,123 @@
"bgcolor": "#353"
},
{
- "id": 100,
- "type": "DualCLIPLoaderGGUFDisTorchMultiGPU",
+ "id": 76,
+ "type": "VAELoaderMultiGPU",
"pos": [
- 2623.7119140625,
- 64.39619445800781
+ 2632.252685546875,
+ 327.2115783691406
],
"size": [
- 348.5331726074219,
- 154
+ 341.07464599609375,
+ 87.14057922363281
],
"flags": {},
- "order": 6,
+ "order": 4,
"mode": 0,
"inputs": [],
"outputs": [
{
- "name": "CLIP",
- "type": "CLIP",
+ "name": "VAE",
+ "type": "VAE",
"links": [
- 154
+ 99
],
"slot_index": 0
}
],
"properties": {
- "Node name for S&R": "DualCLIPLoaderGGUFDisTorchMultiGPU"
+ "Node name for S&R": "VAELoaderMultiGPU"
},
"widgets_values": [
- "clip_l.safetensors",
- "llava-llama-3-8B-v1_1-Q4_K_M.gguf",
- "hunyuan_video",
- "cuda:1",
- "cuda:1,0.33;cpu,0.15"
+ "hunyuan_video_vae_bf16.safetensors",
+ "cuda:1"
],
"color": "#233",
"bgcolor": "#355"
},
+ {
+ "id": 75,
+ "type": "SamplerCustomAdvanced",
+ "pos": [
+ 3517.626953125,
+ -117.69574737548828
+ ],
+ "size": [
+ 292.4319763183594,
+ 479.03521728515625
+ ],
+ "flags": {
+ "collapsed": false
+ },
+ "order": 12,
+ "mode": 0,
+ "inputs": [
+ {
+ "name": "noise",
+ "type": "NOISE",
+ "link": 87,
+ "slot_index": 0
+ },
+ {
+ "name": "guider",
+ "type": "GUIDER",
+ "link": 88,
+ "slot_index": 1
+ },
+ {
+ "name": "sampler",
+ "type": "SAMPLER",
+ "link": 89,
+ "slot_index": 2
+ },
+ {
+ "name": "sigmas",
+ "type": "SIGMAS",
+ "link": 90,
+ "slot_index": 3
+ },
+ {
+ "name": "latent_image",
+ "type": "LATENT",
+ "link": 155,
+ "slot_index": 4
+ }
+ ],
+ "outputs": [
+ {
+ "name": "output",
+ "type": "LATENT",
+ "links": [
+ 85
+ ],
+ "slot_index": 0,
+ "shape": 3
+ },
+ {
+ "name": "denoised_output",
+ "type": "LATENT",
+ "links": null,
+ "shape": 3
+ }
+ ],
+ "properties": {
+ "Node name for S&R": "SamplerCustomAdvanced"
+ },
+ "widgets_values": []
+ },
{
"id": 88,
"type": "UnetLoaderGGUFDisTorchMultiGPU",
"pos": [
- 2622.552734375,
- -105.02156829833984
+ 2624.267822265625,
+ -66.1816177368164
],
"size": [
358.005859375,
- 128.5614013671875
+ 154
],
"flags": {},
- "order": 7,
+ "order": 5,
"mode": 0,
"inputs": [],
"outputs": [
@@ -656,9 +594,48 @@
"Node name for S&R": "UnetLoaderGGUFDisTorchMultiGPU"
},
"widgets_values": [
- "flux1-dev-Q8_0.gguf",
+ "hunyuan-video-t2v-720p-Q8_0.gguf",
"cuda:0",
- "cuda:0,0.33;cuda:1,0.33;cpu,0.0825"
+ 8,
+ false,
+ ""
+ ],
+ "color": "#233",
+ "bgcolor": "#355"
+ },
+ {
+ "id": 120,
+ "type": "DualCLIPLoaderGGUFMultiGPU",
+ "pos": [
+ 2625.07763671875,
+ 132.1048126220703
+ ],
+ "size": [
+ 348.16229248046875,
+ 130
+ ],
+ "flags": {},
+ "order": 6,
+ "mode": 0,
+ "inputs": [],
+ "outputs": [
+ {
+ "name": "CLIP",
+ "type": "CLIP",
+ "links": [
+ 156
+ ],
+ "slot_index": 0
+ }
+ ],
+ "properties": {
+ "Node name for S&R": "DualCLIPLoaderGGUFMultiGPU"
+ },
+ "widgets_values": [
+ "clip_l.safetensors",
+ "llava-llama-3-8B-v1_1-Q4_K_M.gguf",
+ "hunyuan_video",
+ "cpu"
],
"color": "#233",
"bgcolor": "#355"
@@ -761,14 +738,6 @@
0,
"MODEL"
],
- [
- 154,
- 100,
- 0,
- 67,
- 0,
- "CLIP"
- ],
[
155,
115,
@@ -776,6 +745,14 @@
75,
4,
"LATENT"
+ ],
+ [
+ 156,
+ 120,
+ 0,
+ 67,
+ 0,
+ "CLIP"
]
],
"groups": [
@@ -783,8 +760,8 @@
"id": 1,
"title": "GGUFDisTorchMultiGPU",
"bounding": [
- 2617.71240234375,
- -173.0873565673828,
+ 2620.569091796875,
+ -121.6815185546875,
360.4816589355469,
537.903076171875
],
@@ -796,15 +773,17 @@
"config": {},
"extra": {
"ds": {
- "scale": 1.283902517749697,
+ "scale": 1.167184107045179,
"offset": [
- -2107.057728768135,
- 344.093734201536
+ -2360.0515813693187,
+ 465.0591039206934
]
},
"ue_links": [],
"VHS_latentpreview": false,
- "VHS_latentpreviewrate": 0
+ "VHS_latentpreviewrate": 0,
+ "VHS_MetadataImage": true,
+ "VHS_KeepIntermediate": true
},
"version": 0.4
}
\ No newline at end of file
diff --git a/examples/hunyuan_ip2v_distorch_gguf.json b/examples/hunyuan_ip2v_distorch_gguf.json
old mode 100644
new mode 100755
index 3809e4a..01b44bb
--- a/examples/hunyuan_ip2v_distorch_gguf.json
+++ b/examples/hunyuan_ip2v_distorch_gguf.json
@@ -159,7 +159,7 @@
452.87860107421875
],
"flags": {},
- "order": 10,
+ "order": 8,
"mode": 0,
"inputs": [
{
@@ -219,43 +219,6 @@
""
]
},
- {
- "id": 115,
- "type": "UnetLoaderGGUFDisTorchMultiGPU",
- "pos": [
- -807.809814453125,
- 227.3965301513672
- ],
- "size": [
- 342.1245422363281,
- 106
- ],
- "flags": {},
- "order": 2,
- "mode": 0,
- "inputs": [],
- "outputs": [
- {
- "name": "MODEL",
- "type": "MODEL",
- "links": [
- 276,
- 277
- ],
- "slot_index": 0
- }
- ],
- "properties": {
- "Node name for S&R": "UnetLoaderGGUFDisTorchMultiGPU"
- },
- "widgets_values": [
- "hunyuan-video-t2v-720p-Q8_0.gguf",
- "cuda:0",
- "cuda:0,0.15;cpu,0.5"
- ],
- "color": "#233",
- "bgcolor": "#355"
- },
{
"id": 112,
"type": "LoadImage",
@@ -268,7 +231,7 @@
314
],
"flags": {},
- "order": 3,
+ "order": 2,
"mode": 0,
"inputs": [],
"outputs": [
@@ -306,7 +269,7 @@
190.77996826171875
],
"flags": {},
- "order": 4,
+ "order": 3,
"mode": 0,
"inputs": [],
"outputs": [],
@@ -332,7 +295,7 @@
"flags": {
"collapsed": true
},
- "order": 8,
+ "order": 9,
"mode": 0,
"inputs": [
{
@@ -372,7 +335,7 @@
"flags": {
"collapsed": true
},
- "order": 9,
+ "order": 10,
"mode": 0,
"inputs": [
{
@@ -540,7 +503,7 @@
"flags": {
"collapsed": true
},
- "order": 5,
+ "order": 4,
"mode": 0,
"inputs": [],
"outputs": [
@@ -572,7 +535,7 @@
130
],
"flags": {},
- "order": 6,
+ "order": 5,
"mode": 0,
"inputs": [],
"outputs": [
@@ -595,41 +558,6 @@
1
]
},
- {
- "id": 25,
- "type": "RandomNoise",
- "pos": [
- 389.7550354003906,
- -21.38304328918457
- ],
- "size": [
- 250.37998962402344,
- 82
- ],
- "flags": {},
- "order": 7,
- "mode": 0,
- "inputs": [],
- "outputs": [
- {
- "name": "NOISE",
- "type": "NOISE",
- "links": [
- 37
- ],
- "shape": 3
- }
- ],
- "properties": {
- "Node name for S&R": "RandomNoise"
- },
- "widgets_values": [
- 5770521,
- "fixed"
- ],
- "color": "#2a363b",
- "bgcolor": "#3f5159"
- },
{
"id": 73,
"type": "VAEDecodeTiled",
@@ -685,7 +613,7 @@
],
"size": [
451.07391357421875,
- 592.0040893554688
+ 334
],
"flags": {},
"order": 16,
@@ -751,6 +679,80 @@
"muted": false
}
}
+ },
+ {
+ "id": 115,
+ "type": "UnetLoaderGGUFDisTorchMultiGPU",
+ "pos": [
+ -810.4765014648438,
+ 219.3965301513672
+ ],
+ "size": [
+ 342.1245422363281,
+ 154
+ ],
+ "flags": {},
+ "order": 6,
+ "mode": 0,
+ "inputs": [],
+ "outputs": [
+ {
+ "name": "MODEL",
+ "type": "MODEL",
+ "links": [
+ 276,
+ 277
+ ],
+ "slot_index": 0
+ }
+ ],
+ "properties": {
+ "Node name for S&R": "UnetLoaderGGUFDisTorchMultiGPU"
+ },
+ "widgets_values": [
+ "hunyuan-video-t2v-720p-Q4_K_M.gguf",
+ "cuda:0",
+ 4,
+ false,
+ ""
+ ],
+ "color": "#233",
+ "bgcolor": "#355"
+ },
+ {
+ "id": 25,
+ "type": "RandomNoise",
+ "pos": [
+ 368.4217834472656,
+ 72.61695861816406
+ ],
+ "size": [
+ 250.37998962402344,
+ 82
+ ],
+ "flags": {},
+ "order": 7,
+ "mode": 0,
+ "inputs": [],
+ "outputs": [
+ {
+ "name": "NOISE",
+ "type": "NOISE",
+ "links": [
+ 37
+ ],
+ "shape": 3
+ }
+ ],
+ "properties": {
+ "Node name for S&R": "RandomNoise"
+ },
+ "widgets_values": [
+ 5770521,
+ "fixed"
+ ],
+ "color": "#2a363b",
+ "bgcolor": "#3f5159"
}
],
"links": [
@@ -901,16 +903,18 @@
"config": {},
"extra": {
"ds": {
- "scale": 0.7247295000001047,
- "offset": [
- 1743.9442159363227,
- 215.31117435648653
- ]
+ "scale": 1,
+ "offset": {
+ "0": 1075.0079345703125,
+ "1": 156.88380432128906
+ }
},
"groupNodes": {},
"ue_links": [],
"VHS_latentpreview": false,
- "VHS_latentpreviewrate": 0
+ "VHS_latentpreviewrate": 0,
+ "VHS_MetadataImage": true,
+ "VHS_KeepIntermediate": true
},
"version": 0.4
}
\ No newline at end of file
diff --git a/examples/hunyuanvideowrapper_diffsynth.json b/examples/hunyuanvideowrapper_diffsynth.json
deleted file mode 100644
index d2e531c..0000000
--- a/examples/hunyuanvideowrapper_diffsynth.json
+++ /dev/null
@@ -1,575 +0,0 @@
-{
- "last_node_id": 57,
- "last_link_id": 75,
- "nodes": [
- {
- "id": 34,
- "type": "VHS_VideoCombine",
- "pos": [
- 847.0758666992188,
- -415.1882629394531
- ],
- "size": [
- 512.8807373046875,
- 1128.39453125
- ],
- "flags": {},
- "order": 9,
- "mode": 0,
- "inputs": [
- {
- "name": "images",
- "type": "IMAGE",
- "link": 69
- },
- {
- "name": "audio",
- "type": "AUDIO",
- "link": null,
- "shape": 7
- },
- {
- "name": "meta_batch",
- "type": "VHS_BatchManager",
- "link": null,
- "shape": 7
- },
- {
- "name": "vae",
- "type": "VAE",
- "link": null,
- "shape": 7
- }
- ],
- "outputs": [
- {
- "name": "Filenames",
- "type": "VHS_FILENAMES",
- "links": null
- }
- ],
- "properties": {
- "Node name for S&R": "VHS_VideoCombine"
- },
- "widgets_values": {
- "frame_rate": 24,
- "loop_count": 0,
- "filename_prefix": "HunyuanVideo",
- "format": "video/h264-mp4",
- "pix_fmt": "yuv420p",
- "crf": 19,
- "save_metadata": true,
- "trim_to_audio": false,
- "pingpong": false,
- "save_output": true,
- "videopreview": {
- "hidden": false,
- "paused": false,
- "params": {
- "filename": "HunyuanVideo_00182.mp4",
- "subfolder": "",
- "type": "output",
- "format": "video/h264-mp4",
- "frame_rate": 24,
- "workflow": "HunyuanVideo_00182.png",
- "fullpath": "/home/johnj/ComfyUI/output/HunyuanVideo_00182.mp4"
- },
- "muted": false
- }
- }
- },
- {
- "id": 45,
- "type": "VAEDecodeTiled",
- "pos": [
- 604.5225830078125,
- -401.70220947265625
- ],
- "size": [
- 210,
- 150
- ],
- "flags": {},
- "order": 8,
- "mode": 0,
- "inputs": [
- {
- "name": "samples",
- "type": "LATENT",
- "link": 70
- },
- {
- "name": "vae",
- "type": "VAE",
- "link": 56
- }
- ],
- "outputs": [
- {
- "name": "IMAGE",
- "type": "IMAGE",
- "links": [
- 69
- ],
- "slot_index": 0
- }
- ],
- "properties": {
- "Node name for S&R": "VAEDecodeTiled"
- },
- "widgets_values": [
- 256,
- 64,
- 64,
- 8
- ]
- },
- {
- "id": 44,
- "type": "VAELoaderMultiGPU",
- "pos": [
- 208.21218872070312,
- -586.37841796875
- ],
- "size": [
- 353.3572692871094,
- 84.02234649658203
- ],
- "flags": {},
- "order": 0,
- "mode": 0,
- "inputs": [],
- "outputs": [
- {
- "name": "VAE",
- "type": "VAE",
- "links": [
- 56
- ],
- "slot_index": 0
- }
- ],
- "properties": {
- "Node name for S&R": "VAELoaderMultiGPU"
- },
- "widgets_values": [
- "hunyuan_video_vae_bf16.safetensors",
- "cuda:1"
- ],
- "color": "#233",
- "bgcolor": "#355"
- },
- {
- "id": 53,
- "type": "HyVideoTextImageEncode",
- "pos": [
- -163.92579650878906,
- -51.38969421386719
- ],
- "size": [
- 367.4625244140625,
- 528.5474853515625
- ],
- "flags": {},
- "order": 6,
- "mode": 0,
- "inputs": [
- {
- "name": "text_encoders",
- "type": "HYVIDTEXTENCODER",
- "link": 74
- },
- {
- "name": "custom_prompt_template",
- "type": "PROMPT_TEMPLATE",
- "link": null,
- "shape": 7
- },
- {
- "name": "clip_l",
- "type": "CLIP",
- "link": null,
- "shape": 7
- },
- {
- "name": "image1",
- "type": "IMAGE",
- "link": 73,
- "shape": 7
- },
- {
- "name": "image2",
- "type": "IMAGE",
- "link": null,
- "shape": 7
- },
- {
- "name": "hyvid_cfg",
- "type": "HYVID_CFG",
- "link": null,
- "shape": 7
- }
- ],
- "outputs": [
- {
- "name": "hyvid_embeds",
- "type": "HYVIDEMBEDS",
- "links": [
- 75
- ],
- "slot_index": 0
- }
- ],
- "properties": {
- "Node name for S&R": "HyVideoTextImageEncode"
- },
- "widgets_values": [
- "The animal shown in appears within its own natural setting, moving calmly or resting in place as a soft light casts delicate shadows across its form. Over the course of five seconds, it makes subtle shifts in posture or position, revealing small details of its features, such as the texture of its skin or fur, and the quiet rhythm of its breathing.",
- "::3",
- true,
- "video",
- ""
- ]
- },
- {
- "id": 54,
- "type": "LoadImage",
- "pos": [
- -526.0797119140625,
- 233.78311157226562
- ],
- "size": [
- 315,
- 314
- ],
- "flags": {},
- "order": 1,
- "mode": 0,
- "inputs": [],
- "outputs": [
- {
- "name": "IMAGE",
- "type": "IMAGE",
- "links": [
- 73
- ],
- "slot_index": 0
- },
- {
- "name": "MASK",
- "type": "MASK",
- "links": null
- }
- ],
- "properties": {
- "Node name for S&R": "LoadImage"
- },
- "widgets_values": [
- "pasted/image (165).png",
- "image"
- ]
- },
- {
- "id": 49,
- "type": "DownloadAndLoadHyVideoTextEncoderMultiGPU",
- "pos": [
- -591.7539672851562,
- -37.14900588989258
- ],
- "size": [
- 380.8912353515625,
- 202
- ],
- "flags": {},
- "order": 2,
- "mode": 0,
- "inputs": [],
- "outputs": [
- {
- "name": "hyvid_text_encoder",
- "type": "HYVIDTEXTENCODER",
- "links": [
- 74
- ],
- "slot_index": 0
- }
- ],
- "properties": {
- "Node name for S&R": "DownloadAndLoadHyVideoTextEncoderMultiGPU"
- },
- "widgets_values": [
- "xtuner/llava-llama-3-8b-v1_1-transformers",
- "openai/clip-vit-large-patch14",
- "bf16",
- false,
- 2,
- "disabled",
- "cuda:0"
- ],
- "color": "#233",
- "bgcolor": "#355"
- },
- {
- "id": 3,
- "type": "HyVideoSampler",
- "pos": [
- 255.96482849121094,
- -403.58502197265625
- ],
- "size": [
- 315,
- 630
- ],
- "flags": {},
- "order": 7,
- "mode": 0,
- "inputs": [
- {
- "name": "model",
- "type": "HYVIDEOMODEL",
- "link": 71
- },
- {
- "name": "hyvid_embeds",
- "type": "HYVIDEMBEDS",
- "link": 75
- },
- {
- "name": "samples",
- "type": "LATENT",
- "link": null,
- "shape": 7
- },
- {
- "name": "stg_args",
- "type": "STGARGS",
- "link": null,
- "shape": 7
- },
- {
- "name": "context_options",
- "type": "HYVIDCONTEXT",
- "link": null,
- "shape": 7
- },
- {
- "name": "feta_args",
- "type": "FETAARGS",
- "link": null,
- "shape": 7
- },
- {
- "name": "teacache_args",
- "type": "TEACACHEARGS",
- "link": null,
- "shape": 7
- }
- ],
- "outputs": [
- {
- "name": "samples",
- "type": "LATENT",
- "links": [
- 70
- ],
- "slot_index": 0
- }
- ],
- "properties": {
- "Node name for S&R": "HyVideoSampler"
- },
- "widgets_values": [
- 768,
- 1216,
- 101,
- 30,
- 7.5,
- 7.5,
- 5770521,
- "fixed",
- true,
- 1,
- "FlowMatchDiscreteScheduler"
- ]
- },
- {
- "id": 52,
- "type": "HyVideoModelLoaderDiffSynthMultiGPU",
- "pos": [
- -217.43194580078125,
- -401.6243896484375
- ],
- "size": [
- 441,
- 218
- ],
- "flags": {},
- "order": 3,
- "mode": 0,
- "inputs": [
- {
- "name": "compile_args",
- "type": "COMPILEARGS",
- "link": null,
- "shape": 7
- },
- {
- "name": "block_swap_args",
- "type": "BLOCKSWAPARGS",
- "link": null,
- "shape": 7
- },
- {
- "name": "lora",
- "type": "HYVIDLORA",
- "link": null,
- "shape": 7
- }
- ],
- "outputs": [
- {
- "name": "model",
- "type": "HYVIDEOMODEL",
- "links": [
- 71
- ],
- "slot_index": 0
- }
- ],
- "properties": {
- "Node name for S&R": "HyVideoModelLoaderDiffSynthMultiGPU"
- },
- "widgets_values": [
- "hunyuan_video_FastVideo_720_fp8_e4m3fn.safetensors",
- "bf16",
- "fp8_e4m3fn_fast",
- "sageattn_varlen",
- "cuda:0",
- "cuda:1"
- ],
- "color": "#233",
- "bgcolor": "#355"
- },
- {
- "id": 57,
- "type": "Note",
- "pos": [
- 256.481689453125,
- 293.84710693359375
- ],
- "size": [
- 389.5359191894531,
- 349.2815856933594
- ],
- "flags": {},
- "order": 4,
- "mode": 0,
- "inputs": [],
- "outputs": [],
- "title": "Explanation of Memory Constraints and OOM Behavior",
- "properties": {},
- "widgets_values": [
- "The settings in this workflow deliberately push memory usage beyond what a single 24GB RTX 3090 can handle. As a result, UNet blocks must be offloaded to an offload_device, which is assumed to be a second 24GB RTX 3090. The chosen resolution and number of frames illustrate that one-megapixel image size videos over four seconds long can be generated—even though it exceeds the primary GPU's native memory capacity using the \"just-in-time\" DiffSynth block-swapping approach. The DiffSynth JiT algorithm is straightforward, but not optimized for speed.\n\n**Expected OOM Behavior**\n\n* One Initial \"Out of Memory\"\nSeeing a single OOM error at the start is normal for this workflow. It happens because kijai's HunyuanVideo Sampler node initially attempts a full load on \"device\", even if it will surpass available VRAM. A second attempt using the same generation parameters will correctly trigger the offload process to \"offload_device\".\n\n* Repeated OOM Errors \nIf you keep getting OOM errors with the same settings (beyond the first one), then the requirements of your generation parameters still exceed what your system—even with offloading—can manage. You may need to lower resolution, shorten video length, or upgrade hardware resources."
- ],
- "color": "#233",
- "bgcolor": "#355"
- },
- {
- "id": 51,
- "type": "Note",
- "pos": [
- -793.3005981445312,
- -466.9315490722656
- ],
- "size": [
- 564.3460693359375,
- 355.50164794921875
- ],
- "flags": {},
- "order": 5,
- "mode": 0,
- "inputs": [],
- "outputs": [],
- "title": "****FIRST ATTEMPT LOADS ONLY ON PRIMARY DEVICE, POSSIBLE OOM****",
- "properties": {},
- "widgets_values": [
- "The MultiGPU implementation for HunyuanVideo now offers two approaches:\n1. The original workflow (examples/hunyuanvideowrapper_native_vae.json)\n2. A new experimental DiffSynth-enabled workflow that enables more efficient multi-GPU usage (examples/hunyuanvideowrapper_DiffSynth.json)\n\nNew DiffSynth-Based Workflow (Experimental - THIS WORKFLOW):\n* Uses the \"HyVideoModelLoaderDiffSynthMultiGPU\" node which implements DiffSynth's memory management approach with TWO device selections\n* Adds a second device selector \"offload_device\" specifically for model offloading\n* Automatically enables efficient block swapping between GPUs via kijai's adaptation of the DiffSynth methodology\n* Provides better memory utilization by keeping actively computed blocks in VRAM\n* Assumes a 2-device solution and sets \"force_offload\" to \"true\" to avoid VAE loader OOM issues. A third 3090, for instance could be used and thus the VAE would stay in memory as well\n\nKnown Behaviors:\n* Initial OOM errors are expected and consistent with the underlying implementation, occurring in the attention mechanism of the first double block\n* Second attempts typically succeed regardless of offload device selection (CPU or GPU)\n* When running, the secondary device shows active utilization, confirming proper block swapping\n\n***** Will require a second attempt if first generation fails with OOM****\n\nNOTE: The original workflow remains the recommended stable approach. The DiffSynth-based version is provided as an experimental alternative for users who want to explore more aggressive memory optimization strategies to extend either resolution or duration options at the expense of speed."
- ],
- "color": "#233",
- "bgcolor": "#355"
- }
- ],
- "links": [
- [
- 56,
- 44,
- 0,
- 45,
- 1,
- "VAE"
- ],
- [
- 69,
- 45,
- 0,
- 34,
- 0,
- "IMAGE"
- ],
- [
- 70,
- 3,
- 0,
- 45,
- 0,
- "LATENT"
- ],
- [
- 71,
- 52,
- 0,
- 3,
- 0,
- "HYVIDEOMODEL"
- ],
- [
- 73,
- 54,
- 0,
- 53,
- 3,
- "IMAGE"
- ],
- [
- 74,
- 49,
- 0,
- 53,
- 0,
- "HYVIDTEXTENCODER"
- ],
- [
- 75,
- 53,
- 0,
- 3,
- 1,
- "HYVIDEMBEDS"
- ]
- ],
- "groups": [],
- "config": {},
- "extra": {
- "ds": {
- "scale": 0.797202450000112,
- "offset": [
- 1219.7255813472686,
- 777.2824223551936
- ]
- },
- "ue_links": [],
- "VHS_latentpreview": false,
- "VHS_latentpreviewrate": 0
- },
- "version": 0.4
-}
\ No newline at end of file
diff --git a/examples/ltxvideo_gguf_distorch_lowvram_highdram.json b/examples/ltxvideo_gguf_distorch_lowvram_highdram.json
old mode 100644
new mode 100755
index 31c66b8..ad2699d
--- a/examples/ltxvideo_gguf_distorch_lowvram_highdram.json
+++ b/examples/ltxvideo_gguf_distorch_lowvram_highdram.json
@@ -1,6 +1,6 @@
{
- "last_node_id": 312,
- "last_link_id": 777,
+ "last_node_id": 313,
+ "last_link_id": 779,
"nodes": [
{
"id": 271,
@@ -266,7 +266,7 @@
"flags": {
"collapsed": false
},
- "order": 12,
+ "order": 11,
"mode": 0,
"inputs": [
{
@@ -282,7 +282,7 @@
{
"name": "vae",
"type": "VAE",
- "link": 719
+ "link": 779
},
{
"name": "image",
@@ -356,7 +356,7 @@
"flags": {
"collapsed": false
},
- "order": 11,
+ "order": 12,
"mode": 0,
"inputs": [
{
@@ -390,95 +390,6 @@
3
]
},
- {
- "id": 274,
- "type": "ImageResizeKJ",
- "pos": [
- 1738.9410400390625,
- -400.31304931640625
- ],
- "size": [
- 210,
- 238
- ],
- "flags": {
- "collapsed": false
- },
- "order": 6,
- "mode": 0,
- "inputs": [
- {
- "name": "image",
- "type": "IMAGE",
- "link": 771
- },
- {
- "name": "get_image_size",
- "type": "IMAGE",
- "link": null,
- "shape": 7
- },
- {
- "name": "width_input",
- "type": "INT",
- "link": null,
- "widget": {
- "name": "width_input"
- },
- "shape": 7
- },
- {
- "name": "height_input",
- "type": "INT",
- "link": null,
- "widget": {
- "name": "height_input"
- },
- "shape": 7
- }
- ],
- "outputs": [
- {
- "name": "IMAGE",
- "type": "IMAGE",
- "links": [
- 720
- ],
- "slot_index": 0
- },
- {
- "name": "width",
- "type": "INT",
- "links": [
- 721,
- 727
- ],
- "slot_index": 1
- },
- {
- "name": "height",
- "type": "INT",
- "links": [
- 722,
- 728
- ],
- "slot_index": 2
- }
- ],
- "properties": {
- "Node name for S&R": "ImageResizeKJ"
- },
- "widgets_values": [
- 768,
- 768,
- "nearest-exact",
- true,
- 16,
- 0,
- 0,
- "center"
- ]
- },
{
"id": 232,
"type": "LoadImage",
@@ -535,7 +446,7 @@
"flags": {
"collapsed": false
},
- "order": 7,
+ "order": 9,
"mode": 0,
"inputs": [
{
@@ -559,42 +470,6 @@
},
"widgets_values": []
},
- {
- "id": 308,
- "type": "UnetLoaderGGUFDisTorchMultiGPU",
- "pos": [
- 1742.724365234375,
- -987.2313232421875
- ],
- "size": [
- 339.9610290527344,
- 126.85929107666016
- ],
- "flags": {},
- "order": 3,
- "mode": 0,
- "inputs": [],
- "outputs": [
- {
- "name": "MODEL",
- "type": "MODEL",
- "links": [
- 772
- ],
- "slot_index": 0
- }
- ],
- "properties": {
- "Node name for S&R": "UnetLoaderGGUFDisTorchMultiGPU"
- },
- "widgets_values": [
- "ltx-video-2b-v0.9-Q5_0.gguf",
- "cuda:0",
- "cuda:0,0.15;cpu,0.8"
- ],
- "color": "#233",
- "bgcolor": "#355"
- },
{
"id": 312,
"type": "CLIPLoaderGGUFMultiGPU",
@@ -607,7 +482,7 @@
123.96922302246094
],
"flags": {},
- "order": 4,
+ "order": 3,
"mode": 0,
"inputs": [],
"outputs": [
@@ -632,53 +507,6 @@
"color": "#233",
"bgcolor": "#355"
},
- {
- "id": 266,
- "type": "CheckpointLoaderSimpleMultiGPU",
- "pos": [
- 1421.14599609375,
- -898.4613647460938
- ],
- "size": [
- 252,
- 128.28294372558594
- ],
- "flags": {},
- "order": 5,
- "mode": 0,
- "inputs": [],
- "outputs": [
- {
- "name": "MODEL",
- "type": "MODEL",
- "links": [],
- "slot_index": 0
- },
- {
- "name": "CLIP",
- "type": "CLIP",
- "links": null
- },
- {
- "name": "VAE",
- "type": "VAE",
- "links": [
- 719,
- 732
- ],
- "slot_index": 2
- }
- ],
- "properties": {
- "Node name for S&R": "CheckpointLoaderSimpleMultiGPU"
- },
- "widgets_values": [
- "ltx-video-2b-v0.9.1.safetensors",
- "cuda:0"
- ],
- "color": "#233",
- "bgcolor": "#355"
- },
{
"id": 283,
"type": "SamplerCustom",
@@ -840,56 +668,6 @@
}
}
},
- {
- "id": 268,
- "type": "CLIPTextEncode",
- "pos": [
- 1720.8240966796875,
- -838.1546630859375
- ],
- "size": [
- 392.4562072753906,
- 194.1393585205078
- ],
- "flags": {
- "collapsed": false
- },
- "order": 8,
- "mode": 0,
- "inputs": [
- {
- "name": "clip",
- "type": "CLIP",
- "link": 776
- }
- ],
- "outputs": [
- {
- "name": "CONDITIONING",
- "type": "CONDITIONING",
- "links": [
- 717
- ],
- "slot_index": 0
- }
- ],
- "title": "LTXV Pos",
- "properties": {
- "Node name for S&R": "CLIPTextEncode",
- "enableTabs": false,
- "tabWidth": 65,
- "tabXOffset": 10,
- "hasSecondTab": false,
- "secondTabText": "Send Back",
- "secondTabOffset": 80,
- "secondTabWidth": 65
- },
- "widgets_values": [
- "This describes the first frame of a video that needs to subtly animated for 5 seconds: The image is a black and white photograph of a waterfall. The waterfall is cascading down a rocky cliff, with the water flowing over the rocks and creating a misty effect. The rocks are large and jagged, and the water is flowing over them. The cliff is covered in trees and shrubs, and there is a hill in the background. The sky is overcast and the overall mood of the image is peaceful and serene."
- ],
- "color": "#232",
- "bgcolor": "#353"
- },
{
"id": 279,
"type": "VAEDecode",
@@ -915,7 +693,7 @@
{
"name": "vae",
"type": "VAE",
- "link": 732
+ "link": 778
}
],
"outputs": [
@@ -954,7 +732,7 @@
"flags": {
"collapsed": false
},
- "order": 9,
+ "order": 8,
"mode": 0,
"inputs": [
{
@@ -989,6 +767,219 @@
],
"color": "#322",
"bgcolor": "#533"
+ },
+ {
+ "id": 268,
+ "type": "CLIPTextEncode",
+ "pos": [
+ 1724.329833984375,
+ -756.010986328125
+ ],
+ "size": [
+ 358.89739990234375,
+ 117.5052490234375
+ ],
+ "flags": {
+ "collapsed": false
+ },
+ "order": 7,
+ "mode": 0,
+ "inputs": [
+ {
+ "name": "clip",
+ "type": "CLIP",
+ "link": 776
+ }
+ ],
+ "outputs": [
+ {
+ "name": "CONDITIONING",
+ "type": "CONDITIONING",
+ "links": [
+ 717
+ ],
+ "slot_index": 0
+ }
+ ],
+ "title": "LTXV Pos",
+ "properties": {
+ "Node name for S&R": "CLIPTextEncode",
+ "enableTabs": false,
+ "tabWidth": 65,
+ "tabXOffset": 10,
+ "hasSecondTab": false,
+ "secondTabText": "Send Back",
+ "secondTabOffset": 80,
+ "secondTabWidth": 65
+ },
+ "widgets_values": [
+ "This describes the first frame of a video that needs to subtly animated for 5 seconds: The image is a black and white photograph of a waterfall. The waterfall is cascading down a rocky cliff, with the water flowing over the rocks and creating a misty effect. The rocks are large and jagged, and the water is flowing over them. The cliff is covered in trees and shrubs, and there is a hill in the background. The sky is overcast and the overall mood of the image is peaceful and serene."
+ ],
+ "color": "#232",
+ "bgcolor": "#353"
+ },
+ {
+ "id": 308,
+ "type": "UnetLoaderGGUFDisTorchMultiGPU",
+ "pos": [
+ 1738.21630859375,
+ -938.6461791992188
+ ],
+ "size": [
+ 339.9610290527344,
+ 154
+ ],
+ "flags": {},
+ "order": 4,
+ "mode": 0,
+ "inputs": [],
+ "outputs": [
+ {
+ "name": "MODEL",
+ "type": "MODEL",
+ "links": [
+ 772
+ ],
+ "slot_index": 0
+ }
+ ],
+ "properties": {
+ "Node name for S&R": "UnetLoaderGGUFDisTorchMultiGPU"
+ },
+ "widgets_values": [
+ "ltx-video-2b-v0.9.1-F16.gguf",
+ "cuda:0",
+ 4,
+ false,
+ ""
+ ],
+ "color": "#233",
+ "bgcolor": "#355"
+ },
+ {
+ "id": 274,
+ "type": "ImageResizeKJ",
+ "pos": [
+ 1734.0921630859375,
+ -350.01019287109375
+ ],
+ "size": [
+ 210,
+ 238
+ ],
+ "flags": {
+ "collapsed": false
+ },
+ "order": 6,
+ "mode": 0,
+ "inputs": [
+ {
+ "name": "image",
+ "type": "IMAGE",
+ "link": 771
+ },
+ {
+ "name": "get_image_size",
+ "type": "IMAGE",
+ "link": null,
+ "shape": 7
+ },
+ {
+ "name": "width_input",
+ "type": "INT",
+ "link": null,
+ "widget": {
+ "name": "width_input"
+ },
+ "shape": 7
+ },
+ {
+ "name": "height_input",
+ "type": "INT",
+ "link": null,
+ "widget": {
+ "name": "height_input"
+ },
+ "shape": 7
+ }
+ ],
+ "outputs": [
+ {
+ "name": "IMAGE",
+ "type": "IMAGE",
+ "links": [
+ 720
+ ],
+ "slot_index": 0
+ },
+ {
+ "name": "width",
+ "type": "INT",
+ "links": [
+ 721,
+ 727
+ ],
+ "slot_index": 1
+ },
+ {
+ "name": "height",
+ "type": "INT",
+ "links": [
+ 722,
+ 728
+ ],
+ "slot_index": 2
+ }
+ ],
+ "properties": {
+ "Node name for S&R": "ImageResizeKJ"
+ },
+ "widgets_values": [
+ 768,
+ 768,
+ "nearest-exact",
+ true,
+ 16,
+ 0,
+ 0,
+ "center"
+ ]
+ },
+ {
+ "id": 313,
+ "type": "VAELoaderMultiGPU",
+ "pos": [
+ 1725.0933837890625,
+ -463.51104736328125
+ ],
+ "size": [
+ 315,
+ 82
+ ],
+ "flags": {},
+ "order": 5,
+ "mode": 0,
+ "inputs": [],
+ "outputs": [
+ {
+ "name": "VAE",
+ "type": "VAE",
+ "links": [
+ 778,
+ 779
+ ],
+ "slot_index": 0
+ }
+ ],
+ "properties": {
+ "Node name for S&R": "VAELoaderMultiGPU"
+ },
+ "widgets_values": [
+ "ltxv-0.9.1-vae.safetensors",
+ "cuda:0"
+ ],
+ "color": "#233",
+ "bgcolor": "#355"
}
],
"links": [
@@ -2560,14 +2551,6 @@
1,
"CONDITIONING"
],
- [
- 719,
- 266,
- 2,
- 273,
- 2,
- "VAE"
- ],
[
720,
274,
@@ -2640,14 +2623,6 @@
0,
"LATENT"
],
- [
- 732,
- 266,
- 2,
- 279,
- 1,
- "VAE"
- ],
[
733,
279,
@@ -2735,6 +2710,22 @@
269,
0,
"CLIP"
+ ],
+ [
+ 778,
+ 313,
+ 0,
+ 279,
+ 1,
+ "VAE"
+ ],
+ [
+ 779,
+ 313,
+ 0,
+ 273,
+ 2,
+ "VAE"
]
],
"groups": [
@@ -2755,15 +2746,17 @@
"config": {},
"extra": {
"ds": {
- "scale": 0.964614964500143,
- "offset": [
- -869.1439186341164,
- 1102.8630885776138
- ]
+ "scale": 1.1,
+ "offset": {
+ "0": -999.0325317382812,
+ "1": 1194.420166015625
+ }
},
"ue_links": [],
"VHS_latentpreview": false,
- "VHS_latentpreviewrate": 0
+ "VHS_latentpreviewrate": 0,
+ "VHS_MetadataImage": true,
+ "VHS_KeepIntermediate": true
},
"version": 0.4
}
\ No newline at end of file