nodes/city96-ComfyUI-GGUF
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
186 Commits 4 Branches 0 Tags
ca582bcb6d1ec29986a180a901f2ebfa260ddde2
Commit Graph
68 Commits
Author SHA1 Message Date
rattus 100c06cda8 nodes: mmap detach weights before they are pinned (#355) 
Comfy core recently introduced a feature where weights may be pinned
when loading, particularly for the case of offloading.

Intercept this, and immediately detached each weight before the pinning.
This avoids a crash that at least some users are experiencing.

Use a little dict on the modules to keep track of whats already done,
and when the catch-all detacher loop comes through, use this dict
(which has already done modules removed) as the iterator basis.
 2025-11-03 15:57:29 +01:00
City 03842fab0a Remove old workaround 2025-05-07 04:42:20 +02:00
City 220f78ec70 Force recalculation of model size on clone 2025-04-21 15:22:11 +02:00
City a355f8b5ea Skip unlink logic if load/offload dev match 
i.e. when forcing it to run on CPU or highvram enabled
 2025-04-18 23:53:12 +02:00
City 3564e95b26 Dynamic CLIP input types (#263) 2025-04-18 23:48:53 +02:00
City e29f526c68 Disable mixing scaled FP8 + GGUF 2025-04-16 16:07:51 +02:00
City 47bec61475 HiDream text encoder support 
LLaMA GGUF logic already works due to being used for HunYuan
 2025-04-16 06:10:28 +02:00
City 794fc28536 Clean up logging 
Also fixes numpy warning
 2025-04-13 23:01:15 +02:00
City 01f6e3ff63 spiece tokenizer from gguf metadata 2025-03-05 04:44:53 +01:00
City 8098c7149f Add PixArt clip type 
Also makes the map more readable
#195
 2025-01-04 21:46:23 +01:00
City 3dc384b233 More robust folder path logic 
Attempts to override the unet_gguf key with our current one if it exists + print a warning.
Should only be triggered if another custom node edits the expected key in some way.
 2024-12-26 21:42:33 +01:00
City eaffa0a7d3 Change clip workaround for SD3 
#186
 2024-12-22 15:24:14 +01:00
City 8d1b56c5e2 Loader logic for LLaMA models 
Tokenizer shape may be mismatched depending which model was used as source (128320 VS 128256) - warning instead of error for now.
k and q attn keys need to be reshaped back to HF format (inverse of convert_hf_to_gguf.py logic)
 2024-12-20 13:39:17 +01:00
City 0a1723baae Split loader into separate file 2024-12-20 13:03:27 +01:00
City 0b65261cc4 HunyuanVideo 
Convert script needs fixing, had to use a second script for the 5D tensor, will clean up and upload it later.
 2024-12-17 07:33:42 +01:00
City 23b9817081 LTX-Video 
Whilst we wait for Hunyuan Video.
 2024-12-08 00:48:59 +01:00
City 6e4790f903 Unused workaround 2024-12-04 15:54:20 +01:00
City a8b5f5d892 Less fragile patch logic 
For new ComfyUI versions. Relatively minimal, hopefully won't break with future updates
#163
 2024-12-04 02:09:01 +01:00
City 4a84328841 Add AuraFlow support 2024-11-23 06:51:29 +01:00
City 57562d54a5 Add ltxv clip type 
#162
 2024-11-23 00:10:27 +01:00
RandomGitUser321 8e898fad4c Add mochi to CLIPLoaderGGUF options (#154) 
* Add mochi to CLIPLoaderGGUF options

ComfyUI implemented it today, so this should be all that's needed to use the T5 GGUF with it

* Update nodes.py
 2024-11-06 14:04:04 +01:00
City 9833348005 Fix for text encoder folder name change 
#151
 2024-11-03 14:03:42 +01:00
City 0ef158d793 Add SD3 to loader arch list 2024-10-22 22:00:37 +02:00
City d090a910ac Rewrite convert.py to work properly with dev de-distill 
Original logic was triggering SDXL specific code to reshape img_in weight, causing forge not to work with it.
Tagging @blepping since this might change how the SDXL code is evaluated, but based on a quick check it should work.
 2024-10-15 03:56:54 +02:00
City d2aaeb0f13 Always init tenc on offload device 
#83
needs updated ComfyUI as well to fix it, issue was with the direct load logic in model management putting the model on the GPU before the weights were loaded, at which point the loading would overwrite the weights with the CPU ones
 2024-09-17 15:26:39 +02:00
City 0342f0a183 never double patch 
#98
Instead of adding/removing patches, sync with `self.patches[key]` - old patch should be GC'd. Most likely no legitimate reasons for double patching. Can simplify logic in ops.py if no issues.
 2024-09-06 18:10:02 +02:00
City b46c206dd7 double patch issue fix for lowvram 
Potential fix, lowvram weights are called via `patch_weight_to_device` on each run and it was possible to apply the same patch multiple times on subsequent generations.
 2024-09-05 05:32:01 +02:00
City 717a0e1d0b Temporarily disable instant patch logic 
Seems to break VRAM estimation and needs more involved fix with upstream ComfyUI repo.
 2024-09-03 17:00:20 +02:00
City c8923a4cd7 mmap unload fix for lowvram 2024-09-03 16:37:35 +02:00
City 6dbb4ba989 Temp fix for disable smart memory 
Model management seems to expect `unpatch_model` to put model on specified device after unpatching. Needs different way to avoid patch reload.
 2024-09-03 05:28:49 +02:00
City 35007f2659 Refactor model loading 
Should make it more readable. Moved import under if to avoid changes to stuff in `tools` breaking normal models.
 2024-09-02 03:44:07 +02:00
City df3378da6b Detect quant 2024-09-02 03:17:16 +02:00
blepping fa1671d63e Sync with upstream 2024-09-01 18:44:31 -06:00
City ba8ce44b4c Rename move_patch function 2024-09-02 02:22:32 +02:00
City c7b902bf9a Patch load/unload fix 
May break other things. For some reason T5 decided to run on the CPU with this change, but only like once and I can't for the life of me reproduce it.
 2024-09-02 02:00:10 +02:00
City 43427eff52 Better patch clear logic 
Reduces RAM overhead
 2024-09-02 01:37:16 +02:00
blepping 3bad1b4e04 Fix loading GGUF t5xxl 
Can't use squeeze proj workaround for stable-diffusion.cpp SD15
 2024-09-01 06:46:50 -06:00
blepping 436ecb4913 Merge upstream ops_refactor 2024-09-01 03:02:59 -06:00
blepping ed0702fefc Merge branch 'city96:main' into feat_load_stable_diffusion.cpp 2024-09-01 02:47:16 -06:00
City 6618801f1c Don't dequantize token embed on load 2024-08-30 06:25:19 +02:00
blepping b88cb9a344 Remove require_prefix in gguf_sd_loader 2024-08-28 12:50:39 -06:00
blepping e814fc1264 Revert ggml_tensor_name addition 2024-08-28 12:34:08 -06:00
City abe5c6a6c9 Fix weights not being copied for lowvram patch 
This can probably be improved since the weights themselves never need to be copied as they cannot be changed anyway.
 2024-08-28 20:23:08 +02:00
blepping f7d8fbe740 Workaround for stable-diffusion.cpp SDXL GGUF files 2024-08-28 11:56:51 -06:00
blepping 6ae0c733b0 Support loading u-net from stable-diffusion.cpp GGUF files 
Small cleanups
 2024-08-28 10:34:58 -06:00
City 3d9869240b Always patch weight 
Should help with #33
 2024-08-28 17:12:24 +02:00
blepping c1693d0f55 Use a GGUF array for orig shape metadata 2024-08-25 13:18:35 -06:00
blepping 0aec56ec4c Split orig shape metadata into individual fields 2024-08-25 12:00:28 -06:00
blepping f3cf802dd6 Cleanups, refactoring 2024-08-25 07:53:19 -06:00
blepping 2832969aea Sync with main 2024-08-25 04:35:52 -06:00