nodes/city96-ComfyUI-GGUF
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
97 Commits 4 Branches 0 Tags
98333480059a2ccafb4718924ebcb9cdcb9b1f43
Commit Graph
97 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
City 9833348005 Fix for text encoder folder name change 
#151
 2024-11-03 14:03:42 +01:00
City 6561064dcf Add SD3.5 links 2024-10-23 04:38:39 +02:00
City b70240508f Fix SD3 patch metadata/tensor issue 
#136
 2024-10-23 03:16:14 +02:00
City 4d7bb93397 Add initial SD3 lcpp patch + instruction 
#136
 2024-10-23 01:39:45 +02:00
City 0ef158d793 Add SD3 to loader arch list 2024-10-22 22:00:37 +02:00
City 68ad5fb214 Update README.md 2024-10-15 04:11:45 +02:00
City d090a910ac Rewrite convert.py to work properly with dev de-distill 
Original logic was triggering SDXL specific code to reshape img_in weight, causing forge not to work with it.
Tagging @blepping since this might change how the SDXL code is evaluated, but based on a quick check it should work.
 2024-10-15 03:56:54 +02:00
City d2aaeb0f13 Always init tenc on offload device 
#83
needs updated ComfyUI as well to fix it, issue was with the direct load logic in model management putting the model on the GPU before the weights were loaded, at which point the loading would overwrite the weights with the CPU ones
 2024-09-17 15:26:39 +02:00
City 454955ead3 Skip initialization for nn.Linear 
Attempt to reserve less memory by not initializing linear layers until the weights are loaded.
#102 #95
 2024-09-09 01:37:11 +02:00
City ac96699b7a Update README.md 2024-09-06 18:41:49 +02:00
City 0342f0a183 never double patch 
#98
Instead of adding/removing patches, sync with `self.patches[key]` - old patch should be GC'd. Most likely no legitimate reasons for double patching. Can simplify logic in ops.py if no issues.
 2024-09-06 18:10:02 +02:00
City b46c206dd7 double patch issue fix for lowvram 
Potential fix, lowvram weights are called via `patch_weight_to_device` on each run and it was possible to apply the same patch multiple times on subsequent generations.
 2024-09-05 05:32:01 +02:00
City 717a0e1d0b Temporarily disable instant patch logic 
Seems to break VRAM estimation and needs more involved fix with upstream ComfyUI repo.
 2024-09-03 17:00:20 +02:00
City c8923a4cd7 mmap unload fix for lowvram 2024-09-03 16:37:35 +02:00
City 6dbb4ba989 Temp fix for disable smart memory 
Model management seems to expect `unpatch_model` to put model on specified device after unpatching. Needs different way to avoid patch reload.
 2024-09-03 05:28:49 +02:00
City 69f0daffc5 Merge pull request #92 from city96/ops_refactor 
Fix LoRA reloading, ops refactor, better lowvram speed, sd.cpp compat
 2024-09-03 03:13:54 +02:00
City 929d15454d Rename ggml_forward 
Makes it clear that this is the same thing as comfy_cast_weights
 2024-09-02 03:58:26 +02:00
City 35007f2659 Refactor model loading 
Should make it more readable. Moved import under if to avoid changes to stuff in `tools` breaking normal models.
 2024-09-02 03:44:07 +02:00
City df3378da6b Detect quant 2024-09-02 03:17:16 +02:00
City cd343d1426 Merge pull request #80 from blepping/feat_load_stable_diffusion.cpp 
Support loading u-net from stable-diffusion.cpp GGUF files
 2024-09-02 02:51:40 +02:00
blepping fa1671d63e Sync with upstream 2024-09-01 18:44:31 -06:00
blepping 379175e7bf Refactor tensor type checking a bit more 2024-09-01 18:30:47 -06:00
City ba8ce44b4c Rename move_patch function 2024-09-02 02:22:32 +02:00
blepping e19fe8e898 Rename ggml_check_delegate to is_ggml_quantized, refactor accordingly 2024-09-01 18:04:31 -06:00
City c7b902bf9a Patch load/unload fix 
May break other things. For some reason T5 decided to run on the CPU with this change, but only like once and I can't for the life of me reproduce it.
 2024-09-02 02:00:10 +02:00
City 43427eff52 Better patch clear logic 
Reduces RAM overhead
 2024-09-02 01:37:16 +02:00
blepping 3bad1b4e04 Fix loading GGUF t5xxl 
Can't use squeeze proj workaround for stable-diffusion.cpp SD15
 2024-09-01 06:46:50 -06:00
blepping a0cb97eceb Add LayerNorm and GroupNorm ops 
Delegate to superclass methods whenever possible
 2024-09-01 05:25:32 -06:00
blepping 436ecb4913 Merge upstream ops_refactor 2024-09-01 03:02:59 -06:00
blepping ed0702fefc Merge branch 'city96:main' into feat_load_stable_diffusion.cpp 2024-09-01 02:47:16 -06:00
City 7f3ced6c3e Consider FP8 as source 
#91
 2024-09-01 00:56:30 +02:00
City 6618801f1c Don't dequantize token embed on load 2024-08-30 06:25:19 +02:00
City c231239fdf cleanup 
Perf difference from `torch.tensor(tensor.data)` doesn't seem realistically measurable and the warning is annoying.
 2024-08-30 03:47:52 +02:00
City ec24e75d89 Remove unused check 2024-08-30 03:40:15 +02:00
City f40b149bc5 Add BF16 support 2024-08-30 03:39:57 +02:00
City 826bd1804c Make ops less jank 
Also possibly faster. Ignore the massively increased system memory usage.
 2024-08-30 03:25:57 +02:00
blepping b88cb9a344 Remove require_prefix in gguf_sd_loader 2024-08-28 12:50:39 -06:00
blepping e814fc1264 Revert ggml_tensor_name addition 2024-08-28 12:34:08 -06:00
City abe5c6a6c9 Fix weights not being copied for lowvram patch 
This can probably be improved since the weights themselves never need to be copied as they cannot be changed anyway.
 2024-08-28 20:23:08 +02:00
blepping f7d8fbe740 Workaround for stable-diffusion.cpp SDXL GGUF files 2024-08-28 11:56:51 -06:00
blepping 6ae0c733b0 Support loading u-net from stable-diffusion.cpp GGUF files 
Small cleanups
 2024-08-28 10:34:58 -06:00
City 3d9869240b Always patch weight 
Should help with #33
 2024-08-28 17:12:24 +02:00
City 258742c369 Fix Conv2D with bias 2024-08-26 17:49:18 +02:00
City a9f6359a23 Merge pull request #63 from blepping/feat_sd1_sdxl 
Add support for SD1x and SDXL models
 2024-08-25 23:46:43 +02:00
blepping df6f1ab39c Sync with upstream 2024-08-25 15:32:45 -06:00
City bfecd32f8c Rewrite llama.cpp patch 
For #63
 2024-08-25 22:32:55 +02:00
blepping c1693d0f55 Use a GGUF array for orig shape metadata 2024-08-25 13:18:35 -06:00
blepping 0aec56ec4c Split orig shape metadata into individual fields 2024-08-25 12:00:28 -06:00
blepping 7413de86e6 Revert tensor construction change in dequantize_tensor 
Eliminate GGMLOpsLayer, include functionality in GGMLLayer
 2024-08-25 10:35:39 -06:00
blepping f3cf802dd6 Cleanups, refactoring 2024-08-25 07:53:19 -06:00