nodes/city96-ComfyUI-GGUF
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
156 Commits 4 Branches 0 Tags
9d282bcdfd3299e2bc9f0f55a4ac535cbf340935
Commit Graph
40 Commits
Author SHA1 Message Date
City 794fc28536 Clean up logging 
Also fixes numpy warning
 2025-04-13 23:01:15 +02:00
Xiangxi Guo (Ryan) 0c3bdf336e Enable more torch.compile optimization by reducing graph breaks (#243) 
As title, there were 2 fundamental sources of graph breaks:
1. using a pre-existing set in a `torch.compile` region, this is a
   [known issue](https://github.com/pytorch/pytorch/issues/145761) in
   PyTorch and the fix is WIP, but it's a lot easier to change the set
   into a tuple here to fix the graph break.
2. using tensor subclass `GGMLTensor`. The full support of tensor
   subclass has been gated for `torch.compile`, but with [a recent WIP
   stack](https://github.com/pytorch/pytorch/pull/149792),
   we'll be able to remove the gating and have `torch.compile` fully
   support the `GGMLTensor` in this repo. The only required change here
   is avoid changing the `__class__` attribute, and instead using
   `torch.Tensor` to convert the subclass object back to Tensor.
 2025-04-10 02:46:33 +02:00
City e024aab10d Torch compile possible workaround 
Seems to help on torch 2.6.0 with triton 3.2.0, less so on old versions.
Includes workaround for 2.0.X and other really old versions
#118
 2025-02-23 18:49:50 +01:00
City 51fa2cbb5c Account for dequantization VRAM to avoid OOM 
#187
 2024-12-22 18:09:51 +01:00
City eaffa0a7d3 Change clip workaround for SD3 
#186
 2024-12-22 15:24:14 +01:00
City 65a7c895bb Better propagation fix 
#163
 2024-12-14 00:38:11 +01:00
City 851564739d Updated Intel Arc fix 
Provided by @simonlui in #50 as a patch
 2024-12-05 03:16:23 +01:00
City 39102df73f Stop custom tensor class propagating past dequant 
#163
 2024-12-04 16:14:54 +01:00
City 454955ead3 Skip initialization for nn.Linear 
Attempt to reserve less memory by not initializing linear layers until the weights are loaded.
#102 #95
 2024-09-09 01:37:11 +02:00
City 929d15454d Rename ggml_forward 
Makes it clear that this is the same thing as comfy_cast_weights
 2024-09-02 03:58:26 +02:00
blepping fa1671d63e Sync with upstream 2024-09-01 18:44:31 -06:00
blepping 379175e7bf Refactor tensor type checking a bit more 2024-09-01 18:30:47 -06:00
City ba8ce44b4c Rename move_patch function 2024-09-02 02:22:32 +02:00
blepping e19fe8e898 Rename ggml_check_delegate to is_ggml_quantized, refactor accordingly 2024-09-01 18:04:31 -06:00
blepping a0cb97eceb Add LayerNorm and GroupNorm ops 
Delegate to superclass methods whenever possible
 2024-09-01 05:25:32 -06:00
blepping 436ecb4913 Merge upstream ops_refactor 2024-09-01 03:02:59 -06:00
City 6618801f1c Don't dequantize token embed on load 2024-08-30 06:25:19 +02:00
City c231239fdf cleanup 
Perf difference from `torch.tensor(tensor.data)` doesn't seem realistically measurable and the warning is annoying.
 2024-08-30 03:47:52 +02:00
City ec24e75d89 Remove unused check 2024-08-30 03:40:15 +02:00
City 826bd1804c Make ops less jank 
Also possibly faster. Ignore the massively increased system memory usage.
 2024-08-30 03:25:57 +02:00
blepping e814fc1264 Revert ggml_tensor_name addition 2024-08-28 12:34:08 -06:00
blepping 6ae0c733b0 Support loading u-net from stable-diffusion.cpp GGUF files 
Small cleanups
 2024-08-28 10:34:58 -06:00
City 258742c369 Fix Conv2D with bias 2024-08-26 17:49:18 +02:00
blepping 7413de86e6 Revert tensor construction change in dequantize_tensor 
Eliminate GGMLOpsLayer, include functionality in GGMLLayer
 2024-08-25 10:35:39 -06:00
blepping f3cf802dd6 Cleanups, refactoring 2024-08-25 07:53:19 -06:00
blepping 2832969aea Sync with main 2024-08-25 04:35:52 -06:00
City 0ba6f64974 Advanced loader with optimizations 
It may help with speed but it may also increase VRAM usage or degrade image quality.
Fastest would be target/target/true.
 2024-08-24 18:39:34 +02:00
City 90a63c0c0e Fix for Intel Arc 
Deep dive & fix by @simonlui in #50
 2024-08-24 08:17:06 +02:00
blepping c93e622a19 SD1/SDXL fixes 2024-08-23 20:19:48 -06:00
City 6b499ec49e Hotfix to work as normal tensor 
Fixes [ViT-L-14-BEST...] not working
 2024-08-23 02:13:48 +02:00
City 2ca943a1c3 Hotfix for clip 
I'm sure this won't break a million other things
 2024-08-20 02:42:22 +02:00
City 2e5e599b54 CLIP Loaders 2024-08-20 02:23:37 +02:00
City 724b2d8559 LoRA issue workaround 
Proposed by @ahiser1117 in #37
 2024-08-18 04:29:58 +02:00
City 22e2c885f0 Async patch load with GPU support 2024-08-17 07:38:55 +02:00
City fed8ff1e70 Add LoRA support 
Initial, will need to change weight function
 2024-08-17 05:28:46 +02:00
City 2877c18573 Less fragile logic 2024-08-17 04:42:59 +02:00
City b5a3646aae Fix state dict names 2024-08-17 01:57:57 +02:00
City de5f6e9dcc mmap loading, lowvram support 
This should fix both types of OOM issues!
 2024-08-16 04:29:14 +02:00
City 88fb6fa001 Improve Q8_0 and Q4_0, Add Q4_1 and Q5_1 
Use FP32 to do the calculations while dequantizing to improve existing Q8_0 quants and to avoid double dtype conversion.
Add Q4_1 and Q5_1 dequant kernels
 2024-08-15 18:39:24 +02:00
City a8ce4e8d7c Initial commit 2024-08-15 05:50:54 +02:00