nodes/city96-ComfyUI-GGUF
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
156 Commits 4 Branches 0 Tags
9d282bcdfd3299e2bc9f0f55a4ac535cbf340935
Commit Graph
156 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
City 9d282bcdfd LLaMA OOM hotfix 2025-04-21 04:25:30 +02:00
City a355f8b5ea Skip unlink logic if load/offload dev match 
i.e. when forcing it to run on CPU or highvram enabled
 2025-04-18 23:53:12 +02:00
City 3564e95b26 Dynamic CLIP input types (#263) 2025-04-18 23:48:53 +02:00
City e29f526c68 Disable mixing scaled FP8 + GGUF 2025-04-16 16:07:51 +02:00
City 47bec61475 HiDream text encoder support 
LLaMA GGUF logic already works due to being used for HunYuan
 2025-04-16 06:10:28 +02:00
City 3605bfbead Keep ffn down for HiDream 2025-04-16 02:22:31 +02:00
City 2ef5a66a9f HiDream support 
#248
 2025-04-16 01:43:15 +02:00
City 794fc28536 Clean up logging 
Also fixes numpy warning
 2025-04-13 23:01:15 +02:00
City 6de4bdba30 Merge pull request #216 from city96/convert_refactor_new 
convert.py/patch refactor
 2025-04-10 08:11:58 +02:00
City ca97e050b3 Update README.md 2025-04-10 08:04:17 +02:00
City de33c984c1 Merge branch 'main' into convert_refactor_new 2025-04-10 07:12:14 +02:00
City 624687939f Refactor 5D patch and save loop 2025-04-10 07:12:10 +02:00
Xiangxi Guo (Ryan) 0c3bdf336e Enable more torch.compile optimization by reducing graph breaks (#243) 
As title, there were 2 fundamental sources of graph breaks:
1. using a pre-existing set in a `torch.compile` region, this is a
   [known issue](https://github.com/pytorch/pytorch/issues/145761) in
   PyTorch and the fix is WIP, but it's a lot easier to change the set
   into a tuple here to fix the graph break.
2. using tensor subclass `GGMLTensor`. The full support of tensor
   subclass has been gated for `torch.compile`, but with [a recent WIP
   stack](https://github.com/pytorch/pytorch/pull/149792),
   we'll be able to remove the gating and have `torch.compile` fully
   support the `GGMLTensor` in this repo. The only required change here
   is avoid changing the `__class__` attribute, and instead using
   `torch.Tensor` to convert the subclass object back to Tensor.
 2025-04-10 02:46:33 +02:00
City 862e4147c4 Make 5D tensor fix less fragile 2025-04-03 00:38:46 +02:00
City c36b7236e6 Update README.md 2025-03-30 18:46:09 +02:00
City f3b968e137 Merge branch 'main' into convert_refactor_new 2025-03-30 18:43:44 +02:00
City bbd89ae547 Update .gitignore 2025-03-30 18:42:17 +02:00
City 65a2bc20bb wan bf16 fix 2025-03-28 04:21:26 +01:00
City 1c873edeb6 Expose strip prefix as function 
For external convert/detection logic
 2025-03-22 20:40:24 +01:00
City bc5223b0e3 Fix qtype logging on newer gguf-py 2025-03-16 20:14:33 +01:00
City 4aebf1bf2c Add compat mode pig 
closes #230
 2025-03-16 19:38:40 +01:00
City 5aa6e98ad7 Merge pull request #224 from city96/tokenizer_convert 
Sentencepiece tokenizer from gguf metadata for UMT5
 2025-03-14 17:53:02 +01:00
City ad69decc8d Update requirements.txt 2025-03-14 17:43:05 +01:00
City 31dffa061b Add error message for missing packages 2025-03-14 17:37:14 +01:00
City 296b0c1419 Clean up field logic 2025-03-14 17:11:47 +01:00
City d99b2d931f force dequant on umT5 to prevent OOM 
not the best solution but that tensor alone needs ~3GB of runtime memory to dequant
 2025-03-14 16:56:48 +01:00
City d40e9bdcfe Use args for 5d fix 2025-03-14 16:48:01 +01:00
City 9d0a0e317e ltxv bf16 fix 
scale shift table loads into nn.parameter, which can't work with BF16
 2025-03-07 22:37:17 +01:00
City c3a6103af5 hyvid subkey load fix 2025-03-07 22:36:51 +01:00
City 01f6e3ff63 spiece tokenizer from gguf metadata 2025-03-05 04:44:53 +01:00
City 929a9e5f0d Keep img_emb in F32 for wan i2v 2025-02-27 05:58:01 +01:00
City 32a8eab8f9 Merge branch 'main' into convert_refactor_new 2025-02-27 03:19:19 +01:00
City 298192ed60 Wan model arch 2025-02-26 07:40:44 +01:00
City cd361cf0e3 Update README.md 2025-02-26 07:39:01 +01:00
YarvixPA 8832d37df2 Solution for patch files line endings and update for /tools README.md (#202) 
* Create convert_lines_ending.py

Solution to convert line ending from Windows (CRLF) to Unix (LF)

* Update README.md

* Rename convert_lines_ending.py to fix_lines_ending.py

* Update README.md
 2025-02-26 07:35:42 +01:00
City 355b43228c Wan2.1 convert logic 2025-02-26 07:06:54 +01:00
City e024aab10d Torch compile possible workaround 
Seems to help on torch 2.6.0 with triton 3.2.0, less so on old versions.
Includes workaround for 2.0.X and other really old versions
#118
 2025-02-23 18:49:50 +01:00
City abdfec54c4 Hunyuan Jank 2025-02-20 05:10:27 +01:00
City 2d46bcd61b Sanity checks in conversion code 2025-01-08 17:45:37 +01:00
City 8d313a677a Unused/old logic 2025-01-08 17:17:49 +01:00
City 5875c52f59 Unpin numpy version 
#196 #7
 2025-01-08 16:30:17 +01:00
City 8098c7149f Add PixArt clip type 
Also makes the map more readable
#195
 2025-01-04 21:46:23 +01:00
City 3dc384b233 More robust folder path logic 
Attempts to override the unet_gguf key with our current one if it exists + print a warning.
Should only be triggered if another custom node edits the expected key in some way.
 2024-12-26 21:42:33 +01:00
City c6ad4f2363 Account for not having quantized weights 
#188
 2024-12-22 19:00:03 +01:00
City 51fa2cbb5c Account for dequantization VRAM to avoid OOM 
#187
 2024-12-22 18:09:51 +01:00
City eaffa0a7d3 Change clip workaround for SD3 
#186
 2024-12-22 15:24:14 +01:00
City 8d1b56c5e2 Loader logic for LLaMA models 
Tokenizer shape may be mismatched depending which model was used as source (128320 VS 128256) - warning instead of error for now.
k and q attn keys need to be reshaped back to HF format (inverse of convert_hf_to_gguf.py logic)
 2024-12-20 13:39:17 +01:00
City 0a1723baae Split loader into separate file 2024-12-20 13:03:27 +01:00
City 0b65261cc4 HunyuanVideo 
Convert script needs fixing, had to use a second script for the 5D tensor, will clean up and upload it later.
 2024-12-17 07:33:42 +01:00
City 65a7c895bb Better propagation fix 
#163
 2024-12-14 00:38:11 +01:00