nodes/city96-ComfyUI-GGUF
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
43 Commits 4 Branches 0 Tags
0ba6f6497487187668753d5381e9ca304dc033ef
Commit Graph
9 Commits
Author SHA1 Message Date
City 0ba6f64974 Advanced loader with optimizations 
It may help with speed but it may also increase VRAM usage or degrade image quality.
Fastest would be target/target/true.
 2024-08-24 18:39:34 +02:00
City 6b499ec49e Hotfix to work as normal tensor 
Fixes [ViT-L-14-BEST...] not working
 2024-08-23 02:13:48 +02:00
City a8bfdcfbd2 Fix Q4_0 
#41
 2024-08-18 09:44:46 +02:00
City 6295603d08 K quants 2024-08-18 04:26:47 +02:00
City 39a45c65d8 Better tensor split 2024-08-18 03:00:44 +02:00
City 2877c18573 Less fragile logic 2024-08-17 04:42:59 +02:00
City d4c75a69c9 Change dequant logic to FP16 
Benefit of FP32 for dequantization seems less noticable and feels more like a sidegrade due to differences in numerical precision. Was causing issues with memory not being freed.
#16
 2024-08-16 01:26:57 +02:00
City 88fb6fa001 Improve Q8_0 and Q4_0, Add Q4_1 and Q5_1 
Use FP32 to do the calculations while dequantizing to improve existing Q8_0 quants and to avoid double dtype conversion.
Add Q4_1 and Q5_1 dequant kernels
 2024-08-15 18:39:24 +02:00
City a8ce4e8d7c Initial commit 2024-08-15 05:50:54 +02:00