City
a8bfdcfbd2
Fix Q4_0
...
#41
2024-08-18 09:44:46 +02:00
City
a39bb22bf6
Merge pull request #40 from city96/kquants
...
K-quant support
2024-08-18 08:23:29 +02:00
City
4b239d2d75
Fix name change / regular unet list crosstalk
...
#39
2024-08-18 05:02:34 +02:00
City
724b2d8559
LoRA issue workaround
...
Proposed by @ahiser1117 in #37
2024-08-18 04:29:58 +02:00
City
d93fbc81f6
Test script
2024-08-18 04:27:17 +02:00
City
6295603d08
K quants
2024-08-18 04:26:47 +02:00
City
39a45c65d8
Better tensor split
2024-08-18 03:00:44 +02:00
City
a1953693b3
Fix invalid file type in metadata
2024-08-18 02:59:49 +02:00
City
4e848f7c95
Change model patcher logic
...
Janky MethodType -> Inheritance
#31
2024-08-17 11:43:51 +02:00
City
e3f0a4b127
Update README.md
2024-08-17 08:44:55 +02:00
City
c85a8c0db8
Merge pull request #29 from city96/lora
...
Native LoRA support
2024-08-17 08:05:56 +02:00
City
96b0ba1021
Allow using the native LoRA loader node
2024-08-17 07:56:38 +02:00
City
22e2c885f0
Async patch load with GPU support
2024-08-17 07:38:55 +02:00
City
fed8ff1e70
Add LoRA support
...
Initial, will need to change weight function
2024-08-17 05:28:46 +02:00
City
2877c18573
Less fragile logic
2024-08-17 04:42:59 +02:00
City
b5a3646aae
Fix state dict names
2024-08-17 01:57:57 +02:00
City
7e09d35f57
Update README.md
2024-08-16 17:01:12 +02:00
ddh0
51c95d7d70
fallback to F32 for 1-dimensional or very small tensors ( #20 )
...
* change fallback data type from F16 to F32
* only fallback to F32 for 1D or very small tensors
* change exception fallback type
* blacklist before small tensor check
2024-08-16 06:43:59 +02:00
City
de5f6e9dcc
mmap loading, lowvram support
...
This should fix both types of OOM issues!
2024-08-16 04:29:14 +02:00
City
d4c75a69c9
Change dequant logic to FP16
...
Benefit of FP32 for dequantization seems less noticable and feels more like a sidegrade due to differences in numerical precision. Was causing issues with memory not being freed.
#16
2024-08-16 01:26:57 +02:00
City
88fb6fa001
Improve Q8_0 and Q4_0, Add Q4_1 and Q5_1
...
Use FP32 to do the calculations while dequantizing to improve existing Q8_0 quants and to avoid double dtype conversion.
Add Q4_1 and Q5_1 dequant kernels
2024-08-15 18:39:24 +02:00
City
303af6ff75
Upload basic quantization code
2024-08-15 08:09:59 +02:00
City
15e3f45ad7
Update README.md
2024-08-15 07:05:43 +02:00
City
6aa75db166
Update README.md
2024-08-15 06:41:47 +02:00
City
8b8416dea1
Update README.md
2024-08-15 06:30:33 +02:00
City
4750e45623
Update README.md
2024-08-15 06:21:19 +02:00
City
a8ce4e8d7c
Initial commit
2024-08-15 05:50:54 +02:00
City
7368f2d391
Initial commit
2024-08-15 05:26:26 +02:00
City