blepping
379175e7bf
Refactor tensor type checking a bit more
2024-09-01 18:30:47 -06:00
blepping
a0cb97eceb
Add LayerNorm and GroupNorm ops
...
Delegate to superclass methods whenever possible
2024-09-01 05:25:32 -06:00
City
c231239fdf
cleanup
...
Perf difference from `torch.tensor(tensor.data)` doesn't seem realistically measurable and the warning is annoying.
2024-08-30 03:47:52 +02:00
City
f40b149bc5
Add BF16 support
2024-08-30 03:39:57 +02:00
blepping
7413de86e6
Revert tensor construction change in dequantize_tensor
...
Eliminate GGMLOpsLayer, include functionality in GGMLLayer
2024-08-25 10:35:39 -06:00
blepping
2832969aea
Sync with main
2024-08-25 04:35:52 -06:00
City
0ba6f64974
Advanced loader with optimizations
...
It may help with speed but it may also increase VRAM usage or degrade image quality.
Fastest would be target/target/true.
2024-08-24 18:39:34 +02:00
blepping
c93e622a19
SD1/SDXL fixes
2024-08-23 20:19:48 -06:00
City
6b499ec49e
Hotfix to work as normal tensor
...
Fixes [ViT-L-14-BEST...] not working
2024-08-23 02:13:48 +02:00
City
a8bfdcfbd2
Fix Q4_0
...
#41
2024-08-18 09:44:46 +02:00
City
6295603d08
K quants
2024-08-18 04:26:47 +02:00
City
39a45c65d8
Better tensor split
2024-08-18 03:00:44 +02:00
City
2877c18573
Less fragile logic
2024-08-17 04:42:59 +02:00
City
d4c75a69c9
Change dequant logic to FP16
...
Benefit of FP32 for dequantization seems less noticable and feels more like a sidegrade due to differences in numerical precision. Was causing issues with memory not being freed.
#16
2024-08-16 01:26:57 +02:00
City
88fb6fa001
Improve Q8_0 and Q4_0, Add Q4_1 and Q5_1
...
Use FP32 to do the calculations while dequantizing to improve existing Q8_0 quants and to avoid double dtype conversion.
Add Q4_1 and Q5_1 dequant kernels
2024-08-15 18:39:24 +02:00
City
a8ce4e8d7c
Initial commit
2024-08-15 05:50:54 +02:00
blepping