City
e024aab10d
Torch compile possible workaround
...
Seems to help on torch 2.6.0 with triton 3.2.0, less so on old versions.
Includes workaround for 2.0.X and other really old versions
#118
2025-02-23 18:49:50 +01:00
City
51fa2cbb5c
Account for dequantization VRAM to avoid OOM
...
#187
2024-12-22 18:09:51 +01:00
City
eaffa0a7d3
Change clip workaround for SD3
...
#186
2024-12-22 15:24:14 +01:00
City
65a7c895bb
Better propagation fix
...
#163
2024-12-14 00:38:11 +01:00
City
851564739d
Updated Intel Arc fix
...
Provided by @simonlui in #50 as a patch
2024-12-05 03:16:23 +01:00
City
39102df73f
Stop custom tensor class propagating past dequant
...
#163
2024-12-04 16:14:54 +01:00
City
454955ead3
Skip initialization for nn.Linear
...
Attempt to reserve less memory by not initializing linear layers until the weights are loaded.
#102 #95
2024-09-09 01:37:11 +02:00
City
929d15454d
Rename ggml_forward
...
Makes it clear that this is the same thing as comfy_cast_weights
2024-09-02 03:58:26 +02:00
blepping
fa1671d63e
Sync with upstream
2024-09-01 18:44:31 -06:00
blepping
379175e7bf
Refactor tensor type checking a bit more
2024-09-01 18:30:47 -06:00
City
ba8ce44b4c
Rename move_patch function
2024-09-02 02:22:32 +02:00
blepping
e19fe8e898
Rename ggml_check_delegate to is_ggml_quantized, refactor accordingly
2024-09-01 18:04:31 -06:00
blepping
a0cb97eceb
Add LayerNorm and GroupNorm ops
...
Delegate to superclass methods whenever possible
2024-09-01 05:25:32 -06:00
blepping
436ecb4913
Merge upstream ops_refactor
2024-09-01 03:02:59 -06:00
City
6618801f1c
Don't dequantize token embed on load
2024-08-30 06:25:19 +02:00
City
c231239fdf
cleanup
...
Perf difference from `torch.tensor(tensor.data)` doesn't seem realistically measurable and the warning is annoying.
2024-08-30 03:47:52 +02:00
City
ec24e75d89
Remove unused check
2024-08-30 03:40:15 +02:00
City
826bd1804c
Make ops less jank
...
Also possibly faster. Ignore the massively increased system memory usage.
2024-08-30 03:25:57 +02:00
blepping
e814fc1264
Revert ggml_tensor_name addition
2024-08-28 12:34:08 -06:00
blepping
6ae0c733b0
Support loading u-net from stable-diffusion.cpp GGUF files
...
Small cleanups
2024-08-28 10:34:58 -06:00
City
258742c369
Fix Conv2D with bias
2024-08-26 17:49:18 +02:00
blepping
7413de86e6
Revert tensor construction change in dequantize_tensor
...
Eliminate GGMLOpsLayer, include functionality in GGMLLayer
2024-08-25 10:35:39 -06:00
blepping
f3cf802dd6
Cleanups, refactoring
2024-08-25 07:53:19 -06:00
blepping
2832969aea
Sync with main
2024-08-25 04:35:52 -06:00
City
0ba6f64974
Advanced loader with optimizations
...
It may help with speed but it may also increase VRAM usage or degrade image quality.
Fastest would be target/target/true.
2024-08-24 18:39:34 +02:00
City
90a63c0c0e
Fix for Intel Arc
...
Deep dive & fix by @simonlui in #50
2024-08-24 08:17:06 +02:00
blepping
c93e622a19
SD1/SDXL fixes
2024-08-23 20:19:48 -06:00
City
6b499ec49e
Hotfix to work as normal tensor
...
Fixes [ViT-L-14-BEST...] not working
2024-08-23 02:13:48 +02:00
City
2ca943a1c3
Hotfix for clip
...
I'm sure this won't break a million other things
2024-08-20 02:42:22 +02:00
City
2e5e599b54
CLIP Loaders
2024-08-20 02:23:37 +02:00
City
724b2d8559
LoRA issue workaround
...
Proposed by @ahiser1117 in #37
2024-08-18 04:29:58 +02:00
City
22e2c885f0
Async patch load with GPU support
2024-08-17 07:38:55 +02:00
City
fed8ff1e70
Add LoRA support
...
Initial, will need to change weight function
2024-08-17 05:28:46 +02:00
City
2877c18573
Less fragile logic
2024-08-17 04:42:59 +02:00
City
b5a3646aae
Fix state dict names
2024-08-17 01:57:57 +02:00
City
de5f6e9dcc
mmap loading, lowvram support
...
This should fix both types of OOM issues!
2024-08-16 04:29:14 +02:00
City
88fb6fa001
Improve Q8_0 and Q4_0, Add Q4_1 and Q5_1
...
Use FP32 to do the calculations while dequantizing to improve existing Q8_0 quants and to avoid double dtype conversion.
Add Q4_1 and Q5_1 dequant kernels
2024-08-15 18:39:24 +02:00
City
a8ce4e8d7c
Initial commit
2024-08-15 05:50:54 +02:00
City