7 Commits

Include renames

Author	SHA1	Message	Date
City	a8bfdcfbd2	Fix Q4_0 ... #41	2024-08-18 09:44:46 +02:00
City	6295603d08	K quants	2024-08-18 04:26:47 +02:00
City	39a45c65d8	Better tensor split	2024-08-18 03:00:44 +02:00
City	2877c18573	Less fragile logic	2024-08-17 04:42:59 +02:00
City	d4c75a69c9	Change dequant logic to FP16 ... Benefit of FP32 for dequantization seems less noticable and feels more like a sidegrade due to differences in numerical precision. Was causing issues with memory not being freed. #16	2024-08-16 01:26:57 +02:00
City	88fb6fa001	Improve Q8_0 and Q4_0, Add Q4_1 and Q5_1 ... Use FP32 to do the calculations while dequantizing to improve existing Q8_0 quants and to avoid double dtype conversion. Add Q4_1 and Q5_1 dequant kernels	2024-08-15 18:39:24 +02:00
City	a8ce4e8d7c	Initial commit	2024-08-15 05:50:54 +02:00