Sometimes gguf will return size-one one-dimensional memmaps for scalar values. I'm not exactly sure when or why, but this will accept any dimension as long as it only contains one value
Resolves#384
* Implement GGUF metadata extraction function
Added function to extract metadata from GGUF files.
* Updated the GGUF model loading and patching classes to include metadata handling.
* Clean up return logic for extra metadata
This should be more future proof in case we need to return other attributes in the future. Possible breaking change for anyone using `gguf_sd_loader` directly either way, though.
---------
Co-authored-by: City <125218114+city96@users.noreply.github.com>
According to #361 the thinking/abliterated versions work even though the model wasn't trained with them, maybe the vision one does too.
Does not have mmproj support, since nothing uses that atm.
Tokenizer shape may be mismatched depending which model was used as source (128320 VS 128256) - warning instead of error for now.
k and q attn keys need to be reshaped back to HF format (inverse of convert_hf_to_gguf.py logic)
jayn7