 WildAi
|
29488e4a1f
|
feat: unload-on-change eviction + quant-resident runtime for GGUF and quantized safetensors (v2.4.0 -> v2.5.0)
Model management (v2.4.0):
- single-active-per-family registry releases the previous model fully
(RAM, VRAM, ComfyUI current_loaded_models) before a new one loads
- file-identity cache keys (basename+mtime+size+attention+q4+dtype)
prevent cross-file collisions; ASR request keys share the consumer
namespace so identical re-runs never spuriously evict
Quant-resident runtime (v2.5.0):
- GGUF weights stay raw-block resident end-to-end: uint8 parameters,
per-matmul dequant kernels for Q8_0/Q4_K/Q5_K/Q6_K pinned bitwise to
the gguf-py oracle; F32/F16/BF16 pass through at native dtype via
zero-copy views; load-time RAM spike (~2x float size) eliminated
- quantized safetensors via *.comfy_quant metadata: rotated ConvRot
INT8 residents through comfy-kitchen, plain rowwise int8 / fp8
e4m3+e5m2 / int8_blockwise dequant-at-load (per-row, scalar, and
per-gs-block scale layouts); unsupported formats hard-fail with
actionable errors; dense gate rejects unplanned quant storages;
rotated non-Linear targets (embeddings) fail with re-export guidance
- dtype casts filter quant-resident storage (_quant_resident markers,
fp32 weight_scale protection); SageAttention wrapper resolves
activation dtype per module kind, fixing uint8-weight crash
Tests: 937 passed / 5 pre-existing failures / 4 skipped
|
2026-08-26 12:17:24 +03:00 |
|