Adds 'mxfp8' quantization option to the model loader dropdown with full
hardware-accelerated matmul via comfy-kitchen on Blackwell GPUs (CC>=10)
and automatic dequantization fallback on unsupported hardware.
Core implementation:
- fp8_optimization.py: new mxfp8_linear_forward(), convert_mxfp8_linear(),
and dequantize_mxfp8_weight() functions. Forward uses ck.quantize_mxfp8
for activation quantization and ck.scaled_mm_mxfp8 for block-scaled
matmul. Fallback dequantizes weights via ck.dequantize_mxfp8 (or manual
from_blocked + E8M0 conversion when ck is unavailable).
- nodes_model_loading.py: mxfp8 in quantization dropdown, auto-detection
of float8_e8m0fnu/uint8 block scales in state dicts (3-key conventions:
.weight_scale / .scale_weight / .block_scale_weight), per-tensor FP8
substring guard ('fp8' in quantization excludes 'mxfp8'), params_to_keep
dequantization for excluded layers (text_embedding, time_*, img_emb,
modulation, adapter, audio_proj), merged/unmerged LoRA compatibility via
load-time dequant, VRAM management guard, torch.compile warning, SD key
collision detection, T5 encoder MXFP8 rejection.
Safety and robustness:
- _E8M0_DTYPES module constant with hasattr guard for older PyTorch
- safetensors uint8↔float8_e8m0fnu normalization throughout the chain
- except Exception with OOM re-raise and one-shot fallback warning
- original_forward double-patch guard in both converters
- fp8_linear_forward MXFP8 rejection for wrong-forward-path protection
- block_scale_weight registered as persistent buffer (state_dict/device)
- padded matmul output slicing, bias device movement, meta-tensor guard
- Nodesampler + SkyReels autocast guards, offload/reload SD-is-None guards
- Error message clarity throughout
Documentation: README.md/README_ZH.md quantization section with hardware
requirements, auto-detection, fallback behavior, and optional install
instructions for comfy-kitchen. NOTICE updated with Apache 2.0 attribution.
comfy-kitchen is NOT a hard dependency — absent on unsupported hardware.
wuwukasi
c0b8e50239