Eric Rollei
ea06cf9157
INT8 Instruct Fix (5 bugs): - blocks_to_swap no longer forced to 0 for INT8 in generate() - Added _load_int8_block_swap() method in loader with CB/SCB guard hooks - Fixed INT8 model size from 40GB to 80GB in memory budget - CB/SCB now fixed after .to(device) in _move_non_block_components_to_gpu() - Quant-specific gb_per_block in _calculate_optimal_config() VAE Decode Fix: - Fixed super() crash from closure cell corruption in cache clearing - Added isinstance(val, type) guard to prevent __class__ cell nuke - Cleaned up instance-level forward attrs after hook removal (4 files) New Features: - Experimental latent control nodes (Empty Latent, Noise Shaping, Generate with Latent) - Unified Generate V2 node (auto-detects NF4/INT8/BF16, block swap, memory mgmt) - HuggingFace upload script for v2 models (quantization/upload_v2_to_hf.py) README: - Updated to v1.3.0 with v2 HuggingFace model links - INT8 Instruct status changed to Working across all tables - Added Latent Control Nodes documentation section - Added v2 download links for base and Instruct models"
Eric_Hunyuan3 Quantization Tools
Tools for quantizing HunyuanImage-3.0 models to NF4 format for efficient GPU usage.
Quick Start
python hunyuan_quantize_nf4.py \
--model-path "/path/to/HunyuanImage-3" \
--output-path "/path/to/HunyuanImage-3-NF4"
What Gets Quantized
NF4 Quantization Applied To:
- Feed-forward networks (FFN/MLP layers)
- Large linear transformations
- Expert layers in MoE architecture
Kept in Full Precision (BF16):
- VAE encoder/decoder (critical for image quality)
- Attention projection layers (q_proj, k_proj, v_proj, o_proj)
- Patch embedding layers
- Time embedding layers
- Final output layers
Results
- Model Size: 80GB → ~20GB on disk
- VRAM Usage: 80GB → ~45GB during inference
- Quality: Minimal degradation (attention kept in full precision)
- Speed: Slightly slower than full BF16, much faster than CPU offload
Requirements
- 80GB+ disk space for source model
- 24GB+ VRAM for quantization process
- bitsandbytes library
- transformers library
Advanced Options
python hunyuan_quantize_nf4.py \
--model-path "./HunyuanImage-3" \
--output-path "./HunyuanImage-3-NF4" \
--no-double-quant \ # Disable nested quantization
--compute-dtype float16 \ # Use float16 instead of bfloat16
--device-map cuda:0 # Specific GPU
Troubleshooting
Out of memory during quantization:
- Close other applications
- Use
--device-map auto(may use CPU) - Ensure you have swap space available
Quantized model produces artifacts:
- Re-run quantization (ensure latest script)
- Check that attention layers are excluded
- Verify compute-dtype is bfloat16
Files Generated
After quantization, you'll have:
HunyuanImage-3-NF4/
├── model-00001-of-00010.safetensors
├── model-00002-of-00010.safetensors
├── ...
├── config.json
├── quantization_metadata.json
├── load_quantized.py (helper script)
└── tokenizer files
Loading Quantized Model
In ComfyUI, use the "Hunyuan 3 Loader (NF4)" node and select "HunyuanImage-3-NF4" from the dropdown.
The quantization config is automatically applied from metadata.