Eric Rollei
779e5c363b
Features: - Multiple loader nodes (BF16, NF4, Multi-GPU, optimized variants) - Advanced generation with official HunyuanImage-3.0 prompt rewriting - Supports any OpenAI-compatible LLM API (DeepSeek, OpenAI, Claude, local LLMs) - Organized resolution controls with MP indicators - Large image generation with CPU offload support - Comprehensive VRAM management and error handling - NF4 quantization tools included License: CC BY-NC 4.0 (Non-Commercial) / Commercial License Available Author: Eric Hiss (github.com/ericRollei) Model: HunyuanImage-3.0 by Tencent (Apache 2.0)
2.2 KiB
2.2 KiB
Eric_Hunyuan3 Quantization Tools
Tools for quantizing HunyuanImage-3.0 models to NF4 format for efficient GPU usage.
Quick Start
python hunyuan_quantize_nf4.py \
--model-path "/path/to/HunyuanImage-3" \
--output-path "/path/to/HunyuanImage-3-NF4"
What Gets Quantized
NF4 Quantization Applied To:
- Feed-forward networks (FFN/MLP layers)
- Large linear transformations
- Expert layers in MoE architecture
Kept in Full Precision (BF16):
- VAE encoder/decoder (critical for image quality)
- Attention projection layers (q_proj, k_proj, v_proj, o_proj)
- Patch embedding layers
- Time embedding layers
- Final output layers
Results
- Model Size: 80GB → ~20GB on disk
- VRAM Usage: 80GB → ~45GB during inference
- Quality: Minimal degradation (attention kept in full precision)
- Speed: Slightly slower than full BF16, much faster than CPU offload
Requirements
- 80GB+ disk space for source model
- 24GB+ VRAM for quantization process
- bitsandbytes library
- transformers library
Advanced Options
python hunyuan_quantize_nf4.py \
--model-path "./HunyuanImage-3" \
--output-path "./HunyuanImage-3-NF4" \
--no-double-quant \ # Disable nested quantization
--compute-dtype float16 \ # Use float16 instead of bfloat16
--device-map cuda:0 # Specific GPU
Troubleshooting
Out of memory during quantization:
- Close other applications
- Use
--device-map auto(may use CPU) - Ensure you have swap space available
Quantized model produces artifacts:
- Re-run quantization (ensure latest script)
- Check that attention layers are excluded
- Verify compute-dtype is bfloat16
Files Generated
After quantization, you'll have:
HunyuanImage-3-NF4/
├── model-00001-of-00010.safetensors
├── model-00002-of-00010.safetensors
├── ...
├── config.json
├── quantization_metadata.json
├── load_quantized.py (helper script)
└── tokenizer files
Loading Quantized Model
In ComfyUI, use the "Hunyuan 3 Loader (NF4)" node and select "HunyuanImage-3-NF4" from the dropdown.
The quantization config is automatically applied from metadata.