Bugs fixed:
1. Conv2d MXFP8/NVFP4 reshape crash: when quantize_mxfp8 pads flattened
dim (non-multiple-of-32), reshape back to 4D fails. Now falls back to
FP8 with descriptive message.
2. get_dequantized_weight None crash: when block_scale AND weight_scale
are both None (e.g., missing state_dict key), the FP8 fallback path
crashed. Now guards against None weight_scale with identity fallback.
3. _apply_quantized_weights overwrites scales with None: direct
assignment ('new_layer.weight_scale = None') even when state_dict
has no key. Now uses conditional assignment like set_quantized_weight().
- Add defensive dtype casts in QuantizedLinear.forward(),
QuantizedLinear._forward_svd(), QuantizedConv2d.forward()
- Fix bias mutation: use local variable instead of self.bias
- Update all node tooltips with GPU requirements, compression ratios,
and actionable guidance for each precision/parameter
- Fix [tool.comfyui] section: proper TOML syntax for version
- Bump version to 0.7.1
- Remove legacy version-specific troubleshooting entries
- Keep only current relevant troubleshooting items
OUTPUT_NODE with RETURN_TYPES=() must return empty tuple (), not dict {}. ComfyUI execution engine was failing with 'object of type NoneType has no len()' because the function had no return statement.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Bug fixes:
1. QuantizedConv2d.set_quantized_weight() now accepts svd_u/svd_s/svd_v parameters
- Was failing with 'unexpected keyword argument svd_u' for all Conv2d layers
- Added SVD buffer registration matching QuantizedLinear
- Conv2d layers still don't use SVD (only for 2D Linear weights) but accept the params
2. Fixed compression ratio calculation in ModelOptSaveQuantized
- Was calculating 'original size' from already-quantized model (wrong)
- Now captures original params/bytes BEFORE quantization in ModelOptQuantizeUNet
- Save node retrieves stored original_bytes from metadata
- Compression ratio now shows realistic values (~5-6x for SD1.5)
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
PyTorch is now an ambient dependency (provided by ComfyUI) - removed from our dependencies list entirely. This eliminates the setuptools version conflict where PyTorch pins setuptools<82 but nvidia-modelopt requires setuptools>=80. The warning was cosmetic only - modelopt works at runtime regardless - but now pip won't even see the conflict.
Also bumped requires-python to include 3.13 for PyTorch nightly compatibility.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Enable loading quantized checkpoints without original base model by storing full model config in safetensors metadata. Added StoredModelConfig class and reconstruction pipeline. Extracted native quantization to dedicated module.
Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)
Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
Added prominent warnings to README and RESEARCH_NOTES documenting
project status and blocking issues.
## Blocking Issue
PyTorch/TorchScript compatibility prevents full end-to-end inference:
- Encode/decode operations don't work with quantized models in PyTorch
- Root cause: Outdated distributed API in public ModelOpt repo
- Workaround exists in Diffusers wrapper (not ComfyUI native)
- Fix pending: PyTorch upstream updates (PR submitted by @marduk191)
## Community Feedback (2025-11-10)
Source: @marduk191
- "encoding/decoding won't work in torch until they update it"
- "works fine in diffusers" but not in native PyTorch/ComfyUI
- Submitted PR to PyTorch for encoder script fixes
- Recommendation: "yea i'll just wait until the update lol"
## Additional Issue Identified
Storage Inefficiency:
- Current loader requires original checkpoint + quantized model = 2x storage
- Defeats purpose of quantization (should reduce storage, not increase)
- Root cause: mto.restore() needs base model architecture
- Potential fix: Save architecture metadata, create empty model from config
- Not implemented due to project hold
## What Works (v0.4.0)
✅ Model quantization - 794 modules unwrapped, 2382 quantizers inserted
✅ Save/Load pipeline - mto.save()/restore() implementation
✅ Calibration starts successfully
## What's Blocked
❌ Full end-to-end inference in ComfyUI
❌ Encode/decode operations
❌ Production use
## Documentation Updates
- README: Added prominent status warning at top
- RESEARCH_NOTES: Added "PROJECT STATUS - ON HOLD" section
- RESEARCH_NOTES: Documented blocking issues in detail
- RESEARCH_NOTES: Added community feedback references
- RESEARCH_NOTES: Linked to related PyTorch issues (#76726, #75005)
- Updated final status to "ON HOLD" with recommendations
Recommendation: Wait for PyTorch upstream fixes before resuming development.
CRITICAL FIX: Complete rewrite of save/load nodes to properly preserve
quantizer infrastructure.
## Problem Identified
Previous implementation lost TensorQuantizer state during save/load:
- Save: Used state_dict() - only saved weights, not quantizer config
- Load: Used comfy.sd.load_model_weights() - created fresh wrapped modules
- Result: Quantized weights without quantization runtime (no speedup)
## Solution Implemented
Use ModelOpt's official save/load functions:
### ModelOptSaveQuantized (nodes/quantizer.py)
- Changed from state_dict() to mto.save(diffusion_model, path)
- Saves both weights AND quantizer infrastructure
- Format: PyTorch .pt files (safetensors not compatible with modelopt_state)
- Verifies quantizer count before saving
- Clear user instructions for loading
### ModelOptUNetLoader (nodes/loader.py)
- Complete rewrite: now requires base_model input
- Changed from comfy.sd.load_model_weights() to mto.restore()
- Workflow: Clone base model -> Restore quantized state -> Return quantized model
- Verifies quantizer count after restoration (fails if 0)
- Clear error messages for architecture mismatch
## User Workflow
1. Load original unquantized model
2. Quantize with ModelOptQuantizeUNet
3. Save with ModelOptSaveQuantized (creates .pt file)
4. Later: Load original model + ModelOptUNetLoader -> Get quantized model
## Technical Details
- No new dependencies required (uses existing ModelOpt)
- TensorRT not needed for PyTorch save/load path
- Loader now takes 2 inputs: base_model + quantized_unet_path
- Architecture must match between quantization and loading
## Version
- Bumped to v0.4.0 (major fix)
- Updated pyproject.toml
- Updated RESEARCH_NOTES.md with detailed changelog
- Marked save/load issue as FIXED in limitations
Tested architecture working. Actual save/load testing pending.
Fixed "Model must not be quantized!" error that occurred after successful
unwrapping. The issue was:
1. Module unwrapping worked perfectly (794 modules replaced)
2. Built-in config test succeeded (2382 quantizers inserted)
3. Code tried to quantize the SAME model again → error
Changes:
- Use copy.deepcopy() for test quantizations to avoid modifying original
- Skip permissive config test if built-in test already succeeded
- Select final config based on which test worked (built-in vs custom)
- Clean up test models after use to free memory
Result: Should now complete full quantization with calibration successfully.
Test showed:
- Unwrapping: 794 modules replaced ✓
- Modules now standard PyTorch (torch.nn.Linear) ✓
- Built-in FP8_DEFAULT_CFG: 2382 quantizers inserted ✓
- Ready for actual quantization with calibration
Fixed bug in module unwrapping that caused 0 modules to be replaced.
Root cause:
- Checked for module path 'comfy.ops.disable_weight_init'
- But actual __module__ attribute is just 'comfy.ops'
- Condition never matched, so no modules were replaced
Fix:
- Changed check from 'comfy.ops.disable_weight_init' to 'comfy.ops'
- Added explicit class name check ('Linear', 'Conv2d', 'Conv1d')
- Kept isinstance() safety check
Expected result:
- Should now replace ~794 modules (743 Linear + 51 Conv2d)
- Modules will be standard torch.nn types
- ModelOpt should recognize them and insert quantizers
Changes:
- nodes/quantizer.py: Fixed module path checks in _unwrap_comfy_ops()
- RESEARCH_NOTES.md: Updated dates to 2025-11-10, documented bug
Root cause identified and fixed:
- ComfyUI uses custom wrapped modules (comfy.ops.disable_weight_init.Linear/Conv2d)
- ModelOpt's quantization only recognizes exact torch.nn.Linear/Conv2d types
- Wrapped modules failed type checks: `type(m) == torch.nn.Linear` → False
Solution implemented:
- Added _unwrap_comfy_ops() method to recursively replace wrapped modules
- Creates standard torch.nn modules with identical parameters
- Preserves weights and biases via torch.no_grad() copy
- Handles Linear, Conv2d, and Conv1d layers
Changes:
1. nodes/quantizer.py:
- New _unwrap_comfy_ops() method (lines 679-782)
- Called before quantization to replace all wrapped modules
- Verifies replacement worked by checking module type
- Should enable ModelOpt to insert quantizers successfully
2. RESEARCH_NOTES.md (NEW):
- Comprehensive R&D tracker for technical findings
- Documents root cause, testing results, solutions
- Tracks what works, what doesn't, and why
- Provides roadmap for future improvements
- Includes references to NVIDIA docs and ComfyUI internals
Expected result:
- ModelOpt should now recognize modules and insert quantizers
- Next test should show "Inserted XXX quantizers" (XXX > 0)
- Quantization should finally work with ComfyUI models
Critical diagnostics to identify why ModelOpt cannot quantize ComfyUI's UNet:
1. Module Type Verification:
- Check if Linear/Conv2d are standard torch.nn or custom subclasses
- Verify module inheritance chain (MRO)
- Compare exact type vs isinstance checks
2. ModelOpt API Tests:
- Test QuantizeConfig parsing
- Check for module filters
- Verify module registration (named_modules)
3. Model Class Hierarchy:
- Check model's __class__ and __bases__
- Verify nn.Module inheritance
- Identify potential class confusion
4. Manual Quantizer Test:
- Try creating TensorQuantizer() manually
- Proves if ModelOpt API works at all with this setup
- Isolates quantize() function vs TensorQuantizer creation
Previous test showed ALL configs (built-in, permissive, diffusion) fail with
0 quantizers, indicating compatibility issue between ModelOpt and ComfyUI's
UNet implementation (comfy.ldm.modules.diffusionmodules.openaimodel.UNetModel)
Extensive debugging to identify why quantizers aren't being inserted:
1. Model Structure Analysis:
- List all Linear/Conv2d layers with details (in/out features, kernels)
- Show layer distribution (first 3, last 3 of each type)
- Display top-level model attributes
- Check for any existing quantizers before mtq.quantize()
2. Config Testing:
- Test with permissive config (default enabled) to see if ANY quantizers insert
- Test with ModelOpt's built-in FP8_DEFAULT_CFG/INT8_DEFAULT_CFG if available
- Compare results to identify if issue is config format or deeper
3. ModelOpt API Inspection:
- Print mtq.quantize docstring
- Enable ModelOpt debug logging
- Check for relevant environment variables
- List available config constants (FP8_DEFAULT_CFG, etc.)
4. Post-Quantization Analysis:
- Count TensorQuantizer modules
- Compare module count before/after
- Show first 15 quantizer locations if any found
- Provide diagnostic guidance if none found
This will definitively identify whether the issue is:
- Wildcard pattern matching
- Config format/structure
- ModelOpt version incompatibility
- Model structure not recognized by ModelOpt
Critical enhancements to model introspection and quantization diagnostics:
1. Add comprehensive diagnostic output:
- ModelOpt version detection
- Sample layer names from model (to verify wildcard matching)
- Full config dump before mtq.quantize() call
- TensorQuantizer module count after quantization
- Sample quantized layer names if quantizers found
2. Fix author attribution:
- Update project author to "enragedantelope"
- Update ComfyUI tool metadata author
This diagnostic output will help identify why quantizers aren't being
inserted despite having 743 Linear and 51 Conv2d layers in the model.
The previous introspection was failing to detect SDXL models correctly.
This update adds multiple fallback detection methods.
Key Changes:
1. **Parameter Count Detection**
- Counts model parameters to identify architecture
- SDXL: ~2.5B params, SD1.5: ~900M params
- If param count > 2B and y_dim not detected, infers y_dim = 2816
2. **Test Forward Pass Detection** (Most Reliable)
- Actually tries a forward pass without y parameter
- If error says "must specify y", we know it needs y
- Automatically infers y_dim = 2816 for SDXL
- This is the definitive test
3. **Comprehensive Attribute Listing**
- Lists ALL model attributes for debugging
- Tracks which important attributes were found
- Records detection events and methods used
4. **Better Debug Output**
- Shows parameter count in billions
- Lists found attributes
- Shows detection events
- Prints quantization config details
5. **Config Debug Output**
- Prints config keys and wildcard patterns
- Helps troubleshoot "Inserted 0 quantizers" issue
This should fix the misidentification of SDXL as SD1.5. The test forward
pass is the most reliable method since it actually tests what the model
needs rather than relying on attributes that may not be present.
For the "Inserted 0 quantizers" issue, we now print config details to help
diagnose whether the default configs have filters that exclude CNN layers.
Version: 0.2.1
This is a major update that should significantly improve compatibility and
reduce manual configuration for different model architectures.
Key Changes:
1. **Comprehensive Model Introspection** (NEW)
- Added introspect_diffusion_model() utility function
- Automatically detects:
* Model architecture (SDXL, SD1.5, SD3, etc.)
* Y dimension from 4 different methods:
- adm_in_channels attribute
- y_dim attribute
- label_emb layer dimensions
- Inferred from time_embed concatenation patterns
* Context dimension for text conditioning
* Latent format (channels and spatial dimensions)
* Forward pass signature requirements
- Prioritizes reliable detection methods
- Falls back gracefully when attributes not found
2. **Fixed Quantization Configs**
- Now uses ModelOpt default configs as base (INT8_DEFAULT_CFG, FP8_DEFAULT_CFG)
- Customizes configs instead of building from scratch
- This should fix "Inserted 0 quantizers" issue
- Disables output quantization for better quality
3. **Improved Calibration**
- Uses introspected model info instead of repeated detection
- Cleaner error handling (detailed error only on first failure)
- Uses correct y_dim, context_dim, and latent_shape from introspection
- Should fix "mat1 and mat2 shapes cannot be multiplied" errors
4. **Simplified Startup Message**
- Changed from verbose listing to simple: "NVIDIA ModelOpt for ComfyUI loaded successfully"
- No more advertising during ComfyUI startup
5. **Better Debug Output**
- Shows detected architecture details:
* Architecture type
* Y dimension and detection source
* Context dimension
* Latent format
* Whether model has y parameter
- Makes troubleshooting much easier
This update addresses the core issues:
- "Inserted 0 quantizers" - should be fixed by using default configs as base
- "Detected y_dim = None" - should be fixed by comprehensive introspection
- "mat1 and mat2 shapes cannot be multiplied" - should be fixed by using detected y_dim
Version: 0.2.0 (major update)
EnragedAntelope