Rebrand the project as "ComfyUI Quantization Toolkit" and introduce native INT4 (ConvRot W4A4) support, mixed INT4/INT8 modes, and expanded quantization policies. Add an INT4 torch.compile compatibility shim, INT4-aware lazy-compile changes, richer LoRA stack nodes with autogrow, and a Quantized LoRA entry/apply pair. Replace ad-hoc prints with structured logging, bump package metadata to 2.0.0, and add documentation (advanced usage, checkpoints, policies) plus many unit tests for compile, lazy-compile, LoRA, adapter caching, saving, and modes. Misc: kernel config, model save/load, and numerous runtime/adapter refactors and robustness fixes.
Introduce a runtime Triton kernel config tuner and several performance enhancements for INT8 kernels and Dynamic LoRA.
Key changes:
- Add new node int8_kernel_config_node.py to apply fixed Triton kernel configs from-graph, optionally run an on-demand microbench to pick the best config, and emit recommended environment variable lines.
- Implement kernel config helpers and microbench in int8_fused_kernel.py: sanitization, getter/setter, candidate enumeration, microbench harness, and env-line formatting. Register new node in __init__.py.
- Update int8_quant.py to support adaptive small-batch fallback, batching of Dynamic LoRA entries, per-device tensor caching, and new env vars to control behavior (INT8_SMALL_BATCH_FALLBACK_MIN_ROWS, INT8_SMALL_BATCH_FALLBACK_ADAPTIVE, INT8_DYNAMIC_LORA_BATCH, INT8_DYNAMIC_LORA_BATCH_MAX_RANK). Add logic to compute batching eligibility and to perform batched LoRA application when possible.
- Update README.md with usage notes for the INT8 Kernel Config node and new runtime env toggles.
These changes enable persistable fixed-kernel configs, automated microbench tuning, and improved runtime performance for small batches and dynamic LoRAs.
ThereforeGames