Root cause: ComfyUI auto-randomizes INT inputs named "seed", generating
a new seed each queue run. Progress was keyed by seed, so each new seed
started from scratch.
- Rename "seed" input to "shuffle_order" to prevent auto-randomization
- Store completed combo hashes in a flat set instead of per-seed keys
(combo hashes are already order-independent, so progress persists
regardless of shuffle order)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Empty filter no longer allowed — there's no point merging across
incompatible architectures. Users must specify prefixes like
'zit/,zib/' to combine compatible LoRA pools.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Filters the LoRA pool by path prefix (e.g. "zimage/" or "sdxl/")
so users can generate combos only for compatible architectures.
Empty string includes all LoRAs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add INPUT_TYPES, RETURN_TYPES, IS_CHANGED, __init__, and get_next_combo
methods so the class works as a ComfyUI node. Register in
NODE_CLASS_MAPPINGS and NODE_DISPLAY_NAME_MAPPINGS. Add 3 tests for the
node interface (input fields, return types, IS_CHANGED returns NaN).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
combo_size now accepts ComfyUI dropdown strings ("2", "3", "2_and_3")
instead of integers. Progress JSON keys use "seed_42" format instead
of bare "42" for clarity.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements core static methods for generating, shuffling, hashing, and
tracking LoRA combinations (pairs/triples) with deterministic seeding
and JSON-based progress persistence. Includes 11 TDD tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add dedicated architecture support for ACE-Step music generation model LoRAs
with optimizations informed by the v1.5 hybrid attention architecture — cross-
attention is the primary voice-identity pathway (always global, attending to
timbre+lyric+text conditioning), while self-attention alternates sliding window
(local timbre) and global GQA (long-range structure).
Changes:
- New `acestep_dit` preset: wider orthogonal band (0.30), higher TIES threshold
(0.35), full magnitude preservation (floor 1.0) — biases toward SLERP over
TIES to prevent voice signal trimming in cross-attn layers
- Fix `to_out` → `o_proj` normalization bug (was producing `out_proj`)
- Add v1.0 detection and normalization (transformer_blocks, speaker_embedder,
lyric_encoder mapped to v1.5 canonical form)
- Refine WAN detection to exclude transformer_blocks (prevents v1.0 misdetect)
- Add `acestep_dit` to all architecture_preset UI dropdowns
- 25 new tests (detection, normalization, preset)
- Technical reference doc with architecture analysis and design rationale
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Single-LoRA prefixes produce identical patches regardless of merge config
(forced to weighted_sum, early return at _merge_diffs). Cache results keyed
by (label_prefix, auto_strength) to skip diff recomputation for candidates
2+ sharing the same auto_strength value.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Single-LoRA patches bypass all merge strategies (early return at line
2868). Score them once on the first candidate and merge as baseline
for subsequent candidates. All composite_score components (norm_cv,
effective_rank, sparsity) are scale-invariant, so auto_strength
differences between candidates don't affect correctness.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
A LoRA file uses one key format consistently. Caching the winning
format index (0-3) after first detection avoids up to 6 wasted
dict lookups per prefix on subsequent calls. Cache miss on stale
entries is safe — falls through to full cascade.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Moving fp16/bf16 tensors to score_device then converting to fp32
halves the PCIe transfer bandwidth vs converting first. For ~240
LoRAAdapter patches per candidate, saves meaningful transfer time.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replaces dead record_dataset param with community_cache dropdown
(disabled / download_only / upload_and_download).
Downloads are anonymous via urllib; uploads require HF_TOKEN and
huggingface_hub. LoRA filenames are never shared — content hashes
(SHA256[:16]) are used as keys. Per-LoRA/pair conflict metrics and
winning merge configs are shared; network errors fall back silently to
the full local sweep.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Loads lora/pair caches before _run_group_analysis and merges new entries
back into on-disk cache files after successful analysis, completing the
analysis cache feature end-to-end.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add lora_caches/pair_caches/lora_hashes params to _run_group_analysis,
skip _analyze_target_group on full cache hit via _pair_lora_cache_hit,
and populate new_lora_entries/new_pair_entries in the return dict on miss.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Correct log prefix in _analysis_partial_save except block from
[AutoTuner Analysis Cache] to [AutoTuner Analysis Partial]
- Add using_partial flag so partial is promoted to full cache even
when all prefixes were already cached (stuck-state fix)
- Move _analysis_partial_delete outside the new_entries guard so it
is always called (safe no-op when file doesn't exist)
- Add test_partial_promoted_to_full_cache_when_all_prefixes_already_done
to TestAnalysisPartialLifecycle exercising the stuck-state fix
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
On startup, fall back to partial checkpoint when no full cache exists.
Define _on_prefix_done callback that writes partial file after each
fresh prefix. Pass callback to _run_group_analysis. Delete partial on
successful full cache write. Delete partial on clear_and_run. Also fix
log prefixes in _analysis_partial_load/save (carry-forward from Task 1
review).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Callers can now pass on_prefix_done=lambda prefix, entry: ... to receive
a callback after each freshly-computed prefix is extracted. The callback
fires only on cache misses and only when track_new_entries=True, so all
existing call sites (which pass neither argument) are unaffected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
On first run cached_analysis is None, causing new_analysis_entries to
stay empty and the cache to never be saved. Replace the
'cached_analysis is not None' guard with an explicit 'track_new_entries'
flag so auto_tune opts in while other callers of _run_group_analysis
avoid the extraction overhead.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- _extract_lora_svd: return None early for 1D tensors (bias vectors) to
prevent torch.linalg.svd crash; add test_1d_delta_returns_none
- LoRAExtractFromModel: add optional base_clip/finetuned_clip inputs;
fix model_lora_keys_clip to use cond_stage_model instead of model;
merge CLIP state dicts into base_sd/fine_sd when both CLIPs provided;
warn when only one CLIP is connected; remove redundant import comfy.lora
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds module-level _extract_lora_svd() that SVD-decomposes a weight
delta into (lora_up, lora_down, alpha), supporting fixed-rank and
auto-rank (energy threshold) modes with a singular value noise floor.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds TestExtractLoRAFromDelta test class and _load_module() helper
to support TDD for the upcoming LoRAExtractFromModel node.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add _execute_merge_tree, _resolve_tree_to_stack, and
_model_to_virtual_lora methods to LoRAOptimizer. Modify optimize_merge
to detect formula metadata in the stack and delegate to the tree
executor for hierarchical sub-group merging. Add safety filter in
_normalize_stack to strip formula metadata entries.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- SaveMergedLoRA now writes safetensors metadata (source LoRAs, merge
settings, architecture) plus optional user prompt/description
- SaveTunerData accepts optional prompt/description, embeds in JSON
- LoadTunerData outputs prompt, description, and metadata_info strings
- AutoTuner records source_loras in tuner_data for provenance
- New LoRAMetadataReader passthrough node extracts metadata from any
safetensors LoRA in a stack (prompt, description, merge info)
- LoRAStack attaches file metadata at load time; _normalize_stack
preserves it through both tuple and dict paths
- Graceful degradation: missing metadata produces empty strings
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Solo LoRAs now create LoraLoader/LoraLoaderModelOnly nodes (based on CLIP)
- Merge groups continue creating LoRAStackDynamic nodes
- Use original strengths instead of auto-strengths for created nodes
- Add create_nodes toggle to disable automatic node creation
- Move smooth_slerp_gate from AutoTunerSettings to shared MergeSettings
- Add smooth_slerp_gate to main LoRAOptimizer and _SIMPLE_DEFAULTS
- Add decision_smoothing (0.25) to _SIMPLE_DEFAULTS
- Remove output_mode from AutoTunerSettings (hardcode "merge")
- Rewrite all settings node tooltips for clarity
- Update tests for new settings structure
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Create a new LoRAMergeSettings node holding the 6 shared parameters
(normalize_keys, architecture_preset, auto_strength_floor,
decision_smoothing, vram_budget, cache_patches) that were duplicated
across LoRAOptimizerSettings and LoRAAutoTunerSettings.
Both mode nodes now accept an optional MERGE_SETTINGS input and fall
back to identical defaults when unconnected — output dicts are unchanged
so execute_simple needs zero modifications.
Also fixes:
- v1 scoring: de-dent composite_score recomputation so it always runs
under scoring_formula=="v1", not only when sparsification is enabled
- RAM leak: clear _autotuner_delegate caches on model change and when
switching between advanced/autotuner modes in LoRAOptimizerSimple
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add two pure-data nodes (LoRAOptimizerSettings, LoRAAutoTunerSettings)
that output OPTIMIZER_SETTINGS dicts. Connecting either to the Simple
node's new 'settings' input transforms it into the Advanced Optimizer
or AutoTuner respectively, so users never need to swap node types.
Settings take priority over tuner_data and defaults. The AutoTuner
delegate is persisted on the Simple instance so cache_patches works
across re-executions. Unknown mode values raise ValueError.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
