Threads the detected architecture family (zimage, wan, flux, qwen_image,
etc.) into the config payload so the 'dit' arch_preset bucket can be
filtered by sub-family downstream (estimator training, targeted cache
reuse). Existing 400 configs on HF were backfilled with zimage in a
separate dataset commit.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
target_key can be a nested tuple like ("key", (dim, start, size)).
JSON round-trips flatten this to ["key", [dim, start, size]].
The shallow tuple() conversion only fixed the outer level, leaving
inner lists unhashable — causing TypeError in single_lora_keys.add().
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Root cause: ComfyUI auto-randomizes INT inputs named "seed", generating
a new seed each queue run. Progress was keyed by seed, so each new seed
started from scratch.
- Rename "seed" input to "shuffle_order" to prevent auto-randomization
- Store completed combo hashes in a flat set instead of per-seed keys
(combo hashes are already order-independent, so progress persists
regardless of shuffle order)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Logs seed, pool size, combo count, completed count, progress file path
on entry, confirms save success/failure, and logs returned combo info.
Improves error message for insufficient LoRAs to show filtered list.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Empty filter no longer allowed — there's no point merging across
incompatible architectures. Users must specify prefixes like
'zit/,zib/' to combine compatible LoRA pools.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Filters the LoRA pool by path prefix (e.g. "zimage/" or "sdxl/")
so users can generate combos only for compatible architectures.
Empty string includes all LoRAs.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Design doc requires logging when a LoRA file is missing and a combo
is skipped. Follows existing project convention of logging.warning
with the [LoRA ...] prefix.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
LoRACombinationGenerator previously raised RuntimeError when all combos
were done, showing a red error in ComfyUI's UI. Now uses ComfyUI's
InterruptProcessingException which sends an "execution_interrupted"
event — clean yellow stop instead of a scary error traceback.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The missing-LoRA handling used unbounded recursion via
`return self.get_next_combo(...)` which would hit Python's recursion
limit of 1000 when many LoRAs are deleted. Replaced with an iterative
while-loop that skips deleted-LoRA combos without growing the stack.
Also removed redundant `OUTPUT_NODE = False` (it's the default).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add INPUT_TYPES, RETURN_TYPES, IS_CHANGED, __init__, and get_next_combo
methods so the class works as a ComfyUI node. Register in
NODE_CLASS_MAPPINGS and NODE_DISPLAY_NAME_MAPPINGS. Add 3 tests for the
node interface (input fields, return types, IS_CHANGED returns NaN).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
combo_size now accepts ComfyUI dropdown strings ("2", "3", "2_and_3")
instead of integers. Progress JSON keys use "seed_42" format instead
of bare "42" for clarity.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements core static methods for generating, shuffling, hashing, and
tracking LoRA combinations (pairs/triples) with deterministic seeding
and JSON-based progress persistence. Includes 11 TDD tests.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add dedicated architecture support for ACE-Step music generation model LoRAs
with optimizations informed by the v1.5 hybrid attention architecture — cross-
attention is the primary voice-identity pathway (always global, attending to
timbre+lyric+text conditioning), while self-attention alternates sliding window
(local timbre) and global GQA (long-range structure).
Changes:
- New `acestep_dit` preset: wider orthogonal band (0.30), higher TIES threshold
(0.35), full magnitude preservation (floor 1.0) — biases toward SLERP over
TIES to prevent voice signal trimming in cross-attn layers
- Fix `to_out` → `o_proj` normalization bug (was producing `out_proj`)
- Add v1.0 detection and normalization (transformer_blocks, speaker_embedder,
lyric_encoder mapped to v1.5 canonical form)
- Refine WAN detection to exclude transformer_blocks (prevents v1.0 misdetect)
- Add `acestep_dit` to all architecture_preset UI dropdowns
- 25 new tests (detection, normalization, preset)
- Technical reference doc with architecture analysis and design rationale
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The SDXL normalizer's .replace('.', '_') was mangling LoRA weight
suffixes (.lora_down.weight → _lora_down_weight), breaking prefix
extraction for diffusers-format keys. Added _split_lora_suffix to
preserve suffixes. Also split SD 1.5 from SDXL detection so single-TE
LoRAs normalize to lora_te_ instead of lora_te1_.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Single-LoRA prefixes produce identical patches regardless of merge config
(forced to weighted_sum, early return at _merge_diffs). Cache results keyed
by (label_prefix, auto_strength) to skip diff recomputation for candidates
2+ sharing the same auto_strength value.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Patch dict keys and single_lora_keys both store target_key values
from _group_target_key(). When target_key is a tuple (weight slices),
the lambda extracted k[0] (a string) which never matched the tuple
in the set. Direct comparison handles both string and tuple keys.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Single-LoRA patches bypass all merge strategies (early return at line
2868). Score them once on the first candidate and merge as baseline
for subsequent candidates. All composite_score components (norm_cv,
effective_rank, sparsity) are scale-invariant, so auto_strength
differences between candidates don't affect correctness.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
A LoRA file uses one key format consistently. Caching the winning
format index (0-3) after first detection avoids up to 6 wasted
dict lookups per prefix on subsequent calls. Cache miss on stale
entries is safe — falls through to full cascade.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Moving fp16/bf16 tensors to score_device then converting to fp32
halves the PCIe transfer bandwidth vs converting first. For ~240
LoRAAdapter patches per candidate, saves meaningful transfer time.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Memory diagnostics on every iteration adds overhead without value.
First and last iteration still show whether memory leaks occur.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace full gc.collect() with gc.collect(0) (youngest generation only,
~10x faster). Remove torch.cuda.empty_cache() — CUDA allocator handles
block reuse, forced sync is pure overhead. Post-loop cleanup at line
9354-9356 still does full gc.collect() + empty_cache().
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Removes the passive download_only option to encourage contribution.
Users who want community results need to also contribute back.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Memory hits had content_hashes and the cached config available but
never uploaded. Now uploads the winning config to community cache on
memory hits too. lora/pair analysis entries are still only uploaded
after a full sweep (they require _run_group_analysis output).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Community download block was inside Pass 1, after the memory check's
early return. Runs with memory_mode=auto never reached the community
code, so nothing was downloaded or uploaded despite community_cache
being set to upload_and_download.
Moved lora/pair cache init + community download block to before the
persistent memory lookup so community hits and uploads work regardless
of memory cache state.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously fired before memory/community checks, causing misleading
"MISS — will run full analysis" followed by "Memory HIT". Now logs
"No cached results — running full sweep" only when all caches miss.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Show mode, LoRA count, and arch at start of check
- Report lora/pair cache hits as X/N (e.g. "2/3 lora hit, 2/3 pair hit")
- Log "No config found — will run full sweep" on config miss
- Log config version mismatch with both versions
- Log "Config HIT" with score and arch on success
- Log "COMMUNITY CACHE HIT" with score on replay
- Track and report upload counts (N lora, M pair, config score)
- Log when config not uploaded due to community score being higher
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
AUTOTUNER_ALGO_VERSION = "1.5.0" — scoring formula, memory/config format
ANALYSIS_CACHE_VERSION = "1.6.0" — per-prefix conflict math (lora/pair/analysis caches)
The clip_strength_multiplier fix only affects CLIP prefix entries.
Models without CLIP (Wan, etc.) had valid lora/pair caches that were
being incorrectly invalidated by the version bump. Now only analysis
caches are invalidated when conflict math changes, and only memory/
community configs are invalidated when scoring changes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fall back to huggingface_hub stored token so users who ran
'huggingface-cli login' can upload without setting HF_TOKEN manually.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Hardcoded "normalize_keys": "enabled" in reconstructed community
tuner_data was always overriding the user's normalize_keys param.
Removed the field so the fallback correctly uses the user's setting.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Download and upload now both resolve arch via _resolve_arch_preset,
so config filenames are consistent (e.g. "dit" not "flux")
- Upload reuses _arch_key_for_community instead of tuner_preset_key
- Community hit banner no longer shows memory_mode hint
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces dead record_dataset param with community_cache dropdown
(disabled / download_only / upload_and_download).
Downloads are anonymous via urllib; uploads require HF_TOKEN and
huggingface_hub. LoRA filenames are never shared — content hashes
(SHA256[:16]) are used as keys. Per-LoRA/pair conflict metrics and
winning merge configs are shared; network errors fall back silently to
the full local sweep.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
magnitude_samples in _analyze_target_group are built with eff_s =
strength * clip_strength_multiplier for CLIP prefixes without a per-LoRA
clip_strength. The extraction function was dividing by raw strength only,
so stored samples were off by csm. Reconstruction then rescaled without
csm, causing wrong magnitudes if csm changed between runs.
Fix: pass clip_strength_multiplier through _extract_for_lora_cache and
_reconstruct_from_pair_lora_cache, and use the same eff_s formula as
_prepare_group_diffs. Non-CLIP prefixes and models without CLIP are
unaffected (csm is never applied to them).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When auto_ignore_strength misses on the names-only hash, scan existing
strength-sensitive entries with the same LoRA names and settings, and
use the one made with the highest total absolute strength (most reliable
strategy rankings). The result is then promoted to the names-only key
so future runs hit directly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New memory_mode option that caches tuning results keyed by LoRA names
only (ignoring strengths). Useful when sweeping strengths on orthogonal
LoRAs where the optimal config (strategy, sparsification, refinement)
doesn't change with strength — avoids full Phase 2 re-run on each
strength change while still replaying the merge with current strengths.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Loads lora/pair caches before _run_group_analysis and merges new entries
back into on-disk cache files after successful analysis, completing the
analysis cache feature end-to-end.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add lora_caches/pair_caches/lora_hashes params to _run_group_analysis,
skip _analyze_target_group on full cache hit via _pair_lora_cache_hit,
and populate new_lora_entries/new_pair_entries in the return dict on miss.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
