LTX-2 (LTXV 2.3) LoRAs have dual video/audio + cross-modal attention with
'*_attn.to_gate_logits'. The krea2 gate detector's '(?=[._])' lookahead let
the '..._attn.to_gate' substring match inside 'to_gate_logits', so LTX-2 LoRAs
were detected as krea2, ran through krea2 normalization (which renames
transformer_blocks->blocks), and then every key failed to map to the model
(1632 skipped, nothing merged). Require a '.' after the gate/proj leaf — always
the LoRA suffix in real krea2 keys — so LTX-2 falls through to the existing ltx
detection (transformer_blocks + attn1/attn2) and its pass-through normalizer.
No version bump (detection-only fix; correctly-detected arches unaffected).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The shape-mismatch capture lived only in the plain-LoRA path, so LoKr/LoHa
concepts (e.g. snofs_krea_v1) whose reconstructed delta doesn't match the
target model weight were dropped silently. Record those mismatches in the
LoKr/LoHa reshape-failure branch too, with the reconstructed-vs-model dims.
Report-text only; no merge-math change, no version bump. 301 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The per-prefix SLERP-imbalance gate changes mode selection for imbalanced
stacks, so bump to invalidate stale AutoTuner rankings and force a re-tune.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
SLERP averages directions and discards magnitude, so on an imbalanced
orthogonal pair it washed out the dominant LoRA (~40% retained at 6x) while
over-amplifying the weak one (~220%). Suppress the weighted_average->SLERP
upgrade when a prefix's max/min Frobenius ratio exceeds slerp_imbalance_ratio
(default 2.0, dit preset); route a strongly-imbalanced orthogonal *pair* to
additive (weighted_sum), which preserves the dominant LoRA without
oversaturating (the dominant defines the auto-strength reference, so scale ~1.0).
Gated to n_loras==2 so a balanced sub-pair among 3+ can't compound past the
auto-strength floor; aligned imbalanced stays weighted_average. No version bump.
Also: shape-incompatible LoRA layers were dropped silently. Now record each
(LoRA name, target key, LoRA dim vs model dim) and render a clear report
section naming the LoRA, the dropped-layer count, examples, and the cause/fix.
300 tests pass (10 new: routing gate + shape-mismatch recording/render).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Layers only one LoRA touches have no conflict to sparsify and no second
diff to align, so TIES/DARE/refinement are no-ops on them. Previously,
turning on sparsification or refinement disabled the exact low-rank fast
path for EVERY group — so single-LoRA layers needlessly materialized a
full dense diff and ran a compression SVD (the 99%-of-Pass-2 cost on the
Krea 2 softwatercolor merge, where each layer only one LoRA touches paid
a large GPU SVD).
Take the exact low-rank fast path for single-LoRA groups regardless of
those toggles, emitting the LoRA's native factors directly. Size-safe:
a single LoRA's compress_rank is max(rank, 64) >= its native rank, so
compression never shrank it anyway (only padded rank<64 up to 64) —
bypassing is always equal-or-smaller, and peak VRAM drops too (no dense
tensor is built). Multi-LoRA groups are unchanged.
No version bump.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The compression SVD (torch.svd_lowrank) can race asynchronously on some
stacks (Blackwell/cu130), corrupting CUDA state and aborting a later
kernel — a C++ abort the Python try/except can't catch. This is what
crashed Krea 2 merges (two different-form LoRAs route through dense
merge + compression, the only path that runs GPU compression SVD here).
Make the per-group cuda.synchronize() permanent, but gate it tightly on
(GPU compression active) so it only fires on compression merges and
fast no-compression merges keep their fully-pipelined speed. Previously
this sync only ran under the PROFILE_MERGE sentinel.
No version bump.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When the merge profiler is enabled (env LORA_OPTIMIZER_PROFILE_MERGE=1 or the
PROFILE_MERGE sentinel), log each group before _merge_one_group and force a
torch.cuda.synchronize() after it. A kernel that faults async then aborts AT
the sync, so the last 'processing N/M: <target_key>' line printed names the
exact layer/group that crashed — no CUDA_LAUNCH_BLOCKING (no global slowdown)
needed. No-op when the profiler is off. Added to pin which Krea-team
full-pipeline LoRA layer faults the live merge.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Krea 2 LoRAs come in trainer/diffusers key forms that DON'T match the model's
weights, and ComfyUI had no krea2 key normalizer so they passed through raw:
- 'krea_2' trainer: diffusion_model.transformer_blocks.N.attn.to_q ...
- diffusers: transformer.transformer_blocks.N.attn.to_q ..., text_fusion.*,
img_in/txt_in/final_layer/time_embed/time_mod_proj
The diffusers 'transformer.*' keys were mis-routed by ComfyUI's FLUX branch
(flux_to_diffusers) to non-existent double_blocks QKV tuple-offset targets, so
most of the LoRA was silently DROPPED — and the bogus offset target is the prime
suspect for the merge crash (Form-2-only merges work; adding a 'transformer.*'
LoRA crashes).
Add _normalize_keys_krea2 (+ register in the dispatcher) mapping both forms to
the model-native diffusion_model.blocks.N.attn.{wq,wk,wv,wo,gate} + mlp.* +
txtfusion.{layerwise,refiner}_blocks + first/last/tmlp/tproj/txtmlp. Mapping
verified key-by-key (name AND shape) against the real krea2_turbo model:
224/224 (trainer) and 264/264 (diffusers) modules map exactly. +4 tests, 288 pass.
No version bump.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reverts the code from cb1ac2d (svd_lowrank -> torch.linalg.svd). That swapped
the SVD primitive for ALL users to fix a torch.svd_lowrank C++ abort that only
occurs on a bleeding-edge cu130/Blackwell stack — too broad a change for main,
and full linalg.svd is slow on huge layers anyway. Restores the original
torch.svd_lowrank everywhere.
Krea 2 architecture detection (2682875, 726d3cf) is KEPT. The real SVD fix
(Gram+eigh, fast + GPU-stable + more accurate) lives on the 'krea2' branch for
validation before any merge to main. pyproject left at 1.7.13 (no version
change, no re-publish).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A Krea 2 merge hard-crashed (Fatal Python error: Aborted / 'terminate
called') inside torch.svd_lowrank -> get_approximate_basis, called from
_compress_to_lowrank. It's a C++ abort, NOT a Python exception, so the
GPU->CPU try/except fallback can't catch it — the process dies.
Root cause is the randomized lowrank QR kernel on a bleeding-edge stack
(Blackwell + torch 2.12/cu130), not the data (both LoRAs are NaN/Inf-free)
and not Krea-specific: the user's usual low-rank weighted_average merges take
the exact-linear fast path (no SVD), while this 2-LoRA mixed-format merge
routes through dense merge + compression. Proof it's svd_lowrank-specific:
the extract feature runs torch.linalg.svd on the SAME cuda device fine.
Fix: route every svd_lowrank call through _svd_lowrank_safe (torch.linalg.svd,
truncated) — none are in the per-candidate hot loop. Also keep an opt-in
LORA_OPTIMIZER_SVD_DEVICE=cpu escape hatch + the existing svd_device=cpu
setting. 284 tests pass. Registry 1.7.13.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The initial krea2 detection only matched the Comfy-Org native form
(blocks.N.attn.wq). Real trainer outputs use diffusers-style keys and were
misdetected: the 'krea_2' trainer (diffusion_model.transformer_blocks.N.attn
.to_q/to_gate) hit the ACE-Step branch, and the diffusers form
(transformer.transformer_blocks.N...) would hit qwen.
The marker common to ALL forms is the per-attention sigmoid GATE
(attn.to_gate / attn.gate / attn.w{q,k,v,o}) — unique among supported archs.
Detect on that, moved BEFORE the qwen/flux/ACE-Step branches. Verified vs 3
real LoRAs (official Comfy-Org rank-64, a krea_2-trained style LoRA, and a
diffusers 'softwatercolor' with text_fusion) — all -> krea2/dit. +2 tests
(ACE-Step/qwen collision guards); 284 pass. Registry 1.7.12.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Krea 2 (krea/Krea-2) is a from-scratch 12B single-stream image DiT (NOT the
older FLUX.1-Krea finetune) released 2026-06-22. Its LoRAs previously fell to
'unknown' -> sd_unet preset (generic, suboptimal merge tuning).
Detect diffusion_model.blocks.N with GQA attn.wq/wk/wv/wo + sigmoid attn.gate
+ SwiGLU mlp.gate/up/down. The attn.w{q,k,v,o} naming is unique (WAN uses
self_attn.q/k/v/o + ffn), so the branch sits before WAN/Anima safely. Maps to
the 'dit' preset like Flux/Z-Image; no key normalization needed (QKV already
split, ComfyUI-native diffusion_model.* keys pass through).
Verified against the official Comfy-Org/Krea-2 rank-64 LoRA (real keys ->
arch=krea2 -> dit). +4 detection tests incl. WAN/flux/qwen collision guards;
282 pass. No algo bump (only affects krea2, which had no prior cache).
Registry 1.7.11 (also publishes the accumulated tooltip fixes).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
'0 = no limit' was ambiguous. Spell it out on all four nodes: 0 = no floor
at all (remove the limit — auto-strength may shrink strengths all the way
down). Tooltip-only.
Prior tooltip wrongly said 'Optimizer/AutoTuner Settings node'. auto_strength
lives only on Optimizer Settings; the AutoTuner sweeps it (enabled AND
disabled) automatically, so it has no toggle. Corrected the wording.
Clarify that the floor only bounds the shrink — to disable auto-strength
entirely, set auto_strength='disabled' on the Optimizer/AutoTuner Settings
node; for on-but-never-shrink, use manual with floor 1.0. Tooltip-only.
The auto_strength_floor widget mixed a -1 'auto' sentinel into its numeric
range and the tooltip was wrong (said 'for orthogonal LoRAs', but explicit
floors apply to every stack since 1.10.2; nothing explained negatives/>1.0).
- LoRAMergeSettings: split the -1 sentinel into an 'auto_strength_floor_mode'
(auto/manual) switch, appended at the END of the widgets so no existing
widget shifts position. The slider is now a clean [0,1] manual floor (default
0.85). build_settings resolves mode -> the same downstream value the old
float produced (auto -> -1, manual -> clamped slider), so cache keys /
numerics / community cache are UNCHANGED (no algo bump).
- js/lora_merge_settings.js: configure() migration appends the mode value for
old 7-widget workflows BEFORE positional restore — an explicit floor (>=0)
becomes 'manual' (preserved), -1 becomes 'auto'. No scramble, no silent loss.
- Corrected the tooltip on all four nodes (it's a floor on a uniform down-scale
multiplier; never scales up/flips signs; covers negatives by magnitude and
>1.0 strengths).
+1 resolution test; 278 pass. Registry 1.7.10.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes the merge profiler so gross pathologies announce themselves
instead of needing a manual read of the numbers:
- '% is now against wall-clock Pass-2 time, and an 'other (cache/move/score)'
row accounts for everything the timed phases miss — the breakdown sums to
~100%, so untimed overhead can't hide.
- Auto-flag: merge MATH (slerp/consensus/ties/weighted_*) may dominate (it's
the real work), but if an OVERHEAD phase (diff_prep/compress/linear_fast/
other) takes >=40% of a >=1s Pass 2, log a warning with a targeted hint.
This is exactly how the diff-cache disk spill would have surfaced on the
first run.
Diagnostic only, still gated behind LORA_OPTIMIZER_PROFILE_MERGE / the
sentinel file. No merge-path or numerics change. Registry 1.7.9.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Robust-slice follow-up to the low-rank bypass. Two changes to _DiffCache:
1. Store fp32, not fp16. A cache hit is now numerically identical to a
recompute, so results are independent of diff_cache_mode and of what fit
in the cache (they match the 'disabled' path). Removes the documented
~1e-5 cache-vs-disabled score shift.
2. 'auto' mode recomputes past the RAM budget instead of spilling dense
diffs to disk (recompute beats disk I/O — that spill once made Pass-2 98%
disk wait). Explicit 'disk' mode is unchanged for those who want it.
Together these make caching purely a perf/memory policy with zero effect on
results. Algo version 1.11.1 -> 1.11.2 (also covers the earlier low-rank
bypass numerics); invalidates rankings cached under the old fp16 regime.
Registry 1.8.0. Tooltips updated; +2 regression tests (277 pass).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Profiling a 6-LoRA Z-Image sweep showed Pass-2 was 98% diff_prep — ~965ms
PER GROUP — while the slerp math was 16ms. Root cause: the diff cache had
spilled ~37GB of dense diffs to a slow/NAS temp volume, and reading them
back dominated. But these are rank-17 LoRAs: recomputing up@down on the GPU
is microseconds and moves far less data than the dense diff.
_prepare_group_diffs now bypasses the cache (no get/put) whenever the LoRA
factors are smaller than the dense diff (r*(out+in) < out*in — true for any
genuine low-rank LoRA), on the GPU path. This skips the disk spill entirely
and frees the cache for diffs that are actually expensive to recompute
(LoKr/LoHa/full-rank). CPU path and existing tests unchanged (gated on GPU).
Expected: diff_prep drops from ~72s to <1s per candidate on the profiled run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Env vars are awkward to set in some ComfyUI launchers (container/service),
so merge profiling could not be enabled. Add a runtime sentinel: profiling
turns on if <models>/autotuner_memory/PROFILE_MERGE exists (checked per
merge, no restart needed), in addition to LORA_OPTIMIZER_PROFILE_MERGE=1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds LORA_OPTIMIZER_PROFILE_MERGE=1 to attribute Pass-2 wall-clock to
diff_prep / merge:<strategy> / compress / linear_fast, with proper CUDA
sync so async kernels are charged to the right phase. Logs total_s, count,
avg_ms and % per phase. Off by default (zero overhead); diagnostic for
targeting merge speedups with real numbers instead of guesses.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root-caused the '__triton_launcher.c' hard crash: the bundled Triton N=2/3
SVD kernels are reachable ONLY via _triton_svdvals -> _batched_svd, which
always passes a single matrix (B=1). At B=1 the batched kernel gives no
speedup over torch.linalg.svdvals on an already-thin (<=32-wide) matrix, yet
it's the sole path that JIT-launches the Triton kernels — which abort the
process uncatchably on some torch/Triton/driver combos (torch 2.12+cu130).
batched_procrustes (refine path) is pure torch and is unaffected.
Bypass the kernel in the single-matrix path; verified svdvals match torch to
<=1e-7 across N=2/3/8/10. Removes the crash vector for all users at zero perf
cost. LORA_OPTIMIZER_DISABLE_TRITON=1 still skips loading the kernel entirely.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The bundled Triton SVD kernel can hard-crash the process ('__triton_launcher.c'
abort with no Python traceback) on some GPU/driver/torch combinations (seen on
torch 2.12+cu130), killing an AutoTuner sweep mid-candidate. The try/except
around the kernel can't catch a C-level abort. Add an env toggle to skip kernel
loading entirely and fall back to torch.linalg; all call sites already guard on
None, so the refine/procrustes path degrades to a no-op instead of crashing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The extractor's synthetic entry has no file on disk, so _lora_content_hash
returned None and the AutoTuner logged 'Could not hash ... disabling
community cache'. Fall back to hashing the in-memory weight tensors (sorted
keys + shapes + float32 bytes) for a stable, deterministic content hash, so
community caching stays enabled for stacks that include an extracted LoRA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
KeyError: 3 crashed the optimizer when a LoRAStackDynamic (tuples) fed the
extractor (dict) into LoRAOptimizerSimple: _compute_cache_key dispatched on
the first element and indexed entry[3] on the dict. Fixed both copies
(optimizer + conflict editor IS_CHANGED) to dispatch per item with a unified
6-field shape. Also fixed LoRAConflictEditor's enriched output to emit dict
format when any entry is a dict, so an in-memory extractor entry keeps its
weights instead of collapsing to a name-only tuple.
Adds a mixed-stack regression test for _compute_cache_key.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
LoRAStackDynamic emits file-ref tuples; LoRAExtractFromModel emits dicts
with in-memory weights. Two bugs blocked combining them:
- _normalize_stack dispatched on the first element's type and dropped the
minority type. Now it dispatches per item.
- LoRAStackDynamic's passthrough flattened in-memory dict entries to a
(name, ...) tuple, losing the weights (optimizer then tried to load a
nonexistent file). Now such entries pass through as-is.
Adds regression tests for both.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Without a stack input the extractor emitted a fresh one-element stack, so
the optimizer only ever saw the single extracted LoRA. Add an optional
lora_stack input that the extracted entry is appended to, so it merges
together with upstream LoRAs (LoRA Stack / another extractor). LORA_DATA
still represents the extracted LoRA alone for SaveMergedLoRA.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
After a merge/sample, ComfyUI offloads both source models to CPU, so the
previous device-alignment defaulted the per-layer subtraction and SVD to
CPU (no GPU activity, slow over a full model). Move each delta to the
compute device for the decomposition; the robust SVD still falls back to
CPU per-layer on non-convergence.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
LoRAExtractFromModel crashed with 'two devices, cuda:0 and cpu' because
ComfyUI dynamic-VRAM offloading leaves base and finetuned weights on
different devices; the per-layer subtraction now aligns both onto one
device (preferring the GPU side). Also harden _extract_lora_svd against
ROCm/MAGMA non-convergence with a GPU->CPU->jitter fallback (matching the
other SVD call sites), and land extracted factors on CPU to bound VRAM.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
These docs/plans/2026-03/04-*.md were untracked local notes from earlier sessions
that an over-broad 'git add -A' committed. Untrack them (kept on disk) to restore
the prior state; this session's 2026-06-22 design doc stays.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reported: community_cache='disabled' + memory_mode='clear_and_run' still showed
'COMMUNITY CACHE HIT'. Cause: the in-node _autotuner_cache (RAM, per node instance)
keyed on at_cache_key, which did NOT include community_cache — so a result cached by
an earlier community-enabled run (banner baked into its report) was replayed after
community was disabled. And clear_and_run never bypassed that lookup.
- Add |cc={community_cache} to at_cache_key so toggling community invalidates it.
- Skip the in-node cache lookup entirely when memory_mode == 'clear_and_run'
(its whole purpose is to clear caches and recompute fresh).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Switching the preserve flag on/off did not retrigger the AutoTuner: the per-LoRA
merge flags (preserve, and already conflict_mode/key_filter) were missing from the
cache keys that gate re-execution.
- New _per_lora_merge_signature(lora_stack): order-independent hash of
(name, conflict_mode, key_filter, preserve), strength-independent.
- LoRAAutoTuner.IS_CHANGED: embed the signature (it only hashed id(lora_stack),
which is content-blind and id-reuse-fragile) so ComfyUI re-invokes the node.
- LoRAMergeSelector.IS_CHANGED: it hashed no stack content at all — add the
signature so a replayed config re-runs when the flags change.
- In-node at_cache_key stack_fp: add preserve (it already had conflict_mode/key_filter).
- Persistent memory_settings (3 sites): add per_lora_flags so settings_hash misses
on toggle instead of replaying a stale ranking. The merge was always correct
(replayed against the live stack); only the ranking could be stale.
- Community shared RANKING is computed without these flags (heuristic, analysis-based)
— skip it when the stack sets any per-LoRA flag and tune locally; analysis caches
(preserve-independent) and uploads are unaffected.
Analysis caches stay keyed on file identity only (conflict/cosine don't depend on
these flags) — correct to keep sharing. 8 new tests (PreserveCacheAwarenessTests); 307 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A real 3-LoRA merge (characters/concept, no style) oversaturated badly (energy
2.52x vs main's 1.02x): additive stacking is wrong for a balanced multi-LoRA blend.
The analyzer cannot tell 'preserve this style' from 'blend these characters' — both
are orthogonal — so additive must never be auto-selected; it has to be user intent.
- Revert _decide_prefix_mode: orthogonal groups blend by default again
(weighted_average / SLERP), exactly like main. No auto sum_preserve.
- Make the preserve flag a general overlay in _merge_diffs: blend the non-preserved
LoRAs with the chosen mode, then add each preserved LoRA's full delta on top
(works for weighted_average/slerp/ties; still exempt from sparsification + TIES
sign-election). Skip the exact-linear fast path when the stack has a preserve flag.
- Remove the standalone sum_preserve mode, its exact-linear handling,
_ORTHO_SUM_HEADROOM, and the sum+ report/legend/strategy-count display.
- Tests: SumPreserveMergeTests -> PrefixModeRoutingTests (asserts additive is never
auto-selected); add preserve-overlay-on-weighted_average tests. 299 pass.
- Docs (Merge-Algorithms, Tips) + design-doc postmortem updated; algo note reworded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The orthogonal-branch reasoning still said 'upgraded to SLERP per-prefix' after
the routing changed to sum_preserve, contradicting the block map (sum+). Reword to
describe the actual sum_preserve upgrade.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Real run showed the first cut didn't fix it by default: the applied config was
the 'full' strategy (SLERP, energy 1.22x), while the style-preserving sum_preserve
config (no_slerp/basic, energy 1.91x) ranked LOWER and wasn't applied. Cause: the
Phase-2 scorer (_score_merge_result) is effective_rank*0.4 + (1-norm_cv)*0.3 +
sparsity_fit*0.3 — no magnitude/energy term, and (1-norm_cv) rewards uniform norms,
which is exactly SLERP's magnitude-flattening. So the scorer structurally prefers
the style-flattening merge over the style-preserving one.
SLERP is strictly worse than sum_preserve for orthogonal groups anyway (it rotates
two independent directions into a 45-degree blend that loses both, vs keeping each
in its own subspace), so orthogonal non-opposing groups now use sum_preserve in ALL
strategy sets — the AutoTuner's top config preserves the style by default, no manual
selection. SLERP is retained for its real sweet spot: non-orthogonal, similar-
direction (cos ~0.25-0.5) low-conflict groups in the full set.
Tests: orthogonal/full -> sum_preserve; non-orthogonal-aligned/full -> slerp;
non-orthogonal-aligned/no_slerp -> weighted_average. 304 pass. Docs + algo note updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a 'preserve' toggle to LoRAStack and LoRAStackDynamic (advanced). A LoRA
tagged preserve=True is exempt from the two mechanisms that delete a style in a
conflict merge:
- sparsification (DARE/DELLA/conflict-aware): its diff is never trimmed/dropped
- TIES sign-election: instead of being zeroed where it holds the minority sign,
its full contribution is added on top of the TIES-merged content
Plumbed as a 6th stack-tuple field / dict key through _normalize_stack,
_merge_one_group -> _merge_diffs(preserve_flags=...), and the IS_CHANGED cache
keys + LoRAConflictEditor tuple rebuilds (so toggling it invalidates caches and
survives the editor). LoRAStackDynamic JS migration extended 8->9 widgets/slot
(preserve appended at each slot tail; legacy 7- and 8-slot saves re-padded).
The auto-strength exclusion from the design was descoped (single uniform scale is
baked into the linear path + group-cache replay; the reduction is mild/floor-bounded).
7 new tests (PreserveFlagTests); conflict-editor tuple test updated to 6-tuple; 302 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per-prefix merging routes orthogonal groups to weighted_average, which divides
by Sum-of-weights and collapses each LoRA to a convex fraction. A style LoRA set
to strength 2 still lands below 1x its delta (s/(s+s') asymptotes to 1.0), so the
style vanishes while the heavier content LoRA survives -- confirmed against a real
Flux run (prodigy at 0.67x vs additive's 1.82x) and a deep-research wave.
Add 'sum_preserve': result = Sum(w_i d_i) at full strength, then a per-group energy
cap of headroom (1.5) x strongest single contribution. Two orthogonal LoRAs combine
to ~1.41x and pass through un-capped (style preserved); three equal ones hit ~1.73x
and scale back to 1.5x (no oversaturation). The cap is per-group and independent of
the global auto-strength floor, so it can't be defeated by the orthogonal-floor clamp
the way a raw weighted_sum flip would be.
Route orthogonal, non-opposing groups (no_slerp/basic strategy sets) to sum_preserve;
'full' keeps the video-tuned SLERP path. Opposing groups keep weighted_average.
Reuses the exact-linear fast path (folds the cap as a scalar from Pass-1 norms) so
full-rank Flux patches stay low-rank. Report/legend/strategy-counts updated.
11 new tests (SumPreserveMergeTests); 295 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Anima is a 2B anime text-to-image DiT that reuses NVIDIA Cosmos-Predict2's
denoiser (split QKV, 28 blocks) with a Qwen3 text encoder + unique llm_adapter.
Verified key forms against live HF files (circlestone-labs/Anima model + real
LoRAs from deadman44/dummy9996).
- Detector 'anima' placed BEFORE ACE-Step/Wan/LTX (its blocks.N.self_attn.q_proj
+ Qwen3 lora_te_layers_N otherwise collide). Discriminators: unique llm_adapter,
Cosmos triple adaln_modulation_{self_attn,cross_attn,mlp}, GPT2 mlp.layer1/2,
{self_attn,cross_attn}.output_proj.
- Normalizer -> canonical diffusion_model.blocks.N.*: handles ComfyUI/diffusion-pipe
(passthrough), Kohya lora_unet_* (underscore->dot restore), diffusers
transformer_blocks.attn1/attn2.to_* (best effort). Split QKV, no fuse/refuse.
- dit preset; arch label 'Anima (Cosmos-Predict2 DiT)'. README + wiki updated.
- 7 tests (detection on all 3 forms + no ACE/Wan/LTX collision; normalization
convergence). 284 passing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pass 1 (analysis) was silent on the console between 'Analyzing N target groups...'
and completion — slow and opaque on big models with several LoRAs (the GUI pbar
updated but nothing printed). Log 'Analyzed X/N target groups (Zs)' ~10 times,
throttled to >=1s apart so a cached/fast analysis stays quiet. Added to both the
AutoTuner and the plain optimizer Pass 1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When the sampled SVD reconstruction error is high (avg>5% or max>20%), the merged
diffs are higher-rank than the save rank captures (typical for TIES/refine merges).
Add a visible warning that names the current rank, the error, and a suggested
larger save_rank for more fidelity. Only nudges in auto mode.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The SVD compression already runs on GPU, but two things made saving slow on big
models (LTX-2, 1772 layers) and looked CPU-bound:
- the reconstruction-error diagnostic reconstructed EVERY diff (1440) on CPU and
dragged vram_budget GPU patches back to CPU. Sample 32 instead — it's just a
quality readout — and only reconstruct diff patches.
- ~37% of LTX-2 layers (664/1772) are all-zero (untrained); they were SVD'd and
written as dead layers. Skip zero diffs entirely: no SVD, smaller file, identical
result.
277 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
With vram_budget>0 the merge keeps some 'diff' patches on GPU, but the saved
low-rank factors (hence 'reconstructed') are on CPU, so the diagnostic
reconstruction-error subtraction crashed with 'tensors on cuda:0 and cpu'.
Compare on a common device and wrap the check so this diagnostic can never abort
the save.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The diff cache caps RAM (ram_pct) but the disk spill (auto/disk modes) was
uncapped — a large model x several LoRAs (e.g. LTX-2 audio-video x 3) writes
full-rank fp16 diffs until the temp volume fills, which can exceed 60GB and was
likely the cause of earlier out-of-disk incidents. Add a free-space guard: keep
~5GB free on the temp volume, then stop disk-caching and recompute the overflow
on demand (a skipped put() is a cache miss, which the caller already recomputes).
Tests + README updated; 277 passing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
