docs: sync README + wiki to v1.7 (new nodes, params, architectures)
Bring the documentation up to date with the 1.7.0 / algo 1.10.2 state: - README: document Conflict Editor, Merge Formula, Metadata Reader, Extract from Model, Combination Generator; fill param gaps (clip_strength_multiplier, dare_dampening, scoring_svd/device/formula, memory_mode, record_dataset, output_strength=-1 auto); clarify HunyuanVideo is dit-preset-only (not auto-detected). - Wiki: add 7 missing nodes to Nodes.md; fix stale option lists (scoring_svd 4 modes, memory_mode auto_ignore_strength, community_cache upload_only); add acestep_dit + Ideogram 4 to architecture tables; fix Home arch count (8->9); replace obsolete quality-level/profile names (enhanced+/maximum -> refine+/full, classic/v1.2 -> basic/full strategy sets). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Ethanfel
co-authored by
Claude Opus 4.8
Claude Opus 4.8
parent
1fc78b94e3
commit
487385cf60
@@ -91,7 +91,12 @@ Restart ComfyUI. Nodes appear under the `loaders` category.
|
||||
| **LoRA AutoTuner** | Sweep 2000+ parameter combos, rank the best configs |
|
||||
| **LoRA Merge Estimator** | Predict the best config via k-NN over the community cache — skip the sweep |
|
||||
| **Merge Selector** | Try alternative ranked configs from AutoTuner results |
|
||||
| **Conflict Editor** | Inspect pairwise conflicts, set per-LoRA conflict modes + merge strategy by hand |
|
||||
| **Compatibility Analyzer** | Check which LoRAs work well together before merging |
|
||||
| **Merge Formula** | Control hierarchical merge order, e.g. `(1+2) + 3` |
|
||||
| **Metadata Reader** | Read embedded prompt / description / merge info from any LoRA |
|
||||
| **Extract from Model** | Recover a LoRA baked into a finetuned model by diffing against the base |
|
||||
| **Combination Generator** | Auto-generate 2-/3-way combos for AutoTuner dataset collection |
|
||||
| **Save Merged LoRA** | Export the merge as a standalone `.safetensors` file |
|
||||
| **Merged LoRA to Hook** | Apply merged LoRA per-conditioning instead of globally |
|
||||
| **LoRA Optimizer (Legacy)** | All parameters on one node (superseded by Optimizer + Settings) |
|
||||
@@ -200,6 +205,7 @@ DARE and DELLA **sparsify each LoRA's diff before merging**, reducing parameter
|
||||
|---------|---------|---------|
|
||||
| `sparsification` | disabled | `disabled`, `dare`, `della`, `dare_conflict`, `della_conflict` |
|
||||
| `sparsification_density` | 0.7 | Fraction of parameters to keep (lower = more aggressive) |
|
||||
| `dare_dampening` | 0.0 | DAREx rescale dampening (0–1, [ICLR 2025](https://arxiv.org/abs/2410.01215)). Only affects `dare` / `dare_conflict` modes. `0` = standard DARE `1/density` rescale. Higher values dampen the rescale to reduce noise amplification at low density |
|
||||
|
||||
</details>
|
||||
|
||||
@@ -300,6 +306,20 @@ Your original strength ratios are always preserved — the algorithm only scales
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Output & CLIP Strength</b></summary>
|
||||
|
||||
The **LoRA Optimizer** has two strength inputs that control the merged result globally:
|
||||
|
||||
| Input | Default | Effect |
|
||||
|-------|---------|--------|
|
||||
| `output_strength` | 1.0 | Master volume for the merged result. `1.0` = full effect, `0.5` = half, `0` = disabled. Set to **`-1` for auto** — the optimizer picks the suggested max strength, compensating for energy lost during the merge |
|
||||
| `clip_strength_multiplier` | 1.0 | How strongly the LoRAs affect text understanding (CLIP). At `1.0` it matches the model strength; lower values reduce LoRA influence on prompts while keeping the visual effect. Leave at `1.0` (or ignore) for video models with no CLIP |
|
||||
|
||||
`output_strength = -1` is the easiest way to let the optimizer fully drive strength: combined with `auto_strength`, it both balances per-LoRA contributions *and* sets a sensible master level for the stack.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Decision Smoothing</b></summary>
|
||||
|
||||
@@ -345,14 +365,17 @@ All numeric thresholds in the optimizer (density estimation, conflict detection,
|
||||
| Preset | Architectures | Key Differences | Orthogonal floor |
|
||||
|--------|--------------|-----------------|------------------|
|
||||
| `sd_unet` | SD 1.5, SDXL | Density range [0.1, 0.9], noise floor 10%, max strength cap 3.0 | 0.85 |
|
||||
| `dit` | Flux, WAN, Z-Image, LTX, HunyuanVideo | Density range [0.4, 0.95], noise floor 5%, max strength cap 5.0 | 0.85 by default, 1.0 for Wan/LTX |
|
||||
| `dit` | Flux, WAN, Z-Image, LTX, Ideogram 4, HunyuanVideo | Density range [0.4, 0.95], noise floor 5%, max strength cap 5.0 | 0.85 by default, 1.0 for Wan/LTX |
|
||||
| `acestep_dit` | ACE-Step (music DiT) | DiT thresholds tuned for music LoRAs: wider orthogonal band + higher TIES threshold to preserve voice/timbre | 1.0 |
|
||||
| `llm` | Qwen-Image, LLaMA-based | Density range [0.1, 0.8], noise floor 15%, max strength cap 3.0 | 0.9 |
|
||||
|
||||
**Why it matters:** DiT architectures have denser weight distributions than UNet — with UNet thresholds, the optimizer underestimates density and clips suggested strength too aggressively. LLM-based models are sparser and benefit from lower density ceilings.
|
||||
**Why it matters:** DiT architectures have denser weight distributions than UNet — with UNet thresholds, the optimizer underestimates density and clips suggested strength too aggressively. LLM-based models are sparser and benefit from lower density ceilings. ACE-Step gets its own preset because music LoRAs are unusually conflict-prone and need a gentler merge to keep the singing voice intact.
|
||||
|
||||
| Setting | Default | Options |
|
||||
|---------|---------|---------|
|
||||
| `architecture_preset` | auto | `auto`, `sd_unet`, `dit`, `llm`. Auto-detection uses the same key pattern matching as key normalization |
|
||||
| `architecture_preset` | auto | `auto`, `sd_unet`, `dit`, `acestep_dit`, `llm`. Auto-detection uses the same key pattern matching as key normalization |
|
||||
|
||||
> **HunyuanVideo:** covered by the `dit` preset's thresholds, but it is not in the auto-detector — select `dit` manually for HunyuanVideo LoRAs.
|
||||
|
||||
**Note:** This is orthogonal to `strategy_set` (which controls *which strategies* are available — consensus, SLERP, etc.). Architecture preset controls the *numeric thresholds* those strategies use.
|
||||
|
||||
@@ -540,6 +563,39 @@ When `auto` mode runs out of disk space, it falls back to RAM automatically.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Scoring Modes & Speed</b></summary>
|
||||
|
||||
The AutoTuner ranks candidate configs with a heuristic score before merging the top-N for real quality measurement. These knobs trade scoring accuracy for speed:
|
||||
|
||||
| Setting | Default | Options / Effect |
|
||||
|---------|---------|------------------|
|
||||
| `scoring_speed` | turbo | How many prefixes each candidate is scored on. `full` = all prefixes (slowest, most accurate); `fast` ≈ every 2nd; `turbo` ≈ every 3rd (recommended); `turbo+` = fastest, biased toward high-conflict prefixes |
|
||||
| `scoring_svd` | disabled | SVD-based scoring. `disabled` = fast norm-only (usually enough); `merge_quality` = SVD on merged diffs (more thorough); `lora_rank` = effective-rank of factors (experimental, changes ranking); `full` = both. Hardware-accelerated when Triton is installed |
|
||||
| `scoring_device` | gpu | Where scoring math runs. `gpu` is much faster, especially with SVD modes |
|
||||
| `scoring_formula` | v2 | `v2` (recommended) = arch-aware sparsity + energy metrics. `v1` = legacy formula with a fixed 40% sparsity target, kept for comparison |
|
||||
|
||||
Ranking stays fair because every candidate is scored on the same subset of prefixes.
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>Persistent Memory</b></summary>
|
||||
|
||||
`memory_mode` caches a stack's tuning result across ComfyUI sessions, keyed by LoRA content hash + config. Re-running the same stack then skips the whole sweep and replays the winning config instantly.
|
||||
|
||||
| Mode | Behavior |
|
||||
|------|----------|
|
||||
| `disabled` | Never read or write the memory store |
|
||||
| `auto` (default) | Load cached result if present, save after tuning |
|
||||
| `auto_ignore_strength` | Same as `auto`, but the cache key **ignores LoRA strengths** — useful when sweeping strengths on orthogonal LoRAs where the ranking doesn't change |
|
||||
| `read_only` | Use cached results but never save new ones |
|
||||
| `clear_and_run` | Delete the cached entry and re-tune from scratch |
|
||||
|
||||
**`record_dataset`** (`disabled` / `enabled`) — when enabled, appends analysis metrics and all scored configs to `user/lora_optimizer_reports/autotuner_dataset.jsonl` for threshold-tuning research. Entries are written only when a full sweep actually runs (cache/memory replays add nothing).
|
||||
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><b>VRAM Budget</b></summary>
|
||||
|
||||
@@ -557,6 +613,7 @@ Results are keyed by **content hash** (SHA256 of file contents, not filename), s
|
||||
| `community_cache` value | Behavior |
|
||||
|---|---|
|
||||
| `disabled` | No community interaction (default) |
|
||||
| `upload_only` | Runs the sweep locally and uploads results, but does **not** replay HF cache hits. Useful for backfilling/enriching configs |
|
||||
| `upload_and_download` | Downloads before analysis; uploads after if local score is higher |
|
||||
|
||||
**Downloads** are anonymous (no setup required). **Uploads** require a `HF_TOKEN` environment variable with write access to the dataset repo.
|
||||
@@ -614,6 +671,73 @@ LoRA AutoTuner → TUNER_DATA → Merge Selector (selection=2) → try the 2nd-r
|
||||
|
||||
---
|
||||
|
||||
### Conflict Editor
|
||||
|
||||
Inserts between a **LoRA Stack** and the **LoRA Optimizer** to give you manual control over conflict handling. It loads every LoRA, computes full-rank diffs, measures pairwise sign disagreement, and auto-suggests a per-LoRA `conflict_mode` plus an overall merge strategy. You can accept the suggestions or override each LoRA's mode by hand.
|
||||
|
||||
- **Inputs:** `lora_stack`, `merge_strategy` (`auto`, `ties`, `consensus`, `slerp`, `weighted_average`, `weighted_sum`), and a per-slot `conflict_mode_1..10` (`auto`, `all`, `low_conflict`, `high_conflict`)
|
||||
- **Outputs:** the enriched `LORA_STACK`, a human-readable `analysis_report`, and the resolved `merge_strategy` string
|
||||
|
||||
Use it when you already know two LoRAs fight and want to force `high_conflict` (TIES) handling on one of them, or to read the conflict report before committing to a merge.
|
||||
|
||||
---
|
||||
|
||||
### Merge Formula
|
||||
|
||||
Passthrough node that attaches a **hierarchical merge order** to the stack. The formula uses numbered LoRA positions, `+` for blends, and parentheses for grouping — e.g. `(1+2) + 3` merges LoRAs 1 and 2 first, then blends the result with LoRA 3. Optional per-component weights are supported: `(1+2):0.6 + 3:0.4`. An empty formula falls back to a flat merge of the whole stack.
|
||||
|
||||
- **Inputs:** `lora_stack`, `formula` (string)
|
||||
- **Outputs:** `lora_stack` (unchanged tensors, formula metadata attached)
|
||||
|
||||
Useful when a clean two-LoRA base should be established before a third, more disruptive LoRA is layered on top.
|
||||
|
||||
---
|
||||
|
||||
### Metadata Reader
|
||||
|
||||
Passthrough node that extracts embedded metadata from every LoRA in the stack — works with any `.safetensors` LoRA, not just ones saved by this pack. It consolidates fields like `source_loras`, `merge_mode`, merge settings, training prompt, and description.
|
||||
|
||||
- **Inputs:** `lora_stack`
|
||||
- **Outputs:** `lora_stack` (unchanged), `prompt`, `description`, `metadata_info`
|
||||
|
||||
Connect the string outputs to **Show Text** nodes to inspect what a downloaded or previously merged LoRA actually contains.
|
||||
|
||||
---
|
||||
|
||||
### Extract from Model
|
||||
|
||||
Recovers a LoRA that was **baked into a finetuned model** by subtracting the original base-model weights and SVD-decomposing the per-layer delta. Can extract both UNet and CLIP layers when the matching base/finetuned CLIP pair is provided.
|
||||
|
||||
| Input | Default | Effect |
|
||||
|-------|---------|--------|
|
||||
| `base_model` / `finetuned_model` | — | The clean base and the finetuned checkpoint to diff |
|
||||
| `base_clip` / `finetuned_clip` (optional) | — | Provide both to also extract CLIP-side layers |
|
||||
| `rank` | 32 | Target rank when `rank_mode = fixed` |
|
||||
| `rank_mode` | auto | `auto` picks rank per layer from `energy_threshold`; `fixed` forces `rank` |
|
||||
| `energy_threshold` | 0.99 | Fraction of singular-value energy to retain in `auto` mode |
|
||||
| `strength` | 1.0 | Strength baked into the emitted stack entry |
|
||||
|
||||
- **Outputs:** `LORA_STACK` (feed straight into the Optimizer/AutoTuner) and `LORA_DATA` (feed into **Save Merged LoRA** to write a `.safetensors`)
|
||||
|
||||
---
|
||||
|
||||
### Combination Generator
|
||||
|
||||
Generates LoRA combinations for **AutoTuner dataset collection**. It cycles through all 2-way and/or 3-way combos with deterministic shuffling and tracks progress in a persistent `combo_progress.json`, so a long collection run can be resumed without repeating combos.
|
||||
|
||||
| Input | Default | Effect |
|
||||
|-------|---------|--------|
|
||||
| `shuffle_order` | 0 | Seed for deterministic shuffle ordering |
|
||||
| `strength` | 1.0 | Strength applied to each LoRA in the emitted combo |
|
||||
| `combo_size` | 2_and_3 | `2`, `3`, or `2_and_3` |
|
||||
| `folder_filter` | "" | Comma-separated path prefixes to restrict the pool (e.g. `zit/,zib/`) |
|
||||
| `rerun_mode` | false | Re-emit already-processed combos for backfilling enriched configs |
|
||||
| `rerun_source` | shuffle | `shuffle` or `original_progress` ordering when re-running |
|
||||
|
||||
- **Outputs:** one `LORA_STACK` at a time plus a `combo_info` string. Drive it with AutoTuner `record_dataset` / `community_cache` to build the shared cache.
|
||||
|
||||
---
|
||||
|
||||
### AutoTuner → Optimizer Bridge
|
||||
|
||||
Chain the AutoTuner and the Legacy optimizer in a single model line for a "rank, then tweak" workflow. Only one node merges at a time — the other passes the model through. The Legacy optimizer's `settings_source` switch controls which node is authoritative, and the UI bridge keeps the paired widgets in sync.
|
||||
@@ -743,7 +867,7 @@ WanVideoLoraSelect → WanVideoModelLoader → WANVIDEOMODEL → WanVideo LoRA O
|
||||
<details>
|
||||
<summary><b>Compatibility</b></summary>
|
||||
|
||||
- **Models:** SD 1.5, SDXL, Flux, Z-Image (Lumina2), Wan 2.1/2.2, LTX Video, ACE-Step, Qwen-Image, and other architectures supported by ComfyUI
|
||||
- **Models:** SD 1.5, SDXL, Flux, Z-Image (Lumina2), Ideogram 4, Wan 2.1/2.2, LTX Video, ACE-Step, Qwen-Image, and other architectures supported by ComfyUI
|
||||
- **LoRA formats:** Standard LoRA, LoCon, and LoRA/LoCon-style trainer variants whose tensors reduce to up/down(/mid) adapters (including many diffusers/PEFT and LyCORIS naming schemes)
|
||||
- **Trainers:** Kohya, AI-Toolkit, LyCORIS, Musubi Tuner, diffusers — auto-normalized when `normalize_keys` is enabled
|
||||
- **Flux sliced weights:** Handled correctly (linear1_qkv offsets)
|
||||
|
||||
@@ -6,11 +6,12 @@ Every parameter on the LoRA Optimizer, organized by what it controls. Each secti
|
||||
|
||||
## Strength Controls
|
||||
|
||||
### `output_strength` (0.0 – 10.0, default 1.0)
|
||||
### `output_strength` (-1.0 – 10.0, default 1.0)
|
||||
|
||||
Master volume for the entire merge. Scales all merged patches uniformly before applying to the model.
|
||||
|
||||
- Start at 1.0 and adjust if the merge is too strong/weak
|
||||
- Set to **`-1` for auto** — the optimizer picks the suggested max strength, compensating for energy lost during the merge
|
||||
- The analysis report includes a **suggested max output strength** — going above this risks oversaturation
|
||||
- Does not affect the merge itself — only the final application strength
|
||||
|
||||
@@ -54,10 +55,11 @@ Tunes numeric thresholds (density ranges, noise floors, strength caps) to match
|
||||
|-------|--------|--------|
|
||||
| `auto` (default) | Auto-detected | Picks the right preset from LoRA key patterns |
|
||||
| `sd_unet` | SD 1.5, SDXL | Lower density floor (0.1), 10% noise floor, max strength 3.0 |
|
||||
| `dit` | Flux, Wan, Z-Image, LTX, HunyuanVideo | Higher density floor (0.4), 5% noise floor, max strength 5.0 |
|
||||
| `dit` | Flux, Wan, Z-Image, LTX, Ideogram 4, HunyuanVideo | Higher density floor (0.4), 5% noise floor, max strength 5.0 |
|
||||
| `acestep_dit` | ACE-Step (music DiT) | DiT thresholds tuned for music LoRAs — wider orthogonal band + higher TIES threshold to preserve voice |
|
||||
| `llm` | Qwen, LLaMA | Tight density range (0.1–0.8), 15% noise floor, max strength 3.0 |
|
||||
|
||||
Generally leave this on `auto`. Override only if auto-detection picks the wrong family.
|
||||
Generally leave this on `auto`. Override only if auto-detection picks the wrong family. **HunyuanVideo** is covered by the `dit` preset but is not auto-detected — select `dit` manually for it.
|
||||
|
||||
### `merge_strategy_override` (optional input)
|
||||
|
||||
@@ -219,9 +221,16 @@ Number of candidate configurations to actually merge and score in Phase 2. Highe
|
||||
- 5–10 → more alternatives to choose from via Merge Selector
|
||||
- 1 → fastest, but no alternatives
|
||||
|
||||
### `scoring_svd` (enabled / disabled)
|
||||
### `scoring_svd` (disabled / merge_quality / lora_rank / full, default disabled)
|
||||
|
||||
When enabled, computes SVD-based effective rank as part of the quality score. More thorough but slower.
|
||||
SVD-based scoring for ranking candidates. Hardware-accelerated when Triton is installed, adding minimal overhead.
|
||||
|
||||
| Value | Behavior |
|
||||
|-------|----------|
|
||||
| `disabled` (default) | Fast norm-only scoring — usually sufficient |
|
||||
| `merge_quality` | SVD on the merged diff tensors — more thorough quality measurement |
|
||||
| `lora_rank` | Effective rank of the LoRA factors — experimental, changes ranking |
|
||||
| `full` | Both `merge_quality` + `lora_rank` |
|
||||
|
||||
### `scoring_device` (cpu / gpu)
|
||||
|
||||
@@ -255,7 +264,7 @@ Controls what the AutoTuner outputs after the sweep.
|
||||
|
||||
`tuning_only` is useful when you want the AutoTuner to decide the settings but apply the merge through a dedicated Optimizer node with additional controls (e.g., a custom `merge_strategy_override`).
|
||||
|
||||
### `memory_mode` (disabled / auto / read_only / clear_and_run, default auto)
|
||||
### `memory_mode` (disabled / auto / auto_ignore_strength / read_only / clear_and_run, default auto)
|
||||
|
||||
Persistent cache for tuning results. When a matching entry exists on disk, the full Phase 1 + Phase 2 sweep is skipped and the cached rankings are replayed in a single merge (~2–5 seconds instead of 30–120+).
|
||||
|
||||
@@ -263,10 +272,11 @@ Persistent cache for tuning results. When a matching entry exists on disk, the f
|
||||
|-------|----------|
|
||||
| `disabled` | No caching — always run the full sweep |
|
||||
| `auto` (default) | Load from cache if available; save new results after a sweep |
|
||||
| `auto_ignore_strength` | Same as `auto`, but the cache key **ignores LoRA strengths** — useful when sweeping strengths on orthogonal LoRAs where rankings don't change |
|
||||
| `read_only` | Use cached results but never write new ones |
|
||||
| `clear_and_run` | Delete the cached entry and re-run from scratch, then save |
|
||||
|
||||
The cache key is the LoRA set (names + strengths) + tuning settings. Changing any LoRA strength or tuning parameter causes a cache miss. Cache files live in `models/autotuner_memory/` as `{lora_hash}_{settings_hash}.memory.json`.
|
||||
The cache key is the LoRA set (names + strengths) + tuning settings. Changing any LoRA strength or tuning parameter causes a cache miss (unless `auto_ignore_strength` is used). Cache files live in `models/autotuner_memory/` as `{lora_hash}_{settings_hash}.memory.json`.
|
||||
|
||||
Use `clear_and_run` after updating a LoRA file in place (same filename, different content), or when you want fresh results after tweaking settings.
|
||||
|
||||
@@ -276,13 +286,14 @@ When a memory hit occurs, `selection` controls which ranked config is replayed.
|
||||
|
||||
This lets you explore alternatives from a cached run without re-running the full sweep: set `memory_mode=auto` and increase `selection` to try the 2nd or 3rd ranked config from the last run.
|
||||
|
||||
### `community_cache` (disabled / upload_and_download)
|
||||
### `community_cache` (disabled / upload_only / upload_and_download)
|
||||
|
||||
Community-backed cache hosted on Hugging Face. Precomputed analysis results (per-LoRA conflict stats, pairwise metrics, winning merge configs) are hardware-agnostic — the same LoRA files always produce the same output regardless of GPU tier.
|
||||
|
||||
| Value | Behavior |
|
||||
|-------|----------|
|
||||
| `disabled` (default) | No community interaction — all computation is local |
|
||||
| `upload_only` | Run the sweep locally and upload results, but do **not** replay HF cache hits. Useful for backfilling/enriching configs. Requires `huggingface-cli login` or `HF_TOKEN` env var |
|
||||
| `upload_and_download` | Download precomputed results and contribute yours back. Requires `huggingface-cli login` or `HF_TOKEN` env var |
|
||||
|
||||
**Privacy:** LoRA filenames are never shared. Only content hashes (SHA256[:16]) are used as keys. Results include conflict metrics and winning configs — no paths, no names.
|
||||
|
||||
+3
-3
@@ -46,9 +46,9 @@ The optimizer handles everything automatically — conflict analysis, strategy s
|
||||
| **Per-prefix adaptive merge** | Each weight group gets its own strategy based on local conflict data |
|
||||
| **5 merge algorithms** | Weighted sum, weighted average, TIES, SLERP, consensus |
|
||||
| **Sparsification** | DARE and DELLA with standard and conflict-aware variants |
|
||||
| **3 quality levels** | Standard, enhanced (DO + column-wise + TALL-masks), maximum (+ KnOTS SVD) |
|
||||
| **Merge refinement** | `none`, `refine` (orthogonalization + TALL-masks), `full` (+ KnOTS SVD alignment) |
|
||||
| **Auto-strength** | Interference-aware energy normalization prevents oversaturation |
|
||||
| **Architecture presets** | Tuned thresholds for UNet, DiT, and LLM architectures |
|
||||
| **Architecture presets** | Tuned thresholds for UNet, DiT, music-DiT (ACE-Step), and LLM architectures |
|
||||
| **Key normalization** | Mix LoRAs from any trainer (Kohya, AI-Toolkit, LyCORIS, diffusers, etc.) |
|
||||
| **SVD compression** | Re-compress merged patches to low-rank for ~32x RAM savings |
|
||||
| **AutoTuner** | Sweep 2,000+ parameter combinations and rank them by internal metrics or an external evaluator |
|
||||
@@ -56,7 +56,7 @@ The optimizer handles everything automatically — conflict analysis, strategy s
|
||||
| **Compatibility analyzer** | Planning node that groups merge-safe LoRAs, surfaces conflicts, and optionally auto-creates optimized node setups |
|
||||
| **Low memory** | Two-pass streaming architecture — peak memory scales with the largest active target group, not the full stack |
|
||||
| **Community cache** | Share AutoTuner results via Hugging Face — skip sweeps when the best config is already known |
|
||||
| **8 architectures** | SD 1.5, SDXL, FLUX, Z-Image, Wan, LTX Video, ACE-Step, Qwen-Image |
|
||||
| **9 architectures** | SD 1.5, SDXL, FLUX, Z-Image, Ideogram 4, Wan, LTX Video, ACE-Step, Qwen-Image |
|
||||
|
||||
---
|
||||
|
||||
|
||||
+17
-10
@@ -281,18 +281,19 @@ When using `global` optimization (or as a fallback), the optimizer picks a singl
|
||||
|
||||
### Architecture Presets
|
||||
|
||||
> _Controlled by: `architecture_preset` (auto / sd_unet / dit / llm)_
|
||||
> _Controlled by: `architecture_preset` (auto / sd_unet / dit / acestep_dit / llm)_
|
||||
|
||||
Different model architectures have different numerical characteristics. The architecture preset tunes the optimizer's internal thresholds — density ranges, noise floors, and strength caps — to match:
|
||||
|
||||
| Preset | Architectures | Density Range | Noise Floor | Max Strength Cap |
|
||||
|--------|--------------|---------------|-------------|-----------------|
|
||||
| `sd_unet` | SD 1.5, SDXL | 0.1 – 0.9 | 10% | 3.0 |
|
||||
| `dit` | Flux, Wan, Z-Image, LTX, HunyuanVideo | 0.4 – 0.95 | 5% | 5.0 |
|
||||
| `dit` | Flux, Wan, Z-Image, LTX, Ideogram 4, HunyuanVideo | 0.4 – 0.95 | 5% | 5.0 |
|
||||
| `acestep_dit` | ACE-Step (music DiT) | 0.4 – 0.95 | 5% | 5.0 |
|
||||
| `llm` | Qwen, LLaMA | 0.1 – 0.8 | 15% | 3.0 |
|
||||
| `auto` (default) | Auto-detected from LoRA keys | Selected automatically | — | — |
|
||||
|
||||
DiT models are denser and tolerate higher strengths. UNet models are sparser. LLM models need tighter density bounds and lower strength caps.
|
||||
DiT models are denser and tolerate higher strengths. UNet models are sparser. LLM models need tighter density bounds and lower strength caps. ACE-Step uses DiT-class thresholds but with a wider orthogonal band and higher TIES threshold, since music LoRAs are conflict-prone and need a gentler merge to keep the singing voice intact. **HunyuanVideo** uses the `dit` preset but is not in the auto-detector — select `dit` manually.
|
||||
|
||||
The architecture preset is **orthogonal** to the behavior profile — the preset controls _numeric thresholds_, while the profile controls _strategy selection logic_.
|
||||
|
||||
@@ -535,19 +536,19 @@ The key to understanding the optimizer is knowing that these algorithms **compos
|
||||
│ ↓ (sets density/strength thresholds) │
|
||||
│ 3. Auto-Strength (if enabled) │
|
||||
│ ↓ (adjusts strength scalars) │
|
||||
│ 4. Quality: KnOTS alignment (if maximum) │
|
||||
│ 4. Quality: KnOTS alignment (if full) │
|
||||
│ ↓ (transforms diff vectors) │
|
||||
│ 5. Quality: DO-Merging (if enhanced+) │
|
||||
│ 5. Quality: DO-Merging (if refine+) │
|
||||
│ ↓ (orthogonalizes directions) │
|
||||
│ 6. Sparsification (if enabled) │
|
||||
│ ↓ (zeros out weights) │
|
||||
│ ├─ TIES mode? → replaces trim step │
|
||||
│ └─ Other modes? → preprocessing │
|
||||
│ 7. Quality: TALL-masks (if enhanced+) │
|
||||
│ 7. Quality: TALL-masks (if refine+) │
|
||||
│ ↓ (separates selfish weights) │
|
||||
│ 8. Merge Algorithm │
|
||||
│ ↓ (combines diffs) │
|
||||
│ 9. TALL-masks re-add (if enhanced+) │
|
||||
│ 9. TALL-masks re-add (if refine+) │
|
||||
│ ↓ (restores selfish weights) │
|
||||
│ 10. SVD Compression (if enabled) │
|
||||
│ ↓ (reduces to low-rank) │
|
||||
@@ -667,7 +668,7 @@ This diagram shows every algorithm in the system and how they relate:
|
||||
│ │
|
||||
│ Recompute Diffs ──► KnOTS ──► DO-Merge ──► DARE/DELLA ──► ┌──────────┐ │
|
||||
│ (with adjusted (if (if (if │ Select │ │
|
||||
│ strengths) maximum) enhanced+) enabled) │ Merge: │ │
|
||||
│ strengths) full) refine+) enabled) │ Merge: │ │
|
||||
│ │ │ │ │
|
||||
│ ┌─────────────┤ │ wt_sum │ │
|
||||
│ │ │ │ wt_avg │ │
|
||||
@@ -785,9 +786,12 @@ Phase 2 is designed to avoid RAM exhaustion:
|
||||
| Parameter | Default | Effect |
|
||||
|-----------|---------|--------|
|
||||
| `top_n` | 3 | How many candidates to actually merge and score |
|
||||
| `scoring_svd` | disabled | Enable SVD-based effective rank scoring (slower, more thorough) |
|
||||
| `scoring_device` | gpu | Where to run SVD scoring (`gpu` is 10-50x faster) |
|
||||
| `scoring_svd` | disabled | SVD-based scoring: `disabled` (norm-only), `merge_quality` (SVD on merged diffs), `lora_rank` (effective rank, experimental), `full` (both) |
|
||||
| `scoring_device` | gpu | Where to run scoring math (`gpu` is much faster with SVD modes) |
|
||||
| `scoring_speed` | turbo | Subsample prefix scoring for faster sweeps (`full`, `fast`, `turbo`, `turbo+`) |
|
||||
| `scoring_formula` | v2 | `v2` (arch-aware sparsity + energy) or `v1` (legacy, fixed 40% sparsity target) |
|
||||
| `memory_mode` | auto | Persistent cross-session result cache (`disabled`, `auto`, `auto_ignore_strength`, `read_only`, `clear_and_run`) |
|
||||
| `community_cache` | disabled | HF community cache (`disabled`, `upload_only`, `upload_and_download`) |
|
||||
| `auto_strength_floor` | -1.0 | Minimum auto-strength floor for orthogonal LoRAs (`-1` = architecture default) |
|
||||
| `decision_smoothing` | 0.25 | Smooth per-prefix decision metrics toward block averages |
|
||||
| `evaluator` | — | External evaluator hook for prompt/reference scoring |
|
||||
@@ -915,9 +919,12 @@ Bridges `LORA_DATA` to a `WANVIDEOMODEL`. Handles the `_orig_mod.` key mismatch
|
||||
| Architecture | Key Detection Patterns | Special Handling |
|
||||
|-------------|----------------------|-----------------|
|
||||
| **FLUX** | `double_blocks`, `single_blocks` | Sliced QKV offsets, multi-trainer unification |
|
||||
| **SD 1.5** | `lora_te_`, `input_blocks`/`down_blocks` | Text encoder + UNet unified |
|
||||
| **SDXL** | `lora_te1_`, `input_blocks` | Text encoder + UNet unified |
|
||||
| **Z-Image** (Lumina2) | `diffusion_model.layers.N.attention` | Fused QKV split/re-fuse, Musubi Tuner format |
|
||||
| **Ideogram 4** (NextDiT) | `layers.N.attention.qkv`/`attention.o`, fal `conditional_transformer.` prefix | ai-toolkit / fal / PEFT prefixes unified; qkv stays fused (detected **before** Z-Image) |
|
||||
| **Wan** 2.1/2.2 | `blocks.N` with `self_attn`/`ffn` | LyCORIS / diffusers / Musubi / Fun LoRA / finetrainer unified, RS-LoRA alpha fix |
|
||||
| **ACE-Step** v1.0/v1.5 | `layers.N` with `self_attn`/`cross_attn` + `q_proj`/`k_proj`/`v_proj` | Attention key unification, music-DiT preset |
|
||||
| **LTX Video** | `adaln_single`, `attn1`/`attn2` | Trainer format unification |
|
||||
| **Qwen-Image** | `img_mlp`/`txt_mlp`/`img_mod`/`txt_mod` | Dual-stream key unification |
|
||||
|
||||
|
||||
@@ -64,7 +64,7 @@ Same as weighted sum, but divided by the sum of strengths. This ensures the outp
|
||||
|
||||
- Per-prefix: 2+ LoRAs present AND sign conflict ratio ≤ 25%
|
||||
- Global: aggregate conflict ≤ 25% AND cosine similarity between 0.3–0.7
|
||||
- Can be upgraded to SLERP in `v1.2` behavior profile when LoRAs are orthogonal
|
||||
- Can be upgraded to SLERP in the `full` strategy set when LoRAs are orthogonal
|
||||
|
||||
### Properties
|
||||
|
||||
@@ -176,8 +176,8 @@ The strongest LoRA anchors the direction; subsequent LoRAs blend in.
|
||||
|
||||
### When Selected
|
||||
|
||||
- v1.2 behavior profile: auto-selected when conflict ≤ 25% AND LoRAs are detected as orthogonal (low cosine similarity)
|
||||
- Not auto-selected in `no_slerp` or `classic` profiles
|
||||
- `full` strategy set: auto-selected when conflict ≤ 25% AND LoRAs are detected as orthogonal (low cosine similarity)
|
||||
- Not auto-selected in the `no_slerp` or `basic` strategy sets
|
||||
- Can be forced via `merge_strategy_override`
|
||||
|
||||
### Properties
|
||||
@@ -236,8 +236,8 @@ This removes noise — singular values with negligible contribution are filtered
|
||||
|
||||
### When Selected
|
||||
|
||||
- v1.2 behavior profile: auto-selected when conflict ≤ 25% AND cosine similarity > 0.7 (highly similar LoRAs)
|
||||
- Not available in `classic` profile
|
||||
- `full` strategy set: auto-selected when conflict ≤ 25% AND cosine similarity > 0.7 (highly similar LoRAs)
|
||||
- Not available in the `basic` strategy set
|
||||
- Can be forced via `merge_strategy_override`
|
||||
|
||||
### Properties
|
||||
@@ -314,7 +314,7 @@ This preserves unique contributions and same-sign reinforcements while still red
|
||||
|
||||
Quality enhancements compose with all merge strategies. They modify the diffs or the merge process to improve results. See [[How It Works#quality-enhancements]] for the pipeline position.
|
||||
|
||||
### DO-Merging (enhanced+)
|
||||
### DO-Merging (refine+)
|
||||
|
||||
*Reference: [arXiv 2505.15875](https://arxiv.org/abs/2505.15875)*
|
||||
|
||||
@@ -329,7 +329,7 @@ Orthogonalizes LoRA direction vectors via Modified Gram-Schmidt:
|
||||
|
||||
After orthogonalization, LoRAs that were partially aligned become truly independent. This reduces directional interference while preserving each LoRA's strength.
|
||||
|
||||
### Column-Wise Conflict Resolution (enhanced+)
|
||||
### Column-Wise Conflict Resolution (refine+)
|
||||
|
||||
*Inspired by [ZipLoRA, Shah et al., 2025](https://arxiv.org/abs/2311.13600)*
|
||||
|
||||
@@ -344,7 +344,7 @@ for each row j of the weight matrix:
|
||||
|
||||
Preserves structural coherence — a neuron's input weights work together as a functional unit. Falls back to element-wise for 1D tensors (biases, layer norms).
|
||||
|
||||
### TALL-Masks (enhanced+)
|
||||
### TALL-Masks (refine+)
|
||||
|
||||
*Reference: [Wang et al., 2024](https://arxiv.org/abs/2406.12832)*
|
||||
|
||||
@@ -364,7 +364,7 @@ for each position i:
|
||||
|
||||
Threshold `lambda=1.0` — a LoRA's contribution must exceed all others combined to be considered selfish. This protects unique features from being averaged away during consensus merging.
|
||||
|
||||
### KnOTS SVD Alignment (maximum)
|
||||
### KnOTS SVD Alignment (full)
|
||||
|
||||
*Reference: [Ramé et al., 2024](https://arxiv.org/abs/2407.09095)*
|
||||
|
||||
|
||||
+138
-15
@@ -75,7 +75,7 @@ Shared configuration node for parameters common to both the optimizer and AutoTu
|
||||
| Input | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `normalize_keys` | COMBO | Yes | `enabled` | `enabled`, `disabled` — architecture-aware key normalization |
|
||||
| `architecture_preset` | COMBO | Yes | `auto` | `auto`, `sd_unet`, `dit`, `llm` — numeric threshold tuning |
|
||||
| `architecture_preset` | COMBO | Yes | `auto` | `auto`, `sd_unet`, `dit`, `acestep_dit`, `llm` — numeric threshold tuning |
|
||||
| `auto_strength_floor` | FLOAT | Yes | -1.0 | Minimum auto-strength scale factor for orthogonal LoRAs (`-1` = architecture default) |
|
||||
| `decision_smoothing` | FLOAT | Yes | 0.25 | Smooth per-prefix decision metrics toward block averages (0 disables) |
|
||||
| `smooth_slerp_gate` | BOOLEAN | Yes | `false` | Use per-prefix cosine similarity for SLERP gate instead of collection average |
|
||||
@@ -103,7 +103,7 @@ Optimizer-specific configuration. Accepts optional Merge Settings input.
|
||||
| `merge_refinement` | COMBO | Yes | `none` | `none`, `refine`, `full` |
|
||||
| `sparsification` | COMBO | Yes | `disabled` | `disabled`, `dare`, `della`, `dare_conflict`, `della_conflict` |
|
||||
| `sparsification_density` | FLOAT | Yes | 0.7 | Fraction of parameters to keep (0.01–1.0) |
|
||||
| `dare_dampening` | FLOAT | Yes | 0.0 | DAREx noise reduction (0–1.0) |
|
||||
| `dare_dampening` | FLOAT | Yes | 0.0 | DAREx noise reduction (0–1.0, only affects `dare`/`dare_conflict` modes) |
|
||||
| `patch_compression` | COMBO | Yes | `smart` | `smart`, `aggressive`, `disabled` |
|
||||
| `svd_device` | COMBO | Yes | `gpu` | `gpu`, `cpu` — device for SVD compression |
|
||||
| `free_vram_between_passes` | COMBO | Yes | `disabled` | `enabled`, `disabled` — release GPU cache between passes |
|
||||
@@ -133,15 +133,16 @@ AutoTuner-specific configuration. Accepts optional Merge Settings input.
|
||||
| Input | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `top_n` | INT | Yes | 3 | Number of candidates to merge and score (1–10) |
|
||||
| `scoring_svd` | COMBO | Yes | `disabled` | `enabled`, `disabled` — SVD-based effective rank scoring |
|
||||
| `scoring_device` | COMBO | Yes | `gpu` | `cpu`, `gpu` — device for SVD scoring |
|
||||
| `scoring_svd` | COMBO | Yes | `disabled` | `disabled`, `merge_quality`, `lora_rank`, `full` — `disabled` = fast norm-only; `merge_quality` = SVD on merged diffs; `lora_rank` = effective-rank of factors (experimental); `full` = both |
|
||||
| `scoring_device` | COMBO | Yes | `gpu` | `cpu`, `gpu` — device for scoring math (much faster on GPU with SVD modes) |
|
||||
| `scoring_speed` | COMBO | Yes | `turbo` | `full`, `fast`, `turbo`, `turbo+` — subsample scoring |
|
||||
| `scoring_formula` | COMBO | Yes | `v2` | `v2`, `v1` — scoring formula version |
|
||||
| `scoring_formula` | COMBO | Yes | `v2` | `v2` (arch-aware sparsity + energy metrics) or `v1` (legacy, fixed 40% sparsity target) |
|
||||
| `diff_cache_mode` | COMBO | Yes | `auto` | `disabled`, `auto`, `ram`, `disk` |
|
||||
| `diff_cache_ram_pct` | FLOAT | Yes | 0.5 | RAM fraction for auto diff cache (0.1–0.9) |
|
||||
| `community_cache` | COMBO | Yes | `disabled` | `disabled`, `upload_and_download` — community HF cache |
|
||||
| `memory_mode` | COMBO | Yes | `auto` | `disabled`, `auto`, `read_only`, `clear_and_run` — persistent tuning result cache across sessions |
|
||||
| `community_cache` | COMBO | Yes | `disabled` | `disabled`, `upload_only`, `upload_and_download` — community HF cache. `upload_only` runs locally + uploads without replaying HF hits |
|
||||
| `memory_mode` | COMBO | Yes | `auto` | `disabled`, `auto`, `auto_ignore_strength`, `read_only`, `clear_and_run` — persistent tuning result cache across sessions. `auto_ignore_strength` keys the cache without LoRA strengths |
|
||||
| `selection` | INT | Yes | 1 | Which ranked config to replay from memory on a cache hit (1 = top-ranked) |
|
||||
| `record_dataset` | COMBO | Yes | `disabled` | `disabled`, `enabled` — append analysis metrics + scored configs to `autotuner_dataset.jsonl` (only on full sweeps) |
|
||||
|
||||
### Optional Inputs
|
||||
|
||||
@@ -168,7 +169,7 @@ The simplified auto-optimizer. Sensible defaults, minimal controls. Auto-strengt
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `model` | MODEL | Yes | — | Base model from checkpoint loader |
|
||||
| `lora_stack` | LORA_STACK | Yes | — | Stack of LoRAs to merge |
|
||||
| `output_strength` | FLOAT | Yes | 1.0 | Master strength multiplier (0–10) |
|
||||
| `output_strength` | FLOAT | Yes | 1.0 | Master strength multiplier (-1 to 10). `-1` = auto: the optimizer picks the suggested max strength |
|
||||
| `clip` | CLIP | No | — | Text encoder (if using CLIP LoRA keys) |
|
||||
| `clip_strength_multiplier` | FLOAT | No | 1.0 | CLIP strength relative to model strength (0–10) |
|
||||
| `tuner_data` | TUNER_DATA | No | — | AutoTuner result — applies the winning config's settings |
|
||||
@@ -212,7 +213,7 @@ All inputs from the simple variant, plus:
|
||||
| `free_vram_between_passes` | COMBO | disabled | `enabled`, `disabled` — release GPU cache between passes |
|
||||
| `normalize_keys` | COMBO | enabled | `enabled`, `disabled` — architecture-aware key normalization |
|
||||
| `strategy_set` | COMBO | full | `full`, `no_slerp`, `basic` — strategy selection logic |
|
||||
| `architecture_preset` | COMBO | auto | `auto`, `sd_unet`, `dit`, `llm` — numeric threshold tuning |
|
||||
| `architecture_preset` | COMBO | auto | `auto`, `sd_unet`, `dit`, `acestep_dit`, `llm` — numeric threshold tuning |
|
||||
| `auto_strength_floor` | FLOAT | -1.0 | Minimum auto-strength scale factor for orthogonal LoRAs (`-1` = architecture default) |
|
||||
| `decision_smoothing` | FLOAT | 0.25 | Smooth per-prefix decision metrics toward the surrounding block average (0 disables smoothing) |
|
||||
| `smooth_slerp_gate` | BOOLEAN | false | Use per-prefix cosine similarity for SLERP gate instead of collection average |
|
||||
@@ -243,16 +244,18 @@ All inputs from the Legacy optimizer, plus:
|
||||
| Input | Type | Default | Description |
|
||||
|-------|------|---------|-------------|
|
||||
| `top_n` | INT | 3 | Number of candidates to actually merge and score (1–10) |
|
||||
| `scoring_svd` | COMBO | disabled | `enabled`, `disabled` — SVD-based effective rank scoring |
|
||||
| `scoring_device` | COMBO | gpu | `cpu`, `gpu` — device for SVD scoring (GPU is 10–50x faster) |
|
||||
| `scoring_svd` | COMBO | disabled | `disabled`, `merge_quality`, `lora_rank`, `full` — `disabled` = fast norm-only; `merge_quality` = SVD on merged diffs; `lora_rank` = effective-rank (experimental); `full` = both |
|
||||
| `scoring_device` | COMBO | gpu | `cpu`, `gpu` — device for scoring math (GPU much faster with SVD modes) |
|
||||
| `scoring_speed` | COMBO | turbo | `full`, `fast`, `turbo`, `turbo+` — subsample scoring for faster sweeps |
|
||||
| `scoring_formula` | COMBO | v2 | `v2` (arch-aware sparsity + energy metrics) or `v1` (legacy, fixed 40% sparsity target) |
|
||||
| `output_mode` | COMBO | merge | `merge`, `tuning_only` — output the top-ranked merged model, or pass the base model through so a downstream optimizer node applies the AutoTuner's settings |
|
||||
| `auto_strength_floor` | FLOAT | -1.0 | Minimum auto-strength scale factor for orthogonal LoRAs |
|
||||
| `decision_smoothing` | FLOAT | 0.25 | Same smoothing control as the optimizer; affects both ranking and final merge |
|
||||
| `evaluator` | AUTOTUNER_EVALUATOR | — | Optional external evaluator hook for prompt/reference scoring |
|
||||
| `community_cache` | COMBO | disabled | `disabled`, `upload_and_download` — community HF cache (no names shared, content hashes only) |
|
||||
| `memory_mode` | COMBO | auto | `disabled`, `auto`, `read_only`, `clear_and_run` — persistent tuning result cache across sessions |
|
||||
| `community_cache` | COMBO | disabled | `disabled`, `upload_only`, `upload_and_download` — community HF cache (no names shared, content hashes only) |
|
||||
| `memory_mode` | COMBO | auto | `disabled`, `auto`, `auto_ignore_strength`, `read_only`, `clear_and_run` — persistent tuning result cache across sessions |
|
||||
| `selection` | INT | 1 | Which ranked config to replay from memory on a cache hit (1 = top-ranked) |
|
||||
| `record_dataset` | COMBO | disabled | `disabled`, `enabled` — log analysis metrics + all scored configs to `autotuner_dataset.jsonl` (full sweeps only) |
|
||||
| `cache_patches` | COMBO | enabled | Cache the final AutoTuner result in RAM for fast re-execution |
|
||||
| `diff_cache_mode` | COMBO | auto | Diff cache mode across candidates: `disabled`, `auto`, `ram`, `disk` |
|
||||
| `diff_cache_ram_pct` | FLOAT | 0.5 | RAM fraction used before `auto` diff cache spills to disk |
|
||||
@@ -275,6 +278,32 @@ See [[How It Works#appendix-autotuner]] for details on the two-phase architectur
|
||||
|
||||
---
|
||||
|
||||
## LoRA Merge Estimator
|
||||
|
||||
Predicts a merge config via k-NN retrieval over the Hugging Face community cache — skips the full AutoTuner sweep. Runs Phase 1 analysis once, retrieves the nearest cached combos, aggregates their winning configs by inverse-distance weighting, and emits `TUNER_DATA`.
|
||||
|
||||
### Inputs
|
||||
|
||||
| Input | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `model` | MODEL | Yes | — | Base model (used by Phase 1 analysis) |
|
||||
| `lora_stack` | LORA_STACK | Yes | — | Stack to estimate a config for |
|
||||
| `k` | INT | Yes | 5 | Number of nearest neighbors to retrieve (1–20) |
|
||||
| `rebuild_index` | COMBO | Yes | `auto` | `auto` rebuilds when the HF dataset/index schema changes; `force` always rebuilds; `skip` never rebuilds (errors if missing) |
|
||||
| `top_n_output` | INT | Yes | 3 | How many aggregated candidates to emit in `TUNER_DATA` (1–10) |
|
||||
| `clip` | CLIP | No | — | Optional; only used by Phase 1 analysis |
|
||||
|
||||
### Outputs
|
||||
|
||||
| Output | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `tuner_data` | TUNER_DATA | Aggregated predicted configs — feed into LoRA Optimizer (`settings_source=from_tuner_data`) or Merge Selector |
|
||||
| `estimator_report` | STRING | Predicted configs with aggregated scores and neighbor distances (says `No neighbors` when no family+combo-size match exists) |
|
||||
|
||||
The first run downloads the community cache and builds a local k-NN index under `ComfyUI/models/estimator/` (~30–60s); later runs complete in seconds.
|
||||
|
||||
---
|
||||
|
||||
## Merge Selector
|
||||
|
||||
Applies a specific ranked configuration from AutoTuner results.
|
||||
@@ -314,11 +343,11 @@ Interactive conflict analysis with per-LoRA override controls.
|
||||
|
||||
| Input | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `model` | MODEL | Yes | Base model |
|
||||
| `lora_stack` | LORA_STACK | Yes | Stack to analyze |
|
||||
| `merge_strategy` | COMBO | Yes | `auto`, `ties`, `consensus`, `slerp`, `weighted_average`, `weighted_sum` |
|
||||
| `conflict_mode_1..10` | COMBO | Yes | Per-slot override: `auto` (node suggests), `all`, `low_conflict`, `high_conflict` |
|
||||
|
||||
Per-LoRA conflict mode overrides appear dynamically based on the stack.
|
||||
The 10 per-slot `conflict_mode_i` widgets are always present; only those matching loaded LoRAs take effect.
|
||||
|
||||
### Outputs
|
||||
|
||||
@@ -341,6 +370,76 @@ Per-LoRA conflict mode overrides appear dynamically based on the stack.
|
||||
|
||||
---
|
||||
|
||||
## LoRA Merge Formula
|
||||
|
||||
Passthrough node that attaches a hierarchical merge order to the stack. Tensors pass through unchanged; only formula metadata is added.
|
||||
|
||||
### Inputs
|
||||
|
||||
| Input | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `lora_stack` | LORA_STACK | Yes | — | Stack to annotate |
|
||||
| `formula` | STRING | Yes | `""` | Merge order using LoRA positions, `+`, and parentheses — e.g. `(1+2) + 3`. Optional weights: `(1+2):0.6 + 3:0.4`. Empty = flat merge |
|
||||
|
||||
### Outputs
|
||||
|
||||
| Output | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `lora_stack` | LORA_STACK | Same stack with the merge formula attached |
|
||||
|
||||
The optimizer reads the formula and merges sub-groups before composing them, instead of merging the whole stack flat. Useful for establishing a clean base before layering on a disruptive LoRA.
|
||||
|
||||
---
|
||||
|
||||
## LoRA Metadata Reader
|
||||
|
||||
Passthrough node that extracts embedded metadata from every LoRA in the stack. Works with any `.safetensors` LoRA, not just ones saved by this pack.
|
||||
|
||||
### Inputs
|
||||
|
||||
| Input | Type | Required | Description |
|
||||
|-------|------|----------|-------------|
|
||||
| `lora_stack` | LORA_STACK | Yes | Stack to read |
|
||||
|
||||
### Outputs
|
||||
|
||||
| Output | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `lora_stack` | LORA_STACK | Unchanged stack (passthrough) |
|
||||
| `prompt` | STRING | Training/activation prompt fields, if present |
|
||||
| `description` | STRING | Description fields, if present |
|
||||
| `metadata_info` | STRING | Consolidated metadata (`source_loras`, `merge_mode`, merge settings, custom fields) |
|
||||
|
||||
Connect the string outputs to **Show Text** nodes to inspect downloaded or previously merged LoRAs.
|
||||
|
||||
---
|
||||
|
||||
## LoRA Extract from Model
|
||||
|
||||
Recovers a LoRA baked into a finetuned model by subtracting the base-model weights and SVD-decomposing the per-layer delta.
|
||||
|
||||
### Inputs
|
||||
|
||||
| Input | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `base_model` | MODEL | Yes | — | The clean base model |
|
||||
| `finetuned_model` | MODEL | Yes | — | The finetuned model to extract from |
|
||||
| `rank` | INT | Yes | 32 | Target rank when `rank_mode=fixed` (1–512) |
|
||||
| `rank_mode` | COMBO | Yes | `auto` | `auto` picks rank per layer from `energy_threshold`; `fixed` forces `rank` |
|
||||
| `energy_threshold` | FLOAT | Yes | 0.99 | Fraction of singular-value energy to retain in `auto` mode (0.5–1.0) |
|
||||
| `strength` | FLOAT | Yes | 1.0 | Strength baked into the emitted stack entry |
|
||||
| `base_clip` | CLIP | No | — | Provide both CLIPs to also extract text-encoder layers |
|
||||
| `finetuned_clip` | CLIP | No | — | Finetuned text encoder |
|
||||
|
||||
### Outputs
|
||||
|
||||
| Output | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `lora_stack` | LORA_STACK | Feed directly into the Optimizer / AutoTuner |
|
||||
| `lora_data` | LORA_DATA | Feed into **Save Merged LoRA** to write a `.safetensors` |
|
||||
|
||||
---
|
||||
|
||||
## Save Merged LoRA
|
||||
|
||||
Exports merged patches as a standalone `.safetensors` file usable with any standard LoRA loader.
|
||||
@@ -406,6 +505,30 @@ Persist AutoTuner rankings for reuse across optimizer runs.
|
||||
|
||||
---
|
||||
|
||||
## LoRA Combination Generator
|
||||
|
||||
Generates LoRA combinations for AutoTuner dataset collection. Cycles through all 2-/3-way combos with deterministic shuffling and tracks progress in a persistent `combo_progress.json`, so long collection runs resume without repeating combos. Pair with AutoTuner `record_dataset` / `community_cache` to build the shared cache.
|
||||
|
||||
### Inputs
|
||||
|
||||
| Input | Type | Required | Default | Description |
|
||||
|-------|------|----------|---------|-------------|
|
||||
| `shuffle_order` | INT | Yes | 0 | Seed for deterministic shuffle ordering |
|
||||
| `strength` | FLOAT | Yes | 1.0 | Strength applied to each LoRA in the emitted combo |
|
||||
| `combo_size` | COMBO | Yes | `2_and_3` | `2`, `3`, or `2_and_3` |
|
||||
| `folder_filter` | STRING | Yes | `""` | Comma-separated path prefixes to restrict the pool (e.g. `zit/,zib/`) |
|
||||
| `rerun_mode` | BOOLEAN | Yes | `false` | Re-emit already-processed combos for backfilling enriched configs |
|
||||
| `rerun_source` | COMBO | Yes | `shuffle` | `shuffle` or `original_progress` ordering when re-running |
|
||||
|
||||
### Outputs
|
||||
|
||||
| Output | Type | Description |
|
||||
|--------|------|-------------|
|
||||
| `lora_stack` | LORA_STACK | One combo at a time |
|
||||
| `combo_info` | STRING | Human-readable description of the emitted combo + progress |
|
||||
|
||||
---
|
||||
|
||||
## LoRA Compatibility Analyzer
|
||||
|
||||
Pre-merge planning node that analyzes overlap, cosine similarity, and conflicts without applying a merge.
|
||||
|
||||
@@ -120,7 +120,7 @@ The optimizer's two-pass architecture usually keeps peak memory near one active
|
||||
|
||||
### "Key normalization picks the wrong architecture"
|
||||
|
||||
This can happen if LoRA keys are ambiguous. Set `architecture_preset` manually to the correct value (`sd_unet`, `dit`, or `llm`) to override auto-detection.
|
||||
This can happen if LoRA keys are ambiguous. Set `architecture_preset` manually to the correct value (`sd_unet`, `dit`, `acestep_dit`, or `llm`) to override auto-detection.
|
||||
|
||||
### "LoRAs from different trainers don't merge"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user