Per-block scaling is derived from the LoRA's own deltas — which blocks it put
energy into — so it is a property of the adapter, not of the architecture. Two
things made it LTX-only anyway:
The gate required image_model in {ltxv, ltxav} and mode ltx2, so a model
FunPack had not heard of got the global fallback without anything ever reading
its weights. Whether per-block can run is now discovered from the patches:
fewer than two numbered blocks and the existing fallback takes over, as before.
The key pattern only matched transformer_blocks.N. — MiniMax H3 names its
stack blocks.N., so even with the gate open every H3 patch counted as global
and per-block would have been a silent no-op. The pattern now covers blocks,
transformer_blocks, double_blocks, single_blocks, joint_blocks and layers.
The semantic anchors (PAG 14, STG 14/19) were absolute indices measured on
LTX's 28-block DiT and meant nothing on a stack of another depth. They are
positions now; a 28-block model still resolves to exactly 14 and 19, so LTX
behaviour is unchanged, and a deeper stack gets the equivalent blocks.
Declines, loudly, when a model numbers blocks in two separate stacks
(Flux-style double/single): both start at 0, so one scale would land on two
unrelated blocks.
Untested on a real model — local runs cannot load one. 13 new tests, 11 of
which fail against the previous code.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The drop message named three keys and a count with no file attached, and
note_on_change keyed it globally, so in a stack only the first LoRA's shapes
were ever printed. Each file now carries its name into its own log key, and
the count is reported as dropped-of-matched: 51 of 400 is a variant mismatch,
51 of 51 is a file that contributes nothing.
Adds one stack-level line. Every LoRA dropping every weight is not a property
of the LoRAs but of the model they were all measured against, and that is the
reading worth making explicit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
comfy merges with mm(...).reshape(weight.shape) and adds the result. The
reshape succeeds on any pair with the right element count, so loosening the
drop test to a count comparison let transposed and differently fused deltas
merge in silence. A correct adapter always produces exactly
(weight.shape[0], weight.shape[1:].numel()); anything else fitting by count
corrupts the weight at load and surfaces mid-render as an all-NaN latent.
The two abstentions kept: a locon mid and a padded reshape target are still
left to comfy rather than dropped.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The check was stricter than the merge it was protecting. comfy merges with
mm(mat1.flatten(1), mat2.flatten(1)).reshape(weight.shape), so the element
count is the constraint, not the two dimensions — and a locon `mid` or a
`reshape` entry changes what the delta has to fit at all. Those two now
abstain instead of dropping, and the count is what is compared.
The message carries both shapes. "Trained against a different variant" says
nothing you can act on; "LoRA 5376x5376 into weight 16128x5376" says whether
every layer is off by one factor (a different width, nothing to do) or only
some are (a fusion or naming difference, fixable).
Comfy materialises the full delta before discovering it will not
reshape: 51 blocks x 96768x2688 on a curve-form H3 checkpoint, ~25 GB
churned in bf16, for adapters that could never have applied.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Curve-form H3 checkpoints read adaLN from a compact basis, so a full-width
turbo LoRA matches all 51 adaLN keys and merges into none. Named explicitly
instead of 51 generic warnings under a clean-looking status line.
Rows are picked from a searchable list of installed files instead of typed into a
table of dropdowns, and an input the node calls advanced (the trained stack) is
folded away unless something feeds it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Empty is a working state now, logged as "No active LoRAs", so the hop every
LoRA needs is already wired into a fresh pipeline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Quantization + a per-model attention backend read from ComfyUI's own registry, so only
installed ones are offered. The LoRA loader now takes its own list and scores wrapper
prefixes against the model's key map, instead of applying a diffusers LoRA as a silent no-op.
LTXAVTEModel.load_sd passes the full checkpoint sd to Gemma3_12B.load_state_dict,
which has vision_model and multi_modal_projector as real attributes with matching
key names. Vision weights are already present after DualCLIPLoader runs.
Drop _vision_loaded flag check; detect vision capability by attribute existence.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Checkpoints converted for ComfyUI use vision_model.* and
multi_modal_projector.* as top-level keys directly, not the HuggingFace
model.vision_tower.* nesting. Accept both formats.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When no vision keys are found, log the top-level key prefixes actually
present in the checkpoint so the user can see whether the vision tower
was stripped. Also rename all [FunPackGemmaLoader] log prefixes to
[FunPackGemmaVision].
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Companion to DualCLIPLoader: takes the already-built CLIP object (which has
the correct LTXAVTEModel structure) and loads the SigLIP vision tower +
multi-modal projector from the Gemma3-12B-IT checkpoint, setting
_vision_loaded=True. Workflow: DualCLIPLoader → FunPackGemmaVision → Studio.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FunPackGemmaLoader: drop-in replacement for Load CLIP Model that additionally
loads the SigLIP vision tower and multi-modal projector from Google's
Gemma3-12B-IT checkpoint into the already-initialized Gemma3_12B module,
enabling reference-image vision prompting without a separate CLIP Vision model.
Vision prompting: _v2_encode_prompt accepts reference_image; when the clip
has _vision_loaded=True, tokenizes with image=reference_image and
skip_template=False so Gemma3 sees the reference frame before generating
conditioning. Cache key includes image fingerprint to avoid stale hits.
source_image from Studio/refine_v2 flows through to _v2_conditioning_source.
K/V injection revamp (Technique 5): identity blocks in i2v mode now use
in-context conditioning instead of lerp injection. Pre-hook prepends reference
hidden-state tokens to vx and extends v_pe with neutral-rotation entries
(cos=1, sin=0) so video tokens self-attend to reference tokens in attn1.
Post-hook crops the reference prefix from the block output. Strength raised
from 0.02 → 1.0 (sigma-gated: active at 0.35–0.80). _extend_v_pe helper
handles 2-D/3-D/4-D PE tensors. Anchor blocks keep the existing lerp path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removed from LORA_BLOCK_TYPE_PROFILES, LORA_TYPE_BLOCK_ZONES,
LORA_REFINER_TYPE_PROFILES, ACTION_LORA_TYPES, the validation
special-case, and the Studio JS LORA_TYPES dropdown.
normalize_lora_type() keeps the concept->action mapping for silent
backwards compatibility with any existing saved stacks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add LORA_TYPE_BLOCK_ZONES: type-aware block zones that suppress bad-fit
blocks to 0.0 and mildly boost good-zone blocks per LoRA type
- Add SEMANTIC_ANCHOR_BLOCKS {14, 19}: confirmed semantic focal points
(PAG default=14, STG defaults=14,19) protected from all suppression
- Add _semantic_anchor_quality: uses LoRA energy share in blocks 14+19
as a quality signal, scaling all block weights 0.92-1.08x accordingly
- Stack priority: weaker LoRA keeps base scale on anchor blocks instead
of being damped, letting the dominant signal carry semantic content
- Fix range display: status line now shows effective weights (model_weight
* scale) instead of raw multipliers; adds zeroed= and anchor_q= fields
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
