set_model_sampler_cfg_function() detects the old (cond, uncond, cond_scale)
calling convention by counting ALL signature parameters, not just
positional ones -- the previous 3-param version (args + two defaulted
closure vars) tripped that check and got called with three raw tensors
instead of the args dict, crashing with "too many indices for tensor of
dimension 3". Fixed by taking a single `args` param and closing over
noref_to/ref_scale normally.
Optional reference_guidance_scale input (default 1.0 = off). When >1,
adds a third forward pass per denoising step with the reference tokens
dropped, isolating the reference's own contribution the same way CFG
isolates the text prompt: denoised += (scale-1)*(with_ref - without_ref).
Implemented via model.set_model_sampler_cfg_function(), reusing the
positive conditioning already passed to KSampler for the extra pass.
Mirrors the same reference-CFG mechanism added to ltx-trainer's
validation_runner.py so training-time validation and this node's
inference now share the same guidance formula.
New optional input `layout` (default "overlap", the only behavior every
checkpoint so far was trained with -- old workflows keep working
unchanged). "st_drc" shifts the whole reference coordinate block past
the target's own extent on every axis (non-overlapping RoPE region),
mirroring ltx_trainer's training_strategies/tass.py apply_tass_layout
st_drc branch -- kept in sync manually since this node can't import the
trainer package, adapted to ComfyUI's own [B,3,N] single-corner
coordinate shape instead of the trainer's [B,3,N,2] patch-bounds shape
(the min/max-per-axis shift math is equivalent either way, verified
against a synthetic test matching apply_tass_layout's own logic).
Only usable with a checkpoint specifically trained with layout='st_drc'
-- using it against an 'overlap'-trained checkpoint will not work,
the model never learned that coordinate convention.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
New IMAGE outputs (appended at the end of RETURN_TYPES, existing wiring
unaffected):
- ref_preview: exactly what gets encoded as the reference, after
whatever resize/crop/letterbox ref_resize_mode applied -- lets you
sanity-check the actual input to the model.
- crop_overlay: the ORIGINAL reference with a green rectangle around the
region that survives match_target's crop (everything outside it gets
discarded). For match_target_letterbox/native_resolution, which never
crop, the box just outlines the full frame.
New optional input crop_anchor (center/top/bottom/left/right, default
"center"): controls which part of the reference match_target's crop
keeps, instead of always the exact center -- e.g. set "top" if your
sheet's face closeup sits in the top row so it survives a crop against
a differently-shaped output. Added under "optional", not "required", so
existing saved workflows keep working with zero changes (falls back to
the previous always-center-crop behavior). Verified byte-for-byte
equivalent to the old center-crop path on a smooth test image when left
at the default; the non-default anchors reuse the same crop math with a
different offset.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
match_target resizes the reference to the output video's pixel size via
a center-crop (comfy.utils.common_upscale crop="center") then resize --
for aspect-mismatched refs (e.g. a landscape character sheet used with a
portrait output) this silently discards whatever isn't in the middle of
the reference, which can crop out the face closeup panel entirely.
Add match_target_letterbox: same target pixel size, but fits the WHOLE
reference inside it preserving its own aspect ratio (no crop), padding
the leftover space with white. Trades some wasted canvas space for
never losing reference content -- recommended over match_target when
ref and output aspect ratios differ, and over native_resolution when
native_resolution's landscape-composition bias causes an off-center/
cropped-looking result on the OUTPUT video itself.
Reported by a LoRA user: portrait output + landscape character sheet +
native_resolution produced an off-center-left result (model composing
as landscape then getting cropped to the portrait canvas); switching to
match_target "fixed" centering but lost face detail from the crop.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
fit_mode='fit' only guaranteed the row-justified block filled canvas
width, never upscaling to also fill height when the natural block was
shorter than the canvas (e.g. 2 refs left a top/bottom bar). Add
fit_mode='cover': same row-justified layout, but when the natural block
underfills the canvas it scales up to fill height and center-crops the
resulting width overflow instead, and when it overfills height it crops
that overflow directly -- either way every pixel of the canvas is
covered, cropping only the minimum shared/symmetric amount.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Replace per-panel independent letterboxing with a row-justified layout:
every row fills the canvas width exactly (each image scaled uniformly,
no distortion), rows are only shrunk together (never upscaled past
natural fill-width size) if they'd collectively overflow the canvas
height. Net effect: at most one shared pair of side/top-bottom bars
instead of padding scattered inside every individual panel.
Ported the identical algorithm into build_multiref_sheet.py (the
training-side script) so inference-time sheets match training-time
sheets pixel-for-pixel when fit_mode='fit' is used -- verified with a
byte-for-byte comparison test. fit_mode='crop' (default) is unchanged.
crop (default, unchanged): zoom+center-crop to fill each panel, matches
the training data. fit: scale each reference to fit entirely inside its
panel with a single uniform scale factor (no distortion), letterboxed
with background padding for any leftover space -- keeps every pixel
of references with extreme aspect ratios instead of cropping them.
Same grid convention as the multi-ref LTX-2 LoRA training pipeline
(build_multiref_sheet.py): deterministic layout by ref count (1x1, 2x1,
3x1, 2x2, 3-top+2-bottom), 512x512 panels, centered onto a fixed
1536x1024 canvas regardless of how many refs are plugged in. Panel
order matches the image0/image1/... indexing used in training.
With a guide node in the graph (LTXAddVideoICLoRAGuide etc.), the per-token
timestep legitimately spans the grid-mask domain (video + guide tokens) and is
indexed later via timestep[:, grid_mask]. The existing oversized heuristic
assumed oversize could only come from audio, trimmed the timestep to
video-only, and the subsequent grid_mask indexing crashed:
IndexError: The shape of the mask [5616] at index 0 does not match the
shape of the indexed tensor [2, 3726, 1] at index 1
Fix: when the incoming per-token timestep length equals grid_mask length,
skip the audio trim, append the reference zeros directly, and let the
existing grid_mask padding keep modulation and tokens in lockstep. The
audio-trim path is unchanged for graphs without guides.
Tested: LTX-2.3 22B distilled, Face-ID LoRA + union-control IC-LoRA pose
guide + frozen audio latent, 97-281 frame renders.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ref_px was always resized via common_upscale(..., lat_w*w_sf, lat_h*h_sf)
i.e. the TARGET video's pixel size, regardless of what resolution the
reference was actually trained at. Fine for old single-face-crop recipes
(ref bucket == video bucket), but breaks checkpoints trained with a fixed,
independent ref resolution (e.g. the face+3views composite ref, always
2048x1024 no matter the output video size): the ref latent ends up with a
different token count than training, destroying fine facial detail.
Adds ref_resize_mode: "match_target" (old behavior, default) vs
"native_resolution" (encode the ref at its own size, rounded to 32px).
Same kwarg collision as prepare_timestep: overlap's process_output was
reading ref_len from kw["_id_ref_len"] (set by multiangle's process_input
to 768) and trimming 768 tokens, then multiangle's process_output trimmed
another 768 — leaving 6912-1536=5376 tokens and causing the einops
unpatchify crash (5376 != 6144).
Fix: use getattr(self, "_id_ref_len", 0) — instance attr only, which is
correctly set to 0 by overlap's own process_input when no single-ref is
active.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two root causes fixed:
1. Kwarg collision (primary): overlap's prepare_timestep was reading
ref_len from kw["_id_ref_len"] which multi-angle's process_input sets
to 768 via additional_args. When overlap is outer (installed after
multi-angle), it incorrectly extended the timestep even with no active
single-ref, causing size mismatches on subsequent multi-angle runs.
Fix: use getattr(self, "_id_ref_len", 0) — instance attr only, which
is correctly set to 0 by overlap's own process_input when no ref found.
2. Combined audio+video timestep (secondary): with audio connected,
LTXAV.process_timestep can produce a per-token v_timestep that covers
both video and audio tokens (e.g. 24576 = 6144 video + 18432 audio).
Both patches now trim to the first tgt_len tokens before extending,
keeping modulation in lockstep with vx (video-only).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Guide nodes inject a grid_mask that filters x tokens in _process_input and indexes the
timestep in _prepare_timestep (timestep[:, grid_mask]). Our identity ref tokens are
appended AFTER that filter, so the mask (sized without them) crashed with
'shape of the mask [N] does not match the indexed tensor [B, N+ref, 1]'. Both nodes now
extend grid_mask with True for the appended ref tokens, keeping the AdaLN modulation in
lockstep with the token sequence. Reported by vlady (LTX Identity Transfer + Add Video
IC-LoRA Guide Advanced / LTX Director Guide).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The AdaLN modulation crashed on other ComfyUI builds (tensor 6912 vs 7680): timestep and
ref-token extension went out of lockstep because kwarg flattening / timestep granularity
differ across versions. Now: (1) measure-based timestep padding — detect per-token,
per-FRAME or already-extended from the actual length and pad by exactly what
process_input appended this forward; (2) read ref latents from flat kwargs OR nested
transformer_options; (3) unique _idma_* state reset every forward (no stale leakage, no
collision with the single-ref node); (4) _dbg tracing in every hook (debug_log=true).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Copy of LTX Identity Transfer generalized to N reference blocks: reference_face_front
(required, sid 2, drives ArcFace projector) + optional reference_back_head (sid 3),
reference_body_front (sid 4), reference_side_profile (sid 5). Empty optional inputs are
omitted, matching the multi-view training's view-dropout. Distinct patch flags so it
never collides with the single-ref node.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Empirically the AdaLN (CAN) modulation destroys the generation; the identity gain came from the
overlap + projector + LoRA, not the CAN. Reverted the Identity Transfer node to overlap + projector,
unregistered the standalone CAN node (file kept).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The CAN needs its own visible face-detection control; before it was implicit. Now a can_face_mode
dropdown, independent of the projector's arcface_mode.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
arcface_mode=disable was meant to turn OFF the weak projector, but it also skipped the CAN — and
the CAN REQUIRES the reference ArcFace (that's the identity it injects into the AdaLN). Now the CAN
forces auto_adjust when the projector mode is 'disable', so it still gets the face.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- The block-forward patch failed on LTXAV (v_context/ax kwargs). Instead add the CAN delta directly
into scale_shift_table (row 0 shift_msa, row 2 gate_msa) via ComfyUI's reversible add_object_patch
— mathematically identical ((sst+delta)+timestep), works on any LTX/LTXAV version, no forward
rewrite. Confirmed AV row order: slice(0,2)=shift/scale_msa, slice(2,3)=gate_msa.
- Integrated CAN into LTX Identity Transfer (overlap) node: new can_weights + can_strength inputs,
applied to the same model+reference (so one node does overlap + CAN). Standalone CAN node kept too.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The ArcFace projector (appended text tokens) has ~0 impact (cross-attn can ignore them). CAN
injects identity into the self-attn AdaLN (shift+gate) of even blocks — a channel the model can't
ignore. Trained on top of the reference LoRA it lifts identity a lot (humo 0.52 -> 0.69) with no
first-frame leak. Node rebuilds CANModulation per even block from identity_adapters_*.safetensors
(can.<i>.*), extracts the reference ArcFace, and patches each even block's native forward with
shift_msa += dshift; gate_msa += tanh(dgate) — the exact training formula. Lazy imports; defensive
__init__. Weight-load + CAN forward validated headless; block patch untested in ComfyUI.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ComfyUI's native LTX Gemma-3 tokenizer already accepts image= (inserts <image_soft_token> +
carries pixels) and encode_from_tokens runs the multimodal Gemma + projection + connector. So the
node is just clip.tokenize(prompt, image=ref) -> encode_from_tokens_scheduled -> CONDITIONING.
Everything from the already-loaded CLIP; no separate Gemma path, no AutoProcessor, no _get_encoder.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Feeds the reference image through the ALREADY-LOADED ComfyUI CLIP (LTX's own multimodal Gemma 3
vision) -> feature_extractor -> connector -> CONDITIONING. Identity is in Gemma's native space
(the reason foreign CLIP/DINO/ArcFace adapters fail on LTX); no reference latent (no copy-paste/
mask), no DiT surgery. Reuses the loaded CLIP (only loads a lightweight AutoProcessor for the
multimodal chat template). Mirrors ComfyUI-LTXVideo LTXVGemmaTextEncoderModel, run multimodally.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
_load_projector now rebuilds the projector MLP from the state_dict (proj.N Linears),
so it accepts both the original 4-token/2-layer projector and the enhanced
16-token/3-layer one. Bump 1.8.0 -> 1.8.1.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
BFSAmvGuideBuilder builds the keyframes+blank guide for the AMV av2av LoRA:
auto_scene / manual_indices / external_images, with audio-beat placement
(librosa beat_track, RMS-onset fallback) to sync cuts to the music. Adds
librosa to requirements.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pure v2v anime->real conditioning for the Bernini-R anime2real LoRA
(trained without reference). Mirrors training: guide -> source_id 1 on
both positive and negative, empty latent -> source_id 0, no source_id 2
and no amplify_reference (no reference to amplify; text-only CFG).
Reuses bernini_patches; registered in __init__.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds an `amplify_reference` option (default on): the head reference is attached to
the positive conditioning only while the guide stays on both, so ComfyUI's CFG
amplifies the identity (approximates Bernini's omega_img guidance). Use CFG ~3-5
and disable LightX2V. Bump to 1.5.0.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Standalone head-swap nodes for the Bernini-R LoRA (Wan2.2-A14B):
- HeadSwapBerniniConditioning: guide video -> source_id 1, head image ->
source_id 2; output follows the guide's native size, head keeps its own.
- HeadSwapLoRADebug: reports applied LoRA patch count on a MODEL.
Bundles the WanModel context_latents patch (vendored from ComfyUI-RH-Bernini,
GPL-3.0) so no external Bernini node package is required.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
