Guide tokens were anchored at pixel position 0 (causal_fix default), which is the same
temporal slot occupied by the overlap frames at the start of continuation chunks. The model
saw two different frame contents claiming position 0 and produced color artifacts. Offset
the guide to chunk_end + 1 latent step so it acts as a forward style anchor instead.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removed per-scene slicing (_decode_video/_decode_tensor) and the aggressive
del-before-decode GC block. Now: whole sampling loop runs to completion →
full latent decoded in one pass → transition pixel effects applied after.
LATENT output is unchanged. Per-scene decode was causing seam cross-contamination
anyway and contradicted the post-process-only contract for transitions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Transition "bad" appearance: blend_half_pixels=64 caused 2+ seconds of
forced full-black at every seam — the flat zone existed to hide latent
blend artifacts, but the blend is a no-op (overlap frames are identical
on both sides, mask=0). Removed blend_half entirely. Transitions are now
a simple V-ramp: full effect at the seam center, linear ramp back to
normal over transition_duration//2 pixel frames on each side.
Memory leak: scene_cond (Python loop variable) stays alive after
del scene_conditionings — still holds the conditioning tuple whose
embedding tensor keeps LTXAVTEModel referenced during VAE decode.
Added scene_cond to the del statement.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_append_i2v_guides only extends the VIDEO tensor by 'protected' frames.
The audio tensor is never touched, so sampled audio after sampling is
already the correct length. Cropping it proportionally removed audio
frames that were never added, making audio progressively shorter across
scenes and playing back faster.
Main's version (crop video only) was always correct.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_match_audio_amplitude mutated the audio tensor of sampled before
_blend_latents, putting rescaled audio into the overlap zone that
becomes pinned context for the next scene. LTXAV generates video and
audio jointly — artificially rescaled audio context degrades video
generation quality, causing progressive burnout across scenes.
Also removed dead code: _has_i2v_reference_latent (defined, never
called), _build_bridge_chunk_UNUSED, and want_latent (unused after
transition-path cleanup). Sampling loop now matches main exactly
except for boundary_entries metadata tracking (read-only).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_crop_video_tail was only cropping the video tensor, leaving audio at its
full (guide-appended) length. _blend_latents then computed audio overlap
from the wrong video/audio ratio — too many audio frames removed per seam —
so audio drifted progressively shorter than video and played back faster
with each additional scene.
Fix: crop all non-video tensors (audio) proportionally using _derived_overlap,
the same ratio logic used everywhere else in the blend path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The NestedTensor fix was based on a wrong assumption — workflow separates
video and audio latents before decoding each independently. Reverted.
Actual cause of gradual burnout: decoding the full concatenated multi-scene
latent in one VAE pass lets temporal attention spread seam artifacts forward
across scenes. Each additional scene adds another seam, compounding the damage.
Fix: decode each scene's latent slice independently using the known boundary
positions from boundary_entries, then concatenate pixel frames. Transitions
are applied on the concatenated pixels as before. The VAE never sees across
a scene boundary.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_decode_video was extracting only the video tensor before calling vae.decode(),
stripping the audio component. LTXAV's video decoder uses audio as temporal
conditioning — without it, decode quality degrades progressively across scenes
(causal: later frames suffer most) and audio timing drifts.
Standard ComfyUI decode passes output["samples"] directly (full NestedTensor).
Match that behavior. Tiled path still extracts video tensor since decode_tiled
likely doesn't support NestedTensor, falling back to full decode on failure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Was added to hide yellow seam artifacts, but those are gone now that
transitions run as pure post-process on a full-video decode. Blur
effect is blur only, no fade to black.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Transitions apply in pixel space after all sampling is complete.
LATENT output is always the raw generated latent, untouched.
IMAGE output decodes the finished latent and applies pixel effects directly.
No re-encode path exists — removed _apply_transitions_latent entirely.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
blur_out_in blurred center frames but never dimmed them — the yellow decode
artifact at the seam was fully visible through the blur. Added the same
brightness ramp as fade_to_black so center dims to 0; blur and fade now
happen together so the seam is both blurry and invisible.
_apply_transitions_latent was decoding individual windows around each
boundary and re-encoding them back into the latent. Window edges lack full
temporal context, so the VAE encoder introduced color artifacts. Replaced
with full-video decode → pixel transition (same as the IMAGE path) →
full-video re-encode, giving the VAE correct context everywhere.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Transition center was placed at cumulative_latent_frames - overlap//2,
inside scene 1's territory. Since overlap frames are pinned (mask=0)
the latent blend is a no-op, so the actual content seam is at
cumulative_latent_frames exactly. Centering there puts the seam at
peak black, hiding any decode artifacts and giving a clean post-processed
fade-to-black feel on each side.
Maps stacking: _capture_finalizer merged new captures with the existing
temp file, including leftovers from previous ComfyUI runs. Each "loved it"
cycle blessed maps that had been tainted by the previous round's injection,
compounding color bias indefinitely. Clear the temp file at build_enhancements
start so each generation captures clean.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
del model/positive/negative was not enough — the loop leaves
scene_conditionings (slice of positive), scene_positive, scene_negative,
chunk, and sampled alive after the loop. All hold conditioning tuples
whose embedding tensors keep the text encoder referenced. Delete all of
them before VAE decode so ComfyUI can unload the TE immediately.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Revert anchor source back to blended cumulative output — prev_sampled
gave scene N+2 a tail dominated by scene N-1 pinned frames, weakening
the scene-N representation and making the i2v guide's reference pull
visually dominant (apparent reset to reference image). Blended anchor
frames are already hidden by the coverage fix so no yellow leaks.
Fix blur edge distortion: avg_pool2d with zero-padding darkened image
edges at high sigma. Now pre-pad with reflect mode and pool with
padding=0, eliminating the dark halo at frame boundaries.
Fix effect curve shape: previous linear V-fade left blended-latent
content visible at ~89% brightness at the blend zone edges, giving a
"generated transition frames" look. New shape holds brightness=0 /
sigma=max flat across the entire blend zone, then ramps back to full
over transition_duration//2 extra frames into clean scene content.
The blend zone is now fully hidden regardless of transition_duration.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_build_continuation_chunk was reading from the blended cumulative
output whose tail (for 3+ scenes) contains interpolated latent frames
from earlier overlap zones. These decode with color artifacts and were
polluting the anchor context for later scenes.
Store prev_sampled per iteration and use it as the anchor source.
The blended output is still used for the final assembled video and
for cumulative_latent_frames accounting; only the anchor source
changes. Transitions remain purely post-process.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- transition_duration=0 now fully skips boundary recording and all
effect application (previously min was 2 and effects always ran)
- Pixel path: half now covers the full blend zone (blend_half_pixels =
video_overlap//2 * time_scale) plus transition_duration//2 extra
frames, so blended latent artifacts are no longer visible outside
the fade region
- Latent path: half_lat now covers blend_half_latent + extra_lat so
the re-encode window includes the full overlap zone; half_pix uses
the full decoded window so the VAE round-trip frames are all faded
- blend_half_latent/blend_half_pixels stored per boundary entry so
each boundary uses the correct overlap geometry
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
i2v reference frames have noise_mask≈0.3 but were receiving full
creativity noise perturbation, shifting their latent representation
and causing the VAE to decode a yellow-tinted anchor frame.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Delete model/positive/negative references before calling vae.decode() so
ComfyUI's model manager can unload the diffusion model and text encoder
cleanly. Previously they stayed live as function parameters, causing the
circular-reference warning.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Detect which outputs are connected via the ComfyUI prompt graph
- IMAGE connected: full decode → pixel transitions → IMAGE (no re-encode)
- LATENT only: window decode → effect → re-encode → splice (original approach)
- Both: pixel path for IMAGE; LATENT is the raw combined output unchanged
- Add decode_tile_size optional input for OOM control
- Split logic into _apply_effect_on_pixels / _apply_transitions_pixel /
_apply_transitions_latent / _decode_video for clarity
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Custom transition triggers in Studio can now have a visual_effect assigned
(fade_to_black, crossfade, blur_out_in). When triggered during scene splitting,
the effect fires at that scene boundary: the sampler decodes the latent window
around the boundary, applies the pixel-space effect, and re-encodes in-place.
No new nodes needed — VAE is already connected to the scene chain sampler.
New transition_duration input controls how many pixel frames are affected.
New scene_boundaries output provides the boundary list for debugging.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two causes of 3-4x slowdown:
1. Hooks registered for blessed maps capture even when reference_injection
was disabled. Now hooks only register when has_i2v is True.
2. Hidden state tensors moved CPU on capture and GPU on inject -
for 150-frame LTXAV the tensors are large. Tensors now stay on the
model device throughout.
Also fixes memory leak: _block_list captured strongly in wrapper closure
stored inside model.model_options created a cycle:
model → model_options → wrapper → _block_list → model
Changed to weakref.ref(_block_list) - dereferenced lazily inside
_register_hooks at call time, breaking the cycle.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Attempted both approaches:
1. Bridge to reference image - visible reset, character snaps back to reference
2. Two-pass: generate scenes independently, bridge between actual scene frames -
still resets to reference because each independently generated scene
starts from the i2v reference frame (it IS the first frame)
Root cause: LTX-2's first-last-frame conditioning anchors outputs to the
pinned frames too strongly. When the "last frame" target is the reference
image (as the start of any scene), the bridge always transitions TO that
reference, producing the exact reset effect we were trying to avoid.
Would require scenes generated without any reference anchor for the
bridge target to be a natural scene composition - which breaks character
consistency. No clean solution without IP-Adapter or equivalent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previous implementation bridged scene N → reference image, which caused
a visible "reset to reference" rather than a smooth transition.
Correct approach (matches LTX Desktop Bridge Shots):
1. Generate ALL scenes independently from latent_template
2. For each consecutive pair, generate a bridge:
- First frame = last frame of scene N
- Last frame = first frame of scene N+1
- Middle = denoised by the model
3. Assemble: scene1 + bridge + scene2 + bridge + scene3 + ...
The model now transitions between where scene N actually ended and where
scene N+1 actually begins, not to a static reference reset.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New boolean input: bridge_shots (default False). When enabled, generates
a short transition clip between scenes by pinning both ends:
- First frame = last frame of the previous scene (where we came from)
- Last frame = first protected frame of the i2v latent (reference image)
- Middle = denoised by the model with the scene's conditioning
Duration: 30 video frames (~1 second at 30fps). Only fires when an i2v
reference is detected in the latent template (noise_mask present with
first frame mask < 0.95). Falls back to standard overlap if no i2v
reference is connected.
Status line reports bridge count. Scene report lists bridge seeds.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Hooks were registered upfront and called on every block, every step -
even when returning None immediately. For 150-frame LTXAV with large
hidden state tensors, Python hook dispatch overhead accumulated.
New approach: hooks are only registered when sigma first drops below
0.95 (the active zone). For the 8-step distilled schedule this means
steps 1-5 have ZERO hook overhead. On scene transition (sigma resets),
hooks are unregistered for a clean state, then re-registered when sigma
enters the active zone again.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two causes of 3-4x slowdown:
1. Hooks registered for blessed maps capture even when reference_injection
is disabled. Now hooks only register when has_i2v is True.
2. Hidden state tensors were moved CPU on capture and GPU on inject -
for 150-frame LTXAV the tensors are large (~100MB per block).
Tensors now stay on the model's device throughout, eliminating the
CPU<->GPU round trip overhead.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removed: 13 extra diagnostic hooks (18→5), scene transition tracking,
per-scene capture buffers, cross-scene similarity reporting, _prev_sigma,
_scene_snapped, _scene_cur_ref/_sca refs in hook remover.
Back to 5 hooks on IDENTITY_BLOCKS only. Diagnostic data collected,
analysis complete - no longer needed in production path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The i2v hidden state injection only produces meaningful results alongside
carry_i2v_guides in the Scene Chain Sampler. Without it the injection
has no reliable reference to anchor from (proven through testing).
New Studio Refiner setting: "Reference injection" toggle (default off).
Hint explains it should be enabled together with carry_i2v_guides.
Passing reference_latent=None to build_enhancements when disabled so
the entire injection path is skipped cleanly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Data from 5-scene run showed:
- b14:0.609, b33:0.581, b34:0.551, b30:0.542, b21:0.538 (most consistent)
- b26:0.403, b27:0.437 (least consistent - scene-specific, removed)
New IDENTITY_BLOCKS = [14, 21, 30, 33, 34] replacing [21, 26, 31].
Also fixes phantom scene count: only save a scene snapshot on transition
if the previous scene was properly snapped (filters model init passes
that triggered false scene transitions, overcounting by 1).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Performance: hooks now only capture when sigma in [0.85, 0.95] - one
step per scene instead of every step. Eliminates the 2x overhead and
5x slowdown on step 6.
Report timing: fires in sigma tracker when sigma < 0.5 AND scene is
snapped AND ≥2 scenes captured. Covers the last sigma=0.421875 step
which was below the old hook_remover threshold of 0.05.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Detects scene transitions via sigma resets (sigma jumping back up = new
scene). At mid-sigma in each scene, snapshots ALL candidate blocks
(14, 19, 20-35). After the run completes, computes pairwise cosine
similarity across scenes and prints ranked results:
Cross-scene block consistency (3 scenes):
b27:0.89 b31:0.84 b24:0.81 ... b14:0.21
-> Candidate identity blocks (sim>=0.70): [24, 27, 29, 31]
-> Scene-specific blocks (sim<=0.40): [14, 19, 20]
Blocks with high similarity = consistent character features across
different scene compositions. Use these as actual injection targets.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Registers capture-only hooks on ALL candidate blocks (14, 19, 20-35).
After each run, saves their hidden states at sigma~0.91 to a diagnostic
file. On the second run, compares per-block cosine similarity against the
first run and prints ranked results:
Block similarity (high=character-consistent, low=scene-specific):
b27:0.89 b31:0.84 b24:0.81 b19:0.43 b14:0.21 ...
Blocks with high similarity across different scenes of the same character
are the true identity blocks. Blocks with low similarity are scene-specific.
Requires refinement_key to be set. Runs silently (no output) on first
generation, reports on second.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
With schedules like [1.0, 0.99, 0.98, ..., 0.725, 0.421, 0.0] the old
snapshot threshold of sigma<0.6 fired on the last step, leaving zero
steps for injection. Changed to sigma<0.95 to snapshot after the
high-noise plateau, giving injection during the two steps that actually
form character identity (sigma 0.725 and 0.421875).
Widened identity injection gate from (0.25, 0.42) to (0.35, 0.80) to
cover those steps. Strength stays at 2%.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
4% was still causing spatial content from reference to ghost into new
scenes. Dropped to 2% and narrowed the injection window to 0.25-0.42
(late identity formation, after coarse composition is already set),
reducing scene composition interference from the reference.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Anchor blocks (14, 19) encode scene composition/structure - injecting
from the reference frame bled the entire reference scene into new
generations. Disabled anchor injection for i2v entirely.
Identity block strength reduced from 10% to 4% - hidden states encode
the full scene context, not just character features, so even a light
touch at 10% was too strong. 4% applies a gentle bias toward the
reference character without overriding scene-specific content.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
patches_replace["dit"] never fires for LTXAV - both capture and inject
must use forward hooks. Each hook now handles both:
- Capture: always writes block output to capture_buf (for blessed maps)
- Inject: when lazy_inject is populated, lerps the block output toward
the reference tensor at the appropriate sigma-gated strength, then
returns the modified output so the model sees the adjusted hidden state
Reference snapshot moved from step 1 (sigma_max, very noisy) to first
time sigma drops below 0.6, giving cleaner reference maps for fine
details like character features. Snapshot only fires once len(cap_buf)
covers all injection blocks.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LTXAV transformer blocks receive transformer_options correctly but never
check patches_replace["dit"], so capture/inject via that mechanism is a
no-op. Switch to nn.Module forward hooks registered directly on the
transformer blocks.
Finds transformer block list via model.model.diffusion_model using common
attribute names (transformer_blocks, blocks, joint_blocks, layers).
Handles dict/tuple/tensor output formats from blocks.
Hooks are removed via a cleanup wrapper once sigma drops below 0.05.
Prints "Registered N forward hooks" if successful, warns if block list
not found.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two bugs:
1. capture_buf was None when refinement_key was empty, so block replacements
never captured anything and the lazy inject snapshot check always failed
silently. capture_buf is now always created when has_i2v is True.
2. _model_ref=model was left in the sigma tracker closure from a previous
refactor - a direct model reference sitting inside model.model_options,
causing the "potential memory leak" warning. Removed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Running apply_fn with a different x tensor causes shape mismatches because
the conditioning in args["c"] was prepared for the actual generation's
input shape, not our reference tensor.
New approach: no separate forward pass. The block capture already runs
during every denoising step. After the first actual step completes,
snapshot capture_buf into the lazy inject objects. From step 2 onwards
injection is active with the step-1 reference maps.
This is shape-safe, model-type agnostic, and requires zero extra inference.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
ModelPatcherDynamic (dynamic VRAM loading) has no apply_model method.
Use apply_fn from the wrapper instead - it's the base model function
and works with all patcher types.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LTX i2v sets the image frame mask to (1 - strength), e.g. 0.3 at
strength=0.7. Our threshold of < 0.1 was too tight and missed all
real-world i2v latents. Changed to < 0.95 - any frame below 1.0
indicates a reference image is present.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The previous approach called model.apply_model with empty conditioning {},
which fails silently for LTX2 (text conditioning required). Reference maps
were never actually populated.
New approach:
- _LazyInject holder: blocks receive a placeholder that starts empty and gets
populated on the first denoising step. No injection until maps are ready.
- _run_reference_extraction: called from _sigma_tracker on the first model
call, where args["c"] contains the real conditioning. Temporarily installs
capture patches, calls model.apply_model at sigma=0.03 with actual
conditioning, collects hidden states, populates lazy inject objects, then
restores all patches.
- Generation proceeds normally from step 1 with injection active.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When an i2v reference is connected, old blessed maps from prior runs
(which may represent a completely different character) must not inject.
If reference map extraction fails, injection is simply skipped rather
than falling back to stale appearance data.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When an i2v latent is connected (noise_mask with protected first frame),
FunPack now automatically extracts hidden state maps from the reference
image at blocks 14, 19, 21, 26, 31 via a single forward pass at sigma=0.03.
These maps are injected during generation to anchor character appearance
without requiring any prior rated runs.
Priority: reference maps from i2v > blessed maps from prior rated run.
T2v latents (no noise_mask or mask=1 throughout) are detected and skipped
cleanly - no injection happens and generation is unaffected.
Reference map injection uses fixed moderate strength (anchor 22%, identity
10%) since there is no reward signal to scale against. Blessed maps continue
to scale with reward as before.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
