If the character description ends with . ! ? or , use a space separator
instead of injecting a redundant comma. Only bare text gets the ", " join.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Option 1 (samplers.py): Add optional clip input to FunPackLTXAVSceneChainSampler.
After each scene is sampled, decode its last frame and re-encode the next scene's
conditioning with that frame as vision context. This gives identical scene texts
different conditioning based on temporal position, so the model knows what state
it came from.
Option 2 (conditioning.py): When building scene conditionings, detect duplicate
scene texts and prepend "Returning to an earlier scene: " before encoding the
second+ occurrence. Stores original text in funpack_scene_text for logging and
the actual encode text in funpack_encode_text so the sampler uses the same prefix.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When frame_overlap=0, use the last 4 frames of the previous scene as soft
continuation guides (mask=0.4) instead of a single fully-pinned frame (mask=0.0).
This lets the model use previous-scene context for identity continuity while
retaining freedom to change orientation from the text prompt. The frames land
at positions 1-4 after the i2v guide at position 0 (when carry_i2v_guides is
enabled). crop count is returned dynamically so the existing crop/blend logic
requires no changes.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LTXAVTEModel.load_sd passes the full checkpoint sd to Gemma3_12B.load_state_dict,
which has vision_model and multi_modal_projector as real attributes with matching
key names. Vision weights are already present after DualCLIPLoader runs.
Drop _vision_loaded flag check; detect vision capability by attribute existence.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Checkpoints converted for ComfyUI use vision_model.* and
multi_modal_projector.* as top-level keys directly, not the HuggingFace
model.vision_tower.* nesting. Accept both formats.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When no vision keys are found, log the top-level key prefixes actually
present in the checkpoint so the user can see whether the vision tower
was stripped. Also rename all [FunPackGemmaLoader] log prefixes to
[FunPackGemmaVision].
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Companion to DualCLIPLoader: takes the already-built CLIP object (which has
the correct LTXAVTEModel structure) and loads the SigLIP vision tower +
multi-modal projector from the Gemma3-12B-IT checkpoint, setting
_vision_loaded=True. Workflow: DualCLIPLoader → FunPackGemmaVision → Studio.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FunPackGemmaLoader: drop-in replacement for Load CLIP Model that additionally
loads the SigLIP vision tower and multi-modal projector from Google's
Gemma3-12B-IT checkpoint into the already-initialized Gemma3_12B module,
enabling reference-image vision prompting without a separate CLIP Vision model.
Vision prompting: _v2_encode_prompt accepts reference_image; when the clip
has _vision_loaded=True, tokenizes with image=reference_image and
skip_template=False so Gemma3 sees the reference frame before generating
conditioning. Cache key includes image fingerprint to avoid stale hits.
source_image from Studio/refine_v2 flows through to _v2_conditioning_source.
K/V injection revamp (Technique 5): identity blocks in i2v mode now use
in-context conditioning instead of lerp injection. Pre-hook prepends reference
hidden-state tokens to vx and extends v_pe with neutral-rotation entries
(cos=1, sin=0) so video tokens self-attend to reference tokens in attn1.
Post-hook crops the reference prefix from the block output. Strength raised
from 0.02 → 1.0 (sigma-gated: active at 0.35–0.80). _extend_v_pe helper
handles 2-D/3-D/4-D PE tensors. Anchor blocks keep the existing lerp path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
update_refinement_sampler_context was loading the main V2 state file
(which contains base64-encoded conditioning tensors) during sampling.
This re-hydrated large conditioning objects in Python's heap mid-sample,
triggering Python's cyclic GC which detected internal reference cycles
in LTXAVTEModel_ and fired the "potential memory leak" warning.
Fix: write sampler context to a tiny separate file (sampler_ctx prefix,
~60 bytes). refine_v2 reads from that file instead of state["last_sampler_context"].
The main state file is never touched during sampling.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Chain sampler accepts optional refinement_key_input (wire from the same
key as the V2 Refiner). On each run it calls update_refinement_sampler_context
to write carry_i2v_guides, frame_overlap, and scene count into the shared
state file. The V2 Refiner reads last_sampler_context from state when it
loads, making it available for the gen_context snapshot with no latent
manipulation and no extra Refiner inputs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Chain sampler now embeds funpack_sampler_context into the output latent
(carry_i2v_guides, frame_overlap, transitions_enabled). V2 Refiner reads
these from its existing latent input — no new Refiner inputs needed.
Removed the three forceInput connectors added in the previous commit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Stores a gen_context snapshot in last_run each iteration: active LoRAs
(name/type/weight), carry_i2v_guides, frame_overlap, transitions_enabled,
and source image fingerprint. On the next run, diffs previous vs. current
context and emits targeted insights in training_info — e.g. "carry_i2v_guides
is now OFF — appearance drift is likely from missing i2v guides, not the
prompt itself" when rating is Wrong appearance. Three new optional forceInput
connectors on the V2 Refiner node: carry_i2v_guides, frame_overlap,
transitions_enabled — wire from your sampler.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Mirrors _v2_transition_scene_texts: the text before the first
transition is a character anchor prepended to every scene, not a scene
itself. Show it separately above the rail. scenes[] now matches what
the Scene Chain sampler actually generates.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extract _v2_split_prompt_by_transitions to module-level
split_prompt_by_transitions. parse_timeline_segments now calls it
directly with placement=silent — same code path as generation.
Eliminates all reimplemented splitting logic.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Empty string is falsy so the stacking guard was skipping the phantom
collapse when the first segment was empty. Remove the truthy check so
both empty and transition-only segments are collapsed correctly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Shortcut expansions that start with a built-in transition phrase create
a phantom segment with no real content, causing N+1 scenes. Apply the
same stacking collapse as _v2_split_prompt_by_transitions: merge the
phantom into the following segment and drop the spurious transition.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The pipeline is shortcuts → transitions → split. The route now runs
apply_prompt_shortcuts first, then parse_timeline_segments on the
expanded text, matching actual generation behavior.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
widgetByName returns nothing when the input is connected. Traverse the
graph link to the source node and read its first widget value instead.
Also fix custom_map always being empty when refinement_key is set.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Read-only horizontal timeline showing detected scenes as boxes with
transition connectors between them. Backed by a new
POST /funpack/parse_timeline route and parse_timeline_segments() in
conditioning.py that reuses the existing transition phrase patterns
and custom transition map.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add confirmed-bad-combo warnings to both parameter tooltips, the sampler
doc, and the changelog. Warn users not to combine frame_overlap=0 with
carry_i2v_guides=True in normal use.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
mask=0.5 caused the continuation frame to be partially denoised, distorting
it and triggering the same yellowing artifacts as the temporal position conflict.
mask=0.0 keeps the frame intact so the model uses it as a clean reference.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When no overlap frames are pinned, the next scene has zero context from
the previous one. Prepend the last latent frame from the previous output
with mask=0.5 so the model can use it as a soft continuation reference
without hard-pinning it. The frame is cropped from the head after sampling
and never appears in the output. Works alongside carry_i2v_guides.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Appending the guide (even offset to chunk-end) caused the model to generate
content converging toward the reference at the end of each scene. Scene 3's
overlap started from that converged tail and compounded into a full visual reset.
Prepending the guide places it at temporal position 0 naturally (causal patchifier
assigns it [0,1]). Overlap frames follow at [1,9], [9,17]... — no temporal slot
conflict, no forward-pulling toward the reference. The guide acts as a clean
i2v anchor at the start. Crop now removes from the head after sampling.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Guide tokens were anchored at pixel position 0 (causal_fix default), which is the same
temporal slot occupied by the overlap frames at the start of continuation chunks. The model
saw two different frame contents claiming position 0 and produced color artifacts. Offset
the guide to chunk_end + 1 latent step so it acts as a forward style anchor instead.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removed per-scene slicing (_decode_video/_decode_tensor) and the aggressive
del-before-decode GC block. Now: whole sampling loop runs to completion →
full latent decoded in one pass → transition pixel effects applied after.
LATENT output is unchanged. Per-scene decode was causing seam cross-contamination
anyway and contradicted the post-process-only contract for transitions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Transition "bad" appearance: blend_half_pixels=64 caused 2+ seconds of
forced full-black at every seam — the flat zone existed to hide latent
blend artifacts, but the blend is a no-op (overlap frames are identical
on both sides, mask=0). Removed blend_half entirely. Transitions are now
a simple V-ramp: full effect at the seam center, linear ramp back to
normal over transition_duration//2 pixel frames on each side.
Memory leak: scene_cond (Python loop variable) stays alive after
del scene_conditionings — still holds the conditioning tuple whose
embedding tensor keeps LTXAVTEModel referenced during VAE decode.
Added scene_cond to the del statement.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_append_i2v_guides only extends the VIDEO tensor by 'protected' frames.
The audio tensor is never touched, so sampled audio after sampling is
already the correct length. Cropping it proportionally removed audio
frames that were never added, making audio progressively shorter across
scenes and playing back faster.
Main's version (crop video only) was always correct.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_match_audio_amplitude mutated the audio tensor of sampled before
_blend_latents, putting rescaled audio into the overlap zone that
becomes pinned context for the next scene. LTXAV generates video and
audio jointly — artificially rescaled audio context degrades video
generation quality, causing progressive burnout across scenes.
Also removed dead code: _has_i2v_reference_latent (defined, never
called), _build_bridge_chunk_UNUSED, and want_latent (unused after
transition-path cleanup). Sampling loop now matches main exactly
except for boundary_entries metadata tracking (read-only).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_crop_video_tail was only cropping the video tensor, leaving audio at its
full (guide-appended) length. _blend_latents then computed audio overlap
from the wrong video/audio ratio — too many audio frames removed per seam —
so audio drifted progressively shorter than video and played back faster
with each additional scene.
Fix: crop all non-video tensors (audio) proportionally using _derived_overlap,
the same ratio logic used everywhere else in the blend path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The NestedTensor fix was based on a wrong assumption — workflow separates
video and audio latents before decoding each independently. Reverted.
Actual cause of gradual burnout: decoding the full concatenated multi-scene
latent in one VAE pass lets temporal attention spread seam artifacts forward
across scenes. Each additional scene adds another seam, compounding the damage.
Fix: decode each scene's latent slice independently using the known boundary
positions from boundary_entries, then concatenate pixel frames. Transitions
are applied on the concatenated pixels as before. The VAE never sees across
a scene boundary.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_decode_video was extracting only the video tensor before calling vae.decode(),
stripping the audio component. LTXAV's video decoder uses audio as temporal
conditioning — without it, decode quality degrades progressively across scenes
(causal: later frames suffer most) and audio timing drifts.
Standard ComfyUI decode passes output["samples"] directly (full NestedTensor).
Match that behavior. Tiled path still extracts video tensor since decode_tiled
likely doesn't support NestedTensor, falling back to full decode on failure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Was added to hide yellow seam artifacts, but those are gone now that
transitions run as pure post-process on a full-video decode. Blur
effect is blur only, no fade to black.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Transitions apply in pixel space after all sampling is complete.
LATENT output is always the raw generated latent, untouched.
IMAGE output decodes the finished latent and applies pixel effects directly.
No re-encode path exists — removed _apply_transitions_latent entirely.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
blur_out_in blurred center frames but never dimmed them — the yellow decode
artifact at the seam was fully visible through the blur. Added the same
brightness ramp as fade_to_black so center dims to 0; blur and fade now
happen together so the seam is both blurry and invisible.
_apply_transitions_latent was decoding individual windows around each
boundary and re-encoding them back into the latent. Window edges lack full
temporal context, so the VAE encoder introduced color artifacts. Replaced
with full-video decode → pixel transition (same as the IMAGE path) →
full-video re-encode, giving the VAE correct context everywhere.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Transition center was placed at cumulative_latent_frames - overlap//2,
inside scene 1's territory. Since overlap frames are pinned (mask=0)
the latent blend is a no-op, so the actual content seam is at
cumulative_latent_frames exactly. Centering there puts the seam at
peak black, hiding any decode artifacts and giving a clean post-processed
fade-to-black feel on each side.
Maps stacking: _capture_finalizer merged new captures with the existing
temp file, including leftovers from previous ComfyUI runs. Each "loved it"
cycle blessed maps that had been tainted by the previous round's injection,
compounding color bias indefinitely. Clear the temp file at build_enhancements
start so each generation captures clean.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
del model/positive/negative was not enough — the loop leaves
scene_conditionings (slice of positive), scene_positive, scene_negative,
chunk, and sampled alive after the loop. All hold conditioning tuples
whose embedding tensors keep the text encoder referenced. Delete all of
them before VAE decode so ComfyUI can unload the TE immediately.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Revert anchor source back to blended cumulative output — prev_sampled
gave scene N+2 a tail dominated by scene N-1 pinned frames, weakening
the scene-N representation and making the i2v guide's reference pull
visually dominant (apparent reset to reference image). Blended anchor
frames are already hidden by the coverage fix so no yellow leaks.
Fix blur edge distortion: avg_pool2d with zero-padding darkened image
edges at high sigma. Now pre-pad with reflect mode and pool with
padding=0, eliminating the dark halo at frame boundaries.
Fix effect curve shape: previous linear V-fade left blended-latent
content visible at ~89% brightness at the blend zone edges, giving a
"generated transition frames" look. New shape holds brightness=0 /
sigma=max flat across the entire blend zone, then ramps back to full
over transition_duration//2 extra frames into clean scene content.
The blend zone is now fully hidden regardless of transition_duration.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_build_continuation_chunk was reading from the blended cumulative
output whose tail (for 3+ scenes) contains interpolated latent frames
from earlier overlap zones. These decode with color artifacts and were
polluting the anchor context for later scenes.
Store prev_sampled per iteration and use it as the anchor source.
The blended output is still used for the final assembled video and
for cumulative_latent_frames accounting; only the anchor source
changes. Transitions remain purely post-process.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- transition_duration=0 now fully skips boundary recording and all
effect application (previously min was 2 and effects always ran)
- Pixel path: half now covers the full blend zone (blend_half_pixels =
video_overlap//2 * time_scale) plus transition_duration//2 extra
frames, so blended latent artifacts are no longer visible outside
the fade region
- Latent path: half_lat now covers blend_half_latent + extra_lat so
the re-encode window includes the full overlap zone; half_pix uses
the full decoded window so the VAE round-trip frames are all faded
- blend_half_latent/blend_half_pixels stored per boundary entry so
each boundary uses the correct overlap geometry
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
i2v reference frames have noise_mask≈0.3 but were receiving full
creativity noise perturbation, shifting their latent representation
and causing the VAE to decode a yellow-tinted anchor frame.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Delete model/positive/negative references before calling vae.decode() so
ComfyUI's model manager can unload the diffusion model and text encoder
cleanly. Previously they stayed live as function parameters, causing the
circular-reference warning.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Detect which outputs are connected via the ComfyUI prompt graph
- IMAGE connected: full decode → pixel transitions → IMAGE (no re-encode)
- LATENT only: window decode → effect → re-encode → splice (original approach)
- Both: pixel path for IMAGE; LATENT is the raw combined output unchanged
- Add decode_tile_size optional input for OOM control
- Split logic into _apply_effect_on_pixels / _apply_transitions_pixel /
_apply_transitions_latent / _decode_video for clarity
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Custom transition triggers in Studio can now have a visual_effect assigned
(fade_to_black, crossfade, blur_out_in). When triggered during scene splitting,
the effect fires at that scene boundary: the sampler decodes the latent window
around the boundary, applies the pixel-space effect, and re-encodes in-place.
No new nodes needed — VAE is already connected to the scene chain sampler.
New transition_duration input controls how many pixel frames are affected.
New scene_boundaries output provides the boundary list for debugging.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Two causes of 3-4x slowdown:
1. Hooks registered for blessed maps capture even when reference_injection
was disabled. Now hooks only register when has_i2v is True.
2. Hidden state tensors moved CPU on capture and GPU on inject -
for 150-frame LTXAV the tensors are large. Tensors now stay on the
model device throughout.
Also fixes memory leak: _block_list captured strongly in wrapper closure
stored inside model.model_options created a cycle:
model → model_options → wrapper → _block_list → model
Changed to weakref.ref(_block_list) - dereferenced lazily inside
_register_hooks at call time, breaking the cycle.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Attempted both approaches:
1. Bridge to reference image - visible reset, character snaps back to reference
2. Two-pass: generate scenes independently, bridge between actual scene frames -
still resets to reference because each independently generated scene
starts from the i2v reference frame (it IS the first frame)
Root cause: LTX-2's first-last-frame conditioning anchors outputs to the
pinned frames too strongly. When the "last frame" target is the reference
image (as the start of any scene), the bridge always transitions TO that
reference, producing the exact reset effect we were trying to avoid.
Would require scenes generated without any reference anchor for the
bridge target to be a natural scene composition - which breaks character
consistency. No clean solution without IP-Adapter or equivalent.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
