If the character description ends with . ! ? or , use a space separator
instead of injecting a redundant comma. Only bare text gets the ", " join.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Option 1 (samplers.py): Add optional clip input to FunPackLTXAVSceneChainSampler.
After each scene is sampled, decode its last frame and re-encode the next scene's
conditioning with that frame as vision context. This gives identical scene texts
different conditioning based on temporal position, so the model knows what state
it came from.
Option 2 (conditioning.py): When building scene conditionings, detect duplicate
scene texts and prepend "Returning to an earlier scene: " before encoding the
second+ occurrence. Stores original text in funpack_scene_text for logging and
the actual encode text in funpack_encode_text so the sampler uses the same prefix.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LTXAVTEModel.load_sd passes the full checkpoint sd to Gemma3_12B.load_state_dict,
which has vision_model and multi_modal_projector as real attributes with matching
key names. Vision weights are already present after DualCLIPLoader runs.
Drop _vision_loaded flag check; detect vision capability by attribute existence.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
FunPackGemmaLoader: drop-in replacement for Load CLIP Model that additionally
loads the SigLIP vision tower and multi-modal projector from Google's
Gemma3-12B-IT checkpoint into the already-initialized Gemma3_12B module,
enabling reference-image vision prompting without a separate CLIP Vision model.
Vision prompting: _v2_encode_prompt accepts reference_image; when the clip
has _vision_loaded=True, tokenizes with image=reference_image and
skip_template=False so Gemma3 sees the reference frame before generating
conditioning. Cache key includes image fingerprint to avoid stale hits.
source_image from Studio/refine_v2 flows through to _v2_conditioning_source.
K/V injection revamp (Technique 5): identity blocks in i2v mode now use
in-context conditioning instead of lerp injection. Pre-hook prepends reference
hidden-state tokens to vx and extends v_pe with neutral-rotation entries
(cos=1, sin=0) so video tokens self-attend to reference tokens in attn1.
Post-hook crops the reference prefix from the block output. Strength raised
from 0.02 → 1.0 (sigma-gated: active at 0.35–0.80). _extend_v_pe helper
handles 2-D/3-D/4-D PE tensors. Anchor blocks keep the existing lerp path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
update_refinement_sampler_context was loading the main V2 state file
(which contains base64-encoded conditioning tensors) during sampling.
This re-hydrated large conditioning objects in Python's heap mid-sample,
triggering Python's cyclic GC which detected internal reference cycles
in LTXAVTEModel_ and fired the "potential memory leak" warning.
Fix: write sampler context to a tiny separate file (sampler_ctx prefix,
~60 bytes). refine_v2 reads from that file instead of state["last_sampler_context"].
The main state file is never touched during sampling.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Chain sampler accepts optional refinement_key_input (wire from the same
key as the V2 Refiner). On each run it calls update_refinement_sampler_context
to write carry_i2v_guides, frame_overlap, and scene count into the shared
state file. The V2 Refiner reads last_sampler_context from state when it
loads, making it available for the gen_context snapshot with no latent
manipulation and no extra Refiner inputs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Chain sampler now embeds funpack_sampler_context into the output latent
(carry_i2v_guides, frame_overlap, transitions_enabled). V2 Refiner reads
these from its existing latent input — no new Refiner inputs needed.
Removed the three forceInput connectors added in the previous commit.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Stores a gen_context snapshot in last_run each iteration: active LoRAs
(name/type/weight), carry_i2v_guides, frame_overlap, transitions_enabled,
and source image fingerprint. On the next run, diffs previous vs. current
context and emits targeted insights in training_info — e.g. "carry_i2v_guides
is now OFF — appearance drift is likely from missing i2v guides, not the
prompt itself" when rating is Wrong appearance. Three new optional forceInput
connectors on the V2 Refiner node: carry_i2v_guides, frame_overlap,
transitions_enabled — wire from your sampler.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Mirrors _v2_transition_scene_texts: the text before the first
transition is a character anchor prepended to every scene, not a scene
itself. Show it separately above the rail. scenes[] now matches what
the Scene Chain sampler actually generates.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extract _v2_split_prompt_by_transitions to module-level
split_prompt_by_transitions. parse_timeline_segments now calls it
directly with placement=silent — same code path as generation.
Eliminates all reimplemented splitting logic.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Empty string is falsy so the stacking guard was skipping the phantom
collapse when the first segment was empty. Remove the truthy check so
both empty and transition-only segments are collapsed correctly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Shortcut expansions that start with a built-in transition phrase create
a phantom segment with no real content, causing N+1 scenes. Apply the
same stacking collapse as _v2_split_prompt_by_transitions: merge the
phantom into the following segment and drop the spurious transition.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Read-only horizontal timeline showing detected scenes as boxes with
transition connectors between them. Backed by a new
POST /funpack/parse_timeline route and parse_timeline_segments() in
conditioning.py that reuses the existing transition phrase patterns
and custom transition map.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Custom transition triggers in Studio can now have a visual_effect assigned
(fade_to_black, crossfade, blur_out_in). When triggered during scene splitting,
the effect fires at that scene boundary: the sampler decodes the latent window
around the boundary, applies the pixel-space effect, and re-encodes in-place.
No new nodes needed — VAE is already connected to the scene chain sampler.
New transition_duration input controls how many pixel frames are affected.
New scene_boundaries output provides the boundary list for debugging.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The i2v hidden state injection only produces meaningful results alongside
carry_i2v_guides in the Scene Chain Sampler. Without it the injection
has no reliable reference to anchor from (proven through testing).
New Studio Refiner setting: "Reference injection" toggle (default off).
Hint explains it should be enabled together with carry_i2v_guides.
Passing reference_latent=None to build_enhancements when disabled so
the entire injection path is skipped cleanly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When an i2v latent is connected (noise_mask with protected first frame),
FunPack now automatically extracts hidden state maps from the reference
image at blocks 14, 19, 21, 26, 31 via a single forward pass at sigma=0.03.
These maps are injected during generation to anchor character appearance
without requiring any prior rated runs.
Priority: reference maps from i2v > blessed maps from prior rated run.
T2v latents (no noise_mask or mask=1 throughout) are detected and skipped
cleanly - no injection happens and generation is unaffected.
Reference map injection uses fixed moderate strength (anchor 22%, identity
10%) since there is no reward signal to scale against. Blessed maps continue
to scale with reward as before.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New placement option 'silent': the transition phrase marks the scene
boundary but is stripped from the output entirely. Neither the previous
nor the next segment contains it. Available globally and as a per-entry
override on custom transitions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Trailing \b after a non-word character (like '.') never matches since
\b requires a word char on one side. Custom triggers ending with
punctuation now use (?=\s|$) as the end boundary instead, so
"Scene cut." correctly splits prompts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Global setting in Studio Refiner tab: "Transition placement" (start/end).
start = transition opens the new segment (existing default behavior).
end = transition closes the previous segment instead.
- Per-entry placement override in Transitions tab (global/start/end).
Custom transitions can pin their placement regardless of the global setting.
- transition items gain a placement field (global/start/end, defaults to global).
- load_custom_transition_triggers returns {trigger_lower: placement|None} so
the split function can do per-trigger lookups efficiently.
- refine_v2 and Studio.run() wired with split_transition_placement parameter.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The [a-z][\w-]* alternative matched any word after "scene", turning
"scene proceeds", "scene features", "scene shows" etc. into false
transition split points. Removed the over-broad alternative - the
pattern now only matches numbered scenes (scene 1, scene -2) and
"scene minus <label>" forms.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Single words like "next", "suddenly", "later", "finally" etc. fire on
normal prompt text and cause spurious scene splits. Removed the entire
single-word temporal markers block. Shortest triggers are now two words
minimum. Also removes "and suddenly" and "and next" which were similarly
too vague.
Stray article-comma cleanup ("the, next" -> "the next") also applied when
building transition segments to handle commas left by the split.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When the stacking fix merges a pure-transition segment into the next one,
it now uses a space instead of ", " if the segment ends with an article
(a/an/the). This prevents "the scene then cuts to the, next view" when
"next" is also a transition word and the segments get merged mid-phrase.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
If a prompt ends with a transition phrase plus nothing (or just a lone
article like "the"), the last segment is now dropped rather than kept
as a near-empty scene. Covers cases like "...K/DA Kai'Sa, the scene
then cuts to the" where the user did not complete the transition.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Shortcuts: allow empty string as a valid replacement to remove matched phrases
from prompts; post-pass cleans up orphaned commas/spaces left by removals
- Shortcuts: add cross-shortcut longer-phrase-wins test confirming ordered
alternation handles multiple shortcuts with overlapping triggers correctly
- Transitions: new user-editable transition word list stored in transitions.json;
custom triggers extend the built-in split pattern at runtime, keyed by name
- Transitions: Studio JS gets a dedicated Transitions tab (name, trigger, enabled)
with Add/Refresh/Import/Export toolbar
- Transitions: four API routes mirroring shortcuts (GET, export, save/delete, import)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
With split_by_transitions outputting a single conditioning, the contrastive
conditioning math has nothing to act on. Removed from Refiner V2 inputs,
refine_v2 signature, Studio JSON settings, Studio popup UI.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Multi-entry conditioning without a working per-window routing mechanism
accumulates all scene conditionings onto every frame, which confuses the
model. The model already places scene cuts naturally from transition words
in the full prompt - splitting the conditioning breaks this.
split_by_transitions now:
- Detects scene segments and shows them in encoded_prompts (transparency)
- Outputs a SINGLE full-prompt conditioning (model uses natural cuts)
- Status shows "Transition split: N scenes detected"
- encoded_prompts shows "Detected scenes" for use with FunPack Scene Noise
FunPackSceneCutWindows still works independently: it reads num_scenes from
conditioning entries, so with single conditioning it stays inactive (safe).
FunPackSceneNoise still seeds noise per temporal region from the seed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Anchor fix: use the ENTIRE first segment (before the first transition word)
as the character anchor for all windows. Previous code split at the first
comma, which broke prompts like "In this anime video, Character is a..."
by using "In this anime video" as the anchor and losing the character desc
from all subsequent windows.
i2v fix: in FunPackSceneCutWindows, scene N>0 tokens now have access to
scene 0's tokens as key/value (in addition to their own scene). The i2v
guide frame lives in the early temporal positions (scene 0 region), so
blocking it from later scenes caused the reference image to be ignored.
Scenes 1 and 2+ remain isolated from each other.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
New phrases: 'the scene begins with', 'after a brief scene cut',
'as the scene transitions', 'in the next scene', 'later another scene
shows', 'another scene shows', 'the video concludes'.
Stacking fix: when two transition phrases are adjacent ('after a brief
scene cut, in the next scene...'), the pure-transition intermediate
segment is prepended to the following segment rather than left as an
empty conditioning fragment. Fixed direction (was appending to previous,
now prepends to next).
Diagnostic log: context_transition.py now logs cond group sizes,
num_windows, detected dim, and x_in shape at INFO level on every
should_use_context call to confirm routing is working.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
conds = [uncond_list, cond_list] - conds[0] is the negative conditioning
which always has exactly 1 entry. Reading len(conds[0]) always gave
num_windows=1, so the schedule produced a single window covering all
frames, and split_conds_to_windows picked the middle scene's conditioning
for the entire video (center_ratio≈0.5 → region 1 of 3).
Fix: max(len(c) for c in conds) correctly reads the positive conditioning
(3 entries for 3 scenes) and creates 3 isolated windows.
Also added 'the final scene' and 'the last scene' to transition phrases.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
For each context window, apply:
window_i = general + contrast × (scene_i − general)
where general = the full-prompt conditioning (refined, with memory
applied). The character description cancels in the delta because it
appears in both scene_i and general; the push focuses on what is
unique to each scene (action, environment).
- contrast=1.0 → plain scene encoding (original behaviour)
- contrast=1.5 → default, 50% extra push toward scene content
- contrast=2.0 → classic contrastive (2×scene − general)
New parameter: transition_contrast (FLOAT, default 1.5, range 1.0-4.0)
on Refiner V2 and in Studio popup Refiner tab.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Transition phrase list: add 'the scene then cuts to', 'the scene cuts to',
'the scene transitions to', 'the scene fades to', 'the scene dissolves to',
and short forms ('the scene then cuts', 'the scene cuts').
encoded_prompts output: when split_by_transitions is active, append a
'Per-window prompts' section showing the exact text encoded for each
context window. Example:
Window 1: A woman in red, walking in a park
Window 2: A woman in red, then sitting on a bench
Window 3: A woman in red, suddenly she runs away
Also updated _v2_encode_per_window_conditionings to return (conditionings,
window_texts) tuple instead of conditionings only.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Transition words ('then', 'suddenly', 'cut to', etc.) are now kept at
the START of the new scene's conditioning text rather than discarded.
They are valuable tokens that signal a scene change to the model.
Example: 'A woman in red, walking, then sitting on a bench, suddenly runs'
Window 0: 'A woman in red, walking'
Window 1: 'A woman in red, then sitting on a bench'
Window 2: 'A woman in red, suddenly runs'
Also added ordinal scene markers to the detection list:
'the first scene' ... 'the tenth scene'
'scene one' ... 'scene ten'
This lets users explicitly label scenes in the prompt.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All configuration is now auto-computed at sampling time - no user-facing
settings except the optional total_frames INT. Window count is derived
from the number of conditioning entries Refiner/Studio outputs. Window
size is total_frames (or actual latent size) / num_windows. Boundaries
are placed at equal intervals. split_conds_to_windows always on.
Studio: move split_by_transitions from direct node input to the Refiner
tab in the popup editor. Removed direct input, added toggle + hint text.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Studio now exposes split_by_transitions as an optional BOOLEAN
input on the node face, giving direct control without editing
the raw settings JSON. The direct input takes precedence over
the popup JSON setting when connected.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Refiner V2 and Studio: new split_by_transitions option detects
transition words in the final prompt (then, suddenly, cut to, etc.),
extracts the character description (text before the first comma),
and encodes one conditioning per scene window with the character
description prepended to each segment. Returns a multi-entry
conditioning list that routes per-window via split_conds_to_windows.
Context Transition Windows: new total_frames optional input.
When > 0 and split_conds_to_windows is enabled, context_length
is auto-computed at sampling time as total_frames / num_cond_entries
so windows size themselves to match Refiner's segment count.
Also fixes pre-existing bugs: early error return in refine_v2 was
missing video_latent (7th return value), and test unpacking was stale.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Three bugs causing i2v latent to be lost:
1. eff_key and prev_reward were defined inside 'if patched_model is not
None' but used outside it. NameError was silently caught but could
cause unexpected behavior. Moved both definitions before the block.
2. load_and_apply_creativity_mask returned bare {'samples': ...} dropping
all other keys from the original latent (noise_mask, type, batch_index,
etc.). Now copies the full original dict and only replaces 'samples'.
3. Modified samples were left on CPU (detached earlier) while the original
latent samples may be on CUDA. Now moves modified tensor to match the
original device and dtype before returning.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
refine_v2: initialise video_latent = latent instead of None so the connected
input passes through even when no key is set or mask computation is skipped.
Mask result only overwrites when it returns something non-None.
load_and_apply_creativity_mask: track explicit_latent separately from the
disk-loaded source. All early-exit and error paths return explicit_latent
(the connected input unchanged) instead of None, preserving the i2v or
previous-generation latent through the node when mask has nothing to add.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sits between Perfect (1.0) and Missing details (0.35). For when the output
looks great and you'd keep it, but it took creative liberty with the prompt
(man grabbed woman's hand instead of the other way - still a hand grab, still
loved, but not word-for-word correct).
Behaviour vs Perfect:
- axis_feedback: same - no missing/wrong axes, repair cleared
- good_streak: increments (same as Perfect)
- liked_dir: updated (quality direction was good)
- liked_conditioning average: NOT updated (intent drift - only Perfect locks in)
- phrase memory: delta 0.35x vs Perfect's 0.55x; category weights +0.08 vs +0.13
- preferred context: base_delta 0.65 vs 1.0
- phrase forgiven_count: -0.10 vs -0.18
- LoRA weights: quality/style +0.44x, action/character +0.24x (vs Perfect's 0.55x flat)
- Attention anchors: blessed (composition was good regardless of prompt fidelity)
- Advisor: not treated as perfect execution - repair signal still possible
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
clear_refinement_data() removes temp attention maps, blessed attention
maps, and creativity latent for the refinement key. Called from
_v2_load_state() at the start of a reset before fresh state is written.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
