Relative (default, unchanged) = per-prompt steering. Absolute = a keyless,
prompt-agnostic "global taste" prior that pulls conditioning toward generally-
liked directions regardless of the prompt; Both layers them.
- New keyless Absolute store (FUNPACK_ABSOLUTE_KEY): pooled liked/disliked
direction + global value function, learned across every rated generation.
- Absolute rating semantics: a rating means "details I like in general" (not
prompt adherence) — reward-driven liked/bad-dir storage, axis attribution
bypassed. Perfect = "more of this everywhere".
- _v2_finalize_conditioning replaces the per-key ascend hook on both returns;
pure Absolute also bypasses per-prompt conditioning memory.
- Scene Chain sampler: embed_guidance_source (relative/absolute) lets the
per-step nudge draw from the global direction/value function.
- Studio popup + V2 node expose Steer mode + Absolute strength.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds s_churn/s_tmin/s_tmax to sample_funpack_distilled_flow: optionally
re-noise the latent within [s_tmin, s_tmax] before each eval, then denoise
from the bumped sigma_hat. No extra model calls (still one eval per step,
at a higher noise level). Fights stale/frozen distilled output (more motion)
and can sharpen by letting the model re-resolve. sigma_hat is clamped to the
schedule start so the model never sees a higher-than-trained noise level, the
final jump-to-zero detail step is never churned, and the default 0.5-1.0 band
protects the sub-0.4 detail finish per the empirical sigma findings.
Churn supersedes the legacy s_noise ancestral path while active and is
video-stream-only on packed LTXAV latents. With s_churn=0 the sampler is
byte-identical to the previous deterministic ODE. Wired into the node and
Studio's Distilled Flow config panel.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Port the Hybrid/RF velocity-bias capture/apply and rating-gated reactive
rescue into sample_funpack_distilled_flow, reusing the shared trajectory
memory helpers. Adds velocity_bias_mode/strength/source/key and
rescue_mode/threshold/strength to the FunPackDistilledFlowSampler node and
wires them into Studio's Distilled Flow config panel (same blank/'default'
velocity-key -> refinement-key fallback as Hybrid). Steering is
magnitude-preserving and, on packed LTXAV latents, video-stream only
(audio-safe). With all controls off the sampler is byte-identical to the
previous deterministic ODE.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Batch Training confirmed working end to end (controlled batch with transitions, distinct
per-variant seeds, node rating ignored during a batch, per-entry rating + VF/axis learning
on submit). Stripped the temporary trace prints (Batch Training check / Batch check / ACTUAL
scene seeds), keeping clean progress logs. Gitignore the local .testvenv.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 'seed=' in the batch log was the loop index seed (base+idx), not the seed that drives
noise. The real noise seed is each scene's funpack_scene_seed (set distinct per variant by
_v2_pack_batch). Print those so we can tell whether distinct seeds actually reach the
sampler vs. stale/old-code identical seeds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Prints (a) the batch_variants value + mode flags the refiner sees and how many packed
entries it produced, and (b) the positive entry count + variant markers + group count the
Chain Sampler sees. Pinpoints whether the break is the arm/settings (batch_variants==1 at
the refiner), the conditioning path (markers stripped before the sampler), or the split
logic. Temporary, to be removed once the cause is found.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per feedback: Batch Training is a complete UI thing driven by Studio (the hub), not a
prompt, and it must NOT start generation itself (ComfyUI's Generate does).
- Removed batch_iterations from the Chain Sampler entirely (input + param). The sampler
now enters batch mode ONLY when Studio packs variant markers into positive; the batch
count lives in Studio. _run_batch_training simplified to variant-entries-only.
- The node's 'Batch Training' button now opens a dedicated window (openBatchPanel) that
shows EITHER a config view (set batch size, Arm/Disarm) when no batch is pending, OR the
rating list (preview + per-entry rating + Submit/Forget) when a batch has finished. It
never queues — arming just sets refiner.batch_variants; the user presses Generate.
- Dynamic button label reflects live state: '(off)', '(armed ×N)', '(Processing…)',
'(Finished — rate)'. Driven by execution_start/success/error listeners, which also
one-shot disarm after a batch runs so the next Generate is normal.
The older in-Refiner-tab batch panel remains functional (secondary path); the node-button
window is now the primary entry point.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Producer (Studio owns conditioning, per the boundary law): refine_v2 gains batch_variants.
When >1, _v2_build_batch_variants packs N conditioning entries into the single positive wire,
each tagged 'funpack_batch_variant'. Variant 0 is the fully-processed base; variants 1..N-1
re-resolve shortcuts with seed+i (random option per entry) and get the same conditioning-memory
treatment, deduped by expansion text. No shortcut variability -> all entries resolve identical
and reuse the base cond -> the batch becomes seed-only (sampler varies the seed per entry).
Captures the pre-shortcut prompt to re-resolve. Not combined with split_by_transitions.
Trigger: a 'Batch Training' button on the Studio NODE (next to Open Studio) — prompts for N,
sets refiner.batch_variants, queues one prompt, then auto-reverts to 1 (one-shot). Studio is
the batch trigger (hub principle); no sampler input needed.
Sampler: now enters batch mode when positive carries variant markers, even if batch_iterations
is 1 (markers are the trigger; variant count wins). Each entry already saves its own cond, so
Submit trains the VF per variant — teaching which prompt variant rated better. Panel hints now
point at the button.
End-to-end: press Batch Training -> N -> Studio makes N variant conds -> sampler samples N ->
previews to temp -> rate in Studio panel -> Submit trains VF per entry -> batch cleared.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per the Studio/sampler boundary law (Studio owns conditioning; sampler only samples), the
sampler no longer encodes anything. It now auto-detects when Studio has packed N shortcut
VARIANTS into positive — each scene entry tagged with 'funpack_batch_variant' = variant
index — via _split_batch_variants: if present, it samples one chain per variant (variant
count wins); otherwise it falls back to seed-only repeats of batch_iterations. Decisions:
'sampler loops, Studio varies' + 'pack into positive + markers'.
Each entry now saves its OWN conditioning (<id>.cond.pt) — identical across a seed-only
batch, distinct per variant — and the manifest records variant index + prompt per item.
Submit (batch_training.py) trains the VF per entry from its own cond, so variant batches
teach the model which prompt variant rated better (not just seed-quality variance). Studio
panel shows the variant tag + prompt per row.
Studio-side variant PRODUCTION (resolve shortcuts N times, encode, pack markers) is the
remaining half — a Studio/refiner feature, tracked next. Sampler side is ready for it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 1 amended: the engine now also saves the frozen conditioning (cond.pt) per batch
so submit can train on (cond, rating).
Phase 2 (panel + API): new batch_training.py registers /funpack/batch/{list,submit,forget}
(follows the existing /funpack/* PromptServer pattern). Studio gains a 'Batch Training'
section under the refiner: lists the pending batch for the active refinement key with each
entry's filename + seed, a Preview button (loads the temp webp via /view?type=temp), and a
per-entry V2-rating dropdown, plus Submit and 'Forget batch'. Previews served straight from
ComfyUI temp — no custom file route needed.
Phase 3 (learning intake): Submit loads cond.pt and trains the value function on
(cond, reward) for every rated entry — same conditioning, N rewards = variance-reduction
RLHF, which directly sharpens conditioning ascent. '-Just forget it-'/unknown ratings are
skipped. Submit consumes (deletes) the batch; Forget discards it without learning. Either
way the panel clears and the sampler is ready for a new batch.
Human stays the rating oracle throughout (no auto-rating) — 'user over everything'.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New optional 'batch_iterations' on FunPackLTXAVSceneChainSampler. When >1, runs the
ENTIRE scene chain N times frozen except the seed (seed + i) — conditioning/settings
provably identical because each run reuses sample() itself as the per-generation
primitive with only the seed changed. This gives a clean controlled-batch RLHF signal
(same conditioning, only noise varies) instead of the confounded one-at-a-time loop
where conditioning drifts between rated gens.
Each iteration is persisted under ComfyUI's TEMP dir (funpack_batches/<key>/<stamp>/) so
artifacts are wiped on restart (ephemeral rating material) and are servable via
/view?type=temp for the upcoming Studio panel: latent (.latent.pt) + a downscaled
animated-webp preview + a batch.json manifest (seed, scene prompts, subfolder, rating
slots). LATENT output passes through the last iteration so the downstream decode/save
path is unchanged.
Batch semantics: an entry is the full set of provided scenes — scenes within an entry
chain with overlap as normal; separate entries are independent sample() calls with no
cross-entry carry. Requires refinement_key_input (degrades to a single run without it).
Phase 2 (Studio rating panel + /funpack/batch routes) and Phase 3 (batch-train intake
in the refiner) to follow.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three issues found tracing the full Studio path with all features on:
1) BUG (silent data loss): a custom velocity_refinement_key captured trajectories
under that key but the refiner committed the rating under the effective refinement
key — so custom velocity keys never learned (banks stayed empty, rescue/velocity a
permanent no-op). This undercut per-key cut-routing. Fix: Studio resolves each pass's
velocity key (same logic as _build_one_sampler) and passes them to refine_v2, which
now commits the verdict to the union of {effective key} + velocity keys (dedup).
Backward compatible (callers without _velocity_keys behave as before).
2) Effectiveness: VELOCITY_BIAS_TARGETS (0.90, 0.80) matched only sigma~0.909 on the
standard 8-step LTX schedule, so rescue fired once per gen at a near-noise step and
never at the detail-forming sigmas. Broadened to (0.90, 0.72, 0.42) -> hits 0.909,
0.725, 0.422. Velocity APPLY still structure-only (highest target); rescue now acts
where runs actually go good/bad. (New detail targets fill as gens are rated.)
3) Redundant value: velocity_aspect_bucket was a manual text field never tied to actual
resolution, and the bank key already includes the full latent shape (which encodes
resolution) — so it was doubly redundant. Removed from the key, all sampler plumbing,
INPUT_TYPES, Studio build, and the JS UI/defaults. Persistence loads old stores fine
(the 'aspect' field is simply ignored; key is now (refkey, target, shape)).
Verified: both py files + JS parse clean; all refactored call signatures aligned.
Tests require the ComfyUI venv (torch) to run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The real limiter wasn't the 0.35 slider cap but the internal fixed 4.5%-of-|x|
delta clamp, which saturated almost immediately — the upper slider range was a
no-op. Now the rotation cap scales with strength (0.30*strength, capped 0.95), so
the whole range is meaningful: ~0.15 subtle spice (matches old feel), 0.3-1.0 clear
action crossover, 2-3 approaches full action replacement without wiping the current
gen. Magnitude-preserving renorm still keeps |x|, so high strength rotates rather
than blows up. Raised the param/UI cap to 3.0 and updated the tooltip.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
eta_final was silently ignored whenever the quality pass was off: the decay was
anchored to quality_sigma_start, which collapses to sigma=0 when high_quality_pct=0,
so effective_eta stayed pinned at eta for every step. Combined with the missing
deterministic finisher, full-strength ancestral noise ran through the final detail
steps with nothing to resolve it -> noisy mess.
New _effective_eta() ramps eta -> eta_final across raw schedule progress
(sigma/sigmas[0]), so eta_final works with the quality pass on or off. Used in both
the RF and eps paths. No-op when eta_final == eta (default), so existing runs are
unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of consistently bad LTXAV audio: ComfyUI packs the video+audio
latents into one [B,1,N] tensor for sampling (pack_latents, video first), and
the RF sampler's ancestral re-noising (eta/s_noise) + motion pulses + velocity/
rescue steering sprayed the whole packed tensor — including the audio slice.
Ancestral re-noising every step destroys audio coherence (video tolerates it,
and in fact benefits from the scene variability).
Fix: recover comfy's per-stream latent_shapes from the guider conds, build a
[1,1,N] video-region mask, and confine every FunPack perturbation to the video
slice. Audio now rides the clean deterministic euler-RF step:
- ancestral noise branch restructured into x_det (deterministic, audio) vs x_anc
(full ancestral, video), combined via the mask.
- motion pulses, velocity-bias apply, rescue, and quality sharpness masked too.
Single-stream LTXV (mask None) is unaffected — behavior identical to before.
Only the CONST/RF path needs this (LTXAV is always CONST); eps path sees no AV.
Prints a one-line diagnostic when packed AV is detected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Compensate the slow quality degradation from velocity-bias 'apply' (additive
mean-pull) with four guards on the Hybrid Euler 2S sampler (both eps and
rectified-flow/LTXAV paths):
- B: apply is now magnitude-preserving — rotate x toward (x+delta) and
renormalize to |x| instead of x+delta, so no energy is injected.
- C: apply fires only at the highest (structure) velocity target and decays
strength with the sigma ratio; detail sigmas left untouched.
- E: new quality_sharpness knob — temporal-average unsharp on the x0 prediction
during the quality phase to restore high-frequency detail (free, no extra eval).
Default 0.0 (off).
- D: new velocity_bias_source = mean|nearest — 'nearest' steers toward the single
best-matching prompt cluster (one real good gen's detail) instead of the
washed-out average. Affects both apply and rescue. Default 'mean' (off).
Plumbed through get_sampler/INPUT_TYPES, Studio _build_one_sampler, and the
Studio JS sampler panel (both presets + UI controls).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The good/bad trajectory banks were in-memory only, so they died on every
ComfyUI restart — losing valuable rating-learned data. Persist them per key
under refinements/attn_maps/velocity_<hash>.pt (same store as blessed maps),
saved on each commit, loaded once-per-process on first access (rescue/apply/
commit). Directions stored as float16 to halve disk. Now only the deliberate
Studio Session-reset button (clear_velocity_bias_memory) deletes the store; a
restart preserves it. Serialized as weights_only-safe lists/dicts (no tuple
keys); load failures degrade gracefully to empty banks.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of "rescued toward an Awful gen": velocity capture was
rating-blind — it wrote every sampled trajectory straight into the one
"good" memory, so a fresh key whose only capture was an awful gen made that
awful run the sole steering target. Ratings (which live in the refiner) never
touched the bank.
Now:
- capture STAGES the current gen's trajectory (VELOCITY_BIAS_PENDING) instead
of committing it; staging runs whenever rescue is on (no separate capture
toggle needed).
- the refiner commits the previous gen's staged trajectory by its rating:
like/loved/nailed -> good bank, awful -> bad bank, everything else dropped.
- rescue steers TOWARD the good bank and AWAY from the bad bank (per prompt
cluster), firing when diverged from good OR aligned with bad beyond
threshold; no-op when neither bank has a prompt match.
- session reset / clear_refinement_data now clears good + bad + pending.
velocity-bias "apply" now also reads the rating-gated good bank. Tooltips and
Studio note updated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
VELOCITY_BIAS_MEMORY is an in-process dict, so reset_session (which only
removed on-disk enhancement files via clear_refinement_data) left stale
good-trajectory memory behind — rescue and velocity bias would keep steering
toward the previous session's content. Add clear_velocity_bias_memory(key)
in samplers.py (drops global averages + prompt clusters for that key) and
call it from clear_refinement_data. Key matches: refine_v2 and the velocity
key default both resolve to the same effective refinement key.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replaces the minimal RF rescue mirror with _sample_const_rf_full: an
RF-correct port of the hybrid sampler so its features actually run on CONST
models (LTXAV) instead of being swallowed by the euler_ancestral_RF fallback.
- early phase: ancestral RF euler (matches sample_euler_ancestral_RF) with
eta decay, AB2 order-2 denoised extrapolation, motion pulses
- quality phase: deterministic Heun corrector (RF-correct 2nd order; DPM++ 2S
is eps-only) with the progressive correction_blend schedule
- velocity-bias capture/apply + reactive rescue around every model eval
Active by default (matches hybrid defaults) per user request; changes the
LTXAV baseline away from plain euler-ancestral. to_d-based euler is identical
to the RF flow update, so early and quality phases stay consistent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of "no rescue output": LTXAV uses CONST (rectified-flow)
sampling, so the hybrid sampler short-circuited to comfy's euler_ancestral
(which routes to euler_ancestral_RF) and skipped the entire custom loop —
velocity bias, motion pulses, and rescue never ran for LTXAV.
Replace the CONST early-return with _sample_const_rf_with_rescue: a faithful
mirror of comfy.sample_euler_ancestral_RF (same noise_scale, downstep/renoise
math) with velocity-bias capture/apply and reactive rescue inserted around
the model eval. With velocity and rescue off it reproduces the stock RF
result step-for-step. Hoist param normalization above the CONST branch so
both paths share it; correct the startup diagnostic accordingly.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
When rescue_mode is on, print at sampler entry: resolved strength/threshold,
whether a prompt signature arrived, whether the model uses CONST sampling
(which bypasses the whole custom loop), and which sigma steps are eligible
velocity targets. Makes it unambiguous why rescue does or does not fire —
no message at all now means the option never reached the hybrid sampler.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The single velocity-target step (norm sigma ~0.9) lands inside the quality
phase on short distilled schedules (8 steps, high_quality_pct=0.35 ->
quality boundary at sigma 0.909, the only matching step). Capture and rescue
were coded only in the early/non-quality branch, so neither ever ran on the
default LTX schedule — no capture, no rescue, no log output. Hoist
velocity_target above the phase split and run capture+apply+rescue after the
quality-phase model eval too. Capture records raw denoised before rescue
edits it in both branches, so the good-trajectory memory stays clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rescue previously steered toward a prompt-blind average of all captured
good trajectories, risking false positives that fight a legitimately
different prompt. Now the velocity memory keeps per-prompt clusters: each
tagged with a pooled+normalized conditioning signature. Capture merges into
the nearest cluster (cosine >= 0.92) or starts a new one (cap 8/key, evict
lowest-count). Rescue builds its reference from clusters similar to the
current prompt (cosine >= 0.5, >= 2 matched captures), similarity-weighted;
with no confident prompt match it skips rather than steering to the global
mean. Studio pools its positive conditioning into a signature and threads it
through _build_samplers -> get_sampler -> sampler extra_options. Standalone
node (no conditioning) falls back to the prior global-average behavior.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Detects when a denoising trajectory diverges from the learned
good-trajectory (velocity-bias) memory and rotates the prediction
back toward it on that step only. Reactive, read-only on the memory,
audio-safe (direction-only, magnitude-preserved on the denoised latent).
New sampler widgets: rescue_mode / rescue_threshold / rescue_strength,
threaded through get_sampler extra_options. Shares velocity_refinement_key
and aspect bucket with velocity bias; no-op until that memory is captured.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
LTX packs all spatial/temporal dims into a single token [1, 1, C*T*H*W]
during sampling. Per-step latent steering requires hooking into the model
internals, not the k-diffusion callback. Removed: compress_latent(),
_load_latent_value_function(), latent VF training, motion floor callback,
latent_value_guidance toggle in Studio.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previously latent_vf_strength was 0 when no latent VF existed, making
motion floor a no-op. Now embed_guidance_strength is always passed so
motion floor applies at the correct strength regardless of latent VF state.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LTX samples in [B, seq, C] token format during sampling, not [B, C, T, H, W].
Use sequence variance as temporal variance proxy — works for both 3D and 5D.
Replace fixed 0.5 sigma threshold with 1-sigma scale so motion floor and
latent VF activate across LTX's distilled high-sigma schedule naturally.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
OUTPUT_NODE = True forces ComfyUI to run the node even with no downstream
connection on the LATENT output. Motion floor now activates as soon as
embed_guidance is on and a refinement key is set, independent of liked_dir.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
When temporal variance of the latent drops below 5% of overall variance,
applies a normalised deviation gradient that pushes frames apart from each
other. Fires only at sigma < 0.5 (same window as other latent guidance),
uses embed_guidance_strength as scale. Active whenever embed_guidance is on
and liked_dir is available — no user input needed.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Trains a second OnlineValueFunction on rated video latents (compressed via
channel mean-pool). At each denoising step, computes ∂reward/∂latent through
the lightweight MLP and applies it as an in-place drift to the noisy latent —
gradients never touch the 22B transformer. Controlled by embed_guidance_strength
alongside the existing conditioning guidance. Studio toggle: "Latent guidance".
Cleared on session reset alongside the conditioning VF.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Before each denoising run, when the value function is ready, gradient-ascend
the text conditioning until predicted reward plateaus (self-terminating: step
size = 0.5% of conditioning norm, stops when delta < 1e-3, caps at 50 steps).
No user-facing params — fully automated. Per-step wrapper still runs on top.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Model weights loaded via torch.load inside ComfyUI's inference_mode become
inference tensors and can't participate in backward. Wrapping the load in
inference_mode(False) ensures parameters are normal autograd tensors.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
MLP lives on CPU, conditioning on GPU — device mismatch silently fell back
to fixed direction. Now moves conditioning to MLP device for the forward/
backward, then returns gradient to original device.
Also surface value function fallback errors instead of swallowing them.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
value_function.py: OnlineValueFunction MLP that trains incrementally on
(conditioning, reward) pairs. Uses replay buffer of 100 samples, 20
gradient steps per update. save/load to .pt file via refinement_state_path.
Studio popup: new "Value guidance" toggle. When on, trains the value function
after every rated generation (good and bad). Falls back to no-op if conditioning
deserialization fails.
chain sampler: _load_value_function reads the trained MLP if ready (5+
samples). _build_embed_guidance_wrapper uses value function gradient
(∂reward/∂conditioning) when available, fixed liked direction as fallback.
Log reports which mode is active and sample count.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extends the Refiner's one-shot conditioning steering to apply at every
denoising step. Reads the liked conditioning direction from the Refiner's
state file via refinement_key_input. At each step, nudges c_crossattn toward
the learned liked direction with sigma-scaling (no effect at sigma=1, full
effect below sigma=0.5 where detail refinement happens).
Requires 3+ liked generations to have a meaningful direction. No-ops silently
if direction is unavailable.
New inputs: embed_guidance (BOOLEAN), embed_guidance_strength (FLOAT 0.005-0.1).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Below 0.25: audio degrades to noise (guide frame in latent confuses
cross-modal attention without enough video coherence), character appearance
drifts while background over-anchors to previous scene.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Empirical testing: 0.25 is the sweet spot — locks character position and
action pixel-perfect while allowing background to drift freely toward the
new scene's prompt. Above 0.35 causes character-space conflicts when scene
composition shifts significantly.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_prepend_soft_continuation creates the noise mask with ones_like(video_tensor)
giving it 128 channels instead of 1. Using torch.full_like(masks[0][:,:,:1])
copies the exact shape signature from the existing mask so the cat always works
regardless of channel count.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
guide_mask was [B,1,1,1,1] but noise_mask is [B,1,F,H,W] — cat on dim=2
requires matching H,W. Now uses masks[0].shape[3:] for spatial dims.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Previous approach (post-block hidden state injection) was fundamentally
broken for LTXAV: modifying video tokens after a block contaminates audio
through joint audio-video attention, causing audio noise and latent bleed.
New approach: append the middle frame of the previous scene as a guide
token to the END of the current chunk latent, with keyframe_idxs pointing
to F//2 of the current scene in pixel space, plus guide_attention_entries
at the specified strength. The model's existing _attention_with_guide_mask
mechanism attenuates attention weights inside the attention computation —
audio tokens never see modified video tokens.
After sampling, _crop_video_tail removes the guide frame.
New inputs: mid_scene_guide (BOOLEAN), mid_scene_guide_strength (FLOAT 0.1-0.8).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Old: average all tokens in middle third → one scalar direction → spread to
surrounding zones. Loses all spatial structure, can't preserve positioning.
New: take the exact spatial token map of the single middle frame (F//2),
inject it position-for-position into every other frame. Token at (h,w) in
the anchor frame maps to (h,w) in all other frames. Character layout and
positioning stays spatially consistent across the scene. Anchor frame
itself is left unchanged.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Re-encoding the next scene's conditioning with the previous scene's
last frame is useless for fade-to-black workflows — the last frame
is black, so it's conditioning on nothing. Removing the whole path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
