Three live-run findings from the rental:
1. Block steering said "profile not ready (needs 2+ rated runs)" with 16
paired runs. The v1 liked/disliked-EMA attribution had no contrast when
ratings cluster on one pole (pole-vs-all-runs-mean cancels to zero) and
dropped rewards in (0, 0.3) entirely. Attribution is now the per-block
COVARIANCE between reward and activity over a rolling 32-run history:
every admissible reward contributes with its actual value, contrast
comes from reward variance, and the sampler log now states the real
blocker ("all rated alike - rate both good AND bad runs") instead of a
misleading run count. v1 profiles are discarded on first pairing.
2. output_guidance stayed "not ready (needs 10+)" forever on LTX-AV: the
end-of-run x0 snapshot was gated on output samples being a plain
torch.Tensor, but AV chain output is a NESTED tensor (video+audio) - so
the output value function never received a single sample. The snapshot
now extracts the video stream from nested outputs.
3. FunPackNVFP4ModelLoader hit a size-mismatch wall (9-vs-6 ada params,
connector dims): it loaded the state dict without the safetensors
METADATA that carries the LTX-AV architecture config, so detection
built the wrong model variant. Metadata now flows through, mirroring
comfy.sd.load_diffusion_model.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RLHF at the model level: which REGIONS of the transformer were doing the
work when a generation came out good or bad, and steer accordingly.
1. Measure: tagged forward hooks record each block's relative video-residual
delta per model call (near-zero cost; covered by the existing leak-strip).
2. Attribute: the end-of-run fingerprint is staged (block_snapshot.pt, same
stage-then-pair convention as x0_snapshot) and paired with the rating
reward on the next Refiner cycle into liked/disliked/all-runs EMA
profiles; when output_guidance's value function is trained, an innermost
model-function wrapper additionally credits blocks per step by
(value delta) x (centered activity) - converges in a handful of runs.
3. Steer: vx' = vx_in + g_b*(vx_out - vx_in) on the video stream only,
audio untouched; scores zero-mean/unit-max so emphasis is redistributed
rather than inflated, gains hard-capped at +/-10%. Exact no-op until the
profile has 2+ paired runs; blocks at gain 1.0 get no hook at all.
Chain Sampler toggle + strength (append-only widgets), exposed same-change
in the editor's Chain Sampler Guidance section via the sampler_inputs
bridge. New sidecars ride the defensive <key>.* sweep in
delete_refinement_key, so atomic key deletion covers them.
Tests: toy AV-block model driving the real hooks (recorder ranking, steer
video-only scaling + native path for unit gains, credit wrapper, snapshot/
profile/scores/gains roundtrip, mild-positive band, depth-mismatch reset,
leak-strip tagging). Full suite failure set identical to the pre-existing
baseline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The free video-only unsharp-on-x0 sharpening (temporal high-pass between
consecutive denoised predictions, no extra model eval) only existed on
Hybrid Euler 2S. Distilled Flow — the recommended sampler for this
pipeline (deterministic, clean audio) — had no equivalent, so the
"quality phase" of the actual production sampler had no sharpening path
at all.
Ports the same _apply_quality_sharpness call into
sample_funpack_distilled_flow, gated to the existing final-correction
(Heun) steps. Threaded through FunPackDistilledFlowSampler's
INPUT_TYPES/get_sampler, Studio's _build_one_sampler, and the Editor's
sampler_panel.js Distilled Flow section — verified live end-to-end
(UI -> studio_settings JSON -> backend build call).
New LatentValueFunction (value_function.py) mirrors OnlineValueFunction but is
trained on a pooled snapshot of the sampler's own predicted output (x0_hat)
instead of the input conditioning, borrowing NoiseTilt's core idea (reward
gradient from the model's own prediction) without its SDE noise-term trick,
which has no analogue in this deterministic-ODE sampler.
New output_guidance / output_guidance_strength toggle on the Chain Sampler
(off by default): per-step correction installed as the outermost
model_function_wrapper (on top of embed_guidance/score_slider), same
near-zero-forward-pass shape as embed_guidance. End of run writes a small
pooled snapshot per refinement key; the next rating cycle trains the value
function on it via a new call site in conditioning.py.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The 0.5 ceiling was only on the widget bounds (samplers.py INPUT_TYPES +
the editor's Chain Sampler knob); the apply path (_append_joyai_memory_guides)
only enforces the 0.25 audio-safe FLOOR, no upper clamp, so higher values
already flow through. Raise max to 10.0 in both widgets so identity can be
pushed past the audio-safe band. Floor (0.25) kept — below it audio degrades
and identity drifts. Tooltip notes 0.25-0.5 is the audio-safe band; higher
pushes identity harder but may degrade audio / over-constrain motion.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On compute capability > (9, 0) GPUs (Blackwell sm_120, e.g. RTX PRO 6000),
xformers' memory_efficient_attention has no kernel that accepts a tensor
attn_bias — fa3F/cutlassF require cap <= (9,0), fa2F/fa3F reject a tensor
bias. So unmasked attention works (i2v ANCHOR scenes generate) but MASKED
attention crashes: scenes generated from GUIDES hit the LTX guide path
(comfy/ldm/lightricks/model.py _attention_with_guide_mask) which passes a
tensor mask → "No operator found for memory_efficient_attention_forward".
This is the ComfyUI-wide issue normally worked around with
--disable-xformers --use-pytorch-cross-attention; when that flag isn't in
effect on an instance, the guide path still reaches xformers.
Fix: _funpack_install_mask_safe_attention(model), called at the top of the
chain sampler's sample(). When the active backend is xformers AND the device
capability is > (9,0), install ComfyUI's per-call optimized_attention_override
(the wrap_attn hook) so MASKED attention falls back to attention_pytorch
(SDPA handles masks on every GPU) while UNMASKED attention stays on the fast
xformers path. Threaded via transformer_options so it reaches every scene's
forward. No global monkeypatch; no recursion (wrap_attn stamps
_inside_attn_wrapper before calling the override). No-op on supported GPUs,
non-xformers backends, or when another override is already installed;
idempotent.
Tests: tests/test_mask_safe_attention.py — 9 cases covering override routing
(masked→pytorch via kwarg or positional, unmasked→original) and install
gating (xformers+Blackwell installs; supported cap / non-xformers / no-CUDA
skip; existing override respected; idempotent). Run with the ComfyUI venv.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
It was silently installing or no-op'ing with zero console output,
unlike ALG and Momentum Guidance which both confirm activation. Added
the same style confirmation line on successful hook install, plus a
distinct skip reason for each no-op path (no sentence split detected,
bad latent shape, narrow frame, missing transformer blocks) so it's
possible to tell whether it's running at all without reading code.
Structured Diffusion Guidance (arXiv:2212.05032)-style upgrade: instead
of tokenizing each sentence-half just to COUNT tokens (an approximate
boundary within one shared tokenization, vulnerable to BPE merge drift
at the seam), encode each half SEPARATELY through the real text encoder
and concatenate the two conditioning tensors. The split point is now
exact by construction, and the two spans can never share a token even
before any masking is applied. Same toggle, same metadata field
(funpack_bound_split_tokens) the sampler already reads — no sampler
changes needed.
MultiDiffusion (the other planned "same family" technique) needs
multiple forward passes per region (~2x+ cost) and is skipped per the
project's low/no-overhead bar, pending explicit user sign-off.
Live testing found decay=0.9 produces garbage at any strength on our
8-step schedule: the EMA barely forgets, so it stays anchored to the
very first (near-pure-noise, wrong-scale) step's direction for nearly
the whole run. decay=0.5 forgets that stale step within 2-3 iterations
and tested clean even at strength=1.0. Paper's longer-schedule decay
assumptions don't transfer to a few-step distilled run.
Masks text cross-attention (attn2 only, never audio or self-attention)
so the left half of the frame can only attend to subject-1's prompt
tokens and the right half only to subject-2's, fighting attribute/
anatomy bleed between two figures in one frame. Pure attention masking
via PyTorch forward-pre-hooks, no extra forward pass, model-level so it
works on any sampler.
Studio always tags each scene with an approximate token-split boundary
(sentence-count heuristic + real tokenization via clip.tokenize) as
conditioning metadata — cheap, no-op unless the new
bounded_attention_enabled toggle on the Scene Chain Sampler is on.
Opt-in, off by default.
Keeps a running EMA of the per-step ODE direction and blends the current
step toward it once sigma drops below mg_sigma_threshold — the
complementary window to ALG's anchor blur, which only acts above its own
threshold. EMA accumulates every step (free) so there's no cold-start
when application kicks in; video-only, audio untouched. Opt-in, off by
default.
Paper default strength=2.5 visibly changed character appearance, even
without JoyAI-Echo compensating. 2.0 holds i2v consistency noticeably
better while still breaking the static shortcut, confirmed independently
twice. 0.975 threshold (narrower blur window) paired with it in the
same tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
min=1.01 offset the whole step grid (1.01, 1.11, 1.21, ... 2.51, ...)
so every value the spinner snapped to ended in .01. Min is now a clean
1.0 -- kappa=1.0 is already just a harmless no-op blur, never needed
the 1.01 floor.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Generalizes the i2v-anchor blur (_alg_blur_video_frame0 -> _alg_blur_frames)
to optionally also blur newly-appended guide-attention frames each scene
(mid_scene_guide / carry_i2v_guides-as-guide / configured guides / JoyAI
memory), reusing the existing guide_tail bookkeeping rather than a mask-value
heuristic (frame_overlap continuation also uses mask=0 and must stay
untouched). New Scene Chain Sampler toggle "Blur i2v guides and JoyAI memory"
mutates the shared KSAMPLER's extra_options per-chunk since guide_tail varies
per scene; always resets so nothing leaks across chunks/samplers. Surfaced in
Engine Settings via the existing Experimental section, no backend/builder
changes needed beyond the new INPUT_TYPES field.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adaptive Low-Pass Guidance (arXiv:2506.08456): blurs the i2v anchor frame
while sigma is in the near-pure-noise region, then swaps back to the sharp
anchor, to stop the model shortcutting to a near-static video that just
matches the reference image. Opt-in, off by default, no-op without an i2v
anchor. Swaps model.latent_image per step (the same attribute ComfyUI's
inpaint compositing already re-pastes every step) instead of monkeypatching
comfy internals.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
build_enhancements registered forward/forward-pre hooks directly on the
shared diffusion_model blocks but only removed them on scene-transition
sigma jumps, never at end-of-sampling. Single-scene runs (and the final
scene of multi-scene runs) thus left hooks installed; because ModelPatcher
clones share the underlying blocks, every subsequent generation stacked
another set, each lerp-ing block output toward a previous run's stale
reference tensor -> progressive corruption that survived Session Reset and
deleting refinements/ (the leak is in-process, not on disk) and only
cleared on a full restart.
Tag every installed hook (_FUNPACK_HOOK_TAG) and add strip_funpack_block_hooks
to pop only tagged entries from each block's hook stores (idempotent, leaves
foreign hooks intact). Strip at the top of build_enhancements and defensively
at FunPackLTXAVSceneChainSampler.sample() so accumulation is bounded to zero.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Extend the JoyAI cross-shot memory bank from video-only to paired audio-video,
the part that distinguishes JoyAI-Echo from a generic video-consistency trick.
- _JoyAIMemoryBank now stores (video, audio) pairs under the same pin/roll policy;
frames()/audio() stay 1:1. Audio half is None until joyai_audio_memory is on, so
the phase-1 video path is byte-identical.
- _append_joyai_audio_memory pins each banked prior-shot audio latent into the
chunk's audio stream as a fully-protected (mask=0) tail, cropped back off after
sampling (_crop_audio_tail). The audio analogue of the video guide bank; rides
native audio self-attention since LTX has no audio guide-attention API.
- _harvest_joyai_audio harvests the paired audio frame, position-matched to video.
- v2a_grad_scale scales the model's trained video->audio cross-attention via
reversible forward hooks on each AV block's video_to_audio_attn submodule
(JoyAI uses 2.0). 1.0 installs no hook -> native behavior, zero overhead.
- joyai_audio_memory + v2a_grad_scale wired through sample(), batch training,
refinement context, run_mechanisms, INPUT_TYPES (append-only), and editor knobs.
Deliberately breaks the audio pass-through invariant, so both are opt-in and
off/native by default. 3 new bank tests; zero regressions (same 12 pre-existing
stale-mock failures before/after).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The inputs were inserted mid-list (after mid_scene_guide_strength). The builder seeds
sampler widget values from the reference workflow's POSITIONAL widgets_values array
(extract_widgets maps it onto INPUT_TYPES order), so inserting widgets shifted every
following widget — landing a numeric on the joyai_frame_select combo ("0 not in
['center','first','random']") and corrupting the seeded defaults for embed_guidance /
score_slider / decode_*. Same class of bug the existing forceInput-ordering comment warns about.
Append-only fix: joyai widgets now sit at the END of the widget sequence (after
embed_guidance_source, before the forceInput sockets). They fall beyond the reference
array so extract_widgets leaves the object_info default ('center') in place, and all
pre-existing widgets realign. No logic change; behavior identical once on.
Requires a ComfyUI restart to reload object_info.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Diagnostic conclusion from the JoyAI-Echo engine: conditioning is fed to the sampler
unchanged (gemma+connector, which Studio already produces), so the ONLY missing sampler
feature is generate_BidirectionalMemory's cross-shot paired AV memory bank. This lands the
VIDEO half: a managed set of clean prior-shot frames injected into each scene so identity
carries across the whole chain — JoyAI's story-level consistency.
Generalizes the single-anchor mid_scene_guide to N frames, reusing the same LTX
guide-attention primitive (_append_guide_latent). _JoyAIMemoryBank holds the policy
(num_fix opening shots pinned as a global anchor + rolling most-recent window capped at
max_size, matching configs/inference.yaml: max_size 7, num_fix_frames 3, center select).
Harvest one clean frame per finished scene; inject the bank before each continuation scene;
supersedes mid_scene_guide when on. Honors the 0.25 audio-safe strength floor.
Off by default, so the audio protect/pass-through invariant is unchanged. Paired audio
memory + v2a_grad_scale coupling is the next phase (needs the ltx_core v2a term).
Inputs: joyai_memory, joyai_memory_size, joyai_fix_frames, joyai_frame_select,
joyai_memory_strength. Threaded through batch-training; logged in run_mechanisms + the
refinement sampler-context. New test_joyai_memory.py covers the bank policy (6 tests).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The score_slider synthesized a symmetric +/-liked pair, so its axis was
"liked vs anti-liked mirror" — meaningless as an away-from-bad signal. When
3+ disliked/awful gens have populated the existing bad_dir slot in the same
taste store, build the slider's minus pole from that real disliked direction
instead. The (eps_+ - eps_-) axis becomes good-vs-bad, actively steering the
noise prediction away from the conditioning region that produced rated-bad
gens. Falls back to the symmetric mirror when no bad_dir exists.
Reuses existing plumbing: bad_dir is already accumulated identically to
liked_dir (running-averaged unit-norm delta from session mean). Generalized
the direction loader to _load_taste_direction(slot); load + pass bad_dir at
the call site; log/run-mechanism note which pole is active.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Training-free concept sliders (arxiv 2511.00103) in noise-prediction space,
sourced from the learned taste (user's choice). New score_slider + strength(eta)
inputs on the Scene Chain sampler, additive/non-disruptive, reusing the same
learned liked direction + embed_guidance_source + refinement_key_input.
Mechanism: a per-scene model_function_wrapper synthesizes a +/- concept pair from
the base conditioning and the liked direction (no concept prompts), runs 3 forward
passes on quality-phase steps and combines eps_mod = eps_base + eta*(eps_+ - eps_-).
High-sigma steps stay single-pass = the paper's k-step base-only warmup, keeping the
2x cost off early steps. Video-only: +/- conds are audio-protected and the eps blend
is masked to the packed video stream. Composes with embed_guidance (its nudge cancels
in the +/- difference). Installed/torn down per scene in reverse order; threaded
through batch-training re-dispatch. Fully guarded -> falls back to base prediction.
Tests: tests/test_score_slider.py (5) — combine, warmup, eta=0 no-op, eta linearity.
Pre-existing 16 suite failures unchanged (stale-mock/splitter set); zero new.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Inversion-free adaptation of 'When to Lock Attention' (arxiv 2603.09657).
Track x0 prediction instability per step (relative step-change of the predicted
clean latent, video-masked = a cheap proxy for the paper's variance-of-x0
hallucination metric over a sliding window). High instability => the model is
still drifting => raise the shared kvlock_scale so BachVid holds the blessed
identity K/V harder; as x0 settles the multiplier relaxes toward base strength.
No DDIM inversion (LTXAV inverts poorly) — locks against the blessed K/V bank.
No-op unless a BachVid bank installed the shared scale list, so the two compose
cleanly. Wired into all three FunPack sampler loops after each model() call;
fully guarded, cannot break sampling. Tests cover capture/inject/scale + the
schedule's stable-vs-jumpy behavior and the no-bank no-op path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Normalization is now available in both Hybrid Euler 2S and Distilled Flow
samplers, making the standalone Normalizing sampler redundant. Removes the
node class, its sampler function, two private helpers (_parse_norm_factors,
_apply_av_norm_factors), the Studio/Movie Editor "Normalizing" type option,
and the defaultNormalizing() factory. Stale "same as the Normalizing sampler"
tooltip references in both sampler nodes are updated.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Flush pending global prompt edits before save/generate so runs use the latest timeline text and anchors. Improve live preview wiring detection, progress polling, overlay stacking, and diagnostics when a VAE is configured.
Wire an optional fast preview VAE to show per-scene animated previews in the program monitor during generation.
Co-authored-by: Cursor <cursoragent@cursor.com>
Progressive quick preview decoded the full cumulative latent after each scene inside the sampler loop, adding large overhead without fitting ComfyUI's single-pass workflow. Preview now arrives only when the run completes.
Use sampler boundary pixel positions for preview/export offsets instead of subtracting frame_overlap on the client, which misaligned i2v-guided multi-scene runs.
After each scene finishes sampling, decode a downscaled temp mp4 and publish
progress so the Movie Editor can update timeline and preview before the full
chain completes.
Introduces a new temporal_style option that cycles motion→hold ramps
across each scene's denoise trajectory via a segmented frame_rate wrapper.
Wired through Studio, engine settings, chain sampler conditioning tags,
and build_enhancements for single-scene workflows.
Each mixed scene is its own queue request (max_scenes=1), not continuous
chain overlap. Scene 2+ uses its anchor via graph Img2Video and borrows
hidden guides from the previous scene. Sampler applies guide stacks on
the first chunk for solo mixed runs.
Mixed i2v anchors no longer blend the previous scene tail into the
starting chunk or the stitched output. Dragging an image onto a
continuation scene auto-sets mixed so montage stays one queue request.
Preview in-points skip overlap before mixed scenes.
Scene chain sampler scene_boundaries JSON now reports pixel/latent overlap
zones, boundary contamination ranges, per-scene mechanisms, and global steering
notes. Pytest conftest stubs the parent FunPack package so light tests run in
the ComfyUI venv without importing folder_paths; align timeline/builder/nodes
tests with current prompt and report formats.
i2v anchors are starting latents via LTXVImgToVideoInplace; i2v
guides remain separate (carry_i2v_guides / custom guide stack).
Removed the incorrect anchor-as-guide path; mixed scenes now build
their chunk from Img2Video then apply overlap and carried guides.
Mixed scenes stay in the chain run (overlap/carry) while applying
their own image anchor at frame 0 alongside the Studio default
guide from scene 1. Per-scene media is copied for sampler anchor
encoding; custom guide stack toggles remain opt-in.
Guide stack is off by default: multi-scene runs still force
carry_i2v_guides (scene 1 template at frame 0). Enabling
guide_settings.stack_enabled passes per-scene guide JSON to the
chain sampler with toggles for accumulate, frame_idx, and apply_at.
Wire ProgressBar callbacks through scene-chain _sample_chunk so denoise
steps update the global progress hook the frontend polls. Reset progress
when a new generation is queued; label the readout as sampling N/M.
DynamicConditioning is an unpacked-video technique; on LTXAV (packed AV latent, 3D
mask) it can only no-op (comfy's process_timestep patchifies expecting 5D). Rather
than keep dead knobs in the Chain Sampler UI, remove i2v_anchor_power /
i2v_anchor_first_frame_only, the install block, and _install_i2v_anchor entirely.
The packed-safe path for cross-cut appearance carry (strengthen the Chain Sampler's
own carry/guide masks pre-pack) remains available as a future native feature.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The i2v anchor (DynamicConditioning port) crashed on LTXAV with 'not enough values
to unpack (expected 5, got 3)': comfy's process_timestep patchifies the video denoise
mask expecting unpacked 5D [B,C,F,H,W], but LTXAV packs the AV latent so its mask is
3D [B,C,N]. The else-branch (first_frame_only=False) modified + propagated the 3D mask
unguarded -> patchify blew up.
Hard-guard the whole denoise_mask_fn to genuine 5D masks; packed/non-5D -> clean no-op
(one-time log). The technique is fundamentally unpacked-video-only, so on LTXAV it does
nothing -- documented in the tooltip. Native packed-safe carry-strength is the follow-up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two optional Chain Sampler inputs (i2v_anchor_power, i2v_anchor_first_frame_only),
default power=1.0 = off. When >1.0, installs a denoise-mask model patch via the stock
comfy API (set_model_denoise_mask_function) that raises the denoise mask to power**step,
progressively holding the i2v conditioning frame harder as denoising proceeds -> anti-drift
from the input image. No modified model, no extra forward pass, ~zero overhead.
Patched before the batch/main split so both paths inherit it. Fully defensive: missing
mask / unexpected (e.g. packed) shape / any error falls through to the unmodified mask.
EXPERIMENTAL on packed AV latents - validate in ComfyUI. pos_embedding_exp_values.py was
also assessed and is a dead end (internal RoPE precision-matching constant, not a technique).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Laplacian pyramid blend from the LTXV pack is a SPATIAL pixel-space tool and
doesn't transfer to our temporal latent crossfade (uniform full-frame alpha over a
2-4 latent-frame overlap -> multi-band adds nothing). The actual seam problem is that
a linear lerp of two latents superimposes them and lets magnitude collapse at ~0.5
(ghosting + washout). Fix in-domain instead:
- _slerp_block: spherical interp of the overlap block per (batch, frame), direction
via slerp + magnitude interpolated linearly, so the crossfade keeps its energy.
Uses torch.linalg.vector_norm (Tensor.norm treats a 3-tuple dim as a matrix norm).
- _blend_tensors: smoothstep alpha (less dwell at the 50/50 ghost point) + slerp for
video; audio/any further stream stays on the untouched linear path (audio-safety).
- Verified: exact endpoints, magnitude preserved across the seam vs linear washout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reverts the separate FunPackVAEDecoderNoise node (boundary-law violation: if the
Chain Sampler can own it, it should). Decode-time noise is now two optional inputs
on the Scene Chain Sampler (decode_noise_scale / decode_timestep), applied to a
shallow VAE copy via _vae_with_decode_noise before any decode so both the main and
batch-training IMAGES paths benefit. Never mutates the shared input VAE.
decode_noise_scale defaults to 0 (off) -> no behavior change unless enabled.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 1 of the LTXVideo-pack technique pull. Standalone VAE->VAE passthrough that
stamps decode_timestep / decode_noise_scale / seed onto the LTX VAE so its internal
decoder restores fine detail/grain. ~zero cost, scale=0 disables. Place before any
VAE decode (Scene Chain Sampler, VAE Decode).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ports LTXVNormalizingSampler's between-step scalar normalization onto the FunPack
Normalizing sampler. Two new STRING inputs on FunPackNormalizingSampler:
video_normalization_factors / audio_normalization_factors (comma-separated per-step
scalars; short lists repeat last value; both default '1' = off/no-op).
Applied to the running latent at the end of each euler step via _apply_av_norm_factors,
using the existing _packed_video_mask to scale the video and audio streams independently.
This is the ONLY FunPack lever that touches audio, and it does so audio-safely by latent
magnitude only (never attention/hidden states) - the same mechanism the reference node
uses. Conservative + opt-in: ships off, audio untouched unless the user authors a schedule
(reference default was '1,1,0.25,1,1,0.25,1,1'). The adaptive normalize_strength remains
the smarter video-side lever; these are a manual per-step gain schedule.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds temporal_style='auto': Studio classifies each scene's prompt for motion
energy and bakes a continuous frame_rate multiplier into that scene's
conditioning metadata; the Scene Chain Sampler applies it per scene via a
layered model_function_wrapper (composes with embed guidance, restored in
install order). Concrete styles keep the global build_enhancements path
unchanged.
- ltx_enhancements: TEMPORAL_STYLES gains 'auto'; single-source TEMPORAL_STYLE_MULT
table; classify_temporal_intent() keyword director (energy->punchier mult<1,
stillness->hold mult>1, loop flag); factored make_temporal_wrapper() shared by
global + per-scene paths; build_enhancements skips the global wrapper for auto.
- conditioning: _v2_apply_auto_temporal() bakes funpack_temporal_mult/label/loop
into each entry inside the single finalize hook; both refine_v2 returns pass
temporal_style + the run prompt as per-scene fallback text. 'auto' added to V2
node dropdown.
- samplers: _scene_temporal_mult() reader + per-scene temporal wrapper around
_sample_chunk in the scene loop.
- web: 'auto' in the Studio dropdown + hint text.
Heuristic is the baseline director, structured so a reward-driven override can
replace the multiplier per intent-family later. loop's RoPE coordinate trick is
still unimplemented (inert flag only); auto detects loop intent but does not yet
honor it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1) Widget order: moved the refinement_key_input forceInput SOCKET to the
end of optional (after decode_tile_size + embed_guidance_source). A
forceInput placed between widgets desynced ComfyUI's widgets_values
mapping on reload -> the combo's "relative" landed on the decode_tile_size
INT (NaN field).
2) RecursionError: _run_batch_training's no-key fallback called self.sample()
with the variant markers still in positive -> sample() re-detected the
batch -> re-entered _run_batch_training -> infinite loop (hit when
Interactive Guessing/batch is armed without refinement_key_input wired).
Now runs keyless as generate-only: loops every variant, concatenates to
the IMAGES output, skips temp-save/manifest/rating. (Per-variant sub-calls
are single-variant -> _split_batch_variants returns None -> no recursion.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New variant producer for Batch Training that, instead of seed/shortcut
variation, scales the conditioning's spread (sigma) on VIDEO channels by a
linear ramp across N rungs, freezing everything else including the noise
seed so the only variable is conditioning strength. Up amplifies toward
overbake, down dampens. Each rung records funpack_guess_factor (in cond
meta + batch manifest) so the ladder is interpretable and Phase-3 learning
can read each rung. Audio channels untouched (scale_conditioning_spread
reuses the LTXAV channel split). Wired refine_v2 -> _v2_pack_batch
(guess_mode branch -> _v2_pack_guess_ladder) and the Studio refine_v2 call
(guess_mode/direction/range from refiner settings).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Previously the batch path returned only the last variant's frames on the
IMAGES slot. Collect each variant's decoded frames (CPU) and concatenate
them so the whole batch comes out IMAGES back-to-back — enabling the
"just make N slight variations, don't rate them" use case. Falls back to
the last variant if frame shapes are incompatible. (Phase 1 of Interactive
Guessing.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reuse _normalize_video_denoised in _sample_const_rf_full (Hybrid CONST/RF
path) and sample_funpack_distilled_flow: new normalize_strength /
normalize_start_sigma (default 0 = off). Applied to the denoised after
rescue/quality-sharpness, video-only, so audio is never touched and the
samplers keep AB2/Heun/velocity/rescue. Wired node INPUT_TYPES (appended)
-> get_sampler -> Studio _build_one_sampler + JS panels for both.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
