Full frontend redesign into a real movie-editor layout:
- menu bar (File/Edit/View/Generate/FunPack) with dropdowns + status chips
- left Media Browser (projects + media bin drag-drop placeholder)
- right Preview/program monitor + context Inspector (clip props or project
settings + live split preview)
- bottom Timeline track: clips with transition seams, click-to-select drives
the inspector, reorder/exclude/delete
- draggable panel splitters; cinematic dark theme (warm near-black, amber clip
accent, teal selection, film grain), Bricolage Grotesque + IBM Plex Sans/Mono
Architecture: central pub/sub store (store.js) + per-zone view modules; all
backend wiring unchanged (same /funpack/movie/api). Editing moved from cards
into the inspector. Render/Export and media drag-drop are stubbed for later.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per user: serve from ComfyUI's own aiohttp PromptServer at /funpack/movie/
instead of a separate FastAPI/uvicorn process. Removes fastapi/uvicorn/httpx
entirely (all on ComfyUI's bundled aiohttp); backend/requirements.txt deleted,
root requirements.txt needs nothing new.
- server.py: aiohttp route registration on PromptServer (projects/preview/
generate/status/result/library + static UI), guarded for standalone tests;
wired into root __init__.py like batch_training
- bridge.py: parse_timeline/transitions IN-PROCESS (direct FunPack imports);
/prompt /history /view over loopback HTTP
- config.comfy_base_url(): detect the REAL port from PromptServer.instance /
comfy.cli_args so non-default --port works; browser uses same-origin relative
URLs (api.js derives base from window.location) so the port is automatic
- delete app.py, routes/, comfy_client.py, backend/requirements.txt
- README updated; tests still 8/8
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Editable scene timeline + transition library feeding the existing Studio ->
Chain Sampler chain. Light FastAPI sidecar (no torch/comfy) that drives a
running ComfyUI over its API and reuses /funpack/* routes for parsing.
- backend: project store (JSON), timeline model + Studio-format prompt
assembly, ComfyUI client, graph-agnostic workflow injector, routes
(projects/timeline preview/generate/library), FastAPI app
- frontend: vanilla-JS editable timeline (scene cards, transitions, exclude,
generate-only, source type), live split preview, result player
- forward-compat data model: scene source (empty/image/generated_frame),
per-scene excluded flag, intro_transition for the anchor boundary
- tests: 8 passing (prompt assembly, project CRUD, workflow injection)
V1 drives the uniform chain in one pass; per-scene length/FPS, Empty/Image
latent building, media browser, frame->scene anchors, selective regen and
cuts/trims are scaffolded (UI/model) but deferred to later phases.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DynamicConditioning is an unpacked-video technique; on LTXAV (packed AV latent, 3D
mask) it can only no-op (comfy's process_timestep patchifies expecting 5D). Rather
than keep dead knobs in the Chain Sampler UI, remove i2v_anchor_power /
i2v_anchor_first_frame_only, the install block, and _install_i2v_anchor entirely.
The packed-safe path for cross-cut appearance carry (strengthen the Chain Sampler's
own carry/guide masks pre-pack) remains available as a future native feature.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The i2v anchor (DynamicConditioning port) crashed on LTXAV with 'not enough values
to unpack (expected 5, got 3)': comfy's process_timestep patchifies the video denoise
mask expecting unpacked 5D [B,C,F,H,W], but LTXAV packs the AV latent so its mask is
3D [B,C,N]. The else-branch (first_frame_only=False) modified + propagated the 3D mask
unguarded -> patchify blew up.
Hard-guard the whole denoise_mask_fn to genuine 5D masks; packed/non-5D -> clean no-op
(one-time log). The technique is fundamentally unpacked-video-only, so on LTXAV it does
nothing -- documented in the tooltip. Native packed-safe carry-strength is the follow-up.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Two optional Chain Sampler inputs (i2v_anchor_power, i2v_anchor_first_frame_only),
default power=1.0 = off. When >1.0, installs a denoise-mask model patch via the stock
comfy API (set_model_denoise_mask_function) that raises the denoise mask to power**step,
progressively holding the i2v conditioning frame harder as denoising proceeds -> anti-drift
from the input image. No modified model, no extra forward pass, ~zero overhead.
Patched before the batch/main split so both paths inherit it. Fully defensive: missing
mask / unexpected (e.g. packed) shape / any error falls through to the unmodified mask.
EXPERIMENTAL on packed AV latents - validate in ComfyUI. pos_embedding_exp_values.py was
also assessed and is a dead end (internal RoPE precision-matching constant, not a technique).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Laplacian pyramid blend from the LTXV pack is a SPATIAL pixel-space tool and
doesn't transfer to our temporal latent crossfade (uniform full-frame alpha over a
2-4 latent-frame overlap -> multi-band adds nothing). The actual seam problem is that
a linear lerp of two latents superimposes them and lets magnitude collapse at ~0.5
(ghosting + washout). Fix in-domain instead:
- _slerp_block: spherical interp of the overlap block per (batch, frame), direction
via slerp + magnitude interpolated linearly, so the crossfade keeps its energy.
Uses torch.linalg.vector_norm (Tensor.norm treats a 3-tuple dim as a matrix norm).
- _blend_tensors: smoothstep alpha (less dwell at the 50/50 ghost point) + slerp for
video; audio/any further stream stays on the untouched linear path (audio-safety).
- Verified: exact endpoints, magnitude preserved across the seam vs linear washout.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reverts the separate FunPackVAEDecoderNoise node (boundary-law violation: if the
Chain Sampler can own it, it should). Decode-time noise is now two optional inputs
on the Scene Chain Sampler (decode_noise_scale / decode_timestep), applied to a
shallow VAE copy via _vae_with_decode_noise before any decode so both the main and
batch-training IMAGES paths benefit. Never mutates the shared input VAE.
decode_noise_scale defaults to 0 (off) -> no behavior change unless enabled.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 1 of the LTXVideo-pack technique pull. Standalone VAE->VAE passthrough that
stamps decode_timestep / decode_noise_scale / seed onto the LTX VAE so its internal
decoder restores fine detail/grain. ~zero cost, scale=0 disables. Place before any
VAE decode (Scene Chain Sampler, VAE Decode).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ports LTXVNormalizingSampler's between-step scalar normalization onto the FunPack
Normalizing sampler. Two new STRING inputs on FunPackNormalizingSampler:
video_normalization_factors / audio_normalization_factors (comma-separated per-step
scalars; short lists repeat last value; both default '1' = off/no-op).
Applied to the running latent at the end of each euler step via _apply_av_norm_factors,
using the existing _packed_video_mask to scale the video and audio streams independently.
This is the ONLY FunPack lever that touches audio, and it does so audio-safely by latent
magnitude only (never attention/hidden states) - the same mechanism the reference node
uses. Conservative + opt-in: ships off, audio untouched unless the user authors a schedule
(reference default was '1,1,0.25,1,1,0.25,1,1'). The adaptive normalize_strength remains
the smarter video-side lever; these are a manual per-step gain schedule.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds temporal_style='auto': Studio classifies each scene's prompt for motion
energy and bakes a continuous frame_rate multiplier into that scene's
conditioning metadata; the Scene Chain Sampler applies it per scene via a
layered model_function_wrapper (composes with embed guidance, restored in
install order). Concrete styles keep the global build_enhancements path
unchanged.
- ltx_enhancements: TEMPORAL_STYLES gains 'auto'; single-source TEMPORAL_STYLE_MULT
table; classify_temporal_intent() keyword director (energy->punchier mult<1,
stillness->hold mult>1, loop flag); factored make_temporal_wrapper() shared by
global + per-scene paths; build_enhancements skips the global wrapper for auto.
- conditioning: _v2_apply_auto_temporal() bakes funpack_temporal_mult/label/loop
into each entry inside the single finalize hook; both refine_v2 returns pass
temporal_style + the run prompt as per-scene fallback text. 'auto' added to V2
node dropdown.
- samplers: _scene_temporal_mult() reader + per-scene temporal wrapper around
_sample_chunk in the scene loop.
- web: 'auto' in the Studio dropdown + hint text.
Heuristic is the baseline director, structured so a reward-driven override can
replace the multiplier per intent-family later. loop's RoPE coordinate trick is
still unimplemented (inert flag only); auto detects loop intent but does not yet
honor it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
clear_refinement_data() early-returned on an empty refinement_key, but a
keyless run stores its blessed/attn maps, creativity latent and velocity-bias
trajectories under the 'default' bucket. So a keyless Session Reset no-opped
and those disk-persisted stores survived restart too -- the first fresh run
felt cleaner than every later run. Normalize empty key to 'default' (matching
the write side) so reset actually wipes that bucket.
Also wire the keyless Absolute taste store (FUNPACK_ABSOLUTE_KEY) into reset:
it learns from every rated run regardless of steer_mode but had no clear path,
so it would survive reset + restart forever. Delete its state json + value fn
on reset_session.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1) Widget order: moved the refinement_key_input forceInput SOCKET to the
end of optional (after decode_tile_size + embed_guidance_source). A
forceInput placed between widgets desynced ComfyUI's widgets_values
mapping on reload -> the combo's "relative" landed on the decode_tile_size
INT (NaN field).
2) RecursionError: _run_batch_training's no-key fallback called self.sample()
with the variant markers still in positive -> sample() re-detected the
batch -> re-entered _run_batch_training -> infinite loop (hit when
Interactive Guessing/batch is armed without refinement_key_input wired).
Now runs keyless as generate-only: loops every variant, concatenates to
the IMAGES output, skips temp-save/manifest/rating. (Per-variant sub-calls
are single-variant -> _split_batch_variants returns None -> no recursion.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add guess_freeze_seed (default True) threaded refine_v2 -> _v2_pack_batch
-> _v2_pack_guess_ladder and the Studio refine_v2 call. Frozen = every rung
shares one seed (only conditioning strength varies; cleanest for learning).
Off = distinct seed per rung (varied compositions too). New "Freeze seed"
toggle in the Interactive Guessing config; persisted on Arm.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
UI: Batch Training tab gains Mode (Regular / Interactive Guessing),
Direction, Range, and a Learning toggle; Arm sets refiner.guess_* +
batch_learning; auto-disarm clears guess_mode; rating list shows each
rung's [x factor]; Submit passes the learn flag.
Learning (batch_training.py): submit honors learn=false (pure generation,
no training). Guess batches detected via guess_factor -> _learn_guess_ceiling
finds the furthest-pushed acceptable rung before quality broke and stores a
per-key safe ceiling (absolute spread + factor) via _store_guess_ceiling.
Dual auto-cap (refine_v2, normal runs only): soft pre-limit on
absolute_strength + hard clamp of the output conditioning's video-channel
spread to the learned-safe value in _v2_finalize_conditioning. Skipped
during the guessing ladder itself. Helpers conditioning_spread /
scale_conditioning_spread.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New variant producer for Batch Training that, instead of seed/shortcut
variation, scales the conditioning's spread (sigma) on VIDEO channels by a
linear ramp across N rungs, freezing everything else including the noise
seed so the only variable is conditioning strength. Up amplifies toward
overbake, down dampens. Each rung records funpack_guess_factor (in cond
meta + batch manifest) so the ladder is interpretable and Phase-3 learning
can read each rung. Audio channels untouched (scale_conditioning_spread
reuses the LTXAV channel split). Wired refine_v2 -> _v2_pack_batch
(guess_mode branch -> _v2_pack_guess_ladder) and the Studio refine_v2 call
(guess_mode/direction/range from refiner settings).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Previously the batch path returned only the last variant's frames on the
IMAGES slot. Collect each variant's decoded frames (CPU) and concatenate
them so the whole batch comes out IMAGES back-to-back — enabling the
"just make N slight variations, don't rate them" use case. Falls back to
the last variant if frame shapes are incompatible. (Phase 1 of Interactive
Guessing.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reuse _normalize_video_denoised in _sample_const_rf_full (Hybrid CONST/RF
path) and sample_funpack_distilled_flow: new normalize_strength /
normalize_start_sigma (default 0 = off). Applied to the denoised after
rescue/quality-sharpness, video-only, so audio is never touched and the
samplers keep AB2/Heun/velocity/rescue. Wired node INPUT_TYPES (appended)
-> get_sampler -> Studio _build_one_sampler + JS panels for both.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New SAMPLER for distilled LTXAV at CFG=1: deterministic 1st-order euler
base (clean ddim look + clean audio) plus per-step video-only latent
normalization to counter overbaking / oversaturation / colour drift at
~zero overhead. Tracks the video latent's robust spread, anchors a
reference once x0 is meaningful (normalize_start_sigma), and compresses
inflated later steps (normalize_strength). Benefit-only, no-op when
healthy, audio never touched (whole-latent for single-stream LTXV).
Velocity bias + rescue included, video-only.
Node FunPackNormalizingSampler + Studio sampler option ("Normalizing")
wired through _build_one_sampler and the JS panel. Registered in __init__.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Instead of full AB2 on every step, optionally ramp the AB2 contribution
linearly 0->1 across the schedule: denoised_eff = lerp(raw, ab2, i/(N-1)).
Early/high-sigma steps stay near 1st-order euler (less overshoot); late
steps get full AB2. Reuses the computed AB2 estimate -> no extra model
evals. Aimed at low-step distilled quality. No-op at order=1; audio still
rides plain euler. Wired node -> get_sampler -> Studio panel (+ JS toggle).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The FunPack Hybrid Euler 2S and Distilled Flow samplers applied AB2 denoised
extrapolation and the Heun corrector to the whole packed LTXAV latent, audio
slice included -> degraded audio. (Stock euler/ddim are clean because they're
1st-order; ComfyUI "ddim" == euler.) Confine both to the video stream via
_video_only so audio always rides plain 1st-order euler on the raw model
denoised, regardless of order/correction settings. Distilled Flow s_noise
diversity injection is now video-only too.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Absolute/Relative steer mode (Refiner V2 + Studio) and audio-safe
conditioning steering on LTXAV.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The conditioning-dim diagnostic logged once per process via _FUNPACK_AV_LOGGED,
so after the first generation it went silent — easy to read as "it never ran".
Fold the detected split into the per-run "Absolute steer applied" line so every
generation states whether audio is PROTECTED (video[:v] of D) or NOT split
(unrecognised width -> steering all channels, audio unprotected). Makes the
no-op case visible instead of silent.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
LTXAV splits one conditioning tensor into video|audio channel-slices
(av_model _prepare_context: torch.split(context,[v_dim,a_dim],-1)) and
cross-attends each stream to its own slice. Steering the whole tensor
corrupted the audio's text conditioning -> degraded audio (observed live).
Add protect_audio_channels: restore the audio channel-slice from the
unsteered conditioning so every edit moves video only. Applied at all
steering chokepoints:
- _v2_apply_conditioning_memory (per-prompt: liked/axis/concept/vf-grad)
- _v2_apply_absolute (global taste pull)
- _v2_ascend_conditioning (relative VF ascend+search)
- attn2 K/V direction patch + phrase emphasis
- FunPackConditioningAdjust (manual)
- sampler embed_guidance wrapper + per-scene VF ascend
Split auto-detected from width (7680->3840, 6144->4096); logs once.
Single-stream LTXV / unknown width = no-op (no regression).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Inserting the combo mid-list shifted every saved widgets_values slot in
existing Scene Chain Sampler nodes, so a leftover numeric value validated
against the combo ("0 not in ['relative','absolute']"). Moved it to the end
of optional inputs; old workflows keep their widget positions and fall back
to the 'relative' default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Relative (default, unchanged) = per-prompt steering. Absolute = a keyless,
prompt-agnostic "global taste" prior that pulls conditioning toward generally-
liked directions regardless of the prompt; Both layers them.
- New keyless Absolute store (FUNPACK_ABSOLUTE_KEY): pooled liked/disliked
direction + global value function, learned across every rated generation.
- Absolute rating semantics: a rating means "details I like in general" (not
prompt adherence) — reward-driven liked/bad-dir storage, axis attribution
bypassed. Perfect = "more of this everywhere".
- _v2_finalize_conditioning replaces the per-key ascend hook on both returns;
pure Absolute also bypasses per-prompt conditioning memory.
- Scene Chain sampler: embed_guidance_source (relative/absolute) lets the
per-step nudge draw from the global direction/value function.
- Studio popup + V2 node expose Steer mode + Absolute strength.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds s_churn/s_tmin/s_tmax to sample_funpack_distilled_flow: optionally
re-noise the latent within [s_tmin, s_tmax] before each eval, then denoise
from the bumped sigma_hat. No extra model calls (still one eval per step,
at a higher noise level). Fights stale/frozen distilled output (more motion)
and can sharpen by letting the model re-resolve. sigma_hat is clamped to the
schedule start so the model never sees a higher-than-trained noise level, the
final jump-to-zero detail step is never churned, and the default 0.5-1.0 band
protects the sub-0.4 detail finish per the empirical sigma findings.
Churn supersedes the legacy s_noise ancestral path while active and is
video-stream-only on packed LTXAV latents. With s_churn=0 the sampler is
byte-identical to the previous deterministic ODE. Wired into the node and
Studio's Distilled Flow config panel.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Port the Hybrid/RF velocity-bias capture/apply and rating-gated reactive
rescue into sample_funpack_distilled_flow, reusing the shared trajectory
memory helpers. Adds velocity_bias_mode/strength/source/key and
rescue_mode/threshold/strength to the FunPackDistilledFlowSampler node and
wires them into Studio's Distilled Flow config panel (same blank/'default'
velocity-key -> refinement-key fallback as Hybrid). Steering is
magnitude-preserving and, on packed LTXAV latents, video-stream only
(audio-safe). With all controls off the sampler is byte-identical to the
previous deterministic ODE.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Document the three undocumented releases (2.7.5 embed guidance + online
value function + mid_scene_guide; 2.7.6 value-function ascent + rating
overhaul; 2.7.7 reactive rescue + Batch Training) and bring the Scene
Chain Sampler, Hybrid Euler-2S, and Studio docs in line with current
parameters and features.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Batch Training confirmed working end to end (controlled batch with transitions, distinct
per-variant seeds, node rating ignored during a batch, per-entry rating + VF/axis learning
on submit). Stripped the temporary trace prints (Batch Training check / Batch check / ACTUAL
scene seeds), keeping clean progress logs. Gitignore the local .testvenv.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The live log showed the single-scene path: 'packed 3 variant conditioning(s)' with ACTUAL
scene_seeds identical across all 3 (all inherited the base entry's funpack_scene_seed). My
earlier seed fix only covered the multi-scene duplication path. When a prompt resolves to one
scene (common), _v2_build_batch_variants reused the base scene seed for every variant ->
identical noise -> identical videos (only tiny diffs from VF ascent).
Fix: _v2_pack_batch now assigns a fresh, well-spread per-variant scene seed (gen-seeded RNG,
reproducible, per-scene offset) to the packed entries in BOTH paths, after building them.
Verified in venv: single-scene 3 variants -> 3 distinct seeds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 'seed=' in the batch log was the loop index seed (base+idx), not the seed that drives
noise. The real noise seed is each scene's funpack_scene_seed (set distinct per variant by
_v2_pack_batch). Print those so we can tell whether distinct seeds actually reach the
sampler vs. stale/old-code identical seeds.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1) All batch videos identical: in the transition path each scene carries a
'funpack_scene_seed' the sampler honors over its own seed, so duplicating the scene-set
left every variant with the same seeds. _v2_pack_batch now assigns DISTINCT per-variant
scene seeds via a gen-seeded RNG (well-spread, reproducible per gen). Verified in venv:
4 variants -> 4 distinct seed sets, reproducible.
2) Ignore the node's rating widget while a batch generates: that selection would apply
learning to the PREVIOUS run, but it's stale during a batch (the batch is rated afterward
in the panel). When batch_variants>1, force rating to '-Just forget it-' (skip_learning),
so all learning paths (phrase/conditioning-memory update, VF training, velocity commit)
are skipped. Conditioning APPLICATION still runs (it has no skip_learning guard), so the
batch still uses the learned directions.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause (found via the diagnostics + the venv tests): the producer was gated on
'not split_by_transitions', but the user's workflow is multi-scene chains, so with
transitions on the producer never ran -> no variant markers -> sampler did one pass.
Fix: defer variant packing to the FINAL conditioning, applied right before BOTH returns
(the transition early-return and the normal return), so it wraps whatever output ends up
being. New _v2_pack_batch dispatcher: single-scene output -> _v2_build_batch_variants
(shortcut variants); multi-scene output (transition split) -> duplicate the whole scene
set per variant (seed-only). Each variant tags ALL its scene entries with the same index,
so the sampler groups by marker into N full chains and keeps variants independent.
Verified in the test venv: 3-scene set x 4 -> 12 entries, markers [0,0,0,1,1,1,2,2,2,3,3,3]
-> sampler sees 4 groups of 3 scenes; a single-variant group returns None (chains normally,
no recursion). Marker detection + producer round-trip all green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Prints (a) the batch_variants value + mode flags the refiner sees and how many packed
entries it produced, and (b) the positive entry count + variant markers + group count the
Chain Sampler sees. Pinpoints whether the break is the arm/settings (batch_variants==1 at
the refiner), the conditioning path (markers stripped before the sampler), or the split
logic. Temporary, to be removed once the cause is found.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
It shouldn't have been added there. The node 'Batch Training' button window is the sole
batch UI now (config when idle, rating list when finished). Removes the loadBatch() panel
and its call from renderRefiner; no leftover references, JS parses clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per feedback: Batch Training is a complete UI thing driven by Studio (the hub), not a
prompt, and it must NOT start generation itself (ComfyUI's Generate does).
- Removed batch_iterations from the Chain Sampler entirely (input + param). The sampler
now enters batch mode ONLY when Studio packs variant markers into positive; the batch
count lives in Studio. _run_batch_training simplified to variant-entries-only.
- The node's 'Batch Training' button now opens a dedicated window (openBatchPanel) that
shows EITHER a config view (set batch size, Arm/Disarm) when no batch is pending, OR the
rating list (preview + per-entry rating + Submit/Forget) when a batch has finished. It
never queues — arming just sets refiner.batch_variants; the user presses Generate.
- Dynamic button label reflects live state: '(off)', '(armed ×N)', '(Processing…)',
'(Finished — rate)'. Driven by execution_start/success/error listeners, which also
one-shot disarm after a batch runs so the next Generate is normal.
The older in-Refiner-tab batch panel remains functional (secondary path); the node-button
window is now the primary entry point.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Submit now does more than train the value function (3a). New refiner method
_v2_learn_from_batch feeds each rated entry's conditioning through the SAME per-axis
direction-memory machinery as a normal rated run (_v2_update_conditioning_memory →
axis directions + liked/bad direction stores), and tallies axis votes across the batch:
because tokens were frozen, an axis rated missing/wrong in >= half the rated entries is a
high-confidence problem (not seed luck). Those consistent axes are written to
active_repair_axes, which the next generation's repair persists until a Perfect — so a
batch where most gens were 'Missing details' makes the next run boost details.
Wiring: batch submit collects (rating, serialized cond) per rated entry, trains the VF,
then calls _v2_learn_from_batch; the resulting insight ('consistent problem axes = ...')
is returned and shown in the Studio submit toast. List route now also returns each item's
variant index + prompt for the panel.
Batch Training is now complete: seed-only + shortcut variants + node button + VF training
+ axis/direction learning, all human-rated.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Producer (Studio owns conditioning, per the boundary law): refine_v2 gains batch_variants.
When >1, _v2_build_batch_variants packs N conditioning entries into the single positive wire,
each tagged 'funpack_batch_variant'. Variant 0 is the fully-processed base; variants 1..N-1
re-resolve shortcuts with seed+i (random option per entry) and get the same conditioning-memory
treatment, deduped by expansion text. No shortcut variability -> all entries resolve identical
and reuse the base cond -> the batch becomes seed-only (sampler varies the seed per entry).
Captures the pre-shortcut prompt to re-resolve. Not combined with split_by_transitions.
Trigger: a 'Batch Training' button on the Studio NODE (next to Open Studio) — prompts for N,
sets refiner.batch_variants, queues one prompt, then auto-reverts to 1 (one-shot). Studio is
the batch trigger (hub principle); no sampler input needed.
Sampler: now enters batch mode when positive carries variant markers, even if batch_iterations
is 1 (markers are the trigger; variant count wins). Each entry already saves its own cond, so
Submit trains the VF per variant — teaching which prompt variant rated better. Panel hints now
point at the button.
End-to-end: press Batch Training -> N -> Studio makes N variant conds -> sampler samples N ->
previews to temp -> rate in Studio panel -> Submit trains VF per entry -> batch cleared.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per the Studio/sampler boundary law (Studio owns conditioning; sampler only samples), the
sampler no longer encodes anything. It now auto-detects when Studio has packed N shortcut
VARIANTS into positive — each scene entry tagged with 'funpack_batch_variant' = variant
index — via _split_batch_variants: if present, it samples one chain per variant (variant
count wins); otherwise it falls back to seed-only repeats of batch_iterations. Decisions:
'sampler loops, Studio varies' + 'pack into positive + markers'.
Each entry now saves its OWN conditioning (<id>.cond.pt) — identical across a seed-only
batch, distinct per variant — and the manifest records variant index + prompt per item.
Submit (batch_training.py) trains the VF per entry from its own cond, so variant batches
teach the model which prompt variant rated better (not just seed-quality variance). Studio
panel shows the variant tag + prompt per row.
Studio-side variant PRODUCTION (resolve shortcuts N times, encode, pack markers) is the
remaining half — a Studio/refiner feature, tracked next. Sampler side is ready for it.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Phase 1 amended: the engine now also saves the frozen conditioning (cond.pt) per batch
so submit can train on (cond, rating).
Phase 2 (panel + API): new batch_training.py registers /funpack/batch/{list,submit,forget}
(follows the existing /funpack/* PromptServer pattern). Studio gains a 'Batch Training'
section under the refiner: lists the pending batch for the active refinement key with each
entry's filename + seed, a Preview button (loads the temp webp via /view?type=temp), and a
per-entry V2-rating dropdown, plus Submit and 'Forget batch'. Previews served straight from
ComfyUI temp — no custom file route needed.
Phase 3 (learning intake): Submit loads cond.pt and trains the value function on
(cond, reward) for every rated entry — same conditioning, N rewards = variance-reduction
RLHF, which directly sharpens conditioning ascent. '-Just forget it-'/unknown ratings are
skipped. Submit consumes (deletes) the batch; Forget discards it without learning. Either
way the panel clears and the sampler is ready for a new batch.
Human stays the rating oracle throughout (no auto-rating) — 'user over everything'.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New optional 'batch_iterations' on FunPackLTXAVSceneChainSampler. When >1, runs the
ENTIRE scene chain N times frozen except the seed (seed + i) — conditioning/settings
provably identical because each run reuses sample() itself as the per-generation
primitive with only the seed changed. This gives a clean controlled-batch RLHF signal
(same conditioning, only noise varies) instead of the confounded one-at-a-time loop
where conditioning drifts between rated gens.
Each iteration is persisted under ComfyUI's TEMP dir (funpack_batches/<key>/<stamp>/) so
artifacts are wiped on restart (ephemeral rating material) and are servable via
/view?type=temp for the upcoming Studio panel: latent (.latent.pt) + a downscaled
animated-webp preview + a batch.json manifest (seed, scene prompts, subfolder, rating
slots). LATENT output passes through the last iteration so the downstream decode/save
path is unchanged.
Batch semantics: an entry is the full set of provided scenes — scenes within an entry
chain with overlap as normal; separate entries are independent sample() calls with no
cross-entry carry. Requires refinement_key_input (degrades to a single run without it).
Phase 2 (Studio rating panel + /funpack/batch routes) and Phase 3 (batch-train intake
in the refiner) to follow.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Audit finding: value_guidance defaulted OFF, and it gated value-function TRAINING — so
by default the VF never trained, conditioning ascent + the chain sampler's embed_guidance
were permanently inert, and a user who ran many gens with it off then enabled it started
from zero (no accumulated learning).
Fix decouples learning from application:
- The VF now ALWAYS trains in the background on rated gens (cheap: small MLP, 20 steps,
no diffusion calls) whenever there's a rated previous run + refinement key. The reward
asset accumulates regardless of the toggle.
- value_guidance now controls only APPLICATION of the learned direction (output ascent +
search), via a new apply= param on _v2_ascend_conditioning. Default ON.
- Defaults flipped to True: refine_v2 signature, Studio settings read, JS default +
migration. Studio toggle relabeled 'Apply value guidance' with corrected help (training
always on; applies after 10+ ratings; off = strict prompt fidelity, learning still
accumulates). Fixed stale '5+ ratings' text (MIN_SAMPLES is 10).
Side effect: embed_guidance (chain sampler) now has a trained VF available by default.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three issues found tracing the full Studio path with all features on:
1) BUG (silent data loss): a custom velocity_refinement_key captured trajectories
under that key but the refiner committed the rating under the effective refinement
key — so custom velocity keys never learned (banks stayed empty, rescue/velocity a
permanent no-op). This undercut per-key cut-routing. Fix: Studio resolves each pass's
velocity key (same logic as _build_one_sampler) and passes them to refine_v2, which
now commits the verdict to the union of {effective key} + velocity keys (dedup).
Backward compatible (callers without _velocity_keys behave as before).
2) Effectiveness: VELOCITY_BIAS_TARGETS (0.90, 0.80) matched only sigma~0.909 on the
standard 8-step LTX schedule, so rescue fired once per gen at a near-noise step and
never at the detail-forming sigmas. Broadened to (0.90, 0.72, 0.42) -> hits 0.909,
0.725, 0.422. Velocity APPLY still structure-only (highest target); rescue now acts
where runs actually go good/bad. (New detail targets fill as gens are rated.)
3) Redundant value: velocity_aspect_bucket was a manual text field never tied to actual
resolution, and the bank key already includes the full latent shape (which encodes
resolution) — so it was doubly redundant. Removed from the key, all sampler plumbing,
INPUT_TYPES, Studio build, and the JS UI/defaults. Persistence loads old stores fine
(the 'aspect' field is simply ignored; key is now (refkey, target, shape)).
Verified: both py files + JS parse clean; all refactored call signatures aligned.
Tests require the ComfyUI venv (torch) to run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
UX/variability win — collapse the expert knob-soup into one control plus a
legible summary.
- Variability macro (Chill/Spicy/Chaos presets + 0..1 slider) orchestrates the
stochastic knobs: eta (video variability), eta_final (late-noise ramp, only at
high chaos), quality_sharpness (detail recovery as noise rises), velocity spice
(action injection past Spicy, source=nearest), and motion pulses. Deliberately
keeps high_quality_pct>0 (disabling the deterministic finisher under ancestral
noise = mess). Writes the existing hybrid fields; all still individually editable.
- 'Active features (this config)' readout: green/grey rows for ancestral noise,
velocity spice, rescue, quality pass, motion pulses — turns the invisible stack
legible. Notes audio is auto-held deterministic on LTXAV.
Frontend-only; no model/Python changes (all fields already plumb through).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The real limiter wasn't the 0.35 slider cap but the internal fixed 4.5%-of-|x|
delta clamp, which saturated almost immediately — the upper slider range was a
no-op. Now the rotation cap scales with strength (0.30*strength, capped 0.95), so
the whole range is meaningful: ~0.15 subtle spice (matches old feel), 0.3-1.0 clear
action crossover, 2-3 approaches full action replacement without wiping the current
gen. Magnitude-preserving renorm still keeps |x|, so high strength rotates rather
than blows up. Raised the param/UI cap to 3.0 and updated the tooltip.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
eta_final was silently ignored whenever the quality pass was off: the decay was
anchored to quality_sigma_start, which collapses to sigma=0 when high_quality_pct=0,
so effective_eta stayed pinned at eta for every step. Combined with the missing
deterministic finisher, full-strength ancestral noise ran through the final detail
steps with nothing to resolve it -> noisy mess.
New _effective_eta() ramps eta -> eta_final across raw schedule progress
(sigma/sigmas[0]), so eta_final works with the quality pass on or off. Used in both
the RF and eps paths. No-op when eta_final == eta (default), so existing runs are
unchanged.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause of consistently bad LTXAV audio: ComfyUI packs the video+audio
latents into one [B,1,N] tensor for sampling (pack_latents, video first), and
the RF sampler's ancestral re-noising (eta/s_noise) + motion pulses + velocity/
rescue steering sprayed the whole packed tensor — including the audio slice.
Ancestral re-noising every step destroys audio coherence (video tolerates it,
and in fact benefits from the scene variability).
Fix: recover comfy's per-stream latent_shapes from the guider conds, build a
[1,1,N] video-region mask, and confine every FunPack perturbation to the video
slice. Audio now rides the clean deterministic euler-RF step:
- ancestral noise branch restructured into x_det (deterministic, audio) vs x_anc
(full ancestral, video), combined via the mask.
- motion pulses, velocity-bias apply, rescue, and quality sharpness masked too.
Single-stream LTXV (mask None) is unaffected — behavior identical to before.
Only the CONST/RF path needs this (LTXAV is always CONST); eps path sees no AV.
Prints a one-line diagnostic when packed AV is detected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Compensate the slow quality degradation from velocity-bias 'apply' (additive
mean-pull) with four guards on the Hybrid Euler 2S sampler (both eps and
rectified-flow/LTXAV paths):
- B: apply is now magnitude-preserving — rotate x toward (x+delta) and
renormalize to |x| instead of x+delta, so no energy is injected.
- C: apply fires only at the highest (structure) velocity target and decays
strength with the sigma ratio; detail sigmas left untouched.
- E: new quality_sharpness knob — temporal-average unsharp on the x0 prediction
during the quality phase to restore high-frequency detail (free, no extra eval).
Default 0.0 (off).
- D: new velocity_bias_source = mean|nearest — 'nearest' steers toward the single
best-matching prompt cluster (one real good gen's detail) instead of the
washed-out average. Affects both apply and rescue. Default 'mean' (off).
Plumbed through get_sampler/INPUT_TYPES, Studio _build_one_sampler, and the
Studio JS sampler panel (both presets + UI controls).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
