LTX 2.5 reuses LTXAVModel/LTXBaseModel behind config flags, so the samplers,
the identity-transfer wrappers and the guide path all bind to it unchanged.
Two places assumed 2.3 specifics, and both would have gone wrong quietly.
The AV channel split was a hardcoded width table. From 2.5 on comfy reads the
video/audio projection widths off the checkpoint rather than assuming them, so
a checkpoint can ship a pair the table has never seen — and an unrecognised
width made ltxav_video_channels return the FULL width, which silently steers
the audio text context along with the video one. The only symptom would have
been degraded audio. The split is now measured off the live model when one is
on hand; the table stays as the fallback.
Decode-time noise only ever reached VideoVAE.decode. 2.5's diffusion decoder
takes no timestep and seeds itself, but it is an nn.Module, so stamping the
attributes onto it succeeded and was then ignored — a knob that reads as live
and does nothing. It is now capability-tested against the real contract
(hasattr decode_timestep, not a class name) and says so when it cannot apply.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three things that were each locked to one implementation detail:
- steps + schedule were a KSampler-branch feature, though every sampler here
walks a sigma list handed in from outside. They are now pass-level, so
Distilled Flow and Hybrid Euler 2S can run a named schedule (and the second
pass can be computed rather than typed) in both frontends and on the node.
- the project's Frames field snapped to the model's grid on every family. On
H3 that is required, but its arrows moved by 1 and the snap put the value
straight back; on LTX the builder rounds up by itself, so the snap only
fought the user. H3 now steps a whole 17 frames per arrow, LTX keeps what
you type and walks 8k+1 with its arrows.
- ALG lived inside the Distilled Flow step loop, so it was unreachable with a
stock KSampler. Its blurred/sharp swap is decided by the step's sigma, and
sigma is an argument of every model call, so a denoiser proxy drives it from
outside any sampler. New alg_anchor toggle on the Chain Sampler drives the
in-loop version too, so there is one control wherever you are.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An image prompt comes back one tag long: comfy builds minimax_token_tags from
the tokenizer's INPUT sequence (embeds.shape[1]) while the conditioning is what
the encoder returned, and the vision block's expansion does not survive 1:1.
The reconcile treated any surplus as corruption and dropped the whole vector —
which on exactly those prompts discards the modality-0 marks for the image span
and leaves the DiT modulating the picture as if it were text.
Position i still means position i, so the tail is trimmed instead. Growing
(appended tokens) is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two independent, silent breaks made an H3 i2v scene generate from the prompt
alone.
1. The Editor always runs split_by_transitions, and the per-scene conditionings
REPLACE the single entry Studio built — but they were re-encoded with no
reference_image and no h3_references. Both the frame-0 keyframe pin and the
Qwen vision presentation were gone before the sampler looked, even for a
one-scene project. The split now carries the anchor onto scene 0 and the
references onto every scene. Gated to H3: on LTX the anchor rides in the
latent, and adding Gemma3 vision to scene 0 would change existing output.
2. A MiniMax H3 Image to Video node hands its first/last frame pins out on its
CONDITIONING output, which had nowhere to go — the sampler's positive comes
from Studio. New optional h3_keyframes input on the Chain Sampler salvages
them: first frame onto the opening scene, last frame re-indexed onto the
closing scene's own final frame (the source node's frame_count is its own,
and a stale index would land mid-clip where PackedLayout refuses it). The
editor auto-wires it for H3 projects.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A SAMPLER object is only the step function, so picking KSampler used to leave
the pass with no schedule at all unless a sigma list was typed by hand.
steps + schedule now build one from the loaded model's model_sampling, exactly
as ComfyUI's BasicScheduler does. The schedule dropdown is the switch between
the two sources rather than a silent precedence rule: "use_user_sigmas" — the
default, so nothing changes for an existing config — runs the Sigmas field as
typed, and any real scheduler computes instead, leaving the typed schedule
parked in the field to switch back to without retyping it. Studio logs the fact
that the field is being ignored, and steps is only shown while it applies.
Both frontends, same change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
minimax_h3.py shipped tags_match / token_tags_length / extend_token_tags with a
docstring saying a mismatch is a hard IndexError mid-forward — and nothing ever
called them. Studio edits conditioning in many places, and H3 tokenizes reference
images into the same span (ref_image_size "match" scales them to the output
resolution), so the token count moves with the prompt AND with the resolution.
That is why the crash looked intermittent and why editing either one made it come
and go, reported only as a bare "list index out of range" at step 0.
Studio now reconciles before returning: extend the tags when tokens were appended,
drop them when tokens were removed (absent tags the DiT handles, wrong tags it does
not), and print which happened — losing the tags changes how the text span is
modulated, so it must not be silent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
H3 support was written against ComfyUI's PR; it has since merged (57500fc5, shipped in
v0.30.0) and the weights are out. Everything the adapter assumed still holds — audio time
axis, 17k+5 grid, batch-1 forward, the imported optimized_attention symbol, every payload
key — so this is what the release itself changed, not a correction of the reading. LTX-2 /
LTXAV is untouched throughout: a project picks a family and both are first-class.
* i2v anchors now work on H3. LTX writes the image into the starting latent
(LTXVImgToVideoInplace); H3 has no such path — its anchor is a condition row packed
beside the text — so a scene's anchor image becomes a frame-0 keyframe pin instead, and
the LTX node is no longer run against an H3 latent. Studio hands the source image's
pixels to the sampler (it owns the CLIP, the sampler owns the VAE). Pins now MERGE, so
a first-frame anchor and a last-frame guide coexist: H3's own first-and-last mode.
* Frame geometry follows the model: frames per scene snap to the family's grid (LTX 8k+1,
H3 17k+5) in both frontends, in the graph builder, and in the sampler, and H3 renders at
its fixed 24 fps. Picking a family moves the project onto its geometry and says so.
* The two checkpoints are named. fl2va (anchors) and ref2va (references) load identically
and never reject each other's conditioning, so the run reports which mode it is in and
the setup panel spells out the split.
* Pipeline setup is no longer a placeholder: released, with the published filenames.
* Reference images can be encoded at the reference pipeline's 2048px short edge.
Fixes found on the way:
* A video reference lost its soundtrack between Studio and the sampler — Studio had
already announced it to the text encoder as <Audio 1>, so every later audio ordinal in
the prompt pointed one reference earlier than written.
* Easy Gen never knew it was on H3: its model config lives on the project, which the
shared capability check did not read, so every H3 warning silently reported LTX.
* An off-grid scene length failed with arithmetic instead of an explanation.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The graph builder is now parameterized by model family instead of assuming LTXAV. H3
keeps the core's shape but three of its nodes do not apply: LTXVConditioning (stamps LTX's
frame_rate onto conditioning, which H3 has no field for), LTXVConcatAVLatent (H3's own
EmptyMiniMaxH3LatentAV emits both streams, so the latent feeds the sampler directly) and
LTXVAudioVAEDecode (reads an output_sample_rate H3's audio VAE does not define — it would
raise). Requirements, wirable ports, role targets, guided-wiring hints and the port picker
all follow the family, so a loader can no longer be wired into a node the graph never emits.
The family is a Models setting, asked and never guessed from a checkpoint filename: a wrong
guess fails deep inside ComfyUI at generate time instead of in the panel that can fix it.
Video references now work, rather than being refused. FunPack owns the pipeline and already
requires ffmpeg for preview/export, so minimax_h3 decodes reference clips itself — once, at
24 fps — and derives both views from that single decode: the 2 fps timestamped frames Qwen
sees, and the full-rate latent block the DiT packs. A clip's own soundtrack rides with it,
presented as its <Audio j> label immediately before its <Video k>, which is what ties the
two together. Capped at 15s because reference tokens ride every sampling step.
Editor UI: a Reference media view (H3 only — there is no LTX equivalent to grey out) with
per-type 1-based tags shown exactly as the prompt must spell them, reorder controls that
renumber both sides at once, and a soundtrack picker on video entries. Warning chips beside
Generate for the LTX-only sampler settings H3 switches off, so that is known before a
generation is spent rather than after; Easy Gen gets the same warnings on its strip and the
same family selector (it already loads models.js and pipeline_caps.js).
Verified live in the Cutting Room: the family toggle flips requirements, ports and hints
both ways and persists per project; tags renumber on reorder; the H3 chips replace the
Best-FaceID chip on H3 and hand back to it on LTXAV.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
H3 packs reference blocks into the sequence itself (own position grid, never denoised,
never rendered), which is the thing Best-FaceID approximated on LTX with overlap tokens
plus a trained ArcFace projector. On H3 there is nothing to project and nothing to slice
back off, so this is the model's own feature, not a port of that one.
A reference has to reach the model twice and in the same order both times: as
presentation (Qwen sees "<Picture 1>: <vision block>" so the prompt can name it) and as
a packed latent block. Those encodes happen in different nodes, so the ordered spec
travels between them:
- Studio owns the CLIP: resolves the reference list, bakes the presentation via the
tokenizer's minimax_ref_items, and records what it presented as funpack_h3_refs on
the conditioning. Reference media and a first-frame anchor are separate conditioning
modes, so source_image steps aside (loudly) when references are present.
- Chain Sampler owns the VAE: encodes exactly that list, in that order, sized against
the scene's canvas, and attaches minimax_refs. Applied on every scene branch so an
identity holds across the whole chain, and cached per run so it is encoded once
rather than once per scene.
Order is the contract — drop or reorder an entry and "<Picture 2>" points at a
different reference than the text encoder saw. Anything that cannot be encoded is
dropped AND named, and an audio reference lost for want of an audio_vae says out loud
that later <Audio j> labels have shifted.
New optional audio_vae input on the Chain Sampler (audio references only; image
references need just the main vae). Video references are refused with a reason rather
than half-implemented — they need 2 fps frame sampling and a video loader FunPack does
not own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Day-0 groundwork for Comfy-Org/ComfyUI#15224 (MiniMax H3). New minimax_h3.py is the
one place that knows how H3 differs from LTXAV; everything else consults it.
Four divergences that failed silently before:
- Audio latent axis. LTXAV audio is [B,C,T,freq] (time on dim 2, same as video);
H3 audio is [B,32,stereo,T]. Both 4-D, so only the model family can say which
axis is time. The Chain Sampler now carries a per-stream time axis through
continuation, blending, tail crops and JoyAI audio memory.
- Frame grid. LTXAV 8k+1 -> k+1; H3 17k+5 -> 5k+2. H3's downscale_index_formula
is the index map, not the count map, so the old uniform formula under-counted
by ~20%. Frame counts now come from vae.downscale_ratio[0], which reproduces
LTXAV's answer exactly.
- Attention patches. LTX calls optimized_attention through its module; H3 binds
it at import, so FunPack's temperature/capture patch was inert on H3. It now
rebinds every module holding a reference, and position-indexed work (BachVid
K/V, attn importance) sits out H3's pre-reshaped [1,heads,S,D] layout.
- Batch size 1. H3's DiT refuses a batched forward and comfy batches cond+uncond
for cfg != 1.0. install_batch_split runs them one at a time, keeping cfg live.
Guides become keyframe pins: H3 packs condition rows into the sequence instead of
appending a masked latent frame, so no tail is appended and none is cropped. The
layout only accepts a first/last pin, and a mid-clip request is refused up front
instead of raising inside PackedLayout.
Studio: reference images route through Qwen3-VL's images= kwarg (the Gemma3 vision
probe returns False for H3 and was dropping them); the attn2 K/V direction patch
reports that H3 has no cross-attention to hook instead of installing a no-op.
Features that depend on LTX transformer structure (bounded attention, Best-FaceID
identity transfer, v2a_grad_scale, segmented detailing, second-pass latent ops) are
now switched off explicitly on H3 with the reason printed, rather than each one
discovering its own missing attribute mid-scene.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Punchy frame_rate mult held over the first/last slice of a scene's
denoise progress, easing to natural in the middle - same progress-
from-sigma mechanism as pulse, shaped as a single edge ramp instead
of repeated segments. Wired through build_enhancements (single-scene),
the Scene Chain Sampler's per-scene wrapper dispatch, and both Studio
and Editor temporal-style dropdowns.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Adds taste_nearest_prompt on the Scene Chain sampler. On every liked rating the
Refiner now records a (pooled-conditioning fingerprint -> liked direction) entry in a
ring buffer beside the global liked_dir. When the toggle is on, each scene retrieves
the similarity-weighted direction of its nearest rated prompts and uses that for
embed_guidance / score_slider instead of the one global average, falling back to the
global direction when nothing is close enough. Non-parametric (cosine lookup + vector
mean, no extra model forward, no value-function collapse risk). Wired into the Editor
Guidance panel. UNVALIDATED LIVE.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Editor ▸ Settings ▸ Editor gains a 'Shortcut revolver' toggle (+ 'Random' sub-option).
When on, each shortcut with several replacements draws from a per-shortcut queue —
sequential (first…last, then restart) or shuffled once per cycle — and nothing repeats
until the whole set has fired. Off (default) keeps the seeded random pick unchanged.
- State is a sidecar (shortcut_revolver.json) next to the shortcut DB: settings +
per-shortcut remaining queue + replacement-set fingerprint (editing a shortcut's
replacements restarts its cycle; changing either setting resets all cycles).
- Generation commits the draw (refine_v2 positive prompt); every preview path
(parse_timeline, split mapping, fragments) peeks the same stored queue without
saving, so previews show exactly what the next Generate will use.
- Batch Training variants each commit a draw: N variants sweep N consecutive
chambers instead of seed-rolled repeats.
- FunPackStudio.IS_CHANGED returns nan while the revolver is on so fixed-seed
re-runs aren't served from ComfyUI's node cache (the next chamber must fire).
Verified live on local ComfyUI: settings round-trip, seed-independent stable peeks,
sequential cycle REP_ONE→TWO→THREE→ONE across processes, peek-after-commit shows the
next chamber. 9 new unit tests; full suite has the same 20 pre-existing failures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
'Awful' / 'Wrong appearance' banked on intrusion semantics; now any rating
whose profile reward marks the run a bad OUTCOME (<= -0.25: Missing
quality, Missing details+quality, Missing action+quality, Missing
details+action+quality) also feeds the negative bank - degradation is
visible in the latent, so steering off that attractor is meaningful.
Near-miss ratings with positive reward (Missing details +0.35, Missing
action +0.05) stay excluded: those latents are mostly-wanted content and
banking them would repel future runs from material the user asked for.
Decision moves into negative_memory.is_negative_profile (intrusion keys OR
reward threshold, robust to future profiles); promoted entries now carry
the rating key as provenance for severity-weighted steering later.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Store what went WRONG, not just what was asked for. Ratings 'Awful' and
'Wrong appearance' promote that run's final video latent (fp16, batch
squeezed) + mean-pooled conditioning into a per-key negative bank
(<key>.negative_latents.pt, ring buffer of 8). The Chain Sampler writes a
pending candidate at the end of every keyed run; the rating that scores
that run promotes or discards it - same pairing as the x0 snapshot, and
the stem glob in delete_refinement_key covers both new sidecars.
Sampling (dynashift toggle + strength/threshold, off by default):
per late step each current video frame is pooled and compared against
every banked negative frame (alignment-free in time, so chain position /
guide tails don't matter; other resolutions are skipped). Frames matching
above threshold get the matched negative frame's aligned component
subtracted (projection removal, coef clamped >= 0), scaled by
above-threshold margin - steering self-releases once the unwanted feature
is gone. Negatives are weighted by prompt similarity so bad gens from
unrelated prompts steer less. No uncond pass, no extra model forward;
audio byte-identical (video span slice + cat). Wrapper rides the tagged
per-scene lifecycle (leak-strip covered) and threads through batch
training.
Editor: DynaShift knobs land in Engine Settings > Guidance same-change
(sampler_inputs pass-through, no backend change), verified rendering live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Block steering: run-averaged block activity is architecture-determined
(varies in the 4th decimal run-to-run), so no scoring scheme - pole EMAs
or reward correlation - could extract a rating signal from it. The idea
needs within-run per-step measurement to have a chance; that is a
research project, not a feature. Removed: module, sampler wiring, the
two trailing widgets (positional reference mapping unaffected), the
Refiner pairing hook, editor knobs, tests. Stale block_snapshot/
block_profile sidecars are inert and covered by the existing
delete_refinement_key sweep.
FunPackNVFP4ModelLoader: the FP4 GEMM path was real (3.35x at 16k
tokens) but end-to-end the user's runs are dominated by everything the
loader cannot touch (attention, prep, encode - the sampler node is
~23s of a ~60s run) while dynamic input quantization added a fixed
per-layer-call toll. Net observed: overhead. Users are better served
by dedicated NVFP4 loaders. Removed: module, registration, tests.
Kept (independently useful): the chain sampler's phase timing report +
quantized-layer census, the nested-AV x0 snapshot fix, and the sm_120
masked-attention work.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RLHF at the model level: which REGIONS of the transformer were doing the
work when a generation came out good or bad, and steer accordingly.
1. Measure: tagged forward hooks record each block's relative video-residual
delta per model call (near-zero cost; covered by the existing leak-strip).
2. Attribute: the end-of-run fingerprint is staged (block_snapshot.pt, same
stage-then-pair convention as x0_snapshot) and paired with the rating
reward on the next Refiner cycle into liked/disliked/all-runs EMA
profiles; when output_guidance's value function is trained, an innermost
model-function wrapper additionally credits blocks per step by
(value delta) x (centered activity) - converges in a handful of runs.
3. Steer: vx' = vx_in + g_b*(vx_out - vx_in) on the video stream only,
audio untouched; scores zero-mean/unit-max so emphasis is redistributed
rather than inflated, gains hard-capped at +/-10%. Exact no-op until the
profile has 2+ paired runs; blocks at gain 1.0 get no hook at all.
Chain Sampler toggle + strength (append-only widgets), exposed same-change
in the editor's Chain Sampler Guidance section via the sampler_inputs
bridge. New sidecars ride the defensive <key>.* sweep in
delete_refinement_key, so atomic key deletion covers them.
Tests: toy AV-block model driving the real hooks (recorder ranking, steer
video-only scaling + native path for unit gains, credit wrapper, snapshot/
profile/scores/gains roundtrip, mild-positive band, depth-mismatch reset,
leak-strip tagging). Full suite failure set identical to the pre-existing
baseline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The free video-only unsharp-on-x0 sharpening (temporal high-pass between
consecutive denoised predictions, no extra model eval) only existed on
Hybrid Euler 2S. Distilled Flow — the recommended sampler for this
pipeline (deterministic, clean audio) — had no equivalent, so the
"quality phase" of the actual production sampler had no sharpening path
at all.
Ports the same _apply_quality_sharpness call into
sample_funpack_distilled_flow, gated to the existing final-correction
(Heun) steps. Threaded through FunPackDistilledFlowSampler's
INPUT_TYPES/get_sampler, Studio's _build_one_sampler, and the Editor's
sampler_panel.js Distilled Flow section — verified live end-to-end
(UI -> studio_settings JSON -> backend build call).
Replace absolute reward regression with Bradley-Terry pairwise ranking
(DPO-style) so the value function learns relative preference between
sampled pairs rather than fitting noisy absolute reward floats.
Add a 3-member bootstrap ensemble; gradient() and ascend() scale by
inter-member agreement so guidance backs off in regions the ensemble
hasn't seen data for, instead of confidently extrapolating (the
static-character collapse this was meant to guard against).
Loosen the "Wrong but loved" exclusion in conditioning.py so a strong
loved_modifier signal is no longer filtered out by skip_value_function
- "Wrong but loved" and "Missing but loved" now train the value
function, per the stronger-signal precedence requested.
Legacy single-net checkpoints migrate transparently on load (weights
remap into all ensemble members, rating buffer preserved intact).
New LatentValueFunction (value_function.py) mirrors OnlineValueFunction but is
trained on a pooled snapshot of the sampler's own predicted output (x0_hat)
instead of the input conditioning, borrowing NoiseTilt's core idea (reward
gradient from the model's own prediction) without its SDE noise-term trick,
which has no analogue in this deterministic-ODE sampler.
New output_guidance / output_guidance_strength toggle on the Chain Sampler
(off by default): per-step correction installed as the outermost
model_function_wrapper (on top of embed_guidance/score_slider), same
near-zero-forward-pass shape as embed_guidance. End of run writes a small
pooled snapshot per refinement key; the next rating cycle trains the value
function on it via a new call site in conditioning.py.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Custom keys (e.g. a 'cs'-tagged cumshot shortcut) only ever trained
through the Movie Editor's per-scene multi-rating path, which requires
BOTH a multi-scene run AND explicit per-scene ratings from the editor.
Most real generations are single-scene with one overall rating, so
those keys were never created at all -- confirmed on a live instance:
zero custom key files existed despite weeks of use.
Single-scene runs already carry their fired keys in
previous_run["scene_refinement_keys"] (Studio always computes this,
the Movie Editor always forces split_by_transitions=True). Train any
non-default keys found there too, additively -- the default key keeps
learning exactly as before; this doesn't replicate the multi-scene
path's exclusivity (custom key trains instead of default), since that
would mean restructuring the heavily-relied-on single-scene rating
path without separately testing the exclusive behavior there.
Structured Diffusion Guidance (arXiv:2212.05032)-style upgrade: instead
of tokenizing each sentence-half just to COUNT tokens (an approximate
boundary within one shared tokenization, vulnerable to BPE merge drift
at the seam), encode each half SEPARATELY through the real text encoder
and concatenate the two conditioning tensors. The split point is now
exact by construction, and the two spans can never share a token even
before any masking is applied. Same toggle, same metadata field
(funpack_bound_split_tokens) the sampler already reads — no sampler
changes needed.
MultiDiffusion (the other planned "same family" technique) needs
multiple forward passes per region (~2x+ cost) and is skipped per the
project's low/no-overhead bar, pending explicit user sign-off.
_V2_PERSISTENT_ENCODE_CACHE survived for the entire ComfyUI process
lifetime, across every generation request, keyed partly by id(clip) —
a CPython object identity that can in principle be reused for an
unrelated object after garbage collection. Conditioning must always
trace back to what's live in the current request; nothing should be
served from a cache that outlives it. The per-call encode_cache dict
(created fresh per sample() invocation, never escapes it) already
covers legitimate within-request de-duplication and is unaffected.
Live testing found decay=0.9 produces garbage at any strength on our
8-step schedule: the EMA barely forgets, so it stays anchored to the
very first (near-pure-noise, wrong-scale) step's direction for nearly
the whole run. decay=0.5 forgets that stale step within 2-3 iterations
and tested clean even at strength=1.0. Paper's longer-schedule decay
assumptions don't transfer to a few-step distilled run.
Masks text cross-attention (attn2 only, never audio or self-attention)
so the left half of the frame can only attend to subject-1's prompt
tokens and the right half only to subject-2's, fighting attribute/
anatomy bleed between two figures in one frame. Pure attention masking
via PyTorch forward-pre-hooks, no extra forward pass, model-level so it
works on any sampler.
Studio always tags each scene with an approximate token-split boundary
(sentence-count heuristic + real tokenization via clip.tokenize) as
conditioning metadata — cheap, no-op unless the new
bounded_attention_enabled toggle on the Scene Chain Sampler is on.
Opt-in, off by default.
Keeps a running EMA of the per-step ODE direction and blends the current
step toward it once sigma drops below mg_sigma_threshold — the
complementary window to ALG's anchor blur, which only acts above its own
threshold. EMA accumulates every step (free) so there's no cold-start
when application kicks in; video-only, audio untouched. Opt-in, off by
default.
Adaptive Low-Pass Guidance (arXiv:2506.08456): blurs the i2v anchor frame
while sigma is in the near-pure-noise region, then swaps back to the sharp
anchor, to stop the model shortcutting to a near-static video that just
matches the reference image. Opt-in, off by default, no-op without an i2v
anchor. Swaps model.latent_image per step (the same attribute ComfyUI's
inpaint compositing already re-pastes every step) instead of monkeypatching
comfy internals.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Project-scoped $name variables resolved DEAD LAST in refine_v2 — after
shortcut-expand and the transition split — so a value containing a comma or
trigger word can never move a scene cut (order: shortcuts -> splits -> vars).
Recursive (a value may reference other vars), cycle-safe (self-reference left
literal, no infinite loop), undefined -> literal $name.
- templates.resolve_variables: recursive, depth-guarded, cycle-safe resolver
- refine_v2: resolve base prompt (single-scene) + per-scene texts (post-split);
variables threaded via studio_settings.refiner (same channel as scenes)
- builder/server/timeline: carry project.variables into the build; persist
variables + prompt_templates on the project
- Composer: templates dropdown + Save above the prompt; collapsible Variables
editor (Add/Remove rows) below it, with live undeclared/cycle hint
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Symmetric to the anchor (prepended): a project-level postfix appended to
each scene's conditioning. Lives next to the negative prompt in the project
inspector with its own enable toggle; never part of the verbatim global prompt.
- timeline: Project.postfix + postfix_enabled, effective_postfix(), carried in
build_generation_scene_segments and folded into the run fingerprint.
- conditioning: split_scenes_from_segments expands the postfix (keys fold into
every scene); the structural fold appends it after anchor+scene text.
- inspector: Postfix field with inline enable toggle after Negative prompt.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Root cause the user pushed on: the editor flattened all scenes into ONE prompt
string and Studio re-derived boundaries by parsing text, so a carry scene with no
transition needed an injected 'scene N' delimiter. The editor already KNOWS every
boundary (each generative unit is a scene) — it never needed to smuggle markers
into the prompt.
- timeline.build_combined_prompt: no longer injects anything (verbatim anchor +
scene texts). New build_generation_scene_segments(project) lists the boundaries
the editor already has (each unit's raw text, prefixed with its REAL transition
so effects survive; carry = plain text).
- conditioning.split_scenes_from_segments: canonical split from an explicit scene
list (reuses split_scenes per item for expansion/keys/effect, folds the anchor),
same output shape as split_scenes. refine_v2 gains scene_segments and uses it
when present (count-preserving: a text-less i2v scene keeps its slot, no anchor
shift); falls back to text-splitting only for raw node-pack use.
- server/builder: editor sends scene_segments -> studio_settings.refiner.scenes.
Net: the generation prompt Studio encodes is now exactly what the user typed —
no 'scene N' anywhere, in the prompt or the conditioning. Tests updated to assert
the verbatim prompt + structural segments; new tests for split_scenes_from_segments
(boundaries, empty-scene preservation, anchor/key fold). Fallback path unchanged
(stash-verified zero regression). NOTE: montage generation needs GPU verification.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Multi-scene bug: the Movie Editor injects a generic 'scene N' separator before
any scene that lacks a leading transition trigger (build_combined_prompt), purely
to force a boundary. split_scenes was splitting on those markers with placement
'start', which KEEPS the marker word in the new scene's text -> 'scene 1',
'scene 2', ... got encoded into every scene's conditioning. With a text-less
leading i2v scene this also produced a label-only first scene (renders as the
anchor with no prompt) and shifted each scene's content down by one.
Fix in the one canonical splitter: a generic 'scene <N>' label (not a real
custom trigger in the transition DB) now cuts SILENTLY -> boundary only, dropped
from both expanded (generation) and raw (editing) text. Real user/DB triggers
keep their configured placement, so 'cut'/'qcut' etc. are unaffected.
Regression test asserts the markers split but never appear in any scene's
expanded/raw text, and that a real trigger still leads its scene.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Training-free character carry across scenes/sessions (arxiv 2510.21696),
built on the existing LTXAV attn-hook infra. Capture raw K/V at IDENTITY_BLOCKS
during the mid-sigma window into a per-refinement-key temp bank; promote to a
blessed bank on liked ratings (EMA-merge, alongside bless_attn_weights). On
later runs with a blessed bank, lerp the running gen's K/V toward the blessed
identity at matched token positions (packed-/RoPE-safe analogue of the paper's
concat-attend). Wholesale inject at identity blocks (no spatial mask yet).
Exposes a shared kvlock_scale multiplier in transformer_options for the Phase 2
KV-Lock variance scheduler to drive injection strength per step (1.0 = off now).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Route the conditioning variant from per-config outcome history, via the one
charter-safe surface: the INTERNAL value-function ascend+search (not user knobs,
so no WYSIWYG / hidden-settings violation). Sampler-owned visual ops stay on the
sampler (boundary law kept).
Directly serves the original complaint (Perfect not locking, Awful reproducing):
- _v2_path_conditioning_plan: matching-arm Beta posteriors -> liked-rate mean ->
"lock" (>=0.6: hold the liked conditioning, skip the stochastic search),
"explore" (<=0.4: widen the search to flee a disliked region), else "normal"
- value_function.search gains eps_scale to widen/tighten the search radius
- threaded conditioning_plan through ascend -> apply_scene_refinement_keys ->
finalize; computed once per run from path_outcomes + prompt concepts; logged
Only active once the per-key VF is ready, since it rides the existing ascend path.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Upgrade the seed explore/exploit gate from a flat reuse chance to a proper
bandit over the path-outcome arms:
- _v2_path_arm_beta: Beta posterior over an arm's liked-rate (liked>=0.3 success,
<0.0 failure, neutral ignored; Laplace prior)
- _v2_path_explore_decision: Thompson-sample theta per concept-matching arm vs a
fresh Beta(1,1) explore arm; good arms exploit, dislike-dominated arms route to
a fresh seed ("do something else")
- wired into _v2_choose_successful_seed (falls back to the flat gate with no data)
- shared _v2_path_matching_arms helper; _V2_KEY_REWARD + liked/disliked consts
5 tests incl. a statistical one: a strongly-liked arm rarely explores, a hated
arm almost always does.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make path_outcomes ACT via the one Studio->sampler lever wired today: the seed
(FunPackStudio.run -> _v2_choose_successful_seed -> seed output -> sampler). No
new data paths invented.
The existing seed chooser only exploited liked seeds with zero avoidance — the
headline gap. Now:
- recorder tags each seed sample avoid = reward<0 and not skip_value_function
(Wrong-* excluded: the seed was fine, the words/identity were off)
- _v2_path_disliked_seeds: seeds that gave a low-quality result for matching concepts
- _v2_choose_successful_seed redraws a disliked fallback, drops liked-then-later-
disliked seeds from reuse, and reports how many it avoided
Broader config levers (guidance/lora) need new wiring and are deferred pending
graph-topology confirmation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Audited every learning sink in the V2 rating flow. Rating-learning and the
Absolute store already honored skip_learning, but two feedback sinks ran gated
only by feedback_prompt — so a 'Just ignore it' run with typed feedback still
wrote advisor_feedback_history + intent_expansion_memory.
Add _v2_rating_is_discard (forget rating or auto-discarded refusal) and gate
both feedback sinks on it, without touching the active continue/fresh_prompt
modes that legitimately keep feedback.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reframe ratings from local steering into global routing: every rating —
not just successful ones — is recorded as a labeled outcome on the per-scene
config "arm" that produced it, so a later planner can route away from arms
that keep generating disliked results.
- _v2_path_signature: arm = family + concepts + guidance config, seed EXCLUDED
(seed is the random draw within an arm); md5 arm_id collapses identical configs
- _v2_update_path_outcomes: learns from EVERY rating, decaying counts so recent
behavior dominates; records seed-level samples for later seed routing
- hooked at the shared per-scene learning hub + single-run path; stored in
per-key state under "path_outcomes"
Pure instrumentation — no generation behavior change yet (Phase 2 = Studio planner).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The canonical generation/preview/key splitter (split_scenes) only treated a
shortcut as a scene cut when the shortcut's TRIGGER word was a key in the
transition DB. But split markers are commonly keyed by the EXPANSION phrase: a
"cut" shortcut expands to "Scene cut.", and the split marker is "scene cut.".
So the lookup `custom_map.get(_norm(raw))` missed, the shortcut was treated as
plain content, and NO cut was made — split_scenes returned one scene for
"A cut B cut C" while the verbatim editor split returned three. That disagreement
collapsed the timeline's middle scenes (and would have generated one scene).
Fix: when the trigger word isn't in the DB, also match the shortcut's EXPANSION
(`custom_map.get(_norm(exp))`). The generic "scene <N>" label is still NOT matched
on expansions, so an incidental "scene 2" inside a content shortcut never cuts.
Verified against the user's on-disk cut/qcut defs: split_scenes now agrees with
the verbatim split (2 scenes for cut x2 / qcut x2 / mixed). Added regression tests.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
With exactly one transition trigger in the prompt (one "cut", one "qcut", etc.),
split_scenes produced anchor + 1 scene — so len(canon_scenes) == 1. The
previous guard `> 1` skipped the clean-text encode path, leaving
output_conditioning built from the raw prompt_to_encode which still contained
the trigger word. The CLIP encoder received "cut" and the model generated it
in audio.
Changing to `>= 1` ensures any split with at least one clean scene uses the
properly stripped scene text, even when there's only one scene to encode.
The chain sampler receiving a single conditioning entry is fine — it just
renders a single pass with no chaining.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Normalization is now available in both Hybrid Euler 2S and Distilled Flow
samplers, making the standalone Normalizing sampler redundant. Removes the
node class, its sampler function, two private helpers (_parse_norm_factors,
_apply_av_norm_factors), the Studio/Movie Editor "Normalizing" type option,
and the defaultNormalizing() factory. Stale "same as the Normalizing sampler"
tooltip references in both sampler nodes are updated.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Deletes the FunPack Scene Builder node and the Studio Scene Builder feature
— the per-key scene database (scene_db / universal_memory / saved scenes),
the Studio "Scene" tab, scene-phrase authority over classification, and the
scene-database wildcard cleanup. It was redundant: helpful on paper but
effectively unused, undrivable from the Cutting Room, and a risk to shortcuts.
Removed:
- node registration (__init__.py, funpack.py) + web/funpack_scene_builder.js
- templates.py: scene-DB storage layer (load/save/normalize/empty_scene_db,
SCENE_DB_VERSION/SCENE_STATE_FIELD/SCENE_CATEGORY_SOURCES, scene_store_path),
all scene_* helpers, resolve_scene_database_wildcards, the FunPackSceneBuilder
class, and the /funpack/scenes* routes
- conditioning.py: _v2_scene_builder_db / _v2_scene_memory_item /
_v2_apply_scene_builder_authority / _v2_scene_builder_category_for_entry /
_v2_sync_scene_builder_memory / _v2_resolve_scene_builder_wildcards (+ wildcard
helpers + _v2_scene_key), the scene_db threading through classify/learning,
the scene_category_locked phrase fields + their training branch, the
state["scene_builder"] preservation, and the Studio scene_builder prompt mode
- web/funpack_studio.js: Scene tab, SB_MODES, scene_builder settings, renderScene,
fetch/saveScene, the Session Scene-Builder section, and the Refiner banner/notice
- tests/test_scene_builder.py (whole file) + scene-builder tests in
test_v2_category_alignment.py; the SceneBuilder example workflow + its doc;
README link; Scene Builder mentions in docs/FunPackStudio.md
Kept intact: shortcuts (apply_prompt_shortcuts / shortcut DB), transitions and
the canonical split_scenes splitter, and FunPackRefinementKeyLoader. Studio now
always uses the connected positive_prompt. No new test failures (the 16/97
remaining in tests/ are pre-existing env/sampler failures); movie_editor 140 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Reworks the "Wrong appearance" rating from a (now-removed) prompt-repair
signal into a conditioning-level consistency anchor (option b). "Wrong
appearance" now means "character consistency wasn't preserved" and steers
the next generation back toward the established character.
Capture (_v2_update_appearance_anchor, wired into both the single-run
learning path and the shared per-scene suite _v2_learn_scene_into_state):
- good rating (like/nailed_it/loved_it/loved) → store the rated run's
conditioning as the per-key "blessed appearance" (overwrite = last good)
- wrong_appearance → store the rejected run's conditioning as the "drift"
global_state is already per-refinement-key, so both live with that key.
Apply (_v2_apply_conditioning_memory_impl, after the intent-family delta):
when the user rated the previous gen wrong_appearance, gently pull this gen
toward the blessed appearance (lerp 0.05) and repel from the drift (0.04),
reusing _v2_apply_conditioning_payload / _v2_repel_conditioning_payload. No
ramping; the existing 7.5% delta cap bounds the net effect. No-op when
nothing is stored or shapes mismatch.
Drops the dead wrong_categories field (zero consumers after Part A) and
keeps skip_value_function (the anchor is a conditioning nudge, not a reward).
Tests: 7 new unit tests (storage + overwrite + skip + pull/repel + idle +
non-trigger); updated test_wrong_appearance_rating_is_available for the new
semantics. Zero new failures vs HEAD (the 10/31 remaining are pre-existing
env failures from missing server/comfy modules).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Removes the now-unreferenced repair-only machinery left behind after
Stage 2 (1/3) neutralized the application:
- _v2_prompt_repair_text_allowed
- _v2_emphasized_prompt
- _v2_repair_prompt_for_missing_axes and the repair-candidate helpers
- perfect_repairs / rejected_repairs storage + perfect-repair eviction
note in the intent-family learn path
Shared helpers still used by intent-alignment (e.g. _v2_repair_intent_roots,
_v2_repair_candidate_matches_intent, _v2_prompt_contains_text) are kept, as
is the repair_feedback / active_repair_axes "what's missing" signal the
advisor reads.
Deletes the 11 category-alignment tests that exercised the removed methods
directly. No new test failures (the 10/31 remaining are pre-existing env
failures from missing server/comfy modules).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Neutralize the repair application in refine(): the intent-aligned prompt is
encoded as-is — perfect-repair phrase injection, missing-axis repair, and the
emphasis pass are gone. The advisor remains the only prompt-rewriting path, and
repair_feedback/active_repair_axes is KEPT (it's the 'what's missing' signal the
advisor still reads). Drop the prompt_repair param from refine(), INPUT_TYPES,
FunPackStudio, and both UIs (web + engine_settings).
The dead repair-only methods (_v2_repair_prompt_for_missing_axes, perfect-repair
phrases, etc.) come out in the next commit; shared helpers like
_v2_repair_intent_roots (used by intent alignment) stay.
Deleted the now-obsolete test_repair_candidates_saved_as_json_lists (tested the
removed injection). No new test failures vs dev.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Generation now reads split_scenes, so the duplicate _v2_split_prompt_by_transitions
method has zero callers — deleted. bridge.parse_timeline (the 'what Studio sees'
preview) now derives from split_scenes too, so the preview's scene count/boundaries
match generation exactly (no spurious 'split produced N but timeline has M' warnings).
The lossless RAW editing split (parse_timeline_verbatim / split_timeline_verbatim)
stays separate — it has a genuinely different lossless requirement (round-trips the
exact typed text). It still uses the old offset-projection path; converting it to a
lossless piece-walker is the one remaining Stage-1 loose end.
Tests: 33 + 140 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Dropping the generic scene-N pattern broke the Movie Editor: build_combined_prompt
(for_generation) INJECTS 'scene N' markers into the generation prompt to force a
split for any unit whose text lacks a leading transition trigger. Without that
recognition the second unit never split -> 'scene 2 exists only on the timeline,
not generated'.
split_scenes now matches the generic scene-N label again, but ONLY in its
literal-text scanner (never over shortcut expansions). So an editor-injected /
user-typed 'scene 2' cuts, while a 'scene 2' buried inside a shortcut expansion
does not — fixing both the user's phantom split AND the broken montage.
For the reported prompt the piece-walker now returns 2 scenes (was 4), matching
the old generation splitter's 2 -> the guard should pass and keys attribute while
the chain/guides keep working.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Generation now builds its per-scene encode texts AND its per-scene keys from
the SAME split_scenes(raw) call (expanded scene text via the existing
_v2_transition_scene_texts anchor-prepend). Attribution, preview, reset and
generation are now one split, so scene count/boundaries can't disagree — the
SCENE SPLIT MISMATCH guard + diagnostic are deleted.
This also fixes WHERE generation cuts: it used to split the EXPANDED prompt
(boundaries from whatever trigger words survived expansion — e.g. the 'cut'
in 'Scene cut.'), not at your qcut/cut markers. Now it cuts at the real raw
triggers, so scene 1 = qcut..(keyA,keyB), scene 2 = ..(cs), as intended.
Behavior-changing on the generation path -> needs an in-app smoke test.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The verbatim/offset-projection splitter mangled real prompts: long shortcut
expansions desynced the raw<->expanded offset map (empty anchor, qcut+cut
lumped), the hard-coded generic 'scene N' pattern split on incidental text
('scene 2' inside an expansion), and re-scanning expansions re-triggered on
words like the 'cut' inside 'Scene cut.' — 4 phantom chunks instead of 2.
split_scenes now walks _expand_with_map's ordered pieces ONCE: a boundary is
created only by a real transition trigger (a shortcut whose trigger word is in
the transition DB, or a literal typed trigger); expansion CONTENT is never
re-scanned; the generic scene-N pattern is dropped. Each keyed shortcut adds
its key to the current scene; the leading run is the anchor (keys fold into
all). Returns raw + expanded text + keys + effect per scene.
For the reported prompt this yields exactly 2 scenes: [keyA,keyB] and [cs],
with 'scene 2'/'Scene cut.' no longer splitting. Tests updated to use real
triggers + a regression test for the expansion-trap. 21 + 140 pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
