LTX 2.5 reuses LTXAVModel/LTXBaseModel behind config flags, so the samplers,
the identity-transfer wrappers and the guide path all bind to it unchanged.
Two places assumed 2.3 specifics, and both would have gone wrong quietly.
The AV channel split was a hardcoded width table. From 2.5 on comfy reads the
video/audio projection widths off the checkpoint rather than assuming them, so
a checkpoint can ship a pair the table has never seen — and an unrecognised
width made ltxav_video_channels return the FULL width, which silently steers
the audio text context along with the video one. The only symptom would have
been degraded audio. The split is now measured off the live model when one is
on hand; the table stays as the fallback.
Decode-time noise only ever reached VideoVAE.decode. 2.5's diffusion decoder
takes no timestep and seeds itself, but it is an nn.Module, so stamping the
attributes onto it succeeded and was then ignored — a knob that reads as live
and does nothing. It is now capability-tested against the real contract
(hasattr decode_timestep, not a class name) and says so when it cannot apply.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three things that were each locked to one implementation detail:
- steps + schedule were a KSampler-branch feature, though every sampler here
walks a sigma list handed in from outside. They are now pass-level, so
Distilled Flow and Hybrid Euler 2S can run a named schedule (and the second
pass can be computed rather than typed) in both frontends and on the node.
- the project's Frames field snapped to the model's grid on every family. On
H3 that is required, but its arrows moved by 1 and the snap put the value
straight back; on LTX the builder rounds up by itself, so the snap only
fought the user. H3 now steps a whole 17 frames per arrow, LTX keeps what
you type and walks 8k+1 with its arrows.
- ALG lived inside the Distilled Flow step loop, so it was unreachable with a
stock KSampler. Its blurred/sharp swap is decided by the step's sigma, and
sigma is an argument of every model call, so a denoiser proxy drives it from
outside any sampler. New alg_anchor toggle on the Chain Sampler drives the
in-loop version too, so there is one control wherever you are.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every Editor preference was per-MACHINE: autocomplete, shortcut ideas, the anchor
toggle and the custom-i2v bypass in browser localStorage, and the shortcut
revolver in a server-side sidecar next to the shortcut DB. A freshly rented box
has neither, so all of them silently revert to their defaults and have to be set
by hand again — the revolver's Random mode every single time.
They now ride along in the project file (Project.editor_settings), and opening a
project puts them back. The revolver is still a server setting shared by every
browser and by generation; the project only restores it, and only POSTs when the
value actually differs, so opening a project that already agrees never restarts a
live cycle.
Two cases that would otherwise leave the feature useless for an existing setup:
* A project saved before this existed, opened on a configured machine, adopts
the machine's current settings instead of staying blank — including reading the
server's current revolver mode. Nothing changes in the moment; it just stops
the project from travelling empty.
* setEditorSetting mirrors into the project BEFORE its unchanged-value return.
Re-picking a value you already have is a no-op, so without this the settings
you never touch again — exactly the ones you'd lose — would never be recorded.
An ABSENT key still means "leave this browser alone": loading an old project must
never overwrite a configured setting with a default.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Engine settings were unreadable: every experimental knob carried a paragraph
about cost figures, sampler caveats and failure modes, so nobody read any of
it. The hint is now ONE sentence saying what the setting does, and the
paragraph moves behind a collapsed "Details" link — still the only place some
of those numbers exist, just not in the way of the checkbox.
cut_opening_frames also learns MiniMax H3. It could never engage there: H3's
anchor is a keyframe condition row rather than a pinned latent prefix, so the
t2v guard read every H3 scene as anchorless, and its video latent sits on a
5k+2 grid an arbitrary cut cannot land on. On H3 the frames now come off the
head of the DECODED batch instead — exact to the frame — with the latent's
audio stream cropped by the same proportion of time, because audio decodes
from this node's LATENT output and would otherwise start N frames early. The
latent keeps its full video stream (off-grid the moment it is cut) and the
scene report says so.
Fixes a latent bug on the way: extra streams were cropped on dim 2 with the
video stream's frame count. On H3 dim 2 is the stereo channel and time is
last, so that would have cropped the audio to mono and left its duration
untouched.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous commit claimed the correction could not reach a stock
sampler. That was wrong. It can: the correction moves from the step to
the prediction, because a rectified-flow euler update is
x*r + denoised*(1-r), so scaling the audio part of (x - denoised) by the
same factor lands the audio on its own grid. A proxy denoiser wraps any
KSAMPLER-shaped SAMPLER and does exactly that.
Tying a model call to a step is the real constraint, and it is checked
rather than assumed: the proxy verifies on every call that the sigma it
was handed is this step's scheduled one, and switches itself off with a
console note the moment that stops holding. Samplers that evaluate twice
per step (heun, dpm_2, dpmpp_2s/sde, seeds_2) trip that on their first
corrector call and are left untouched.
Measured against comfy's real samplers with a perfect rectified-flow
predictor, so every number is pure schedule error (audio error as a % of
the stream's range, 4/8/20 steps):
euler 85/38/14 -> 0/0/0 exact at every step count
res_multistep 69/18/1 -> 21/23/15 helped at 4 steps, HURT at 20
dpmpp_2m 69/18/0.4 -> 20/22/14 same
heun, dpm_2 unchanged (proxy stands down)
The lesson worth keeping: this is a first-order integration error, so a
higher-order sampler already absorbs most of it once the schedule is
fine — which explains why res_multistep and er_sde behaved well on H3 at
20 steps while plain-euler paths did not. At 4 steps nothing absorbs it
(everything sits at 68-85%) and only the clock fixes it, which is the
regime turbo LoRAs put you in.
Not gated on any of this: the user picks the sampler, and the tooltip,
the Editor hint and a per-run console note carry the numbers so the
choice is informed. Also corrects two claims from the previous commit —
"no-op above 20 steps" (it is 14% on euler at 20) and "video is
bit-identical" (the streams share one attention sequence).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
MiniMax H3 denoises video and audio on two different flow schedules
(shift 12 / shift 3) but hands the sampler a single sigma grid, so the
DiT reconciles them by scaling the audio velocity by d(sigma_a)/d(sigma_v)
measured at the START of each step. Exact in the continuous limit; on a
few-step schedule the tangent badly overshoots the chord it stands in
for. On a 4-step shift-12 grid the final step drives the audio ~2.5x past
where its own schedule puts it, which is heard as distortion — the reason
the turbo-LoRA community ships a custom sampler for it.
h3_audio_clock on the Chain Sampler swaps that start-of-step slope for
the chord that actually spans the step, so audio lands on its own grid.
One scalar multiply per step: no extra model call, no extra memory. The
video stream is untouched. Above ~20 steps the two agree and it does
nothing, so it is aimed at turbo / distilled schedules.
Wired into the two FunPack sampler functions H3 can reach (Distilled
Flow, and Hybrid Euler 2S via its CONST rectified-flow path). A stock
ComfyUI sampler cannot be reached from outside its loop and says so
rather than looking active. Off by default, UNVALIDATED LIVE.
Editor: Settings > Engine > Experimental, plus the mirror-image inert
chip — H3-only settings left on against an LTX pipeline now warn beside
Generate the same way H3-dead settings already do.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An image prompt comes back one tag long: comfy builds minimax_token_tags from
the tokenizer's INPUT sequence (embeds.shape[1]) while the conditioning is what
the encoder returned, and the vision block's expansion does not survive 1:1.
The reconcile treated any surplus as corruption and dropped the whole vector —
which on exactly those prompts discards the modality-0 marks for the image span
and leaves the DiT modulating the picture as if it were text.
Position i still means position i, so the tail is trimmed instead. Growing
(appended tokens) is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two independent, silent breaks made an H3 i2v scene generate from the prompt
alone.
1. The Editor always runs split_by_transitions, and the per-scene conditionings
REPLACE the single entry Studio built — but they were re-encoded with no
reference_image and no h3_references. Both the frame-0 keyframe pin and the
Qwen vision presentation were gone before the sampler looked, even for a
one-scene project. The split now carries the anchor onto scene 0 and the
references onto every scene. Gated to H3: on LTX the anchor rides in the
latent, and adding Gemma3 vision to scene 0 would change existing output.
2. A MiniMax H3 Image to Video node hands its first/last frame pins out on its
CONDITIONING output, which had nowhere to go — the sampler's positive comes
from Studio. New optional h3_keyframes input on the Chain Sampler salvages
them: first frame onto the opening scene, last frame re-indexed onto the
closing scene's own final frame (the source node's frame_count is its own,
and a stale index would land mid-clip where PackedLayout refuses it). The
editor auto-wires it for H3 projects.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two things, both from the same debugging session.
v2a_grad_scale is JoyAI's coupling knob and is never installed without
joyai_audio_memory (see the call site), so a left-over value with the feature
off was being reported as an H3 limitation — blaming the model for a knob that
would be equally inert on LTX. Both the sampler's pre-run notice and the
Editor's inert-setting hint now gate on joyai_audio_memory.
The NaN guard only saw the sampler's OUTPUT, which cannot tell "the model
computed garbage" from "the model was handed garbage and propagated it" — and
those have disjoint suspect lists. _sample_chunk now checks the incoming latent
and the schedule first, and says which it was. The schedule check is there
because it is hand-typed and our solvers divide by sigma (er = sigma_next /
sigma): an interior zero or a repeated value is an instant Inf that poisons
every value downstream. A trailing zero stays legal — it is only ever a step's
target, never a divisor.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A chunk that samples to completion and returns NaN/Inf currently sails through
everything: the scene blend spreads it into the already-finished scenes, the VAE
decodes it, and the first thing that objects is ffmpeg's AAC encoder — "Input
contains (near) NaN/+-Inf", naming nothing that produced it, after the whole
montage has been paid for. The video half never objects at all: NaN through
astype(np.uint8) is undefined but silent, so it degrades with a warning.
_sample_chunk now checks the returned latent and raises with the run-phase label
("scene 2/3 · pass 1 of 2"), which stream, how many values and what dtype, plus
the causes worth testing in order — mismatched LoRA, a mis-inferred checkpoint
layout, a dtype the model cannot hold. It raises rather than warns because a
non-finite latent is unrecoverable and blending it forward destroys the scenes
that were still good. One isfinite() per chunk against a multi-second sample is
not a cost worth weighing; a latent the helper cannot read is ignored, never
fatal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A SAMPLER object is only the step function, so picking KSampler used to leave
the pass with no schedule at all unless a sigma list was typed by hand.
steps + schedule now build one from the loaded model's model_sampling, exactly
as ComfyUI's BasicScheduler does. The schedule dropdown is the switch between
the two sources rather than a silent precedence rule: "use_user_sigmas" — the
default, so nothing changes for an existing config — runs the Sigmas field as
typed, and any real scheduler computes instead, leaving the typed schedule
parked in the field to switch back to without retyping it. Studio logs the fact
that the field is being ignored, and steps is only shown while it applies.
Both frontends, same change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
minimax_h3.py shipped tags_match / token_tags_length / extend_token_tags with a
docstring saying a mismatch is a hard IndexError mid-forward — and nothing ever
called them. Studio edits conditioning in many places, and H3 tokenizes reference
images into the same span (ref_image_size "match" scales them to the output
resolution), so the token count moves with the prompt AND with the resolution.
That is why the crash looked intermittent and why editing either one made it come
and go, reported only as a bare "list index out of range" at step 0.
Studio now reconciles before returning: extend the tags when tokens were appended,
drop them when tokens were removed (absent tags the DiT handles, wrong tags it does
not), and print which happened — losing the tags changes how the text span is
modulated, so it must not be silent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every one of them was a real test of a live feature. 18 failed only in a full-suite run and
passed alone, 4 asserted behaviour that was deliberately changed and never got updated.
Ordering (18): each test module built its own partial `comfy` stub and registered it with
`sys.modules.setdefault(...)`, so the first module to import won and every later module's
richer stub was silently discarded — the scene chain sampler then died on `comfy` having no
`.sample`, and the mask-safe attention install bailed at its first guard because
`import comfy.ldm.modules.attention` resolves through the PARENT's attribute, which one
module set and another did not. tests/_comfy_stubs.py now builds that package tree once,
wiring every parent attribute, and test modules attach their fakes as attributes on the
shared modules (per-test, via fixtures, where the fake is only theirs — the idiom
test_detailing.py already used). Same file also stubs `server.PromptServer`, whose absence
made every module that imports templates.py (i.e. anything importing conditioning) fail on
`No module named 'server'`.
Stale assertions (4):
* the "every learning rating" loop excluded non-learning ratings by hardcoded LABEL, so the
later-added `Wrong appearance|loved` and the two control sentinels were asserted to train
phrase memory. Now filtered by profile key, with the reason per key.
* three split_by_transitions tests still used "scene ten" / "cut to" as cut markers. The
unified splitter cuts on a real transition trigger or the generic `scene <N>` label, on
purpose — an incidental phrase must never split a prompt. Rewritten on the current
contract, and they now also pin that the delimiter is dropped rather than encoded.
Verified: 584 pass, and every file passes alone, in reverse order, and with movie_editor's
suite standalone. Mutation-checked the rewritten split tests (disabling the scene-label
pattern fails 4 of 5) so they are not passing vacuously.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
H3 support was written against ComfyUI's PR; it has since merged (57500fc5, shipped in
v0.30.0) and the weights are out. Everything the adapter assumed still holds — audio time
axis, 17k+5 grid, batch-1 forward, the imported optimized_attention symbol, every payload
key — so this is what the release itself changed, not a correction of the reading. LTX-2 /
LTXAV is untouched throughout: a project picks a family and both are first-class.
* i2v anchors now work on H3. LTX writes the image into the starting latent
(LTXVImgToVideoInplace); H3 has no such path — its anchor is a condition row packed
beside the text — so a scene's anchor image becomes a frame-0 keyframe pin instead, and
the LTX node is no longer run against an H3 latent. Studio hands the source image's
pixels to the sampler (it owns the CLIP, the sampler owns the VAE). Pins now MERGE, so
a first-frame anchor and a last-frame guide coexist: H3's own first-and-last mode.
* Frame geometry follows the model: frames per scene snap to the family's grid (LTX 8k+1,
H3 17k+5) in both frontends, in the graph builder, and in the sampler, and H3 renders at
its fixed 24 fps. Picking a family moves the project onto its geometry and says so.
* The two checkpoints are named. fl2va (anchors) and ref2va (references) load identically
and never reject each other's conditioning, so the run reports which mode it is in and
the setup panel spells out the split.
* Pipeline setup is no longer a placeholder: released, with the published filenames.
* Reference images can be encoded at the reference pipeline's 2048px short edge.
Fixes found on the way:
* A video reference lost its soundtrack between Studio and the sampler — Studio had
already announced it to the text encoder as <Audio 1>, so every later audio ordinal in
the prompt pointed one reference earlier than written.
* Easy Gen never knew it was on H3: its model config lives on the project, which the
shared capability check did not read, so every H3 warning silently reported LTX.
* An off-grid scene length failed with arithmetic instead of an explanation.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The graph builder is now parameterized by model family instead of assuming LTXAV. H3
keeps the core's shape but three of its nodes do not apply: LTXVConditioning (stamps LTX's
frame_rate onto conditioning, which H3 has no field for), LTXVConcatAVLatent (H3's own
EmptyMiniMaxH3LatentAV emits both streams, so the latent feeds the sampler directly) and
LTXVAudioVAEDecode (reads an output_sample_rate H3's audio VAE does not define — it would
raise). Requirements, wirable ports, role targets, guided-wiring hints and the port picker
all follow the family, so a loader can no longer be wired into a node the graph never emits.
The family is a Models setting, asked and never guessed from a checkpoint filename: a wrong
guess fails deep inside ComfyUI at generate time instead of in the panel that can fix it.
Video references now work, rather than being refused. FunPack owns the pipeline and already
requires ffmpeg for preview/export, so minimax_h3 decodes reference clips itself — once, at
24 fps — and derives both views from that single decode: the 2 fps timestamped frames Qwen
sees, and the full-rate latent block the DiT packs. A clip's own soundtrack rides with it,
presented as its <Audio j> label immediately before its <Video k>, which is what ties the
two together. Capped at 15s because reference tokens ride every sampling step.
Editor UI: a Reference media view (H3 only — there is no LTX equivalent to grey out) with
per-type 1-based tags shown exactly as the prompt must spell them, reorder controls that
renumber both sides at once, and a soundtrack picker on video entries. Warning chips beside
Generate for the LTX-only sampler settings H3 switches off, so that is known before a
generation is spent rather than after; Easy Gen gets the same warnings on its strip and the
same family selector (it already loads models.js and pipeline_caps.js).
Verified live in the Cutting Room: the family toggle flips requirements, ports and hints
both ways and persists per project; tags renumber on reorder; the H3 chips replace the
Best-FaceID chip on H3 and hand back to it on LTXAV.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
H3 packs reference blocks into the sequence itself (own position grid, never denoised,
never rendered), which is the thing Best-FaceID approximated on LTX with overlap tokens
plus a trained ArcFace projector. On H3 there is nothing to project and nothing to slice
back off, so this is the model's own feature, not a port of that one.
A reference has to reach the model twice and in the same order both times: as
presentation (Qwen sees "<Picture 1>: <vision block>" so the prompt can name it) and as
a packed latent block. Those encodes happen in different nodes, so the ordered spec
travels between them:
- Studio owns the CLIP: resolves the reference list, bakes the presentation via the
tokenizer's minimax_ref_items, and records what it presented as funpack_h3_refs on
the conditioning. Reference media and a first-frame anchor are separate conditioning
modes, so source_image steps aside (loudly) when references are present.
- Chain Sampler owns the VAE: encodes exactly that list, in that order, sized against
the scene's canvas, and attaches minimax_refs. Applied on every scene branch so an
identity holds across the whole chain, and cached per run so it is encoded once
rather than once per scene.
Order is the contract — drop or reorder an entry and "<Picture 2>" points at a
different reference than the text encoder saw. Anything that cannot be encoded is
dropped AND named, and an audio reference lost for want of an audio_vae says out loud
that later <Audio j> labels have shifted.
New optional audio_vae input on the Chain Sampler (audio references only; image
references need just the main vae). Video references are refused with a reason rather
than half-implemented — they need 2 fps frame sampling and a video loader FunPack does
not own.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Day-0 groundwork for Comfy-Org/ComfyUI#15224 (MiniMax H3). New minimax_h3.py is the
one place that knows how H3 differs from LTXAV; everything else consults it.
Four divergences that failed silently before:
- Audio latent axis. LTXAV audio is [B,C,T,freq] (time on dim 2, same as video);
H3 audio is [B,32,stereo,T]. Both 4-D, so only the model family can say which
axis is time. The Chain Sampler now carries a per-stream time axis through
continuation, blending, tail crops and JoyAI audio memory.
- Frame grid. LTXAV 8k+1 -> k+1; H3 17k+5 -> 5k+2. H3's downscale_index_formula
is the index map, not the count map, so the old uniform formula under-counted
by ~20%. Frame counts now come from vae.downscale_ratio[0], which reproduces
LTXAV's answer exactly.
- Attention patches. LTX calls optimized_attention through its module; H3 binds
it at import, so FunPack's temperature/capture patch was inert on H3. It now
rebinds every module holding a reference, and position-indexed work (BachVid
K/V, attn importance) sits out H3's pre-reshaped [1,heads,S,D] layout.
- Batch size 1. H3's DiT refuses a batched forward and comfy batches cond+uncond
for cfg != 1.0. install_batch_split runs them one at a time, keeping cfg live.
Guides become keyframe pins: H3 packs condition rows into the sequence instead of
appending a masked latent frame, so no tail is appended and none is cropped. The
layout only accepts a first/last pin, and a mid-clip request is refused up front
instead of raising inside PackedLayout.
Studio: reference images route through Qwen3-VL's images= kwarg (the Gemma3 vision
probe returns False for H3 and was dropping them); the attn2 K/V direction patch
reports that H3 has no cross-attention to hook instead of installing a no-op.
Features that depend on LTX transformer structure (bounded attention, Best-FaceID
identity transfer, v2a_grad_scale, segmented detailing, second-pass latent ops) are
now switched off explicitly on H3 with the reason printed, rather than each one
discovering its own missing attribute mid-scene.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
_downscale_to used mode="area" — a 2x2 box for the exact-2x ratio both
callers hit. A box loses on both axes at once: its passband droops well
before Nyquist (detail comes back soft) and its stopband leaks (the detail
the upsampler just invented folds back as aliasing). Soft AND speckled is
precisely the reported "sharpen leaves smearing and artifacts".
Antialiased bicubic instead. On a full-band chirp, RMS error against an
ideal brick-wall decimation drops from 0.316 to 0.244 (~23% closer).
Constant input still round-trips exactly, so the DC behaviour both callers
rely on is unchanged.
This is shared with segmented detailing's 'repair' paste, which downscales
its refined crop through the same helper — that path gets the same change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Renaming the combo's choices to core's spellings broke queueing for anyone who
had one of the old ones saved: ComfyUI validates combo VALUES at queue time,
before the node ever runs, so the alias map added with the rename never got a
chance. The three legacy names are back in the list (they resolve through the
alias map at runtime), and the Editor now displays a stored legacy value as the
name it maps to and writes that back, so the dropdown can't show one setting
while another runs.
And that rejection was unreadable. ComfyUI's top-level `error` for a validation
failure is a fixed blob ("Prompt outputs failed validation", details ""); the
part naming the node, the widget and the bad value is in `node_errors`, which
`error or node_errors` never reached because the blob is always truthy. Every
rejected prompt therefore reported the same nothing. Both are formatted now:
Prompt outputs failed validation; FunPackLTXAVSceneChainSampler: Value not in
list — context_window_schedule: 'static_standard' not in [...]
Also: the progress poll no longer imports anything. current_progress() reads
run_phase's label straight off `sys`. It fires every 700ms on ComfyUI's event
loop WHILE the worker thread samples, and that is not a path to do import work
on — the direct read is both cheaper and the correct one, since run_phase keeps
its state there anyway.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three things, all about a run telling you what it is doing.
1. The second pass is now visible. ComfyUI's progress channel carries only
(value, max), so a chain sampling some scenes twice was one anonymous bar.
The sampler publishes a live label ("scene 2/3 · pass 2 of 2") through the
new run_phase module; /api/progress hands it to the editor, which appends it
to the readout, and pass 2 also announces itself on the console the moment it
starts rather than only in the after-the-fact run report.
run_phase keeps its state on `sys`, not in module globals: the writer reaches
it package-relative and the reader imports FunPack modules top-level by name,
so they can hold different module objects and the label would never arrive.
2. The progress bar total now counts both passes. It only ever counted the main
schedule, so with a second pass on it overflowed during scene 1 and jumped
backwards at scene 2, whose offset is a multiple of the per-scene stride.
3. context_windows was unusable. The knob offered "uniform_standard" etc.;
core's names are the other way round ("standard_uniform"), so every schedule
except "batched" raised ValueError out of the sampler and FAILED THE RENDER.
Core's names are the choices now, the old spellings are aliased so saved
projects keep working, and an unknown name is refused with a reason naming
what core accepts instead of crashing.
Same call had a second, quieter break: a keyword core's handler doesn't take
(latent_retain_index_list is absent on some builds, including this Mac's)
raised TypeError, which was reported as "ComfyUI core too old" — silently
switching off a feature core fully supports. Extras are dropped now, with a
note naming the one knob that stops working.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two-pass shift regrew the frames it dropped, and the invented ending
consistently came out with worse or missing movement — not worth paying
steps for. What is left is the part that works: generate the scene exactly
as normal, anchor pinned at full strength the whole way, then cut the
opening off the finished clip. Nothing is regrown, so the scene comes out
that much shorter and the audio is cropped to match.
Six knobs collapse into one INT, cut_opening_frames (0 = off), in the
Editor under Chain Sampler ▸ Timing & Seed rather than Experimental.
Gone with the shift: the mid-schedule split, the rewind re-noise
(_renoise_to_sigma), the tail refill modes, the fresh-audio reset, and the
schedule_context/continue_from_state plumbing that existed only so a split
scene would not re-time the sampler inside each half.
second_pass is untouched — it never used any of that machinery.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
User, after a live run: "First pass runs its own FULL schedule. Then 2nd pass begins.
If I want to stop 1st pass earlier - I remove sigmas from schedule... 'Continue from'
is ONLY relevant in our shifting technique which is a separate entity."
Correct, and the cut was mine, not theirs. I carried anchor_shift's cut/restart
semantics into a feature that has no use for them. Consequences they hit on the rental:
- Steps. Pass 1 was cut at the second schedule's first sigma, so a 9-step main plus a
4-step second pass ran 3+4=7, not 13. They noticed the missing steps.
- Distortion. With the cut, the between-pass operation ran on a MID-TRAJECTORY latent,
so "sharpen" was sharpening the noise still in it. It only makes sense on a finished
clip.
Now: pass 1 runs the main schedule in full, exactly as written. The operation runs on
that finished clip. Pass 2 runs second_pass_sigmas in full, exactly as written, entering
the way any img2img refine does — comfy's ordinary CONST scaling on a clean latent, so
continue_from_state stays off. Total steps are the two schedules added up. To make pass 1
shorter, shorten the main schedule.
second_pass_sigma and second_pass_restart_sigma are deleted outright: with both schedules
run as written there is nothing to cut and nothing to continue from. The second schedule's
first sigma is now simply the strength dial, the way it is in every img2img refine.
_second_pass_plan is replaced by _second_pass_schedule, which only validates.
Also reported: "I have only sharpening enabled but log tells me upsampling is enabled as
well." The load line printed "[FunPackSceneChain] segmented detailing upsampler: ..."
regardless of which feature pulled it in, and sharpen legitimately loads that model —
so a sharpen-only run announced a feature that was off. Now "latent upsampler loaded".
Kept: the malformed-schedule refusals (ascending, or not ending at 0) — both are silent
in the output rather than loud at runtime, which is how the distorted run read.
tests/test_second_pass.py rewritten around the corrected shape, including that a 9-step
main plus a 4-step second pass is 13. Suites at baseline: 22 pre-existing failures, 467
passing. Verified live that typing a schedule sets second_pass with no cut knob leaking.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Per the user's reminder that these are OPTIONS, not defaults: second_pass_op is a
combo defaulting to "none", and nothing runs unless it is picked. The expensive one
is therefore not a tax on anything — it costs what it costs only when chosen.
- "sharpen": one forward of Lightricks' trained 2x LatentUpsampler, resampled straight
back to the original size. No video-model calls, so a fraction of a step; pass 2 then
re-denoises the sharpened latent, which is what makes it stick. Adds detail consistent
with what is there and CANNOT fix wrong structure — the same limit segmented
detailing's sharpen mode documents.
- "upscale_2x": same upsampler, result KEPT at 2x, so pass 2 runs at 4x the pixels and
the scene decodes at double resolution. 3-5x the second half's cost.
Reuses detailing.py's _run_upsampler / _downscale_to and its lazily-loaded upsampler
(same model, same detail_upsampler knob, "auto" downloads the official file) rather
than adding a second copy of any of it.
Ordering that matters: the op runs BEFORE the rewind re-noise, because sharpening a
latent that has just had noise added back sharpens the noise.
Two failure modes handled rather than left implicit:
- upscale_2x changes the latent resolution, so the chunk's i2v anchor and mask are the
wrong size. Rescaling them would be inventing an anchor, so the pin is dropped and
the scene report SAYS the pass runs unpinned and may drift from the reference image.
- A missing or throwing upsampler degrades to "second pass, no operation" with the
reason, instead of killing a whole montage over an optional sharpen.
Found while writing the tests: an unrecognised op name was validated only AFTER the
upsampler forward had already run, so a typo paid full price and then reported the
wrong reason. Name is checked first now.
Editor + Easy Gen get the combo under Engine > Experimental; verified live that it
renders with none/sharpen/upscale_2x and defaults to none. Suite at baseline: 22
pre-existing failures, 306 passing.
Not built: downscale-then-rescale (DeepShrink-shaped). It is not a between-passes op —
pass 1 itself has to run at reduced resolution, which means downscaling the chunk, its
mask and the anchor before pass 1, so it needs its own insertion point rather than
this hook.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
anchor_shift built the two-pass machinery — stop the schedule, hand the state over,
re-enter exactly (_sample_chunk continue_from_state, _renoise_to_sigma, _schedule_view).
The user's point: that is generally useful, not anchor-specific. This exposes it on its
own, and keeps the two features independent as they asked: reaching the second_pass
branch means anchor_shift did not act on this scene, so they can never nest.
second_pass stop partway and finish in a second pass
second_pass_sigma where pass 1 stops (walked against the real schedule)
second_pass_restart_sigma where pass 2 re-enters; 0 = carry straight on
second_pass_sigmas OPTIONAL SIGMAS socket: pass 2 runs THAT schedule instead
At restart 0 with no second schedule the split is behaviour-neutral by construction —
pass 2 resumes from the exact handed-over state, nothing added, and 5+4 steps on a
9-step schedule is still 9. It earns its keep when the second half is made different:
re-entering higher (the model reworks what it committed to) or finishing on a different
schedule entirely, so the tail can have its own step count, spacing or shift.
Two things a split gets silently wrong if you let it, both handled:
- The i2v anchor. _sample_chunk drops noise_mask from what it returns, so pass 2 would
run UNPINNED and re-denoise the reference frame. anchor_shift wants exactly that;
a plain second pass must not. _restore_pinned_prefix puts the mask back AND restores
the pinned frames from the original chunk, since a rewind re-noise adds noise
everywhere including frames that are meant to be frozen.
- The sampler's own phases. Pass 2's schedule_context is the alternate schedule at
offset 0 when one is connected, or the main schedule at the slice offset when not —
so quality steps / final-correction window / AB2 ramp are measured against whichever
schedule pass 2 is actually running.
Every refusal names its fix (cut outside the schedule, entry below the cut, a
single-sigma SIGMAS input that is not a schedule) and lands in the scene report as
second_pass(SKIPPED: ...) — never a silent no-op.
Editor + Easy Gen get the three widgets under Engine > Experimental. The SIGMAS socket
is node-graph only for now: the Editor's fixed graph takes sigmas from ("studio", 5) and
has no second scheduler node, so exposing it there is a graph-topology change I have not
made unilaterally.
Latent/pixel-space pass-2 operations (sharpen, downscale->rescale, upscale) are next,
as opt-in options with nothing on by default.
New tests/test_second_pass.py: 10 tests over the split, the separate schedule, the
refusals and the pin. Suite at baseline: 22 pre-existing failures, 299 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported by the user: setting the shift sigma to 1 or 0 with restart 0 got them
nowhere, and what they wanted was "generate me a full video and don't make the 2nd
pass, just cut off the beginning".
That was a gap, not a crash. Both extremes fell through to the splitter, which
correctly refused (a cut at the first sigma leaves pass 1 no step; a cut at the last
leaves pass 2 none) and reported a skip — so anchor_shift silently did nothing and
the skip reason did not name the thing they were actually after.
anchor_shift_sigma=0 now means it: a cut at sigma 0 IS the end of the schedule, so
pass 1 is the entire run and there is nothing left to denoise. The scene samples
exactly as if anchor_shift were off — anchor pinned at full strength the whole way,
which is the point — and the slide is applied to the FINISHED latent. Zero extra
sampling: the cheapest form of this feature by a wide margin, and with tail=crop it
is plainly "generate the video, then delete the opening". Widget min lowered 0.05 -> 0.
Two things are refused rather than half-worked, each naming its fix:
- tail=extend_last / empty, because with no second pass a refilled tail is never
denoised — extend_last would freeze the last frame for anchor_shift_frames and
empty would leave an empty latent on screen. crop and wrap are pure rearrangements
of finished content, so both are allowed.
- a non-zero restart sigma, which has nothing to restart.
And anchor_shift_fresh_audio is forced off in this mode (reported, not silent):
resetting the audio with nothing left to regenerate it would just produce a silent
clip.
The old "schedule never reaches ..." skip reason now points at sigma=0 as the way to
run the whole schedule and only cut the head, so the next person hitting the extreme
is told what to set.
Suite at baseline: 22 pre-existing failures, 289 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
User's idea, as an OPTION (default stays extend_last). The three existing tail modes
all refill the freed end so the clip keeps its length, which means pass 2 has to invent
an ending and blend a joint — the source of the "thin ending" and of most of the reason
to reach for a rewind restart. 'crop' regrows nothing: the scene ends anchor_shift_frames
earlier and every surviving frame was generated as part of one continuous shot.
Checked the model side before assuming this was free, since a bad latent size would
mismatch rather than degrade:
- LTXAV's video patchifier is SymmetricPatchifier(1) -> temporal patch size 1, so there
is no divisibility rule on video latent frames.
- The streams reach the transformer as independent tensors (separate_audio_and_video_
latents just unbinds the list); no video:audio ratio is enforced there.
- comfy re-packs per sample_custom call and re-derives latent_shapes, and the denoise
masks off them (comfy/samplers.py outer_sample), so nothing is baked in from
scene-build time and a shorter latent needs no conditioning rebuild.
- VAE decode is (f-1)*time_scale+1 for any f >= 1.
So no size is rejected. What IS real is proportion: audio timings come from the audio
stream's own index and video's from its own RoPE, so cropping the two by different
amounts of TIME desyncs the clip silently — no error, just drifting sound. Each extra
stream's KEPT length is therefore derived from the new video length directly rather than
by subtracting a separately-rounded drop, so rounding cannot leave them describing clips
of different durations. fresh_audio zeroes after the crop, at the new length.
Downstream needed nothing: _prepend_soft_continuation already clamps to the previous
scene's real length and _derived_overlap clamps to the stream, so a shorter scene flows
through the chain as-is.
The scene report now states the cropped length in latent frames and in real frames,
because a scene coming out shorter than the length that was set must never be something
the user has to infer from the output.
Tests: crop shortens instead of refilling and the surviving frames are the untouched
originals; audio/video stay in proportion (including four rates that do not divide
evenly); fresh_audio resets at the cropped length; the other three modes still keep the
clip length. Suite at baseline: 22 pre-existing failures, 288 passing (+5).
Verified live in the running app: the Engine ▸ Experimental dropdown offers
extend_last/wrap/empty/crop, default unchanged, hint present.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Raised by the user: "it should read the actual sampler schedule." They were right,
and this is a bigger deal than anchor_shift.
Both FunPack samplers derive their phase boundaries from len(sigmas) — the tensor
they are handed. anchor_shift runs a scene as two sample_custom calls over slices of
one schedule, so each half re-derived every phase INSIDE the slice:
_sample_const_rf_full (Hybrid Euler 2S, and the LTXAV path of it)
- quality phase: late_start = pct of the SLICE. On the user's 9-step schedule at
high_quality_pct 0.35 the quality phase starts at sigma 0.603; split at 0.482 it
starts at 0.812 in pass 1 and 0.482 in pass 2.
- motion-pulse anchors keyed by step index within the slice
- eta decay + velocity-bias ratio measured against the slice's first sigma
- the euler->Heun correction ramp restarted per call
sample_funpack_distilled_flow
- final-correction window: the last N steps OF THE SLICE, so pass 1 ran a "final"
Heun correction in the middle of the trajectory
- AB2 ramp w = i/(total-1) restarted from 0 in pass 2
- velocity-bias sigma ratio against the slice's first sigma
So turning anchor_shift on silently re-timed the sampler's own machinery in both
halves. Present since 015f274 and a far better explanation for a visible joint or a
thin ending than anything in the shift itself.
Fix: _schedule_view(sigmas, schedule_context) -> (real schedule, step offset). The
chain sampler passes {"sigmas": <the scene's schedule>, "offset": <slice start>} via
extra_options (the same chunk-scoped mechanism the alg_guide_* options already use,
always written even as None so it cannot leak into a later chunk). Both samplers take
the kwarg; every phase-relative quantity now reads the schedule and every step index
is step_abs = i + offset. No context = a normal single call = sigmas IS the schedule,
offset 0, byte-identical behaviour.
_anchor_shift_split_sigmas also returns pass 2's index into the schedule, and its
docstring now states the boundary rule the way the user framed it: walk the schedule,
the last pre-shift step is the one whose NEXT sigma has crossed the threshold. On
1.000 0.955 0.893 0.812 0.715 0.603 0.482 0.241 0.121 0.0 with shift 0.482 and restart
0.812 that gives pass 1 = 1.000..0.482 and pass 2 = 0.812..0.0, which is now asserted
directly as a test on that schedule.
Not changed, deliberately: sample_funpack_hybrid_euler_2s's own non-CONST loop (LTXAV
always takes the RF port, so no anchor_shift run reaches it) and segmented detailing's
stage-2 tail (a separate refine pass over a finished latent with its own schedule by
design, not a slice of one).
Suite at baseline: 22 pre-existing failures, 283 passing (+4).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported by the user from a scene report: with a restart sigma set, the log said
anchor_shift_sigma was ignored. It was. That is my doing, and it should never have
shipped that way.
The original implementation did what was designed: cut pass 1 at anchor_shift_sigma,
shift, restart pass 2 at anchor_shift_restart_sigma, the two independent. It produced
noise artifacts because pass 2 re-entered through comfy's CONST scaling, which assumes
a CLEAN latent. There were two ways to fix that — write a re-noise that is valid on a
half-denoised latent, or make pass 1 always run to completion so the input is clean.
327f544 took the second. That silently reduced a two-knob design to one knob per mode,
was described in the commit as the modes being "genuinely different" rather than as a
parameter being removed, and left the user sweeping a value that could not do anything.
This does the work that should have been done then.
_anchor_shift_renoise: given x_a mid-trajectory and a target b >= a,
x_b = alpha*x_a + beta*n, alpha = (1-b)/(1-a), beta = sqrt(b^2 - (a*alpha)^2)
alpha reproduces x0's coefficient exactly, beta tops the noise term back up to b, so
the result lands on the manifold the model expects at b. Both old modes are special
cases of the one formula: b == a gives alpha=1, beta=0 (an exact resume) and a == 0
gives comfy's own fresh-noise scaling. beta has a real root only when b >= a, which is
why a restart BELOW the cut is now refused with a message saying why — noise can be
added back, never removed.
Streams wiped by anchor_shift_fresh_audio are not mid-trajectory, so they re-enter at
a=0 and get the full b*n a from-scratch run starts from, not a partial top-up of a
latent that no longer exists.
Continue mode (restart 0, the validated config) is unchanged and still bit-identical:
no re-noise, cost one full schedule. Rewind now costs the cut plus the restart tail
rather than a whole schedule plus the tail.
The scene-report NOTE about an ignored parameter is gone, because nothing is ignored.
Tooltips in samplers.py and engine_settings.js (Editor + Easy Gen share it) rewritten.
Tests: the old test asserting pass 1 runs to completion is replaced by one asserting it
stops at the cut; plus the re-noise identity at b==a, equivalence with comfy's scaling
at a==0, the coefficient/variance invariants on a synthetic latent with known x0 and
eps, reset-stream handling, non-mutation, and the below-the-cut refusal. Suite at
baseline: 22 pre-existing failures, 279 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported from the first live run: the idea works (opening reads as t2v,
character details preserved) but the output is full of noise artifacts, as if
under-denoised, with the last steps seemingly skipped. That was a real bug, not
the concept.
comfy's CONST scaling is x = sigma*noise + (1-sigma)*latent_image, and it
treats latent_image as a CLEAN image. Pass 2 was handed the PARTIALLY DENOISED
latent as latent_image, so at a 0.725 cut it entered at
0.725*fresh_noise + 0.275*picture — pass 1's content attenuated to a quarter
and buried under a full dose of new noise, with only the tail steps left to
recover. Exactly the reported symptom.
I cited segmented detailing's stage-2 tail as the precedent for this re-entry,
but that recipe feeds a FINISHED latent, where re-noising is correct. The
precedent does not transfer mid-trajectory. My error.
Fix — the two modes are now actually different things:
- CONTINUE (restart sigma 0, the default): pass 2 resumes from exactly the
state pass 1 handed over. Passing that state as BOTH noise and latent_image
collapses the CONST expression to an identity (s*x + (1-s)*x == x), so no
noise is added at all. Cost still one full schedule.
- REWIND (restart sigma > 0): pass 1 now runs the WHOLE schedule to a finished
clip first, the shift is applied to that, and pass 2 re-noises it upward and
denoises down. That is the img2img recipe, and it is valid precisely because
the input is finished. Costs a full schedule plus the tail.
Also fixed pass 2's progress offset, which restarted at pass 1's, so the bar
appeared to stall short of the end.
Two new tests reproduce the CONST arithmetic directly and assert the identity
holds in continue mode and does not in rewind mode. Their comfy stubs are now
snapshot/restored — the first version leaked into other modules and shifted the
suite off baseline (3 unrelated tests changed result). Back to baseline: 22
pre-existing failures, 272 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Found while checking whether this works in Easy Gen. It does - the knobs render
and persist there through the same shared engine_settings.js - but Easy Gen's
default scene has no anchor image, which exposed a hole: nothing checked that
the scene actually HAS a pinned i2v anchor. On a genuine t2v scene the slide
would have dropped real generated frames and regrown them at the end, for no
benefit at all. This is the "fake t2v" trick; it needs a real anchor to fake
it with. Now refused with that reason in the scene report.
Second guard: if anchor_shift_frames drops FEWER latent frames than the scene
actually pins, part of the anchor survives and stays visible - the exact
outcome the feature exists to prevent. The report now names the pinned count
and the value to raise the knob to.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
User's idea, and the framing is the point: this is "faking t2v" when it is
really i2v. Every existing way of stopping the anchor from dominating the
opening WEAKENS it first - ALG blurs it and loses character detail, and if you
under-blur you get both failure modes at once; Best-FaceID overlap tokens
approximate it and lose some detail too. This weakens nothing. The anchor is
pinned at full strength, untouched, for as long as it takes to transfer
identity/style/composition - and is then removed from the clip entirely.
Two passes:
pass 1 sigma[0] .. anchor_shift_sigma, anchor pinned as usual
shift drop the first N latent frames, slide everything left, refill the
freed tail, drop the pin (clip LENGTH is unchanged - nothing extra
is ever generated; the dropped head is regrown at the end)
pass 2 restart sigma .. 0, unpinned
Two passes rather than a roll between steps for a specific reason: pass 2
re-enters through comfy's CONST noise scaling (x = s*noise + (1-s)*latent),
which puts the shifted latent back ON the rectified-flow manifold. A raw
in-flight roll skips that, and project_sigmas records that this model cannot
recover from off-manifold latent states - that is what killed churn. Same
on-manifold re-entry that segmented detailing's stage-2 tail already uses.
Every choice is a knob, because WHERE to cut is the entire experiment and I
have no measurement for it: shift sigma (default 0.909, the first step where
structure actually forms - above it nothing has formed and the anchor would be
discarded before doing any work), frames dropped, pass-2 restart sigma
(0 = continue, higher = rewind for more freedom at the joint), what the
regrown tail starts from, and whether audio is rewritten in pass 2 (on by
default: pass 1's audio was scored against frames that have since moved).
Refused, loudly and with the reason in the scene report, on continuation
scenes (the head is carried frames, not an anchor), on scenes carrying guide
frames (the slide would overwrite them) and on scenes with JoyAI audio memory
(resetting audio would destroy the injected tail).
One real bug caught by the new tests while writing them: SIGMAS are float32,
so a threshold typed as 0.725 is a float64 sitting just below the schedule's
float32 0.725, and a bare `v <= s` cut one step LATER than the number entered.
Now matched against core's own isclose tolerance.
11 new tests; suite at baseline (22 pre-existing failures, no new). Editor
checked live: knobs render, gate on the toggle, persist to sampler_inputs.
UNVALIDATED LIVE - the shift sigma is yours to find.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two things, both driven by upstream ComfyUI moving 7 weeks ahead of the
local checkout (2cdaaf4a 2026-06-06 -> e651b7be 2026-07-28, 263 commits).
1. Segmented detailing crashed on ComfyUI v0.29.0. Upstream f8a3fd9d moved
the LTX latent upsampler onto DynamicVram and made `operations` a required
argument of LatentUpsampler.from_config; detailing.py still called it with
one arg, so the pending live probe would have died before detecting
anything. Now passes comfy.ops.disable_weight_init (what core's own loader
does) with a TypeError fallback, since rentals run whatever they pulled.
2. New Chain Sampler option: context_windows. Upstream cd77c551 (CORE-3)
added context-window sampling with real LTX2/LTXAV support - it unpacks
the packed AV latent, maps each video window onto its proportional audio
window, and re-slices denoise masks, keyframe_idxs and
guide_attention_entries per window, so anchors, mid-scene guides and JoyAI
memory survive inside a window. Nothing is ported: we build core's handler
and hang it on the scene's model_options exactly as the stock
LTXVContextWindows node does. Core picks it up in
comfy.samplers.calc_cond_batch, which every FunPack sampler already routes
through, so no sampler-loop change was needed and no graph change either -
it rides the sampler_inputs bridge into both Editor and Easy Gen.
Refuses (loudly, once per run, with a scene-report note) on cores without
BaseModel.map_context_window_to_modalities: the handler class predates
LTXAV support and would window the packed tensor blindly, which is the
same packed-vs-unpacked trap that killed DynamicConditioning. Install and
removal are per-scene under the existing wrapper-hygiene finally, so an
interrupt cannot leak the handler or its two patcher wrappers into later
runs.
Knobs (all exposed, none hardcoded): length, overlap, schedule, blend,
FreeNoise, and pin-anchor-in-every-window. Off by default. Engages only
on scenes longer than the window, and says so in the scene report when it
does not, so a scene that sampled whole never reads as a broken feature.
Verified: 8 new unit tests for the install/remove/capability-gate contract;
full suite at baseline (22 pre-existing cross-module stub failures, no new).
Editor checked live - knobs render in Engine > Chain Sampler > Experimental,
dependents gate on the toggle, and values persist into sampler_inputs.
UNVALIDATED LIVE on a rental.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ComfyUI 7c59a07 ("Use comfy kitchen rope functions in ltx models", 2026-07-24)
changed the LTX positional-embedding payload from (cos, sin, split_mode) to
(rotation_matrix, split_mode), where the matrix is (B, T, heads, D, 2, 2)
holding [[cos, -sin], [sin, cos]].
rotate_overlap_freqs read pe[1] as the sin tensor — now a bool — so it raised,
and prepare_pe's bare `except: return pe` swallowed it. The Best-FaceID
reference tokens were still appended on the target's frame-0 coordinate grid
but WITHOUT the source-phase tag that marks them as a reference, so the model
read them as literal frame 0 and opened the clip on the reference image.
Anyone on ComfyUI >= 2026-07-24 saw i2v-like behaviour they couldn't bypass.
rotate_overlap_freqs now detects the layout at runtime (pe[1] is a tensor =>
legacy cos/sin; otherwise the matrix form, tokens on axis 1, frequencies on
-3) and composes the same R(angle + phase) either way. An unrecognised payload
raises instead of silently skipping the tag.
The four LTXBaseModel wrappers no longer fail silently: each reports once per
install that identity conditioning is not being applied and that a ComfyUI
update is the likely cause. A logged warning would have made this obvious.
Tests: both layouts rotate only the ref-token tail, produce identical angles,
reject unknown payloads, and the matrix form survives end-to-end through
_install_identity_overlap.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Recovers the original pitch: 'cheap, on-the-fly sharpen' was quietly
upgraded into a full re-denoise pass during design without ever offering
the free version as an option. The trained latent upsampler net was
already an intermediate step in the pipeline (crop_up) — sharpen mode
just stops there and pastes it back, with ZERO video-model forward
passes (only the small upsampler CNN runs). repair mode (default,
unchanged, byte-identical behavior) keeps the existing re-noise + 3-step
tail for cases that need actual structural correction, since only the
generative model itself can decide a region should look structurally
different — a super-res net can only sharpen what's already there.
Also extends the Editor's knobVisible with a array (AND'd
conditions) — detail_denoise needed to gate on BOTH segmented_detailing
AND detail_mode=='repair', which the existing single dependsOn/dependsValue
couldn't express without silently dropping one of the two checks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Live report: a hand was detected, masked, and upsampled, but the pasted
result looked like a plain upscale, not an actually-corrected region.
Root cause is mechanical, not a bug: 0.85 is Lightricks' own official
re-entry sigma, but it was designed for their use case (sharpen an
already-correct frame at higher resolution), not ours (repair a likely-
malformed crop). At that re-entry point the tail mostly smooths the
upsampled interpolation back down rather than giving the model enough
noise/freedom to genuinely reconstruct structure.
detail_denoise (0.3-0.99, default 0.85 = unchanged prior behavior) now
controls the re-entry point; the tail's shape (ratio between its 3 steps)
scales with it via the new stage2_sigmas() instead of a fixed tuple, so
raising it doesn't just enlarge the first jump. Higher = more genuine
repair freedom at some risk of drifting from the surrounding frame.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 35% area cap was refusing a user-requested 'hand' region (47% of
frame) with no way to override it — the code was unilaterally deciding
the request 'wasn't a detail pass' when the user never asked for that
judgment. Fair complaint: cost guardrails are not the same as content
decisions.
detail_max_area is now threaded end-to-end (detailing.area_cap param ->
sampler widget -> Editor knob in Chain Sampler > Experimental, 0.05-1.0,
default 0.35 unchanged). Raising it lets a region of any size through;
1.0 = no cap. The refusal note now points at the actual setting name
instead of citing a fixed constant.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Live run hit 'no region matched' with zero diagnostics: CLIPSeg's raw
sigmoid score for a real, correctly-named region is often well under a
naive 0.5, so a miss against the hardcoded 0.35 threshold was
indistinguishable from 'nothing there.' Two fixes:
- find_tube now takes an optional diag dict and always records max_heat
(plus area_cap vs below_threshold reason on a miss).
- detail_refine_scene always returns a descriptive note once detection
actually ran ('no match: max CLIPSeg score 0.XX < threshold 0.35 — try
a lower detail_threshold...'), replacing the generic
'no region matched' the sampler used to fabricate.
- detail_threshold is now a real widget/sampler_input (was a hardcoded
constant with no UI — a hidden-setting violation once real runs started
needing it), in Engine > Chain Sampler > Experimental next to the other
detailing knobs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Live rental run hit it immediately on 'detail_targets' with more than one
target: CLIPSegProcessor tokenizes each prompt to a different length, and
without padding=True torch can't stack them into a batch ('excessive
nesting... type list where int is expected'). Reproduced against the real
CIDAS/clipseg-rd64-refined model before and after the fix. Single-target
runs were unaffected (batch of 1 never needed padding), which is why the
earlier probe steps didn't catch it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first live attempt silently no-oped: detail_upsampler defaulted to 'None'
(nothing installed on the rental) and the guard skipped without a word.
Two fixes:
- detail_upsampler default is now 'auto': prefer the official installed file,
else the highest-versioned spatial upscaler (version-aware compare — plain
sort ranked ltx2_3_… over ltx-2.3-…-1.1), else hf_hub_download of
Lightricks/LTX-2.3 ltx-2.3-spatial-upscaler-x2-1.1.safetensors (~1 GB, once)
with an actionable error if the repo is gated. Legacy 'None' = auto.
- With the toggle on, every outcome is visible: upsampler choice and skips
print to console AND land in the scene report (segmented_detail(SKIPPED: …) /
(no region matched)); resolve/load failure disables the pass for the run
instead of retrying the download per scene. A failed pass still never costs
the scene.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ADetailer-for-video probe core. After a scene denoises, CLIPSeg locates
user-named regions (hands, feet, ...) on decoded keyframes; the region is
cropped as a spatiotemporal tube, 2x-upsampled through Lightricks' trained
LatentUpsampler (comfy core, no port needed), re-denoised over the official
stage-2 tail (0.85/0.725/0.4219/0), downscaled back and pasted through the
feathered CLIPSeg silhouette. Final resolution unchanged; audio protected by
construction (refined audio discarded); tubes >35% frame refused; empty
targets/no-detection = bit-identical no-op. UNVALIDATED LIVE.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Guide-carrying scenes no longer disable the loop temporal style. Appended
guide frames always live at the tail of the video stream's T range, and
keyframe_idxs is per-token — so the wrapper derives the tail frame count
(tokens / (H*W), same math as comfy's get_keyframe_idxs) and rolls only the
content region [0, T - tail); the tail, keyframe_idxs, and attention entries
that reference it stay canonical. The audio stream's appended memory (JoyAI,
rides a trailing mask=0 run) is detected the same way from audio_denoise_mask.
Guides keep informing every rolled position — their influence smears across
the cycle, which is what identity/style references should do in a loop — and
are never dragged into the rendered content.
Unparseable guide contexts (malformed keyframe_idxs, entries without idxs)
still run canonical: never risk a corrupting roll. Content-region size gates
the minimum-frames check now, so a short clip whose frames are mostly guide
tail stays canonical too.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 'loop' temporal style (in the style list since forever, but inert) now does
what WanVideoWrapper's Loop Args does for WAN: on eligible denoise steps the
packed latent is cyclically rolled along its temporal dim before the forward and
the prediction unrolled after, so the model repeatedly sees the video's seam
(last|first frame) as an interior frame pair at step-varying positions (van der
Corput sequence, reset per scene) and smooths it into a natural loop. Video and
audio streams roll by their own frame counts and stay time-aligned; denoise
masks roll in step, so i2v anchor per-token timesteps stay consistent.
Near-zero overhead: two clones + torch.roll per eligible step, no extra forward
passes. Rolls only below the near-noise plateau (nothing to smooth up there, and
the plateau step-cache stays valid). Calls carrying guides are left canonical —
keyframe_idxs pins appended guide frames to absolute positions inside the same
T range, so rolling them would corrupt; on guide-carrying scenes the roll is
inert and the run mechanism log says so.
Wiring: concrete style via the global build_enhancements path (innermost
relative to per-scene wrappers, as required — prediction-modifying wrappers see
canonical orientation only); auto director's funpack_temporal_loop via a tagged
per-scene wrapper on the Scene Chain sampler, installed below the guidance
stack. Editor already exposes 'loop' in the temporal style dropdown.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
MixCache/Chorus-family step-caching adapted to LTX2.3's distilled schedule.
On the near-pure-noise plateau (sigma >= threshold, default 0.975) the
transformer output barely changes, so compute the base-model forward once
and reuse it for the remaining plateau steps, skipping ~3-4 of 8 full
48-block forwards. Below threshold every step computes for real and the
cache drops, so structure formation is never approximated. Deterministic
given seed -> safe in Batch Training, no rating-diversity impact.
Installed innermost as a per-scene model_function_wrapper (existing
tag/strip leak hygiene) so guidance wrappers still post-process each step's
cached-or-fresh prediction. Cache keyed by (input.shape, cond_or_uncond).
Editor: plateau_cache + plateau_cache_threshold in Engine -> Experimental,
auto-wired via the sampler_inputs bridge (no backend/builder change).
Off by default, UNVALIDATED LIVE. (Chorus cross-variant reuse rejected --
would blur batch-rating diversity; FreeSpec dropped -- wrong topology.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
model=object() lacked model_options (needed by the per-scene wrapper
snapshot/restore added for the scene-wrapper-leak fix), and FakeVAE lacked
decode() (needed since the sampler always decodes IMAGES when the caller
doesn't declare that output as disconnected) — together these silently
killed every test in the file that calls sample(). Added FakeModel and
FakeVAE.decode, then fixed the resulting return-tuple arity (sample() now
returns 6 values, these tests unpacked 4) and three genuinely stale
assertions: the per-scene report line gained a timing segment, carry_i2v_guides
now prepends its guide frame instead of appending it, and _append_i2v_guides
no longer routes through keyframe_idxs/guide_attention_entries.
Verified in isolation: all 14 tests in this file now pass. Running the full
tests/ suite still shows the same file failing — that's a separate,
pre-existing cross-file sys.modules stubbing collision (confirmed via
git stash on a clean dev checkout) affecting this file plus
test_mask_safe_attention.py and test_v2_category_alignment.py; not fixed
here since it isn't a one-line root cause.
carry_overlap_through_anchor lets a scene keep the previous scene's latent
tail for background/environment continuity even when it switches to its own
i2v anchor, instead of the current hard cut. The anchor's own leading frame
stays untouched; only the frames after it carry the old tail.
Also fixes a real gap found along the way: the mixed-anchor branch never
called _apply_configured_guides, so Best-FaceID identity_transfer could
never resolve an identity_pin on an anchor-swap scene — exactly where it's
most useful. Added a standalone lookup so it resolves there too.
Replaces the source-phase-RoPE-only approximation with a faithful native port of
ComfyUI-BFSNodes' LTX Identity Transfer (read upstream source directly): the reference
face is now injected as separate, non-rendered overlap tokens (clean timestep, RoPE
source-phase tag, trimmed before output) instead of blending into Continuity's identity-pin
keyframe, plus an optional ArcFace embedding -> IdentityProjector -> context-token channel
(identity_projector, source_id, phase_scale, id_strength, arcface_mode, debug_log). Stays
on Chain Sampler per the Studio/Sampler boundary law's guide-ops carve-out. New
identity_transfer.py module; insightface added as a soft pip dependency.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Reverts the earlier ComfyUI-BFSNodes graph-splice approach (builder.py/timeline.py/
server.py scaffolding) in favor of porting the missing piece natively: FunPack's own
guide-attention path (LTXVAddGuide.add_keyframe_index + guide_attention_entries)
already places a reference image on the frame-0 grid exactly like BFSNodes' overlap
technique, and Continuity's existing "Identity pin" already applies one reference
face to every scene. The only real gap was the LoRA's trained-time source_phase RoPE
tag, added as a tag+strip patch on FunPackLTXAVSceneChainSampler scoped to the exact
guide-entry token range (never co-active mid_scene/carry/prior-scene guides sharing
the same keyframe_idxs tensor). New "Best-FaceID compatibility" + "Source-phase id"
knobs live under Engine Settings -> Experimental, alongside every other unvalidated
technique. LoRA loading itself reuses the existing generic Models-menu lora role.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
alg_blur_guides no longer borrows the sampler's anchor knobs: the Scene Chain
Sampler gains alg_guide_blur_strength (default 2.0) and
alg_guide_blur_sigma_threshold (default 0.975), threaded per-chunk via
extra_options like the tail count. The sampler now precomputes one latent per
(anchor blurred?, tail blurred?) combination — the frame sets are disjoint, so
the both-blurred variant composes the two passes — and each region follows its
own sigma window per step. Anchor ALG (alg_enabled/alg_strength/
alg_sigma_threshold) is untouched and back to anchor-only semantics.
Editor: 'Guide blur strength' and 'Guide blur sigma threshold' rows appear
under Engine ▸ Experimental when the blur toggle is on (sampler_inputs bridge,
names match the new node widgets; inputs appended at the block end to keep the
builder's positional widget mapping aligned).
Verified live: object_info shows both widgets appended last; Editor rows
appear on toggle and persist to project.sampler_inputs. 11 ALG unit tests
pass (new: composed independent-strength blur); same 20 pre-existing suite
failures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Editor ▸ Settings ▸ Editor gains a 'Shortcut revolver' toggle (+ 'Random' sub-option).
When on, each shortcut with several replacements draws from a per-shortcut queue —
sequential (first…last, then restart) or shuffled once per cycle — and nothing repeats
until the whole set has fired. Off (default) keeps the seeded random pick unchanged.
- State is a sidecar (shortcut_revolver.json) next to the shortcut DB: settings +
per-shortcut remaining queue + replacement-set fingerprint (editing a shortcut's
replacements restarts its cycle; changing either setting resets all cycles).
- Generation commits the draw (refine_v2 positive prompt); every preview path
(parse_timeline, split mapping, fragments) peeks the same stored queue without
saving, so previews show exactly what the next Generate will use.
- Batch Training variants each commit a draw: N variants sweep N consecutive
chambers instead of seed-rolled repeats.
- FunPackStudio.IS_CHANGED returns nan while the revolver is on so fixed-seed
re-runs aren't served from ComfyUI's node cache (the next chamber must fire).
Verified live on local ComfyUI: settings round-trip, seed-independent stable peeks,
sequential cycle REP_ONE→TWO→THREE→ONE across processes, peek-after-commit shows the
next chamber. 9 new unit tests; full suite has the same 20 pre-existing failures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
'Awful' / 'Wrong appearance' banked on intrusion semantics; now any rating
whose profile reward marks the run a bad OUTCOME (<= -0.25: Missing
quality, Missing details+quality, Missing action+quality, Missing
details+action+quality) also feeds the negative bank - degradation is
visible in the latent, so steering off that attractor is meaningful.
Near-miss ratings with positive reward (Missing details +0.35, Missing
action +0.05) stay excluded: those latents are mostly-wanted content and
banking them would repel future runs from material the user asked for.
Decision moves into negative_memory.is_negative_profile (intrusion keys OR
reward threshold, robust to future profiles); promoted entries now carry
the rating key as provenance for severity-weighted steering later.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
