Renaming the combo's choices to core's spellings broke queueing for anyone who
had one of the old ones saved: ComfyUI validates combo VALUES at queue time,
before the node ever runs, so the alias map added with the rename never got a
chance. The three legacy names are back in the list (they resolve through the
alias map at runtime), and the Editor now displays a stored legacy value as the
name it maps to and writes that back, so the dropdown can't show one setting
while another runs.
And that rejection was unreadable. ComfyUI's top-level `error` for a validation
failure is a fixed blob ("Prompt outputs failed validation", details ""); the
part naming the node, the widget and the bad value is in `node_errors`, which
`error or node_errors` never reached because the blob is always truthy. Every
rejected prompt therefore reported the same nothing. Both are formatted now:
Prompt outputs failed validation; FunPackLTXAVSceneChainSampler: Value not in
list — context_window_schedule: 'static_standard' not in [...]
Also: the progress poll no longer imports anything. current_progress() reads
run_phase's label straight off `sys`. It fires every 700ms on ComfyUI's event
loop WHILE the worker thread samples, and that is not a path to do import work
on — the direct read is both cheaper and the correct one, since run_phase keeps
its state there anyway.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Answering "how does sharpen operate — I never saw the upsampler download?":
it uses the same file segmented detailing does, resolved the same way (any
model already in models/latent_upscale_models wins; the ~1 GB official file is
downloaded only when that folder is empty). So no download is expected on a
machine that already has one — but nothing in the Editor said so.
The upsampler picker rendered only under Experimental, and only while segmented
detailing was on, so choosing 'sharpen' alone gave no model control and no hint
that a 1 GB file was involved. It now also renders in the Second pass group
whenever an operation is selected, and the knob's hint states the dependency
and that the pass still runs (operation skipped, reason in the scene report)
when the file can't be obtained.
That reason is also worth something now: a failure said only "no latent
upsampler could be loaded", which is the same message for a missing
huggingface_hub, a failed download and an unreadable file. The real cause is
carried into the scene report and printed to the console once — previously only
the successful download printed, so a silent skip read as the op having run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three things, all about a run telling you what it is doing.
1. The second pass is now visible. ComfyUI's progress channel carries only
(value, max), so a chain sampling some scenes twice was one anonymous bar.
The sampler publishes a live label ("scene 2/3 · pass 2 of 2") through the
new run_phase module; /api/progress hands it to the editor, which appends it
to the readout, and pass 2 also announces itself on the console the moment it
starts rather than only in the after-the-fact run report.
run_phase keeps its state on `sys`, not in module globals: the writer reaches
it package-relative and the reader imports FunPack modules top-level by name,
so they can hold different module objects and the label would never arrive.
2. The progress bar total now counts both passes. It only ever counted the main
schedule, so with a second pass on it overflowed during scene 1 and jumped
backwards at scene 2, whose offset is a multiple of the per-scene stride.
3. context_windows was unusable. The knob offered "uniform_standard" etc.;
core's names are the other way round ("standard_uniform"), so every schedule
except "batched" raised ValueError out of the sampler and FAILED THE RENDER.
Core's names are the choices now, the old spellings are aliased so saved
projects keep working, and an unknown name is refused with a reason naming
what core accepts instead of crashing.
Same call had a second, quieter break: a keyword core's handler doesn't take
(latent_retain_index_list is absent on some builds, including this Mac's)
raised TypeError, which was reported as "ComfyUI core too old" — silently
switching off a feature core fully supports. Extras are dropped now, with a
note naming the one knob that stops working.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two-pass shift regrew the frames it dropped, and the invented ending
consistently came out with worse or missing movement — not worth paying
steps for. What is left is the part that works: generate the scene exactly
as normal, anchor pinned at full strength the whole way, then cut the
opening off the finished clip. Nothing is regrown, so the scene comes out
that much shorter and the audio is cropped to match.
Six knobs collapse into one INT, cut_opening_frames (0 = off), in the
Editor under Chain Sampler ▸ Timing & Seed rather than Experimental.
Gone with the shift: the mid-schedule split, the rewind re-noise
(_renoise_to_sigma), the tail refill modes, the fresh-audio reset, and the
schedule_context/continue_from_state plumbing that existed only so a split
scene would not re-time the sampler inside each half.
second_pass is untouched — it never used any of that machinery.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
User asked why there is an "img2img re-noise" at all — "you just pass the original latent
from 1st pass as latent_image into 2nd pass and that's it, sampler does everything itself."
That is exactly what the code does. Pass 1's latent goes in as latent_image with
continue_from_state off, so _sample_chunk calls prepare_noise and comfy's own noise_scaling
handles the entry; _renoise_to_sigma is never called on this path (it is anchor_shift's
only, for its genuinely mid-trajectory case). Between the passes there are just two things,
one of them usually nothing: the optional op (skipped at "none") and re-attaching the i2v
noise_mask so pass 2 does not re-denoise the anchor.
So no code change — but the docs said "re-noising the finished clip", which named comfy's
internal step as if it were machinery I had added, and that is what prompted the question.
Reworded to say the sampler does it.
The same wording hid the answer to their actual symptom (softness). CONST scaling is
x = s*noise + (1-s)*picture, so the schedule's first sigma is a literal strength: at 0.812
pass 2 starts from 81% fresh noise over 19% of the pass-1 picture with 4 steps to resolve
it, which is a very strong img2img and is expected to read soft. Both tooltips now give the
0.8 / 0.4 / 0.2 scale so the dial is legible instead of implicit.
Suites at baseline: 22 pre-existing failures, 467 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
User, after a live run: "First pass runs its own FULL schedule. Then 2nd pass begins.
If I want to stop 1st pass earlier - I remove sigmas from schedule... 'Continue from'
is ONLY relevant in our shifting technique which is a separate entity."
Correct, and the cut was mine, not theirs. I carried anchor_shift's cut/restart
semantics into a feature that has no use for them. Consequences they hit on the rental:
- Steps. Pass 1 was cut at the second schedule's first sigma, so a 9-step main plus a
4-step second pass ran 3+4=7, not 13. They noticed the missing steps.
- Distortion. With the cut, the between-pass operation ran on a MID-TRAJECTORY latent,
so "sharpen" was sharpening the noise still in it. It only makes sense on a finished
clip.
Now: pass 1 runs the main schedule in full, exactly as written. The operation runs on
that finished clip. Pass 2 runs second_pass_sigmas in full, exactly as written, entering
the way any img2img refine does — comfy's ordinary CONST scaling on a clean latent, so
continue_from_state stays off. Total steps are the two schedules added up. To make pass 1
shorter, shorten the main schedule.
second_pass_sigma and second_pass_restart_sigma are deleted outright: with both schedules
run as written there is nothing to cut and nothing to continue from. The second schedule's
first sigma is now simply the strength dial, the way it is in every img2img refine.
_second_pass_plan is replaced by _second_pass_schedule, which only validates.
Also reported: "I have only sharpening enabled but log tells me upsampling is enabled as
well." The load line printed "[FunPackSceneChain] segmented detailing upsampler: ..."
regardless of which feature pulled it in, and sharpen legitimately loads that model —
so a sharpen-only run announced a feature that was off. Now "latent upsampler loaded".
Kept: the malformed-schedule refusals (ascending, or not ending at 0) — both are silent
in the output rather than loud at runtime, which is how the distorted run read.
tests/test_second_pass.py rewritten around the corrected shape, including that a 9-step
main plus a 4-step second pass is 13. Suites at baseline: 22 pre-existing failures, 467
passing. Verified live that typing a schedule sets second_pass with no cut knob leaking.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three corrections from the user, all right.
1. The pass-2 schedule needed no graph-topology change at all — I was wrong to leave
it out. Studio has ALWAYS emitted a second schedule: _build_samplers builds both
samplers_cfg["high"] and ["low"], and low_pass_sigmas is output 7. The Editor simply
never wired it (sampler_panel.js even says so: "outputs 6+7 are not connected").
So this is one CORE_LINKS entry, "second_pass_sigmas": ("studio", 7), plus surfacing
the low pass's existing `sigmas` text field. No new node, no new output.
Only the SIGMAS is taken, not the low-pass SAMPLER: pass 2 reuses the sampler
configured above it. So the panel deliberately shows just that one field rather than
the whole low-pass sampler section — rendering its algorithm settings would offer
knobs that quietly do nothing, which is the thing the editor rules forbid.
2. No re-entry sigma in the Editor. With an explicit pass-2 schedule the first sigma of
that schedule IS where pass 2 re-enters, so second_pass_restart_sigma was a redundant
second way to say the same thing. Dropped from the Editor; still on the node for
graph users with no second schedule wired, where it is the only way to say it.
3. Not experimental. At its defaults the split is behaviour-neutral by construction —
pass 2 resumes from exactly the state pass 1 handed over, same step count, same
output — so calling it experimental overstated it. Moved from Chain Sampler >
Experimental to FunPack Studio > Sampler algorithm, next to the schedule field it
depends on, and the EXPERIMENTAL / UNVALIDATED LIVE framing is gone from both
tooltips.
Verified live in the running app: the Sampler algorithm pane shows the Second pass
group and the schedule field, the re-entry knob is gone, and typing a schedule persists
to studio_settings.samplers.low.sigmas (confirmed the high pass is untouched). Suites at
baseline: 22 pre-existing failures / 306 passing, movie_editor 163 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Per the user's reminder that these are OPTIONS, not defaults: second_pass_op is a
combo defaulting to "none", and nothing runs unless it is picked. The expensive one
is therefore not a tax on anything — it costs what it costs only when chosen.
- "sharpen": one forward of Lightricks' trained 2x LatentUpsampler, resampled straight
back to the original size. No video-model calls, so a fraction of a step; pass 2 then
re-denoises the sharpened latent, which is what makes it stick. Adds detail consistent
with what is there and CANNOT fix wrong structure — the same limit segmented
detailing's sharpen mode documents.
- "upscale_2x": same upsampler, result KEPT at 2x, so pass 2 runs at 4x the pixels and
the scene decodes at double resolution. 3-5x the second half's cost.
Reuses detailing.py's _run_upsampler / _downscale_to and its lazily-loaded upsampler
(same model, same detail_upsampler knob, "auto" downloads the official file) rather
than adding a second copy of any of it.
Ordering that matters: the op runs BEFORE the rewind re-noise, because sharpening a
latent that has just had noise added back sharpens the noise.
Two failure modes handled rather than left implicit:
- upscale_2x changes the latent resolution, so the chunk's i2v anchor and mask are the
wrong size. Rescaling them would be inventing an anchor, so the pin is dropped and
the scene report SAYS the pass runs unpinned and may drift from the reference image.
- A missing or throwing upsampler degrades to "second pass, no operation" with the
reason, instead of killing a whole montage over an optional sharpen.
Found while writing the tests: an unrecognised op name was validated only AFTER the
upsampler forward had already run, so a typo paid full price and then reported the
wrong reason. Name is checked first now.
Editor + Easy Gen get the combo under Engine > Experimental; verified live that it
renders with none/sharpen/upscale_2x and defaults to none. Suite at baseline: 22
pre-existing failures, 306 passing.
Not built: downscale-then-rescale (DeepShrink-shaped). It is not a between-passes op —
pass 1 itself has to run at reduced resolution, which means downscaling the chunk, its
mask and the anchor before pass 1, so it needs its own insertion point rather than
this hook.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
anchor_shift built the two-pass machinery — stop the schedule, hand the state over,
re-enter exactly (_sample_chunk continue_from_state, _renoise_to_sigma, _schedule_view).
The user's point: that is generally useful, not anchor-specific. This exposes it on its
own, and keeps the two features independent as they asked: reaching the second_pass
branch means anchor_shift did not act on this scene, so they can never nest.
second_pass stop partway and finish in a second pass
second_pass_sigma where pass 1 stops (walked against the real schedule)
second_pass_restart_sigma where pass 2 re-enters; 0 = carry straight on
second_pass_sigmas OPTIONAL SIGMAS socket: pass 2 runs THAT schedule instead
At restart 0 with no second schedule the split is behaviour-neutral by construction —
pass 2 resumes from the exact handed-over state, nothing added, and 5+4 steps on a
9-step schedule is still 9. It earns its keep when the second half is made different:
re-entering higher (the model reworks what it committed to) or finishing on a different
schedule entirely, so the tail can have its own step count, spacing or shift.
Two things a split gets silently wrong if you let it, both handled:
- The i2v anchor. _sample_chunk drops noise_mask from what it returns, so pass 2 would
run UNPINNED and re-denoise the reference frame. anchor_shift wants exactly that;
a plain second pass must not. _restore_pinned_prefix puts the mask back AND restores
the pinned frames from the original chunk, since a rewind re-noise adds noise
everywhere including frames that are meant to be frozen.
- The sampler's own phases. Pass 2's schedule_context is the alternate schedule at
offset 0 when one is connected, or the main schedule at the slice offset when not —
so quality steps / final-correction window / AB2 ramp are measured against whichever
schedule pass 2 is actually running.
Every refusal names its fix (cut outside the schedule, entry below the cut, a
single-sigma SIGMAS input that is not a schedule) and lands in the scene report as
second_pass(SKIPPED: ...) — never a silent no-op.
Editor + Easy Gen get the three widgets under Engine > Experimental. The SIGMAS socket
is node-graph only for now: the Editor's fixed graph takes sigmas from ("studio", 5) and
has no second scheduler node, so exposing it there is a graph-topology change I have not
made unilaterally.
Latent/pixel-space pass-2 operations (sharpen, downscale->rescale, upscale) are next,
as opt-in options with nothing on by default.
New tests/test_second_pass.py: 10 tests over the split, the separate schedule, the
refusals and the pin. Suite at baseline: 22 pre-existing failures, 299 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported by the user: setting the shift sigma to 1 or 0 with restart 0 got them
nowhere, and what they wanted was "generate me a full video and don't make the 2nd
pass, just cut off the beginning".
That was a gap, not a crash. Both extremes fell through to the splitter, which
correctly refused (a cut at the first sigma leaves pass 1 no step; a cut at the last
leaves pass 2 none) and reported a skip — so anchor_shift silently did nothing and
the skip reason did not name the thing they were actually after.
anchor_shift_sigma=0 now means it: a cut at sigma 0 IS the end of the schedule, so
pass 1 is the entire run and there is nothing left to denoise. The scene samples
exactly as if anchor_shift were off — anchor pinned at full strength the whole way,
which is the point — and the slide is applied to the FINISHED latent. Zero extra
sampling: the cheapest form of this feature by a wide margin, and with tail=crop it
is plainly "generate the video, then delete the opening". Widget min lowered 0.05 -> 0.
Two things are refused rather than half-worked, each naming its fix:
- tail=extend_last / empty, because with no second pass a refilled tail is never
denoised — extend_last would freeze the last frame for anchor_shift_frames and
empty would leave an empty latent on screen. crop and wrap are pure rearrangements
of finished content, so both are allowed.
- a non-zero restart sigma, which has nothing to restart.
And anchor_shift_fresh_audio is forced off in this mode (reported, not silent):
resetting the audio with nothing left to regenerate it would just produce a silent
clip.
The old "schedule never reaches ..." skip reason now points at sigma=0 as the way to
run the whole schedule and only cut the head, so the next person hitting the extreme
is told what to set.
Suite at baseline: 22 pre-existing failures, 289 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
User's idea, as an OPTION (default stays extend_last). The three existing tail modes
all refill the freed end so the clip keeps its length, which means pass 2 has to invent
an ending and blend a joint — the source of the "thin ending" and of most of the reason
to reach for a rewind restart. 'crop' regrows nothing: the scene ends anchor_shift_frames
earlier and every surviving frame was generated as part of one continuous shot.
Checked the model side before assuming this was free, since a bad latent size would
mismatch rather than degrade:
- LTXAV's video patchifier is SymmetricPatchifier(1) -> temporal patch size 1, so there
is no divisibility rule on video latent frames.
- The streams reach the transformer as independent tensors (separate_audio_and_video_
latents just unbinds the list); no video:audio ratio is enforced there.
- comfy re-packs per sample_custom call and re-derives latent_shapes, and the denoise
masks off them (comfy/samplers.py outer_sample), so nothing is baked in from
scene-build time and a shorter latent needs no conditioning rebuild.
- VAE decode is (f-1)*time_scale+1 for any f >= 1.
So no size is rejected. What IS real is proportion: audio timings come from the audio
stream's own index and video's from its own RoPE, so cropping the two by different
amounts of TIME desyncs the clip silently — no error, just drifting sound. Each extra
stream's KEPT length is therefore derived from the new video length directly rather than
by subtracting a separately-rounded drop, so rounding cannot leave them describing clips
of different durations. fresh_audio zeroes after the crop, at the new length.
Downstream needed nothing: _prepend_soft_continuation already clamps to the previous
scene's real length and _derived_overlap clamps to the stream, so a shorter scene flows
through the chain as-is.
The scene report now states the cropped length in latent frames and in real frames,
because a scene coming out shorter than the length that was set must never be something
the user has to infer from the output.
Tests: crop shortens instead of refilling and the surviving frames are the untouched
originals; audio/video stay in proportion (including four rates that do not divide
evenly); fresh_audio resets at the cropped length; the other three modes still keep the
clip length. Suite at baseline: 22 pre-existing failures, 288 passing (+5).
Verified live in the running app: the Engine ▸ Experimental dropdown offers
extend_last/wrap/empty/crop, default unchanged, hint present.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Raised by the user: "it should read the actual sampler schedule." They were right,
and this is a bigger deal than anchor_shift.
Both FunPack samplers derive their phase boundaries from len(sigmas) — the tensor
they are handed. anchor_shift runs a scene as two sample_custom calls over slices of
one schedule, so each half re-derived every phase INSIDE the slice:
_sample_const_rf_full (Hybrid Euler 2S, and the LTXAV path of it)
- quality phase: late_start = pct of the SLICE. On the user's 9-step schedule at
high_quality_pct 0.35 the quality phase starts at sigma 0.603; split at 0.482 it
starts at 0.812 in pass 1 and 0.482 in pass 2.
- motion-pulse anchors keyed by step index within the slice
- eta decay + velocity-bias ratio measured against the slice's first sigma
- the euler->Heun correction ramp restarted per call
sample_funpack_distilled_flow
- final-correction window: the last N steps OF THE SLICE, so pass 1 ran a "final"
Heun correction in the middle of the trajectory
- AB2 ramp w = i/(total-1) restarted from 0 in pass 2
- velocity-bias sigma ratio against the slice's first sigma
So turning anchor_shift on silently re-timed the sampler's own machinery in both
halves. Present since 015f274 and a far better explanation for a visible joint or a
thin ending than anything in the shift itself.
Fix: _schedule_view(sigmas, schedule_context) -> (real schedule, step offset). The
chain sampler passes {"sigmas": <the scene's schedule>, "offset": <slice start>} via
extra_options (the same chunk-scoped mechanism the alg_guide_* options already use,
always written even as None so it cannot leak into a later chunk). Both samplers take
the kwarg; every phase-relative quantity now reads the schedule and every step index
is step_abs = i + offset. No context = a normal single call = sigmas IS the schedule,
offset 0, byte-identical behaviour.
_anchor_shift_split_sigmas also returns pass 2's index into the schedule, and its
docstring now states the boundary rule the way the user framed it: walk the schedule,
the last pre-shift step is the one whose NEXT sigma has crossed the threshold. On
1.000 0.955 0.893 0.812 0.715 0.603 0.482 0.241 0.121 0.0 with shift 0.482 and restart
0.812 that gives pass 1 = 1.000..0.482 and pass 2 = 0.812..0.0, which is now asserted
directly as a test on that schedule.
Not changed, deliberately: sample_funpack_hybrid_euler_2s's own non-CONST loop (LTXAV
always takes the RF port, so no anchor_shift run reaches it) and segmented detailing's
stage-2 tail (a separate refine pass over a finished latent with its own schedule by
design, not a slice of one).
Suite at baseline: 22 pre-existing failures, 283 passing (+4).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported by the user from a scene report: with a restart sigma set, the log said
anchor_shift_sigma was ignored. It was. That is my doing, and it should never have
shipped that way.
The original implementation did what was designed: cut pass 1 at anchor_shift_sigma,
shift, restart pass 2 at anchor_shift_restart_sigma, the two independent. It produced
noise artifacts because pass 2 re-entered through comfy's CONST scaling, which assumes
a CLEAN latent. There were two ways to fix that — write a re-noise that is valid on a
half-denoised latent, or make pass 1 always run to completion so the input is clean.
327f544 took the second. That silently reduced a two-knob design to one knob per mode,
was described in the commit as the modes being "genuinely different" rather than as a
parameter being removed, and left the user sweeping a value that could not do anything.
This does the work that should have been done then.
_anchor_shift_renoise: given x_a mid-trajectory and a target b >= a,
x_b = alpha*x_a + beta*n, alpha = (1-b)/(1-a), beta = sqrt(b^2 - (a*alpha)^2)
alpha reproduces x0's coefficient exactly, beta tops the noise term back up to b, so
the result lands on the manifold the model expects at b. Both old modes are special
cases of the one formula: b == a gives alpha=1, beta=0 (an exact resume) and a == 0
gives comfy's own fresh-noise scaling. beta has a real root only when b >= a, which is
why a restart BELOW the cut is now refused with a message saying why — noise can be
added back, never removed.
Streams wiped by anchor_shift_fresh_audio are not mid-trajectory, so they re-enter at
a=0 and get the full b*n a from-scratch run starts from, not a partial top-up of a
latent that no longer exists.
Continue mode (restart 0, the validated config) is unchanged and still bit-identical:
no re-noise, cost one full schedule. Rewind now costs the cut plus the restart tail
rather than a whole schedule plus the tail.
The scene-report NOTE about an ignored parameter is gone, because nothing is ignored.
Tooltips in samplers.py and engine_settings.js (Editor + Easy Gen share it) rewritten.
Tests: the old test asserting pass 1 runs to completion is replaced by one asserting it
stops at the cut; plus the re-noise identity at b==a, equivalence with comfy's scaling
at a==0, the coefficient/variance invariants on a synthetic latent with known x0 and
eps, reset-stream handling, non-mutation, and the below-the-cut refusal. Suite at
baseline: 22 pre-existing failures, 279 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Live result: 48 frames (6 latent) was needed for a quick-cut prompt to open
strictly on the requested action - the default 8 removes only the anchor and
leaves the settling-in stretch where the shot is still departing the reference
still and nothing happens yet.
Cited in the tooltips as a starting point FOR THIS PIPELINE (768x768x305@30 i2v
chain, quick-cut prompt), not a universal default, and the node default stays
8 - one workflow's tuned value is not evidence for another's.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Rewind mode runs pass 1 over the whole schedule (so the re-noise gets a clean
latent), which means anchor_shift_sigma has no effect there at all. A knob that
silently does nothing reads as broken - the scene report now names it and says
to set restart sigma to 0 if you want the cut point to matter. Noted in both
tooltips too.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Reported from the first live run: the idea works (opening reads as t2v,
character details preserved) but the output is full of noise artifacts, as if
under-denoised, with the last steps seemingly skipped. That was a real bug, not
the concept.
comfy's CONST scaling is x = sigma*noise + (1-sigma)*latent_image, and it
treats latent_image as a CLEAN image. Pass 2 was handed the PARTIALLY DENOISED
latent as latent_image, so at a 0.725 cut it entered at
0.725*fresh_noise + 0.275*picture — pass 1's content attenuated to a quarter
and buried under a full dose of new noise, with only the tail steps left to
recover. Exactly the reported symptom.
I cited segmented detailing's stage-2 tail as the precedent for this re-entry,
but that recipe feeds a FINISHED latent, where re-noising is correct. The
precedent does not transfer mid-trajectory. My error.
Fix — the two modes are now actually different things:
- CONTINUE (restart sigma 0, the default): pass 2 resumes from exactly the
state pass 1 handed over. Passing that state as BOTH noise and latent_image
collapses the CONST expression to an identity (s*x + (1-s)*x == x), so no
noise is added at all. Cost still one full schedule.
- REWIND (restart sigma > 0): pass 1 now runs the WHOLE schedule to a finished
clip first, the shift is applied to that, and pass 2 re-noises it upward and
denoises down. That is the img2img recipe, and it is valid precisely because
the input is finished. Costs a full schedule plus the tail.
Also fixed pass 2's progress offset, which restarted at pass 1's, so the bar
appeared to stall short of the end.
Two new tests reproduce the CONST arithmetic directly and assert the identity
holds in continue mode and does not in rewind mode. Their comfy stubs are now
snapshot/restored — the first version leaked into other modules and shifted the
suite off baseline (3 unrelated tests changed result). Back to baseline: 22
pre-existing failures, 272 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Found while checking whether this works in Easy Gen. It does - the knobs render
and persist there through the same shared engine_settings.js - but Easy Gen's
default scene has no anchor image, which exposed a hole: nothing checked that
the scene actually HAS a pinned i2v anchor. On a genuine t2v scene the slide
would have dropped real generated frames and regrown them at the end, for no
benefit at all. This is the "fake t2v" trick; it needs a real anchor to fake
it with. Now refused with that reason in the scene report.
Second guard: if anchor_shift_frames drops FEWER latent frames than the scene
actually pins, part of the anchor survives and stays visible - the exact
outcome the feature exists to prevent. The report now names the pinned count
and the value to raise the knob to.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
User's idea, and the framing is the point: this is "faking t2v" when it is
really i2v. Every existing way of stopping the anchor from dominating the
opening WEAKENS it first - ALG blurs it and loses character detail, and if you
under-blur you get both failure modes at once; Best-FaceID overlap tokens
approximate it and lose some detail too. This weakens nothing. The anchor is
pinned at full strength, untouched, for as long as it takes to transfer
identity/style/composition - and is then removed from the clip entirely.
Two passes:
pass 1 sigma[0] .. anchor_shift_sigma, anchor pinned as usual
shift drop the first N latent frames, slide everything left, refill the
freed tail, drop the pin (clip LENGTH is unchanged - nothing extra
is ever generated; the dropped head is regrown at the end)
pass 2 restart sigma .. 0, unpinned
Two passes rather than a roll between steps for a specific reason: pass 2
re-enters through comfy's CONST noise scaling (x = s*noise + (1-s)*latent),
which puts the shifted latent back ON the rectified-flow manifold. A raw
in-flight roll skips that, and project_sigmas records that this model cannot
recover from off-manifold latent states - that is what killed churn. Same
on-manifold re-entry that segmented detailing's stage-2 tail already uses.
Every choice is a knob, because WHERE to cut is the entire experiment and I
have no measurement for it: shift sigma (default 0.909, the first step where
structure actually forms - above it nothing has formed and the anchor would be
discarded before doing any work), frames dropped, pass-2 restart sigma
(0 = continue, higher = rewind for more freedom at the joint), what the
regrown tail starts from, and whether audio is rewritten in pass 2 (on by
default: pass 1's audio was scored against frames that have since moved).
Refused, loudly and with the reason in the scene report, on continuation
scenes (the head is carried frames, not an anchor), on scenes carrying guide
frames (the slide would overwrite them) and on scenes with JoyAI audio memory
(resetting audio would destroy the injected tail).
One real bug caught by the new tests while writing them: SIGMAS are float32,
so a threshold typed as 0.725 is a float64 sitting just below the schedule's
float32 0.725, and a bare `v <= s` cut one step LATER than the number entered.
Now matched against core's own isclose tolerance.
11 new tests; suite at baseline (22 pre-existing failures, no new). Editor
checked live: knobs render, gate on the toggle, persist to sampler_inputs.
UNVALIDATED LIVE - the shift sigma is yours to find.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
overlap/length (~28%) is the per-window redundancy, not the whole-clip cost.
The real figure is stride-based: at 145/40 each window advances 105 new frames,
so a windowed scene does ~1.45x the per-frame work regardless of length. That
is offset by attention, which is quadratic in one pass and near-flat when
windowed: ~1.05x at 200 frames, 0.68x at 305, 0.42x at 500. Net: roughly
break-even around 200 frames, a real win past ~300.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The plateau step-cache is keyed by (input.shape, cond_or_uncond), and every
context window inside one step calls the model with the SAME shape. With both
toggles on, windows 2..N would have been handed window 1's cached prediction
on every plateau step - one window's content bleeding across the whole clip,
silently, on a combination that is two ticked checkboxes away.
Context windows now wins and the cache is skipped, with the reason in the
scene report and in both tooltips, rather than quietly honouring one of two
requested settings. Caught by reasoning through the interaction, not by a
live run - neither feature has been validated on a rental yet.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two things, both driven by upstream ComfyUI moving 7 weeks ahead of the
local checkout (2cdaaf4a 2026-06-06 -> e651b7be 2026-07-28, 263 commits).
1. Segmented detailing crashed on ComfyUI v0.29.0. Upstream f8a3fd9d moved
the LTX latent upsampler onto DynamicVram and made `operations` a required
argument of LatentUpsampler.from_config; detailing.py still called it with
one arg, so the pending live probe would have died before detecting
anything. Now passes comfy.ops.disable_weight_init (what core's own loader
does) with a TypeError fallback, since rentals run whatever they pulled.
2. New Chain Sampler option: context_windows. Upstream cd77c551 (CORE-3)
added context-window sampling with real LTX2/LTXAV support - it unpacks
the packed AV latent, maps each video window onto its proportional audio
window, and re-slices denoise masks, keyframe_idxs and
guide_attention_entries per window, so anchors, mid-scene guides and JoyAI
memory survive inside a window. Nothing is ported: we build core's handler
and hang it on the scene's model_options exactly as the stock
LTXVContextWindows node does. Core picks it up in
comfy.samplers.calc_cond_batch, which every FunPack sampler already routes
through, so no sampler-loop change was needed and no graph change either -
it rides the sampler_inputs bridge into both Editor and Easy Gen.
Refuses (loudly, once per run, with a scene-report note) on cores without
BaseModel.map_context_window_to_modalities: the handler class predates
LTXAV support and would window the packed tensor blindly, which is the
same packed-vs-unpacked trap that killed DynamicConditioning. Install and
removal are per-scene under the existing wrapper-hygiene finally, so an
interrupt cannot leak the handler or its two patcher wrappers into later
runs.
Knobs (all exposed, none hardcoded): length, overlap, schedule, blend,
FreeNoise, and pin-anchor-in-every-window. Off by default. Engages only
on scenes longer than the window, and says so in the scene report when it
does not, so a scene that sampled whole never reads as a broken feature.
Verified: 8 new unit tests for the install/remove/capability-gate contract;
full suite at baseline (22 pre-existing cross-module stub failures, no new).
Editor checked live - knobs render in Engine > Chain Sampler > Experimental,
dependents gate on the toggle, and values persist into sampler_inputs.
UNVALIDATED LIVE on a rental.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
ComfyUI 7c59a07 ("Use comfy kitchen rope functions in ltx models", 2026-07-24)
changed the LTX positional-embedding payload from (cos, sin, split_mode) to
(rotation_matrix, split_mode), where the matrix is (B, T, heads, D, 2, 2)
holding [[cos, -sin], [sin, cos]].
rotate_overlap_freqs read pe[1] as the sin tensor — now a bool — so it raised,
and prepare_pe's bare `except: return pe` swallowed it. The Best-FaceID
reference tokens were still appended on the target's frame-0 coordinate grid
but WITHOUT the source-phase tag that marks them as a reference, so the model
read them as literal frame 0 and opened the clip on the reference image.
Anyone on ComfyUI >= 2026-07-24 saw i2v-like behaviour they couldn't bypass.
rotate_overlap_freqs now detects the layout at runtime (pe[1] is a tensor =>
legacy cos/sin; otherwise the matrix form, tokens on axis 1, frequencies on
-3) and composes the same R(angle + phase) either way. An unrecognised payload
raises instead of silently skipping the tag.
The four LTXBaseModel wrappers no longer fail silently: each reports once per
install that identity conditioning is not being applied and that a ComfyUI
update is the likely cause. A logged warning would have made this obvious.
Tests: both layouts rotate only the ref-token tail, produce identical angles,
reject unknown payloads, and the matrix form survives end-to-end through
_install_identity_overlap.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Recovers the original pitch: 'cheap, on-the-fly sharpen' was quietly
upgraded into a full re-denoise pass during design without ever offering
the free version as an option. The trained latent upsampler net was
already an intermediate step in the pipeline (crop_up) — sharpen mode
just stops there and pastes it back, with ZERO video-model forward
passes (only the small upsampler CNN runs). repair mode (default,
unchanged, byte-identical behavior) keeps the existing re-noise + 3-step
tail for cases that need actual structural correction, since only the
generative model itself can decide a region should look structurally
different — a super-res net can only sharpen what's already there.
Also extends the Editor's knobVisible with a array (AND'd
conditions) — detail_denoise needed to gate on BOTH segmented_detailing
AND detail_mode=='repair', which the existing single dependsOn/dependsValue
couldn't express without silently dropping one of the two checks.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Live report: a hand was detected, masked, and upsampled, but the pasted
result looked like a plain upscale, not an actually-corrected region.
Root cause is mechanical, not a bug: 0.85 is Lightricks' own official
re-entry sigma, but it was designed for their use case (sharpen an
already-correct frame at higher resolution), not ours (repair a likely-
malformed crop). At that re-entry point the tail mostly smooths the
upsampled interpolation back down rather than giving the model enough
noise/freedom to genuinely reconstruct structure.
detail_denoise (0.3-0.99, default 0.85 = unchanged prior behavior) now
controls the re-entry point; the tail's shape (ratio between its 3 steps)
scales with it via the new stage2_sigmas() instead of a fixed tuple, so
raising it doesn't just enlarge the first jump. Higher = more genuine
repair freedom at some risk of drifting from the surrounding frame.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 35% area cap was refusing a user-requested 'hand' region (47% of
frame) with no way to override it — the code was unilaterally deciding
the request 'wasn't a detail pass' when the user never asked for that
judgment. Fair complaint: cost guardrails are not the same as content
decisions.
detail_max_area is now threaded end-to-end (detailing.area_cap param ->
sampler widget -> Editor knob in Chain Sampler > Experimental, 0.05-1.0,
default 0.35 unchanged). Raising it lets a region of any size through;
1.0 = no cap. The refusal note now points at the actual setting name
instead of citing a fixed constant.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Live run hit 'no region matched' with zero diagnostics: CLIPSeg's raw
sigmoid score for a real, correctly-named region is often well under a
naive 0.5, so a miss against the hardcoded 0.35 threshold was
indistinguishable from 'nothing there.' Two fixes:
- find_tube now takes an optional diag dict and always records max_heat
(plus area_cap vs below_threshold reason on a miss).
- detail_refine_scene always returns a descriptive note once detection
actually ran ('no match: max CLIPSeg score 0.XX < threshold 0.35 — try
a lower detail_threshold...'), replacing the generic
'no region matched' the sampler used to fabricate.
- detail_threshold is now a real widget/sampler_input (was a hardcoded
constant with no UI — a hidden-setting violation once real runs started
needing it), in Engine > Chain Sampler > Experimental next to the other
detailing knobs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The first live attempt silently no-oped: detail_upsampler defaulted to 'None'
(nothing installed on the rental) and the guard skipped without a word.
Two fixes:
- detail_upsampler default is now 'auto': prefer the official installed file,
else the highest-versioned spatial upscaler (version-aware compare — plain
sort ranked ltx2_3_… over ltx-2.3-…-1.1), else hf_hub_download of
Lightricks/LTX-2.3 ltx-2.3-spatial-upscaler-x2-1.1.safetensors (~1 GB, once)
with an actionable error if the repo is gated. Legacy 'None' = auto.
- With the toggle on, every outcome is visible: upsampler choice and skips
print to console AND land in the scene report (segmented_detail(SKIPPED: …) /
(no region matched)); resolve/load failure disables the pass for the run
instead of retrying the download per scene. A failed pass still never costs
the scene.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ADetailer-for-video probe core. After a scene denoises, CLIPSeg locates
user-named regions (hands, feet, ...) on decoded keyframes; the region is
cropped as a spatiotemporal tube, 2x-upsampled through Lightricks' trained
LatentUpsampler (comfy core, no port needed), re-denoised over the official
stage-2 tail (0.85/0.725/0.4219/0), downscaled back and pasted through the
feathered CLIPSeg silhouette. Final resolution unchanged; audio protected by
construction (refined audio discarded); tubes >35% frame refused; empty
targets/no-detection = bit-identical no-op. UNVALIDATED LIVE.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Punchy frame_rate mult held over the first/last slice of a scene's
denoise progress, easing to natural in the middle - same progress-
from-sigma mechanism as pulse, shaped as a single edge ramp instead
of repeated segments. Wired through build_enhancements (single-scene),
the Scene Chain Sampler's per-scene wrapper dispatch, and both Studio
and Editor temporal-style dropdowns.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Guide-carrying scenes no longer disable the loop temporal style. Appended
guide frames always live at the tail of the video stream's T range, and
keyframe_idxs is per-token — so the wrapper derives the tail frame count
(tokens / (H*W), same math as comfy's get_keyframe_idxs) and rolls only the
content region [0, T - tail); the tail, keyframe_idxs, and attention entries
that reference it stay canonical. The audio stream's appended memory (JoyAI,
rides a trailing mask=0 run) is detected the same way from audio_denoise_mask.
Guides keep informing every rolled position — their influence smears across
the cycle, which is what identity/style references should do in a loop — and
are never dragged into the rendered content.
Unparseable guide contexts (malformed keyframe_idxs, entries without idxs)
still run canonical: never risk a corrupting roll. Content-region size gates
the minimum-frames check now, so a short clip whose frames are mostly guide
tail stays canonical too.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 'loop' temporal style (in the style list since forever, but inert) now does
what WanVideoWrapper's Loop Args does for WAN: on eligible denoise steps the
packed latent is cyclically rolled along its temporal dim before the forward and
the prediction unrolled after, so the model repeatedly sees the video's seam
(last|first frame) as an interior frame pair at step-varying positions (van der
Corput sequence, reset per scene) and smooths it into a natural loop. Video and
audio streams roll by their own frame counts and stay time-aligned; denoise
masks roll in step, so i2v anchor per-token timesteps stay consistent.
Near-zero overhead: two clones + torch.roll per eligible step, no extra forward
passes. Rolls only below the near-noise plateau (nothing to smooth up there, and
the plateau step-cache stays valid). Calls carrying guides are left canonical —
keyframe_idxs pins appended guide frames to absolute positions inside the same
T range, so rolling them would corrupt; on guide-carrying scenes the roll is
inert and the run mechanism log says so.
Wiring: concrete style via the global build_enhancements path (innermost
relative to per-scene wrappers, as required — prediction-modifying wrappers see
canonical orientation only); auto director's funpack_temporal_loop via a tagged
per-scene wrapper on the Scene Chain sampler, installed below the guidance
stack. Editor already exposes 'loop' in the temporal style dropdown.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
sample_funpack_distilled_flow built the alg_latents dict eagerly, and the
(True, True) value used 'anchor_blurred or tail_blurred' as its fallback when
both_blurred was None. When anchor blur is on but tail blur is off (or the tail
blur returned None), anchor_blurred is a multi-element tensor, so 'or' calls
bool() on it -> 'Boolean value of Tensor with more than one value is ambiguous'.
Surfaces only on continuation scenes: an anchorless first scene (identity/empty
latent, no denoise_mask) skips ALG entirely, but a later scene with a guide mask
enters ALG and trips the eager dict build. Replaced the tensor 'or' with explicit
None chaining (both -> anchor -> tail). Pure fallback; when (True,True) is actually
indexed both blurs are on, so both_blurred is non-None and the singles are never
reached. Independent of taste_nearest_prompt, which was merely on in the same run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds taste_nearest_prompt on the Scene Chain sampler. On every liked rating the
Refiner now records a (pooled-conditioning fingerprint -> liked direction) entry in a
ring buffer beside the global liked_dir. When the toggle is on, each scene retrieves
the similarity-weighted direction of its nearest rated prompts and uses that for
embed_guidance / score_slider instead of the one global average, falling back to the
global direction when nothing is close enough. Non-parametric (cosine lookup + vector
mean, no extra model forward, no value-function collapse risk). Wired into the Editor
Guidance panel. UNVALIDATED LIVE.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
MixCache/Chorus-family step-caching adapted to LTX2.3's distilled schedule.
On the near-pure-noise plateau (sigma >= threshold, default 0.975) the
transformer output barely changes, so compute the base-model forward once
and reuse it for the remaining plateau steps, skipping ~3-4 of 8 full
48-block forwards. Below threshold every step computes for real and the
cache drops, so structure formation is never approximated. Deterministic
given seed -> safe in Batch Training, no rating-diversity impact.
Installed innermost as a per-scene model_function_wrapper (existing
tag/strip leak hygiene) so guidance wrappers still post-process each step's
cached-or-fresh prediction. Cache keyed by (input.shape, cond_or_uncond).
Editor: plateau_cache + plateau_cache_threshold in Engine -> Experimental,
auto-wired via the sampler_inputs bridge (no backend/builder change).
Off by default, UNVALIDATED LIVE. (Chorus cross-variant reuse rejected --
would blur batch-rating diversity; FreeSpec dropped -- wrong topology.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
carry_overlap_through_anchor lets a scene keep the previous scene's latent
tail for background/environment continuity even when it switches to its own
i2v anchor, instead of the current hard cut. The anchor's own leading frame
stays untouched; only the frames after it carry the old tail.
Also fixes a real gap found along the way: the mixed-anchor branch never
called _apply_configured_guides, so Best-FaceID identity_transfer could
never resolve an identity_pin on an anchor-swap scene — exactly where it's
most useful. Added a standalone lookup so it resolves there too.
Replaces the source-phase-RoPE-only approximation with a faithful native port of
ComfyUI-BFSNodes' LTX Identity Transfer (read upstream source directly): the reference
face is now injected as separate, non-rendered overlap tokens (clean timestep, RoPE
source-phase tag, trimmed before output) instead of blending into Continuity's identity-pin
keyframe, plus an optional ArcFace embedding -> IdentityProjector -> context-token channel
(identity_projector, source_id, phase_scale, id_strength, arcface_mode, debug_log). Stays
on Chain Sampler per the Studio/Sampler boundary law's guide-ops carve-out. New
identity_transfer.py module; insightface added as a soft pip dependency.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Reverts the earlier ComfyUI-BFSNodes graph-splice approach (builder.py/timeline.py/
server.py scaffolding) in favor of porting the missing piece natively: FunPack's own
guide-attention path (LTXVAddGuide.add_keyframe_index + guide_attention_entries)
already places a reference image on the frame-0 grid exactly like BFSNodes' overlap
technique, and Continuity's existing "Identity pin" already applies one reference
face to every scene. The only real gap was the LoRA's trained-time source_phase RoPE
tag, added as a tag+strip patch on FunPackLTXAVSceneChainSampler scoped to the exact
guide-entry token range (never co-active mid_scene/carry/prior-scene guides sharing
the same keyframe_idxs tensor). New "Best-FaceID compatibility" + "Source-phase id"
knobs live under Engine Settings -> Experimental, alongside every other unvalidated
technique. LoRA loading itself reuses the existing generic Models-menu lora role.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
alg_blur_guides no longer borrows the sampler's anchor knobs: the Scene Chain
Sampler gains alg_guide_blur_strength (default 2.0) and
alg_guide_blur_sigma_threshold (default 0.975), threaded per-chunk via
extra_options like the tail count. The sampler now precomputes one latent per
(anchor blurred?, tail blurred?) combination — the frame sets are disjoint, so
the both-blurred variant composes the two passes — and each region follows its
own sigma window per step. Anchor ALG (alg_enabled/alg_strength/
alg_sigma_threshold) is untouched and back to anchor-only semantics.
Editor: 'Guide blur strength' and 'Guide blur sigma threshold' rows appear
under Engine ▸ Experimental when the blur toggle is on (sampler_inputs bridge,
names match the new node widgets; inputs appended at the block end to keep the
builder's positional widget mapping aligned).
Verified live: object_info shows both widgets appended last; Editor rows
appear on toggle and persist to project.sampler_inputs. 11 ALG unit tests
pass (new: composed independent-strength blur); same 20 pre-existing suite
failures.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
alg_blur_guides now works with the sampler's alg_enabled off: the anchor
stays sharp and only the newly-appended guide-attention frames (mid-scene
guide, carried guides, per-scene guides, JoyAI memory) are blurred, reusing
the same alg_strength / alg_sigma_threshold parameters and sigma window.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
'Awful' / 'Wrong appearance' banked on intrusion semantics; now any rating
whose profile reward marks the run a bad OUTCOME (<= -0.25: Missing
quality, Missing details+quality, Missing action+quality, Missing
details+action+quality) also feeds the negative bank - degradation is
visible in the latent, so steering off that attractor is meaningful.
Near-miss ratings with positive reward (Missing details +0.35, Missing
action +0.05) stay excluded: those latents are mostly-wanted content and
banking them would repel future runs from material the user asked for.
Decision moves into negative_memory.is_negative_profile (intrusion keys OR
reward threshold, robust to future profiles); promoted entries now carry
the rating key as provenance for severity-weighted steering later.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Store what went WRONG, not just what was asked for. Ratings 'Awful' and
'Wrong appearance' promote that run's final video latent (fp16, batch
squeezed) + mean-pooled conditioning into a per-key negative bank
(<key>.negative_latents.pt, ring buffer of 8). The Chain Sampler writes a
pending candidate at the end of every keyed run; the rating that scores
that run promotes or discards it - same pairing as the x0 snapshot, and
the stem glob in delete_refinement_key covers both new sidecars.
Sampling (dynashift toggle + strength/threshold, off by default):
per late step each current video frame is pooled and compared against
every banked negative frame (alignment-free in time, so chain position /
guide tails don't matter; other resolutions are skipped). Frames matching
above threshold get the matched negative frame's aligned component
subtracted (projection removal, coef clamped >= 0), scaled by
above-threshold margin - steering self-releases once the unwanted feature
is gone. Negatives are weighted by prompt similarity so bad gens from
unrelated prompts steer less. No uncond pass, no extra model forward;
audio byte-identical (video span slice + cat). Wrapper rides the tagged
per-scene lifecycle (leak-strip covered) and threads through batch
training.
Editor: DynaShift knobs land in Engine Settings > Guidance same-change
(sampler_inputs pass-through, no backend change), verified rendering live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two audit findings on the experimental feature stack:
1. Leak: an interrupt or OOM inside _sample_chunk skipped the per-feature
unwind blocks, leaking embed_guidance/score_slider/output_guidance/temporal
wrappers on the shared ModelPatcher - every later run then chained new
wrappers onto the leaked ones and double-steered (same in-process
progressive-corruption mode as the old block-hook leak).
- one base-wrapper snapshot per scene, restored in the same finally that
removes the v2a hooks; the four conditional unwind blocks are gone
- every per-scene wrapper is tagged with its predecessor; sample() strips
leaked tagged chains at run start (mirrors strip_funpack_block_hooks)
2. Inert feature: output_guidance applied the RAW value-fn gradient, whose
per-element magnitude is O(1/pool_window) - at video-latent sizes the
correction was ~1e-6 relative, i.e. numerically a no-op. It also fed the
full packed AV tensor to a value function trained on the video stream
alone (shifted pooling buckets + audio contamination).
- gradient now computed on the packed latent's video span only (matches
the training domain of _save_output_value_snapshot)
- delta norm-calibrated: strength = fraction of video-stream norm per
fully-ramped step (0.02 = 2%), same relative convention as the slider
- audio region byte-identical by construction (cat, not post-hoc mask)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The per-scene line now includes run_mechanisms, so a slow guide scene
attributes itself fully: which feature appended the tokens
(custom_guide_stack / carry_i2v_guides / joyai_memory / mid_scene_guide
/ soft_continuation) alongside the tail counts and sampling seconds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Live report: guide scenes suddenly ~3.5x slower than the anchor scene
(6-scene 768x768x305 montage at 7 min vs ~2.5 expected), on a nightly
ComfyUI. Audit of the upstream delta found nothing touching the default
guide attention path (context windows is opt-in, int8 irrelevant, the
guide-strength mask machinery dates to February) - so before theorizing
further, make each run attribute itself: every scene line now carries
its sampling seconds plus what was appended to its sequence (carried
head frames, guide tail, audio tail), printed live and in the report.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Block steering: run-averaged block activity is architecture-determined
(varies in the 4th decimal run-to-run), so no scoring scheme - pole EMAs
or reward correlation - could extract a rating signal from it. The idea
needs within-run per-step measurement to have a chance; that is a
research project, not a feature. Removed: module, sampler wiring, the
two trailing widgets (positional reference mapping unaffected), the
Refiner pairing hook, editor knobs, tests. Stale block_snapshot/
block_profile sidecars are inert and covered by the existing
delete_refinement_key sweep.
FunPackNVFP4ModelLoader: the FP4 GEMM path was real (3.35x at 16k
tokens) but end-to-end the user's runs are dominated by everything the
loader cannot touch (attention, prep, encode - the sampler node is
~23s of a ~60s run) while dynamic input quantization added a fixed
per-layer-call toll. Net observed: overhead. Users are better served
by dedicated NVFP4 loaders. Removed: module, registration, tests.
Kept (independently useful): the chain sampler's phase timing report +
quantized-layer census, the nested-AV x0 snapshot fix, and the sm_120
masked-attention work.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Live report: "flat activity across runs" with a working profile. Root
cause is real, not a threshold tuning issue: the fingerprint is each
block's MEAN residual activity over ~1200 steps, which is practically a
property of the architecture - run-to-run variation sits in the 4th
decimal, so raw reward-covariance is numerically dead. Lowering the
epsilon would have "fixed" it by steering on noise.
Now: per-block Pearson CORRELATION between reward and activity (scale-
free - tiny variations count when they consistently track ratings),
fingerprints stored mean-normalized so run-global scale (step counts,
schedulers) can't masquerade as block signal, and a confidence ramp
(min(1,(runs-2)/6)) so a 3-run profile steers gently instead of not at
all, reaching full strength ~8 runs. MIN_RATED_RUNS 2->3 (correlation
over 2 runs is degenerate). Status lines now carry the evidence:
"ready (N runs, peak |corr| X, confidence Y)". Older profiles are
discarded on first pairing (fingerprint semantics changed).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
"The matmuls got faster" and "the video arrives sooner" are different
claims. The run report and log now show total = sampling + VAE decode +
other with percentages, so an optimization's real ceiling is visible
(at 768x768x305 the ~22k-token attention rivals all linears combined,
and decode/prep are NVFP4-blind - a 2x on quantized GEMMs can cap out
around seconds of a one-minute run).
Also logs a quantized-layer census at run start: layers forced OFF the
quantized matmul path by weight patches (LoRA / refiner attn deltas) or
emulation dequantize on EVERY call - silently slower than bf16.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Three live-run findings from the rental:
1. Block steering said "profile not ready (needs 2+ rated runs)" with 16
paired runs. The v1 liked/disliked-EMA attribution had no contrast when
ratings cluster on one pole (pole-vs-all-runs-mean cancels to zero) and
dropped rewards in (0, 0.3) entirely. Attribution is now the per-block
COVARIANCE between reward and activity over a rolling 32-run history:
every admissible reward contributes with its actual value, contrast
comes from reward variance, and the sampler log now states the real
blocker ("all rated alike - rate both good AND bad runs") instead of a
misleading run count. v1 profiles are discarded on first pairing.
2. output_guidance stayed "not ready (needs 10+)" forever on LTX-AV: the
end-of-run x0 snapshot was gated on output samples being a plain
torch.Tensor, but AV chain output is a NESTED tensor (video+audio) - so
the output value function never received a single sample. The snapshot
now extracts the video stream from nested outputs.
3. FunPackNVFP4ModelLoader hit a size-mismatch wall (9-vs-6 ada params,
connector dims): it loaded the state dict without the safetensors
METADATA that carries the LTX-AV architecture config, so detection
built the wrong model variant. Metadata now flows through, mirroring
comfy.sd.load_diffusion_model.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RLHF at the model level: which REGIONS of the transformer were doing the
work when a generation came out good or bad, and steer accordingly.
1. Measure: tagged forward hooks record each block's relative video-residual
delta per model call (near-zero cost; covered by the existing leak-strip).
2. Attribute: the end-of-run fingerprint is staged (block_snapshot.pt, same
stage-then-pair convention as x0_snapshot) and paired with the rating
reward on the next Refiner cycle into liked/disliked/all-runs EMA
profiles; when output_guidance's value function is trained, an innermost
model-function wrapper additionally credits blocks per step by
(value delta) x (centered activity) - converges in a handful of runs.
3. Steer: vx' = vx_in + g_b*(vx_out - vx_in) on the video stream only,
audio untouched; scores zero-mean/unit-max so emphasis is redistributed
rather than inflated, gains hard-capped at +/-10%. Exact no-op until the
profile has 2+ paired runs; blocks at gain 1.0 get no hook at all.
Chain Sampler toggle + strength (append-only widgets), exposed same-change
in the editor's Chain Sampler Guidance section via the sampler_inputs
bridge. New sidecars ride the defensive <key>.* sweep in
delete_refinement_key, so atomic key deletion covers them.
Tests: toy AV-block model driving the real hooks (recorder ranking, steer
video-only scaling + native path for unit gains, credit wrapper, snapshot/
profile/scores/gains roundtrip, mild-positive band, depth-mismatch reset,
leak-strip tagging). Full suite failure set identical to the pre-existing
baseline.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The free video-only unsharp-on-x0 sharpening (temporal high-pass between
consecutive denoised predictions, no extra model eval) only existed on
Hybrid Euler 2S. Distilled Flow — the recommended sampler for this
pipeline (deterministic, clean audio) — had no equivalent, so the
"quality phase" of the actual production sampler had no sharpening path
at all.
Ports the same _apply_quality_sharpness call into
sample_funpack_distilled_flow, gated to the existing final-correction
(Heun) steps. Threaded through FunPackDistilledFlowSampler's
INPUT_TYPES/get_sampler, Studio's _build_one_sampler, and the Editor's
sampler_panel.js Distilled Flow section — verified live end-to-end
(UI -> studio_settings JSON -> backend build call).
