ec90c19d2f8873ba32c72b2768b046135b00db37
176
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 digital-garbage
|
e1a907a93b |
fix: redirect the legacy /movie and /easy API prefix, not just the page
Old frontends build their API base from location.pathname, so a page left open on the old URL polled /funpack/<old>/api/health, got 404, and sat on "Reloading ComfyUI" forever — never reaching the reload that would have hit the page redirect. |
||
|
digital-garbageandClaude Opus 5
|
bf73835e3b |
fix: wiring survives a project load; Frames says when it reaches nothing; anchor survives upscale_2x
Three unrelated reports. Wires lost or "(not allowed)" after loading a project. /api/pipeline-ports read the family from the GLOBAL models.json, so it described whichever project was saved last. Open one on the other family and its saved wires point at ports that family has no rule for. It takes ?pid= now, and the panel asks for the config before the rules that depend on it. Reconcile also dropped a port wire when core_overrides named a deleted slot — it cleared every other slot's wire to that port, then failed to add the replacement. Guided mode also filtered harder than the builder validates. A node added through "Any node…" has role `custom`, which no rule table mentions, so a custom VAE loader could not reach the pipeline at all. A role with no rule for an output type now reaches any port of that type; a role that HAS one keeps it, and internal links stay closed. Both sides read the same list, since the panel hiding what the builder accepts is what produced "(not allowed)". Frames / scene silently missed shots. Any timeline resize, trim or split moves a scene off project length for good, and nothing said so — typing 305 and rendering 450 read as the edit being ignored. The Project panel now says how many shots it doesn't reach and offers to put them back. The panel's fingerprint had to learn about scene state, or the warning outlived the fix. second_pass_op=upscale_2x no longer drops the i2v pin. The pinned frames in the upscaled latent ARE the anchor — carried through the same upsampler as the rest of the clip — so only the mask needed scaling to the new grid. Pass 2 at 4x the pixels was the run most likely to drift off the reference, and it was the one running unpinned. The pin now holds the upscaled anchor rather than the encoded source image, and the report says so. Guide keyframes are still dropped: those are token indices, not pixels. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
b56900b499 |
fix(refiner): follow the encoder instead of assuming Gemma3
LTX-2.5 encodes with Gemma4 where 2.3 used Gemma3. Concept steering finds a phrase by matching its token ids inside the prompt's, so a tokenizer from the wrong family still returns spans — pointing at the wrong tokens. Nothing raises; steering just stops working. The tokenizer now comes from the connected CLIP, which is by construction the one that produced the conditioning. The download stays as the fallback for runs with no CLIP, and says so rather than claiming to be Gemma3. Learned state is stamped with its encoder and cleared when that changes, on both the V1 and V2 paths. Clearing reuses Session Reset so the value function, x0 snapshots and the keyless Absolute store go with it instead of being orphaned. An unstamped key is adopted, never wiped: every key predating this has no stamp. The vision probe no longer depends on the submodule being called gemma3_12b — it still is on 2.5, where it holds Gemma4. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
7329cd9680 |
fix: multi-scene chains work with a resolution-changing second pass
second_pass_op="upscale_2x" hands a scene back at twice the latent size while every LATER scene is still built from latent_template at the original size. Everything that crosses a scene boundary — carried overlap frames, the anchor's continuation, the soft join, the JoyAI memory frame, per-scene guide sources — is spliced into a chunk on the template's grid, so the second scene died on the shape mismatch and the combination was simply unavailable. One choke point, _match_template_resolution, brings the carried material back before it is used, applied at the three places a finished scene crosses into the next one. Each scene still samples and OUTPUTS at 2x: only what is carried gets resampled, and only downwards, which is the direction that survives it — those frames exist to say "continue from here", which a resample preserves far better than invented detail would. Identity no-op when the grids already agree, so any run without a resizing op is bit-identical. Audio has no spatial axes and is never touched. A stale noise_mask is dropped rather than reapplied at the wrong scale, and a resample that throws costs continuity with a console line, not the run. No grid guard on the chain blend: second_pass and second_pass_op are run-level settings applied to every scene, so a chain cannot hold two resolutions — every scene is upscaled or none is. Both tooltips said this combination dropped multi-scene; they now say how it works. Also: About reads "FunPack 3" with the codename on its own line, rather than attaching the major to the codename. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
98d48e78e3 |
fix: cut_opening_frames cuts decoded pixels, not latents (noisy first frame)
The opening frame came out noisy because the cut was made in latent space. The LTX video VAE is causal: latent frame 0 is the temporal origin and decodes to a single pixel frame, while every later latent frame was generated as a CONTINUATION of the frames before it. Slicing the front off the latent promoted a continuation frame to position 0, and the decoder then applied the temporal-origin handling that frame was never generated for. H3 already cut decoded images for unrelated reasons and so never had the bug; LTX now does the same thing. The count was wrong too, quietly. A latent cut could only remove whole latent frames, and the promoted frame changed from covering 1 pixel to covering `scale`, so dropping k latent frames shortened the clip by k*scale rather than the (k-1)*scale+1 it actually spanned. The value was also rounded up to a whole latent frame on the way in. It is now exact: N means N, matching H3. Each eligible scene records a (start, count) pixel span instead of cutting; the spans are removed after decode, later ones first so earlier removals cannot shift their indices. Mid-chain hard-cut scenes keep working — the span is computed from the scene's position in the chain, which is exact because anchor scenes append with zero blend overlap. The latent's audio stream loses the same stretches of TIME, mapped by proportion on its own axis, or sound would run ahead of picture with no error. The latent's VIDEO stream is deliberately left whole (cutting it is the bug), so video comes from the IMAGES output — which the Editor already wires. A run with IMAGES unconnected now skips with a reason instead of silently doing nothing. _cut_opening_latent is removed rather than left dead, since its docstring documented the broken approach as correct. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
e7b624d684 |
fix: drop guide keyframes on a resized second pass; unify the two ALG switches
Two unrelated fixes.
**Guide keyframes vs a resolution-changing second_pass_op.** Guide keyframes are
recorded as TOKEN indices into pass 1's latent grid, so upscale_2x — which
doubles each spatial dim and thus quadruples the tokens per latent frame —
leaves them addressing the wrong tokens. Since the LTX-2.5 commit core rejects
that outright ("keyframe_idxs holds N tokens, which is not a whole number of
M-token latent frames"), which is how this surfaced; older cores silently
mis-placed the guides instead, which is worse. Pass 2 now drops them and says
so in the scene report, exactly as segmented detailing already does for its
crop — so detailing's stripper becomes public rather than being duplicated. Only
pass 2 is affected; pass 1 ran at the grid the guides were recorded against and
keeps every one. The pin loss and the guide loss now share one predicate
(_latent_spatial_changed) so they cannot drift apart, and an unreadable shape
answers "no change" so a probe failure never costs a guide.
**One ALG control instead of two.** The Distilled Flow panel had its own
alg_enabled while the chain sampler's alg_anchor did the same guidance on any
sampler and already drove the Distilled Flow one when set — two switches for one
behaviour, with precedence depending on which you had touched. The Editor now
shows only alg_anchor. The node input stays for hand-built graphs; what moved is
the Editor's control. A project still carrying the old switch is migrated once,
strength and threshold included, and told so, rather than left running ALG with
nothing on screen to turn it off.
The migration is deferred out of the render pass and guarded by a module-level
set of project ids: every store write re-renders the pane synchronously, so
migrating inline recursed until the stack blew (caught in the browser, not by
the suite).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
digital-garbageandClaude Opus 5
|
03158b4edf |
fix: two LTX-2.5 assumptions that would have failed silently
LTX 2.5 reuses LTXAVModel/LTXBaseModel behind config flags, so the samplers, the identity-transfer wrappers and the guide path all bind to it unchanged. Two places assumed 2.3 specifics, and both would have gone wrong quietly. The AV channel split was a hardcoded width table. From 2.5 on comfy reads the video/audio projection widths off the checkpoint rather than assuming them, so a checkpoint can ship a pair the table has never seen — and an unrecognised width made ltxav_video_channels return the FULL width, which silently steers the audio text context along with the video one. The only symptom would have been degraded audio. The split is now measured off the live model when one is on hand; the table stays as the fallback. Decode-time noise only ever reached VideoVAE.decode. 2.5's diffusion decoder takes no timestep and seeds itself, but it is an nn.Module, so stamping the attributes onto it succeeded and was then ignored — a knob that reads as live and does nothing. It is now capability-tested against the real contract (hasattr decode_timestep, not a class name) and says so when it cannot apply. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
48ff7f0f48 |
feat: schedules for every sampler, a usable frames field, ALG on any sampler
Three things that were each locked to one implementation detail: - steps + schedule were a KSampler-branch feature, though every sampler here walks a sigma list handed in from outside. They are now pass-level, so Distilled Flow and Hybrid Euler 2S can run a named schedule (and the second pass can be computed rather than typed) in both frontends and on the node. - the project's Frames field snapped to the model's grid on every family. On H3 that is required, but its arrows moved by 1 and the snap put the value straight back; on LTX the builder rounds up by itself, so the snap only fought the user. H3 now steps a whole 17 frames per arrow, LTX keeps what you type and walks 8k+1 with its arrows. - ALG lived inside the Distilled Flow step loop, so it was unreachable with a stock KSampler. Its blurred/sharp swap is decided by the step's sigma, and sigma is an argument of every model call, so a denoiser proxy drives it from outside any sampler. New alg_anchor toggle on the Chain Sampler drives the in-loop version too, so there is one control wherever you are. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
4e6c68bf51 |
feat: the project remembers your Editor settings, so a new rental doesn't reset them
Every Editor preference was per-MACHINE: autocomplete, shortcut ideas, the anchor toggle and the custom-i2v bypass in browser localStorage, and the shortcut revolver in a server-side sidecar next to the shortcut DB. A freshly rented box has neither, so all of them silently revert to their defaults and have to be set by hand again — the revolver's Random mode every single time. They now ride along in the project file (Project.editor_settings), and opening a project puts them back. The revolver is still a server setting shared by every browser and by generation; the project only restores it, and only POSTs when the value actually differs, so opening a project that already agrees never restarts a live cycle. Two cases that would otherwise leave the feature useless for an existing setup: * A project saved before this existed, opened on a configured machine, adopts the machine's current settings instead of staying blank — including reading the server's current revolver mode. Nothing changes in the moment; it just stops the project from travelling empty. * setEditorSetting mirrors into the project BEFORE its unchanged-value return. Re-picking a value you already have is a no-op, so without this the settings you never touch again — exactly the ones you'd lose — would never be recorded. An ABSENT key still means "leave this browser alone": loading an old project must never overwrite a configured setting with a default. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
c37319fb2a |
feat: settings say what they do in one line; cut_opening_frames works on H3
Engine settings were unreadable: every experimental knob carried a paragraph about cost figures, sampler caveats and failure modes, so nobody read any of it. The hint is now ONE sentence saying what the setting does, and the paragraph moves behind a collapsed "Details" link — still the only place some of those numbers exist, just not in the way of the checkbox. cut_opening_frames also learns MiniMax H3. It could never engage there: H3's anchor is a keyframe condition row rather than a pinned latent prefix, so the t2v guard read every H3 scene as anchorless, and its video latent sits on a 5k+2 grid an arbitrary cut cannot land on. On H3 the frames now come off the head of the DECODED batch instead — exact to the frame — with the latent's audio stream cropped by the same proportion of time, because audio decodes from this node's LATENT output and would otherwise start N frames early. The latent keeps its full video stream (off-grid the moment it is cut) and the scene report says so. Fixes a latent bug on the way: extra streams were cropped on dim 2 with the video stream's frame count. On H3 dim 2 is the stereo channel and time is last, so that would have cropped the audio to mono and left its duration untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
532361fb49 |
feat: h3_audio_clock works with stock ComfyUI samplers too
The previous commit claimed the correction could not reach a stock sampler. That was wrong. It can: the correction moves from the step to the prediction, because a rectified-flow euler update is x*r + denoised*(1-r), so scaling the audio part of (x - denoised) by the same factor lands the audio on its own grid. A proxy denoiser wraps any KSAMPLER-shaped SAMPLER and does exactly that. Tying a model call to a step is the real constraint, and it is checked rather than assumed: the proxy verifies on every call that the sigma it was handed is this step's scheduled one, and switches itself off with a console note the moment that stops holding. Samplers that evaluate twice per step (heun, dpm_2, dpmpp_2s/sde, seeds_2) trip that on their first corrector call and are left untouched. Measured against comfy's real samplers with a perfect rectified-flow predictor, so every number is pure schedule error (audio error as a % of the stream's range, 4/8/20 steps): euler 85/38/14 -> 0/0/0 exact at every step count res_multistep 69/18/1 -> 21/23/15 helped at 4 steps, HURT at 20 dpmpp_2m 69/18/0.4 -> 20/22/14 same heun, dpm_2 unchanged (proxy stands down) The lesson worth keeping: this is a first-order integration error, so a higher-order sampler already absorbs most of it once the schedule is fine — which explains why res_multistep and er_sde behaved well on H3 at 20 steps while plain-euler paths did not. At 4 steps nothing absorbs it (everything sits at 68-85%) and only the clock fixes it, which is the regime turbo LoRAs put you in. Not gated on any of this: the user picks the sampler, and the tooltip, the Editor hint and a per-run console note carry the numbers so the choice is informed. Also corrects two claims from the previous commit — "no-op above 20 steps" (it is 14% on euler at 20) and "video is bit-identical" (the streams share one attention sequence). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
132feed08b |
feat: H3 audio clock — integrate audio on its own flow schedule
MiniMax H3 denoises video and audio on two different flow schedules (shift 12 / shift 3) but hands the sampler a single sigma grid, so the DiT reconciles them by scaling the audio velocity by d(sigma_a)/d(sigma_v) measured at the START of each step. Exact in the continuous limit; on a few-step schedule the tangent badly overshoots the chord it stands in for. On a 4-step shift-12 grid the final step drives the audio ~2.5x past where its own schedule puts it, which is heard as distortion — the reason the turbo-LoRA community ships a custom sampler for it. h3_audio_clock on the Chain Sampler swaps that start-of-step slope for the chord that actually spans the step, so audio lands on its own grid. One scalar multiply per step: no extra model call, no extra memory. The video stream is untouched. Above ~20 steps the two agree and it does nothing, so it is aimed at turbo / distilled schedules. Wired into the two FunPack sampler functions H3 can reach (Distilled Flow, and Hybrid Euler 2S via its CONST rectified-flow path). A stock ComfyUI sampler cannot be reached from outside its loop and says so rather than looking active. Off by default, UNVALIDATED LIVE. Editor: Settings > Engine > Experimental, plus the mirror-image inert chip — H3-only settings left on against an LTX pipeline now warn beside Generate the same way H3-dead settings already do. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
fdee1fea1a |
fix: spare H3 token tags are trimmed, not thrown away
An image prompt comes back one tag long: comfy builds minimax_token_tags from the tokenizer's INPUT sequence (embeds.shape[1]) while the conditioning is what the encoder returned, and the vision block's expansion does not survive 1:1. The reconcile treated any surplus as corruption and dropped the whole vector — which on exactly those prompts discards the modality-0 marks for the image span and leaves the DiT modulating the picture as if it were text. Position i still means position i, so the tail is trimmed instead. Growing (appended tokens) is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
fdf39bbd03 |
fix: H3 image conditioning reaches the model
Two independent, silent breaks made an H3 i2v scene generate from the prompt alone. 1. The Editor always runs split_by_transitions, and the per-scene conditionings REPLACE the single entry Studio built — but they were re-encoded with no reference_image and no h3_references. Both the frame-0 keyframe pin and the Qwen vision presentation were gone before the sampler looked, even for a one-scene project. The split now carries the anchor onto scene 0 and the references onto every scene. Gated to H3: on LTX the anchor rides in the latent, and adding Gemma3 vision to scene 0 would change existing output. 2. A MiniMax H3 Image to Video node hands its first/last frame pins out on its CONDITIONING output, which had nowhere to go — the sampler's positive comes from Studio. New optional h3_keyframes input on the Chain Sampler salvages them: first frame onto the opening scene, last frame re-indexed onto the closing scene's own final frame (the source node's frame_count is its own, and a stale index would land mid-clip where PackedLayout refuses it). The editor auto-wires it for H3 projects. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
f40ae828a5 |
fix: the H3 v2a hint only fires when JoyAI audio memory is on; guard the inputs too
Two things, both from the same debugging session. v2a_grad_scale is JoyAI's coupling knob and is never installed without joyai_audio_memory (see the call site), so a left-over value with the feature off was being reported as an H3 limitation — blaming the model for a knob that would be equally inert on LTX. Both the sampler's pre-run notice and the Editor's inert-setting hint now gate on joyai_audio_memory. The NaN guard only saw the sampler's OUTPUT, which cannot tell "the model computed garbage" from "the model was handed garbage and propagated it" — and those have disjoint suspect lists. _sample_chunk now checks the incoming latent and the schedule first, and says which it was. The schedule check is there because it is hand-typed and our solvers divide by sigma (er = sigma_next / sigma): an interior zero or a repeated value is an instant Inf that poisons every value downstream. A trailing zero stays legal — it is only ever a step's target, never a divisor. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
9ea0be0713 |
feat: a non-finite latent stops the run where it happened
A chunk that samples to completion and returns NaN/Inf currently sails through
everything: the scene blend spreads it into the already-finished scenes, the VAE
decodes it, and the first thing that objects is ffmpeg's AAC encoder — "Input
contains (near) NaN/+-Inf", naming nothing that produced it, after the whole
montage has been paid for. The video half never objects at all: NaN through
astype(np.uint8) is undefined but silent, so it degrades with a warning.
_sample_chunk now checks the returned latent and raises with the run-phase label
("scene 2/3 · pass 1 of 2"), which stream, how many values and what dtype, plus
the causes worth testing in order — mismatched LoRA, a mis-inferred checkpoint
layout, a dtype the model cannot hold. It raises rather than warns because a
non-finite latent is unrecoverable and blending it forward destroys the scenes
that were still good. One isfinite() per chunk against a multi-second sample is
not a cost worth weighing; a latent the helper cannot read is ignored, never
fatal.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
digital-garbageandClaude Opus 5
|
0bf1a80820 |
feat: the KSampler pass can build its own schedule from steps + a scheduler
A SAMPLER object is only the step function, so picking KSampler used to leave the pass with no schedule at all unless a sigma list was typed by hand. steps + schedule now build one from the loaded model's model_sampling, exactly as ComfyUI's BasicScheduler does. The schedule dropdown is the switch between the two sources rather than a silent precedence rule: "use_user_sigmas" — the default, so nothing changes for an existing config — runs the Sigmas field as typed, and any real scheduler computes instead, leaving the typed schedule parked in the field to switch back to without retyping it. Studio logs the fact that the field is being ignored, and steps is only shown while it applies. Both frontends, same change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
e001853f7a |
fix: H3 token tags are reconciled with the conditioning Studio hands on
minimax_h3.py shipped tags_match / token_tags_length / extend_token_tags with a docstring saying a mismatch is a hard IndexError mid-forward — and nothing ever called them. Studio edits conditioning in many places, and H3 tokenizes reference images into the same span (ref_image_size "match" scales them to the output resolution), so the token count moves with the prompt AND with the resolution. That is why the crash looked intermittent and why editing either one made it come and go, reported only as a bare "list index out of range" at step 0. Studio now reconciles before returning: extend the tags when tokens were appended, drop them when tokens were removed (absent tags the DiT handles, wrong tags it does not), and print which happened — losing the tags changes how the text span is modulated, so it must not be silent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
ab85ef4ca9 |
test: kill the 22 chronic failures — they were stub collisions, not redundant tests
Every one of them was a real test of a live feature. 18 failed only in a full-suite run and passed alone, 4 asserted behaviour that was deliberately changed and never got updated. Ordering (18): each test module built its own partial `comfy` stub and registered it with `sys.modules.setdefault(...)`, so the first module to import won and every later module's richer stub was silently discarded — the scene chain sampler then died on `comfy` having no `.sample`, and the mask-safe attention install bailed at its first guard because `import comfy.ldm.modules.attention` resolves through the PARENT's attribute, which one module set and another did not. tests/_comfy_stubs.py now builds that package tree once, wiring every parent attribute, and test modules attach their fakes as attributes on the shared modules (per-test, via fixtures, where the fake is only theirs — the idiom test_detailing.py already used). Same file also stubs `server.PromptServer`, whose absence made every module that imports templates.py (i.e. anything importing conditioning) fail on `No module named 'server'`. Stale assertions (4): * the "every learning rating" loop excluded non-learning ratings by hardcoded LABEL, so the later-added `Wrong appearance|loved` and the two control sentinels were asserted to train phrase memory. Now filtered by profile key, with the reason per key. * three split_by_transitions tests still used "scene ten" / "cut to" as cut markers. The unified splitter cuts on a real transition trigger or the generic `scene <N>` label, on purpose — an incidental phrase must never split a prompt. Rewritten on the current contract, and they now also pin that the delimiter is dropped rather than encoded. Verified: 584 pass, and every file passes alone, in reverse order, and with movie_editor's suite standalone. Mutation-checked the rewritten split tests (disabling the scene-label pattern fails 4 of 5) so they are not passing vacuously. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
4aec53e01b |
feat: MiniMax-H3 against the merged release — real i2v anchors, model-owned frame geometry
H3 support was written against ComfyUI's PR; it has since merged (57500fc5, shipped in v0.30.0) and the weights are out. Everything the adapter assumed still holds — audio time axis, 17k+5 grid, batch-1 forward, the imported optimized_attention symbol, every payload key — so this is what the release itself changed, not a correction of the reading. LTX-2 / LTXAV is untouched throughout: a project picks a family and both are first-class. * i2v anchors now work on H3. LTX writes the image into the starting latent (LTXVImgToVideoInplace); H3 has no such path — its anchor is a condition row packed beside the text — so a scene's anchor image becomes a frame-0 keyframe pin instead, and the LTX node is no longer run against an H3 latent. Studio hands the source image's pixels to the sampler (it owns the CLIP, the sampler owns the VAE). Pins now MERGE, so a first-frame anchor and a last-frame guide coexist: H3's own first-and-last mode. * Frame geometry follows the model: frames per scene snap to the family's grid (LTX 8k+1, H3 17k+5) in both frontends, in the graph builder, and in the sampler, and H3 renders at its fixed 24 fps. Picking a family moves the project onto its geometry and says so. * The two checkpoints are named. fl2va (anchors) and ref2va (references) load identically and never reject each other's conditioning, so the run reports which mode it is in and the setup panel spells out the split. * Pipeline setup is no longer a placeholder: released, with the published filenames. * Reference images can be encoded at the reference pipeline's 2048px short edge. Fixes found on the way: * A video reference lost its soundtrack between Studio and the sampler — Studio had already announced it to the text encoder as <Audio 1>, so every later audio ordinal in the prompt pointed one reference earlier than written. * Easy Gen never knew it was on H3: its model config lives on the project, which the shared capability check did not read, so every H3 warning silently reported LTX. * An off-grid scene length failed with arithmetic instead of an explanation. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
990b29b8c9 |
feat: MiniMax-H3 in the Editor and Easy Gen — family-aware graph, ref2va UI, video references
The graph builder is now parameterized by model family instead of assuming LTXAV. H3 keeps the core's shape but three of its nodes do not apply: LTXVConditioning (stamps LTX's frame_rate onto conditioning, which H3 has no field for), LTXVConcatAVLatent (H3's own EmptyMiniMaxH3LatentAV emits both streams, so the latent feeds the sampler directly) and LTXVAudioVAEDecode (reads an output_sample_rate H3's audio VAE does not define — it would raise). Requirements, wirable ports, role targets, guided-wiring hints and the port picker all follow the family, so a loader can no longer be wired into a node the graph never emits. The family is a Models setting, asked and never guessed from a checkpoint filename: a wrong guess fails deep inside ComfyUI at generate time instead of in the panel that can fix it. Video references now work, rather than being refused. FunPack owns the pipeline and already requires ffmpeg for preview/export, so minimax_h3 decodes reference clips itself — once, at 24 fps — and derives both views from that single decode: the 2 fps timestamped frames Qwen sees, and the full-rate latent block the DiT packs. A clip's own soundtrack rides with it, presented as its <Audio j> label immediately before its <Video k>, which is what ties the two together. Capped at 15s because reference tokens ride every sampling step. Editor UI: a Reference media view (H3 only — there is no LTX equivalent to grey out) with per-type 1-based tags shown exactly as the prompt must spell them, reorder controls that renumber both sides at once, and a soundtrack picker on video entries. Warning chips beside Generate for the LTX-only sampler settings H3 switches off, so that is known before a generation is spent rather than after; Easy Gen gets the same warnings on its strip and the same family selector (it already loads models.js and pipeline_caps.js). Verified live in the Cutting Room: the family toggle flips requirements, ports and hints both ways and persists per project; tags renumber on reorder; the H3 chips replace the Best-FaceID chip on H3 and hand back to it on LTXAV. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
c77fd95f12 |
feat: MiniMax-H3 ref2va — native reference conditioning, replacing the LTX identity path
H3 packs reference blocks into the sequence itself (own position grid, never denoised, never rendered), which is the thing Best-FaceID approximated on LTX with overlap tokens plus a trained ArcFace projector. On H3 there is nothing to project and nothing to slice back off, so this is the model's own feature, not a port of that one. A reference has to reach the model twice and in the same order both times: as presentation (Qwen sees "<Picture 1>: <vision block>" so the prompt can name it) and as a packed latent block. Those encodes happen in different nodes, so the ordered spec travels between them: - Studio owns the CLIP: resolves the reference list, bakes the presentation via the tokenizer's minimax_ref_items, and records what it presented as funpack_h3_refs on the conditioning. Reference media and a first-frame anchor are separate conditioning modes, so source_image steps aside (loudly) when references are present. - Chain Sampler owns the VAE: encodes exactly that list, in that order, sized against the scene's canvas, and attaches minimax_refs. Applied on every scene branch so an identity holds across the whole chain, and cached per run so it is encoded once rather than once per scene. Order is the contract — drop or reorder an entry and "<Picture 2>" points at a different reference than the text encoder saw. Anything that cannot be encoded is dropped AND named, and an audio reference lost for want of an audio_vae says out loud that later <Audio j> labels have shifted. New optional audio_vae input on the Chain Sampler (audio references only; image references need just the main vae). Video references are refused with a reason rather than half-implemented — they need 2 fps frame sampling and a video loader FunPack does not own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
c731d8d451 |
feat: MiniMax-H3 foundation — adapter module, stream axes, frame grid, keyframe guides
Day-0 groundwork for Comfy-Org/ComfyUI#15224 (MiniMax H3). New minimax_h3.py is the one place that knows how H3 differs from LTXAV; everything else consults it. Four divergences that failed silently before: - Audio latent axis. LTXAV audio is [B,C,T,freq] (time on dim 2, same as video); H3 audio is [B,32,stereo,T]. Both 4-D, so only the model family can say which axis is time. The Chain Sampler now carries a per-stream time axis through continuation, blending, tail crops and JoyAI audio memory. - Frame grid. LTXAV 8k+1 -> k+1; H3 17k+5 -> 5k+2. H3's downscale_index_formula is the index map, not the count map, so the old uniform formula under-counted by ~20%. Frame counts now come from vae.downscale_ratio[0], which reproduces LTXAV's answer exactly. - Attention patches. LTX calls optimized_attention through its module; H3 binds it at import, so FunPack's temperature/capture patch was inert on H3. It now rebinds every module holding a reference, and position-indexed work (BachVid K/V, attn importance) sits out H3's pre-reshaped [1,heads,S,D] layout. - Batch size 1. H3's DiT refuses a batched forward and comfy batches cond+uncond for cfg != 1.0. install_batch_split runs them one at a time, keeping cfg live. Guides become keyframe pins: H3 packs condition rows into the sequence instead of appending a masked latent frame, so no tail is appended and none is cropped. The layout only accepts a first/last pin, and a mid-clip request is refused up front instead of raising inside PackedLayout. Studio: reference images route through Qwen3-VL's images= kwarg (the Gemma3 vision probe returns False for H3 and was dropping them); the attn2 K/V direction patch reports that H3 has no cross-attention to hook instead of installing a no-op. Features that depend on LTX transformer structure (bounded attention, Best-FaceID identity transfer, v2a_grad_scale, segmented detailing, second-pass latent ops) are now switched off explicitly on H3 with the reason printed, rather than each one discovering its own missing attribute mid-scene. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
0b8dbf91f0 |
fix: stop the sharpen round-trip from smearing through a box filter
_downscale_to used mode="area" — a 2x2 box for the exact-2x ratio both callers hit. A box loses on both axes at once: its passband droops well before Nyquist (detail comes back soft) and its stopband leaks (the detail the upsampler just invented folds back as aliasing). Soft AND speckled is precisely the reported "sharpen leaves smearing and artifacts". Antialiased bicubic instead. On a full-band chirp, RMS error against an ideal brick-wall decimation drops from 0.316 to 0.244 (~23% closer). Constant input still round-trips exactly, so the DC behaviour both callers rely on is unchanged. This is shared with segmented detailing's 'repair' paste, which downscales its refined crop through the same helper — that path gets the same change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
2011b18b95 |
fix: don't reject projects saved with the old context_window_schedule names
Renaming the combo's choices to core's spellings broke queueing for anyone who
had one of the old ones saved: ComfyUI validates combo VALUES at queue time,
before the node ever runs, so the alias map added with the rename never got a
chance. The three legacy names are back in the list (they resolve through the
alias map at runtime), and the Editor now displays a stored legacy value as the
name it maps to and writes that back, so the dropdown can't show one setting
while another runs.
And that rejection was unreadable. ComfyUI's top-level `error` for a validation
failure is a fixed blob ("Prompt outputs failed validation", details ""); the
part naming the node, the widget and the bad value is in `node_errors`, which
`error or node_errors` never reached because the blob is always truthy. Every
rejected prompt therefore reported the same nothing. Both are formatted now:
Prompt outputs failed validation; FunPackLTXAVSceneChainSampler: Value not in
list — context_window_schedule: 'static_standard' not in [...]
Also: the progress poll no longer imports anything. current_progress() reads
run_phase's label straight off `sys`. It fires every 700ms on ComfyUI's event
loop WHILE the worker thread samples, and that is not a path to do import work
on — the direct read is both cheaper and the correct one, since run_phase keeps
its state there anyway.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
digital-garbageandClaude Opus 5
|
7681c599a0 |
progress: say which scene and which pass is running; fix context_window_schedule
Three things, all about a run telling you what it is doing.
1. The second pass is now visible. ComfyUI's progress channel carries only
(value, max), so a chain sampling some scenes twice was one anonymous bar.
The sampler publishes a live label ("scene 2/3 · pass 2 of 2") through the
new run_phase module; /api/progress hands it to the editor, which appends it
to the readout, and pass 2 also announces itself on the console the moment it
starts rather than only in the after-the-fact run report.
run_phase keeps its state on `sys`, not in module globals: the writer reaches
it package-relative and the reader imports FunPack modules top-level by name,
so they can hold different module objects and the label would never arrive.
2. The progress bar total now counts both passes. It only ever counted the main
schedule, so with a second pass on it overflowed during scene 1 and jumped
backwards at scene 2, whose offset is a multiple of the per-scene stride.
3. context_windows was unusable. The knob offered "uniform_standard" etc.;
core's names are the other way round ("standard_uniform"), so every schedule
except "batched" raised ValueError out of the sampler and FAILED THE RENDER.
Core's names are the choices now, the old spellings are aliased so saved
projects keep working, and an unknown name is refused with a reason naming
what core accepts instead of crashing.
Same call had a second, quieter break: a keyword core's handler doesn't take
(latent_retain_index_list is absent on some builds, including this Mac's)
raised TypeError, which was reported as "ComfyUI core too old" — silently
switching off a feature core fully supports. Extras are dropped now, with a
note naming the one knob that stops working.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
digital-garbageandClaude Opus 5
|
39533dfde4 |
anchor_shift: remove the shift, keep only the head cut
The two-pass shift regrew the frames it dropped, and the invented ending consistently came out with worse or missing movement — not worth paying steps for. What is left is the part that works: generate the scene exactly as normal, anchor pinned at full strength the whole way, then cut the opening off the finished clip. Nothing is regrown, so the scene comes out that much shorter and the audio is cropped to match. Six knobs collapse into one INT, cut_opening_frames (0 = off), in the Editor under Chain Sampler ▸ Timing & Seed rather than Experimental. Gone with the shift: the mid-schedule split, the rewind re-noise (_renoise_to_sigma), the tail refill modes, the fresh-audio reset, and the schedule_context/continue_from_state plumbing that existed only so a split scene would not re-time the sampler inside each half. second_pass is untouched — it never used any of that machinery. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
63f225e231 |
second_pass: both schedules run in FULL — remove the cut I invented
User, after a live run: "First pass runs its own FULL schedule. Then 2nd pass begins. If I want to stop 1st pass earlier - I remove sigmas from schedule... 'Continue from' is ONLY relevant in our shifting technique which is a separate entity." Correct, and the cut was mine, not theirs. I carried anchor_shift's cut/restart semantics into a feature that has no use for them. Consequences they hit on the rental: - Steps. Pass 1 was cut at the second schedule's first sigma, so a 9-step main plus a 4-step second pass ran 3+4=7, not 13. They noticed the missing steps. - Distortion. With the cut, the between-pass operation ran on a MID-TRAJECTORY latent, so "sharpen" was sharpening the noise still in it. It only makes sense on a finished clip. Now: pass 1 runs the main schedule in full, exactly as written. The operation runs on that finished clip. Pass 2 runs second_pass_sigmas in full, exactly as written, entering the way any img2img refine does — comfy's ordinary CONST scaling on a clean latent, so continue_from_state stays off. Total steps are the two schedules added up. To make pass 1 shorter, shorten the main schedule. second_pass_sigma and second_pass_restart_sigma are deleted outright: with both schedules run as written there is nothing to cut and nothing to continue from. The second schedule's first sigma is now simply the strength dial, the way it is in every img2img refine. _second_pass_plan is replaced by _second_pass_schedule, which only validates. Also reported: "I have only sharpening enabled but log tells me upsampling is enabled as well." The load line printed "[FunPackSceneChain] segmented detailing upsampler: ..." regardless of which feature pulled it in, and sharpen legitimately loads that model — so a sharpen-only run announced a feature that was off. Now "latent upsampler loaded". Kept: the malformed-schedule refusals (ascending, or not ending at 0) — both are silent in the output rather than loud at runtime, which is how the distorted run read. tests/test_second_pass.py rewritten around the corrected shape, including that a 9-step main plus a 4-step second pass is 13. Suites at baseline: 22 pre-existing failures, 467 passing. Verified live that typing a schedule sets second_pass with no cut knob leaking. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
16045e086c |
second_pass: optional latent operations between the passes (none by default)
Per the user's reminder that these are OPTIONS, not defaults: second_pass_op is a combo defaulting to "none", and nothing runs unless it is picked. The expensive one is therefore not a tax on anything — it costs what it costs only when chosen. - "sharpen": one forward of Lightricks' trained 2x LatentUpsampler, resampled straight back to the original size. No video-model calls, so a fraction of a step; pass 2 then re-denoises the sharpened latent, which is what makes it stick. Adds detail consistent with what is there and CANNOT fix wrong structure — the same limit segmented detailing's sharpen mode documents. - "upscale_2x": same upsampler, result KEPT at 2x, so pass 2 runs at 4x the pixels and the scene decodes at double resolution. 3-5x the second half's cost. Reuses detailing.py's _run_upsampler / _downscale_to and its lazily-loaded upsampler (same model, same detail_upsampler knob, "auto" downloads the official file) rather than adding a second copy of any of it. Ordering that matters: the op runs BEFORE the rewind re-noise, because sharpening a latent that has just had noise added back sharpens the noise. Two failure modes handled rather than left implicit: - upscale_2x changes the latent resolution, so the chunk's i2v anchor and mask are the wrong size. Rescaling them would be inventing an anchor, so the pin is dropped and the scene report SAYS the pass runs unpinned and may drift from the reference image. - A missing or throwing upsampler degrades to "second pass, no operation" with the reason, instead of killing a whole montage over an optional sharpen. Found while writing the tests: an unrecognised op name was validated only AFTER the upsampler forward had already run, so a typo paid full price and then reported the wrong reason. Name is checked first now. Editor + Easy Gen get the combo under Engine > Experimental; verified live that it renders with none/sharpen/upscale_2x and defaults to none. Suite at baseline: 22 pre-existing failures, 306 passing. Not built: downscale-then-rescale (DeepShrink-shaped). It is not a between-passes op — pass 1 itself has to run at reduced resolution, which means downscaling the chunk, its mask and the anchor before pass 1, so it needs its own insertion point rather than this hook. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
b7fa3756f1 |
Chain Sampler: general "Enable second pass", with an optional separate schedule
anchor_shift built the two-pass machinery — stop the schedule, hand the state over,
re-enter exactly (_sample_chunk continue_from_state, _renoise_to_sigma, _schedule_view).
The user's point: that is generally useful, not anchor-specific. This exposes it on its
own, and keeps the two features independent as they asked: reaching the second_pass
branch means anchor_shift did not act on this scene, so they can never nest.
second_pass stop partway and finish in a second pass
second_pass_sigma where pass 1 stops (walked against the real schedule)
second_pass_restart_sigma where pass 2 re-enters; 0 = carry straight on
second_pass_sigmas OPTIONAL SIGMAS socket: pass 2 runs THAT schedule instead
At restart 0 with no second schedule the split is behaviour-neutral by construction —
pass 2 resumes from the exact handed-over state, nothing added, and 5+4 steps on a
9-step schedule is still 9. It earns its keep when the second half is made different:
re-entering higher (the model reworks what it committed to) or finishing on a different
schedule entirely, so the tail can have its own step count, spacing or shift.
Two things a split gets silently wrong if you let it, both handled:
- The i2v anchor. _sample_chunk drops noise_mask from what it returns, so pass 2 would
run UNPINNED and re-denoise the reference frame. anchor_shift wants exactly that;
a plain second pass must not. _restore_pinned_prefix puts the mask back AND restores
the pinned frames from the original chunk, since a rewind re-noise adds noise
everywhere including frames that are meant to be frozen.
- The sampler's own phases. Pass 2's schedule_context is the alternate schedule at
offset 0 when one is connected, or the main schedule at the slice offset when not —
so quality steps / final-correction window / AB2 ramp are measured against whichever
schedule pass 2 is actually running.
Every refusal names its fix (cut outside the schedule, entry below the cut, a
single-sigma SIGMAS input that is not a schedule) and lands in the scene report as
second_pass(SKIPPED: ...) — never a silent no-op.
Editor + Easy Gen get the three widgets under Engine > Experimental. The SIGMAS socket
is node-graph only for now: the Editor's fixed graph takes sigmas from ("studio", 5) and
has no second scheduler node, so exposing it there is a graph-topology change I have not
made unilaterally.
Latent/pixel-space pass-2 operations (sharpen, downscale->rescale, upscale) are next,
as opt-in options with nothing on by default.
New tests/test_second_pass.py: 10 tests over the split, the separate schedule, the
refusals and the pin. Suite at baseline: 22 pre-existing failures, 299 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
digital-garbageandClaude Opus 5
|
71e001afff |
anchor_shift: sigma 0 = no second pass, just cut the head off the finished clip
Reported by the user: setting the shift sigma to 1 or 0 with restart 0 got them nowhere, and what they wanted was "generate me a full video and don't make the 2nd pass, just cut off the beginning". That was a gap, not a crash. Both extremes fell through to the splitter, which correctly refused (a cut at the first sigma leaves pass 1 no step; a cut at the last leaves pass 2 none) and reported a skip — so anchor_shift silently did nothing and the skip reason did not name the thing they were actually after. anchor_shift_sigma=0 now means it: a cut at sigma 0 IS the end of the schedule, so pass 1 is the entire run and there is nothing left to denoise. The scene samples exactly as if anchor_shift were off — anchor pinned at full strength the whole way, which is the point — and the slide is applied to the FINISHED latent. Zero extra sampling: the cheapest form of this feature by a wide margin, and with tail=crop it is plainly "generate the video, then delete the opening". Widget min lowered 0.05 -> 0. Two things are refused rather than half-worked, each naming its fix: - tail=extend_last / empty, because with no second pass a refilled tail is never denoised — extend_last would freeze the last frame for anchor_shift_frames and empty would leave an empty latent on screen. crop and wrap are pure rearrangements of finished content, so both are allowed. - a non-zero restart sigma, which has nothing to restart. And anchor_shift_fresh_audio is forced off in this mode (reported, not silent): resetting the audio with nothing left to regenerate it would just produce a silent clip. The old "schedule never reaches ..." skip reason now points at sigma=0 as the way to run the whole schedule and only cut the head, so the next person hitting the extreme is told what to set. Suite at baseline: 22 pre-existing failures, 289 passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
9f7e8bb3c6 |
anchor_shift: add tail=crop — no regrown tail, the clip just ends earlier
User's idea, as an OPTION (default stays extend_last). The three existing tail modes all refill the freed end so the clip keeps its length, which means pass 2 has to invent an ending and blend a joint — the source of the "thin ending" and of most of the reason to reach for a rewind restart. 'crop' regrows nothing: the scene ends anchor_shift_frames earlier and every surviving frame was generated as part of one continuous shot. Checked the model side before assuming this was free, since a bad latent size would mismatch rather than degrade: - LTXAV's video patchifier is SymmetricPatchifier(1) -> temporal patch size 1, so there is no divisibility rule on video latent frames. - The streams reach the transformer as independent tensors (separate_audio_and_video_ latents just unbinds the list); no video:audio ratio is enforced there. - comfy re-packs per sample_custom call and re-derives latent_shapes, and the denoise masks off them (comfy/samplers.py outer_sample), so nothing is baked in from scene-build time and a shorter latent needs no conditioning rebuild. - VAE decode is (f-1)*time_scale+1 for any f >= 1. So no size is rejected. What IS real is proportion: audio timings come from the audio stream's own index and video's from its own RoPE, so cropping the two by different amounts of TIME desyncs the clip silently — no error, just drifting sound. Each extra stream's KEPT length is therefore derived from the new video length directly rather than by subtracting a separately-rounded drop, so rounding cannot leave them describing clips of different durations. fresh_audio zeroes after the crop, at the new length. Downstream needed nothing: _prepend_soft_continuation already clamps to the previous scene's real length and _derived_overlap clamps to the stream, so a shorter scene flows through the chain as-is. The scene report now states the cropped length in latent frames and in real frames, because a scene coming out shorter than the length that was set must never be something the user has to infer from the output. Tests: crop shortens instead of refilling and the surviving frames are the untouched originals; audio/video stay in proportion (including four rates that do not divide evenly); fresh_audio resets at the cropped length; the other three modes still keep the clip length. Suite at baseline: 22 pre-existing failures, 288 passing (+5). Verified live in the running app: the Engine ▸ Experimental dropdown offers extend_last/wrap/empty/crop, default unchanged, hint present. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
4c2a6b6b54 |
anchor_shift: the sampler must see the real schedule, not its own half of it
Raised by the user: "it should read the actual sampler schedule." They were right,
and this is a bigger deal than anchor_shift.
Both FunPack samplers derive their phase boundaries from len(sigmas) — the tensor
they are handed. anchor_shift runs a scene as two sample_custom calls over slices of
one schedule, so each half re-derived every phase INSIDE the slice:
_sample_const_rf_full (Hybrid Euler 2S, and the LTXAV path of it)
- quality phase: late_start = pct of the SLICE. On the user's 9-step schedule at
high_quality_pct 0.35 the quality phase starts at sigma 0.603; split at 0.482 it
starts at 0.812 in pass 1 and 0.482 in pass 2.
- motion-pulse anchors keyed by step index within the slice
- eta decay + velocity-bias ratio measured against the slice's first sigma
- the euler->Heun correction ramp restarted per call
sample_funpack_distilled_flow
- final-correction window: the last N steps OF THE SLICE, so pass 1 ran a "final"
Heun correction in the middle of the trajectory
- AB2 ramp w = i/(total-1) restarted from 0 in pass 2
- velocity-bias sigma ratio against the slice's first sigma
So turning anchor_shift on silently re-timed the sampler's own machinery in both
halves. Present since 015f274 and a far better explanation for a visible joint or a
thin ending than anything in the shift itself.
Fix: _schedule_view(sigmas, schedule_context) -> (real schedule, step offset). The
chain sampler passes {"sigmas": <the scene's schedule>, "offset": <slice start>} via
extra_options (the same chunk-scoped mechanism the alg_guide_* options already use,
always written even as None so it cannot leak into a later chunk). Both samplers take
the kwarg; every phase-relative quantity now reads the schedule and every step index
is step_abs = i + offset. No context = a normal single call = sigmas IS the schedule,
offset 0, byte-identical behaviour.
_anchor_shift_split_sigmas also returns pass 2's index into the schedule, and its
docstring now states the boundary rule the way the user framed it: walk the schedule,
the last pre-shift step is the one whose NEXT sigma has crossed the threshold. On
1.000 0.955 0.893 0.812 0.715 0.603 0.482 0.241 0.121 0.0 with shift 0.482 and restart
0.812 that gives pass 1 = 1.000..0.482 and pass 2 = 0.812..0.0, which is now asserted
directly as a test on that schedule.
Not changed, deliberately: sample_funpack_hybrid_euler_2s's own non-CONST loop (LTXAV
always takes the RF port, so no anchor_shift run reaches it) and segmented detailing's
stage-2 tail (a separate refine pass over a finished latent with its own schedule by
design, not a slice of one).
Suite at baseline: 22 pre-existing failures, 283 passing (+4).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
digital-garbageandClaude Opus 5
|
2b26c89ebb |
anchor_shift: restore the design — pass 1 always stops at the shift sigma
Reported by the user from a scene report: with a restart sigma set, the log said
anchor_shift_sigma was ignored. It was. That is my doing, and it should never have
shipped that way.
The original implementation did what was designed: cut pass 1 at anchor_shift_sigma,
shift, restart pass 2 at anchor_shift_restart_sigma, the two independent. It produced
noise artifacts because pass 2 re-entered through comfy's CONST scaling, which assumes
a CLEAN latent. There were two ways to fix that — write a re-noise that is valid on a
half-denoised latent, or make pass 1 always run to completion so the input is clean.
327f544 took the second. That silently reduced a two-knob design to one knob per mode,
was described in the commit as the modes being "genuinely different" rather than as a
parameter being removed, and left the user sweeping a value that could not do anything.
This does the work that should have been done then.
_anchor_shift_renoise: given x_a mid-trajectory and a target b >= a,
x_b = alpha*x_a + beta*n, alpha = (1-b)/(1-a), beta = sqrt(b^2 - (a*alpha)^2)
alpha reproduces x0's coefficient exactly, beta tops the noise term back up to b, so
the result lands on the manifold the model expects at b. Both old modes are special
cases of the one formula: b == a gives alpha=1, beta=0 (an exact resume) and a == 0
gives comfy's own fresh-noise scaling. beta has a real root only when b >= a, which is
why a restart BELOW the cut is now refused with a message saying why — noise can be
added back, never removed.
Streams wiped by anchor_shift_fresh_audio are not mid-trajectory, so they re-enter at
a=0 and get the full b*n a from-scratch run starts from, not a partial top-up of a
latent that no longer exists.
Continue mode (restart 0, the validated config) is unchanged and still bit-identical:
no re-noise, cost one full schedule. Rewind now costs the cut plus the restart tail
rather than a whole schedule plus the tail.
The scene-report NOTE about an ignored parameter is gone, because nothing is ignored.
Tooltips in samplers.py and engine_settings.js (Editor + Easy Gen share it) rewritten.
Tests: the old test asserting pass 1 runs to completion is replaced by one asserting it
stops at the cut; plus the re-noise identity at b==a, equivalence with comfy's scaling
at a==0, the coefficient/variance invariants on a synthetic latent with known x0 and
eps, reset-stream handling, non-mutation, and the below-the-cut refusal. Suite at
baseline: 22 pre-existing failures, 279 passing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
digital-garbageandClaude Opus 5
|
9ed53c8346 |
Fix anchor_shift under-denoising: pass 2 was re-noising a mid-trajectory latent
Reported from the first live run: the idea works (opening reads as t2v, character details preserved) but the output is full of noise artifacts, as if under-denoised, with the last steps seemingly skipped. That was a real bug, not the concept. comfy's CONST scaling is x = sigma*noise + (1-sigma)*latent_image, and it treats latent_image as a CLEAN image. Pass 2 was handed the PARTIALLY DENOISED latent as latent_image, so at a 0.725 cut it entered at 0.725*fresh_noise + 0.275*picture — pass 1's content attenuated to a quarter and buried under a full dose of new noise, with only the tail steps left to recover. Exactly the reported symptom. I cited segmented detailing's stage-2 tail as the precedent for this re-entry, but that recipe feeds a FINISHED latent, where re-noising is correct. The precedent does not transfer mid-trajectory. My error. Fix — the two modes are now actually different things: - CONTINUE (restart sigma 0, the default): pass 2 resumes from exactly the state pass 1 handed over. Passing that state as BOTH noise and latent_image collapses the CONST expression to an identity (s*x + (1-s)*x == x), so no noise is added at all. Cost still one full schedule. - REWIND (restart sigma > 0): pass 1 now runs the WHOLE schedule to a finished clip first, the shift is applied to that, and pass 2 re-noises it upward and denoises down. That is the img2img recipe, and it is valid precisely because the input is finished. Costs a full schedule plus the tail. Also fixed pass 2's progress offset, which restarted at pass 1's, so the bar appeared to stall short of the end. Two new tests reproduce the CONST arithmetic directly and assert the identity holds in continue mode and does not in rewind mode. Their comfy stubs are now snapshot/restored — the first version leaked into other modules and shifted the suite off baseline (3 unrelated tests changed result). Back to baseline: 22 pre-existing failures, 272 passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
83124a37ab |
anchor_shift: refuse on t2v scenes, and warn when the anchor survives the slide
Found while checking whether this works in Easy Gen. It does - the knobs render and persist there through the same shared engine_settings.js - but Easy Gen's default scene has no anchor image, which exposed a hole: nothing checked that the scene actually HAS a pinned i2v anchor. On a genuine t2v scene the slide would have dropped real generated frames and regrown them at the end, for no benefit at all. This is the "fake t2v" trick; it needs a real anchor to fake it with. Now refused with that reason in the scene report. Second guard: if anchor_shift_frames drops FEWER latent frames than the scene actually pins, part of the anchor survives and stays visible - the exact outcome the feature exists to prevent. The report now names the pinned count and the value to raise the knob to. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
e158d02e46 |
Chain Sampler: anchor_shift - keep the real i2v anchor, then delete it
User's idea, and the framing is the point: this is "faking t2v" when it is
really i2v. Every existing way of stopping the anchor from dominating the
opening WEAKENS it first - ALG blurs it and loses character detail, and if you
under-blur you get both failure modes at once; Best-FaceID overlap tokens
approximate it and lose some detail too. This weakens nothing. The anchor is
pinned at full strength, untouched, for as long as it takes to transfer
identity/style/composition - and is then removed from the clip entirely.
Two passes:
pass 1 sigma[0] .. anchor_shift_sigma, anchor pinned as usual
shift drop the first N latent frames, slide everything left, refill the
freed tail, drop the pin (clip LENGTH is unchanged - nothing extra
is ever generated; the dropped head is regrown at the end)
pass 2 restart sigma .. 0, unpinned
Two passes rather than a roll between steps for a specific reason: pass 2
re-enters through comfy's CONST noise scaling (x = s*noise + (1-s)*latent),
which puts the shifted latent back ON the rectified-flow manifold. A raw
in-flight roll skips that, and project_sigmas records that this model cannot
recover from off-manifold latent states - that is what killed churn. Same
on-manifold re-entry that segmented detailing's stage-2 tail already uses.
Every choice is a knob, because WHERE to cut is the entire experiment and I
have no measurement for it: shift sigma (default 0.909, the first step where
structure actually forms - above it nothing has formed and the anchor would be
discarded before doing any work), frames dropped, pass-2 restart sigma
(0 = continue, higher = rewind for more freedom at the joint), what the
regrown tail starts from, and whether audio is rewritten in pass 2 (on by
default: pass 1's audio was scored against frames that have since moved).
Refused, loudly and with the reason in the scene report, on continuation
scenes (the head is carried frames, not an anchor), on scenes carrying guide
frames (the slide would overwrite them) and on scenes with JoyAI audio memory
(resetting audio would destroy the injected tail).
One real bug caught by the new tests while writing them: SIGMAS are float32,
so a threshold typed as 0.725 is a float64 sitting just below the schedule's
float32 0.725, and a bare `v <= s` cut one step LATER than the number entered.
Now matched against core's own isclose tolerance.
11 new tests; suite at baseline (22 pre-existing failures, no new). Editor
checked live: knobs render, gate on the toggle, persist to sampler_inputs.
UNVALIDATED LIVE - the shift sigma is yours to find.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
digital-garbageandClaude Opus 5
|
e01af72445 |
Context windows on the Chain Sampler + fix upsampler load on ComfyUI v0.29.0
Two things, both driven by upstream ComfyUI moving 7 weeks ahead of the local checkout (2cdaaf4a 2026-06-06 -> e651b7be 2026-07-28, 263 commits). 1. Segmented detailing crashed on ComfyUI v0.29.0. Upstream f8a3fd9d moved the LTX latent upsampler onto DynamicVram and made `operations` a required argument of LatentUpsampler.from_config; detailing.py still called it with one arg, so the pending live probe would have died before detecting anything. Now passes comfy.ops.disable_weight_init (what core's own loader does) with a TypeError fallback, since rentals run whatever they pulled. 2. New Chain Sampler option: context_windows. Upstream cd77c551 (CORE-3) added context-window sampling with real LTX2/LTXAV support - it unpacks the packed AV latent, maps each video window onto its proportional audio window, and re-slices denoise masks, keyframe_idxs and guide_attention_entries per window, so anchors, mid-scene guides and JoyAI memory survive inside a window. Nothing is ported: we build core's handler and hang it on the scene's model_options exactly as the stock LTXVContextWindows node does. Core picks it up in comfy.samplers.calc_cond_batch, which every FunPack sampler already routes through, so no sampler-loop change was needed and no graph change either - it rides the sampler_inputs bridge into both Editor and Easy Gen. Refuses (loudly, once per run, with a scene-report note) on cores without BaseModel.map_context_window_to_modalities: the handler class predates LTXAV support and would window the packed tensor blindly, which is the same packed-vs-unpacked trap that killed DynamicConditioning. Install and removal are per-scene under the existing wrapper-hygiene finally, so an interrupt cannot leak the handler or its two patcher wrappers into later runs. Knobs (all exposed, none hardcoded): length, overlap, schedule, blend, FreeNoise, and pin-anchor-in-every-window. Off by default. Engages only on scenes longer than the window, and says so in the scene report when it does not, so a scene that sampled whole never reads as a broken feature. Verified: 8 new unit tests for the install/remove/capability-gate contract; full suite at baseline (22 pre-existing cross-module stub failures, no new). Editor checked live - knobs render in Engine > Chain Sampler > Experimental, dependents gate on the toggle, and values persist into sampler_inputs. UNVALIDATED LIVE on a rental. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 5
|
3948f64033 |
Identity transfer: support ComfyUI's new RoPE matrix layout
ComfyUI 7c59a07 ("Use comfy kitchen rope functions in ltx models", 2026-07-24)
changed the LTX positional-embedding payload from (cos, sin, split_mode) to
(rotation_matrix, split_mode), where the matrix is (B, T, heads, D, 2, 2)
holding [[cos, -sin], [sin, cos]].
rotate_overlap_freqs read pe[1] as the sin tensor — now a bool — so it raised,
and prepare_pe's bare `except: return pe` swallowed it. The Best-FaceID
reference tokens were still appended on the target's frame-0 coordinate grid
but WITHOUT the source-phase tag that marks them as a reference, so the model
read them as literal frame 0 and opened the clip on the reference image.
Anyone on ComfyUI >= 2026-07-24 saw i2v-like behaviour they couldn't bypass.
rotate_overlap_freqs now detects the layout at runtime (pe[1] is a tensor =>
legacy cos/sin; otherwise the matrix form, tokens on axis 1, frequencies on
-3) and composes the same R(angle + phase) either way. An unrecognised payload
raises instead of silently skipping the tag.
The four LTXBaseModel wrappers no longer fail silently: each reports once per
install that identity conditioning is not being applied and that a ComfyUI
update is the likely cause. A logged warning would have made this obvious.
Tests: both layouts rotate only the ref-token tail, produce identical angles,
reject unknown payloads, and the matrix form survives end-to-end through
_install_identity_overlap.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
||
|
digital-garbageandClaude Fable 5
|
726d88a547 |
Add detail_mode: sharpen (near-free, no diffusion) vs repair (current)
Recovers the original pitch: 'cheap, on-the-fly sharpen' was quietly upgraded into a full re-denoise pass during design without ever offering the free version as an option. The trained latent upsampler net was already an intermediate step in the pipeline (crop_up) — sharpen mode just stops there and pastes it back, with ZERO video-model forward passes (only the small upsampler CNN runs). repair mode (default, unchanged, byte-identical behavior) keeps the existing re-noise + 3-step tail for cases that need actual structural correction, since only the generative model itself can decide a region should look structurally different — a super-res net can only sharpen what's already there. Also extends the Editor's knobVisible with a array (AND'd conditions) — detail_denoise needed to gate on BOTH segmented_detailing AND detail_mode=='repair', which the existing single dependsOn/dependsValue couldn't express without silently dropping one of the two checks. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Fable 5
|
1644dde829 |
Expose the crop re-noise strength: fixed 0.85 was sharpening, not repairing
Live report: a hand was detected, masked, and upsampled, but the pasted result looked like a plain upscale, not an actually-corrected region. Root cause is mechanical, not a bug: 0.85 is Lightricks' own official re-entry sigma, but it was designed for their use case (sharpen an already-correct frame at higher resolution), not ours (repair a likely- malformed crop). At that re-entry point the tail mostly smooths the upsampled interpolation back down rather than giving the model enough noise/freedom to genuinely reconstruct structure. detail_denoise (0.3-0.99, default 0.85 = unchanged prior behavior) now controls the re-entry point; the tail's shape (ratio between its 3 steps) scales with it via the new stage2_sigmas() instead of a fixed tuple, so raising it doesn't just enlarge the first jump. Higher = more genuine repair freedom at some risk of drifting from the surrounding frame. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Fable 5
|
0372ee8287 |
Make the area cap a user knob, not a silent content judgment
The 35% area cap was refusing a user-requested 'hand' region (47% of frame) with no way to override it — the code was unilaterally deciding the request 'wasn't a detail pass' when the user never asked for that judgment. Fair complaint: cost guardrails are not the same as content decisions. detail_max_area is now threaded end-to-end (detailing.area_cap param -> sampler widget -> Editor knob in Chain Sampler > Experimental, 0.05-1.0, default 0.35 unchanged). Raising it lets a region of any size through; 1.0 = no cap. The refusal note now points at the actual setting name instead of citing a fixed constant. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Fable 5
|
30b27b734a |
Instrument CLIPSeg misses + expose detail_threshold knob
Live run hit 'no region matched' with zero diagnostics: CLIPSeg's raw
sigmoid score for a real, correctly-named region is often well under a
naive 0.5, so a miss against the hardcoded 0.35 threshold was
indistinguishable from 'nothing there.' Two fixes:
- find_tube now takes an optional diag dict and always records max_heat
(plus area_cap vs below_threshold reason on a miss).
- detail_refine_scene always returns a descriptive note once detection
actually ran ('no match: max CLIPSeg score 0.XX < threshold 0.35 — try
a lower detail_threshold...'), replacing the generic
'no region matched' the sampler used to fabricate.
- detail_threshold is now a real widget/sampler_input (was a hardcoded
constant with no UI — a hidden-setting violation once real runs started
needing it), in Engine > Chain Sampler > Experimental next to the other
detailing knobs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
digital-garbageandClaude Fable 5
|
5727c39189 |
Fix CLIPSeg multi-prompt crash: missing padding/truncation on tokenizer call
Live rental run hit it immediately on 'detail_targets' with more than one
target: CLIPSegProcessor tokenizes each prompt to a different length, and
without padding=True torch can't stack them into a batch ('excessive
nesting... type list where int is expected'). Reproduced against the real
CIDAS/clipseg-rd64-refined model before and after the fix. Single-target
runs were unaffected (batch of 1 never needed padding), which is why the
earlier probe steps didn't catch it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
digital-garbageandClaude Fable 5
|
c864f62480 |
Segmented detailing: auto-resolve/download upsampler + loud skips
The first live attempt silently no-oped: detail_upsampler defaulted to 'None' (nothing installed on the rental) and the guard skipped without a word. Two fixes: - detail_upsampler default is now 'auto': prefer the official installed file, else the highest-versioned spatial upscaler (version-aware compare — plain sort ranked ltx2_3_… over ltx-2.3-…-1.1), else hf_hub_download of Lightricks/LTX-2.3 ltx-2.3-spatial-upscaler-x2-1.1.safetensors (~1 GB, once) with an actionable error if the repo is gated. Legacy 'None' = auto. - With the toggle on, every outcome is visible: upsampler choice and skips print to console AND land in the scene report (segmented_detail(SKIPPED: …) / (no region matched)); resolve/load failure disables the pass for the run instead of retrying the download per scene. A failed pass still never costs the scene. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Fable 5
|
a6e4a934f1 |
Segmented detailing: CLIPSeg-targeted latent crop-refine on Scene Chain
ADetailer-for-video probe core. After a scene denoises, CLIPSeg locates user-named regions (hands, feet, ...) on decoded keyframes; the region is cropped as a spatiotemporal tube, 2x-upsampled through Lightricks' trained LatentUpsampler (comfy core, no port needed), re-denoised over the official stage-2 tail (0.85/0.725/0.4219/0), downscaled back and pasted through the feathered CLIPSeg silhouette. Final resolution unchanged; audio protected by construction (refined audio discarded); tubes >35% frame refused; empty targets/no-detection = bit-identical no-op. UNVALIDATED LIVE. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Fable 5
|
80329c9f74 |
Loop roll v2: content-only roll — guides and audio memory stay pinned
Guide-carrying scenes no longer disable the loop temporal style. Appended guide frames always live at the tail of the video stream's T range, and keyframe_idxs is per-token — so the wrapper derives the tail frame count (tokens / (H*W), same math as comfy's get_keyframe_idxs) and rolls only the content region [0, T - tail); the tail, keyframe_idxs, and attention entries that reference it stay canonical. The audio stream's appended memory (JoyAI, rides a trailing mask=0 run) is detected the same way from audio_denoise_mask. Guides keep informing every rolled position — their influence smears across the cycle, which is what identity/style references should do in a loop — and are never dragged into the rendered content. Unparseable guide contexts (malformed keyframe_idxs, entries without idxs) still run canonical: never risk a corrupting roll. Content-region size gates the minimum-frames check now, so a short clip whose frames are mostly guide tail stays canonical too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Fable 5
|
0214144a9d |
Loop temporal style: Mobius latent roll — seamless looping on LTX2.3
The 'loop' temporal style (in the style list since forever, but inert) now does what WanVideoWrapper's Loop Args does for WAN: on eligible denoise steps the packed latent is cyclically rolled along its temporal dim before the forward and the prediction unrolled after, so the model repeatedly sees the video's seam (last|first frame) as an interior frame pair at step-varying positions (van der Corput sequence, reset per scene) and smooths it into a natural loop. Video and audio streams roll by their own frame counts and stay time-aligned; denoise masks roll in step, so i2v anchor per-token timesteps stay consistent. Near-zero overhead: two clones + torch.roll per eligible step, no extra forward passes. Rolls only below the near-noise plateau (nothing to smooth up there, and the plateau step-cache stays valid). Calls carrying guides are left canonical — keyframe_idxs pins appended guide frames to absolute positions inside the same T range, so rolling them would corrupt; on guide-carrying scenes the roll is inert and the run mechanism log says so. Wiring: concrete style via the global build_enhancements path (innermost relative to per-scene wrappers, as required — prediction-modifying wrappers see canonical orientation only); auto director's funpack_temporal_loop via a tagged per-scene wrapper on the Scene Chain sampler, installed below the guidance stack. Editor already exposes 'loop' in the temporal style dropdown. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
digital-garbageandClaude Opus 4.8
|
432e5c54bb |
Plateau step-cache: reuse near-noise plateau forward on Scene Chain sampler
MixCache/Chorus-family step-caching adapted to LTX2.3's distilled schedule. On the near-pure-noise plateau (sigma >= threshold, default 0.975) the transformer output barely changes, so compute the base-model forward once and reuse it for the remaining plateau steps, skipping ~3-4 of 8 full 48-block forwards. Below threshold every step computes for real and the cache drops, so structure formation is never approximated. Deterministic given seed -> safe in Batch Training, no rating-diversity impact. Installed innermost as a per-scene model_function_wrapper (existing tag/strip leak hygiene) so guidance wrappers still post-process each step's cached-or-fresh prediction. Cache keyed by (input.shape, cond_or_uncond). Editor: plateau_cache + plateau_cache_threshold in Engine -> Experimental, auto-wired via the sampler_inputs bridge (no backend/builder change). Off by default, UNVALIDATED LIVE. (Chorus cross-variant reuse rejected -- would blur batch-rating diversity; FreeSpec dropped -- wrong topology.) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
digital-garbage
|
449532358f |
Fix stale FakeModel/FakeVAE mocks and expectations in test_scene_chain_sampler.py
model=object() lacked model_options (needed by the per-scene wrapper snapshot/restore added for the scene-wrapper-leak fix), and FakeVAE lacked decode() (needed since the sampler always decodes IMAGES when the caller doesn't declare that output as disconnected) — together these silently killed every test in the file that calls sample(). Added FakeModel and FakeVAE.decode, then fixed the resulting return-tuple arity (sample() now returns 6 values, these tests unpacked 4) and three genuinely stale assertions: the per-scene report line gained a timing segment, carry_i2v_guides now prepends its guide frame instead of appending it, and _append_i2v_guides no longer routes through keyframe_idxs/guide_attention_entries. Verified in isolation: all 14 tests in this file now pass. Running the full tests/ suite still shows the same file failing — that's a separate, pre-existing cross-file sys.modules stubbing collision (confirmed via git stash on a clean dev checkout) affecting this file plus test_mask_safe_attention.py and test_v2_category_alignment.py; not fixed here since it isn't a one-line root cause. |