A video model only follows a guide inside the clip length it was trained on, so long
sources are split into shots that fit and run once per shot.
- BFS Shot Planner: interactive Vue timeline (thumbnails, cut-detector curve, draggable
boundaries, split/merge, per-shot reference gallery with upload, per-shot prompt).
Splits at camera cuts via PySceneDetect (adaptive/content, built-in fallback), fixed
length, or by hand; lengths snap to the model grid (H3 17n+5, LTX/Wan 8n+1, 4n+1).
Outputs the shots as a ComfyUI list so everything downstream runs once per shot.
- BFS Shot Unpack: guide frames, references, prompt, length, audio of one shot.
- BFS Shot H3 Conditioning: native MiniMax H3 conditioning per shot (aligned guide,
native reference video, or both).
- BFS Shot Join: concatenates in order, trims each shot to its true length, cross-fades
soft joins with the real overlap frames, returns the soundtrack.
- Queue loop mode: one shot per run stored on disk, auto re-queue, nodes after the join
blocked until the last shot.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Getting a recipe from the interactive node into the save node meant copying the rules JSON
between two text widgets, which is a bad way to move state and easy to get wrong.
Two paths now. BFS LoRA Surgery gained a `rules` output that can be wired straight into the
save node's `rules` input, so the file is produced as part of the run. And the panel has a
"save as file" button with an optional name, which posts to a new /bfs/lora/save route and
reports the result inline, for when you are still exploring and do not want a second node in
the graph.
Both go through one write_surgery function, so the two paths cannot drift apart. That
function is the old node body lifted out, not a copy.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The parser took the last numeric component of a key as the block index. That is wrong
whenever a name carries more than one number: transformer_blocks.12.attn1.to_q would be read
as block 1 of a family ending in attn, and a nested layout such as
down_blocks.0.attentions.1.transformer_blocks.0.ff.net.0.proj collapsed into a family called
ff.net. The block axis is now chosen per path template as the numeric slot that varies most
across the whole LoRA, which is information a single key does not have. Numbers outside that
slot stay in the module name, so attn1 and attn2 stay distinct rather than folding together.
The grouped tree is a summary, and a summary can hide things, so the panel now also has an
"every layer" view: the full module list with a filter box, each row showing its norm and
rank, and a + that writes a rule matching exactly that path. The structure endpoint returns
that list alongside the grouped one.
Checked against the body-swap recipe: same 136 of 192 modules kept, 56 dropped, 96 scaled as
before the change, and attn.to_out.0 is still treated as attention rather than as block 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two fixes.
The panel lost its configuration when a workflow was reopened. The Vue app mounts in
nodeCreated, which runs before ComfyUI restores widget values in configure, so it read an
empty widget and the first edit would then overwrite the stored recipe. The node now wraps
onConfigure and tells the panel to re-read once values have landed, and the rules widget is
explicitly marked serializable with a serializeValue, so the configuration travels with a
shared workflow rather than living only in the session.
The widget now stores {rules, ui} instead of a bare array, so the selected family, type,
block range and scale come back too, not just the rules. A bare array is still accepted, for
workflows saved by the previous version and because the Python side takes either shape;
verified both produce identical results.
The inline guide named gate_up as if it were universal. It now says up front that names
depend on the base model, gives the three architectures verified so far with their actual
family and module names and block counts, and frames the finding as the MLP input projection
whatever the model calls it, explicitly a starting point rather than a rule.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Filenames could exceed the OS limit: the slug was capped at 110 characters but the final
name is stem + slug, so a long source name pushed it past NAME_MAX (255 bytes on Linux, and
Windows caps the whole path near 260 without long paths). Names are now fitted to 160 bytes,
trimming both halves and appending a short hash of the full recipe when needed, so two
different recipes still never produce the same name. A path over 240 characters logs a
warning, since the folder above us is not ours to control.
Metadata now records what was applied rather than just that something was. The original
metadata is preserved untouched, and next to it:
bfs_surgery_summary one readable line, for viewers that know nothing about this tool
bfs_surgery tool, UTC timestamp, source filename and its sha256, the readable
recipe, the exact rules, and module counts (total/kept/dropped/scaled)
ss_output_name updated to the new filename
The sha256 matters more than the filename: it ties the result to the checkpoint it came from
even after either one is renamed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
BFS LoRA Surgery (save) applies the same rules and writes a .safetensors into models/loras,
for when a recipe is settled and you want it usable outside the workflow that produced it.
With no filename given, the name is derived from the rules themselves:
mylora__attn-x1.15__gate_up-b8_15-off__all-b16_23-off.safetensors. Names get sanitized, so a
type like *mlp* or a regex match still yields a valid file. The recipe also goes into the
safetensors metadata under bfs_surgery, next to the original metadata, so a shared file can
explain itself later.
Verified against the study output: saving the body-swap recipe produces the same 136 of 192
modules, and the effective dW matches the file produced by the standalone script to 3e-3
relative. The factors differ because the node scales only the up/B side while the script
split the scale across both, but the product is what the model sees.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Shift-click was the wrong gesture inside a node: on the ComfyUI canvas it starts a text
selection, so picking a range fought the editor. Ranges are now painted by dragging across
the ruler, with pointerdown calling preventDefault and stopPropagation so the drag does not
reach the canvas and move the node.
Click-to-toggle still works, since a click is a one-cell drag, and the gesture adds or
removes depending on the cell you start from. Also added a text field that both shows and
accepts the selection (8-15, 0,4,7, 8-15,24-31) and shortcuts for all / first half / last
half / invert / none, which cover most of what the ruler is used for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The panel opened with the layer tree, which pushed the controls off screen on a LoRA with
32 blocks: you had to scroll past the reference material to do anything. Order is now
header, guide, Selection, Rules, Layers.
The guide is a collapsed block that explains what the node is for, how to read the ||dW||
bars, where to look first (the MLP input projection), that later rules override earlier
ones, and the two things worth knowing before trusting a result: a variant can win every
metric and still have destroyed what you trained, and pruning after training is not the
same as training with those layers excluded.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Bumps the version so the registry workflow publishes, and keeps the vendored Vue out of
ComfyUI's extension scan: server.py globs **/*.js recursively under WEB_DIRECTORY and
imports every hit as an extension, which would have pulled in the Vue bundle as if it
were one. Renaming it to .mjs sidesteps that — mimetypes still resolves it to
text/javascript, so the module import is unaffected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A defect a LoRA learned — smoothed skin, identity drifting at distance, a pose it
refuses to copy — usually lives in one module family and a range of blocks, not in the
whole adapter. This node makes that testable without retraining: pick a family, pick
blocks, set a scale, queue. Nothing is written to disk, so iterating is a slider away.
Model-agnostic by construction. The structure is discovered from key names rather than
hardcoded, covering the diffusers (lora_A/lora_B) and kohya (lora_down/lora_up + alpha)
conventions. Verified against three architectures with different naming: Qwen-Image-2.1
(transformer_blocks, 32), Krea 2 (blocks 28 + txtfusion) and MiniMax-H3 (blocks 50 +
token_refiner).
Scaling multiplies only the up/B factor, since (sB)A = s(BA) — exact under either alpha
convention, and negative scales stay usable.
The panel is a Vue app in a DOM widget, with Vue vendored so it works offline. It lists
each family's module types sorted by ||dW||, which reads as a map of where training
invested. Selections convert to a regex on demand for cases the tree cannot express.
Rules are ordered and later ones win, so the UI says so and lets you reorder.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
1.41.0 fixed the wrong half of this. The deformation came from the crop and the
latent having different ASPECTS -- a mask around a face gives a portrait box,
sampled in a landscape latent it is stretched on the way in and squeezed back on
the way out -- and rebuilding the latent at the crop's size did remove the
mismatch, but by throwing away the resolution that cropping exists to buy. A
face 200 px tall in the source was then sampled at 200 px, which is the problem
the crop was there to solve.
The planner already takes an aspect_ratio and we were passing 0.0, letting the
box follow the subject. It now gets the connected latent's aspect, and
_static_box grows its short side to match when the planner is not in play. The
box still contains the subject and stays divisible by the grid.
So the latent's size is yours again -- it is the sampling resolution, choose it
for the quality you want -- and its shape no longer has to be guessed.
68 tests; the new ones pin that a landscape latent gives a landscape box, that
the box still contains the subject, and that a free aspect still follows it.
The crop's size is derived from the mask, so an EmptyLTXVLatentVideo built
upstream cannot know it -- and a latent that disagreed was not an error
anywhere. The model sampled at the latent's shape and the paste-back squeezed
the result into the box: a face crop is portrait, sampled into a landscape
latent it came back visibly deformed, and under crop_mode zoomed the deformation
changed frame to frame. The only cure was to run once, read the size off the
debug string, rebuild the latent and run again, after every change to the mask.
The sampler now rebuilds the latent's video stream at the crop's shape when they
disagree, keeps the audio stream as it is, and reports it in debug. Nothing is
lost: at denoise 1.0 the initial latent contributes nothing, and the inpainting
path overwrites the video stream with the encoded guide regardless.
BFS Crop Size stays useful for building the latent at the right size up front
(skipping the rebuild) and for taking the per-frame boxes straight to BFS Paste
Back, but it is no longer the price of a correct result.
68 tests.
The sampler derives its box from the mask, so the size to build the empty latent
at was only knowable after a run: sample a whole clip, read one number off the
debug string, rebuild the latent, sample again -- and repeat after every change
to the mask or the crop widgets.
Getting it wrong does not look like a sizing problem. A mask around a face in a
1920x1080 frame gives a portrait box, 512x672 say; sampled into a 768x512
landscape latent the crop is stretched horizontally, and the paste-back then
squeezes the result back into the portrait box. The face comes out deformed, and
under crop_mode: zoomed the deformation changes frame to frame.
The new node runs the same planner on the same mask and returns width, height,
the per-frame boxes and a note, so the latent can be built up front. The planner
is deterministic, so identical crop widgets give the sampler the identical box.
Also factored the per-frame box normalisation the sampler did inline into
_boxes_per_frame, which both nodes now share.
63 tests.
Chunked sampling built its chunks with an overlap for continuity and then
concatenated them whole, so the overlap was emitted twice. 97 frames at tile 72
overlap 16 gave 15 latent frames where the clip has 13: the output ran longer
than the input, and from the first seam on, frame i was no longer moment i.
Pasting back made it obvious rather than subtle -- crop_bboxes has one box per
source frame, so every frame after the seam went into a box belonging to a
different moment, and with crop_mode: zoomed into a box of a different size too.
That is the "crooked and out of sync" this was reported as.
Each chunk after the first now contributes only its tail, the part the previous
chunk did not already cover, so the concatenation is exactly the clip's latent
length. Single-chunk runs are untouched.
`import comfy.nested_tensor` inside execute() binds `comfy` as a LOCAL for the
entire function, so every other comfy.* use in it raises UnboundLocalError -- at
compile time Python decides the name is local, and the AV branch that would have
bound it does not run on a video-only latent.
It shipped in 1.37.0 with the chunk-offload line:
got = got.to(comfy.model_management.intermediate_device())
which only runs when a clip is long enough to be sampled in more than one chunk.
That is why it survived the tests and the short runs and failed on the first
real long video:
UnboundLocalError: cannot access local variable 'comfy'
Imported as `from comfy import nested_tensor as _nested` now, which binds
`_nested` and leaves the module-level `comfy` visible. A test asserts neither
node's execute() has `comfy` in co_varnames, which catches the whole class of it.
_paste_back gave up when there was no crop box and returned the sampler's frame
whole. Nothing outside the mask *changed*, but the entire frame had been through
the VAE's encode/decode, so every pixel the edit never touched came back softer.
For a pointwise edit, or anything sharpness-adjacent, that is the opposite of
what the mask was for.
With no crop and a mask, the full frame is now composited into the ORIGINAL
frames through that mask, the way the crop path already did: outside the mask
the output is the source verbatim, never a round trip of it. The mask's own blur
supplies the soft edge -- there is no box border to feather here.
This makes crop_mode a resolution decision again. Before it, preserving the
untouched pixels required turning the crop on even when the region already had
all the resolution it needed.
52 tests; the new ones pin that outside is the source byte for byte, inside is
the result, no mask still composites nothing, and a result of another size is
brought back to the frame first.
Three things.
`identity_image` moves from required to optional. The node was never head-swap
specific below the surface -- the reference slot is simply not packed when it is
empty, and everything else (crop, mask, inpainting, chunking, paste-back) is
indifferent to which LoRA is running. An instruction-edit, restyle or sharpening
IC-LoRA now runs on the guide alone. IMAGE is a link, not a widget, so this
costs no widget position.
Renamed to "BFS Sampler (crop · mask · loop)" and "BFS Paste Back". The
registration keys are deliberately unchanged: saved workflows reference those,
and only the display name is free to move.
And the fix for the reason this was noticed: 1.37.0 inserted auto_config and
identity_headroom BEFORE crop_mode. ComfyUI stores widgets_values positionally,
so every later value in every saved workflow shifted two slots -- crop_mode's
string landed in auto_config, crop_scale's float in identity_headroom, and the
graph failed validation with "input out of range" on crop_scale and
crop_divisible_by. Both widgets are appended at the end now, with a comment
saying why, and a test pins the historical order so the next addition cannot
repeat it. ltx_multiple_controls.py already carried that rule in a comment;
this file did not follow it.
48 tests.
Two compounding mistakes in 1.37.0's derivation, reported as a hole much larger
than the head with the new head generated floating inside it:
- the upward slack was scaled by the mask's HEIGHT (hh * delta / tilt). Height
swings with how much neck the mask took, and a portrait-shaped identity image
(tilt < 1) multiplied it further: a 300x400 head with a 0.6 reference asked
for 136 px above. Every slack is now a fraction of the head's WIDTH, the
stable dimension, capped at 25% sideways and 30% up.
- latent_mask_dilate was derived from mask_grow and forced to at least one
cell, which added a whole 32 px on EVERY side -- including down, which the
pixel grow deliberately holds at the base -- on top of a grow that already
carried the slack. It is 0 now: _mask_to_latent reduces with MAX, so any cell
the mask touches is fully editable already. The same head asked for 4 cells,
128 px, before this.
Also undo the guider injections in reverse: with several chunks each injection
captured the state the previous one left, so undoing forwards restored chunk
1's specs onto the guider instead of clearing them.
Tests pin both: growth stays under 45% of the head's width across every
reference aspect and headroom, the latent dilation stays 0, and the crop box
always holds the grow plus the feather.
auto_config measures the subject in subject_mask -- width, height, travel, size
change, per-frame step -- and derives crop mode, crop scale, mask grow/blur,
paste feather and latent dilation from it. Every amount in this node is in
pixels, and a pixel only means something relative to how big the head is in
frame: the same 8 px is generous on a distant head and invisible on a close
one, and that mismatch is what leaves a seam.
The mask is the OLD head, so growing it by a fixed fraction only moves the
clip when the new head is wider or has more hair. The reference's proportions
are measured (identity_image's aspect against the mask's) and decide WHERE the
slack goes; its absolute size cannot be measured at all -- a cropped head
carries no scale -- so that part is identity_headroom, default 1.15. Growth is
anisotropic: up and sideways, never down into the neck and collar, which stay
the guide's own pixels. The margin between mask and crop border is a budget:
the grow takes what it needs, the feather gets half of what is left, floored --
a ramp wider than its share fades the head itself.
Two fixes for state that outlived a run:
- the transformer_options written into the guider are now undone in a finally.
The guider belongs to another node and survives in the execution cache, so
the reference latents stayed reachable (VRAM never freed) and a later run
that did not overwrite the same keys sampled with the previous clip's specs.
- chunked clips move each sampled chunk to the offload device as it comes out,
instead of holding all of them on the sampling device.
40 tests, including that each side grows by exactly its own amount (the pad is
mirrored: padding the top grows the mask downward) and that the feather never
reaches past the margin it has.
The node was feeding one mask to two jobs with opposite needs. A soft edge
hides the seam when compositing, but in the denoise mask it means partial
denoising: the original latent, and with it the original identity, blends
back in exactly at the edge of the head. mask_hard_for_inpaint (default on)
binarises the mask for the sampler and leaves the soft one for the paste.
latent_mask_dilate grows the mask by whole latent cells after the reduction,
which is the scale that actually decides whether the head sits inside
editable blocks -- 32 px per cell, 8 video frames per latent frame. On a test
mask one cell went from 11% to 25% of the grid. latent_mask_dilate_frames
does the same along time. The latent_mask output reflects both.
Internal decoding exists only because the paste-back needs pixels. For
anything past a single pass the graph's own VAE Decode (Tiled) is the better
tool -- tunable, visible, already there -- so the docs now say so and point
at decode: none plus the paste-back node.
A second pass after a 2x upscaler leaves a latent the full decode cannot
hold: VRAM pins at 99%, the GPU oscillates between 0 and 100 as tensors
shuttle, and it reads as a hang. Graphs solve this with VAE Decode (Tiled);
the node had no way to.
decode: full | tiled | none, with the usual tile/overlap and temporal
controls. none returns the latent alone, for feeding your own tiled decode
downstream -- and then there is nothing to paste back, so compositing is
skipped too.
Without forceInput the BOUNDING_BOX input renders as editable x/y/width/height
fields, inviting a hand-typed box -- which would paste every frame into the
same fixed rectangle instead of following the planned path.
The frontend builds node help as /extensions/<pack>/docs/<NodeID>.md, served
from the pack's registered web dir -- which ComfyUI only registers when the
folder exists at boot, and which the registry path reads from [tool.comfy]
web. Declared both, plus DESCRIPTION on each class, since the classic API
ignores docstrings.
With cropping on, the sampled latent is the crop's, which is exactly what a
refinement pass wants: the face has pixels there and the surrounding frame
never needed a second pass. But the node composited before returning, so
there was no way to get between the two.
paste_back: off now keeps images and latent in crop space, crop_bboxes comes
out alongside them, and the new BFS Head Swap Paste Back node composites at
the end with the same edge-aware feather and mask confinement.
Two behaviours the reference uncrop has and ours did not.
Sides of the crop that sit on the image edge are no longer feathered: there
is nothing outside to blend into, and fading there leaves a washed band along
the frame border.
paste_confine_to_mask (default on) pastes only inside the mask rather than
the whole crop rectangle, so whatever the model invented in the crop's
background never reaches the final frame.
Inpainting preserves the INITIAL latent outside the mask. The node was
handing it the empty latent, so everything outside the mask came back grey
instead of the source video -- the encode that a normal inpaint graph does
before Set Latent Noise Mask was simply missing.
The guide chunk is now VAE-encoded and becomes the sampling latent whenever
a mask is in play. On AV latents the audio stream from the connected latent
is kept and only the video stream is replaced.
The vendored planner emits boxes as {"x","y","width","height"} dicts, not
(x,y,w,h) tuples, so tracked/zoomed died with KeyError: 0 on paste-back.
Both shapes are accepted now.
mask_over_source also draws the moving box, one per frame, so the planned
modes can be watched the same way the static one could.
The frontend fetches node help from /extensions/<module>/docs/<NodeID>.md,
which is served out of WEB_DIRECTORY -- a docs/ folder at the pack root is
never reached. Moved to web/docs/.
mask_strength controls how completely the masked region is replaced. It is
NOT how you get expressions: those arrive through the aligned guide, which
carries the whole source performance regardless of the mask. Below 1.0 the
original latent blends back, keeping the original geometry and dragging the
original identity along with it -- an average of both faces. Documented as
such so it is not reached for by mistake.
The sampler no longer reaches into a sibling pack at runtime: the crop path
planner and its helpers now live here as bfs_crop_planner.py and
bfs_subject_crop.py, copied from drozbay/MaskVidExperiments (d98cc89, v0.2.0).
Both packs are GPL-3.0, which is what makes the copy legal; attribution is in
NOTICE and in each file's header. Only the ComfyUI node classes were dropped
-- they stay in the original pack, where they belong.
tracked/zoomed therefore work out of the box now, and the per-frame paste-back
is ours rather than a call into their uncrop node.
Also adds the node's help page (docs/BFSHeadSwapMaskedSampler.md, the '?' on
the node): what each crop mode does, why the mask is native inpainting rather
than a LoRA feature, why latent_mask is the output to check first, how to size
the latent to the crop, and the order to switch things on when a run looks
wrong. scipy is now a declared dependency (the planner uses ndimage).
The sampled size comes from the connected latent, and the zoomed crop mode
rescales crops by design, so the result is not guaranteed to match the box
it came from -- pasting then fails on a shape mismatch. Resize first, and
report the size the latent should be to avoid the resample entirely.
Debugging this pipeline blind is the slow way. Five outputs now say what the
node actually did:
mask_over_source the mask painted on the ORIGINAL frames with the crop box
outlined -- is the mask where you think it is?
cropped_guide exactly what the model was fed
crop_mask the mask after grow/blur, in the crop's pixel space
latent_mask the mask as the SAMPLER sees it: max-reduced to the latent
grid and upsampled back. One frame per latent frame, which
is the real resolution of the edit and where a thin mask
disappears entirely
debug crop mode and box, mask ops, tiling
CFGGuider captures model_patcher.model_options BY REFERENCE when it is
constructed (comfy/samplers.py:937), so swapping guider.model_patcher does
not propagate -- the guider keeps sampling with the dict it captured, and
the reference specs LTXMultipleControls wrote on the clone never reach the
forward. That is why the guide had no effect at all despite the node
reporting two active slots. The keys are now mutated into the guider's own
transformer_options in place, the way the Edit Anything looping sampler
already does it.
LTX-2.5 latents are AV, and the video VAE takes the video stream alone --
what LTXVSeparateAVLatent does in the stock graphs.
LTX-2.5 latents are AV: video and audio nested together, which is why the
working graphs carry an LTXVSeparateAVLatent. Building a plain
[1,128,t,h,w] tensor hands the model a structure it does not expect, and
the guide stops reaching it -- the run returns the reference face and none
of the source clip.
The node now takes an optional latent (connect EmptyLTXVLatentVideo, sized
to the crop when cropping), slices it per chunk when it is video-only, and
refuses chunked sampling on a nested AV latent rather than corrupting it.
The old fabricated latent stays as a last resort, with a warning.
Two bugs from the first cut, both caught on the first real run.
_sample_chunk samples through guider.model_patcher, so passing the patched
clone as the model argument did nothing: the reference specs sat in
transformer_options that the forward never read, and guide + identity were
silently inert -- sampling ran to completion and ignored both. The pack
already had _set_guider_conds for exactly this; use it.
vae.decode takes the sample tensor, not a latent dict.
The LoRA keeps doing only its job -- guide on source_id 1, identity on
source_id 2. The mask never reaches it: it becomes latent["noise_mask"],
which the guider consumes as denoise_mask, so only the masked region is
denoised and the original face stays visible to the model.
Masks are reduced to the latent grid with MAX rather than ComfyUI's
trilinear resize, which blurs a mask across frames and lets the original
content bleed through the edit.
Crop exists for the case a LoRA cannot fix: a face too small to carry
identity. tracked/zoomed borrow drozbay/MaskVidExperiments' planner when
that pack is installed -- its boxes hold still through mask noise and
occlusion, and a jittering crop reads as camera motion to a video model.
Without it, one static box around the subject's whole travel.
Helpers are unit-tested; the sampling path still needs a GPU run.
ComfyUI's pixel coords are int64 -- latent corners multiplied by integer VAE scale factors --
while centring a reference on the target's height and taking a clip midpoint are half-pixel
operations. Adding a float shift into a Long tensor in place raises "result type Float can't
be cast to the desired output type Long", which is where the sidecar layout landed next.
_apply_tass_layout now promotes both coordinate tensors to float32 for the shift and rounds
back to the caller's dtype on the way out. Rounding matters: the caller casts the reference
coords to the target grid's dtype when concatenating, and a cast truncates where rounding is
what a half-pixel centre wants.
The earlier tests missed this because they built float tensors -- test_tass_layout now runs
every layout on int64 input and checks both the dtype and that the result stays within half
a pixel of the float placement.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HGYHCVjHe6acihEpCRyn1K
1.29.1 stopped the crash but only got the placement right on 2.5. The temporal span the
trainer writes -- bounds = (t_min, t_max) of the target clip -- is collapsed to one number
per token by the model, and the two LTX generations do it differently:
use_middle_indices_grid=True (2.5, and any checkpoint whose metadata declares it)
-> (start + end) / 2, i.e. the middle of the clip
use_middle_indices_grid=False (2.3 checkpoints that predate the flag)
-> the start bound alone, i.e. frame zero
So the same bounds mean "centre of the clip" on 2.5 and "pinned to frame 0" on 2.3 -- one
LoRA, two placements, and the 2.3 one is exactly the single-instant conditioning sidecar
exists to avoid. The layout now takes the model's own flag (read off the transformer in
_process_input) and writes the midpoint into both bounds when the model reads the start,
which lands the reference in the same place under either rule. Legacy [B, 3, N] coordinates
get the midpoint for the same reason.
Verified by collapsing the coordinates the way generate_freqs does and measuring the
reference against the target in the same frame: 2.5 is bit-identical to the trainer, and
2.3 matches to within half a patch (the coordinate convention's own offset, which applies
to the target's tokens equally).
tests/test_tass_layout.py covers both shapes, both collapse rules, the batch-1 reference vs
batch-2 target that crashed, the margin, and that patch spans are translated rather than
squeezed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HGYHCVjHe6acihEpCRyn1K
Two wrong assumptions in 1.29.0, both hit by the first real run:
shape -- LTX-2.5 hands out patch bounds, [B, 3, N, 2], not the one-corner-per-token
[B, 3, N] the node was written against. The other layouts survived it by
accident (their math is elementwise), but each was reducing over tokens only,
so a patch's start and end bound were shifted by their own separate minima --
which squeezes the span instead of translating it.
batch -- layouts run BEFORE the reference tokens are expanded to the CFG batch, so the
target arrives [2, ...] while the reference is still [1, ...]. overlap/st_drc
broadcast out of it; sidecar assigns in place and raised
"The expanded size of the tensor (1) must match the existing size (2)".
_tass_axis_stats now reduces over every dimension except the T/H/W one -- tokens, the
bounds axis when present, and the batch -- which fixes both at once and keeps the older
[B, 3, N] shape working. Verified against ltx_core.conditioning.reference_layout: all four
layouts now agree with the trainer bit-for-bit on the 2.5 shape, in the exact batch-1
reference vs batch-2 target configuration that crashed.
Also passes the spec's sidecar_margin_pixels through instead of always using the default.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HGYHCVjHe6acihEpCRyn1K
The trainer has had four reference layouts for a while — overlap, st_drc, strata
and sidecar — but the nodes only ever exposed three. Training a LoRA with the
reference in sidecar therefore had no matching inference path, which is how you
end up running a checkpoint in a layout it was never trained in.
sidecar places the reference beside the target on the W axis, vertically centred
on it, and stretches its T span to cover the target's whole clip, so an object
reference stays available for every frame instead of sitting at one instant. It
is the layout the replace_refedit recipe validated on 2.3 for exactly this role.
Mirrors ltx_core.conditioning.reference_layout, adapted to ComfyUI's [B, 3, N]
coordinate tensor instead of the trainer's [B, 3, N, 2] patch bounds, and takes
sidecar_margin_pixels like the trainer does.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two formatting rules the VLM breaks consistently, and that no amount of prompt
work fixed — five rounds of rewriting plus temperature 0 still produced the same
violations:
no reference -> the marker sentence must not exist, and a swap is a plain
Replace. The model emits Add + marker + Remove instead, which
is the reference form applied where it does not belong.
reference -> the marker may only follow a sentence starting with Add. When
it lands on a Replace, that pair appears in zero training
samples.
Both are decidable in code: the caller already knows whether it attached a
reference image, so the node takes that boolean rather than asking the model to
infer it. Add+Remove is recomposed into Replace, and Replace+marker into
Add+Remove, preserving the object description the VLM did get right.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The stock SaveLatent calls .contiguous() straight on samples["samples"], which
fails on H3 because the latent is a NestedTensor holding the video and audio
components. This unbinds them first and writes each as its own tensor, so an H3
latent can be taken out of the graph and decoded or inspected offline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The node read the device off the decoder parameters, but ComfyUI keeps the VAE on
CPU until something loads it — vae.decode() calls load_models_gpu first, and the
direct path skipped that. The result was a full ViT decode on CPU: minutes of
work, fans up, no output.
Now it calls comfy.model_management.load_models_gpu([vae.patcher]) and uses
vae.device / vae.vae_dtype like the native path, returning to the intermediate
device. Measured on a 768x1280 decode: 2.4s on cuda:0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The H3 latent arrives as a NestedTensor holding the video and audio components,
so `z.dim()` raised AttributeError as soon as the node was fed a real generated
latent. The native nodes unbind and keep the first component (nodes.py:335); this
does the same, and re-adds the batch dimension that unbind drops.
Verified both paths produce identical output (MSE 0.00e+00) on the same latent.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Decoding a DiT-generated H3 latent through the stock VAEDecode produces a visible
grid. The reference path does not use vae.decode() at all: both the checkpoint
card and the multimodalart/MiniMax-H3-images Space call the decoder directly.
This node is that recipe, with the four details its author flags as silently
corrupting if changed:
1. vae.decoder(vae.post_quant_conv(z)) directly, never vae.decode(), whose
temporal chunker cannot form a chunk from the few latent slices an image
generation produces;
2. decoded[:, :, -1] -- the last output slice, not index 0;
3. latent denormalisation, z * latents_std + latents_mean;
4. ImageNet pixel denormalisation, without which the frame comes out dark and
over-contrasted like a tone-mapping bug.
Reconstruction check against a real image: 22.43 dB, block ratio 1.30 at the
32px patch boundary -- matching the raw decoder path measured separately, which
is what confirms the node reproduces the reference rather than approximating it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Ports the sharpest-frame selection from scanpe_experiments/h3_image.py, which
takes it from IF-Edit (arXiv:2511.19435): decoded frames carry different amounts
of residual motion blur, so score by Laplacian variance and keep the crispest
instead of a fixed index.
Pairs with running MiniMax-H3 as an image generator: the VAE's minimum clip is 5
pixel frames (2 latents), decode yields frames of uneven sharpness, and the best
one is not always at the same index.
The node documents a limitation found while testing it, because it is easy to
misuse: Laplacian variance is a BLUR heuristic, not a quality metric -- artifacts
are high-frequency too. On a real H3 decode it picked the frame with the WORST
PSNR of its block (19.72 dB against 23.19 for its neighbour) because that frame
was the most artifacted. It is sound when the candidates are the same content
differing only in blur, which is the IF-Edit setting; it is meaningless for
choosing among frames from different moments.
Also relevant and not obvious: the "last frame of each decoded block is the valid
one" rule (decoded[:, :, -1] in the upstream example) held for the first block
here (34.49 dB against 26.80 for its neighbour) but inverted on the second.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
