Commit Graph
56 Commits
Author SHA1 Message Date
alisson-anjos f1f639f389 Fix Shot Planner splitting and TextGenerate compatibility 2026-10-13 07:12:26 +00:00
alisson-anjos ceeff4803d Add simplified H3 shot conditioning 2026-10-12 13:13:43 +00:00
alisson-anjos 8aeb958ab6 Planner: stabilize VLM descriptions and default videos to 15s 2026-10-12 10:27:36 +00:00
alisson-anjosandClaude Opus 5.5 a04f3116cf {details} filled automatically by the render nodes too: the planner leaves it when it has no VLM, BFS Shot H3 Conditioning / Shot H3 Duet (new vlm input) describe the shot's references
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 12:48:30 +00:00
alisson-anjosandClaude Opus 5.5 2e64d989f6 BFS Shot H3 Duet: continuity 'history' too (the previous end before frame 0, panel strip filled with the source under it); shared shot_history() pops it from the shot
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 02:17:41 +00:00
alisson-anjosandClaude Opus 5.5 b1e0fcbbb6 Shot Planner: continuity 'history' is the default for new plans and unset shots (cuts still skip it)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 02:14:20 +00:00
alisson-anjosandClaude Opus 5.5 a03e0be342 Shot Planner: continuity 'history' - the last 17/34/51 frames of the previous result placed just before frame 0 on H3's timeline (sliding history, after Akatz's H3 Relay / Ethanfel's Context Loop), with every mask mode, crop and the duet canvas; cuts reset it
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 01:37:36 +00:00
alisson-anjosandClaude Opus 5.5 596a1c2538 Duet prompts: TSC's newer strategy - the finished split screen as plain footage (no kept footage / generated / in sync), the pinned region named by place + one clause of what is seen ({panel_region}/{video_region}: top quarter, bottom three quarters...), motion beat by beat with times on the video clock, sounds of the new scene
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 21:18:07 +00:00
alisson-anjosandClaude Opus 5.5 4e4fc20fdd Duet: optional VLM prompt - writes it from the task, or fills the {...} fields of a hand-written template (TSC's); setting picture described as <Subject 2>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 21:08:35 +00:00
alisson-anjosandClaude Opus 5.5 ad99a9f91a Duet: panel_static (TV static over face/person, SAM 3) + setting picture input + BFS Setting Picture node
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 20:52:18 +00:00
alisson-anjosandClaude Opus 5.5 e43cc14d9a Points: Track -> whole video - one selection tracked by SAM 3 through the whole video or range (chunked, forward and backward, re-found by text after cuts); shots without their own selection take their part (1.81.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 19:47:21 +00:00
alisson-anjosandClaude Opus 5.5 4fab22caa9 Shot Planner range: a player in the Video tab, start / end in frames (start here / end here take the player's exact frame; seconds of older plans still read), aligned layout (1.80.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 19:09:13 +00:00
alisson-anjosandClaude Opus 5.5 34ffcef172 Shot Planner: Start at / End at range - only that part of the video is split, analysed, generated and joined; timeline dims the rest; max total seconds counts from the start (1.79.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 17:42:42 +00:00
alisson-anjosandClaude Opus 5.5 9ae5fc9614 Fit the prompt: only for the short preset, the VLM is told to reply with the blank's words only, and an echo of the prompt sentence is cut; Checks warn when {details} is used with a paragraph preset (1.78.2)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 15:14:58 +00:00
alisson-anjosandClaude Opus 5.5 8e573275b8 Fit the prompt: descriptions ({details}, {target}) are written for the prompt sentence that holds them (blank in place, article matched); {target} is described automatically at run time for shots with a selection and no text (1.78.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:34:08 +00:00
alisson-anjosandClaude Opus 5.5 a5535dc77c Describe refs for any subject (person, creature, robot, animal, cartoon: says what it is first); new default preset 'short' - a noun phrase that fits '<Subject 1> is the {details} shown in <Picture 1>'; {target} describes animals/creatures too (1.77.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:35:11 +00:00
alisson-anjosandClaude Opus 5.5 39610ab1ef mask_guide_look: what the mask guide shows of the subject - grey blurred (default), colour, silhouette, edges, or pose (people, YOLO pose model from models/ultralytics, no new packages) (1.76.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 15:47:14 +00:00
alisson-anjosandClaude Opus 5.5 f1388f0463 mask_guide (renamed from person_guide, works for any subject): masked region as an extra guide, instead of the full guide, or as a native reference video (<Video n>, with mask_ref_size); Mask only shots only; Frame + paste takes a per-shot 'new subject' text for the new outline (1.75.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:51:56 +00:00
alisson-anjosandClaude Opus 5.5 5c93d23c88 H3 Conditioning: experimental person_guide - a guide latent with only the masked person (rest grey), added next to the full guide or instead of it (1.74.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:45:55 +00:00
alisson-anjosandClaude Opus 5.5 7d9d7a9e26 Frame + paste mode: the whole frame is generated (pose follows the guide, the new person may be bigger) and BFS Shot Join pastes only the person (shot mask + the new outline found by SAM 3) onto the original frame; mask strength renamed mask opacity (1.73.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:22:16 +00:00
alisson-anjosandClaude Opus 5.5 1de7753be9 Mask strength per shot (mask modes): the masked rows start at strength x the noise level and keep part of the original, like an opacity (1.72.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:09:17 +00:00
alisson-anjosandClaude Opus 5.5 aa545097b7 External masks instead of SAM 3: a mask video per shot, the planner's optional mask input (whole video) or a mask video for the whole video in the panel; used by every mode, {target}, the setting picture and the comparison (1.71.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:01:13 +00:00
alisson-anjosandClaude Opus 5.5 2022b48e07 Shot Join comparison: mask-only shots show their generation mask (as grown by expand) over the original column, not only cropped shots
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 11:50:44 +00:00
alisson-anjosandClaude Opus 5.5 907e923816 Shot H3 Conditioning: inpaint 'only the mask' also without crop (whole frame, no crop/uncrop); the mask's expand now grows the generation mask (1.69.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 09:30:48 +00:00
alisson-anjosandClaude Opus 5.5 52000210c3 Shot H3 Conditioning: inpaint 'only the mask (crop)' - latent starts from the cropped shot, the SAM 3 mask becomes the H3 generation mask; credit to Neko (Nekodificador) and AbleJones (1.68.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 08:44:33 +00:00
alisson-anjosandClaude Opus 5.5 3fc67735b8 Shot Join: stitch finishing for cropped shots - edge hardness, shot-wide colour match (ring around the subject for swaps), seamless edges; adapted from Nekodificador's NKD Inpaint Stitch (MIT) (1.67.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 08:39:34 +00:00
alisson-anjosandClaude Opus 5.5 ceaec97e63 Shot Join comparison: mask (red) and crop box (yellow) over the original column for cropped shots (comparison_mask) (1.66.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 08:26:58 +00:00
alisson-anjosandClaude Opus 5.5 1468279071 Shot Planner: {target} placeholder - pick the person (SAM 3 points / text), the VLM describes them, the prompt names who is replaced (1.64.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 08:16:18 +00:00
alisson-anjosandClaude Opus 5.5 90face345c Shot H3 Conditioning: setting_ref, the shot's frame with the person in TV static as an extra reference (TSC's tip), generation or source size (1.63.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 23:38:02 +00:00
alisson-anjosandClaude Opus 5.5 6015065898 Shot H3 Conditioning: optional duet prompt writer (task + instruction, VLM or template, prompt output); {layout} filled per duet mode; template without split-screen wording in shifted mode (1.62.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 23:26:57 +00:00
alisson-anjosandClaude Opus 5.5 4454bd617f Shot Planner: detect the video's audio, use it or a silent track; planner, join and duet always output valid audio (PyAV fallback) (1.58.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 14:52:23 +00:00
alisson-anjosandClaude Opus 5.5 43a31ed934 Shot Join: comparison frames on a multiple of 16 (x264 needs even sizes) (1.56.1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 00:10:58 +00:00
alisson-anjosandClaude Opus 5.5 c37667c70c Shot H3 Conditioning: duet option (canvas / shifted RoPE), the join cuts the panel off (1.56.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-03 00:07:59 +00:00
alisson-anjosandClaude Opus 5.5 16674de83f Shot Planner: SAM 3 mask & crop per shot with uncrop in the join, global mask settings, points editor, VLM suggestions (1.55.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 23:58:03 +00:00
alisson-anjosandClaude Opus 5.5 7017a7633d Shot Planner: per-shot continuity from the previous shot's result (reference or first frame) (1.54.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 23:31:44 +00:00
alisson-anjosandClaude Opus 5.5 4ddac428c9 H3 Duet: dynamic references, task draft prompts, example and credits; Shot Join comparison output (1.53.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 22:16:20 +00:00
alisson-anjosandClaude Opus 5.5 51a622c457 H3 Duet / Side Panel nodes (training-free pinned panel); cast on CPU with top faces per frame; Unpack shot output (1.50.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 21:03:00 +00:00
alisson-anjosandClaude Opus 5.5 fc7a026ab7 Shot Planner: face-tracked cast, link a reference per person, split/assign/skip shots by who is on screen (1.49.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 20:20:48 +00:00
alisson-anjosandClaude Opus 5.5 e67b22a0ea Shot loop: BFS Shot Repack to put edited pieces back into a shot (1.48.2)
Unpack -> any processing (e.g. remove the reference background) -> Repack. Unconnected
inputs keep the shot's values; replacement guide frames are fitted to the shot's
generation size and length, and timing, cuts and audio are unchanged so the join still
works. Runs once per shot like the rest of the loop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 20:10:32 +00:00
alisson-anjosandClaude Opus 5.5 bb87918e57 Shot Planner: content filters with YOLO person/face detection and timeline fill
- Filters per shot from a few sampled frames: needs a person, minimum person size,
  maximum people, needs a face, skip dark/fades, skip static shots, minimum frames.
  People and faces use YOLO from models/ultralytics (Impact Pack person/face models)
  with OpenCV fallbacks; stats are cached per shot.
- Per-shot override (filters decide / always run / always skip) and content badges
  on every shot card; "Analyse people & faces" previews the numbers.
- Planner outputs a timeline with every shot, including the ones that do not run.
  BFS Shot Join fills skipped shots with the original video (full duration and
  soundtrack) or removes them together with their audio.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 17:39:01 +00:00
alisson-anjosandClaude Opus 5.5 bf473e1dbd Shot Planner / Shot Loop: split long videos into model-sized shots (1.48.0)
A video model only follows a guide inside the clip length it was trained on, so long
sources are split into shots that fit and run once per shot.

- BFS Shot Planner: interactive Vue timeline (thumbnails, cut-detector curve, draggable
  boundaries, split/merge, per-shot reference gallery with upload, per-shot prompt).
  Splits at camera cuts via PySceneDetect (adaptive/content, built-in fallback), fixed
  length, or by hand; lengths snap to the model grid (H3 17n+5, LTX/Wan 8n+1, 4n+1).
  Outputs the shots as a ComfyUI list so everything downstream runs once per shot.
- BFS Shot Unpack: guide frames, references, prompt, length, audio of one shot.
- BFS Shot H3 Conditioning: native MiniMax H3 conditioning per shot (aligned guide,
  native reference video, or both).
- BFS Shot Join: concatenates in order, trims each shot to its true length, cross-fades
  soft joins with the real overlap frames, returns the soundtrack.
- Queue loop mode: one shot per run stored on disk, auto re-queue, nodes after the join
  blocked until the last shot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 17:32:32 +00:00
Codex f145527ea5 Add H3 reference RoPE options and caption-only identity conditioning 2026-09-30 20:30:01 +00:00
Codex bd23236bdd Add aligned downscaled MiniMax H3 guides and image/video targets 2026-09-30 17:26:56 +00:00
alisson-anjos 12bd7a73b1 Shape the crop to the latent's aspect, not the latent to the crop (1.41.1)
1.41.0 fixed the wrong half of this. The deformation came from the crop and the
latent having different ASPECTS -- a mask around a face gives a portrait box,
sampled in a landscape latent it is stretched on the way in and squeezed back on
the way out -- and rebuilding the latent at the crop's size did remove the
mismatch, but by throwing away the resolution that cropping exists to buy. A
face 200 px tall in the source was then sampled at 200 px, which is the problem
the crop was there to solve.

The planner already takes an aspect_ratio and we were passing 0.0, letting the
box follow the subject. It now gets the connected latent's aspect, and
_static_box grows its short side to match when the planner is not in play. The
box still contains the subject and stays divisible by the grid.

So the latent's size is yours again -- it is the sampling resolution, choose it
for the quality you want -- and its shape no longer has to be guessed.

68 tests; the new ones pin that a landscape latent gives a landscape box, that
the box still contains the subject, and that a free aspect still follows it.
2026-09-03 17:08:17 +00:00
alisson-anjos dab4233e48 Rebuild the latent to the crop instead of asking for it (1.41.0)
The crop's size is derived from the mask, so an EmptyLTXVLatentVideo built
upstream cannot know it -- and a latent that disagreed was not an error
anywhere. The model sampled at the latent's shape and the paste-back squeezed
the result into the box: a face crop is portrait, sampled into a landscape
latent it came back visibly deformed, and under crop_mode zoomed the deformation
changed frame to frame. The only cure was to run once, read the size off the
debug string, rebuild the latent and run again, after every change to the mask.

The sampler now rebuilds the latent's video stream at the crop's shape when they
disagree, keeps the audio stream as it is, and reports it in debug. Nothing is
lost: at denoise 1.0 the initial latent contributes nothing, and the inpainting
path overwrites the video stream with the encoded guide regardless.

BFS Crop Size stays useful for building the latent at the right size up front
(skipping the rebuild) and for taking the per-frame boxes straight to BFS Paste
Back, but it is no longer the price of a correct result.

68 tests.
2026-09-03 16:52:09 +00:00
alisson-anjos c145b091d7 BFS Crop Size: the crop's size before anything is sampled (1.40.0)
The sampler derives its box from the mask, so the size to build the empty latent
at was only knowable after a run: sample a whole clip, read one number off the
debug string, rebuild the latent, sample again -- and repeat after every change
to the mask or the crop widgets.

Getting it wrong does not look like a sizing problem. A mask around a face in a
1920x1080 frame gives a portrait box, 512x672 say; sampled into a 768x512
landscape latent the crop is stretched horizontally, and the paste-back then
squeezes the result back into the portrait box. The face comes out deformed, and
under crop_mode: zoomed the deformation changes frame to frame.

The new node runs the same planner on the same mask and returns width, height,
the per-frame boxes and a note, so the latent can be built up front. The planner
is deterministic, so identical crop widgets give the sampler the identical box.

Also factored the per-frame box normalisation the sampler did inline into
_boxes_per_frame, which both nodes now share.

63 tests.
2026-09-03 16:50:39 +00:00
alisson-anjos b5d08490b0 Drop the overlap when concatenating chunks (1.39.2)
Chunked sampling built its chunks with an overlap for continuity and then
concatenated them whole, so the overlap was emitted twice. 97 frames at tile 72
overlap 16 gave 15 latent frames where the clip has 13: the output ran longer
than the input, and from the first seam on, frame i was no longer moment i.

Pasting back made it obvious rather than subtle -- crop_bboxes has one box per
source frame, so every frame after the seam went into a box belonging to a
different moment, and with crop_mode: zoomed into a box of a different size too.
That is the "crooked and out of sync" this was reported as.

Each chunk after the first now contributes only its tail, the part the previous
chunk did not already cover, so the concatenation is exactly the clip's latent
length. Single-chunk runs are untouched.
2026-09-03 15:45:58 +00:00
alisson-anjos 221d9382a0 Stop import comfy.nested_tensor from shadowing the module (1.39.1)
`import comfy.nested_tensor` inside execute() binds `comfy` as a LOCAL for the
entire function, so every other comfy.* use in it raises UnboundLocalError -- at
compile time Python decides the name is local, and the AV branch that would have
bound it does not run on a video-only latent.

It shipped in 1.37.0 with the chunk-offload line:

    got = got.to(comfy.model_management.intermediate_device())

which only runs when a clip is long enough to be sampled in more than one chunk.
That is why it survived the tests and the short runs and failed on the first
real long video:

    UnboundLocalError: cannot access local variable 'comfy'

Imported as `from comfy import nested_tensor as _nested` now, which binds
`_nested` and leaves the module-level `comfy` visible. A test asserts neither
node's execute() has `comfy` in co_varnames, which catches the whole class of it.
2026-09-03 14:44:49 +00:00
alisson-anjos c79629dc80 Confine the paste to the mask even with no crop (1.39.0)
_paste_back gave up when there was no crop box and returned the sampler's frame
whole. Nothing outside the mask *changed*, but the entire frame had been through
the VAE's encode/decode, so every pixel the edit never touched came back softer.
For a pointwise edit, or anything sharpness-adjacent, that is the opposite of
what the mask was for.

With no crop and a mask, the full frame is now composited into the ORIGINAL
frames through that mask, the way the crop path already did: outside the mask
the output is the source verbatim, never a round trip of it. The mask's own blur
supplies the soft edge -- there is no box border to feather here.

This makes crop_mode a resolution decision again. Before it, preserving the
untouched pixels required turning the crop on even when the region already had
all the resolution it needed.

52 tests; the new ones pin that outside is the source byte for byte, inside is
the result, no mask still composites nothing, and a result of another size is
brought back to the frame first.
2026-09-03 14:27:51 +00:00
alisson-anjos df32bdc0fe BFS Sampler: optional identity, and put the new widgets back at the end (1.38.0)
Three things.

`identity_image` moves from required to optional. The node was never head-swap
specific below the surface -- the reference slot is simply not packed when it is
empty, and everything else (crop, mask, inpainting, chunking, paste-back) is
indifferent to which LoRA is running. An instruction-edit, restyle or sharpening
IC-LoRA now runs on the guide alone. IMAGE is a link, not a widget, so this
costs no widget position.

Renamed to "BFS Sampler (crop · mask · loop)" and "BFS Paste Back". The
registration keys are deliberately unchanged: saved workflows reference those,
and only the display name is free to move.

And the fix for the reason this was noticed: 1.37.0 inserted auto_config and
identity_headroom BEFORE crop_mode. ComfyUI stores widgets_values positionally,
so every later value in every saved workflow shifted two slots -- crop_mode's
string landed in auto_config, crop_scale's float in identity_headroom, and the
graph failed validation with "input out of range" on crop_scale and
crop_divisible_by. Both widgets are appended at the end now, with a comment
saying why, and a test pins the historical order so the next addition cannot
repeat it. ltx_multiple_controls.py already carried that rule in a comment;
this file did not follow it.

48 tests.
2026-09-03 14:07:39 +00:00