Compare commits

..
Author SHA1 Message Date
alisson-anjosandClaude Opus 5.5 7db9eca1c9 Shot Planner: points on several key frames of a shot, the mask over every frame (1.116.0)
- A selection (the shot's mask and each Colored Mark person) has key frames, each with its
  own points: SAM 3 selects the subject on a key frame and tracks it up to the next one
  (before the first, backward), so points added on a later frame correct the mask from there
  on. Plans with one key frame (points + key) work as before; keys are saved next to them.
- The points window: after Segment the mask is shown over every frame of the shot, with a
  play button, frame stepping (arrows), key-frame marks on the slider and a key-frame list
  (jump / remove). Edited points mark the preview as old until Segment again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 18:19:16 +00:00
alisson-anjosandClaude Opus 5.5 488cb77710 Shot Planner: Auto split with Max shots no longer leaves a giant last shot (1.115.1)
Auto split saved only the starts of the shots the server returned under Max shots, so the
last kept shot had no end and ran to the end of the range. It now asks for every boundary
(Max shots only picks how many shots run), so the kept shots keep their length and raising
Max shots later shows the next ones split too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 17:08:10 +00:00
alisson-anjosandClaude Opus 5.5 0cd1c04d17 Video Compare: several reference pictures in a strip below the videos (1.115.0)
- The reference input takes the refs list of BFS Shot Planner / Unpack (pictures of any size)
  or a batch, up to 12 (was: one batch, 4 shown).
- The player shows them in a strip below the videos, numbered Picture 1..n, instead of over
  the video's corner; ⬇ save / ● rec put the same strip below the videos in the file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 16:01:15 +00:00
alisson-anjosandClaude Opus 5.5 857ba260f3 Shot loop: every picture always reaches H3 (use_ref_2 removed); Auto prompt caption style; loop-proof writer (1.114.0)
- BFS Shot H3 Conditioning and its Simple version always pass every picture of the shot: the
  use_ref_2 switch is gone (the pictures are chosen in the planner). The published v1.1
  character-swap workflow had it off, so only <Picture 1> reached the model. Saved
  workflows are upgraded on load: their widget values are put back by name (checked with
  the published v1.1 and tutorial workflows).
- 🤖 Auto prompt: Style = H3 official (six sections) or OpenJutsu captions, the short
  'Replace the ... in the video with ... from <Picture n>. Keep ...' style of the training
  captions (written from the request, the frames and the pictures).
- The writer samples (repetition and presence penalties) instead of greedy decoding, which
  looped on long answers; repeated sentences and a cut sentence are removed, missing sound
  sections completed; an answer with problems (missing <Subject n> in detailed_description,
  a loop, H3 sections in a caption) is written once more, told what was wrong; labels still
  missing are tied after [Shot 1]. A subject acts with its own body (hands, not paws).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 15:22:30 +00:00
alisson-anjosandClaude Opus 5.5 a93cea17a9 Shot Planner: Auto prompt applies the panel request to the prompt input's template; real cut times (1.113.0)
- With Auto prompt on, the planner's prompt input is the template (e.g. a LoRA's body_swap
  prompt) and the panel's text is the request applied to it; the run tells the panel the
  template so 🤖 Write uses the same one (shown in the card). Without a VLM the template is the
  prompt. The template is part of a written prompt's 'made from'.
- The writer gets the shot's real cut times from the cut detector and at least one key frame
  per piece between cuts; a shot without cuts gets no timestamps (the ones the VLM writes from
  the frame times are removed), and [Shot 1] never has one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 02:08:18 +00:00
alisson-anjosandClaude Opus 5.5 7b57eacb7b Shot Planner: Auto prompt keeps a template request (trigger word first), timestamps only at cuts, labels with brackets (1.113.0)
- A request that is a whole prompt (H3 sections, e.g. a LoRA's body_swap template) is kept:
  same sentences and order, its trigger word first; the writer adds the pictures' features and
  the shot's camera, place, light and actions after [Shot 1].
- Writer rules: a timestamp only starts a new [Shot N] at a real cut, never inside a shot; the
  replacement is stated (the person of the guide video is replaced by <Subject N>); the summary
  names 'the guide video'; {details@Picture n} left in the request is filled by the writer.
- Picture / Subject / Video / Audio N written without brackets get them back.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 01:59:22 +00:00
alisson-anjosandClaude Opus 5.5 c37ce27ea8 Shot Planner: saved Advanced conditioning nodes keep their widget values after the duet widgets left (1.113.0)
The pre-1.113 BFS Shot H3 Conditioning (Advanced) saved its values in order with the duet
widgets in the middle; on load they are put back by name, and widgets the saved workflow
did not have go back to their default. Checked with the published tutorial workflow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 01:46:09 +00:00
alisson-anjosandClaude Opus 5.5 0d831b0bb5 Shot Planner: 🤖 Auto prompt in H3's official format; duet out of the planner and conditioning (1.113.0)
- 🤖 Auto prompt (Prompts & refs): the prompt boxes become requests and the VLM writes each
  shot's MiniMax H3 prompt in the full-reference format of MiniMax-H3's
  VIDEO_PROMPT_WRITING_GUIDE_ref_en (six sections, task types, retention markers, [Shot N],
  official camera terms) from key frames of the shot (auto: ~1.5/s, 3-8), the shot's
  pictures and their descriptions, the request, Colored Marks, selection and mode, citing
  the source as the guide video or <Video 1>. Write all / missing in the panel, per shot
  Write again and edit; shots without one are written at run time (before SAM 3) and the
  panel stores them; out-of-date prompts are flagged. A written prompt is part of the
  stored shot's signature.
- Duet removed from the planner (its split-screen prompt writer, /bfs/shotloop/write) and
  from BFS Shot H3 Conditioning (Advanced): duet, panel_*, rope_gap, task, instruction.
  The model input stays as a pass-through so workflows that routed the model keep working.
  BFS Shot H3 Duet and the Side Panel nodes keep the duet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 01:44:46 +00:00
alisson-anjosandClaude Opus 5.5 d94e1c2b19 Shot Planner: Colored Mark per picture, refs outputs, object preset, fixed points window (1.113.0)
- Colored Mark: every picture has a mark color (12-color palette, editable under each
  global picture); a shot's selection and each extra person ("+ Mark another person",
  text or points) pick the picture they become ("Picture 3 · red", with its
  thumbnail) and are drawn in its color. The prompt gets a model-agnostic mapping
  ("the person marked in red becomes the character from <Picture 3>"). The
  conditioning node checks each marked picture is passed and colors are not shared.
  Plans with the old second region open with it as <Picture 2>.
- Points window: fixed size; the frame fits a fixed stage (points are stored relative to
  the frame); one point set per marked person.
- BFS Shot Planner outputs refs (every picture the shots use, as a list) and ref_count
  where ref_image / ref_image_2 were.
- Saved workflows are upgraded on load: the planner's ref_image / ref_image_2 inputs are
  removed, old ref outputs become refs / ref_count, and what fed Repack's ref_image
  feeds refs.
- New describe preset "object" (a noun phrase: what it is, shape, materials, colors,
  text), written for the prompt sentence like "short".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 01:27:19 +00:00
alisson-anjosandClaude Opus 5.5 6f335175d9 Shot Planner: one description per picture, Unpack refs list, PictureN chips (1.113.0)
- Describe references is per picture now: one row per picture the plan uses, each with
  its own preset (default = the card's; custom with its own instruction) and 🔄 to
  describe only that one. {details} = picture 1, {details2} = picture 2,
  {details@Picture n} = picture n. Descriptions saved per reference set by older plans
  are read as those pictures'. The separate "Describe each picture" section is gone.
- BFS Shot Unpack: refs (every picture of the shot, as a list: the next node runs once
  per picture) and ref_count replace ref_image / ref_image_2 in the same output slots.
  BFS Shot Repack's refs input takes the list back and splits it per shot; ref_image
  links from older workflows still work.
- Prompt chips read PictureN (they insert <Picture N>).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 01:16:27 +00:00
alisson-anjosandClaude Opus 5.5 7da6f7427f Shot Planner: up to 12 reference pictures, @Picture chips, {details@Picture n} (1.113.0)
- Plans keep a list of pictures (global_refs, per-shot refs) instead of ref / ref2; older
  plans are converted on load and ref / ref2 stay in the JSON as pictures 1 and 2. A
  shot's empty slot uses the global picture at the same position.
- Panel: the global list starts with one slot and + adds more (several files at once);
  drop image files on a slot to replace it or on the box to add them; recent thumbnails
  can be dragged onto any slot. Chips under every prompt insert <Picture N> at the cursor
  (@PictureN) or {details@Picture N} (📝).
- {details@Picture n}: the description of the shot's n-th picture alone. 📝 Describe each
  picture writes one per image (editable, saved as picture_details); without one, the
  planner's VLM, then the conditioning node's VLM, describes it at run time.
- The planner's ref_image / ref_image_2 inputs are gone (a workflow saved with them
  connected still runs and the log says they are ignored). Conditioning and Duet pass
  every picture of the shot (use_ref_2 off: only the first) and log when there are
  more than 9.
- Stored shots keep their signature when a shot has at most 2 pictures and no
  per-picture descriptions are set.
- A two-picture shot no longer gets the automatic <Subject 2> {details2} line when the
  prompt already names <Picture 2> or uses {details@Picture n}.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 00:18:36 +00:00
alisson-anjosandClaude Opus 5.5 084e1d22b5 Shot loop: a new LoRA, seed or sampler runs the shots again; no preview on BFS Shot Join (1.112.0)
- The stored-shot signature now includes what renders the shots: the planner reads the
  queued workflow (hidden PROMPT input) and fingerprints every node upstream of BFS Shot
  Join's images (loaders, LoRAs and strengths, sampler, seed, conditioning settings).
  Changing any of them runs every shot again on the next Run, instead of only
  re-assembling the stored ones. The join's own settings (crossfade, comparison) and
  display nodes don't count. A seed on "randomize" changes every Run, so fix it to keep
  the stored shots.
- BFS Shot Join no longer shows a video preview on the node; the video still goes out
  through images and the file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 14:48:59 +00:00
alisson-anjosandClaude Opus 5.5 55b120d14b Video Compare: choose the format of save / rec: MP4, small MP4, GIF or the browser file (1.111.0)
The browser records WebM (or a fragmented MP4 without a duration) that many apps and chat
previews won't play. The player now uploads the recording to ComfyUI in 8 MB pieces and
converts it there (ffmpeg, PyAV without it):
- MP4: H.264 + AAC, yuv420p, faststart, CRF 18.
- MP4 small: the same with the bitrate set to stay under 10 MB (smaller frame when it gets low).
- GIF: one palette per video, 15 fps, made smaller until it fits 10 MB, no sound.
- browser file: the recording as it is, no conversion.
The choice is remembered per browser.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 12:34:44 +00:00
alisson-anjosandClaude Opus 5.5 526745455f Source blur: choose what stays sharp, masked area, Sapiens2 segmenter, invert colours (1.110.0)
- Modes read "<region> (<what stays sharp>)": face / head + hands / person / masked area,
  with keep mouth (lips, teeth, tongue) / keep lips + tongue (teeth blurred too) /
  mouth too (nothing sharp). The 1.91 values keep working in saved workflows.
- masked area: the shot's own mask (the person being replaced, with its expand), no extra
  segmentation; without a mask it blurs the whole person.
- source_blur_segmenter: SAM 3.1 (as before) or Sapiens2 (facebook/sapiens2-seg-0.4b,
  body-part classes with lips, teeth and tongue apart from the face, one pass; downloads
  to models/sapiens2 the first time, runs in half precision, back to RAM after each shot).
  Frames keep their aspect ratio at the trained area instead of being stretched.
- source_blur_effect: blur, or invert colours (shapes and expression stay, skin tone and
  colours go).
- On BFS Shot H3 Conditioning (+ Simple), BFS Shot H3 Duet, BFS H3 Duet and BFS H3 Duet
  Conditioning.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 12:02:24 +00:00
alisson-anjosandClaude Opus 5.5 78a806103e Video Compare: save / record, reference picture, side-by-side size fix; wheel zoom over panels (1.109.0)
- BFS Video Compare: "save" plays the video once from the start and downloads it as
  shown (layout, divider, labels); "rec" records what you do in the player until stop.
  Drawn into a canvas and encoded in the browser (MP4 where it can, else WebM).
- Optional "reference" input: up to 4 pictures shown in a corner of the player and in
  what save / rec download ("ref" toggles them).
- Side by side no longer grows the player out of the node (the videos stay absolutely
  positioned, one per half); save / rec / labels moved to their own row so the top bar
  fits in one line.
- The top video follows the bottom one by nudging its speed (it used to seek every
  frame: it stuttered, and drew blank into the canvas while seeking).
- BFS Shot Join: turning the comparison on after a run reads the original frames from
  the source video, so the guide column is there (before only the target came back).
- Mouse wheel over the Shot Planner panel and the Video Compare player zooms the canvas,
  except on something that can still scroll that way; Ctrl / Cmd + wheel always zooms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-17 11:50:49 +00:00
alisson-anjosandClaude Opus 5.5 ce3ca59b17 Shot loop: run again without changes only assembles; unload models before the join (1.108.0)
- Every shot stored and up to date: the planner hands out an assemble-only placeholder
  (no VLM, SAM 3 or history) and BFS Shot Join assembles the stored shots. Its images
  input is lazy, so conditioning, sampler and decode are not executed again (before,
  the last shot was generated again just so the join had something to join).
- Run tab: "Unload models before the join" (on by default): once every shot is
  rendered, unload the models so the join and the nodes after it get the memory.
  Not part of the shot signature, so toggling it keeps the stored shots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 22:15:41 +00:00
alisson-anjosandClaude Opus 5.5 c11f3f9f12 Shot Planner: second pass size in the Run tab (1.107.0)
The Run tab gets a 'Second pass (upscale)' section: on/off, and the upscaled
size by multiplier, target dimensions or megapixels with an alignment, the
same arithmetic as Minimax H3 Latent Upscaler (3D). The planner outputs
second_pass (BOOLEAN), upscale_width and upscale_height (INT); the graph does
the latent upscale and the second sampling with its own nodes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 20:04:57 +00:00
alisson-anjosandClaude Opus 5.5 bacdbe9155 Revert the 15 s default range in the node (1.106.3)
The 15 s limit belongs to the published workflow, not to the node: a plan with
no range works on the whole video again, as before 1.106.2. Choosing a video in
the panel still starts with its first 15 s, as it always did.

This reverts commit d62a721.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 12:52:59 +00:00
alisson-anjosandClaude Opus 5.5 d62a72147c Shot Planner: a plan with no range works on the first 15 s (1.106.2)
A plan that never set its range (fresh node, older workflow, plan made outside
the panel) used the whole video, so a long video run through a workflow
nobody adjusted generated every shot of it. Unset now means the first 15 s,
read the same way by the panel and the server; the whole video is the
explicit 'whole video' choice (end frame 0), as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 12:52:14 +00:00
alisson-anjosandClaude Opus 5.5 35ef99090b Write MP4s without an ffmpeg program; manual splits beat Max shots (1.106.1)
Issue #12: on installs with no ffmpeg on PATH (Windows portable), Shot Join's
mp4 mode - also picked automatically for long videos when the RAM is short -
failed with WinError 2. Video files now go through bfs_video_io: ffmpeg from
PATH, else imageio-ffmpeg's, else PyAV (what ComfyUI's Save Video uses), for
writing and for reading the MP4 back. BFS Video Compare uses the same writer.

Issue #13: a workflow saved with Max shots = 1 silently dropped every shot
after the first, so double-click splits did nothing. A split made by hand now
raises Max shots to keep it, and the Video tab says when detected shots are
left out by the limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 12:40:08 +00:00
alisson-anjosandClaude Opus 5.5 ea9841662a Add BFS Video Compare and a guide output on Shot Join (1.106.0)
BFS Video Compare plays two videos in one player (wipe, overlay or side by
side, any of guide / target / difference on top) and measures a per-pixel
difference map on the real frames, with the difference per frame as a
clickable strip. Shot Join gains a 'guide' output: the original frames under
each output frame (the comparison's original column), so the two connect
straight into the compare node.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-16 12:18:48 +00:00
alisson-anjosandClaude Opus 5.5 738e9c6955 Turn keep_colors off by default in Shot Join (1.105.6)
The colour anchor applies one chroma shift to the whole shot, so with
continuity "head" (whose first frames already continue the previous shot)
it showed as a skin colour jump at every seam. Off by default; turn it on
only for a long take whose colours visibly drift.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-15 22:05:41 +00:00
alisson-anjosandClaude Opus 5.5 06c75bb85e Stop {details} from repeating the prompt words
The VLM is given the prompt sentence around {details} and sometimes answers
with it ('is the ... shown in', '... is the shown in <Picture 1>'), so the
prompt read 'is the is the ... shown in shown in'. The short preset now drops
those words, and every description (also ones already saved in a plan) is
cleaned when it goes into the prompt. Bump BFSNodes version to 1.105.5.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:56:08 +00:00
alisson-anjosandClaude Opus 5.5 982b0e7dcb Embed the workflow in the MP4s the join writes
With save_to = mp4 file (chosen or by the RAM guard) the join writes the
video and comparison itself, without the prompt / workflow tags Save Video
adds, so the files could not be dropped back into ComfyUI. They now carry
the same tags (JSON, use_metadata_tags), passed through an ffmetadata file
so large workflows fit. Bump BFSNodes version to 1.105.4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 19:22:48 +00:00
alisson-anjosandClaude Opus 5.5 77bb00ab74 Fit the crop box to the frames the shot delivers
The box covered every generated frame: the continuity head before the shot
and the frames generated past its end (often another scene after a cut),
both cut by the join. It now holds only the shot's own frames, the same box
the panel's mask preview shows. Bump BFSNodes version to 1.105.3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:48:48 +00:00
alisson-anjosandClaude Opus 5.5 41ef1a549f Advise mp4-file output for long videos in the panel
save_to = images stays the default. The memory status now reports the free
RAM, and the Checks warn when the final video as images (with or without
the comparison) would not fit, advising save_to = mp4 file. Bump BFSNodes
version to 1.105.2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 17:41:59 +00:00
alisson-anjosandClaude Opus 5.5 90be29aa99 Document the filled outputs of the mp4-file join
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:54:21 +00:00
alisson-anjosandClaude Opus 5.5 d01ca239ce Never leave the join's outputs empty in mp4-file mode
With save_to = mp4 file, chosen or forced by the RAM guard, the images and
comparison outputs were blocked, so the result nodes stayed empty and looked
broken. They now carry the video read back from the written files, scaled to
fit the free RAM (the files keep the full size), and the join shows the
files on its node. Bump BFSNodes version to 1.105.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 16:54:06 +00:00
alisson-anjosandClaude Opus 5.5 d88683357d Keep colors: anchor the subject's chroma across a continuous take
Shots that start on the previous result come out more saturated right after
it, and the next shot inherits that (measured: subject saturation 75 -> 124
over 4 joins, flat inside each shot). The first shot of a take records its
subject's CIELAB a/b statistics; every continuing shot is matched to them
before it is stored (one transform per shot, subject only, soft edge), so the
next head starts from the anchored colours. On the reported video the subject
stays at 69-73. New BFS Shot Join options keep_colors / keep_colors_strength.
Bump BFSNodes version to 1.105.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 14:09:46 +00:00
alisson-anjosandClaude Opus 5.5 88216b41cf Guard the final join's RAM and stream the comparison in chunks
- Before an images join, drop earlier cached outputs (ComfyUI's RAM-pressure
  release) and estimate what the video and comparison need; when it does not
  fit, write MP4 files instead of letting the OS kill ComfyUI.
- The comparison is built 16 frames at a time, straight into its output or
  to ffmpeg: file mode with comparison_height 1024 went from +9.9 GB to +2.1 GB.
- The comparison's original frames are stored as shot_NNNN_orig.mkv instead
  of raw inside the .pt (245 MB per shot at full height).
Bump BFSNodes version to 1.104.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 13:10:38 +00:00
alisson-anjosandClaude Opus 5.5 dd86935962 Store shots in the output folder, lossless and cached per shot; low-memory join
- Stored shots move from ComfyUI's temp folder (emptied at every start) to
  output/bfs_shotloop (Shot cache = keep; temp still available).
- Each shot is saved as a lossless FFV1 video next to a small .pt (~12x
  smaller than raw frames, exact); Smaller / Smallest (H.264 RGB) and Raw
  are selectable.
- One run folder per source video; each shot carries a signature of its
  settings, so a Queue runs only the shots that changed, plus the ones that
  continue from a new take. Redo (with or without cascade) from the panel.
- The final join reads one stored shot at a time into the output (measured
  +15.8 GB -> +8.2 GB on 11 shots at 960x544); save to = mp4 file streams
  the video and the comparison to ffmpeg (+2.1 GB). New video_path output.
- Combine stored shots: an MP4 of what is stored now, without generating.
- tools/bfs_assemble_shots.py reads the new format; docs updated.
Bump BFSNodes version to 1.104.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 12:05:16 +00:00
alisson-anjosandClaude Opus 5.5 12c6b8080b Bump BFSNodes version to 1.103.1
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 11:51:38 +00:00
alisson-anjosandClaude Opus 5.5 226cebb0ab Add a low-memory tool to assemble stored shots into an MP4
For a run whose final join did not finish: reads the stored shots one by
one, trims and cross-fades them like BFS Shot Join, and streams the frames
to ffmpeg, so memory stays at about one shot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 11:45:14 +00:00
alisson-anjosandClaude Opus 5.5 deee5ff4f3 Document the memory status; bump BFSNodes version to 1.103.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:59:48 +00:00
alisson-anjosandClaude Opus 5.5 e2784998ca Add a memory status and a between-shots memory option
BFS Shot Join records every shot it receives (latent tokens = generated
frames x resolution, with the continuity head) and the peak VRAM of its
run, per GPU. The panel compares each planned shot with the biggest one
that already ran on this GPU: a status at the top, a Memory card and a
warning in Checks when a shot is much bigger (possible out of memory).
No fixed GB guess, since ComfyUI offloads weights as needed.
New Run option 'Between shots': keep models loaded (default) or free
memory (unload every model between shots, for low VRAM).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:58:19 +00:00
alisson-anjosandClaude Opus 5.5 9f94159c83 Make the shot-by-shot run the default
Tested on real H3 runs: continuity holds and memory stays as in the other
modes. It needs no open browser tab, unlike the queue loop. Saved plans
keep their own run mode. Bump BFSNodes version to 1.102.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:48:43 +00:00
alisson-anjosandClaude Opus 5.5 222414b48d Refresh the loop progress when the plan changes
Every plan change (a new video, a prompt, a setting) is a new loop run on
the server, but the panel only re-read the progress on open or on Status,
so it kept showing the previous video's shots as done.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:47:21 +00:00
alisson-anjosandClaude Opus 5.5 7d2e0160c6 Keep the sound of skipped shots in the join
The planner's audio output covers only the shots that run, so with skipped
shots kept as original video the join padded the end with silence and
shifted the sound after a skipped shot. The planner now marks that track
and the join uses the timeline's soundtrack instead; a soundtrack from
elsewhere is still used as given. Bump BFSNodes version to 1.101.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:45:09 +00:00
alisson-anjosandClaude Opus 5.5 e8533cc92c Keep the queue loop as the default run
The shot-by-shot run stays available as an option until it is tested
under a real H3 load.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 10:04:02 +00:00
alisson-anjosandClaude Opus 5.5 b485d55dd4 Document the shot-by-shot run mode
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 09:50:04 +00:00
alisson-anjosandClaude Opus 5.5 23823f3cd1 Add a recursive shot-by-shot run mode
One Queue renders every shot in order inside the same prompt: after a shot
is stored, BFS Shot Join expands the planner-to-join body (ComfyUI node
expansion) for the next one, so continuity head/history works without the
browser re-queueing. Graph walk adapted from Ethanfel's MiniMax H3
Context Loop. Queue loop and auto loop are unchanged; new plans default to
the recursive run.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 09:47:48 +00:00
alisson-anjosandClaude Opus 5.5 81576e4c99 Default the shot loop to the queue loop
Continuity head/history needs each shot to read the previous result, which
only exists in the queue loop (the auto loop conditions every shot before
sampling any). New plans start in the queue loop with auto-queue on, and
the Run checks warn when a shot asks for head/history in the auto loop.
Bump BFSNodes version to 1.101.0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 09:40:14 +00:00
alisson-anjosandClaude Opus 5.5 aa6a4da0e0 Bump BFSNodes version to 1.100.0
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 09:27:27 +00:00
alisson-anjosandClaude Opus 5.5 18f500ac59 Fix H3 history lengths and add head continuity
H3's VAE encodes 17-frame clips into 5 latent steps and drops the last 3
steps of the whole video, so a 17/34/51-frame history came back as only
its first 5/22/39 frames: a 12-frame hole between the history and the
shot, seen as a jump/drift at every join. History now uses 5/22/39/56
(17k+5); old plans with 17/34/51 read as 22/39/56.

New continuity mode "head" (now the default, 22 frames): the shot is
generated from N frames earlier, the previous shot's result is encoded
into the first latent steps with noise mask 0 (kept, not swapped again),
and BFS Shot Join cuts those frames. Queue loop only; skipped at cuts,
after a skipped shot and off the H3 grid.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-14 09:23:51 +00:00
alisson-anjos 957f3e482c Set crop paste and seamless join defaults 2026-10-14 02:34:11 +00:00
alisson-anjos 2b3071c0e7 Align planner color mark prompt with training 2026-10-13 22:38:36 +00:00
alisson-anjos 22692236f6 Fix Colored Mark points modal syntax 2026-10-13 18:47:04 +00:00
alisson-anjos 40e8779d98 Preserve colored mark overlays in shot comparisons 2026-10-13 18:06:02 +00:00
alisson-anjos 1c99bba23d Add independent colored mark point regions 2026-10-13 17:51:29 +00:00
alisson-anjos ffa0e75362 Add dual-reference prompts and resolution controls 2026-10-13 17:35:52 +00:00
alisson-anjos e13c1c709c Expose Colored Mark settings in shot state 2026-10-13 17:18:56 +00:00
alisson-anjos 16efe73169 Expand second Colored Mark controls when ref two is set 2026-10-13 17:11:34 +00:00
alisson-anjos dca37ddc0f Apply Colored Mark color to mask previews 2026-10-13 17:06:14 +00:00
alisson-anjos 507d5bc60a Always show Colored Mark region color 2026-10-13 16:59:56 +00:00
alisson-anjos 383fed81a1 Refine Colored Mark on updated main 2026-10-13 16:34:09 +00:00
alisson-anjos 2b0caac30a Fix Color Guide reference history and add global colors 2026-10-13 16:28:18 +00:00
alisson-anjos 27090c979a Add Shot Planner language selector and optional tour 2026-10-13 16:27:56 +00:00
alisson-anjos 3e2d112b03 Add optional Color Guide to Shot Planner 2026-10-13 16:27:29 +00:00
alisson-anjos 5f6e3c8446 Add five-frame shot history continuity 2026-10-13 12:38:58 +00:00
alisson-anjos 72af0aa0c7 Move shot filters to People Filters Masks tab 2026-10-13 07:33:51 +00:00
alisson-anjos d1ae8bee0a Bump BFSNodes version to 1.94.4 2026-10-13 07:26:11 +00:00
alisson-anjos 391115b78a Remove duplicate total duration control 2026-10-13 07:23:12 +00:00
alisson-anjos a55054aa0c Move max shots control to Video tab 2026-10-13 07:20:45 +00:00
alisson-anjos 154698f46f Re-split shots when cut settings change 2026-10-13 07:19:54 +00:00
alisson-anjos f1f639f389 Fix Shot Planner splitting and TextGenerate compatibility 2026-10-13 07:12:26 +00:00
alisson-anjos ceeff4803d Add simplified H3 shot conditioning 2026-10-12 13:13:43 +00:00
alisson-anjos 8aeb958ab6 Planner: stabilize VLM descriptions and default videos to 15s 2026-10-12 10:27:36 +00:00
alisson-anjosandClaude Opus 5.5 29cedad7be Shot loop: one-shot split mode, BFS Shot Seed, BFS Shot Noise, BFS Sigmas Preview (1.94.0)
- Planner: Mode "One shot (whole range)" (the range as a single shot, Max seconds / shot not applied) and
  "Range length, seconds" next to it, so a whole shot or pieces of at most X are set in the Video tab alone.
- BFS Shot Seed: one seed per shot from a base seed (same / increment / random per shot) with per-shot overrides.
- BFS Shot Noise (experimental, optional): starting noise for SamplerCustomAdvanced on H3's video + audio latent;
  every amount at 0 is exactly RandomNoise. shared_base (the coarse pattern from a base seed, the same across shots,
  resolutions and lengths), static_share, distribution, spectrum_tilt, pyramid, guide_structure, ref_tint.
- BFS Sigmas Preview: chart of a sigma schedule (steps on layout / shapes / detail), pass-through.

Tested: split logic, seed logic, noise statistics (unit variance, identity at 0) and the chart, in Python.
Not tested yet inside ComfyUI / the panel in a browser.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 23:47:53 +00:00
alisson-anjosandClaude Opus 5.5 187c9a1776 Find people / Cast without insightface: OpenCV Zoo YuNet + SFace (Apache-2.0, run by OpenCV, downloaded once into models/opencv_face); InsightFace only when installed; SAM 3.1 + colours as last resort
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 19:00:36 +00:00
alisson-anjosandClaude Opus 5.5 a04f3116cf {details} filled automatically by the render nodes too: the planner leaves it when it has no VLM, BFS Shot H3 Conditioning / Shot H3 Duet (new vlm input) describe the shot's references
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 12:48:30 +00:00
alisson-anjosandClaude Opus 5.5 84a953e8c5 source_blur (SAM 3.1, keeps the mouth): blur the source person in the guide / native ref video / duet panel so the model reads pose, motion and lips but has no face to copy - Shot Conditioning, Shot Duet, Duet, Duet Conditioning
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 12:10:27 +00:00
alisson-anjosandClaude Opus 5.5 0bf03a8fb7 Planner: chunked upload for big files (8 MB pieces to /bfs/upload/chunk, assembled on disk) - any size, past ComfyUI's 100 MB --max-upload-size
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 09:03:16 +00:00
alisson-anjosandClaude Opus 5.5 390f9e8714 Planner UI: ✕ clear all - removes every reference from the plan (global, shots, cast) and the recent list, after a confirm
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 07:50:53 +00:00
alisson-anjosandClaude Opus 5.5 515ab73c70 Planner UI: collapsible sections keep what the user clicked (re-renders snapped them shut); ✕ clear for the recent reference pictures
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 07:47:46 +00:00
alisson-anjosandClaude Opus 5.5 2e64d989f6 BFS Shot H3 Duet: continuity 'history' too (the previous end before frame 0, panel strip filled with the source under it); shared shot_history() pops it from the shot
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 02:17:41 +00:00
alisson-anjosandClaude Opus 5.5 b1e0fcbbb6 Shot Planner: continuity 'history' is the default for new plans and unset shots (cuts still skip it)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 02:14:20 +00:00
alisson-anjosandClaude Opus 5.5 0b5315c45a History / chain: load the previous result on the CPU, slice uint8 before converting, keep history as uint8 in the shot and drop it after conditioning (no idle RAM/VRAM)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 01:39:46 +00:00
alisson-anjosandClaude Opus 5.5 a03e0be342 Shot Planner: continuity 'history' - the last 17/34/51 frames of the previous result placed just before frame 0 on H3's timeline (sliding history, after Akatz's H3 Relay / Ethanfel's Context Loop), with every mask mode, crop and the duet canvas; cuts reset it
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-09 01:37:36 +00:00
alisson-anjosandClaude Opus 5.5 78aa214584 Shot Planner: {details} described automatically before any SAM 3 work (it ran after the masks, which breaks the VLM: the button worked, the run did not); never leaves a hole in the sentence
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 23:03:47 +00:00
alisson-anjosandClaude Opus 5.5 05184747b2 Duet VLM prompt: place and look as lower-case noun phrases inside the sentences
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 21:32:31 +00:00
alisson-anjosandClaude Opus 5.5 596a1c2538 Duet prompts: TSC's newer strategy - the finished split screen as plain footage (no kept footage / generated / in sync), the pinned region named by place + one clause of what is seen ({panel_region}/{video_region}: top quarter, bottom three quarters...), motion beat by beat with times on the video clock, sounds of the new scene
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 21:18:07 +00:00
alisson-anjosandClaude Opus 5.5 4e4fc20fdd Duet: optional VLM prompt - writes it from the task, or fills the {...} fields of a hand-written template (TSC's); setting picture described as <Subject 2>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 21:08:35 +00:00
alisson-anjosandClaude Opus 5.5 458099a2aa Duet: size refs for the whole canvas (as TSC's Reference to Video at canvas size)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 21:01:00 +00:00
alisson-anjosandClaude Opus 5.5 ad99a9f91a Duet: panel_static (TV static over face/person, SAM 3) + setting picture input + BFS Setting Picture node
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 20:52:18 +00:00
alisson-anjosandClaude Opus 5.5 f763b51779 BFS H3 Duet: side_by_side output - the same comparison as BFS Shot Join (original | references | result, info on top, prompt below) (1.83.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 20:21:36 +00:00
alisson-anjosandClaude Opus 5.5 2b462b639a BFS H3 Duet: megapixels input removed again (the size is computed in the workflow instead) (1.82.1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 19:58:56 +00:00
alisson-anjosandClaude Opus 5.5 864a28b161 BFS H3 Duet / Duet Conditioning: megapixels - output size from the clip's aspect ratio at that area (0 = width / height) (1.82.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 19:54:02 +00:00
alisson-anjosandClaude Opus 5.5 e43cc14d9a Points: Track -> whole video - one selection tracked by SAM 3 through the whole video or range (chunked, forward and backward, re-found by text after cuts); shots without their own selection take their part (1.81.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 19:47:21 +00:00
alisson-anjosandClaude Opus 5.5 4fab22caa9 Shot Planner range: a player in the Video tab, start / end in frames (start here / end here take the player's exact frame; seconds of older plans still read), aligned layout (1.80.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 19:09:13 +00:00
alisson-anjosandClaude Opus 5.5 96f7e8a1b1 VLM descriptions: temperature and seed (0 = greedy as before), a dice button to try another description, the short preset capped at 30 words (1.79.1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 18:02:53 +00:00
alisson-anjosandClaude Opus 5.5 34ffcef172 Shot Planner: Start at / End at range - only that part of the video is split, analysed, generated and joined; timeline dims the rest; max total seconds counts from the start (1.79.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 17:42:42 +00:00
alisson-anjosandClaude Opus 5.5 9ae5fc9614 Fit the prompt: only for the short preset, the VLM is told to reply with the blank's words only, and an echo of the prompt sentence is cut; Checks warn when {details} is used with a paragraph preset (1.78.2)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 15:14:58 +00:00
alisson-anjosandClaude Opus 5.5 7d5f60ce86 insightface and onnxruntime are optional: no pin in the requirements (the exact insightface pin and its own requirements conflicted with many environments); Find people / LTX Identity say what to install (1.78.1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:50:54 +00:00
alisson-anjosandClaude Opus 5.5 8e573275b8 Fit the prompt: descriptions ({details}, {target}) are written for the prompt sentence that holds them (blank in place, article matched); {target} is described automatically at run time for shots with a selection and no text (1.78.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 13:34:08 +00:00
alisson-anjosandClaude Opus 5.5 a5535dc77c Describe refs for any subject (person, creature, robot, animal, cartoon: says what it is first); new default preset 'short' - a noun phrase that fits '<Subject 1> is the {details} shown in <Picture 1>'; {target} describes animals/creatures too (1.77.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-08 11:35:11 +00:00
alisson-anjosandClaude Opus 5.5 7af3fa5618 pose look: clearer error (package vs model, with the download link) and no restart needed after adding the model (1.76.1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 16:07:09 +00:00
alisson-anjosandClaude Opus 5.5 39610ab1ef mask_guide_look: what the mask guide shows of the subject - grey blurred (default), colour, silhouette, edges, or pose (people, YOLO pose model from models/ultralytics, no new packages) (1.76.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 15:47:14 +00:00
alisson-anjosandClaude Opus 5.5 aa92d72990 Faster masks: separable dilation / blur (same result, ~7x faster on few CPU threads) and the generation mask computed once per shot (1.75.1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 15:35:14 +00:00
alisson-anjosandClaude Opus 5.5 f1388f0463 mask_guide (renamed from person_guide, works for any subject): masked region as an extra guide, instead of the full guide, or as a native reference video (<Video n>, with mask_ref_size); Mask only shots only; Frame + paste takes a per-shot 'new subject' text for the new outline (1.75.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:51:56 +00:00
alisson-anjosandClaude Opus 5.5 5c93d23c88 H3 Conditioning: experimental person_guide - a guide latent with only the masked person (rest grey), added next to the full guide or instead of it (1.74.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:45:55 +00:00
alisson-anjosandClaude Opus 5.5 7d9d7a9e26 Frame + paste mode: the whole frame is generated (pose follows the guide, the new person may be bigger) and BFS Shot Join pastes only the person (shot mask + the new outline found by SAM 3) onto the original frame; mask strength renamed mask opacity (1.73.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:22:16 +00:00
alisson-anjosandClaude Opus 5.5 1de7753be9 Mask strength per shot (mask modes): the masked rows start at strength x the noise level and keep part of the original, like an opacity (1.72.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:09:17 +00:00
alisson-anjosandClaude Opus 5.5 aa545097b7 External masks instead of SAM 3: a mask video per shot, the planner's optional mask input (whole video) or a mask video for the whole video in the panel; used by every mode, {target}, the setting picture and the comparison (1.71.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 14:01:13 +00:00
alisson-anjosandClaude Opus 5.5 1b04a822b3 Console logs for masks: H3 Conditioning prints each shot's mode, where its mask comes from, coverage (and empty frames) and how much of the latent is regenerated; SAM 3 runs print what, how many frames and how long (1.70.1)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 13:45:07 +00:00
alisson-anjosandClaude Opus 5.5 61a9e732f2 Shot Planner refactor: tabbed panel, per-shot generation modes, Copy to other shots, checks (1.70.0)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-07 11:54:22 +00:00
34 changed files with 9582 additions and 1068 deletions
+9 -2
View File
@@ -46,6 +46,11 @@ cd ComfyUI-BFSNodes
pip install -r requirements.txt
```
Optional: `pip install insightface` with `onnxruntime` or `onnxruntime-gpu`, only for LTX Identity. The Shot Planner's
*Find people* / Cast no longer needs it (1.93.0): it uses OpenCV's YuNet + SFace (Apache-2.0, downloaded once), and uses
InsightFace only when it is installed. They are not in the requirements because insightface pins packages
that conflict with many environments; without them everything else works and those features say what to install.
Restart ComfyUI after installation.
## LoRA Placement
@@ -135,7 +140,8 @@ workflow once per shot, and join the results back with the original timing and s
- **BFS Shot Unpack** — guide frames, references, prompt and length of one shot, for any model.
- **BFS Shot H3 Conditioning** — MiniMax H3 conditioning for one shot with the native nodes
(aligned guide and/or native reference video).
- **BFS Shot Join** — concatenates, trims to the true lengths, cross-fades soft joins, returns audio.
- **BFS Shot Join** — concatenates, trims to the true lengths, cross-fades soft joins, returns audio. With `comparison` on, its `guide` output is the original video frame for frame.
- **BFS Video Compare** — one player for two videos (Join's `guide` and `images`): wipe ↔ / ↕, overlay or side by side, any of guide / target / difference on top, plus a per-pixel difference map and the difference per frame (click to jump).
**Cast (1.49.0)**: *Find people* tracks faces through the video (InsightFace) and groups them
into people. Link each person to a reference and every shot uses the reference of the person
@@ -156,7 +162,8 @@ Training-free split-screen generation for MiniMax H3 (TSC's latent pin): a panel
or a reference) is pinned beside the video with a noise mask, the video is generated in sync with
it, and the panel is cropped off before decoding. **BFS H3 Duet** does it all in one node, **BFS Shot H3 Duet** renders each shot of the Shot Loop that way (or with an aligned guide for body-swap LoRAs);
**BFS H3 Side Panel** + **BFS H3 Side Panel Crop** are the building blocks. Optional aligned latent
guide. See [`web/js/docs/BFSH3Duet.md`](web/js/docs/BFSH3Duet.md).
guide. 1.84.0: `panel_static` (TV static over the face / person of the pinned clip, SAM 3) and a `setting`
picture (**BFS Setting Picture**) for same-world swaps. See [`web/js/docs/BFSH3Duet.md`](web/js/docs/BFSH3Duet.md).
## Requirements
+16
View File
@@ -86,6 +86,12 @@ try:
except Exception as _e: # noqa
print(f"[BFSNodes] Shot loop nodes not loaded: {_e!r}")
SHOT_NODE_CLASS_MAPPINGS, SHOT_NODE_DISPLAY_NAME_MAPPINGS = {}, {}
try:
from .bfs_shot_noise import NODE_CLASS_MAPPINGS as SNOISE_NODE_CLASS_MAPPINGS
from .bfs_shot_noise import NODE_DISPLAY_NAME_MAPPINGS as SNOISE_NODE_DISPLAY_NAME_MAPPINGS
except Exception as _e: # noqa
print(f"[BFSNodes] BFS Shot Noise node not loaded: {_e!r}")
SNOISE_NODE_CLASS_MAPPINGS, SNOISE_NODE_DISPLAY_NAME_MAPPINGS = {}, {}
try:
from .minimax_h3_downscaled_guide import NODE_CLASS_MAPPINGS as H3GUIDE_NODE_CLASS_MAPPINGS
from .minimax_h3_downscaled_guide import NODE_DISPLAY_NAME_MAPPINGS as H3GUIDE_NODE_DISPLAY_NAME_MAPPINGS
@@ -106,6 +112,12 @@ try:
except Exception as _e: # noqa
print(f"[BFSNodes] LoRA Surgery node not loaded: {_e!r}")
LSURG_NODE_CLASS_MAPPINGS, LSURG_NODE_DISPLAY_NAME_MAPPINGS = {}, {}
try:
from .bfs_video_compare import NODE_CLASS_MAPPINGS as VCMP_NODE_CLASS_MAPPINGS
from .bfs_video_compare import NODE_DISPLAY_NAME_MAPPINGS as VCMP_NODE_DISPLAY_NAME_MAPPINGS
except Exception as _e: # noqa
print(f"[BFSNodes] Video Compare node not loaded: {_e!r}")
VCMP_NODE_CLASS_MAPPINGS, VCMP_NODE_DISPLAY_NAME_MAPPINGS = {}, {}
NODE_CLASS_MAPPINGS = {
**H3GUIDE_NODE_CLASS_MAPPINGS,
@@ -127,11 +139,13 @@ NODE_CLASS_MAPPINGS = {
**MSP_NODE_CLASS_MAPPINGS,
**MMVAE_NODE_CLASS_MAPPINGS,
**SHARP_NODE_CLASS_MAPPINGS,
**SNOISE_NODE_CLASS_MAPPINGS,
**MMDD_NODE_CLASS_MAPPINGS,
**SVH3_NODE_CLASS_MAPPINGS,
**EPF_NODE_CLASS_MAPPINGS,
**HSMS_NODE_CLASS_MAPPINGS,
**LSURG_NODE_CLASS_MAPPINGS,
**VCMP_NODE_CLASS_MAPPINGS,
}
NODE_DISPLAY_NAME_MAPPINGS = {
@@ -154,11 +168,13 @@ NODE_DISPLAY_NAME_MAPPINGS = {
**MSP_NODE_DISPLAY_NAME_MAPPINGS,
**MMVAE_NODE_DISPLAY_NAME_MAPPINGS,
**SHARP_NODE_DISPLAY_NAME_MAPPINGS,
**SNOISE_NODE_DISPLAY_NAME_MAPPINGS,
**MMDD_NODE_DISPLAY_NAME_MAPPINGS,
**SVH3_NODE_DISPLAY_NAME_MAPPINGS,
**EPF_NODE_DISPLAY_NAME_MAPPINGS,
**HSMS_NODE_DISPLAY_NAME_MAPPINGS,
**LSURG_NODE_DISPLAY_NAME_MAPPINGS,
**VCMP_NODE_DISPLAY_NAME_MAPPINGS,
}
WEB_DIRECTORY = "./web/js"
+163
View File
@@ -0,0 +1,163 @@
"""Sliding history for MiniMax H3: the end of the previous shot's result as clean frames placed just BEFORE the video
on H3's timeline, so a long continuous take is generated window by window with motion (and look) carried across.
Idea from Akatz's H3 Relay windowed edit and Ethanfel's ComfyUI-MiniMaxH3-Context-Loop (both GPL-3.0); this is an
independent implementation. A guide keyframe marked ``anchor = "history"`` is packed like any H3 guide, then its rows
are kept at the target's origin and everything else on the target timeline (the video, its audio, the other guides)
moves forward by the history's span. Text and reference rows are untouched.
The layout is patched in memory only (ComfyUI files are never edited), once, and only changes layouts that carry a
history keyframe; when ComfyUI's own H3 layout already understands history anchors the patch stands down.
"""
from __future__ import annotations
import logging
import torch
LOG = logging.getLogger("bfs.h3_history")
# H3's VAE encodes 17-frame clips into 5 tokens (1+4+4+4+4 frames) and drops the last 3 tokens of the whole video, so only
# 17k+5 frames encode without loss: 17 frames come back as 2 tokens = the FIRST 5 frames, leaving a 12-frame hole between
# the history and the shot it sits before
HISTORY_FRAMES = (5, 22, 39, 56)
def history_length(wanted: int, available: int) -> int:
"""The longest valid history (17k+5 frames) up to `wanted` and `available`; 0 when not even 5 fit. A wanted value
off the grid (old plans saved 17/34/51) is read as the next valid one up (22/39/56)."""
wanted, available = max(0, int(wanted)), max(0, int(available))
if wanted < 5:
return 0
wanted = 5 + -(-(wanted - 5) // 17) * 17
n = min(wanted, available)
return 0 if n < 5 else 5 + ((n - 5) // 17) * 17
_PATCH = "_bfs_history_patch"
def history_span(latent_t: int) -> float:
"""Timeline span of `latent_t` video latent steps (FRAME_RESCALE per pixel frame)."""
import comfy.ldm.minimax.model as mm
return float(sum(mm._video_t_spans(int(latent_t))))
def relocate(layout, keyframes) -> bool:
"""Moves the target timeline past the history keyframes. Returns whether anything changed."""
hist = [i for i, kf in enumerate(keyframes or ()) if kf.get("anchor") == "history"]
if not hist:
return False
# keyframe -> its packed segments: every guide with a video latent owns the next "cond" segment, every guide
# with audio the next "cond_audio" one, in keyframe order (the order PackedLayout packs them)
cond = [s for s in layout.segments if s[2] == "cond"]
cond_a = [s for s in layout.segments if s[2] == "cond_audio"]
own, ci, ai = {}, 0, 0
for i, kf in enumerate(keyframes):
segs = []
if kf.get("latent") is not None and ci < len(cond):
segs.append(cond[ci]); ci += 1
if kf.get("audio_latent") is not None and ai < len(cond_a):
segs.append(cond_a[ai]); ai += 1
own[i] = segs
span = sum(history_span(keyframes[i]["latent"].shape[2]) for i in hist if keyframes[i].get("latent") is not None)
if span <= 0:
return False
keep = torch.zeros(layout.position_ids.shape[0], dtype=torch.bool)
for i in hist:
for a, b, _ in own[i]:
keep[a:b] = True
vid = next(((a, b) for a, b, k in layout.segments if k == "video"), None)
if vid is not None and keep.any():
# another history implementation (e.g. H3 Relay's, or a future ComfyUI) already moved the target: skip
if float(layout.position_ids[vid[0]:vid[1], 0].min()) - float(layout.position_ids[keep, 0].min()) >= span - 1e-6:
return False
move = torch.zeros_like(keep)
for a, b, kind in layout.segments:
if kind in ("video", "audio", "cond", "cond_audio"):
move[a:b] = True
move &= ~keep
pos = layout.position_ids.clone()
pos[move, 0] += span
layout.position_ids = pos
LOG.info("H3 history: %d rows before the video, target timeline moved by %.2f", int(keep.sum()), span)
return True
def install() -> bool:
"""Wraps PackedLayout.__init__ once (process-local). False when ComfyUI is missing or already supports it."""
try:
import comfy.ldm.minimax.model as mm
except ImportError:
return False
cls = mm.PackedLayout
if getattr(cls.__init__, _PATCH, False):
return True
import inspect
if "history" in inspect.getsource(cls): # native history anchors: nothing to do
return False
orig = cls.__init__
def __init__(self, text_len, latent_t, latent_h, latent_w, audio_t, keyframes=None, refs=None):
orig(self, text_len, latent_t, latent_h, latent_w, audio_t, keyframes=keyframes, refs=refs)
relocate(self, keyframes)
setattr(__init__, _PATCH, True)
cls.__init__ = __init__
return True
def add_history(positive, vae, frames: torch.Tensor):
"""Appends the last 17k+5 frames of `frames` ([N,H,W,3]) from the previous shot as an H3 history keyframe.
Returns (positive, frames used)."""
import node_helpers
n = history_length(frames.shape[0], frames.shape[0])
if n < 5:
return positive, 0
install()
# encoded from CPU frames; the latent comes back on ComfyUI's intermediate device like any H3 guide, and the VAE
# is loaded / freed by ComfyUI's model management (nothing here keeps GPU memory)
with torch.no_grad():
lat = vae.encode(frames[-n:, ..., :3].float().cpu())
kfs = list(positive[0][1].get("minimax_keyframes", []))
kfs.append({"resolved_frame_index": 0, "latent": lat, "anchor": "history"})
return node_helpers.conditioning_set_values(positive, {"minimax_keyframes": kfs}), n
def pin_head(latent: dict, vae, frames: torch.Tensor):
"""Continuity 'head': the end of the previous shot's result (17k+5 frames, [N,H,W,3]) encoded over the FIRST latent
steps of this shot's video, with the sampler's noise mask at 0 there, so those frames are kept exactly as they were
(no new swap on them) and the rest of the shot is generated continuing from them. BFS Shot Join cuts them off again.
Same masked-prefix idea as Ethanfel's Context Loop 'masked AV' continuation. Returns (latent, frames pinned)."""
import comfy.nested_tensor
n = history_length(frames.shape[0], frames.shape[0])
samples = latent["samples"]
nested = getattr(samples, "is_nested", False)
video = samples.tensors[0] if nested else samples
T, h, w = int(video.shape[2]), int(video.shape[3]), int(video.shape[4])
if n < 5:
return latent, 0
px = frames[-n:, ..., :3].float().cpu()
if tuple(px.shape[1:3]) != (h * 16, w * 16):
px = torch.nn.functional.interpolate(px.movedim(-1, 1), size=(h * 16, w * 16), mode="bilinear",
align_corners=False).movedim(1, -1)
with torch.no_grad():
enc = vae.encode(px).to(video)
t = int(enc.shape[2])
if t >= T:
LOG.warning("H3 head: %d pinned latent steps leave nothing to generate in a %d-step video", t, T)
return latent, 0
video = video.clone()
video[:, :, :t] = enc
old = latent.get("noise_mask")
vm = old.tensors[0] if getattr(old, "is_nested", False) else old
vm = torch.ones((1, 1, T, h, w), dtype=torch.float32) if vm is None else vm.float().clone()
vm = torch.broadcast_to(vm, (vm.shape[0], vm.shape[1], T, h, w)).clone()
vm[:, :, :t] = 0.0
out = dict(latent)
if nested:
audio = samples.tensors[1]
am = old.tensors[1] if getattr(old, "is_nested", False) else torch.ones_like(audio)
out["samples"] = comfy.nested_tensor.NestedTensor((video, audio))
out["noise_mask"] = comfy.nested_tensor.NestedTensor((vm.to(video.device), am))
else:
out["samples"], out["noise_mask"] = video, vm.to(video.device)
LOG.info("H3 head: first %d frames (%d latent steps) pinned from the previous shot", n, t)
return out, n
+359 -101
View File
@@ -26,30 +26,31 @@ PATCH_PX = 32 # one DiT patch (2x2 latent cells of 16 px)
FRAME_PER_TOKEN = (1, 4, 4, 4, 4) # H3 video latent: frames per latent step, k % 5
PANEL_PROMPT_HINT = (
"How to refer to the side panel in the prompt: the panel has NO tag (it is not <Picture n> or <Video n>; "
"the text encoder never sees it). Name it by its place: 'the LEFT half is the kept footage' (write {layout} "
"to insert that sentence for the current size and side). Describe only the generated part: never describe "
"the panel's performer, clothes or room, even to contrast them; what you leave undescribed is copied from "
"the panel. Restate the new identity in every shot ('her face from <Picture 1>' plus two or three face, hair "
"or outfit words). Give exact times for cuts ('[Shot 2] At 00:03.708, both halves cut together to ...').")
"How to write a duet prompt (TSC, newer strategy): H3 has no idea what is pinned and what is noised, it reads one "
"description of one video. Describe the finished split screen as plain footage: never write 'kept footage', 'kept "
"exactly', 'generated', 'preserved', 'in sync with', 'seamlessly'. The pinned clip gets no tag and no retention "
"line: name its region by where it is plus what is seen there, in one clause ('the top quarter shows a man dancing "
"in a small bedroom'; {panel_region} / {video_region} give the region names for the current layout, {layout} the "
"whole split-screen sentence). Write the other region as its own scene in a real place with concrete details. "
"Spell out the motion beat by beat, with times for the big moments (cuts, strikes, jumps, a line), on the video's "
"clock. The soundscape is the sounds of the new scene. Cuts: '[Shot 2] At MM:SS.mmm, both halves cut together to ...'.")
DUET_PROMPT_TEMPLATE = """subject_definitions:
<Subject 1> is the woman whose appearance comes from <Picture 1>: {skin, face shape, eyes, brows, nose, lips}, {hair colour, length and cut}, wearing {the outfit, piece by piece}.
<Subject 1> is {name or plain description} whose appearance comes from <Picture 1>: {concrete look words}, wearing {outfit piece by piece}.
summary:
[reference generation] The target video is a split screen: the kept footage beside <Subject 1>, who moves in sync with it, in the same room.
[reference generation] The target video is a split screen: {panel_region} shows {what is seen in the source, one clause}, and {video_region} shows <Subject 1> doing the same movements at the same moments, in {place}.
retention_analysis:
<Subject 1> (appears in [Shot 1]): fully_preserved - {her key look words} are retained.
The kept footage: fully_preserved - the panel is kept exactly.
<Subject 1> (appears in [Shot 1]): fully_preserved - {key look words} are retained.
detailed_description:
The target video is in a realistic style, as {the medium, e.g. handheld vertical smartphone footage under soft daylight}. {layout}
The target video is in a {realistic / 3D CG / anime} style, as {the medium of the source clip}. A split screen divided by a thin straight {vertical / horizontal} line: {panel_region} shows {what is seen there, one clause}; {video_region} shows <Subject 1> performing the same movements at the same moments, in {place with concrete details}.
[Shot 1] {Camera distance, angle and movement.} <Subject 1>, her face from <Picture 1> with {two face words}, {two hair or outfit words}, {what she does, one sentence per real action}.
[Shot 1] {Camera distance, angle and movement as the source shows it}. <Subject 1>, the face from <Picture 1>, {what they do, one sentence per real action}. At {00:01.500} {big beat}.
overall_soundscape:
{The sounds of the scene and when they happen.}
{The sounds of the new scene and when they happen.}
non_diegetic_music:
None."""
@@ -63,27 +64,26 @@ CREDIT = ("Credits: the initial idea for this H3 node came from TSC's latent-pin
"New here: the shifted RoPE layout (the video keeps its own grid and the panel sits past its edge, with an "
"optional gap), dynamic references, task prompts and the shot-loop node.")
PROMPT_EXAMPLE = """Example (character swap, source clip pinned on the left, one face picture and one full-body picture):
PROMPT_EXAMPLE = """Example (character swap, source clip pinned on the left half, one face picture and one full-body picture):
subject_definitions:
<Subject 1> is the woman whose appearance comes from <Picture 1> and <Picture 2>: fair skin, a narrow oval face, grey-green eyes, light brown hair in a low ponytail, wearing a white long-sleeved top under a navy denim apron.
<Subject 1> is a young woman whose appearance comes from <Picture 1> and <Picture 2>: fair skin, a narrow oval face, grey-green eyes, light brown hair in a low ponytail, wearing a white long-sleeved top under a navy denim apron.
summary:
[reference generation] The target video is a split screen: the kept footage beside <Subject 1>, who moves in sync with it, in the same bedroom.
[reference generation] The target video is a split screen: the left half shows a man talking to a phone camera in a bedroom, and the right half shows <Subject 1> doing the same movements at the same moments, in a sunny kitchen with white tiles.
retention_analysis:
<Subject 1> (appears in [Shot 1], [Shot 2]): fully_preserved - her face, ponytail, white top and navy apron are retained.
The kept footage: fully_preserved - the panel is kept exactly.
detailed_description:
The target video is in a realistic style, as handheld vertical smartphone footage under soft daylight. {layout}
The target video is in a realistic style, as handheld vertical smartphone footage under soft daylight. A split screen divided by a thin straight vertical line: the left half shows a man talking to a phone camera in a bedroom; the right half shows <Subject 1> performing the same movements at the same moments, in a sunny kitchen with white tiles and a window behind her.
[Shot 1] A medium close-up at chest height, the phone steady at eye level. <Subject 1>, her face from <Picture 1> with grey-green eyes and soft pink lips, her light brown ponytail and navy apron, talks to the camera with small nods.
[Shot 1] A medium close-up at chest height, the phone steady at eye level. <Subject 1>, her face from <Picture 1> with grey-green eyes and soft pink lips, talks to the camera with small nods. At 00:01.800 she laughs and tilts her head back.
[Shot 2] At 00:03.708, both halves cut together to a closer framing. <Subject 1>, her face from <Picture 1>, the white sleeves and apron straps visible, raises one open hand beside her mouth and smiles.
overall_soundscape:
Her voice speaking to the camera in a quiet bedroom.
Her voice speaking to the camera, a kettle starting to hiss in the kitchen at 00:02.500.
non_diegetic_music:
None."""
@@ -205,30 +205,66 @@ def panel_mask(info: dict, latent_t: int, hold: str, target_mask: torch.Tensor |
return m
def layout_text(info: dict, rope_mode: str = "canvas") -> str:
"""How the kept region is named in a prompt (TSC's wording: 42-58% of the canvas is 'the LEFT half').
shifted RoPE: the video sits on its own grid, so a split-screen sentence would make the model split the video
itself; the sentence then only says that the whole frame follows the kept footage."""
if rope_mode == "shifted":
return "The whole frame shows the generated video, moving in sync with the kept footage frame by frame."
_SHARES = ((1 / 2, "half"), (1 / 3, "third"), (2 / 3, "two thirds"), (1 / 4, "quarter"), (3 / 4, "three quarters"),
(1 / 5, "fifth"), (4 / 5, "four fifths"))
def _share_name(side: str, share: float) -> str:
for v, w in _SHARES:
if abs(share - v) <= 0.035:
return f"the {side} {w}"
return f"the {side} {round(share * 100)}% of the frame"
def region_names(info: dict) -> tuple[str, str, str]:
"""(pinned region, generated region, divider) as plain words: ('the top quarter', 'the bottom three quarters',
'horizontal'). TSC: H3 does not know what is pinned, so the regions are named by where they are."""
pos = info["position"]
side = {"left": "LEFT", "right": "RIGHT", "top": "TOP", "bottom": "BOTTOM"}[pos]
other = {"left": "RIGHT", "right": "LEFT", "top": "BOTTOM", "bottom": "TOP"}[pos]
other = {"left": "right", "right": "left", "top": "bottom", "bottom": "top"}[pos]
horizontal = pos in ("left", "right")
strip = info["strip_w"] if horizontal else info["strip_h"]
share = strip / max(1, strip + (info["w"] if horizontal else info["h"]))
return _share_name(pos, share), _share_name(other, 1 - share), "vertical" if horizontal else "horizontal"
def _verb(region: str) -> str:
return "show" if region.endswith(("thirds", "quarters", "fifths")) else "shows"
def layout_text(info: dict, rope_mode: str = "canvas", seen: str = "", who: str = "", place: str = "") -> str:
"""The split-screen sentence of detailed_description, in TSC's newer wording: the finished frame described as plain
footage, each region by where it is and what is seen there (no 'kept footage', 'generated', 'in sync').
shifted RoPE: the video sits on its own grid, so there is no split screen to describe."""
if rope_mode == "shifted":
return ""
pin, gen, line = region_names(info)
seen = seen or "the original performance"
horizontal = info["position"] in ("left", "right")
strip = info["strip_w"] if horizontal else info["strip_h"]
total = strip + (info["w"] if horizontal else info["h"])
share = strip / total
word = "half" if 0.42 <= share <= 0.58 else None
kept = f"the {side} half" if word else f"the {side} {round(share * 100)}% of the frame (a narrow strip)"
gen = f"the {other} half" if word else f"the {other} {round((1 - share) * 100)}% of the frame"
line = "vertical" if horizontal else "horizontal"
text = (f"A split screen divided by a thin straight {line} line: {kept} is the kept footage; "
f"{gen} is generated and moves in sync with it.")
if (1 - share) > 1.3 * share:
text += (f" The generated area is LARGER than the kept panel ({round((1 - share) * 100)}% of the frame against "
f"{round(share * 100)}%): restage the scene at that larger size, the same shots, framing proportions and "
"timing, a bigger picture, not a pixel-for-pixel mirror.")
return text
wider = ", a wider view of the same action" if (total - strip) > 1.3 * strip else ""
doing = f"{who} performing the same movements" if who else "the same movements"
return (f"A split screen divided by a thin straight {line} line: {pin} {_verb(pin)} {seen}; {gen} {_verb(gen)} "
f"{doing} at the same moments{', in ' + place if place else ''}{wider}.")
def fill_layout(text: str, info: dict | None, rope_mode: str = "canvas") -> str:
"""{layout}, {panel_region}, {video_region}, {split_line} and {wider} of a prompt, from the canvas layout."""
if info is None or rope_mode == "shifted":
for k in ("{layout}", "{panel_region}", "{split_line}", "{wider}"):
text = text.replace(k, "")
return text.replace("{video_region} shows", "the frame shows").replace("{video_region}", "the frame").replace(" ", " ")
pin, gen, line = region_names(info)
horizontal = info["position"] in ("left", "right")
strip = info["strip_w"] if horizontal else info["strip_h"]
total = strip + (info["w"] if horizontal else info["h"])
wider = ", a wider view of the same action" if (total - strip) > 1.3 * strip else ""
who = "<Subject 1>" if "<Subject 1>" in text else ""
text = text.replace("{layout}", layout_text(info, rope_mode, who=who))
text = text.replace("{split_line}", line).replace("{wider}", wider)
for k, v in (("{panel_region}", pin), ("{video_region}", gen)):
text = text.replace(k + " shows", f"{v} {_verb(v)}").replace(k, v)
return text.replace(" ", " ")
ROPE_MODES = ["canvas", "shifted"]
@@ -455,56 +491,59 @@ class BFSH3SidePanelCrop:
def build_prompt(task: str, instruction: str, n_pictures: int, n_videos: int, rope_mode: str = "canvas") -> str:
"""A six-section REF2VA prompt for a duet task. `{layout}` stays in it and is filled at render time.
shifted: no split-screen wording (it makes the model split the video itself)."""
"""A six-section REF2VA duet prompt without a VLM, in TSC's newer wording: the finished split screen described as
plain footage ({panel_region} / {video_region} / {layout} are filled from the canvas at render time). Without a VLM
the motion cannot be spelled out: write it into `instruction`, or connect a VLM."""
inst = instruction.strip().rstrip(".")
pics = [f"<Picture {i}>" for i in range(1, n_pictures + 1)]
pics_txt = " and ".join(pics) if len(pics) <= 2 else ", ".join(pics[:-1]) + " and " + pics[-1]
seen = f" as shown in {pics_txt}" if pics else ""
sync = "every movement, gesture and expression in sync with the kept footage"
split = rope_mode != "shifted"
if task == "character swap":
look = f"whose appearance comes from {pics_txt}" if pics else "the new person"
defs = f"<Subject 1> is the person {look}" + (f": {inst}." if inst else ".")
summary = ("[reference generation] The target video is a split screen: the kept footage beside <Subject 1>, "
"who moves in sync with it, in the same place.")
summary = ("[reference generation] The target video is a split screen: {panel_region} shows the original "
"performance, and {video_region} shows <Subject 1> doing the same movements at the same moments, in "
"the same place." if split else
"[reference generation] The target video shows <Subject 1> doing the movements of the original "
"performance at the same moments, in the same place.")
keep = (f"<Subject 1> (appears in [Shot 1]): fully_preserved - the face, hair and outfit"
f"{' from ' + pics[0] if pics else ''} are retained.")
face = f", the face from {pics[0]}" if pics else ""
shot = (f"[Shot 1] The same framing and camera movement as the kept footage. <Subject 1>{face}"
f"{', ' + inst if inst else ''}, performs {sync}.")
style = "The target video keeps the medium and the light of the kept footage."
shot = (f"[Shot 1] The same framing and camera movement. <Subject 1>{face}{', ' + inst if inst else ''}, "
"performs every movement, gesture and expression of the original performance at the same moments.")
style = "The target video is in a realistic style, as the same footage."
sound = "The sounds <Subject 1>'s movements make in the place, at the same moments."
else:
what = {"style": f"restyled as {inst or 'the style'}{seen}",
"setting": f"moved to {inst or 'the place'}{seen}",
"appearance": f"with the performer now {inst or 'changed'}{seen}",
"lighting / weather": f"under {inst or 'the new light'}{seen}"}[task]
defs = (f"<Subject 1> is {inst or 'the reference'}{seen}." if pics else "")
summary = (f"[reference generation] The target video is a split screen: the kept footage beside the same "
f"performance, {what}.")
keep = f"The performance and camera: fully_preserved - {sync}."
shot = f"[Shot 1] The same framing, people and camera movement as the kept footage, {what}, {sync}."
summary = (f"[reference generation] The target video is a split screen: {{panel_region}} shows the original "
f"performance, and {{video_region}} shows the same performance, {what}, the same movements at the "
"same moments." if split else
f"[reference generation] The target video shows the original performance, {what}.")
keep = f"The performance: fully_preserved - every movement and expression, at the same moments."
shot = f"[Shot 1] The same framing, people and camera movement, {what}, every movement at the same moments."
style = (f"The target video is in {inst}{'' if 'style' in inst.lower() else ' style'}." if task == "style" and inst
else "The target video keeps the medium of the kept footage.")
else "The target video is in a realistic style, as the same footage.")
sound = "The sounds of the scene, at the same moments."
sections = [
"subject_definitions:\n" + (defs or "The kept footage is the motion reference of the split screen."),
"subject_definitions:\n" + (defs or "<Subject 1> is the performer of the original performance."),
"summary:\n" + summary,
"retention_analysis:\n" + keep + "\nThe kept footage: fully_preserved - the panel is kept exactly.",
"detailed_description:\n" + style + " {layout}\n\n" + shot,
"overall_soundscape:\nThe sounds of the kept footage, in sync.",
"retention_analysis:\n" + keep,
"detailed_description:\n" + style + (" {layout}" if split else "") + "\n\n" + shot,
"overall_soundscape:\n" + sound,
"non_diegetic_music:\nNone.",
]
text = "\n\n".join(sections)
if rope_mode == "shifted":
text = (text.replace("The target video is a split screen: the kept footage beside ", "The target video shows ")
.replace("who moves in sync with it", "who moves in sync with the kept footage")
.replace("the motion reference of the split screen", "the motion reference"))
return text
return "\n\n".join(sections)
def h3_render(model, clip, vae, audio_vae, prompt, refs, width, height, length, steps, sampler_name, scheduler,
seed, panel=None, guide=None, guide_frame_idx=0, position="left", size=1.0, fit="contain", gap=0,
panel_noise=0.0, hold="all frames", ref_image_size="match", decode_canvas=False,
rope_mode="canvas", rope_gap=0.0, first_frame=None):
rope_mode="canvas", rope_gap=0.0, first_frame=None, history=None, history_src=None):
"""Reference to Video -> optional pinned panel -> optional aligned guide -> sample -> crop -> decode.
`refs` holds the Reference to Video inputs ({"ref_images": {...}, "ref_videos": {...}, ...}). Without a
@@ -514,12 +553,22 @@ def h3_render(model, clip, vae, audio_vae, prompt, refs, width, height, length,
from comfy_extras.nodes_minimax_h3 import MiniMaxH3AddGuide, MiniMaxH3ReferenceToVideo
from comfy_extras.nodes_custom_sampler import Guider_Basic, Noise_RandomNoise, SamplerCustomAdvanced
text = layout_text(make_info(width, height, position, size, gap * PATCH_PX), rope_mode) if panel is not None else ""
prompt = prompt.replace("{layout}", text)
linfo = make_info(width, height, position, size, gap * PATCH_PX) if panel is not None else None
text = layout_text(linfo, rope_mode) if linfo is not None else ""
prompt = fill_layout(prompt, linfo, rope_mode)
refs = {k: v for k, v in (refs or {}).items() if v}
# with a panel the refs are sized for the whole canvas, as in TSC's duet (Reference to Video at the canvas size):
# 'match' scales refs to the generation's area, and sizing them for the video half alone shrinks them a lot
rw, rh = width, height
if panel is not None:
ci = make_info(width, height, position, size, gap * PATCH_PX)
rw, rh = width + ci["strip_w"] * 16, height + ci["strip_h"] * 16
positive, latent = MiniMaxH3ReferenceToVideo.execute(
clip=clip, prompt=prompt, width=width, height=height, length=length, ref_image_size=ref_image_size,
clip=clip, prompt=prompt, width=rw, height=rh, length=length, ref_image_size=ref_image_size,
vae=vae, audio_vae=audio_vae, **refs).args[:2]
if (rw, rh) != (width, height):
from comfy_extras.nodes_minimax_h3 import _empty_av_latent
latent = _empty_av_latent(width, height, length)[0]
if first_frame is not None: # anchored at frame 0; the panel step moves it onto the canvas
positive = MiniMaxH3AddGuide.execute(positive=positive, latent=latent, frame_idx=0, vae=vae,
image=first_frame[:1]).args[0]
@@ -531,6 +580,28 @@ def h3_render(model, clip, vae, audio_vae, prompt, refs, width, height, length,
positive = MiniMaxH3AddGuide.execute(positive=positive, latent=latent, frame_idx=guide_frame_idx,
vae=vae, image=guide).args[0]
if history is not None: # continuity 'history': the previous result's end just before frame 0 (added last)
if info is not None and rope_mode == "shifted":
print("[BFS H3 Duet] history is not used with rope_mode 'shifted' (use canvas)", flush=True)
else:
hist = history[..., :3].float().cpu()
if tuple(hist.shape[1:3]) != (height, width):
hist = torch.nn.functional.interpolate(hist.movedim(-1, 1), size=(height, width), mode="bilinear",
align_corners=False).movedim(1, -1)
if info is not None: # the canvas: the source frames under the history go into the panel strip
under = history_src if history_src is not None and history_src.shape[0] >= hist.shape[0] else hist
strip = strip_frames(under[-hist.shape[0]:], list(range(hist.shape[0])), width, height, position,
size, gap * PATCH_PX, fit)
hist = compose(hist, strip, position)
del strip
try:
from .bfs_h3_history import add_history
except ImportError:
from bfs_h3_history import add_history
positive, n = add_history(positive, vae, hist)
del hist
if n:
print(f"[BFS H3 Duet] history: the previous {n} frames go before frame 0 on H3's timeline", flush=True)
if info is not None and rope_mode == "shifted":
model = patch_model_rope(model, info, rope_gap)
guider = Guider_Basic(model)
@@ -641,32 +712,159 @@ if _io is not None:
input=_io.Audio.Input("ref_video_audio", tooltip="Soundtrack of the same-numbered reference video"), prefix="ref_video_audio_", min=0, max=3)),
_io.Autogrow.Input("ref_audios", optional=True, template=_io.Autogrow.TemplatePrefix(
input=_io.Audio.Input("ref_audio", tooltip="<Audio n> reference"), prefix="ref_audio_", min=0, max=3)),
_io.Combo.Input("source_blur", options=_blur_modes(), default="off", optional=True, tooltip=
BLUR_TT + " Works with or instead of panel_static."),
_io.Combo.Input("source_blur_segmenter", options=_blur_lists().SEGMENTERS, default="SAM 3.1",
optional=True, tooltip=SEGMENTER_TT),
_io.Combo.Input("source_blur_effect", options=_blur_lists().EFFECTS, default="blur", optional=True,
tooltip=EFFECT_TT),
_io.Combo.Input("panel_static", options=PANEL_STATIC, default="off", optional=True, tooltip=
"TSC's fix for swaps in the same world: cover the face (or the whole person) of the pinned clip in TV "
"static, every frame (SAM 3), so the generated half cannot copy it and takes the identity from the "
"reference. The output is the generated half only, so the static never shows."),
_io.Image.Input("setting", optional=True, tooltip=
"The place picture (BFS Setting Picture: a frame of the clip with the person in static). It becomes "
"the last <Picture n> and the prompt says it shows the setting (write {setting} to place its tag)."),
_io.Clip.Input("vlm", optional=True, tooltip=
"Optional VLM (CLIPLoader with a Qwen3-VL text encoder) for the prompt. Empty prompt: it writes one "
"from the task + instruction (TSC's rules). Prompt with {...} fields (TSC's template): it fills them "
"from the clip and the pictures. Prompt without fields: used as written."),
_io.Int.Input("vlm_max_tokens", default=1024, min=128, max=4096, optional=True, advanced=True),
],
outputs=[_io.Image.Output(display_name="images"), _io.Audio.Output(display_name="audio"),
_io.Image.Output(display_name="canvas"), _io.String.Output(display_name="layout_text"),
_io.Latent.Output(display_name="latent"), _io.String.Output(display_name="prompt")],
_io.Latent.Output(display_name="latent"), _io.String.Output(display_name="prompt"),
_io.Image.Output(display_name="side_by_side", tooltip="Comparison like BFS Shot Join's: "
"original | references | result, the info on top and the prompt below.")],
)
@classmethod
def execute(cls, model, clip, vae, task, instruction, prompt, width, height, length, position, size, fit, gap,
panel_noise, hold, rope_mode, rope_gap, ref_image_size, steps, sampler_name, scheduler, seed,
decode_canvas, audio_vae=None, panel=None, guide=None, guide_frame_idx=0, ref_images=None,
ref_videos=None, ref_video_audios=None, ref_audios=None):
ref_videos=None, ref_video_audios=None, ref_audios=None, panel_static="off", setting=None, vlm=None, source_blur="off",
vlm_max_tokens=1024, source_blur_segmenter="SAM 3.1", source_blur_effect="blur"):
if panel is None and guide is None:
raise ValueError("BFS H3 Duet needs a panel, a guide, or both")
raw_panel = panel
panel = static_over_panel(_blur(panel, source_blur, 1.0, "[BFS H3 Duet] panel", source_blur_segmenter,
source_blur_effect), panel_static)
guide = _blur(guide, source_blur, 1.0, "[BFS H3 Duet] guide", source_blur_segmenter, source_blur_effect)
if length <= 0:
src = guide if guide is not None else panel
length = src.shape[0] if src.shape[0] >= 5 else 124
ref_images = {k: v for k, v in (ref_images or {}).items() if v is not None}
ref_videos = {k: v for k, v in (ref_videos or {}).items() if v is not None}
text = _resolve_prompt(prompt, task, instruction, len(ref_images), len(ref_videos))
text = _shot_loop().duet_prompt(vlm, prompt, guide if guide is not None else raw_panel, list(ref_images.values()),
task, instruction, int(vlm_max_tokens), rope_mode, setting)
text = _resolve_prompt(text, task, instruction, len(ref_images), len(ref_videos))
if setting is not None: # the place picture is the last <Picture n>
ref_images = {f"ref_image_{i}": v for i, v in enumerate([*ref_images.values(), setting])}
text = with_setting(text, len(ref_images), task == "character swap" or "<Subject 1>" in text)
refs = {"ref_images": ref_images, "ref_videos": ref_videos,
"ref_video_audios": {k: v for k, v in (ref_video_audios or {}).items() if v is not None},
"ref_audios": {k: v for k, v in (ref_audios or {}).items() if v is not None}}
return _io.NodeOutput(*h3_render(model, clip, vae, audio_vae, text, refs, width, height, length, steps,
sampler_name, scheduler, seed, panel, guide, guide_frame_idx, position,
size, fit, gap, panel_noise, hold, ref_image_size, decode_canvas,
rope_mode, rope_gap))
out = h3_render(model, clip, vae, audio_vae, text, refs, width, height, length, steps,
sampler_name, scheduler, seed, panel, guide, guide_frame_idx, position,
size, fit, gap, panel_noise, hold, ref_image_size, decode_canvas,
rope_mode, rope_gap)
return _io.NodeOutput(*out, side_by_side(guide if guide is not None else raw_panel, out[0],
list(ref_images.values()), out[5]))
def _shot_loop():
try:
from . import bfs_shot_loop as SL
except ImportError:
import bfs_shot_loop as SL
return SL
PANEL_STATIC = ["off", "face", "person"]
def _blur_lists():
try:
from . import bfs_source_blur as B
except ImportError:
import bfs_source_blur as B
return B
def _blur_modes():
return _blur_lists().SOURCE_BLUR
BLUR_TT = ("Blur the source person in what the model sees (the panel / the guide): '<region> (<what stays sharp>)'. "
"keep mouth: lips, teeth and tongue stay sharp; keep lips + tongue: the teeth are blurred too; mouth too: "
"nothing stays sharp. masked area: the shot's own mask (elsewhere: the whole person). It reads the pose and "
"motion but has no face to copy. The output is never blurred.")
SEGMENTER_TT = ("What finds the face / head / hands / person and the mouth for source_blur. SAM 3.1: text prompts. "
"Sapiens2: body-part classes in one pass, finer lips and teeth (downloads facebook/sapiens2-seg-0.4b, "
"1.6 GB, to models/sapiens2 the first time). 'keep lips + tongue' always uses Sapiens2 for the lips.")
EFFECT_TT = "blur: no detail left. invert colours: keeps the shapes and the expression but not the skin tone or colours."
def _blur(frames, mode, strength=1.0, log="", segmenter="SAM 3.1", effect="blur", area=None):
if frames is None or not mode or mode == "off":
return frames
return _blur_lists().blur_source(frames, mode, float(strength), log, segmenter, effect, area)
def _small(frames: torch.Tensor, side: int = 640) -> torch.Tensor:
H, W = frames.shape[1:3]
s = side / max(H, W)
if s >= 1:
return frames[..., :3].float()
return torch.nn.functional.interpolate(frames[..., :3].movedim(-1, 1).float(), size=(max(32, int(H * s) // 2 * 2),
max(32, int(W * s) // 2 * 2)), mode="bilinear", align_corners=False).movedim(1, -1)
def static_over_panel(frames: torch.Tensor | None, what: str) -> torch.Tensor | None:
"""TSC's fix for same-world swaps: the face (or the whole person) of the pinned clip covered in TV static, every
frame, so the generated half cannot copy it and takes the identity from the reference. Tracked by SAM 3."""
if frames is None or not what or what == "off":
return frames
SL = _shot_loop()
spec = dict(SL.DEFAULT_MASK, text=what, max_objects=4, threshold=0.4)
masks = SL.segment_frames(_small(frames), spec)
out = torch.cat([SL.tv_static(frames[i:i + 1, ..., :3].float(), masks[min(i, masks.shape[0] - 1)],
grow=0.05 if what == "face" else 0.03, seed=i) for i in range(frames.shape[0])], 0)
cov = float((masks > 0.5).float().mean())
print(f"[BFS H3 Duet] panel_static '{what}': covered {cov * 100:.1f}% of the panel (SAM 3, {frames.shape[0]} frames)",
flush=True)
return out
def setting_from_clip(frames: torch.Tensor, position: float = 0.5, subject: str = "person") -> torch.Tensor:
"""TSC's setting picture: one frame of the clip with the subject covered in TV static, so the generated half sees the
place (light, framing, background) in full detail and not the person."""
SL = _shot_loop()
k = int(round((frames.shape[0] - 1) * min(1.0, max(0.0, position))))
img = frames[k:k + 1, ..., :3].float()
m = SL.segment_frames(_small(img), dict(SL.DEFAULT_MASK, text=subject or "person", max_objects=8))[0]
return SL.tv_static(img, m)
def with_setting(text: str, k: int, swap: bool) -> str:
"""{setting} becomes <Picture k>; otherwise TSC's sentence goes at the end of subject_definitions."""
return _shot_loop().add_setting(text, k, swap)
def side_by_side(source: torch.Tensor | None, result: torch.Tensor, refs: list | None = None, prompt: str = "",
label: str = "BFS H3 Duet") -> torch.Tensor:
"""The same comparison as BFS Shot Join's: original | references | result, the info on top and the prompt below."""
try:
from .bfs_shot_loop import comparison_video
except ImportError:
from bfs_shot_loop import comparison_video
res = result[..., :3].float().cpu()
L = res.shape[0]
refs = [r for r in (refs or []) if r is not None]
shot = {"fps": 24.0, "start": 0, "end": L, "gen_length": L, "prompt": prompt,
"ref": refs[0] if refs else None, "ref2": refs[1] if len(refs) > 1 else None}
orig = source[..., :3].float().cpu() if source is not None else None
return comparison_video(res, [(shot, L, orig)], label, height=min(720, res.shape[1]), show_mask=False)
if _io is not None:
@@ -712,7 +910,9 @@ if _io is not None:
_io.Float.Input("rope_gap", default=0.0, min=0.0, max=256.0, step=1.0, advanced=True),
_io.Combo.Input("ref_image_size", options=["match", "max"], default="match", advanced=True),
_io.Int.Input("vlm_max_tokens", default=1024, min=128, max=4096, advanced=True),
_io.Clip.Input("vlm", optional=True, tooltip="Optional VLM (CLIPLoader with a Qwen3-VL text encoder) that writes the prompt."),
_io.Clip.Input("vlm", optional=True, tooltip="Optional VLM (CLIPLoader with a Qwen3-VL text encoder). Empty prompt: "
"it writes one from the task + instruction. Prompt with {...} fields: it fills them from the "
"clip and the pictures. Prompt without fields: used as written."),
_io.Model.Input("model", optional=True, tooltip="Needed for rope_mode = shifted."),
_io.Vae.Input("audio_vae", optional=True),
_io.Image.Input("guide", optional=True, tooltip="Optional aligned latent guide in the video area."),
@@ -723,6 +923,19 @@ if _io is not None:
_io.Image.Input("keep_video", optional=True, tooltip="The video kept outside the mask (default: the panel clip)."),
_io.Autogrow.Input("ref_images", optional=True, template=_io.Autogrow.TemplatePrefix(
input=_io.Image.Input("ref_image", tooltip="<Picture n>, in order"), prefix="ref_image_", min=0, max=9)),
_io.Combo.Input("source_blur", options=_blur_modes(), default="off", optional=True, tooltip=
BLUR_TT + " Works with or instead of panel_static."),
_io.Combo.Input("source_blur_segmenter", options=_blur_lists().SEGMENTERS, default="SAM 3.1",
optional=True, tooltip=SEGMENTER_TT),
_io.Combo.Input("source_blur_effect", options=_blur_lists().EFFECTS, default="blur", optional=True,
tooltip=EFFECT_TT),
_io.Combo.Input("panel_static", options=PANEL_STATIC, default="off", optional=True, tooltip=
"TSC's fix for swaps in the same world: cover the face (or the whole person) of the pinned clip in TV "
"static, every frame (SAM 3), so the generated half cannot copy it and takes the identity from the "
"reference. The output is the generated half only, so the static never shows."),
_io.Image.Input("setting", optional=True, tooltip=
"The place picture (BFS Setting Picture: a frame of the clip with the person in static). It becomes "
"the last <Picture n> and the prompt says it shows the setting (write {setting} to place its tag)."),
],
outputs=[_io.Conditioning.Output(display_name="positive"), _io.Latent.Output(display_name="latent"),
_io.Model.Output(display_name="model"), PANEL_INFO_T.Output(display_name="panel_info"),
@@ -732,28 +945,32 @@ if _io is not None:
@classmethod
def execute(cls, clip, vae, panel, task, instruction, prompt, width, height, length, position, size, panel_noise,
rope_mode, fit, gap, hold, rope_gap, ref_image_size, vlm_max_tokens, vlm=None, model=None,
audio_vae=None, guide=None, ref_images=None, keep_mask=None, keep_video=None):
audio_vae=None, guide=None, ref_images=None, keep_mask=None, keep_video=None, panel_static="off",
setting=None, source_blur="off", source_blur_segmenter="SAM 3.1", source_blur_effect="blur"):
from comfy_extras.nodes_minimax_h3 import MiniMaxH3ReferenceToVideo
refs = [v for v in (ref_images or {}).values() if v is not None]
raw_panel = panel
panel = static_over_panel(_blur(panel, source_blur, 1.0, "[BFS H3 Duet Conditioning] panel",
source_blur_segmenter, source_blur_effect), panel_static)
guide = _blur(guide, source_blur, 1.0, "[BFS H3 Duet Conditioning] guide", source_blur_segmenter,
source_blur_effect)
if length <= 0:
length = panel.shape[0] if panel.shape[0] >= 5 else 124
text = prompt if prompt and prompt.strip() else ""
if not text and vlm is not None:
try:
from .bfs_shot_loop import write_duet_prompt
except ImportError:
write_duet_prompt = None
if write_duet_prompt:
idx = sorted(set(int(round(x)) for x in torch.linspace(0, panel.shape[0] - 1, min(4, panel.shape[0])).tolist()))
text = write_duet_prompt(vlm, [panel[i:i + 1] for i in idx], refs, task, instruction, int(vlm_max_tokens),
rope_mode)
text = _shot_loop().duet_prompt(vlm, prompt, raw_panel, refs, task, instruction, int(vlm_max_tokens),
rope_mode, setting)
if not text:
text = build_prompt(task if task != "custom" else "appearance", instruction, len(refs), 0, rope_mode)
if setting is not None: # the place picture is the last <Picture n>
refs = refs + [setting]
text = with_setting(text, len(refs), task == "character swap" or "<Subject 1>" in text)
info = make_info(width, height, position, size, gap * PATCH_PX)
text = text.replace("{layout}", layout_text(info, rope_mode))
positive, latent = MiniMaxH3ReferenceToVideo.execute(
clip=clip, prompt=text, width=width, height=height, length=length, ref_image_size=ref_image_size, vae=vae,
audio_vae=audio_vae, ref_images={f"ref_image_{i}": r for i, r in enumerate(refs)} or None).args[:2]
text = fill_layout(text, info, rope_mode)
rw, rh = width + info["strip_w"] * 16, height + info["strip_h"] * 16 # refs sized for the canvas
positive = MiniMaxH3ReferenceToVideo.execute(
clip=clip, prompt=text, width=rw, height=rh, length=length, ref_image_size=ref_image_size, vae=vae,
audio_vae=audio_vae, ref_images={f"ref_image_{i}": r for i, r in enumerate(refs)} or None).args[0]
from comfy_extras.nodes_minimax_h3 import _empty_av_latent
latent = _empty_av_latent(width, height, length)[0]
positive, latent, info, preview, _ = BFSH3SidePanel().apply(
positive, latent, vae, panel, position, size, fit, gap, hold, panel_noise, guide, 0,
keep_video=keep_video, keep_mask=keep_mask)
@@ -799,7 +1016,13 @@ class BFSShotH3Duet:
},
"optional": {"audio_vae": ("VAE",),
"rope_mode": (ROPE_MODES, {"default": "canvas", "advanced": True}),
"rope_gap": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 256.0, "step": 1.0, "advanced": True})},
"rope_gap": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 256.0, "step": 1.0, "advanced": True}),
"source_blur": (_blur_modes(), {"default": "off", "tooltip": BLUR_TT}),
"source_blur_strength": ("FLOAT", {"default": 1.0, "min": 0.1, "max": 1.0, "step": 0.05}),
"source_blur_segmenter": (_blur_lists().SEGMENTERS, {"default": "SAM 3.1", "tooltip": SEGMENTER_TT}),
"source_blur_effect": (_blur_lists().EFFECTS, {"default": "blur", "tooltip": EFFECT_TT}),
"vlm": ("CLIP", {"tooltip": "Optional VLM (Qwen3-VL CLIPLoader): fills {details} in the shot's "
"prompt from its references when the planner left it."})},
}
RETURN_TYPES = ("IMAGE", "AUDIO", "IMAGE", "STRING", "STRING")
@@ -813,39 +1036,74 @@ class BFSShotH3Duet:
def render(self, shot, model, clip, vae, mode, task, instruction, use_ref_2, position, size, fit, gap,
panel_noise, ref_image_size, steps, sampler_name, scheduler, seed, decode_canvas, audio_vae=None,
rope_mode="canvas", rope_gap=0.0):
imgs = {}
if shot.get("ref") is not None:
imgs["ref_image_1"] = shot["ref"]
if use_ref_2 and shot.get("ref2") is not None:
imgs[f"ref_image_{len(imgs) + 1}"] = shot["ref2"]
rope_mode="canvas", rope_gap=0.0, source_blur="off", source_blur_strength=1.0, vlm=None,
source_blur_segmenter="SAM 3.1", source_blur_effect="blur"):
own = _shot_loop().shot_refs(shot)
imgs = {f"ref_image_{k + 1}": r for k, r in enumerate(own[:_shot_loop().MAX_REFS if use_ref_2 else 1])}
duet, guided = mode != self.MODES[1], mode != self.MODES[0]
try:
from .bfs_shot_loop import chain_image, remember_result
from .bfs_shot_loop import chain_image, remember_result, shot_history
except ImportError:
chain_image = remember_result = None
chain_image = remember_result = shot_history = None
prev = chain_image(shot) if chain_image else None
if prev is not None and shot.get("chain") == "reference":
imgs[f"ref_image_{len(imgs) + 1}"] = prev
text = _resolve_prompt(shot.get("prompt", ""), task, instruction, len(imgs), 0)
if "{details" in text:
text = _shot_loop().fill_details_now(text, vlm, own, f"[BFS Shot H3 Duet] shot {int(shot.get('index', 0)) + 1}",
shot.get("ref_names"))
hist, hsrc = shot_history(shot) if shot_history else (None, None)
area = None
if source_blur and source_blur.startswith("masked area") and (shot.get("mask_src") or (shot.get("crop") or {}).get("crop_mask") is not None):
area = _shot_loop().generation_mask(shot)
seen = _blur(shot["frames"], source_blur, source_blur_strength,
f"[BFS Shot H3 Duet] shot {int(shot.get('index', 0)) + 1}", source_blur_segmenter, source_blur_effect, area)
out = h3_render(model, clip, vae, audio_vae, text, {"ref_images": imgs}, shot["width"], shot["height"],
shot["gen_length"], steps, sampler_name, scheduler, seed + int(shot.get("index", 0)),
panel=shot["frames"] if duet else None, guide=shot["frames"] if guided else None,
panel=seen if duet else None, guide=seen if guided else None,
position=position, size=size, fit=fit, gap=gap, panel_noise=panel_noise,
ref_image_size=ref_image_size, decode_canvas=decode_canvas,
rope_mode=rope_mode, rope_gap=rope_gap,
first_frame=prev if prev is not None and shot.get("chain") == "first frame" else None)
first_frame=prev if prev is not None and shot.get("chain") == "first frame" else None,
history=hist, history_src=hsrc)
del hist, hsrc
if remember_result:
remember_result(shot, out[0]) # the next shot can continue from it (auto loop)
return out[:4] + (out[5],)
class BFSSettingPicture:
"""TSC's setting picture: a frame of the clip with the person covered in TV static (the place, not the person)."""
@classmethod
def INPUT_TYPES(cls):
return {"required": {
"images": ("IMAGE", {"tooltip": "The source clip (or one frame)."}),
"position": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.05,
"tooltip": "Which frame: 0 = first, 0.5 = middle, 1 = last."}),
"subject": ("STRING", {"default": "person", "tooltip": "What SAM 3 covers with static (e.g. person, man, dog)."}),
}}
RETURN_TYPES = ("IMAGE",)
RETURN_NAMES = ("setting",)
FUNCTION = "make"
CATEGORY = "BFS/MiniMax H3"
DESCRIPTION = ("TSC's setting picture for a duet: one frame of the clip with the person covered in TV static. Give it "
"to BFS H3 Duet's `setting` input: the generated half then takes the place, light and framing from "
"it instead of from the character picture.")
def make(self, images, position, subject):
return (setting_from_clip(images, position, subject),)
NODE_CLASS_MAPPINGS = {
"BFSSettingPicture": BFSSettingPicture,
"BFSShotH3Duet": BFSShotH3Duet,
"BFSH3SidePanel": BFSH3SidePanel,
"BFSH3SidePanelCrop": BFSH3SidePanelCrop,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"BFSSettingPicture": "BFS Setting Picture (place, person in static)",
"BFSShotH3Duet": "BFS Shot H3 Duet (render one shot)",
"BFSH3SidePanel": "BFS H3 Side Panel (virtual reference panel)",
"BFSH3SidePanelCrop": "BFS H3 Side Panel Crop",
+97
View File
@@ -0,0 +1,97 @@
"""Sapiens2 body-part segmentation (Meta, facebook/sapiens2-seg-*): one class per pixel out of 29 (28 parts +
background), lips, teeth and tongue separate from the face. Used by the source blur for parts SAM 3.1 can't
find reliably from a text prompt (lips, tongue). It marks every person in the frame (no instances).
The weights (0.4B, 1.6 GB) go to models/sapiens2/sapiens2-seg-0.4b, downloaded from Hugging Face the first time.
Sapiens2 License: https://github.com/facebookresearch/sapiens2/blob/main/LICENSE.md
"""
from __future__ import annotations
import os
import torch
REPO = "facebook/sapiens2-seg-0.4b"
FILES = ("config.json", "preprocessor_config.json", "model.safetensors")
CLASSES = ("Background", "Apparel", "Eyeglass", "Face_Neck", "Hair", "Left_Foot", "Left_Hand", "Left_Lower_Arm",
"Left_Lower_Leg", "Left_Shoe", "Left_Sock", "Left_Upper_Arm", "Left_Upper_Leg", "Lower_Clothing",
"Right_Foot", "Right_Hand", "Right_Lower_Arm", "Right_Lower_Leg", "Right_Shoe", "Right_Sock",
"Right_Upper_Arm", "Right_Upper_Leg", "Torso", "Upper_Clothing", "Lower_Lip", "Upper_Lip", "Lower_Teeth",
"Upper_Teeth", "Tongue")
ID = {n: i for i, n in enumerate(CLASSES)}
PIXELS = 1024 * 768 # the size it was trained at; other shapes keep this area and their aspect ratio
_MEAN = (0.485, 0.456, 0.406)
_STD = (0.229, 0.224, 0.225)
_MODEL: dict = {}
def model_dir() -> str:
try:
import folder_paths
base = folder_paths.models_dir
except Exception:
base = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "..", "models")
return os.path.join(base, "sapiens2", REPO.split("/")[-1])
def _load():
if "m" in _MODEL:
return _MODEL["m"]
d = model_dir()
if not all(os.path.exists(os.path.join(d, f)) for f in FILES):
print(f"[BFS Sapiens2] downloading {REPO} (1.6 GB) to {d}", flush=True)
from huggingface_hub import hf_hub_download
for f in FILES:
hf_hub_download(REPO, f, local_dir=d)
from transformers import Sapiens2ForSemanticSegmentation
_MODEL["m"] = Sapiens2ForSemanticSegmentation.from_pretrained(d, dtype=torch.float32).eval()
return _MODEL["m"]
def _device():
try:
import comfy.model_management as mm
return mm.get_torch_device()
except Exception:
return torch.device("cuda" if torch.cuda.is_available() else "cpu")
def input_size(h: int, w: int) -> tuple[int, int]:
"""(H, W) with the trained area, the frame's aspect ratio, multiples of the 16 px patch."""
s = (PIXELS / (h * w)) ** 0.5
return max(16, round(h * s / 16) * 16), max(16, round(w * s / 16) * 16)
@torch.no_grad()
def segment(frames: torch.Tensor, out_size: tuple[int, int] | None = None, batch: int = 4) -> torch.Tensor:
"""frames [N,H,W,3] 0-1 -> class ids [N,h,w] uint8 at `out_size` (default the frames' size)."""
m = _load()
dev = _device()
half = dev.type == "cuda"
m.to(dev, dtype=torch.float16 if half else torch.float32)
N, H, W = frames.shape[:3]
oh, ow = out_size or (H, W)
ih, iw = input_size(H, W)
mean = torch.tensor(_MEAN, device=dev).view(1, 3, 1, 1)
std = torch.tensor(_STD, device=dev).view(1, 3, 1, 1)
out = torch.empty(N, oh, ow, dtype=torch.uint8)
try:
for a in range(0, N, batch):
x = frames[a:a + batch, ..., :3].to(dev).float().movedim(-1, 1)
x = torch.nn.functional.interpolate(x, size=(ih, iw), mode="bilinear", align_corners=False, antialias=True)
x = ((x - mean) / std).to(torch.float16 if half else torch.float32)
lg = m(pixel_values=x).logits.float()
lg = torch.nn.functional.interpolate(lg, size=(oh, ow), mode="bilinear", align_corners=False)
out[a:a + batch] = lg.argmax(1).to(torch.uint8).cpu()
del x, lg
finally:
m.to("cpu") # 0.4B: back to RAM, the VRAM is the video model's
if half:
torch.cuda.empty_cache()
return out
def parts(classes: torch.Tensor, names) -> torch.Tensor:
"""[N,h,w] float mask of the pixels whose class is one of `names`."""
ids = torch.tensor([ID[n] for n in names], dtype=classes.dtype)
return torch.isin(classes, ids).float()
+3583 -548
View File
File diff suppressed because it is too large Load Diff
+323
View File
@@ -0,0 +1,323 @@
"""BFS Shot Noise: the sampler's starting noise, with optional structure. Every option is off at 0, and with all of
them off the output is exactly ComfyUI's RandomNoise for the same seed.
What can be shaped (the video part of the latent only; the audio noise is always plain):
seed per shot same for every shot / increment / random per shot / overrides (as BFS Shot Seed)
shared_base the COARSE pattern of the noise (block averages on a fixed grid) comes from a base seed instead
of the shot's seed; the fine detail stays the shot's own. The coarse pattern decides layout and
who the subject becomes in the first steps, so shots of one video (or re-rolls of one clip)
agree on it. The grid is defined in blocks, not in latent cells, so the same base seed gives the
same coarse pattern at any resolution, and along the timeline it is one long tape: a shot reads
the part at its own position, whatever its length.
distribution gaussian (default) / laplacian / student-t / uniform, all at unit variance
spectrum_tilt 1/f^alpha spectrum: more (or less) energy in the large blobs ("fractal" noise)
pyramid coarser layers of noise added on top
static_share a part of the noise that is the same in every frame (temporally correlated noise)
guide_structure outside the mask, a little of the guide video's own latent is mixed into the noise, so the
background starts closer to the guide; inside the mask the noise stays free (no pull toward
the original person)
ref_tint inside the mask, the noise leans toward the reference picture's average latent (its colours)
All mixes keep the noise at unit variance (a*signal + sqrt(1-a^2)*noise, block means replaced by block means of the same
variance). The model was trained on plain noise: small values, and compare with 0."""
import hashlib
import math
import torch
import comfy.nested_tensor
import comfy.sample
try:
from .bfs_shot_loop import shot_seed, BFSShotSeed
from .bfs_h3_side_panel import latent_mask
except ImportError: # run as a script (tests)
from bfs_shot_loop import shot_seed, BFSShotSeed
from bfs_h3_side_panel import latent_mask
_pick_seed = shot_seed # BFSShotNoise.make has an input of the same name
def _gen(*key) -> torch.Generator:
return torch.Generator().manual_seed(int.from_bytes(hashlib.sha256("/".join(str(k) for k in key).encode()).digest()[:8], "big") >> 1)
def _edges(n: int, blocks: int) -> torch.Tensor:
"""Block id of each of n cells for `blocks` near-equal blocks (blocks is capped at n)."""
blocks = max(1, min(int(blocks), n))
return torch.clamp((torch.arange(n) * blocks) // n, max=blocks - 1)
def replace_coarse(x: torch.Tensor, base_seed: int, amount: float, grid: int = 6, time_block: int = 2, t0: int = 0) -> torch.Tensor:
"""x [C,T,h,w] unit noise. The mean of every (time block, y block, x block) cell is replaced (amount 1) or mixed
with a mean drawn from `base_seed`; the result is unit noise again. Blocks: `grid` along the short side, the long
side in proportion; `time_block` latent frames, counted from timeline position t0."""
if amount <= 0:
return x
C, T, h, w = x.shape
hb, wb = (grid, max(1, round(grid * w / h))) if h <= w else (max(1, round(grid * h / w)), grid)
yid, xid = _edges(h, hb), _edges(w, wb)
hb, wb = int(yid.max()) + 1, int(xid.max()) + 1
tid_abs = (t0 + torch.arange(T)) // max(1, int(time_block))
tids = torch.unique(tid_abs)
tloc = torch.searchsorted(tids, tid_abs)
nb = len(tids) * hb * wb
ids = ((tloc[:, None, None] * hb + yid[None, :, None]) * wb + xid[None, None, :]).reshape(-1)
flat = x.reshape(C, -1).double()
cnt = torch.zeros(nb, dtype=torch.float64).index_add_(0, ids, torch.ones_like(ids, dtype=torch.float64))
mean = torch.zeros(C, nb, dtype=torch.float64).index_add_(1, ids, flat) / cnt
z = torch.stack([torch.randn(C, hb, wb, generator=_gen("coarse", base_seed, int(t), hb, wb), dtype=torch.float64) for t in tids], 1).reshape(C, nb)
a = float(min(1.0, amount))
new = math.sqrt(1.0 - a * a) * mean + a * z / cnt.sqrt()
return (flat + (new - mean)[:, ids]).reshape(C, T, h, w).to(x.dtype)
DISTRIBUTIONS = ["gaussian", "laplacian", "student-t", "uniform"]
def draw(shape, kind: str, seed: int) -> torch.Tensor:
"""Unit-variance noise of another distribution (heavier or lighter tails than the gaussian the model was trained on)."""
g = _gen("dist", kind, seed)
if kind == "laplacian":
u = torch.rand(shape, generator=g, dtype=torch.float64) - 0.5
return (-torch.sign(u) * torch.log1p(-2 * u.abs().clamp(max=0.5 - 1e-12)) / math.sqrt(2.0)).float()
if kind == "student-t": # 5 degrees of freedom
n = torch.randn(shape, generator=g, dtype=torch.float64)
chi = torch.randn((5,) + tuple(shape), generator=g, dtype=torch.float64).pow(2).sum(0)
return (n / torch.sqrt(chi / 5.0) * math.sqrt(3.0 / 5.0)).float()
if kind == "uniform":
return ((torch.rand(shape, generator=g, dtype=torch.float64) - 0.5) * math.sqrt(12.0)).float()
return torch.randn(shape, generator=g)
def tilt_spectrum(x: torch.Tensor, alpha: float) -> torch.Tensor:
"""x [C,T,h,w]: power spectrum multiplied by 1/f^alpha over time and space (alpha > 0 = more energy in the large
blobs, 'fractal' noise; alpha < 0 = more in the fine grain), then back to zero mean and unit variance per channel."""
if alpha == 0:
return x
C, T, h, w = x.shape
f = torch.sqrt(torch.fft.fftfreq(T)[:, None, None] ** 2 + torch.fft.fftfreq(h)[None, :, None] ** 2 + torch.fft.rfftfreq(w)[None, None, :] ** 2)
f[0, 0, 0] = f[f > 0].min()
y = torch.fft.irfftn(torch.fft.rfftn(x.double(), dim=(1, 2, 3)) * f.pow(-alpha / 2.0), s=(T, h, w), dim=(1, 2, 3))
y = y - y.mean(dim=(1, 2, 3), keepdim=True)
return (y / y.std(dim=(1, 2, 3), keepdim=True).clamp_min(1e-8)).to(x.dtype)
def add_pyramid(x: torch.Tensor, amount: float, seed: int, levels: int = 4) -> torch.Tensor:
"""x [C,T,h,w] plus coarser copies of fresh noise (each level half the size, weight amount^level), unit variance."""
if amount <= 0:
return x
C, T, h, w = x.shape
y = x.double()
for i in range(1, levels + 1):
hh, ww = max(1, h >> i), max(1, w >> i)
if hh == 1 and ww == 1:
break
n = torch.randn(C, T, hh, ww, generator=_gen("pyramid", seed, i), dtype=torch.float64)
y = y + (amount ** i) * torch.nn.functional.interpolate(n, size=(h, w), mode="bilinear", align_corners=False)
y = y - y.mean(dim=(1, 2, 3), keepdim=True)
return (y / y.std(dim=(1, 2, 3), keepdim=True).clamp_min(1e-8)).to(x.dtype)
def mix(x: torch.Tensor, signal: torch.Tensor, weight: torch.Tensor) -> torch.Tensor:
weight = weight.clamp(0, 1)
return torch.sqrt(1.0 - weight ** 2) * x + weight * signal
class ShotNoise:
def __init__(self, seed, base_seed, opts, guide=None, tint=None, mask=None, t0=0):
self.seed, self.base_seed, self.o, self.guide, self.tint, self.mask, self.t0 = int(seed), int(base_seed), opts, guide, tint, mask, int(t0)
def generate_noise(self, input_latent):
samples = input_latent["samples"]
inds = input_latent["batch_index"] if "batch_index" in input_latent else None
noise = comfy.sample.prepare_noise(samples, self.seed, inds) # exactly RandomNoise
o = self.o
if not any(o[k] > 0 for k in ("shared_base", "static_share", "guide_structure", "ref_tint", "pyramid")) \
and o["spectrum_tilt"] == 0 and o["distribution"] == "gaussian":
return noise
nested = getattr(noise, "is_nested", False)
parts = list(noise.unbind()) if nested else [noise]
v = parts[0]
if v.ndim != 5:
return noise
out = []
for b in range(v.shape[0]):
x = v[b].float()
C, T, h, w = x.shape
if o["distribution"] != "gaussian":
x = draw((C, T, h, w), o["distribution"], self.seed + b)
x = add_pyramid(tilt_spectrum(x, o["spectrum_tilt"]), o["pyramid"], self.seed + b)
if o["static_share"] > 0:
s = torch.randn(C, 1, h, w, generator=_gen("static", self.base_seed, h, w))
x = mix(x, s.expand_as(x), torch.tensor(float(o["static_share"])))
x = replace_coarse(x, self.base_seed, o["shared_base"], o["base_grid"], o["time_block"], self.t0)
m = None
if self.mask is not None:
m = latent_mask(self.mask, T, h, w, grow=1)[0].to(x) # [1,T,h,w], 1 = the person
if o["guide_structure"] > 0 and self.guide is not None:
g = self.guide[min(b, self.guide.shape[0] - 1)].float().cpu()
if tuple(g.shape[1:]) != (T, h, w):
g = torch.nn.functional.interpolate(g[None], size=(T, h, w), mode="trilinear", align_corners=False)[0]
g = (g - g.mean(dim=(1, 2, 3), keepdim=True)) / g.std(dim=(1, 2, 3), keepdim=True).clamp_min(1e-6)
x = mix(x, g, float(o["guide_structure"]) * ((1.0 - m) if m is not None else torch.ones(1, T, h, w)))
if o["ref_tint"] > 0 and self.tint is not None:
x = mix(x, self.tint.view(C, 1, 1, 1).expand_as(x), float(o["ref_tint"]) * (m if m is not None else torch.ones(1, T, h, w)))
out.append(x.to(v.dtype))
parts[0] = torch.stack(out)
if nested:
return comfy.nested_tensor.NestedTensor(parts)
return parts[0]
class BFSShotNoise:
"""Starting noise for the sampler with optional structure (see the node description)."""
@classmethod
def INPUT_TYPES(cls):
f = lambda tip, mx=1.0: ("FLOAT", {"default": 0.0, "min": 0.0, "max": mx, "step": 0.01, "tooltip": tip})
return {
"required": {
"seed": ("INT", {"default": 0, "min": 0, "max": 0xffffffffffffffff, "control_after_generate": True,
"tooltip": "The seed (the base seed when a shot is connected)."}),
"shot_seed": (BFSShotSeed.MODES, {"default": BFSShotSeed.MODES[0], "tooltip":
"Only with a shot connected: the seed of each shot (same / counting up / unrelated but repeatable)."}),
"shared_base": f("0 = off (plain noise). The coarse pattern of the noise comes from base_seed instead of "
"the shot's seed: every shot, any length or size, agrees on it; the fine detail stays "
"per shot. 1 = fully shared. Try 0.5 - 1."),
"static_share": f("0 = off. A part of the noise that is identical in every frame (temporally "
"correlated noise). Try 0.1 - 0.3; too much freezes the motion.", 0.9),
"guide_structure": f("0 = off. Needs guide_frames + vae. Outside the mask the guide's latent is mixed "
"into the noise, so the background starts closer to the guide (without a mask: "
"everywhere, which also pulls toward the original person). Try 0.05 - 0.2.", 0.6),
"distribution": (DISTRIBUTIONS, {"default": "gaussian", "tooltip":
"gaussian = what the model was trained on. laplacian / student-t: more extreme values (stronger, "
"sparser kicks). uniform: none. All at unit variance. Like AdvancedNoise's types, but for H3's "
"video + audio latent (the audio noise always stays gaussian)."}),
"spectrum_tilt": ("FLOAT", {"default": 0.0, "min": -2.0, "max": 2.0, "step": 0.05, "tooltip":
"0 = off (white noise). > 0: more energy in the large blobs (fractal / pink noise, AdvancedNoise's "
"'fractal' alpha): the coarse layout, where the swap is decided, gets a harder push. < 0: more "
"fine grain. Try 0.2 - 0.6; large values burn colours."}),
"pyramid": f("0 = off. Adds coarser layers of noise (each half the size, weight = this value per "
"level): another way to put more energy in the large scales. Try 0.3 - 0.6.", 0.95),
"ref_tint": f("0 = off. Needs ref_image + vae. Inside the mask (everywhere without one) the noise "
"leans toward the reference's average latent, i.e. its colours. Try 0.05 - 0.15.", 0.6),
},
"optional": {
"shot": ("BFS_SHOT", {"tooltip": "From BFS Shot Planner / Unpack: per-shot seeds, and the shared base is "
"read at the shot's position on the timeline."}),
"base_seed": ("INT", {"default": -1, "min": -1, "max": 0xffffffffffffffff, "tooltip":
"Seed of the shared coarse pattern and of static_share. -1 = the seed above. Put here the seed of "
"a result that worked to keep its coarse noise while the detail changes with the seed."}),
"overrides": ("STRING", {"default": "", "tooltip": "Seeds for single shots: 'shot:seed, shot:seed' (shots count from 1)."}),
"base_grid": ("INT", {"default": 6, "min": 1, "max": 32, "advanced": True, "tooltip":
"Blocks of the coarse pattern along the short side (the long side in proportion). Fewer = only the "
"broadest layout is shared."}),
"time_block": ("INT", {"default": 2, "min": 1, "max": 32, "advanced": True, "tooltip": "Latent frames per block of the coarse pattern."}),
"vae": ("VAE", {"tooltip": "The video VAE, for guide_structure and ref_tint."}),
"guide_frames": ("IMAGE", {"tooltip": "The guide video frames (guide_structure)."}),
"ref_image": ("IMAGE", {"tooltip": "The reference picture (ref_tint)."}),
"mask": ("MASK", {"tooltip": "The person, one mask per frame (white = the person): guide_structure stays "
"outside it, ref_tint inside it."}),
},
}
RETURN_TYPES = ("NOISE", "INT")
RETURN_NAMES = ("noise", "seed")
OUTPUT_TOOLTIPS = ("For SamplerCustomAdvanced.", "The seed this shot used.")
FUNCTION = "make"
CATEGORY = "BFS/shot loop"
DESCRIPTION = ("Optional structured starting noise. With every amount at 0 it is exactly RandomNoise. shared_base: "
"the coarse noise pattern is the same for every shot / size / length (from base_seed), the detail "
"per shot. static_share: a part identical in all frames. guide_structure: the guide's latent mixed "
"in outside the mask. ref_tint: lean toward the reference's colours inside the mask. Experimental: "
"the model was trained on plain noise, so use small amounts and compare with 0.")
def make(self, seed, shot_seed, shared_base, static_share, guide_structure, ref_tint, distribution="gaussian", spectrum_tilt=0.0,
pyramid=0.0, shot=None, base_seed=-1, overrides="",
base_grid=6, time_block=2, vae=None, guide_frames=None, ref_image=None, mask=None):
index = int(shot.get("index", 0)) if shot else 0
s = _pick_seed(int(seed), index, shot_seed, overrides) if shot else int(seed)
base = int(seed) if int(base_seed) < 0 else int(base_seed)
guide = tint = None
if guide_structure > 0:
if vae is None or guide_frames is None:
raise ValueError("BFS Shot Noise: guide_structure needs vae and guide_frames")
guide = vae.encode(guide_frames[..., :3])
guide = guide.tensors[0] if getattr(guide, "is_nested", False) else guide
if ref_tint > 0:
if vae is None or ref_image is None:
raise ValueError("BFS Shot Noise: ref_tint needs vae and ref_image")
r = vae.encode(ref_image[:1, ..., :3])
r = (r.tensors[0] if getattr(r, "is_nested", False) else r).float().cpu()
tint = (r.mean(dim=(0, 2, 3, 4)) / r.std(dim=(0, 2, 3, 4)).clamp_min(1e-6)).clamp(-2, 2)
t0 = int(round(int(shot.get("start", 0)) / 4)) if shot else 0 # timeline position in latent frames
opts = {"shared_base": float(shared_base), "static_share": float(static_share), "guide_structure": float(guide_structure),
"ref_tint": float(ref_tint), "base_grid": int(base_grid), "time_block": int(time_block),
"distribution": distribution, "spectrum_tilt": float(spectrum_tilt), "pyramid": float(pyramid)}
return (ShotNoise(s, base, opts, guide, tint, mask, t0), s)
def sigmas_chart(sigmas: torch.Tensor, other: torch.Tensor | None = None, w: int = 640, h: int = 360):
"""(image [1,h,w,3], text): the noise level at the start of every step, with the zones where the steps are spent."""
import numpy as np
from PIL import Image, ImageDraw
s = [float(v) for v in sigmas.flatten().tolist()]
top = max(1e-6, s[0])
img = Image.new("RGB", (w, h), (24, 24, 28)); d = ImageDraw.Draw(img)
L, R, T, B = 52, w - 16, 34, h - 40
y = lambda v: B - (v / top) * (B - T)
zones = ((0.85, 1.01, (60, 40, 40), "layout / identity"), (0.5, 0.85, (48, 46, 36), "shapes"), (-0.01, 0.5, (36, 44, 52), "detail"))
for lo, hi, col, name in zones:
d.rectangle([L, y(min(hi, 1.0) * top), R, y(max(lo, 0.0) * top)], fill=col)
d.text((R - 6 - 6 * len(name), y(min(hi, 1.0) * top) + 3), name, fill=(150, 150, 150))
for v in (0.0, 0.25, 0.5, 0.75, 1.0):
d.line([L, y(v * top), R, y(v * top)], fill=(70, 70, 76)); d.text((8, y(v * top) - 6), f"{v * top:.2f}", fill=(170, 170, 170))
def line(vals, col, labels):
n = len(vals) - 1
pts = [(L + (R - L) * i / max(1, n), y(v)) for i, v in enumerate(vals)]
d.line(pts, fill=col, width=2)
for i, (px, py) in enumerate(pts):
d.ellipse([px - 3, py - 3, px + 3, py + 3], fill=col)
if labels and i < n:
d.text((px - 10, py - 16 if i % 2 == 0 else py + 6), f"{vals[i]:.2f}", fill=(235, 235, 235))
d.text((px - 3, B + 8), str(i + 1), fill=(170, 170, 170))
if other is not None:
line([float(v) for v in other.flatten().tolist()], (120, 120, 200), False)
line(s, (255, 190, 70), True)
steps = s[:-1]
cnt = [sum(lo * top < v <= hi * top or (hi > 1 and v >= lo * top) for v in steps) for lo, hi, _, _ in zones]
d.text((L, 10), f"{len(steps)} steps: {cnt[0]} on layout / identity (> {0.85 * top:.2f}), {cnt[1]} on shapes, {cnt[2]} on detail (< {0.5 * top:.2f})"
f" last jump {steps[-1]:.2f} -> {s[-1]:.2f}", fill=(235, 235, 235))
d.text((R - 24, B + 8), "step", fill=(170, 170, 170))
text = (f"{len(steps)} steps | layout/identity {cnt[0]}, shapes {cnt[1]}, detail {cnt[2]} | sigmas: " + ", ".join(f"{v:.3f}" for v in s))
return torch.from_numpy(np.asarray(img).astype("float32") / 255.0)[None], text
class BFSSigmasPreview:
"""Draws a sigma schedule: where the sampler's steps are spent (pass-through)."""
@classmethod
def INPUT_TYPES(cls):
return {"required": {"sigmas": ("SIGMAS", {"tooltip": "The schedule that goes to the sampler (after the scheduler and the shift)."})},
"optional": {"compare": ("SIGMAS", {"tooltip": "Another schedule drawn behind it, e.g. the same steps with another shift."})}}
RETURN_TYPES = ("SIGMAS", "IMAGE", "STRING")
RETURN_NAMES = ("sigmas", "chart", "text")
OUTPUT_TOOLTIPS = ("The same sigmas, untouched.", "The chart: connect a Preview Image.", "The values and the step count per zone.")
FUNCTION = "show"
CATEGORY = "BFS/shot loop"
DESCRIPTION = ("Shows a sigma schedule: the noise level at the start of each step and how many steps fall on layout / "
"identity (high noise), shapes and detail (low noise). With H3's video shift of 12 most steps sit at "
"the top; lowering the shift moves steps toward detail. Changes nothing: sigmas pass through.")
def show(self, sigmas, compare=None):
chart, text = sigmas_chart(sigmas, compare)
return (sigmas, chart, text)
NODE_CLASS_MAPPINGS = {"BFSShotNoise": BFSShotNoise, "BFSSigmasPreview": BFSSigmasPreview}
NODE_DISPLAY_NAME_MAPPINGS = {"BFSShotNoise": "BFS Shot Noise", "BFSSigmasPreview": "BFS Sigmas Preview"}
+159
View File
@@ -0,0 +1,159 @@
"""Optional blur of the source person in what the model SEES of the source (aligned guide, native reference video,
duet panel): the model still reads the pose, the motion and (if kept sharp) the lips, but has no face to copy, so the
identity comes from the reference. JalenBrunson's trick for reliable ref2v swaps. The kept area (inpaint) and the
output are never blurred.
Modes are "<region> (<what stays sharp>)":
region face / head + hands / person: found by the segmenter, every person in the frame;
masked area: the shot's own mask (the person being replaced, with its expand), no extra segmentation
sharp keep mouth: lips, teeth and tongue; keep lips + tongue: the teeth are blurred too (always Sapiens2);
mouth too: nothing stays sharp
Segmenter: SAM 3.1 (text prompts, already used by the planner) or Sapiens2 (body-part classes, one pass, finer lips).
Effect: blur, or invert the colours (keeps the shapes and the expression, loses skin tone and colours).
"""
from __future__ import annotations
import torch
_REGIONS = ("face", "head + hands", "person", "masked area")
_SHARP = ("keep mouth", "keep lips + tongue", "mouth too")
SOURCE_BLUR = ["off"] + [f"{r} ({k})" for k in _SHARP for r in _REGIONS]
SEGMENTERS = ["SAM 3.1", "Sapiens2"]
EFFECTS = ["blur", "invert colours"]
_SAM_PARTS = {"face": ("face",), "head + hands": ("head", "hand"), "person": ("person",)}
_SAP_FACE = ("Face_Neck", "Eyeglass", "Lower_Lip", "Upper_Lip", "Lower_Teeth", "Upper_Teeth", "Tongue")
_SAP_PARTS = {"face": _SAP_FACE, "head + hands": _SAP_FACE + ("Hair", "Left_Hand", "Right_Hand")}
_SAP_SHARP = {"keep mouth": ("Lower_Lip", "Upper_Lip", "Lower_Teeth", "Upper_Teeth", "Tongue"),
"keep lips + tongue": ("Lower_Lip", "Upper_Lip", "Tongue")}
_CACHE: dict = {}
def parse(mode: str) -> tuple[str, str] | None:
"""'face (keep mouth)' -> ('face', 'keep mouth'); None for off / unknown."""
if not mode or "(" not in mode:
return None
region, sharp = mode.rstrip(")").split(" (", 1)
return (region, sharp) if region in _REGIONS and sharp in _SHARP else None
def _sl():
try:
from . import bfs_shot_loop as SL
except ImportError:
import bfs_shot_loop as SL
return SL
def _sap():
try:
from . import bfs_sapiens as SP
except ImportError:
import bfs_sapiens as SP
return SP
def _small(frames: torch.Tensor, side: int = 640) -> torch.Tensor:
H, W = frames.shape[1:3]
s = side / max(H, W)
x = frames[..., :3].float()
if s >= 1:
return x
return torch.nn.functional.interpolate(x.movedim(-1, 1), size=(max(32, int(H * s) // 2 * 2), max(32, int(W * s) // 2 * 2)),
mode="bilinear", align_corners=False).movedim(1, -1)
def _grow(m: torch.Tensor, px: int) -> torch.Tensor:
if px <= 0:
return m
return torch.nn.functional.max_pool2d(m[:, None], 2 * px + 1, stride=1, padding=px)[:, 0]
def blur_mask(frames: torch.Tensor, mode: str, log: str = "", segmenter: str = "SAM 3.1",
area: torch.Tensor | None = None) -> torch.Tensor | None:
"""[N,h,w] float mask (working size) of what to blur: the region for `mode` minus what stays sharp (grown a
little). `area` [N,H,W]: the shot's own mask, for the 'masked area' region."""
pr = parse(mode)
if not pr:
return None
region, sharp = pr
if region == "masked area" and area is None:
if log:
print(f"{log}: source blur 'masked area' needs a mask on the shot (SAM 3 text or points): blurring the "
"whole person instead", flush=True)
region = "person"
SL = _sl()
small = _small(frames)
h, w = small.shape[1:3]
sap_region = region != "masked area" and segmenter == "Sapiens2"
sap_sharp = sharp == "keep lips + tongue" or (sharp == "keep mouth" and segmenter == "Sapiens2")
key = (region, sharp, segmenter, tuple(small.shape), SL._image_key(frames[:1]), SL._image_key(frames[-1:]),
SL._image_key(frames[frames.shape[0] // 2:frames.shape[0] // 2 + 1]),
None if area is None else (tuple(area.shape), float(area.float().sum())))
if key in _CACHE:
return _CACHE[key]
cls = _sap().segment(small) if sap_region or sap_sharp else None
if region == "masked area":
m = torch.nn.functional.interpolate(area[:, None].float(), size=(h, w), mode="nearest")[:, 0].clamp(0, 1)
m_grow = 0 # already grown by the mask's own expand
elif sap_region:
m = (cls > 0).float() if region == "person" else _sap().parts(cls, _SAP_PARTS[region])
m_grow = max(1, int(min(h, w) * 0.01))
else:
m = torch.zeros(small.shape[:3])
for p in _SAM_PARTS[region]:
m = torch.maximum(m, (SL.segment_frames(small, dict(SL.DEFAULT_MASK, text=p, max_objects=8, threshold=0.4)) > 0.5).float())
m_grow = max(1, int(min(h, w) * 0.015))
if sharp == "mouth too":
keep, found = torch.zeros_like(m), None
elif sap_sharp:
keep = _sap().parts(cls, _SAP_SHARP[sharp])
keep = _grow(keep, max(1, int(min(h, w) * 0.005)))
found = float(keep.sum()) > 0
else:
mouth = (SL.segment_frames(small, dict(SL.DEFAULT_MASK, text="mouth", max_objects=8, threshold=0.3)) > 0.5).float()
keep = _grow(mouth, max(1, int(min(h, w) * 0.012)))
found = float(mouth.sum()) > 0
out = (_grow(m, m_grow) * (1 - keep)).clamp(0, 1)
if log:
kept = "nothing kept sharp" if found is None else \
f"{sharp[5:]} kept ({'found' if found else 'not found: the whole region is blurred'})"
used = "shot mask" if region == "masked area" else "Sapiens2" if sap_region else "SAM 3.1"
print(f"{log}: source blur '{region}' ({used}{' + Sapiens2 lips' if sap_sharp and not sap_region else ''}): "
f"{float(m.mean()) * 100:.1f}% of the frame, {kept}", flush=True)
if len(_CACHE) > 8:
_CACHE.pop(next(iter(_CACHE)))
_CACHE[key] = out
return out
def _soft(m: torch.Tensor, H: int, W: int, N: int) -> torch.Tensor:
"""The working-size mask at the frame size with a soft edge, one per frame ([N,H,W,1])."""
mm = torch.nn.functional.interpolate(m[:, None], size=(H, W), mode="bilinear", align_corners=False)
mm = torch.nn.functional.avg_pool2d(mm, 9, stride=1, padding=4)[:, 0, ..., None].clamp(0, 1)
if mm.shape[0] != N: # one mask per frame; a shorter list holds its last one
mm = mm[[min(i, mm.shape[0] - 1) for i in range(N)]]
return mm
def blur_source(frames: torch.Tensor | None, mode: str, strength: float = 1.0, log: str = "",
segmenter: str = "SAM 3.1", effect: str = "blur", area: torch.Tensor | None = None) -> torch.Tensor | None:
"""The frames with the source person blurred (or colour-inverted) per `mode`. `strength` 0-1 sets how coarse the
blur is; `area`: the shot's mask for the 'masked area' modes."""
if frames is None or not mode or mode == "off" or strength <= 0:
return frames
m = blur_mask(frames, mode, log, segmenter, area)
if m is None or float(m.sum()) == 0:
return frames
x = frames[..., :3].float()
N, H, W = x.shape[:3]
if effect == "invert colours":
mm = _soft(m, H, W, N)
return x * (1 - mm) + (1 - x) * mm
# a heavy blur: down to ~1/12-1/40 of the size and back, then a soft pass; blocky detail never survives it
f = max(4, int(min(H, W) * (0.025 + 0.06 * float(strength))))
low = torch.nn.functional.interpolate(x.movedim(-1, 1), size=(max(2, H // f), max(2, W // f)), mode="area")
blur = torch.nn.functional.interpolate(low, size=(H, W), mode="bilinear", align_corners=False)
blur = torch.nn.functional.avg_pool2d(blur, 5, stride=1, padding=2).movedim(1, -1)
mm = _soft(m, H, W, N)
out = x * (1 - mm) + blur * mm
del low, blur, mm
return out
+341
View File
@@ -0,0 +1,341 @@
"""BFS Video Compare: two videos in one player (wipe, overlay or side by side, either one on top) plus a per-pixel
difference map. Made for BFS Shot Join's `guide` and `images` outputs, but any two IMAGE batches work.
The difference is measured here on the frames themselves (not on the browser's compressed preview): per pixel, the mean
absolute RGB difference, drawn black (equal) -> red -> yellow -> white (very different), scaled by `gain`. The player
gets three temporary MP4s (guide, target, difference) and the per-frame difference, so it can jump to the worst frames.
"""
import math
import os
import re
import subprocess
import uuid
import numpy as np
import torch
import torch.nn.functional as F
import folder_paths
try:
from . import bfs_video_io as VIO
except ImportError: # loaded as a plain module (the tests): the file next to this one
import importlib.util as _ilu
_vio = _ilu.spec_from_file_location("bfs_video_io", os.path.join(os.path.dirname(os.path.abspath(__file__)),
"bfs_video_io.py"))
VIO = _ilu.module_from_spec(_vio)
_vio.loader.exec_module(VIO)
CHUNK = 16
def _scale(x: torch.Tensor, h: int, w: int) -> torch.Tensor:
if tuple(x.shape[1:3]) == (h, w):
return x
return F.interpolate(x.permute(0, 3, 1, 2), size=(h, w), mode="bilinear", align_corners=False,
antialias=True).permute(0, 2, 3, 1).clamp(0, 1)
def heat(v: torch.Tensor) -> torch.Tensor:
"""[..., H, W] in 0..1 -> RGB: black (0), red, yellow, white (1)."""
return torch.stack([(3 * v).clamp(0, 1), (3 * v - 1).clamp(0, 1), (3 * v - 2).clamp(0, 1)], -1)
def difference(guide: torch.Tensor, target: torch.Tensor, gain: float = 4.0):
"""Both [N, H, W, 3] in 0..1 (same size). Returns (map [N, H, W, 3], mean abs difference per frame in 0..255,
mean squared error per frame in 0..1)."""
d = (guide[..., :3].float() - target[..., :3].float()).abs()
mae = d.mean(dim=(1, 2, 3)) * 255
mse = (d * d).mean(dim=(1, 2, 3))
return heat((d.mean(-1) * float(gain)).clamp(0, 1)), mae, mse
def psnr(mse: float) -> float:
return float("inf") if mse <= 0 else 10 * math.log10(1.0 / mse)
MAX_REFS = 12
def ref_pictures(reference) -> list:
"""The reference pictures, one [H,W,3] each: from a list of IMAGEs of any sizes (e.g. the planner's or BFS Shot
Unpack's refs) and / or IMAGE batches."""
out = []
for item in reference if isinstance(reference, list) else [reference]:
if item is not None:
out += [item[i] for i in range(int(item.shape[0]))]
return out[:MAX_REFS]
def save_refs(reference, out_dir: str, height: int) -> list:
"""The reference pictures as PNGs no taller than the player (no upscale) -> file names."""
from PIL import Image
names = []
for i, pic in enumerate(ref_pictures(reference)):
im = Image.fromarray((pic[..., :3].float().clamp(0, 1).cpu().numpy() * 255).round().astype(np.uint8))
if im.height > height:
im = im.resize((max(1, round(im.width * height / im.height)), height), Image.LANCZOS)
names.append(f"ref_{i}.png")
im.save(os.path.join(out_dir, names[-1]))
return names
class BFSVideoCompare:
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"guide": ("IMAGE", {"tooltip": "The original video (e.g. BFS Shot Join's 'guide' output)."}),
"target": ("IMAGE", {"tooltip": "The result to compare with it (e.g. BFS Shot Join's 'images')."}),
},
"optional": {
"fps": ("FLOAT", {"default": 24.0, "min": 1.0, "max": 240.0, "step": 0.001, "tooltip":
"Frame rate of both videos (connect BFS Shot Join's 'fps')."}),
"audio": ("AUDIO", {"tooltip": "Soundtrack for the player (e.g. BFS Shot Join's 'audio')."}),
"gain": ("FLOAT", {"default": 2.0, "min": 0.5, "max": 64.0, "step": 0.5, "tooltip":
"How strongly differences light up in the map: a mean RGB difference of 1/gain "
"(128/255 at 2) is already white. Raise it to see small changes."}),
"size": (["guide", "target"], {"default": "guide", "tooltip":
"Which video's size the other is scaled to before comparing. "
"The Join's guide is at comparison_height, so 'guide' is lighter."}),
"preview_height": ("INT", {"default": 720, "min": 0, "max": 4096, "step": 16, "tooltip":
"Height of the player's videos (0 = the compared size). Only the preview: "
"the difference is measured before it."}),
"reference": ("IMAGE", {"tooltip": "Optional reference pictures (e.g. the people swapped in), shown in a "
"strip below the videos, numbered as <Picture n>, and in what ⬇ save / ● rec "
"download. Takes the refs list of BFS Shot Planner / Unpack (pictures of any "
"size) or a batch; up to 12."}),
"label_guide": ("STRING", {"default": "guide"}),
"label_target": ("STRING", {"default": "target"}),
},
}
# lists in: the reference can be the refs list of the planner / Unpack (pictures of different sizes); the other
# inputs take their first item
INPUT_IS_LIST = True
RETURN_TYPES = ("IMAGE", "STRING", "FLOAT")
RETURN_NAMES = ("difference", "report", "mean_difference")
OUTPUT_TOOLTIPS = ("The difference map, one frame per compared frame (black = equal, white = very different).",
"Frame count, sizes, mean difference, PSNR and the most / least different frames.",
"Mean absolute RGB difference over all frames, 0-255.")
OUTPUT_NODE = True
FUNCTION = "compare"
CATEGORY = "BFS/shot loop"
DESCRIPTION = ("Play two videos in one player (wipe, overlay or side by side; choose which is on top) and see "
"where they differ, pixel by pixel.")
def compare(self, guide, target, fps=24.0, audio=None, gain=2.0, size="guide", preview_height=720,
label_guide="guide", label_target="target", reference=None):
one = lambda v: v[0] if isinstance(v, list) and v else v
guide, target, fps, audio, gain, size, preview_height, label_guide, label_target = map(
one, (guide, target, fps, audio, gain, size, preview_height, label_guide, label_target))
notes = []
n = min(guide.shape[0], target.shape[0])
if guide.shape[0] != target.shape[0]:
notes.append(f"lengths differ (guide {guide.shape[0]}, target {target.shape[0]} frames): "
f"compared the first {n}")
ref = guide if size == "guide" else target
H, W = int(ref.shape[1]), int(ref.shape[2])
for name, v in (("guide", guide), ("target", target)):
if tuple(v.shape[1:3]) != (H, W):
notes.append(f"{name} scaled from {v.shape[2]}x{v.shape[1]} to {W}x{H}")
ph = min(H, int(preview_height)) if preview_height and preview_height > 0 else H
PH, PW = max(16, ph // 2 * 2), max(16, int(round(W * ph / H)) // 2 * 2)
out_dir = os.path.join(folder_paths.get_temp_directory(), "bfs_compare", uuid.uuid4().hex[:12])
os.makedirs(out_dir, exist_ok=True)
names = {"guide": "guide.mp4", "target": "target.mp4", "difference": "difference.mp4"}
writers = {k: VIO.Mp4Writer(os.path.join(out_dir, f), PW, PH, float(fps), audio if k == "target" else None,
crf=14, who="BFS Video Compare") for k, f in names.items()}
maps, mae, mse = [], [], []
try:
for a in range(0, n, CHUNK):
e = min(a + CHUNK, n) # the longer video stops where the shorter one ends
g = _scale(guide[a:e, ..., :3].float(), H, W)
t = _scale(target[a:e, ..., :3].float(), H, W)
m, e, s = difference(g, t, gain)
maps.append(m)
mae.append(e)
mse.append(s)
for k, v in (("guide", g), ("target", t), ("difference", m)):
writers[k].write(_scale(v, PH, PW))
finally:
for w in writers.values():
w.close()
diff_map = torch.cat(maps, 0)
mae, mse = torch.cat(mae), torch.cat(mse)
mean = float(mae.mean())
worst, best = int(mae.argmax()), int(mae.argmin())
report = "\n".join([
f"{label_guide} vs {label_target}: {n} frames at {fps:g} fps, compared at {W}x{H}",
f"mean difference {mean:.2f}/255, PSNR {psnr(float(mse.mean())):.2f} dB",
f"most different: frame {worst} ({worst / fps:.2f} s) {float(mae[worst]):.2f}/255",
f"least different: frame {best} ({best / fps:.2f} s) {float(mae[best]):.2f}/255",
*notes])
print(f"[BFS Video Compare] {report}", flush=True)
sub = os.path.relpath(out_dir, folder_paths.get_temp_directory())
files = {k: {"filename": f, "subfolder": sub, "type": "temp"} for k, f in names.items()}
refs = [{"filename": f, "subfolder": sub, "type": "temp"} for f in save_refs(reference, out_dir, PH)]
ui = {"bfs_compare": [{**files, "fps": float(fps), "frames": n, "width": PW, "height": PH,
"labels": [label_guide, label_target], "mae": [round(float(x), 2) for x in mae],
"refs": refs,
"report": report}]}
return {"ui": ui, "result": (diff_map, report, mean)}
# ---------------------------------------------------------------------------- save / record: the browser's file -> a shareable one
EXPORT_FORMATS = { # what the player's format list offers; "webm" never reaches the server (the browser's own file)
"mp4": "MP4 (H.264)",
"small": "MP4 small (under 10 MB)",
"gif": "GIF (under 10 MB, no sound)",
}
LIMIT_BYTES = int(9.5 * 1024 * 1024) # under the 10 MB many chat apps accept for free
_ID = re.compile(r"^[0-9a-f]{8,32}$")
def _export_dir() -> str:
d = os.path.join(folder_paths.get_temp_directory(), "bfs_compare", "exports")
os.makedirs(d, exist_ok=True)
return d
def write_part(rid: str, offset: int, data: bytes) -> int:
"""One piece of a recording uploaded by the player (pieces keep big recordings under the upload limit)."""
if not _ID.match(rid or "") or offset < 0:
raise ValueError("bad recording id / offset")
path = os.path.join(_export_dir(), f"{rid}.rec")
with open(path, "r+b" if os.path.exists(path) else "wb") as f:
f.seek(offset)
f.write(data)
return len(data)
def _run(cmd: list) -> None:
r = subprocess.run(cmd, capture_output=True, text=True)
if r.returncode != 0:
raise RuntimeError(f"ffmpeg failed: {r.stderr.strip()[-600:]}")
def encode(src: str, fmt: str, seconds: float, out_base: str) -> str:
"""The recording `src` (WebM / fragmented MP4 from MediaRecorder, which many apps won't play or preview) as
`fmt` -> the written path. Both MP4s are H.264 + AAC, yuv420p, faststart. mp4: CRF 18. small: the bitrate set so
the file stays under 9.5 MB (smaller frame when it gets low). gif: palette per video, 15 fps, made smaller until
it fits 9.5 MB."""
if fmt not in EXPORT_FORMATS:
raise ValueError(f"unknown format {fmt!r}")
exe = VIO.ffmpeg_bin()
if exe is None:
return _encode_av(src, fmt, out_base)
seconds = max(0.5, float(seconds or 0))
even = "scale=trunc(iw/2)*2:trunc(ih/2)*2"
if fmt == "gif":
out = out_base + ".gif"
for width, fps in ((720, 15), (540, 15), (480, 12), (360, 12), (270, 10)):
vf = f"fps={fps},scale='min({width},iw)':-2:flags=lanczos,split[a][b];[a]palettegen=stats_mode=diff[p];[b][p]paletteuse=dither=bayer:bayer_scale=4"
_run([exe, "-y", "-v", "error", "-i", src, "-vf", vf, "-loop", "0", out])
if os.path.getsize(out) <= LIMIT_BYTES:
break
return out
out = out_base + ".mp4"
if fmt == "mp4":
_run([exe, "-y", "-v", "error", "-i", src, "-vf", even, "-c:v", "libx264", "-preset", "medium", "-crf", "18",
"-pix_fmt", "yuv420p", "-c:a", "aac", "-b:a", "192k", "-movflags", "+faststart", out])
return out
audio_k = 128
for shrink in (1.0, 0.92, 0.8): # a VBR encode can overshoot a little: a second, lower try
kbps = int(min(8000, max(250, LIMIT_BYTES * 8 / 1000 / seconds * shrink - audio_k)))
vf = even if kbps >= 1500 else "scale=-2:'min(720,ih)':flags=lanczos," + even if kbps >= 700 else \
"scale=-2:'min(480,ih)':flags=lanczos," + even
_run([exe, "-y", "-v", "error", "-i", src, "-vf", vf, "-c:v", "libx264", "-preset", "medium",
"-b:v", f"{kbps}k", "-maxrate", f"{int(kbps * 1.4)}k", "-bufsize", f"{kbps * 2}k", "-pix_fmt", "yuv420p",
"-r", "30", "-c:a", "aac", "-b:a", f"{audio_k}k", "-ac", "2", "-movflags", "+faststart", out])
if os.path.getsize(out) <= LIMIT_BYTES:
break
return out
def _encode_av(src: str, fmt: str, out_base: str) -> str:
"""No ffmpeg program: decode with PyAV and write an MP4 (no sound) or a GIF (PIL); 'small' at most 720p."""
import av
from PIL import Image
frames, fps = [], 30.0
with av.open(src) as c:
st = c.streams.video[0]
fps = float(st.average_rate or st.guessed_rate or 30)
for f in c.decode(st):
frames.append(f.to_ndarray(format="rgb24"))
if not frames:
raise RuntimeError("the recording has no frames")
h, w = frames[0].shape[:2]
if fmt == "gif":
out = out_base + ".gif"
step = max(1, round(fps / 12))
for side in (540, 400, 300, 220):
k = min(1.0, side / max(h, w))
ims = [Image.fromarray(x).resize((max(2, int(w * k)), max(2, int(h * k))), Image.LANCZOS) for x in frames[::step]]
ims[0].save(out, save_all=True, append_images=ims[1:], duration=int(1000 * step / fps), loop=0, optimize=True)
if os.path.getsize(out) <= LIMIT_BYTES:
break
return out
out = out_base + ".mp4"
k = 1.0 if fmt == "mp4" else min(1.0, 720 / h)
W2, H2 = int(w * k) // 2 * 2, int(h * k) // 2 * 2
wr = VIO.Mp4Writer(out, W2, H2, fps, None, crf=18 if fmt == "mp4" else 27, who="BFS Video Compare")
try:
for a in range(0, len(frames), 16):
x = torch.from_numpy(np.stack(frames[a:a + 16])).float() / 255
wr.write(_scale(x, H2, W2))
finally:
wr.close()
print("[BFS Video Compare] no ffmpeg program: saved without sound", flush=True)
return out
def finish_export(rid: str, fmt: str, seconds: float, name: str) -> dict:
src = os.path.join(_export_dir(), f"{rid}.rec")
if not _ID.match(rid or "") or not os.path.exists(src):
raise ValueError("recording not found")
stem = re.sub(r"[^\w.-]+", "_", os.path.splitext(name or "bfs_compare")[0])[:80] or "bfs_compare"
try:
out = encode(src, fmt, seconds, os.path.join(_export_dir(), f"{stem}_{rid[:6]}"))
finally:
os.remove(src)
sub = os.path.relpath(os.path.dirname(out), folder_paths.get_temp_directory()).replace(os.sep, "/")
mb = os.path.getsize(out) / 1024 / 1024
print(f"[BFS Video Compare] saved {os.path.basename(out)} ({EXPORT_FORMATS[fmt]}, {mb:.1f} MB)", flush=True)
return {"filename": os.path.basename(out), "subfolder": sub, "type": "temp", "mb": round(mb, 2)}
try:
import asyncio
from aiohttp import web
from server import PromptServer
@PromptServer.instance.routes.post("/bfs/compare/part")
async def _bfs_compare_part(request):
try:
data = await request.read()
n = await asyncio.get_running_loop().run_in_executor(
None, lambda: write_part(request.query.get("id", ""), int(request.query.get("offset", "-1")), data))
except Exception as exc: # noqa: BLE001
return web.json_response({"error": f"{type(exc).__name__}: {exc}"}, status=400)
return web.json_response({"written": n})
@PromptServer.instance.routes.post("/bfs/compare/export")
async def _bfs_compare_export(request):
try:
b = await request.json()
r = await asyncio.get_running_loop().run_in_executor(
None, lambda: finish_export(b.get("id", ""), b.get("format", ""), float(b.get("seconds") or 0), b.get("name", "")))
except Exception as exc: # noqa: BLE001
return web.json_response({"error": f"{type(exc).__name__}: {exc}"}, status=400)
return web.json_response(r)
except Exception as _exc: # noqa: BLE001 - outside ComfyUI (tests)
if "PromptServer" not in repr(_exc) and "server" not in repr(_exc):
print(f"[BFS Video Compare] export routes not registered: {_exc!r}", flush=True)
NODE_CLASS_MAPPINGS = {"BFSVideoCompare": BFSVideoCompare}
NODE_DISPLAY_NAME_MAPPINGS = {"BFSVideoCompare": "BFS Video Compare"}
+217
View File
@@ -0,0 +1,217 @@
"""Video files for the BFS nodes, with or without an ffmpeg program.
The ffmpeg program is found on PATH, else the one bundled by imageio-ffmpeg; with neither (e.g. a Windows portable
ComfyUI), MP4s are written and read through PyAV, the library ComfyUI's own Save Video uses, so it is always there.
"""
import json
import math
import os
import re
import shutil
import subprocess
from fractions import Fraction
import numpy as np
import torch
import torch.nn.functional as F
_BIN: dict = {}
def ffmpeg_bin() -> str | None:
"""The ffmpeg program to run, or None."""
if "ffmpeg" not in _BIN:
exe = shutil.which("ffmpeg")
if exe is None:
try:
import imageio_ffmpeg
exe = imageio_ffmpeg.get_ffmpeg_exe()
except Exception: # noqa: BLE001 - optional
exe = None
_BIN["ffmpeg"] = exe
return _BIN["ffmpeg"]
def ffprobe_bin() -> str | None:
if "ffprobe" not in _BIN:
_BIN["ffprobe"] = shutil.which("ffprobe")
return _BIN["ffprobe"]
def _wav(audio: dict) -> str:
import tempfile
import wave
fd, path = tempfile.mkstemp(suffix=".wav")
os.close(fd)
wf = audio["waveform"][0].clamp(-1, 1)
with wave.open(path, "wb") as f:
f.setnchannels(int(wf.shape[0]))
f.setsampwidth(2)
f.setframerate(int(audio["sample_rate"]))
f.writeframes((wf.t().contiguous() * 32767).round().short().numpy().tobytes())
return path
def _has_audio(audio) -> bool:
try:
return audio is not None and audio["waveform"].ndim == 3 and audio["waveform"].shape[-1] > 0
except Exception: # noqa: BLE001
return False
class Mp4Writer:
"""Frames streamed into an MP4 (H.264, yuv420p, even size) with the soundtrack, one chunk at a time.
metadata: tags like Save Video's (prompt, workflow...), values JSON-encoded when not strings."""
def __init__(self, path: str, w: int, h: int, fps: float, audio: dict | None = None, crf: int = 17,
metadata: dict | None = None, who: str = "BFS"):
self.path, self.who, self.wav, self.meta, self.proc, self.av = path, who, None, None, None, None
os.makedirs(os.path.dirname(path) or ".", exist_ok=True)
exe = ffmpeg_bin()
if exe is None:
self._open_av(w, h, fps, audio, crf, metadata)
return
inputs = ["-f", "rawvideo", "-pix_fmt", "rgb24", "-s", f"{w}x{h}", "-r", f"{fps:g}", "-i", "-"]
maps, outs = ["-map", "0:v"], []
if _has_audio(audio):
self.wav = _wav(audio)
inputs += ["-i", self.wav]
maps += ["-map", "1:a"]
outs += ["-c:a", "aac", "-b:a", "192k"]
if metadata: # through an ffmetadata file
import tempfile
fd, self.meta = tempfile.mkstemp(suffix=".txt")
esc = lambda v: re.sub(r"([=;#\\\n])", r"\\\1", v)
with os.fdopen(fd, "w", encoding="utf-8") as f:
f.write(";FFMETADATA1\n")
for k, v in metadata.items():
f.write(f"{esc(str(k))}={esc(v if isinstance(v, str) else json.dumps(v))}\n")
maps += ["-map_metadata", str(inputs.count("-i"))]
inputs += ["-f", "ffmetadata", "-i", self.meta]
cmd = [exe, "-v", "error", "-y", *inputs, *maps, *outs,
"-c:v", "libx264", "-preset", "medium", "-crf", str(int(crf)), "-pix_fmt", "yuv420p",
"-vf", "pad=ceil(iw/2)*2:ceil(ih/2)*2", "-movflags", "+use_metadata_tags+faststart", path]
self.proc = subprocess.Popen(cmd, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
def _open_av(self, w, h, fps, audio, crf, metadata):
import av
self.av = av.open(self.path, mode="w", options={"movflags": "use_metadata_tags+faststart"})
for k, v in (metadata or {}).items():
self.av.metadata[str(k)] = v if isinstance(v, str) else json.dumps(v)
self.vs = self.av.add_stream("h264", rate=Fraction(round(float(fps) * 1000), 1000))
self.vs.width, self.vs.height = w + w % 2, h + h % 2
self.vs.pix_fmt = "yuv420p"
self.vs.options = {"crf": str(int(crf)), "preset": "medium"}
self.audio = audio if _has_audio(audio) else None
if self.audio is not None:
ch = int(self.audio["waveform"].shape[1])
self.layout = {1: "mono", 2: "stereo"}.get(ch, "stereo")
self.as_ = self.av.add_stream("aac", rate=int(self.audio["sample_rate"]), layout=self.layout)
def write(self, frames: torch.Tensor) -> None:
data = (frames[..., :3].clamp(0, 1) * 255).round().to(torch.uint8).contiguous().numpy()
if self.proc is not None:
self.proc.stdin.write(data.tobytes())
return
import av
h, w = data.shape[1:3]
for fr in data:
if (h % 2) or (w % 2):
fr = np.pad(fr, ((0, h % 2), (0, w % 2), (0, 0)), mode="edge")
for p in self.vs.encode(av.VideoFrame.from_ndarray(fr, format="rgb24").reformat(format="yuv420p")):
self.av.mux(p)
def close(self) -> str:
if self.proc is not None:
self.proc.stdin.close()
err = self.proc.stderr.read().decode(errors="replace")
self.proc.wait()
self._cleanup()
if self.proc.returncode != 0:
raise RuntimeError(f"{self.who}: ffmpeg could not write {self.path}: {err.strip()[-400:]}")
return self.path
import av
try:
for p in self.vs.encode(None):
self.av.mux(p)
if self.audio is not None:
wf = self.audio["waveform"][0, :2].float().clamp(-1, 1).contiguous().numpy()
fr = av.AudioFrame.from_ndarray(wf, format="fltp", layout=self.layout)
fr.sample_rate, fr.pts = int(self.audio["sample_rate"]), 0
for p in self.as_.encode(fr):
self.av.mux(p)
for p in self.as_.encode(None):
self.av.mux(p)
finally:
self.av.close()
return self.path
def abort(self) -> None:
try:
if self.proc is not None:
self.proc.kill()
elif self.av is not None:
self.av.close()
except Exception: # noqa: BLE001
pass
self._cleanup()
def _cleanup(self) -> None:
for f in (self.wav, self.meta):
if f and os.path.exists(f):
os.remove(f)
def read_video_scaled(path: str, budget: float) -> tuple[torch.Tensor, float]:
"""A video file as IMAGE, scaled down (never up) so it takes at most `budget` bytes; streamed into one tensor.
Returns (frames, scale)."""
exe, probe = ffmpeg_bin(), ffprobe_bin()
if exe is None or probe is None:
return _read_video_av(path, budget)
info = json.loads(subprocess.run([probe, "-v", "error", "-select_streams", "v:0", "-count_packets", "-show_entries",
"stream=width,height,nb_read_packets", "-of", "json", path],
capture_output=True, check=True).stdout)["streams"][0]
w, h, n = int(info["width"]), int(info["height"]), int(info["nb_read_packets"])
s = min(1.0, math.sqrt(max(budget, 1.0) / max(1, n * h * w * 3 * 4)))
W, H = max(16, int(w * s) // 2 * 2), max(16, int(h * s) // 2 * 2)
out = torch.empty((n, H, W, 3), dtype=torch.float32)
proc = subprocess.Popen([exe, "-v", "error", "-i", path, "-vf", f"scale={W}:{H}:flags=area", "-f", "rawvideo",
"-pix_fmt", "rgb24", "-"], stdout=subprocess.PIPE)
fb, i = W * H * 3, 0
while i < n:
raw = proc.stdout.read(fb * 16)
if not raw:
break
k = len(raw) // fb
out[i:i + k] = torch.frombuffer(bytearray(raw[:k * fb]), dtype=torch.uint8).view(k, H, W, 3).float() / 255.0
i += k
proc.stdout.close()
proc.wait()
return out[:max(1, i)], s
def _read_video_av(path: str, budget: float) -> tuple[torch.Tensor, float]:
import av
with av.open(path) as c:
st = c.streams.video[0]
w, h = int(st.codec_context.width), int(st.codec_context.height)
n = int(st.frames or 0) or int(round(float(st.duration * st.time_base) * float(st.average_rate)) if st.duration
else 0)
s = min(1.0, math.sqrt(max(budget, 1.0) / max(1, max(n, 1) * h * w * 3 * 4)))
W, H = max(16, int(w * s) // 2 * 2), max(16, int(h * s) // 2 * 2)
chunks, buf = [], []
for fr in c.decode(st):
buf.append(torch.from_numpy(fr.to_ndarray(format="rgb24")))
if len(buf) == 16:
chunks.append(_shrink(torch.stack(buf), H, W))
buf = []
if buf:
chunks.append(_shrink(torch.stack(buf), H, W))
return (torch.cat(chunks) if chunks else torch.zeros(1, H, W, 3)), s
def _shrink(x: torch.Tensor, h: int, w: int) -> torch.Tensor:
x = x.float() / 255.0
if tuple(x.shape[1:3]) == (h, w):
return x
return F.interpolate(x.permute(0, 3, 1, 2), size=(h, w), mode="area").permute(0, 2, 3, 1)
+5 -1
View File
@@ -67,7 +67,11 @@ def _dbg(*a):
def _get_face_app():
global _FACE_APP
if _FACE_APP is None:
from insightface.app import FaceAnalysis
try:
from insightface.app import FaceAnalysis
except ImportError as e:
raise RuntimeError("This node needs the optional insightface package: pip install insightface (with onnxruntime "
"or onnxruntime-gpu), then restart ComfyUI.") from e
providers = (["CUDAExecutionProvider", "CPUExecutionProvider"] if _USE_GPU else ["CPUExecutionProvider"])
app = FaceAnalysis(name="buffalo_l", providers=providers)
app.prepare(ctx_id=0 if _USE_GPU else -1, det_size=(640, 640))
+2 -4
View File
@@ -1,7 +1,7 @@
[project]
name = "bfsnodes"
description = "Custom ComfyUI nodes for reference-guided video: BFS Sampler (guide-driven IC-LoRAs, auto crop, mask inpainting, temporal chunking), LTX Multiple Controls / Identity Transfer / Edit Anything, Head-Swap and Anime2Real conditioning, MiniMax-H3 latent guides, RoPE layouts and loaders, LoRA Surgery, and Shot Planner / Shot Loop: split long videos into shots at camera cuts, face-tracked cast, SAM 3 mask and crop per shot (text or click points, one selection for the whole video), optional Qwen3-VL to suggest settings, describe references ({details}) and the person to replace ({target}), continuity between shots, H3 conditioning with an optional duet and prompt writer, join with the original timing, audio and a side-by-side comparison. H3 Duet: pin a source clip beside the video, training-free."
version = "1.69.0"
description = "Custom ComfyUI nodes for reference-guided video: BFS Sampler (guide-driven IC-LoRAs, auto crop, mask inpainting, temporal chunking), LTX Multiple Controls / Identity Transfer / Edit Anything, Head-Swap and Anime2Real conditioning, MiniMax-H3 latent guides, RoPE layouts and loaders, LoRA Surgery, and Shot Planner / Shot Loop (tabbed panel): split long videos into shots at camera cuts, face-tracked cast, SAM 3 mask per shot with full frame, paste, mask, crop and crop + mask generation modes, optional translucent Colored Mark overlay for manually selected regions, optional Qwen3-VL to suggest settings, describe references ({details}) and the person to replace ({target}), continuity between shots, H3 conditioning with an optional duet and prompt writer, join with the original timing, audio and a side-by-side comparison. H3 Duet: pin a source clip beside the video, training-free."
version = "1.116.0"
license = {file = "LICENSE"}
dependencies = [
"torch",
@@ -11,8 +11,6 @@ dependencies = [
"librosa",
"opencv-python",
"scenedetect",
"insightface==0.7.3",
"onnxruntime",
"transformers>=4.55.0",
"scipy",
]
-2
View File
@@ -4,6 +4,4 @@ Pillow
safetensors
librosa
opencv-python
insightface==0.7.3
onnxruntime
scenedetect
+48
View File
@@ -0,0 +1,48 @@
import importlib.util
import types
import unittest
from pathlib import Path
import torch
ROOT = Path(__file__).resolve().parents[1]
SPEC = importlib.util.spec_from_file_location("bfs_h3_history", ROOT / "bfs_h3_history.py")
HI = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(HI)
class Layout:
"""text 4 rows | guide cond 6 rows | history cond 4 rows | audio 3 | video 6, all at the target origin 4.0"""
def __init__(self):
self.segments = [(0, 4, "text"), (4, 10, "cond"), (10, 14, "cond"), (14, 17, "audio"), (17, 23, "video")]
p = torch.zeros(23, 3)
p[:4, 0] = torch.arange(4.)
p[4:, 0] = 4.0
self.position_ids = p
class HistoryTest(unittest.TestCase):
def setUp(self):
HI.history_span = lambda t: 10.0 * t
def test_target_moves_past_the_history_only(self):
lay = Layout()
kfs = [{"latent": torch.zeros(1, 24, 3, 2, 2)}, {"latent": torch.zeros(1, 24, 2, 2, 2), "anchor": "history"}]
self.assertTrue(HI.relocate(lay, kfs))
t = lay.position_ids[:, 0]
self.assertTrue(torch.equal(t[:4], torch.arange(4.))) # text untouched
self.assertTrue(bool((t[10:14] == 4.0).all())) # history stays at the origin
self.assertTrue(bool((t[4:10] == 24.0).all())) # the other guide moves with the video
self.assertTrue(bool((t[14:] == 24.0).all())) # audio and video move by the span (2 steps)
self.assertFalse(HI.relocate(lay, kfs)) # already moved: no double shift
def test_no_history_no_change(self):
lay = Layout()
before = lay.position_ids.clone()
self.assertFalse(HI.relocate(lay, [{"latent": torch.zeros(1, 24, 3, 2, 2)}]))
self.assertTrue(torch.equal(before, lay.position_ids))
if __name__ == "__main__":
unittest.main()
+40 -3
View File
@@ -72,10 +72,18 @@ class SidePanelTest(unittest.TestCase):
def test_layout_text_follows_the_share(self):
half = SP.layout_text({"position": "left", "h": 64, "w": 36, "strip_h": 0, "strip_w": 36})
self.assertIn("the LEFT half is the kept footage", half)
self.assertIn("the left half shows the original performance; the right half shows", half)
self.assertNotIn("kept", half)
narrow = SP.layout_text({"position": "left", "h": 64, "w": 72, "strip_h": 0, "strip_w": 36})
self.assertIn("narrow strip", narrow)
self.assertIn("LARGER", narrow)
self.assertIn("the left third shows", narrow)
self.assertIn("the right two thirds show", narrow)
self.assertIn("a wider view of the same action", narrow)
top = {"position": "top", "h": 48, "w": 36, "strip_h": 16, "strip_w": 0}
txt = SP.fill_layout("{panel_region} shows a man dancing, and {video_region} shows <Subject 1>.", top)
self.assertEqual(txt, "the top quarter shows a man dancing, and the bottom three quarters show <Subject 1>.")
p = SP.build_prompt("character swap", "", 1, 0)
for w in ("kept footage", "kept exactly", "in sync", "generated"):
self.assertNotIn(w, SP.fill_layout(p, top))
def test_apply_and_crop_round_trip(self):
lat = _latent()
@@ -121,6 +129,35 @@ class KeepMaskTest(unittest.TestCase):
self.assertEqual(float(m[0, 0, 3, 30, 36 + 18]), 1.0) # inside the mask: generated
self.assertEqual(float(m[0, 0, 3, 30, 5]), 0.0) # panel: pinned
def test_panel_static_covers_the_subject_every_frame(self):
seen = {}
def seg(imgs, spec):
seen["spec"] = spec
m = torch.zeros(imgs.shape[0], *imgs.shape[1:3]); m[:, 10:20, 10:20] = 1
return m
def tv(img, mask, grow=0.03, seed=0):
out = img.clone(); m = torch.nn.functional.interpolate(mask[None, None].float(), size=img.shape[1:3])[0, 0]
out[0][m > 0.5] = -1.0
return out
fake = types.SimpleNamespace(DEFAULT_MASK={"text": "", "threshold": 0.5}, segment_frames=seg, tv_static=tv,
add_setting=lambda text, k, swap: text + f" <Picture {k}> setting")
old, SP._shot_loop = SP._shot_loop, lambda: fake
try:
fr = torch.full((5, 64, 48, 3), 0.5)
self.assertIs(SP.static_over_panel(fr, "off"), fr)
out = SP.static_over_panel(fr, "face")
self.assertEqual(seen["spec"]["text"], "face")
self.assertEqual(tuple(out.shape), (5, 64, 48, 3))
self.assertTrue(bool((out[:, 15, 15] == -1).all()) and float(out[0, 40, 40, 0]) == 0.5)
st = SP.setting_from_clip(fr, 1.0, "dog")
self.assertEqual(tuple(st.shape), (1, 64, 48, 3)); self.assertEqual(seen["spec"]["text"], "dog")
self.assertTrue(SP.with_setting("x", 2, True).endswith("<Picture 2> setting"))
finally:
SP._shot_loop = old
if __name__ == "__main__":
unittest.main()
File diff suppressed because it is too large Load Diff
+126
View File
@@ -0,0 +1,126 @@
"""Source blur modes: '<region> (<what stays sharp>)', SAM 3.1 or Sapiens2, blur or inverted colours.
The segmenters are replaced by fixed masks, so this checks which pixels end up blurred, not the models."""
from __future__ import annotations
import os
import sys
import types
import unittest
import torch
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.insert(0, ROOT)
# only what the blur uses of the shot loop: the frame key and SAM 3 (a fixed 'mouth' / part mask here)
_SL = types.ModuleType("bfs_shot_loop")
_SL._image_key = lambda t: (tuple(t.shape), float(t.float().sum()))
_SL.DEFAULT_MASK = {}
_SL.SAM_CALLS = []
def _sam(frames, spec):
_SL.SAM_CALLS.append(spec["text"])
m = torch.zeros(frames.shape[:3])
if spec["text"] == "mouth":
m[:, 22:25, 28:36] = 1
else:
m[:, 10:30, 20:44] = 1
return m
_SL.segment_frames = _sam
sys.modules.setdefault("bfs_shot_loop", _SL)
import bfs_sapiens as SP # noqa: E402
import bfs_source_blur as B # noqa: E402
H, W = 48, 64
FACE = (slice(10, 30), slice(20, 44))
LIPS = (slice(22, 24), slice(28, 36))
TEETH = (slice(24, 25), slice(28, 36))
def fake_classes(frames, out_size=None, batch=4):
c = torch.zeros(frames.shape[0], H, W, dtype=torch.uint8)
c[:, 5:45, 10:54] = SP.ID["Upper_Clothing"]
c[(slice(None),) + FACE] = SP.ID["Face_Neck"]
c[(slice(None),) + LIPS] = SP.ID["Upper_Lip"]
c[(slice(None),) + TEETH] = SP.ID["Upper_Teeth"]
return c
class SourceBlurTest(unittest.TestCase):
def setUp(self):
B._CACHE.clear()
self._seg = SP.segment
SP.segment = fake_classes
self.frames = torch.rand(3, H, W, 3)
def tearDown(self):
SP.segment = self._seg
def mask(self, mode, **kw):
return B.blur_mask(self.frames, mode, segmenter=kw.pop("segmenter", "Sapiens2"), **kw)
def test_saved_workflows_keep_their_values(self):
for old in ("off", "face (keep mouth)", "head + hands (keep mouth)", "person (keep mouth)"):
self.assertIn(old, B.SOURCE_BLUR)
self.assertEqual(B.parse("masked area (mouth too)"), ("masked area", "mouth too"))
self.assertIsNone(B.parse("off"))
def test_keep_lips_and_tongue_blurs_the_teeth(self):
m = self.mask("face (keep lips + tongue)")
self.assertEqual(float(m[0, 15, 25]), 1.0) # face
self.assertEqual(float(m[0, 23, 32]), 0.0) # lips stay sharp
self.assertEqual(float(m[0, 24, 32]), 0.0) # teeth are inside the lips' small grow here...
big = fake_classes(self.frames)
big[:, 26:29, 28:36] = SP.ID["Lower_Teeth"] # ...so check teeth away from the lips
SP.segment = lambda *a, **k: big
B._CACHE.clear()
m = self.mask("face (keep lips + tongue)")
self.assertEqual(float(m[0, 28, 32]), 1.0)
m = self.mask("face (keep mouth)")
self.assertEqual(float(m[0, 28, 32]), 0.0) # keep mouth keeps the teeth too
def test_mouth_too_blurs_everything_in_the_region(self):
m = self.mask("face (mouth too)")
self.assertEqual(float(m[0, 23, 32]), 1.0)
self.assertEqual(float(m[0, 2, 2]), 0.0) # background untouched
def test_masked_area_uses_the_shot_mask_without_segmenting(self):
SP.segment = lambda *a, **k: self.fail("masked area + mouth too must not segment")
area = torch.zeros(3, H, W)
area[:, 0:10, 0:10] = 1
m = self.mask("masked area (mouth too)", area=area)
self.assertEqual(float(m[0, 5, 5]), 1.0)
self.assertEqual(float(m[0, 20, 30]), 0.0)
def test_masked_area_without_a_mask_blurs_the_whole_person(self):
m = self.mask("masked area (mouth too)")
self.assertEqual(float(m[0, 40, 15]), 1.0) # clothing = person
def test_invert_colours_inside_the_mask_only(self):
x = torch.full((3, H, W, 3), 0.2)
out = B.blur_source(x, "face (mouth too)", 1.0, segmenter="Sapiens2", effect="invert colours")
self.assertAlmostEqual(float(out[0, 18, 30, 0]), 0.8, places=4)
self.assertAlmostEqual(float(out[0, 2, 2, 0]), 0.2, places=4)
def test_sam_keeps_the_old_behaviour(self):
SP.segment = lambda *a, **k: self.fail("SAM 3.1 + keep mouth must not run Sapiens2")
_SL.SAM_CALLS.clear()
m = self.mask("face (keep mouth)", segmenter="SAM 3.1")
self.assertEqual(_SL.SAM_CALLS, ["face", "mouth"])
self.assertEqual(float(m[0, 15, 25]), 1.0)
self.assertEqual(float(m[0, 23, 32]), 0.0)
class SapiensSizeTest(unittest.TestCase):
def test_input_keeps_the_area_and_the_aspect(self):
for h, w in ((1080, 1920), (1920, 1080), (512, 512)):
ih, iw = SP.input_size(h, w)
self.assertEqual((ih % 16, iw % 16), (0, 0))
self.assertAlmostEqual(iw / ih, w / h, delta=0.03) # never stretched to 1024x768
self.assertAlmostEqual(ih * iw / SP.PIXELS, 1.0, delta=0.04)
if __name__ == "__main__":
unittest.main()
+169
View File
@@ -0,0 +1,169 @@
from __future__ import annotations
import importlib.util
import os
import sys
import tempfile
import types
import unittest
from pathlib import Path
import torch
ROOT = Path(__file__).resolve().parents[1]
_TMP = tempfile.mkdtemp(prefix="bfs_compare_test_")
_fp = types.ModuleType("folder_paths")
_fp.get_temp_directory = lambda: _TMP
sys.modules.setdefault("folder_paths", _fp)
SPEC = importlib.util.spec_from_file_location("bfs_video_compare", ROOT / "bfs_video_compare.py")
VC = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(VC)
class DifferenceTest(unittest.TestCase):
def test_equal_frames_are_black_and_score_zero(self):
x = torch.rand(3, 8, 8, 3)
m, mae, mse = VC.difference(x, x.clone())
self.assertEqual(float(m.abs().max()), 0.0)
self.assertEqual(float(mae.max()), 0.0)
self.assertEqual(VC.psnr(float(mse.mean())), float("inf"))
def test_map_lights_up_only_where_it_changed(self):
g = torch.zeros(1, 8, 8, 3)
t = g.clone()
t[:, :4, :4] = 1.0
m, mae, _ = VC.difference(g, t, gain=1.0)
self.assertEqual(m[0, 0, 0].tolist(), [1.0, 1.0, 1.0]) # fully different: white
self.assertEqual(m[0, 7, 7].tolist(), [0.0, 0.0, 0.0]) # unchanged: black
self.assertAlmostEqual(float(mae[0]), 255 / 4, places=3) # a quarter of the frame
def test_heat_scale(self):
self.assertEqual(VC.heat(torch.tensor(1 / 3)).tolist(), [1.0, 0.0, 0.0])
self.assertEqual(VC.heat(torch.tensor(2 / 3)).tolist()[:2], [1.0, 1.0])
class CompareNodeTest(unittest.TestCase):
def test_compares_at_the_guide_size_and_writes_three_previews(self):
import cv2
guide = torch.rand(20, 32, 48, 3)
target = torch.rand(18, 64, 96, 3) # bigger and shorter, like the Join's images vs its guide
audio = {"waveform": torch.zeros(1, 2, 48000), "sample_rate": 48000}
out = VC.BFSVideoCompare().compare(guide, target, fps=24.0, audio=audio, preview_height=32)
diff, report, mean = out["result"]
self.assertEqual(tuple(diff.shape), (18, 32, 48, 3))
self.assertGreater(mean, 0)
self.assertIn("lengths differ", report)
self.assertIn("target scaled from 96x64 to 48x32", report)
ui = out["ui"]["bfs_compare"][0]
self.assertEqual((ui["frames"], len(ui["mae"]), ui["width"], ui["height"]), (18, 18, 48, 32))
for k in ("guide", "target", "difference"):
path = os.path.join(_TMP, ui[k]["subfolder"], ui[k]["filename"])
cap = cv2.VideoCapture(path)
n = 0
while cap.read()[0]:
n += 1
self.assertEqual(n, 18, k)
def test_reference_pictures_go_to_the_player_no_taller_than_it(self):
from PIL import Image
guide, target = torch.rand(4, 32, 48, 3), torch.rand(4, 32, 48, 3)
ref = torch.rand(6, 80, 40, 3) # taller than the preview
ui = VC.BFSVideoCompare().compare(guide, target, preview_height=32, reference=ref)["ui"]["bfs_compare"][0]
self.assertEqual([r["filename"] for r in ui["refs"]], [f"ref_{k}.png" for k in range(6)])
im = Image.open(os.path.join(_TMP, ui["refs"][0]["subfolder"], "ref_0.png"))
self.assertEqual(im.size, (16, 32))
# a refs list as the planner / Unpack give it: pictures of different sizes, as ComfyUI hands a list input
refs = [torch.rand(1, 64, 32, 3), torch.rand(1, 20, 60, 3)] + [torch.rand(1, 8, 8, 3)] * 12
ui = VC.BFSVideoCompare().compare([guide], [target], preview_height=[32], reference=refs)["ui"]["bfs_compare"][0]
self.assertEqual(len(ui["refs"]), 12) # at most 12
sizes = [Image.open(os.path.join(_TMP, r["subfolder"], r["filename"])).size for r in ui["refs"][:2]]
self.assertEqual(sizes, [(16, 32), (60, 20)])
none = VC.BFSVideoCompare().compare(guide, target, preview_height=32)["ui"]["bfs_compare"][0]
self.assertEqual(none["refs"], [])
@unittest.skipUnless(VC.VIO.ffmpeg_bin(), "needs an ffmpeg program")
class ExportTest(unittest.TestCase):
"""The player's recording (WebM from MediaRecorder) converted for sharing."""
@classmethod
def setUpClass(cls):
import subprocess
cls.src = os.path.join(_TMP, "rec_src.webm")
# 20 s of noise at 720p with sound: far above 10 MB at the browser's bitrate
subprocess.run([VC.VIO.ffmpeg_bin(), "-y", "-v", "error", "-f", "lavfi", "-i", "nullsrc=s=1280x720:r=30:d=20,geq=random(1)*255:128:128",
"-f", "lavfi", "-i", "sine=frequency=440:duration=20", "-c:v", "libvpx", "-b:v", "16M",
"-deadline", "realtime", "-cpu-used", "8", "-c:a", "libopus", "-shortest", cls.src], check=True)
def probe(self, path):
import json
import subprocess
r = subprocess.run([VC.VIO.ffmpeg_bin().replace("ffmpeg", "ffprobe"), "-v", "error", "-show_streams", "-of", "json", path],
capture_output=True, text=True)
return {st["codec_type"]: st for st in json.loads(r.stdout)["streams"]}
def test_small_mp4_is_h264_aac_faststart_under_the_limit(self):
self.assertGreater(os.path.getsize(self.src), VC.LIMIT_BYTES)
out = VC.encode(self.src, "small", 20.0, os.path.join(_TMP, "exp_small"))
self.assertLessEqual(os.path.getsize(out), VC.LIMIT_BYTES)
st = self.probe(out)
self.assertEqual((st["video"]["codec_name"], st["video"]["pix_fmt"], st["audio"]["codec_name"]), ("h264", "yuv420p", "aac"))
head = open(out, "rb").read(256 * 1024)
self.assertLess(head.find(b"moov"), head.find(b"mdat") if b"mdat" in head else 1 << 30) # faststart: moov first
def test_gif_fits_the_limit(self):
out = VC.encode(self.src, "gif", 20.0, os.path.join(_TMP, "exp_gif"))
self.assertTrue(out.endswith(".gif"))
self.assertLessEqual(os.path.getsize(out), VC.LIMIT_BYTES)
def test_bad_ids_and_formats_are_refused(self):
with self.assertRaises(ValueError):
VC.write_part("../../etc", 0, b"x")
with self.assertRaises(ValueError):
VC.encode(self.src, "avi", 1.0, os.path.join(_TMP, "x"))
rid = "abcdef0123"
VC.write_part(rid, 0, b"12345")
VC.write_part(rid, 5, b"678")
self.assertEqual(open(os.path.join(_TMP, "bfs_compare", "exports", rid + ".rec"), "rb").read(), b"12345678")
class NoFfmpegTest(unittest.TestCase):
"""Windows portable installs often have no ffmpeg program: everything must go through PyAV."""
def setUp(self):
self._saved = dict(VC.VIO._BIN)
VC.VIO._BIN.update(ffmpeg=None, ffprobe=None)
def tearDown(self):
VC.VIO._BIN.clear()
VC.VIO._BIN.update(self._saved)
def test_writer_and_reader_work_without_ffmpeg(self):
import av
path = os.path.join(_TMP, "noffmpeg.mp4")
w = VC.VIO.Mp4Writer(path, 33, 21, 24.0, {"waveform": torch.zeros(1, 2, 48000), "sample_rate": 48000},
metadata={"prompt": {"1": 2}})
yy, xx = torch.meshgrid(torch.linspace(0, 1, 21), torch.linspace(0, 1, 33), indexing="ij")
x = torch.stack([torch.stack([xx, yy, (xx + yy + t / 30) % 1], -1) for t in range(30)]) # smooth, like video
w.write(x[:16])
w.write(x[16:])
w.close()
with av.open(path) as c:
self.assertEqual(len(c.streams.audio), 1)
self.assertEqual((c.streams.video[0].codec_context.width, c.streams.video[0].codec_context.height), (34, 22))
self.assertEqual(c.metadata.get("prompt"), '{"1": 2}')
frames, s = VC.VIO.read_video_scaled(path, 1e9)
self.assertEqual((tuple(frames.shape), s), ((30, 22, 34, 3), 1.0))
self.assertLess(float((frames[:, :21, :33] - x).abs().mean()), 0.05) # lossy, but the same pictures
small, s = VC.VIO.read_video_scaled(path, 30 * 22 * 34 * 3 * 4 / 4) # a quarter of the bytes
self.assertLess(small.shape[1], 22)
def test_compare_node_without_ffmpeg(self):
out = VC.BFSVideoCompare().compare(torch.rand(10, 32, 48, 3), torch.rand(10, 32, 48, 3), preview_height=32)
ui = out["ui"]["bfs_compare"][0]
self.assertTrue(os.path.getsize(os.path.join(_TMP, ui["guide"]["subfolder"], ui["guide"]["filename"])) > 0)
if __name__ == "__main__":
unittest.main()
+94
View File
@@ -0,0 +1,94 @@
"""Assemble the shots a BFS Shot Planner run stored on disk into one MP4, one shot at a time (low RAM).
For a run whose final join did not finish (ComfyUI killed, out of memory...). Every shot is stored by BFS Shot Join as
<run folder>/shot_0000.pt (+ shot_0000.mkv), shot_0001.pt ... already pasted back / uncropped, without its continuity head; this reads
them in order, trims each to its length, cross-fades joins that are not camera cuts (like BFS Shot Join), and streams
the frames to ffmpeg, so memory stays at about one shot whatever the video length.
python bfs_assemble_shots.py <run folder> out.mp4 [--audio source.mp4] [--crossfade 4] [--crf 17]
--audio: the source video; its sound is cut to the stored shots' time range (shot 1's start to the last shot's end).
Shots that did not run (disabled / filtered) are not stored, so they are not in the output.
"""
from __future__ import annotations
import argparse
import glob
import os
import subprocess
import sys
import torch
def main() -> int:
ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
ap.add_argument("run", help="the run folder with shot_0000.pt ... (ComfyUI output/bfs_shotloop/<run id>; "
"temp/bfs_shotloop/<run id> before 1.104)")
ap.add_argument("out", help="output .mp4")
ap.add_argument("--audio", help="source video to take the sound from", default="")
ap.add_argument("--crossfade", type=int, default=4, help="frames blended at joins that are not cuts (0 = hard)")
ap.add_argument("--crf", type=int, default=17)
a = ap.parse_args()
files = sorted(glob.glob(os.path.join(a.run, "shot_*.pt")))
if not files:
print(f"no shot_*.pt in {a.run}", file=sys.stderr)
return 1
metas = [torch.load(f, map_location="cpu", weights_only=False)["shot"] for f in files[:1]]
m0 = metas[0]
count = int(m0.get("count") or len(files))
idx = sorted(int(os.path.basename(f)[5:9]) for f in files)
missing = [i for i in range(count) if i not in idx]
if missing:
print(f"warning: shots {[i + 1 for i in missing]} of {count} are not stored; the video skips them", file=sys.stderr)
fps = float(m0.get("fps") or 24.0)
proc, prev_tail, first_start, last_end, n_out = None, None, None, None, 0
for f in files:
d = torch.load(f, map_location="cpu", weights_only=False)
meta = d["shot"]
if "frames" in d: # raw frames (older runs, Shot storage = raw)
frames = d["frames"] # uint8 [N, H, W, 3]
else: # the frames are in shot_NNNN.mkv next to the .pt
n, h, w, _ = d["shape"]
raw = subprocess.run(["ffmpeg", "-v", "error", "-i", os.path.join(a.run, d["video"]), "-f", "rawvideo",
"-pix_fmt", "rgb24", "-"], capture_output=True, check=True).stdout
frames = torch.frombuffer(bytearray(raw[:n * h * w * 3]), dtype=torch.uint8).view(n, h, w, 3)
L = int(meta["length"])
body = frames[:L]
if body.shape[0] < L: # shorter than planned: hold the last frame
body = torch.cat([body, body[-1:].expand(L - body.shape[0], -1, -1, -1)], 0)
if proc is None:
H, W = int(body.shape[1]), int(body.shape[2])
cmd = ["ffmpeg", "-v", "error", "-y", "-f", "rawvideo", "-pix_fmt", "rgb24", "-s", f"{W}x{H}",
"-r", f"{fps:g}", "-i", "-"]
if a.audio:
t0 = float(meta["start"]) / fps
cmd += ["-ss", f"{t0:.4f}", "-i", a.audio, "-map", "0:v", "-map", "1:a?", "-c:a", "aac", "-shortest"]
cmd += ["-c:v", "libx264", "-crf", str(a.crf), "-pix_fmt", "yuv420p",
"-vf", "pad=ceil(iw/2)*2:ceil(ih/2)*2", a.out]
proc = subprocess.Popen(cmd, stdin=subprocess.PIPE)
elif body.shape[1:3] != (H, W):
body = torch.nn.functional.interpolate(body.permute(0, 3, 1, 2).float(), size=(H, W), mode="bilinear",
align_corners=False).round().clamp(0, 255).byte().permute(0, 2, 3, 1)
if a.crossfade > 0 and prev_tail is not None and not meta.get("cut_before"):
n = min(a.crossfade, prev_tail.shape[0], body.shape[0])
if n > 0:
w = torch.linspace(0, 1, n + 2)[1:-1].view(-1, 1, 1, 1)
body = body.clone()
body[:n] = (prev_tail[:n].float() * (1 - w) + body[:n].float() * w).round().byte()
proc.stdin.write(body.contiguous().numpy().tobytes())
prev_tail = frames[L:].clone() if frames.shape[0] > L else None
first_start = meta["start"] if first_start is None else first_start
last_end, n_out = meta["end"], n_out + L
print(f"shot {int(meta['index']) + 1}/{count}: {L} frames", flush=True)
del d, frames, body
proc.stdin.close()
proc.wait()
print(f"{a.out}: {n_out} frames ({n_out / fps:.2f} s at {fps:g} fps)")
return proc.returncode
if __name__ == "__main__":
sys.exit(main())
+9 -6
View File
@@ -94,12 +94,15 @@ function html(type) {
<tr><td>appearance</td><td><code>an elderly woman with short grey hair</code></td></tr>
<tr><td>lighting / weather</td><td><code>night, lit by pink and blue neon signs</code></td></tr>
</table>
<p>Rules that matter (render-tested): the panel has <b>no tag</b> (it is not &lt;Picture n&gt; or &lt;Video n&gt;): call it
"the kept footage" by its side, or write <code>{layout}</code> to insert the sentence. <b>Never describe the panel's person,
clothes or room</b> (what you describe gets drawn; what you leave out is copied from the panel). Restate the new identity in
every shot ("her face from &lt;Picture 1&gt;" + 2–3 face/hair/outfit words). Give exact times for cuts
("[Shot 2] At 00:03.708, both halves cut together to…"). No negations ("no hat" draws a hat). The full example is in the
prompt tooltip.</p>
<p>Rules that matter (TSC, newer strategy): <b>H3 has no idea what is pinned and what is noised</b>, it reads one
description of one video. Describe the finished split screen as plain footage: never write "kept footage", "generated",
"preserved", "in sync with", "seamlessly". The pinned clip gets <b>no tag and no retention line</b>: name its region by where
it is plus what is seen there, in one clause ("the top quarter shows a man dancing in a small bedroom";
<code>{panel_region}</code> / <code>{video_region}</code> give the region names, <code>{layout}</code> the whole sentence).
Write the other region as its own scene in a real place. <b>Spell out the motion</b> beat by beat, with times for the big
moments on the video's clock. The soundscape is the sounds of the new scene. Cuts: "[Shot 2] At 00:03.708, both halves cut
together to…". No negations ("no hat" draws a hat). Connect a VLM to <code>vlm</code> to write the prompt or fill a
template's <code>{...}</code> fields. The full example is in the prompt tooltip.</p>
<h3>Layout</h3>
<p><b>position</b>: the side of the panel.</p>
+561 -95
View File
@@ -11,16 +11,20 @@
import { app } from "../../scripts/app.js";
import { api } from "../../scripts/api.js";
import { showGuide } from "./bfs_md_view.js";
import { createApp, ref, reactive, computed, watch, nextTick, onMounted, onBeforeUnmount, h } from "./vendor/vue.esm-browser.prod.mjs";
import { DEFAULTS, MASK_DEFAULTS, VLM_DEFAULTS, snapUp, snapDown, segKey, hasMask, pill } from "./shotloop/common.js";
import { wheelToCanvas } from "./bfs_wheel.js";
import { createApp, ref, reactive, computed, watch, nextTick, onMounted, onBeforeUnmount } from "./vendor/vue.esm-browser.prod.mjs";
import { h, rawH, t, uiLanguage, setUiLanguage, DEFAULTS, MASK_DEFAULTS, VLM_DEFAULTS, snapUp, snapDown, segKey, hasMask, pill,
MAX_REFS, refList, shotRefNames, MARK_PALETTE, pictureColor, markRegions, AUTO_DEFAULTS } from "./shotloop/common.js";
import { styles } from "./shotloop/styles.js";
// history lengths H3's VAE encodes whole (17k+5); old plans saved 17/34/51, read as the next valid one up
const historyLen = n => (n = Number(n) || 22) <= 5 ? 5 : 5 + Math.ceil((n - 5) / 17) * 17;
import { timelineCard } from "./shotloop/view_timeline.js";
import { videoTab } from "./shotloop/view_video.js";
import { shotsTab } from "./shotloop/view_shots.js";
import { peopleTab } from "./shotloop/view_people.js";
import { promptsTab } from "./shotloop/view_prompts.js";
import { runTab } from "./shotloop/view_run.js";
import { busyBar, pointsModal } from "./shotloop/view_modal.js";
import { busyBar, pointsModal, tourModal } from "./shotloop/view_modal.js";
const viewUrl = name => {
if (!name) return "";
@@ -33,7 +37,7 @@ const store = { // per-browser UI conveniences (never the plan itself)
set: (k, v) => { try { localStorage.setItem(k, JSON.stringify(v)); } catch { /* private mode */ } },
};
const DEFAULT_W = 1180, MIN_W = 1040, MIN_H = 1000; // node size: wide enough for the four-column settings
const TABS = [["video", "🎞 Video"], ["shots", "🎬 Shots"], ["people", "👥 People & masks"], ["prompts", "📝 Prompts & refs"], ["run", "▶ Run"]];
const TABS = [["video", "🎞 Video"], ["shots", "🎬 Shots"], ["people", "👥 People / Filters / Masks"], ["prompts", "📝 Prompts & refs"], ["run", "▶ Run"]];
function Panel(io) {
const plan = reactive({ ...DEFAULTS });
@@ -54,22 +58,42 @@ function Panel(io) {
watch(sel, () => nextTick(() => rootEl.value?.querySelector(".shots .sc.sel")?.scrollIntoView({ block: "nearest", inline: "nearest" })));
const hover = ref(null); // {frame, x}
const prog = reactive({ done: 0, count: 0 });
const cache = reactive({ ok: [], stored: [], mb: 0, folder: "", temp: false, combined: null });
// what the server learned about memory on this GPU (BFS Shot Join records every shot it receives)
const vram = reactive({ gpu: "", total_gb: 0, free_gb: 0, max_tokens: 0, max_peak_gb: 0, runs: 0, ram_free_gb: 0 });
async function refreshVram() {
try { const r = await api.fetchApi("/bfs/shotloop/vram"); const j = await r.json(); if (!j.error) Object.assign(vram, j); } catch { /* optional */ }
}
const tlEl = ref(null);
const stats = ref([]); // per-shot content stats from /bfs/shotloop/filters
const people = ref([]); // cast from /bfs/shotloop/cast: [{id, thumb, share, first, last}]
const maskPrev = ref({}); // "start-end" -> {frames, box, coverage, empty} from /bfs/shotloop/mask
const showMasks = ref(true); // overlay the mask preview on the shot cards
const vlmSug = ref({}); // "start-end" -> {segment, shot, people, recommend, reason} from the VLM
const modal = reactive({ open: false, idx: -1, key: 0, points: [], text: "", prev: null, busy: "" });
const modal = reactive({ open: false, idx: -1, key: 0, points: [], text: "", region: 1, groups: {}, prev: null, stale: false,
showMask: true, playing: 0, busy: "" });
const segPeople = ref({}); // "start-end" -> {people, main} from /bfs/shotloop/plan
const targetCrop = ref({}); // "start-end" -> data URL of the subject the description was made from
const tab = ref(store.get("bfs.shotloop.tab", "shots"));
watch(tab, v => store.set("bfs.shotloop.tab", v));
const tour = ref(0); // opened only by the user's Tour button; never shown automatically
// what "Copy to other shots" copies (remembered per browser)
const copyOpts = reactive(store.get("bfs.shotloop.copy", { ref: true, prompt: false, mask_text: false, points: false, crop: false,
target: false, chain: false, scope: "all" }));
target: false, chain: false, color_guide: false, scope: "all" }));
watch(copyOpts, v => store.set("bfs.shotloop.copy", { ...v }), { deep: true });
// one description per picture: picture_details, plus the ones plans before 1.113 saved per reference set in
// ref_details ("a|b" -> {details, details2}, or one text for a); same rule as bfs_shot_loop.saved_picture_details
const pictureDetailsOf = p => {
const out = {};
Object.entries(p.ref_details || {}).forEach(([key, v]) => {
const [a, b] = key.split("|");
const pairs = v && typeof v === "object" ? [[a, v.details || v.legacy], [b, v.details2]] : [[a, v]];
pairs.forEach(([name, text]) => { if (name && text && !(name in out)) out[name] = String(text); });
});
Object.entries(p.picture_details || {}).forEach(([name, text]) => { if (text) out[name] = text; });
return out;
};
const load = () => {
let p = {};
try { p = JSON.parse(io.getPlan() || "{}"); } catch { p = {}; }
@@ -79,42 +103,79 @@ function Panel(io) {
plan.mask_cfg = { ...MASK_DEFAULTS, ...(p.mask_cfg || {}) };
plan.vlm_cfg = { ...VLM_DEFAULTS, ...(p.vlm_cfg || {}) };
plan.ref_details = { ...(p.ref_details || {}) };
plan.picture_details = { ...pictureDetailsOf(p) };
plan.picture_cfg = { ...(p.picture_cfg || {}) };
plan.auto_prompt = { ...AUTO_DEFAULTS, ...(p.auto_prompt || {}) };
plan.auto_prompts = { ...(p.auto_prompts || {}) };
if (!Array.isArray(plan.bounds)) plan.bounds = [];
if (!Array.isArray(plan.segs)) plan.segs = [];
// plans before 1.113 had two references (ref / ref2): now a list of up to 12 pictures
plan.global_refs = refList(p.global_refs, p.global_ref, p.global_ref2);
plan.segs.forEach(m => { if (m) m.refs = refList(m.refs, m.ref, m.ref2); });
// Colored Mark: a color per picture (was: two marker colors), regions tied to a picture (was: region 2)
if (!Array.isArray(p.picture_colors)) plan.picture_colors = [p.global_color_guide_ref1 || MARK_PALETTE[0][0], p.global_color_guide_ref2 || MARK_PALETTE[1][0]];
plan.segs.forEach(m => {
if (!m) return;
m.color_guide_regions = markRegions(m);
["color_guide_text2", "color_guide_points2", "color_guide_key2", "color_guide_color", "color_guide_color2"].forEach(k => delete m[k]);
});
Object.values(plan.cast).forEach(person => { if (person && typeof person === "object") { delete person.ref; delete person.ref2; delete person.guide_color; } });
plan.segs.forEach(meta => { if (meta) { delete meta.color_guide_person; delete meta.color_guide_person2; } });
};
const save = () => io.setPlan(JSON.stringify(plan));
// any change of the plan (a new video, a prompt, a setting...) is a new loop run on the server: show its progress
let progTimer = 0;
// ref / ref2 (and global_ref / global_ref2) keep pictures 1 and 2 for older readers of the plan
const syncRefs = () => {
plan.global_refs = refList(plan.global_refs); [plan.global_ref = "", plan.global_ref2 = ""] = plan.global_refs;
plan.global_color_guide_ref1 = pictureColor(plan, 1); plan.global_color_guide_ref2 = pictureColor(plan, 2);
plan.segs.forEach(m => { if (m) { m.refs = refList(m.refs); [m.ref = "", m.ref2 = ""] = m.refs; } });
};
const save = () => { syncRefs(); io.setPlan(JSON.stringify(plan)); clearTimeout(progTimer); progTimer = setTimeout(() => progress(), 400); };
const n = computed(() => {
if (!an.value) return 0;
const cap = plan.max_total_s > 0 ? Math.round(plan.max_total_s * plan.fps) : Infinity;
return Math.min(an.value.n, cap);
const n = computed(() => (an.value ? an.value.n : 0)); // the whole video on the timeline
// the part the plan works on [lo, hi): start / end (0 = the end), then "max total seconds" counted from the start
const range = computed(() => {
const N = n.value; if (!N) return { lo: 0, hi: 0 };
// frames (older plans saved seconds)
const a = plan.start_f != null ? +plan.start_f : Math.round((+plan.start_s || 0) * plan.fps);
const b = plan.end_f != null ? +plan.end_f : (+plan.end_s > 0 ? Math.round(+plan.end_s * plan.fps) : 0);
const lo = Math.max(0, Math.min(N - 1, a));
let hi = b > 0 ? Math.max(lo + 1, Math.min(N, b)) : N;
if (+plan.max_total_s > 0) hi = Math.min(hi, lo + Math.max(1, Math.round(+plan.max_total_s * plan.fps)));
return { lo, hi };
});
const maxLen = computed(() => snapDown(Math.round(plan.max_s * plan.fps), plan.grid));
const segs = computed(() => {
const N = n.value; if (!N) return [];
const b = [0, ...plan.bounds.filter(x => x > 0 && x < N), N].sort((a, c) => a - c);
const { lo, hi } = range.value;
const b = [lo, ...plan.bounds.filter(x => x > lo && x < hi), hi].sort((a, c) => a - c);
const cs = new Set(cuts.value);
let out = [];
for (let i = 0; i < b.length - 1; i++) {
if (b[i + 1] <= b[i]) continue;
const m = plan.segs[out.length] || {};
out.push({ start: b[i], end: b[i + 1], len: b[i + 1] - b[i], gen: snapUp(b[i + 1] - b[i], plan.grid),
cut: cs.has(b[i]), enabled: m.enabled !== false, ref: m.ref || "", ref2: m.ref2 || "", prompt: m.prompt || "",
force: m.force || "auto", chain: m.chain || "off", chainFrame: m.chain_frame || "first",
crop: !!m.crop, inpaint: !!m.inpaint, mask: m.mask || {}, target: m.target || "" });
cut: cs.has(b[i]), enabled: m.enabled !== false, refs: refList(m.refs, m.ref, m.ref2), prompt: m.prompt || "",
force: m.force || "auto", chain: m.chain || "head", chainFrame: m.chain_frame || "first", chainHistory: historyLen(m.chain_history),
color_guide: !!m.color_guide, color_guide_picture: +m.color_guide_picture || 1,
color_guide_opacity: m.color_guide_opacity ?? 0.35, regions: markRegions(m),
crop: !!m.crop, inpaint: !!m.inpaint, paste: !!m.paste, pasteText: m.paste_text || "", strength: m.strength ?? 1, mask: m.mask || {}, target: m.target || "",
extMask: !!plan.mask_video || !!(plan.track?.points?.length), trackMask: !!(plan.track?.points?.length) });
}
out.forEach(s => { s.names = shotRefNames(s.refs, plan.global_refs); [s.ref = "", s.ref2 = ""] = s.refs; });
if (plan.max_parts > 0) out = out.slice(0, plan.max_parts);
return out;
});
// shots on the timeline before "Max shots" keeps the first ones: more than segs means some are left out
const allSegCount = computed(() => {
if (!n.value) return 0;
const { lo, hi } = range.value;
const b = [lo, ...plan.bounds.filter(x => x > lo && x < hi), hi].sort((a, c) => a - c);
return b.reduce((k, x, i) => k + (i && x > b[i - 1] ? 1 : 0), 0);
});
const whoIn = s => segPeople.value[segKey(s)] || null;
const castOf = id => plan.cast[String(id)] || {};
const linked = id => !!castOf(id).ref && !castOf(id).ignore;
// reference a shot gets from its main person when it has none of its own
const castRef = (s, field) => {
if (!plan.cast_assign) return "";
const who = whoIn(s); if (!who || who.main < 0 || castOf(who.main).ignore) return "";
return castOf(who.main)[field] || "";
};
const included = id => !!personOf(id) && !castOf(id).ignore;
const personOf = id => people.value.find(p => p.id === id) || null;
const statFor = s => stats.value.find(x => x.start === s.start && x.end === s.end) || null;
const filtersOn = computed(() => { const f = plan.filters; return !!(f.person || f.face || f.skip_dark || f.skip_static || f.max_persons > 0 || f.min_frames > 0); });
@@ -125,7 +186,7 @@ function Panel(io) {
const why = filtersOn.value ? (statFor(s)?.skip_reason || "") : "";
if (why) return why;
const who = whoIn(s);
if (plan.cast_only && who && !who.people.some(p => linked(p))) return "no linked person";
if (plan.cast_only && who && !who.people.some(p => included(p))) return "no detected person";
return "";
};
const active = computed(() => segs.value.filter(s => !skipWhy(s)));
@@ -137,30 +198,76 @@ function Panel(io) {
// how a shot is generated: crop (the planner crops, the join pastes back) x inpaint (only the mask is regenerated,
// read by BFS Shot H3 Conditioning with inpaint = per shot)
const MODES = [["frame", "Full frame", false, false], ["mask", "Mask only", false, true], ["crop", "Crop", true, false], ["cropmask", "Crop + mask", true, true]];
const modeOf = s => MODES.find(m => m[2] === !!s.crop && m[3] === !!s.inpaint)[0];
// [key, name, crop, inpaint, paste]; paste = the whole frame is generated and BFS Shot Join pastes only the person back
const MODES = [["frame", "Full frame", false, false, false], ["paste", "Frame + paste", false, false, true], ["mask", "Mask only", false, true, false],
["crop", "Crop", true, false, false], ["cropmask", "Crop + mask", true, true, false]];
const modeOf = s => (MODES.find(m => m[2] === !!s.crop && m[3] === !!s.inpaint && m[4] === (!!s.paste && !s.crop && !s.inpaint))
|| MODES[0])[0];
const modeName = s => MODES.find(m => m[0] === modeOf(s))[1];
const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return plan.segs[i]; };
// the reference a shot really uses: its own, its main person's, or the global one
const refOf = (s, k) => s[k] || castRef(s, k) || plan["global_" + k] || "";
// the pictures a shot uses: "ref" = <Picture 1>, "ref2" = <Picture 2> (its own or the global ones)
const refOf = (s, k) => s.names[k === "ref2" ? 1 : 0] || "";
const promptOf = s => s.prompt || plan.global_prompt || "";
const detailsOf = s => plan.ref_details?.[`${refOf(s, "ref")}|${refOf(s, "ref2")}`] || "";
// the pictures a prompt describes ({details} = 1, {details2} = 2, {details@Picture n}) all have a description
const detailsOf = (s, prompt) => {
const want = new Set();
if (prompt.includes("{details}")) want.add(1);
if (prompt.includes("{details2}")) want.add(2);
for (const m of prompt.matchAll(/\{details@Picture\s*(\d+)\}/g)) want.add(+m[1]);
return [...want].every(k => !s.names[k - 1] || plan.picture_details?.[s.names[k - 1]]);
};
// things worth fixing before a run, per shot (only shots that run)
// ---- memory status: each shot's size (latent tokens = generated frames x resolution) against the biggest shot that
// already ran on this GPU. Not a GB figure: ComfyUI offloads weights as needed, so only what ran here is a reference.
const H3G = "H3 (17n+5)";
const headOf = (s, i) => plan.run !== "auto" && s.chain === "head" && i > 0 && !s.cut && (plan.grid || H3G) === H3G
? Math.min(s.chainHistory || 22, 39) : 0;
const shotTokens = (s, i) => {
const hd = headOf(s, i), gen = hd ? snapUp(s.len + hd, plan.grid) : s.gen;
const steps = (plan.grid || H3G) === H3G ? 5 * Math.ceil(gen / 17) - 3 : Math.ceil(gen / 4);
return Math.max(1, steps) * Math.ceil((size.h || 0) / 16) * Math.ceil((size.w || 0) / 16);
};
const memory = computed(() => {
const list = segs.value.map((s, i) => ({ i, tok: skipWhy(s) ? 0 : shotTokens(s, i), gen: s.gen + headOf(s, i) })).filter(x => x.tok > 0);
const top = list.reduce((a, b) => (b.tok > a.tok ? b : a), { i: -1, tok: 0, gen: 0 });
const ratio = vram.max_tokens ? top.tok / vram.max_tokens : 0;
const level = !top.tok || !size.w ? "" : !vram.max_tokens ? "info" : ratio <= 1.05 ? "ok" : ratio <= 1.3 ? "note" : "warn";
return { list, top, ratio, level };
});
// the final video as images (BFS Shot Join, save_to = images) is held in RAM whole: about 12 bytes per pixel per frame,
// ~3.5x more with the comparison. Long videos should be written straight to a file instead.
const joinRam = computed(() => {
const frames = segs.value.reduce((t, s) => t + (skipWhy(s) && plan.skip_fill === "drop" ? 0 : s.len), 0);
const gb = frames * (size.w || 0) * (size.h || 0) * 12 / 2 ** 30;
return { frames, gb, long: vram.ram_free_gb > 0 && gb * 3.5 > 0.6 * vram.ram_free_gb };
});
const checks = computed(() => {
const out = [];
if (joinRam.value.long && active.value.length)
out.push({ lvl: "warn", shot: 0, text: `long video: as images the final video needs ~${joinRam.value.gb.toFixed(1)} GB of RAM `
+ `(~${(joinRam.value.gb * 3.5).toFixed(1)} GB with the comparison) and ${vram.ram_free_gb} GB is free. Set BFS Shot Join `
+ `save_to = mp4 file: it writes the video straight to disk, shows it on the node and still gives the result to the next nodes` });
segs.value.forEach((s, i) => {
if (skipWhy(s)) return;
const add = (lvl, text) => out.push({ lvl, shot: i, text });
const p = promptOf(s);
if (s.len > maxLen.value) add("warn", `longer than ${maxLen.value} frames: split it`);
if ((s.crop || s.inpaint) && !hasMask(s)) add("warn", `${modeName(s)} needs a mask (text or points) on the shot`);
if (s.len > maxLen.value) add(plan.mode === "single" ? "note" : "warn", plan.mode === "single"
? `one shot of ${s.len} frames (${(s.len / plan.fps).toFixed(1)} s), longer than Max seconds / shot: the model may not hold it`
: `longer than ${maxLen.value} frames: split it`);
if ((s.crop || s.inpaint || s.paste) && !hasMask(s)) add("warn", `${modeName(s)} needs a mask (text, points or a mask video; or the planner's mask input)`);
if (hasMask(s) && maskPrev.value[segKey(s)]?.empty) add("warn", "the mask preview found nothing");
if (/\{target\}/.test(p) && !s.target) add("warn", "the prompt uses {target} but the shot has no target description");
if (/\{details\}/.test(p) && !detailsOf(s) && !plan.vlm_cfg?.enabled) add("info", "the prompt uses {details} but its references have no description yet");
if (/\{details/.test(p) && !detailsOf(s, p) && !plan.vlm_cfg?.enabled) add("info", "the prompt uses {details} but its pictures have no description yet");
if (/\{details2?\}/.test(p) && (plan.vlm_cfg?.describe_preset || "short") !== "short")
add("warn", "{details} sits inside a sentence of the prompt: use the 'short' describe preset (Prompts & refs), the other presets write a paragraph");
if (!p) add("info", "no prompt (fine when the node's prompt input is connected)");
if (!refOf(s, "ref")) add("info", "no reference (fine when the node's ref_image input is connected)");
if (!refOf(s, "ref")) add("info", "no reference picture");
if (vram.max_tokens && shotTokens(s, i) > 1.3 * vram.max_tokens)
add("warn", "much bigger than any shot that ran on this GPU (over 1.3×): it may run out of memory. Lower Megapixels or Max seconds / shot, use head 5, or set Between shots = free memory (Run tab)");
if (i > 0 && plan.run === "auto" && (s.chain === "head" || s.chain === "history") && !s.cut)
add("warn", `continuity '${s.chain}' does not work in the auto loop (the previous result does not exist yet): use Run = one run, shot by shot`);
});
return out;
});
@@ -188,7 +295,9 @@ const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return pla
if (!plan.video) return;
busy.value = "Detecting cuts…"; error.value = "";
try {
const body = { plan: { ...plan, bounds: apply ? [] : plan.bounds } };
// every detected boundary is kept (Max shots only picks how many shots run): asked for without the limit, the
// last kept shot would otherwise have no end and run to the end of the range
const body = { plan: { ...plan, bounds: apply ? [] : plan.bounds, max_parts: apply ? 0 : plan.max_parts } };
const r = await api.fetchApi("/bfs/shotloop/plan", { method: "POST", body: JSON.stringify(body) });
const j = await r.json();
if (j.error) throw new Error(j.error);
@@ -197,9 +306,11 @@ const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return pla
.map(s => [`${s.start}-${s.end}`, { people: s.people, main: s.main }]));
if (apply) {
plan.bounds = j.segs.slice(1).map(s => s.start);
const keep = plan.segs; plan.segs = j.segs.map((_, i) => ({ enabled: true, ref: keep[i]?.ref || "", ref2: keep[i]?.ref2 || "", prompt: keep[i]?.prompt || "",
chain: keep[i]?.chain || "off", chain_frame: keep[i]?.chain_frame || "first",
crop: !!keep[i]?.crop, inpaint: !!keep[i]?.inpaint, mask: keep[i]?.mask || {}, target: keep[i]?.target || "" }));
const keep = plan.segs; plan.segs = j.segs.map((_, i) => ({ enabled: true, refs: refList(keep[i]?.refs), prompt: keep[i]?.prompt || "",
chain: keep[i]?.chain || "head", chain_frame: keep[i]?.chain_frame || "first", chain_history: historyLen(keep[i]?.chain_history),
crop: !!keep[i]?.crop, inpaint: !!keep[i]?.inpaint, paste: !!keep[i]?.paste, paste_text: keep[i]?.paste_text || "", strength: keep[i]?.strength ?? 1, mask: keep[i]?.mask || {}, target: keep[i]?.target || "",
color_guide: !!keep[i]?.color_guide, color_guide_picture: keep[i]?.color_guide_picture || 1, color_guide_opacity: keep[i]?.color_guide_opacity ?? 0.35,
color_guide_regions: (keep[i]?.color_guide_regions || []).map(r => ({ ...r })) }));
sel.value = 0; save();
}
} catch (e) { error.value = String(e.message || e); }
@@ -228,7 +339,6 @@ const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return pla
}
const setCast = (id, k, v) => {
plan.cast = { ...plan.cast, [String(id)]: { ...castOf(id), [k]: v } };
if (k !== "ignore") remember(v);
save();
};
async function describeTarget(i) {
@@ -239,19 +349,21 @@ const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return pla
const j = await r.json(); if (j.error) throw new Error(j.error);
targetCrop.value = { ...targetCrop.value, [segKey(s)]: j.crop };
if (j.desc) setMeta(i, "target", j.desc);
else error.value = "No VLM yet: connect it to the planner's vlm input and run once, or type the description yourself.";
else error.value = "The SAM 3 target cutout was created (shown below), but no VLM is loaded to describe it. Connect Qwen3-VL to the Shot Planner's vlm input and queue the workflow once; then click Describe target again. You can also type the target description yourself.";
} catch (e) { error.value = String(e.message || e); }
busy.value = "";
}
// intoModal: every frame of the shot, for the points window's player (count 0)
async function previewMask(i, spec, intoModal = false) {
const s = segs.value[i]; if (!s) return;
const label = "Segmenting with SAM 3… (the first time downloads/loads the model)";
if (intoModal) { modal.busy = label; busySince.value = Date.now(); Object.assign(status, { stage: "", done: 0, total: 0 }); } else busy.value = label;
error.value = "";
try {
const r = await api.fetchApi("/bfs/shotloop/mask", { method: "POST", body: JSON.stringify({ plan: { ...plan }, index: i, mask: spec || null }) });
const body = { plan: { ...plan }, index: i, mask: spec || null, ...(intoModal ? { count: 0 } : {}) };
const r = await api.fetchApi("/bfs/shotloop/mask", { method: "POST", body: JSON.stringify(body) });
const j = await r.json(); if (j.error) throw new Error(j.error);
if (intoModal) modal.prev = j; else maskPrev.value = { ...maskPrev.value, [segKey(s)]: j };
if (intoModal) { modal.prev = j; modal.stale = false; modal.showMask = true; } else maskPrev.value = { ...maskPrev.value, [segKey(s)]: j };
} catch (e) { error.value = String(e.message || e); }
if (intoModal) modal.busy = ""; else busy.value = "";
}
@@ -262,14 +374,84 @@ const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return pla
if (s && maskPrev.value[segKey(s)]) { const mp = { ...maskPrev.value }; delete mp[segKey(s)]; maskPrev.value = mp; }
};
const setMaskCfg = (k, v) => { plan.mask_cfg = { ...plan.mask_cfg, [k]: v }; save(); maskPrev.value = {}; };
const openPoints = i => {
// The points window. A selection (region 1 = the shot's own mask, 2.. = its extra Colored Mark regions) has key
// frames, each with its own points: SAM 3 starts from a key frame and tracks up to the next one, so points added on a
// later frame correct the mask from there on. modal.key is the frame shown, modal.points its points (being edited).
const keyFramesOf = m => (Array.isArray(m?.keys) && m.keys.length ? m.keys.map(k => ({ key: +k.key || 0, points: [...(k.points || [])] }))
: (m?.points || []).length ? [{ key: +m.key || 0, points: [...m.points] }] : []);
const pointGroup = () => modal.groups[modal.region];
const keepFrame = () => { // the frame being edited back into its selection's key frames
const g = pointGroup(); if (!g) return;
g.keys = g.keys.filter(k => k.key !== modal.key);
if (modal.points.length) g.keys = [...g.keys, { key: modal.key, points: [...modal.points] }].sort((a, b) => a.key - b.key);
g.text = modal.text;
};
const showFrame = f => {
keepFrame();
modal.key = Math.max(0, Math.min((segs.value[modal.idx]?.len || 1) - 1, f));
modal.points = [...(pointGroup()?.keys.find(k => k.key === modal.key)?.points || [])];
};
const dropKeyFrame = f => {
const g = pointGroup(); g.keys = g.keys.filter(k => k.key !== f);
if (f === modal.key) modal.points = [];
modal.stale = !!modal.prev;
};
const openPoints = (i, region = 1) => {
const s = segs.value[i]; if (!s) return;
Object.assign(modal, { open: true, idx: i, key: s.mask.key ?? Math.floor(s.len / 2), points: [...(s.mask.points || [])],
text: s.mask.text || "", prev: null, busy: "" });
const groups = { 1: { keys: keyFramesOf(s.mask), text: s.mask.text || "" } };
if (s.color_guide) s.regions.forEach((r, j) => { groups[j + 2] = { keys: keyFramesOf(r), text: r.text || "" }; });
const r0 = groups[region] ? region : 1, key = groups[r0].keys[0]?.key ?? Math.floor(s.len / 2);
Object.assign(modal, { open: true, idx: i, region: r0, groups, key, points: [...(groups[r0].keys[0]?.points || [])],
text: groups[r0].text, prev: null, stale: false, showMask: true, busy: "" });
};
const switchPointRegion = region => {
keepFrame();
modal.region = region; modal.text = pointGroup()?.text || "";
modal.points = [...(pointGroup()?.keys.find(k => k.key === modal.key)?.points || [])]; modal.prev = null;
};
// the selection being edited as a mask spec (what Segment sends)
const pointSpec = () => { keepFrame(); const g = pointGroup(); return { keys: g.keys, points: g.keys[0]?.points || [], key: g.keys[0]?.key ?? modal.key, text: g.text }; };
const commitPointGroups = i => {
keepFrame();
const m = meta(i), g1 = modal.groups[1];
// keys is what 1.116 reads; points + key (the first key frame) for older versions
m.mask = { ...(m.mask || {}), keys: g1.keys, points: g1.keys[0]?.points || [], key: g1.keys[0]?.key ?? 0, text: g1.text };
if (segs.value[i]?.color_guide) {
m.color_guide_regions = markRegions(m).map((r, j) => {
const g = modal.groups[j + 2];
return g ? { ...r, keys: g.keys, points: g.keys[0]?.points || [], key: g.keys[0]?.key ?? 0, text: g.text } : r;
});
}
};
// the picture a region becomes (region 1 = the shot's own selection) and the extra regions
const setMarkPicture = (i, region, k) => {
const m = meta(i);
if (region === 1) m.color_guide_picture = k;
else m.color_guide_regions = markRegions(m).map((r, j) => (j === region - 2 ? { ...r, picture: k } : r));
save();
};
const addMarkRegion = i => {
const m = meta(i), used = new Set([+m.color_guide_picture || 1, ...markRegions(m).map(r => r.picture)]);
const n = Math.max(2, segs.value[i]?.names.length || 0);
let k = 1; while (used.has(k) && k < n) k++;
m.color_guide_regions = [...markRegions(m), { picture: k, text: "", points: [], key: 0 }]; save();
};
const setMarkRegion = (i, j, field, v) => {
const m = meta(i); m.color_guide_regions = markRegions(m).map((r, q) => (q === j ? { ...r, [field]: v } : r)); save();
};
const removeMarkRegion = i => j => { const m = meta(i); m.color_guide_regions = markRegions(m).filter((_, q) => q !== j); save(); };
const setPictureColor = (k, color) => {
const list = [...(plan.picture_colors || [])];
while (list.length < k) list.push(pictureColor(plan, list.length + 1));
list[k - 1] = color; plan.picture_colors = list; save();
};
const savePoints = () => {
const m = meta(modal.idx); m.mask = { ...(m.mask || {}), points: modal.points, key: modal.key, text: modal.text }; save();
if (modal.prev) maskPrev.value = { ...maskPrev.value, [segKey(segs.value[modal.idx])]: modal.prev };
const i = modal.idx; commitPointGroups(i); save();
if (modal.prev && !modal.stale) { // the card's strip shows 6 of the window's frames
const fr = modal.prev.frames, pick = [...new Set(Array.from({ length: Math.min(6, fr.length) }, (_, k) => Math.round(k * (fr.length - 1) / Math.max(1, Math.min(6, fr.length) - 1))))];
maskPrev.value = { ...maskPrev.value, [segKey(segs.value[modal.idx])]: { ...modal.prev, frames: pick.map(k => fr[k]) } };
}
clearInterval(modal.playing); modal.playing = 0;
modal.open = false;
};
const targets = (scope, from) => segs.value.map((_, i) => i).filter(i => i !== from && (scope === "all" || i > from));
@@ -279,34 +461,62 @@ const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return pla
const src = segs.value[fromIdx]; const rel = src ? key / Math.max(1, src.len - 1) : 0.5;
[fromIdx, ...targets(scope, fromIdx)].forEach(i => {
const x = segs.value[i], m = meta(i);
m.mask = { ...(m.mask || {}), points: points.map(p => ({ ...p })), key: Math.round(rel * Math.max(0, x.len - 1)), text: text ?? (m.mask || {}).text };
const key = Math.round(rel * Math.max(0, x.len - 1));
m.mask = { ...(m.mask || {}), keys: [{ key, points: points.map(p => ({ ...p })) }], points: points.map(p => ({ ...p })), key, text: text ?? (m.mask || {}).text };
});
maskPrev.value = {}; save();
};
const savePointsAll = () => { pointsTo(modal.points, modal.key, modal.text, modal.idx); modal.open = false; };
const savePointsAll = () => {
if (segs.value[modal.idx]?.color_guide) { commitPointGroups(modal.idx); save(); }
else pointsTo(modal.points, modal.key, modal.text, modal.idx);
modal.open = false;
};
// one selection tracked by SAM 3 through the whole video (or range): every shot without its own selection takes its
// part of that track, so the subject is found wherever it moves (copied points can fall on the background)
const saveTrack = () => {
const s0 = segs.value[modal.idx]; if (!s0 || !modal.points.length) return;
const own = segs.value.filter((x, i) => i !== modal.idx && (x.mask.text || (x.mask.points || []).length || (x.mask.keys || []).length)).length;
if (own && !window.confirm(`${own} other shot(s) have their own mask text or points, which would win over the track. Clear them so every shot uses the track?`)) {
// keep them: those shots keep their own selection
} else {
segs.value.forEach((_, i) => { const m = meta(i); if (m.mask) { m.mask = { ...m.mask, points: [], keys: [], text: "" }; } });
}
const m = meta(modal.idx); m.mask = { ...(m.mask || {}), points: [], keys: [], text: "" };
plan.track = { points: modal.points.map(p => ({ ...p })), frame: s0.start + modal.key, text: modal.text || "",
threshold: plan.mask_cfg?.threshold ?? 0.5 };
maskPrev.value = {}; save(); modal.open = false;
};
const clearTrack = () => { plan.track = null; maskPrev.value = {}; save(); };
const clearMask = i => {
const m = meta(i); m.mask = {}; m.crop = false; m.inpaint = false;
const m = meta(i); m.mask = {}; m.color_guide_regions = [];
m.crop = false; m.inpaint = false; m.paste = false;
const k = segs.value[i] ? segKey(segs.value[i]) : null;
if (k) { const mp = { ...maskPrev.value }; delete mp[k]; maskPrev.value = mp; }
save();
};
// copy the chosen settings of shot `from` to the other shots (all of them, or the ones after it)
const COPY_FIELDS = { ref: "references", prompt: "prompt", mask_text: "mask text", points: "mask points", crop: "mode",
target: "{target}", chain: "continuity" };
const COPY_FIELDS = { ref: "references", prompt: "prompt", mask_text: "mask text", points: "mask points", mask_video: "mask video", crop: "mode",
target: "{target}", chain: "continuity", color_guide: "Colored Mark" };
const copyFrom = (from, fields = copyOpts, scope = copyOpts.scope) => {
const src = segs.value[from]; if (!src) return 0;
const to = targets(scope, from);
to.forEach(i => {
const m = meta(i), x = segs.value[i];
if (fields.ref) { m.ref = src.ref; m.ref2 = src.ref2; }
if (fields.ref) m.refs = [...src.refs];
if (fields.prompt) m.prompt = src.prompt;
if (fields.mask_text) m.mask = { ...(m.mask || {}), text: src.mask.text || "" };
if (fields.crop && (hasMask(x) || src.mask.text || fields.mask_text)) { m.crop = src.crop; m.inpaint = src.inpaint; }
if (fields.mask_video) m.mask = { ...(m.mask || {}), video: src.mask.video || "" };
if (fields.crop && (hasMask(x) || src.mask.text || fields.mask_text)) { m.crop = src.crop; m.inpaint = src.inpaint; m.paste = src.paste; m.paste_text = src.pasteText; m.strength = src.strength; }
if (fields.target) m.target = src.target;
if (fields.chain && i > 0) { m.chain = src.chain; m.chain_frame = src.chainFrame; }
if (fields.color_guide) {
m.color_guide = src.color_guide; m.color_guide_picture = src.color_guide_picture;
m.color_guide_opacity = src.color_guide_opacity;
m.color_guide_regions = src.regions.map(r => ({ ...r, points: [...(r.points || [])] }));
}
if (fields.chain && i > 0) { m.chain = src.chain; m.chain_frame = src.chainFrame; m.chain_history = src.chainHistory; }
});
if (fields.points && (src.mask.points || []).length) pointsTo(src.mask.points, src.mask.key ?? Math.floor(src.len / 2), undefined, from, scope);
if (fields.mask_text || fields.points) maskPrev.value = {};
if (fields.mask_text || fields.points || fields.mask_video) maskPrev.value = {};
save();
return to.length;
};
@@ -320,26 +530,55 @@ const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return pla
} catch (e) { error.value = String(e.message || e); }
busy.value = "";
}
// reference sets used by the plan (global, every shot's, every cast person's): one description each
const refSets = computed(() => {
// reference sets used by the plan (global and per shot): one description each
// every picture the plan uses, once, with where it is <Picture n>: one description each ({details@Picture n})
const pictures = computed(() => {
const out = new Map();
const add = (a, b, where) => { if (!a && !b) return; const k = `${a || ""}|${b || ""}`; if (!out.has(k)) out.set(k, { a, b, where: [] }); out.get(k).where.push(where); };
add(plan.global_ref, plan.global_ref2, "global");
segs.value.forEach((x, i) => add(x.ref || plan.global_ref, x.ref2 || plan.global_ref2, `#${i + 1}`));
Object.entries(plan.cast || {}).forEach(([id, c]) => { if (!c.ignore) add(c.ref, c.ref2, `person ${id}`); });
return [...out.entries()].map(([key, v]) => ({ key, ...v }));
const add = (name, where) => { if (!name) return; if (!out.has(name)) out.set(name, []); out.get(name).push(where); };
plan.global_refs.forEach((x, k) => add(x, `global ${k + 1}`));
segs.value.forEach((x, i) => x.names.forEach((name, k) => { if (x.refs[k] === name) add(name, `#${i + 1}: ${k + 1}`); }));
return [...out.entries()].map(([name, where]) => ({ name, where }));
});
async function describeRefs() {
busy.value = "Describing the references with the VLM…"; error.value = "";
// names: the pictures to describe (all of them by default)
async function describePictures(names = pictures.value.map(x => x.name)) {
busy.value = "Describing the pictures with the VLM…"; error.value = "";
try {
const sets = refSets.value.map(x => [x.a || "", x.b || ""]);
const r = await api.fetchApi("/bfs/shotloop/describe", { method: "POST", body: JSON.stringify({ plan: { ...plan }, sets }) });
const r = await api.fetchApi("/bfs/shotloop/describe", { method: "POST",
body: JSON.stringify({ plan: { ...plan }, pictures: names }) });
const j = await r.json(); if (j.error) throw new Error(j.error);
plan.ref_details = { ...plan.ref_details, ...j.texts }; save();
plan.picture_details = { ...plan.picture_details, ...j.pictures }; save();
} catch (e) { error.value = String(e.message || e); }
busy.value = "";
}
const setDetail = (k, v) => { plan.ref_details = { ...plan.ref_details, [k]: v }; save(); };
const setPictureDetail = (name, v) => { plan.picture_details = { ...plan.picture_details, [name]: v }; save(); };
// a picture's own describe preset ("" = the card's) and custom instruction
// 🤖 Auto prompt: what a shot's written prompt was made from (same string as bfs_shot_loop.auto_prompt_for), so the
// panel shows when the request, pictures, marks or mode changed after it was written
const autoFor = s => {
const marks = s.color_guide ? [[s.color_guide_picture, pictureColor(plan, s.color_guide_picture)],
...s.regions.filter(r => String(r.text || "").trim() || (r.points || []).length).map(r => [r.picture, pictureColor(plan, r.picture)])] : [];
return JSON.stringify([s.prompt || plan.global_prompt || "", s.names, marks, modeOf(s), s.mask?.text || "", s.target || "",
plan.auto_prompt?.template || "", plan.auto_prompt?.style || "h3"]);
};
const writtenOf = s => plan.auto_prompts?.[segKey(s)] || null;
const setAuto = (k, v) => { plan.auto_prompt = { ...plan.auto_prompt, [k]: v }; save(); };
const setWritten = (s, text) => {
const k = segKey(s), cur = plan.auto_prompts[k] || {};
plan.auto_prompts = { ...plan.auto_prompts, [k]: { ...cur, text, for: cur.for ?? autoFor(s) } }; save();
};
// the VLM writes the prompts of these shots (all by default)
async function writePrompts(indices = segs.value.map((_, i) => i)) {
busy.value = `Writing ${indices.length} prompt(s) with the VLM…`; error.value = "";
try {
const r = await api.fetchApi("/bfs/shotloop/autoprompt", { method: "POST", body: JSON.stringify({ plan: { ...plan }, indices }) });
const j = await r.json(); if (j.error) throw new Error(j.error);
plan.auto_prompts = { ...plan.auto_prompts, ...j.prompts }; save();
} catch (e) { error.value = String(e.message || e); }
busy.value = "";
}
const setPictureCfg = (name, k, v) => {
plan.picture_cfg = { ...plan.picture_cfg, [name]: { ...(plan.picture_cfg[name] || {}), [k]: v } }; save();
};
const setVlmCfg = (k, v) => { plan.vlm_cfg = { ...plan.vlm_cfg, [k]: v }; save(); };
const applySug = (i, what) => {
const s = segs.value[i], g = s && vlmSug.value[segKey(s)]; if (!g) return;
@@ -358,6 +597,10 @@ const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return pla
const setCastOpt = (k, v) => { plan[k] = v; save(); if (k === "cast_split") autoSplit(true); };
const setFilter = (k, v) => { plan.filters = { ...plan.filters, [k]: v }; save(); if (stats.value.length) analyzeContent(); };
const setPlan = (k, v, after) => { plan[k] = v; save(); after && after(); };
// the range changed: split again inside it (the per-shot settings stay with the shot at the same position)
// start / end in timeline frames (end 0 = the end of the video); the old seconds fields are cleared
const setRange = (k, v) => { plan[k] = Math.max(0, Math.round(+v || 0)); plan.start_s = 0; plan.end_s = 0; save(); autoSplit(true); };
const setGlobalMaskVideo = v => { plan.mask_video = v; save(); maskPrev.value = {}; };
// XHR instead of fetch: big videos show how much has been sent
const postWithProgress = (url, body, onPct) => new Promise((resolve, reject) => {
const x = new XMLHttpRequest();
@@ -368,14 +611,39 @@ const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return pla
x.onerror = () => reject(new Error("network error"));
x.send(body);
});
// big files go in 8 MB pieces: ComfyUI refuses any single request over --max-upload-size (100 MB by default)
const CHUNK = 8 * 1024 * 1024, CHUNKED_FROM = 32 * 1024 * 1024;
async function uploadChunked(file, onPct) {
const id = `${Date.now().toString(36)}_${Math.random().toString(36).slice(2, 12)}`;
for (let off = 0; off < file.size; off += CHUNK) {
const piece = file.slice(off, Math.min(file.size, off + CHUNK));
for (let tries = 0; ; tries++) { // a dropped piece is sent again (it is written at its offset)
try {
await postWithProgress(`/bfs/upload/chunk?id=${id}&offset=${off}`, piece,
f => onPct((off + f * piece.size) / file.size));
break;
} catch (e) { if (tries >= 3) throw e; await new Promise(r => setTimeout(r, 1000 * (tries + 1))); }
}
}
const r = await fetch(api.apiURL("/bfs/upload/finish"), { method: "POST", headers: { "Content-Type": "application/json" },
body: JSON.stringify({ id, name: file.name, size: file.size }) });
const j = await r.json();
if (!r.ok || j.error) throw new Error(j.error || `HTTP ${r.status}`);
return j.name;
}
async function upload(file, cb) {
const fd = new FormData(); fd.append("image", file); fd.append("type", "input"); fd.append("overwrite", "true");
const mb = (file.size / 1e6).toFixed(1);
busy.value = `Uploading ${file.name} (${mb} MB)…`; upPct.value = 0;
try {
const j = await postWithProgress("/upload/image", fd, f => { upPct.value = f; });
let name;
if (file.size >= CHUNKED_FROM) name = await uploadChunked(file, f => { upPct.value = f; });
else {
const fd = new FormData(); fd.append("image", file); fd.append("type", "input"); fd.append("overwrite", "true");
const j = await postWithProgress("/upload/image", fd, f => { upPct.value = f; });
name = j.subfolder ? `${j.subfolder}/${j.name}` : j.name;
}
await refreshFiles();
cb(j.subfolder ? `${j.subfolder}/${j.name}` : j.name);
cb(name);
} catch (e) { error.value = `Upload failed: ${e}`; }
busy.value = "";
}
@@ -383,20 +651,65 @@ const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return pla
const inp = document.createElement("input"); inp.type = "file"; inp.accept = accept;
inp.onchange = () => inp.files[0] && upload(inp.files[0], cb); inp.click();
};
// several files: uploaded one after the other, cb gets the names in order
const uploadAll = async (files, cb) => {
const names = [];
for (const f of files) await upload(f, name => names.push(name));
if (names.length) cb(names);
};
const pickFiles = (accept, cb) => {
const inp = document.createElement("input"); inp.type = "file"; inp.accept = accept; inp.multiple = true;
inp.onchange = () => uploadAll([...inp.files], cb); inp.click();
};
async function progress(reset = false) {
if (!plan.video || plan.run !== "queue") return;
if (!plan.video || plan.run === "auto") { prog.done = 0; prog.count = 0; Object.assign(cache, { ok: [], stored: [], mb: 0 }); return; }
try {
const r = await api.fetchApi("/bfs/shotloop/progress", { method: "POST", body: JSON.stringify({ plan: { ...plan }, plan_raw: io.getPlan(), reset }) });
const j = await r.json(); if (!j.error) { prog.done = j.done; prog.count = j.count || active.value.length; }
const j = await r.json();
if (!j.error) { prog.done = j.done; prog.count = j.count || active.value.length; Object.assign(cache, j); }
} catch { /* ignore */ }
}
// ---- stored shots: redo some, or combine what is stored into an MP4 (low memory, on the server)
async function redo(shots, cascade = true) {
try {
const r = await api.fetchApi("/bfs/shotloop/redo", { method: "POST", body: JSON.stringify({ plan: { ...plan }, shots, cascade }) });
const j = await r.json(); if (j.error) error.value = j.error; else { Object.assign(cache, j); prog.done = j.done; }
} catch (e) { error.value = `${e}`; }
}
async function combine() {
busy.value = "Combining the stored shots into an MP4";
try {
const r = await api.fetchApi("/bfs/shotloop/combine", { method: "POST", body: JSON.stringify({ plan: { ...plan },
crossfade: plan.combine_crossfade ?? 4, crf: plan.combine_crf ?? 17 }) });
const j = await r.json();
if (j.error) error.value = j.error;
else cache.combined = { ...j, url: api.apiURL(`/view?filename=${encodeURIComponent(j.filename)}&subfolder=${encodeURIComponent(j.subfolder)}&type=output`) };
} catch (e) { error.value = `${e}`; }
busy.value = "";
}
// a shot of the panel (any, skipped ones too) -> its cache state: "ok" stored and up to date, "run" runs next time
const cacheOf = s => { const k = active.value.indexOf(s); return k < 0 || !cache.ok.length ? "" : cache.ok[k] ? "ok" : "run"; };
// ---- editing
const setVideo = v => { plan.video = v; plan.bounds = []; plan.segs = []; save(); analyze(); if (v) tab.value = "shots"; };
const setVideo = v => {
plan.video = v; plan.bounds = []; plan.segs = [];
if (v) {
// A new source starts as one manageable clip. The user can widen the range or ask for an automatic split later;
// do not surprise them with a wall of detected shots on first load.
plan.mode = "single";
plan.start_f = 0;
plan.end_f = Math.round(15 * plan.fps);
plan.start_s = 0; plan.end_s = 0; plan.max_total_s = 0;
tab.value = "video";
}
save(); analyze();
};
const splitAt = f => {
const N = n.value; f = Math.round(f);
if (f <= 0 || f >= N || plan.bounds.includes(f)) return;
const { lo, hi } = range.value; f = Math.round(f);
if (f <= lo || f >= hi || plan.bounds.includes(f)) return;
const i = segs.value.findIndex(s => f > s.start && f < s.end);
// a split you make by hand always shows: "Max shots" (e.g. 1 in a one-shot workflow) only limits detected shots
if (plan.max_parts > 0 && allSegCount.value >= plan.max_parts) plan.max_parts = allSegCount.value + 1;
plan.bounds = [...plan.bounds, f].sort((a, c) => a - c);
plan.segs.splice(i + 1, 0, { ...(plan.segs[i] || {}) });
sel.value = i + 1; save();
@@ -425,14 +738,14 @@ const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return pla
removeCut(sel.value);
};
const setMeta = (i, k, v) => { meta(i)[k] = v; save(); };
const setMode = (i, mode) => { const m = MODES.find(x => x[0] === mode); Object.assign(meta(i), { crop: m[2], inpaint: m[3] }); save(); };
const setMode = (i, mode) => { const m = MODES.find(x => x[0] === mode); Object.assign(meta(i), { crop: m[2], inpaint: m[3], paste: m[4] }); save(); };
const drag = (k, ev) => {
ev.preventDefault(); ev.stopPropagation();
const box = tlEl.value.getBoundingClientRect();
const move = e => {
const x = e.clientX - box.left + tlEl.value.scrollLeft;
const f = Math.round(x / pxPerFrame.value);
const lo = (plan.bounds[k - 1] ?? 0) + 1, hi = (plan.bounds[k + 1] ?? n.value) - 1;
const lo = (plan.bounds[k - 1] ?? range.value.lo) + 1, hi = (plan.bounds[k + 1] ?? range.value.hi) - 1;
plan.bounds[k] = Math.max(lo, Math.min(hi, f));
hover.value = { frame: plan.bounds[k], x };
};
@@ -480,52 +793,120 @@ const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return pla
const seek = f => { const v = vid.value; if (v) { v.currentTime = f / plan.fps + 1e-3; play.frame = f; } };
// ---- queue loop events
const onProg = e => { prog.done = e.detail.done; prog.count = e.detail.count; };
const onProg = e => { prog.done = e.detail.done; prog.count = e.detail.count; refreshVram(); };
const onStatus = e => { if (busy.value || modal.busy) Object.assign(status, e.detail || {}); };
let clock = 0;
const onVlm = e => { if (e.detail?.video === plan.video) mergeSug(e.detail.segs); };
// prompts the planner's VLM wrote while running (shots that had none): kept in the plan, so they stay the same
const onAutoPrompt = e => {
const d = e.detail; if (d?.video !== plan.video) return;
const tpl = typeof d.template === "string" && d.template !== (plan.auto_prompt.template || "");
if (tpl) plan.auto_prompt = { ...plan.auto_prompt, template: d.template }; // the prompt input of the last run
if (tpl || Object.keys(d.prompts || {}).length) { plan.auto_prompts = { ...plan.auto_prompts, ...(d.prompts || {}) }; save(); }
};
const onNext = e => { onProg(e); if (plan.run === "queue" && plan.auto_continue) setTimeout(() => app.queuePrompt(0, 1), 300); };
onMounted(() => {
load(); refreshFiles(); analyze(); progress(); raf = requestAnimationFrame(loop);
load(); refreshFiles(); analyze(); progress(); refreshVram(); raf = requestAnimationFrame(loop);
if (!plan.video) tab.value = "video";
api.addEventListener("bfs-shotloop-progress", onProg); api.addEventListener("bfs-shotloop-next", onNext);
api.addEventListener("bfs-shotloop-vlm", onVlm);
api.addEventListener("bfs-shotloop-autoprompt", onAutoPrompt);
api.addEventListener("bfs-shotloop-status", onStatus);
clock = setInterval(() => { if (busy.value || modal.busy) now.value = Date.now(); }, 500);
});
onBeforeUnmount(() => { cancelAnimationFrame(raf); api.removeEventListener("bfs-shotloop-progress", onProg); api.removeEventListener("bfs-shotloop-next", onNext); api.removeEventListener("bfs-shotloop-vlm", onVlm); api.removeEventListener("bfs-shotloop-status", onStatus); clearInterval(clock); });
onBeforeUnmount(() => { cancelAnimationFrame(raf); api.removeEventListener("bfs-shotloop-progress", onProg); api.removeEventListener("bfs-shotloop-next", onNext); api.removeEventListener("bfs-shotloop-vlm", onVlm); api.removeEventListener("bfs-shotloop-autoprompt", onAutoPrompt); api.removeEventListener("bfs-shotloop-status", onStatus); clearInterval(clock); });
io.expose({ reload: () => { load(); analyze(); progress(); } });
// 🧠 the memory status at the top: the plan's biggest shot against the biggest one that ran on this GPU
const kTok = n => (n >= 1000 ? `${(n / 1000).toFixed(n >= 100000 ? 0 : 1)}k` : `${n}`);
function memTip(m) {
const gpu = vram.gpu ? `${vram.gpu} · ${vram.total_gb} GB (${vram.free_gb} GB free now)` : "no CUDA GPU seen by the server";
const top = m.top.i >= 0 ? `Biggest shot in this plan: #${m.top.i + 1}, ${m.top.gen} frames at ${size.w}×${size.h} = ${kTok(m.top.tok)} tokens` : "";
const ref = vram.max_tokens
? `Biggest shot that ran on this GPU: ${kTok(vram.max_tokens)} tokens, peak ${vram.max_peak_gb} GB (${vram.runs} run(s) recorded)`
: "No shot has run on this GPU yet: after the first one, this status compares every shot with what already ran here.";
const say = { ok: "✓ Every shot is within what already ran on this GPU.",
note: "A bit bigger than anything that ran here: it will probably fit (ComfyUI offloads weights), maybe slower.",
warn: "⚠ Much bigger than anything that ran here: it may run out of memory. Lower Megapixels or Max seconds / shot, use head 5, or set Between shots = free memory (Run tab).",
info: "" }[m.level];
return [gpu, top, ref, say, "Size = generated frames (with the continuity head) × resolution; skipped shots are not counted."].filter(Boolean).join("\n");
}
function memPill() {
const m = memory.value;
if (!m.level) return null;
const text = { ok: "🧠 memory ✓", note: `🧠 ${m.ratio.toFixed(2)}× biggest run`, warn: `🧠 ⚠ ${m.ratio.toFixed(1)}× biggest run · may OOM`,
info: "🧠 no memory data yet" }[m.level];
return h("span", { class: ["pill", { ok: "ok", warn: "warn" }[m.level] || ""], style: "cursor:pointer", title: memTip(m),
onClick: () => { tab.value = "run"; refreshVram(); } }, text);
}
// ---- references
// the last 10 references picked, remembered across workflows (per browser)
const RECENT_KEY = "bfs.shotloop.recentRefs";
const recent = ref(store.get(RECENT_KEY, []));
const isColorValue = name => typeof name === "string" && /^#[0-9a-f]{6}$/i.test(name);
const savedRecent = store.get(RECENT_KEY, []);
const recent = ref((Array.isArray(savedRecent) ? savedRecent : []).filter(name => typeof name === "string" && !isColorValue(name)));
if (recent.value.length !== (Array.isArray(savedRecent) ? savedRecent.length : 0)) store.set(RECENT_KEY, recent.value);
const remember = name => {
if (!name) return;
if (!name || typeof name !== "string" || isColorValue(name)) return;
recent.value = [name, ...recent.value.filter(x => x !== name)].slice(0, 10);
store.set(RECENT_KEY, recent.value);
};
// target: shot index or "global"; field: ref | ref2 (global_ref | global_ref2 for "global")
const setRef = (target, field, name) => {
if (target === "global") plan[field] = name; else meta(target)[field] = name;
// target: shot index or "global"; k: the picture's slot (0 = <Picture 1>). "" removes it: a global picture is taken
// out of the list (the next ones move up), a shot's own picture falls back to the global one at that slot
const refsOf = target => (target === "global" ? plan.global_refs : (meta(target).refs ||= []));
const setRef = (target, k, name) => {
if (isColorValue(name)) return; // colors belong to Colored Mark settings, never to image references
const list = refsOf(target);
if (name) { while (list.length < k) list.push(""); list[k] = name; }
else if (target === "global" || k >= plan.global_refs.length) list.splice(k, 1);
else list[k] = "";
remember(name); save();
};
const uploadRef = (target, field) => pickFile("image/*", name => setRef(target, field, name));
// pictures put in from slot k on (k = the list's end: added), never past 12
const putRefs = (target, k, names) => {
const list = refsOf(target);
names.slice(0, Math.max(0, MAX_REFS - k)).forEach((name, j) => { while (list.length < k + j) list.push(""); list[k + j] = name; remember(name); });
save();
};
const uploadRef = (target, k, many = false) => many
? pickFiles("image/*", names => putRefs(target, k, names))
: pickFile("image/*", name => setRef(target, k, name));
// a drop on slot k replaces it (more files go on the next slots); k = null (the box): added at the end. A recent
// thumbnail dragged in carries its name, image files are uploaded
const dropRefs = (target, k, e) => {
const at = k ?? refsOf(target).length;
const name = e.dataTransfer?.getData("text/bfs-ref");
if (name) { putRefs(target, at, [name]); return; }
const files = [...(e.dataTransfer?.files || [])].filter(f => f.type.startsWith("image/"));
if (files.length) uploadAll(files, names => putRefs(target, at, names));
};
const clearRecent = () => { recent.value = []; store.set(RECENT_KEY, []); };
// everything: the recent list and every reference this plan uses (global and per shot)
const clearAllRefs = () => {
if (!confirm("Remove every reference picture from this plan (global, every shot) and forget the recent list?")) return;
clearRecent();
plan.global_refs = [];
plan.segs.forEach(m => { if (m) m.refs = []; });
save();
};
const usedRefs = computed(() => {
const set = new Set([...recent.value, plan.global_ref, plan.global_ref2, ...plan.segs.flatMap(m => [m?.ref, m?.ref2])].filter(Boolean));
const set = new Set([...recent.value, ...plan.global_refs, ...plan.segs.flatMap(m => m?.refs || [])].filter(Boolean));
return [...set].slice(0, 10);
});
// everything the views need
const c = {
api, plan, files, an, cuts, detectorUsed, size, busy, error, status, busySince, now, upPct, sel, zoom, hover, prog, tlEl,
stats, people, maskPrev, showMasks, vlmSug, modal, segPeople, targetCrop, tab, copyOpts, COPY_FIELDS, vid, play, recent,
n, maxLen, segs, active, pxPerFrame, tlWidth, filtersOn, refSets, usedRefs, checks,
MODES, modeOf, modeName, viewUrl, save, meta, setMeta, setMode, setPlan, whoIn, castOf, linked, castRef, personOf, statFor, skipWhy, refOf, promptOf, checksFor,
stats, people, maskPrev, showMasks, vlmSug, modal, segPeople, targetCrop, tab, copyOpts, COPY_FIELDS, vid, play, recent, clearRecent, clearAllRefs,
n, range, setRange, maxLen, segs, allSegCount, active, pxPerFrame, tlWidth, filtersOn, usedRefs, checks, tour, memory, vram, refreshVram, cache, cacheOf, redo, combine,
MODES, modeOf, modeName, viewUrl, save, meta, setMeta, setMode, setPlan, whoIn, castOf, included, personOf, statFor, skipWhy, refOf, promptOf, checksFor,
refreshFiles, analyze, autoSplit, analyzeContent, findPeople, setCast, describeTarget, previewMask, setMask, setMaskCfg,
openPoints, savePoints, savePointsAll, pointsTo, clearMask, copyFrom, analyseVLM, describeRefs, setDetail, setVlmCfg,
applySug, applyAllSug, setCastOpt, setFilter, pickFile, progress, setVideo, splitAt, mergeNext, removeCut, select, drag,
frameAt, thumbFor, playShot, playAll, stop, seek, setRef, uploadRef,
openPoints, switchPointRegion, savePoints, savePointsAll, saveTrack, clearTrack, pointsTo, clearMask, copyFrom, analyseVLM, setVlmCfg,
applySug, applyAllSug, setCastOpt, setGlobalMaskVideo, setFilter, pickFile, progress, setVideo, splitAt, mergeNext, removeCut, select, drag,
frameAt, thumbFor, playShot, playAll, stop, seek, setRef, putRefs, uploadRef, dropRefs, showFrame, dropKeyFrame, pointSpec, keyFramesOf, pictures, describePictures, setPictureDetail, setPictureCfg,
setMarkPicture, addMarkRegion, setMarkRegion, removeMarkRegion, setPictureColor,
autoFor, writtenOf, setAuto, setWritten, writePrompts,
};
return () => {
@@ -534,10 +915,20 @@ const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return pla
const guideUrl = new URL("./docs/BFSShotPlanner.md", import.meta.url).href;
const header = h("div", { class: "hdr" }, [
h("span", { class: "ttl" }, "🎬 Shot Planner"),
an.value ? pill(`${active.value.length}/${S.length} shots · ${(N / fps).toFixed(1)}s`, "ok") : null,
an.value ? pill(`${active.value.length}/${S.length} shots · ${((range.value.hi - range.value.lo) / fps).toFixed(1)}s`
+ (range.value.lo > 0 || range.value.hi < N ? " (range)" : ""), "ok") : null,
warns ? h("span", { class: "pill warn", style: "cursor:pointer", title: "open the checks (Run tab)", onClick: () => { tab.value = "run"; } }, `⚠ ${warns}`) : null,
h("span", { class: "grow" }),
pill(plan.run === "queue" ? "queue loop" : "auto loop"),
h("label", { class: "langpick", title: t("Language") }, [
h("span", { class: "hint" }, "🌐"),
h("select", { value: uiLanguage.value, "aria-label": t("Language"), onChange: e => setUiLanguage(e.target.value) }, [
rawH("option", { value: "en" }, "English"), rawH("option", { value: "pt" }, "Português"),
rawH("option", { value: "es" }, "Español"), rawH("option", { value: "zh" }, "中文"),
]),
]),
memPill(),
pill({ queue: "queue loop", recursive: "shot by shot" }[plan.run] || "auto loop"),
h("button", { title: t("Shot Planner tour"), onClick: () => { tour.value = 1; tab.value = "video"; } }, t("🧭 Tour")),
h("button", { title: "Open the full guide (every setting explained, with examples)", onClick: () => showGuide(guideUrl) }, "📖 Guide"),
]);
const counts = { shots: S.length || null, people: people.value.length || null };
@@ -554,12 +945,86 @@ const meta = i => { while (plan.segs.length <= i) plan.segs.push({}); return pla
an.value ? timelineCard(c) : null,
views[cur](c),
pointsModal(c),
tourModal({ tour, tab }),
]);
};
}
// A workflow saved before 1.113 keeps the old slots of these nodes (ComfyUI restores a node's saved slots): the
// planner's ref_image / ref_image_2 inputs are gone, the planner's and Unpack's ref_image / ref_image_2 outputs are now
// refs / ref_count, and what fed Repack's ref_image now feeds its refs input.
// Saved widget orders of the conditioning nodes (ComfyUI restores values by position): use_ref_2 (always second) left
// in 1.114 and the Advanced node's duet widgets in 1.113, so older values are put back by name
const OLD_ADVANCED_WIDGETS = ["guide_mode", "use_ref_2", "first_frame", "ref_image_size", "with_audio", "duet",
"panel_position", "panel_size", "panel_noise", "rope_gap", "task", "instruction", "setting_ref", "inpaint", "mask_guide",
"source_blur", "source_blur_strength", "source_blur_segmenter", "source_blur_effect", "mask_guide_look", "mask_ref_size"];
const ADVANCED_1113 = ["guide_mode", "use_ref_2", "first_frame", "ref_image_size", "with_audio", "setting_ref", "inpaint",
"mask_guide", "source_blur", "source_blur_strength", "source_blur_segmenter", "source_blur_effect", "mask_guide_look", "mask_ref_size"];
const OLD_SIMPLE_WIDGETS = ["guide_mode", "use_ref_2", "ref_image_size", "with_audio", "inpaint", "source_blur",
"source_blur_strength", "source_blur_segmenter", "source_blur_effect"];
const NO_BLUR_PICKERS = n => n !== "source_blur_segmenter" && n !== "source_blur_effect"; // added later
function oldWidgetNames(cls, v) {
if (typeof v[1] !== "boolean") return null; // already without use_ref_2
if (cls === "BFSShotH3ConditioningSimple") return v.length >= 9 ? OLD_SIMPLE_WIDGETS : OLD_SIMPLE_WIDGETS.filter(NO_BLUR_PICKERS);
if (["off", "canvas", "shifted RoPE"].includes(v[5]) && ["left", "right", "top", "bottom"].includes(v[6]))
return v.length >= 21 ? OLD_ADVANCED_WIDGETS : OLD_ADVANCED_WIDGETS.filter(NO_BLUR_PICKERS);
return v.length >= 14 ? ADVANCED_1113 : ADVANCED_1113.filter(NO_BLUR_PICKERS);
}
function upgradeConditioningWidgets(node, info) {
const v = info?.widgets_values;
const names = Array.isArray(v) && oldWidgetNames(node.comfyClass, v);
if (!names) return;
names.forEach((name, k) => { const w = node.widgets?.find(x => x.name === name); if (w && k < v.length) w.value = v[k]; });
// widgets the saved workflow did not have got a shifted value: back to their default (a list's first choice)
node.widgets?.forEach(w => {
const choices = w.options?.values;
if (!names.includes(w.name) && Array.isArray(choices) && !choices.includes(w.value)) w.value = w.options.default ?? choices[0];
});
}
function upgradeLegacySlots(node, info) {
const cls = node.comfyClass;
if (cls === "BFSShotH3Conditioning" || cls === "BFSShotH3ConditioningSimple") { upgradeConditioningWidgets(node, info); return; }
// ref_image / ref_image_2 outputs at slots `at`, at+1 -> refs (list) / ref_count
const listOutputs = at => {
const a = node.outputs?.[at], b = node.outputs?.[at + 1];
if (a?.name === "ref_image") Object.assign(a, { name: "refs", label: undefined, localized_name: "refs", shape: 6 });
if (b?.name === "ref_image_2") { node.disconnectOutput(at + 1); Object.assign(b, { name: "ref_count", label: undefined, localized_name: "ref_count", type: "INT", shape: undefined }); }
};
if (cls === "BFSShotPlanner") {
for (const name of ["ref_image", "ref_image_2"]) {
const k = node.inputs?.findIndex(x => x.name === name) ?? -1;
if (k >= 0) node.removeInput(k);
}
listOutputs(8);
} else if (cls === "BFSShotUnpack") {
listOutputs(1);
} else if (cls === "BFSShotRepack") {
// what fed ref_image now feeds refs; ref_image / ref_image_2 go away
const at = name => node.inputs?.findIndex(x => x.name === name) ?? -1;
for (const name of ["ref_image", "ref_image_2"]) {
const k = at(name); if (k < 0) continue;
const lid = node.inputs[k].link, l = lid == null ? null : (node.graph?.links?.get?.(lid) ?? node.graph?.links?.[lid]);
const origin = l ? node.graph.getNodeById(l.origin_id) : null;
node.removeInput(k);
const to = at("refs");
if (name === "ref_image" && origin && to >= 0 && node.inputs[to].link == null) origin.connect(l.origin_slot, node, to);
}
} else return;
node.setDirtyCanvas?.(true, true);
}
app.registerExtension({
name: "BFSNodes.ShotPlanner",
async beforeRegisterNodeDef(nodeType, nodeData) {
if (!["BFSShotPlanner", "BFSShotUnpack", "BFSShotRepack", "BFSShotH3Conditioning", "BFSShotH3ConditioningSimple"].includes(nodeData.name)) return;
const prev = nodeType.prototype.onConfigure;
nodeType.prototype.onConfigure = function (info) {
const r = prev?.apply(this, arguments);
upgradeLegacySlots(this, info);
return r;
};
},
async nodeCreated(node) {
if (node.comfyClass !== "BFSShotPlanner") return;
styles();
@@ -573,6 +1038,7 @@ app.registerExtension({
// node (Vue nodes mode sizes widgets by their content); min-width keeps it from collapsing to its content width
host.style.cssText = "position:relative;width:100%;height:100%;min-width:900px;min-height:640px";
node.addDOMWidget("bfs_shot_planner", "div", host, { serialize: false, hideOnZoom: false });
wheelToCanvas(host); // the wheel zooms the canvas over the panel too, except on something that scrolls
const handle = {};
createApp({
setup: () => Panel({
+492
View File
@@ -0,0 +1,492 @@
/**
* BFS Video Compare: one player for two videos.
*
* Wipe (drag the divider), overlay (drag to fade) or side by side, with any of guide / target / difference on top
* or below. The strip under the scrubber is the difference per frame, measured by the node on the real frames:
* click it to jump to a frame. The node sends three temporary MP4s and that list when it runs.
*
* ⬇ save plays the video once from the start and downloads it as shown (same layout, divider, labels); ● rec records
* whatever you do in the player (dragging the divider, pausing, scrubbing) until ■ stop. Both draw the two videos into
* a canvas and record it in the browser, in real time. The browser's file (WebM, or a fragmented MP4 many apps won't
* play) is then converted by ComfyUI to the chosen format: MP4 (H.264), a small MP4 (under 10 MB) or a GIF.
*/
import { app } from "../../scripts/app.js";
import { api } from "../../scripts/api.js";
import { wheelToCanvas } from "./bfs_wheel.js";
const SOURCES = ["guide", "target", "difference"];
const MODES = [["wipe", "wipe ↔"], ["wipe-v", "wipe ↕"], ["overlay", "overlay"], ["side", "side by side"]];
const FORMATS = [["mp4", "MP4"], ["small", "MP4 small (<10 MB)"], ["gif", "GIF"], ["webm", "browser file (WebM)"]];
const FORMAT_KEY = "bfs.videoCompare.format";
const PART = 8 * 1024 * 1024; // upload pieces, under ComfyUI's request limit
function styles() {
if (document.getElementById("bfs-vcmp-css")) return;
const el = document.createElement("style");
el.id = "bfs-vcmp-css";
el.textContent = `
.bfsvc{display:flex;flex-direction:column;gap:6px;height:100%;box-sizing:border-box;padding:6px;
font:12px/1.4 var(--font-family,system-ui,sans-serif);color:var(--descrip-text,#c9c9cf);
background:var(--comfy-input-bg,#1b1b1f);border-radius:8px}
.bfsvc .stage{position:relative;flex:1 1 0;min-height:160px;background:#0e0e11;border-radius:6px;overflow:hidden;
cursor:ew-resize;touch-action:none;user-select:none}
.bfsvc .stage.v{cursor:ns-resize}.bfsvc .stage.o{cursor:col-resize}.bfsvc .stage.s{cursor:default}
.bfsvc video{position:absolute;inset:0;width:100%;height:100%;object-fit:contain;pointer-events:none}
.bfsvc .stage.s video{width:calc(50% - 2px)}.bfsvc .stage.s video.r{left:auto;right:0}
.bfsvc .refs{flex:0 0 auto;height:84px;display:flex;gap:6px;justify-content:center;overflow-x:auto;padding:4px;
background:#0e0e11;border-radius:6px}
.bfsvc .refs figure{margin:0;height:100%;position:relative;flex:none}
.bfsvc .refs img{height:100%;border-radius:4px;box-shadow:0 0 0 1px #000a;background:#000;display:block}
.bfsvc .refs figcaption{position:absolute;left:3px;top:2px;font-size:10px;padding:0 4px;border-radius:3px;background:#000a;color:#fff}
.bfsvc .line{position:absolute;background:#fff;box-shadow:0 0 0 1px #0008;pointer-events:none}
.bfsvc .tag{position:absolute;top:6px;padding:1px 7px;border-radius:4px;background:#000a;color:#fff;font-size:11px;
pointer-events:none}
.bfsvc .row{display:flex;gap:6px;align-items:center;flex-wrap:wrap}
.bfsvc select,.bfsvc button{background:#131317;border:1px solid #34343d;color:#e6e6ea;border-radius:4px;
padding:2px 6px;font:inherit;cursor:pointer}
.bfsvc button:hover,.bfsvc select:hover{border-color:#3a86ff}
.bfsvc button.rec{border-color:#e0573a;color:#ff7a5c}
.bfsvc button:disabled{opacity:.45;cursor:default}
.bfsvc label{display:flex;gap:3px;align-items:center;cursor:pointer}
.bfsvc input[type=range]{flex:1;accent-color:#3a86ff;min-width:80px}
.bfsvc canvas{width:100%;height:26px;cursor:pointer;border-radius:3px;background:#131317;display:block}
.bfsvc .info{font-family:ui-monospace,monospace;font-size:11px;white-space:nowrap}
.bfsvc .empty{position:absolute;inset:0;display:flex;align-items:center;justify-content:center;color:#8b8b93;
text-align:center;padding:12px}
`;
document.head.appendChild(el);
}
const url = f => api.apiURL(`/view?filename=${encodeURIComponent(f.filename)}&subfolder=${encodeURIComponent(f.subfolder || "")}`
+ `&type=${f.type || "temp"}&t=${Date.now()}`);
function el(tag, attrs = {}, kids = []) {
const e = document.createElement(tag);
for (const [k, v] of Object.entries(attrs)) {
if (k === "style") e.style.cssText = v; else if (k.startsWith("on")) e.addEventListener(k.slice(2), v);
else e.setAttribute(k, v);
}
for (const c of [].concat(kids)) e.append(c);
return e;
}
function buildPlayer(host) {
const st = { data: null, top: "target", bottom: "guide", mode: "wipe", pos: 0.5, playing: false, muted: true, frame: 0,
labels: true, refs: true };
const vTop = el("video", { playsinline: "", preload: "auto", loop: "" });
const vBot = el("video", { playsinline: "", preload: "auto", loop: "" });
vTop.muted = vBot.muted = true;
const line = el("div", { class: "line" });
const tagTop = el("div", { class: "tag" }), tagBot = el("div", { class: "tag" });
const empty = el("div", { class: "empty" }, "Run the node to load the guide and the target.");
const refBox = el("div", { class: "refs", style: "display:none" });
const stage = el("div", { class: "stage", tabindex: "0" }, [vBot, vTop, line, tagTop, tagBot, empty]);
const sel = (opts, val, on) => {
const s = el("select", {}, opts.map(([v, l]) => { const o = el("option", { value: v }, l); if (v === val) o.selected = true; return o; }));
s.addEventListener("change", () => on(s.value));
return s;
};
const srcOpts = () => SOURCES.map(s => [s, s === "guide" ? (st.data?.labels?.[0] || "guide") : s === "target" ? (st.data?.labels?.[1] || "target") : "difference"]);
let selTop, selBot;
const selMode = sel(MODES, st.mode, v => { st.mode = v; layout(); });
const refreshSelects = () => {
const t = sel(srcOpts(), st.top, v => { st.top = v; load(); }), b = sel(srcOpts(), st.bottom, v => { st.bottom = v; load(); });
selTop ? selTop.replaceWith(t) : null; selBot ? selBot.replaceWith(b) : null;
selTop = t; selBot = b;
};
refreshSelects();
const btnSwap = el("button", { title: "Swap top and bottom", onclick: () => { [st.top, st.bottom] = [st.bottom, st.top]; refreshSelects(); load(); } }, "⇅ swap");
const btnPlay = el("button", { title: "Play / pause (space)", onclick: () => toggle() }, "▶");
const btnPrev = el("button", { title: "Previous frame (←)", onclick: () => step(-1) }, "◀|");
const btnNext = el("button", { title: "Next frame (→)", onclick: () => step(1) }, "|▶");
const btnMute = el("button", { title: "Sound (from the target)", onclick: () => { st.muted = !st.muted; sound(); } }, "🔇");
const btnFull = el("button", { title: "Full screen", onclick: () => (document.fullscreenElement ? document.exitFullscreen() : host.requestFullscreen?.())?.catch?.(() => {}) }, "⛶");
const btnSave = el("button", { title: "Play the video once from the start and download it as shown", onclick: () => saveFull() }, "⬇ save");
let format = "mp4";
try { format = localStorage.getItem(FORMAT_KEY) || format; } catch { }
if (!FORMATS.some(([v]) => v === format)) format = "mp4";
const selFormat = sel(FORMATS, format, v => { format = v; try { localStorage.setItem(FORMAT_KEY, v); } catch { } });
selFormat.title = "Format of what save / rec download. MP4: H.264 + AAC, plays and previews anywhere. MP4 small: "
+ "the same, sized to stay under 10 MB. GIF: no sound, under 10 MB. browser file: the recording as it is, no conversion.";
const btnRec = el("button", { title: "Record what you do in the player until stop, then download it", onclick: () => (rec.mr ? stopRec() : startRec()) }, "● rec");
const chkLabels = el("input", { type: "checkbox" });
chkLabels.checked = true;
chkLabels.addEventListener("change", () => { st.labels = chkLabels.checked; });
const chkRefs = el("input", { type: "checkbox" });
chkRefs.checked = true;
const showRefs = () => { refBox.style.display = st.refs && refBox.childElementCount ? "flex" : "none"; };
chkRefs.addEventListener("change", () => { st.refs = chkRefs.checked; showRefs(); });
const lblRefs = el("label", { title: "Show the reference pictures in a strip below the videos (also in what save / rec download)", style: "display:none" }, [chkRefs, "refs"]);
const scrub = el("input", { type: "range", min: "0", max: "0", value: "0", step: "1" });
scrub.addEventListener("input", () => seek(+scrub.value));
const info = el("span", { class: "info" }, "");
const strip = el("canvas", { title: "Difference per frame: click to jump" });
strip.addEventListener("pointerdown", e => {
const r = strip.getBoundingClientRect(), n = st.data?.frames || 0;
if (n) seek(Math.min(n - 1, Math.max(0, Math.floor((e.clientX - r.left) / r.width * n))));
});
host.append(el("div", { class: "bfsvc" }, [
el("div", { class: "row" }, [el("span", {}, "top"), selTop, btnSwap, el("span", {}, "bottom"), selBot, selMode, btnFull]),
stage,
refBox,
el("div", { class: "row" }, [btnPrev, btnPlay, btnNext, scrub, btnMute, info]),
el("div", { class: "row" }, [btnSave, btnRec, selFormat, el("label", { title: "Draw the names on the saved / recorded video" }, [chkLabels, "labels"]), lblRefs]),
strip,
]));
const fps = () => st.data?.fps || 24;
const frameOf = t => Math.min((st.data?.frames || 1) - 1, Math.max(0, Math.floor(t * fps() + 1e-3)));
const timeOf = f => (f + 0.5) / fps();
const videos = () => [vBot, vTop];
function sound() {
btnMute.textContent = st.muted ? "🔇" : "🔊";
for (const v of [vTop, vBot]) v.muted = true;
if (!st.muted) { const v = st.bottom === "target" ? vBot : st.top === "target" ? vTop : null; if (v) v.muted = false; }
}
function layout() {
const m = st.mode, p = st.pos;
stage.className = "stage" + (m === "wipe-v" ? " v" : m === "overlay" ? " o" : m === "side" ? " s" : "");
vTop.style.clipPath = m === "wipe" ? `inset(0 ${(1 - p) * 100}% 0 0)` : m === "wipe-v" ? `inset(0 0 ${(1 - p) * 100}% 0)` : "none";
vTop.style.opacity = m === "overlay" ? String(p) : "1";
line.style.display = m === "wipe" || m === "wipe-v" ? "block" : "none";
if (m === "wipe") line.style.cssText = `display:block;left:calc(${p * 100}% - 1px);top:0;bottom:0;width:2px`;
if (m === "wipe-v") line.style.cssText = `display:block;top:calc(${p * 100}% - 1px);left:0;right:0;height:2px`;
stage.prepend(vTop); stage.prepend(vBot); // under the divider and tags; side by side: bottom left, top right
vTop.classList.toggle("r", m === "side");
const name = s => s === "guide" ? (st.data?.labels?.[0] || "guide") : s === "target" ? (st.data?.labels?.[1] || "target") : "difference";
tagTop.textContent = name(st.top) + (m === "overlay" ? ` ${Math.round(p * 100)}%` : "");
tagBot.textContent = name(st.bottom);
tagTop.style.cssText = m === "side" ? "right:6px" : "left:6px";
tagBot.style.cssText = m === "side" ? "left:6px" : m === "wipe-v" ? "left:6px;top:auto;bottom:6px" : "right:6px";
if (m === "overlay") tagBot.style.cssText = "right:6px";
}
function drag(e) {
if (st.mode === "side" || !st.data) return;
const r = stage.getBoundingClientRect();
st.pos = Math.min(1, Math.max(0, st.mode === "wipe-v" ? (e.clientY - r.top) / r.height : (e.clientX - r.left) / r.width));
layout();
}
stage.addEventListener("pointerdown", e => { stage.setPointerCapture(e.pointerId); drag(e); });
stage.addEventListener("pointermove", e => { if (stage.hasPointerCapture(e.pointerId)) drag(e); });
stage.addEventListener("keydown", e => {
if (e.key === " ") { toggle(); e.preventDefault(); }
else if (e.key === "ArrowLeft") { step(-1); e.preventDefault(); }
else if (e.key === "ArrowRight") { step(1); e.preventDefault(); }
});
function drawStrip() {
const d = st.data, ctx = strip.getContext("2d");
const w = strip.width = Math.max(50, strip.clientWidth * devicePixelRatio), h = strip.height = 26 * devicePixelRatio;
ctx.clearRect(0, 0, w, h);
if (!d?.mae?.length) return;
// from the least to the most different frame, so the changes along the video stand out
const mx = Math.max(...d.mae), mn = Math.min(...d.mae), span = Math.max(mx - mn, 1e-6), n = d.mae.length;
ctx.fillStyle = "#e0573a";
d.mae.forEach((v, i) => { const bh = 2 + (v - mn) / span * (h - 4); ctx.fillRect(i / n * w, h - bh, Math.max(1, w / n), bh); });
ctx.fillStyle = "#3a86ff";
ctx.fillRect(st.frame / n * w, 0, Math.max(2, w / n), h);
}
function show(f) {
st.frame = f;
scrub.value = String(f);
const d = st.data;
const t = (f / fps());
info.textContent = d ? `frame ${f}/${d.frames - 1} ${t.toFixed(2)}s diff ${(d.mae?.[f] ?? 0).toFixed(1)}/255` : "";
drawStrip();
}
function seek(f) {
if (!st.data) return;
if (st.playing) toggle();
for (const v of videos()) v.currentTime = timeOf(f);
show(f);
}
const step = k => seek(Math.min(st.data ? st.data.frames - 1 : 0, Math.max(0, st.frame + k)));
function toggle() {
if (!st.data) return;
st.playing = !st.playing;
btnPlay.textContent = st.playing ? "⏸" : "▶";
if (st.playing) { vTop.currentTime = vBot.currentTime; for (const v of videos()) v.play().catch(() => {}); tick(); }
else for (const v of videos()) { v.pause(); v.playbackRate = 1; }
}
function tick() { // the bottom video is the clock; the top one follows it within half a frame
if (!st.playing) return;
// small drift: nudge the top one's speed (seeking it every frame kept it seeking, so it stuttered and drew blank)
const drift = vTop.currentTime - vBot.currentTime;
if (!vTop.seeking) {
if (Math.abs(drift) > 0.25) { vTop.currentTime = vBot.currentTime; vTop.playbackRate = 1; }
else vTop.playbackRate = Math.abs(drift) > 0.5 / fps() ? Math.min(1.05, Math.max(0.95, 1 - 2 * drift)) : 1;
}
show(frameOf(vBot.currentTime));
requestAnimationFrame(tick);
}
function load() {
const d = st.data;
if (!d) return;
const f = st.frame, was = st.playing;
if (was) toggle();
let pending = 2;
const ready = () => { if (--pending === 0) { for (const v of videos()) v.currentTime = timeOf(f); show(f); sound(); if (was) toggle(); } };
for (const [v, s] of [[vTop, st.top], [vBot, st.bottom]]) {
v.addEventListener("loadeddata", ready, { once: true });
v.src = url(d[s]);
v.load();
}
layout();
}
// --- save / record: the two videos drawn into a canvas the way the stage shows them, encoded by MediaRecorder
const rec = { mr: null, chunks: [], raf: 0, t0: 0, full: false, onEnd: null, type: "", busy: "" };
function pickType(audio) { // name an audio codec only with a sound track: Firefox stalls on one it never gets
const list = audio ? ["video/mp4;codecs=avc1.640028,mp4a.40.2", "video/mp4", "video/webm;codecs=vp9,opus", "video/webm;codecs=vp8,opus", "video/webm"]
: ["video/mp4;codecs=avc1.640028", "video/mp4;codecs=avc1", "video/mp4", "video/webm;codecs=vp9", "video/webm;codecs=vp8", "video/webm"];
return list.find(t => window.MediaRecorder?.isTypeSupported?.(t)) || "";
}
function fit(aw, ah, x, y, w, h) { // the largest aw:ah box centred in (x, y, w, h)
const k = Math.min(w / aw, h / ah), bw = aw * k, bh = ah * k;
return [x + (w - bw) / 2, y + (h - bh) / 2, bw, bh];
}
function label(ctx, text, x, y, size, right = false, bottom = false) {
ctx.font = `${size}px system-ui,sans-serif`;
const pad = size * 0.45, w = ctx.measureText(text).width + 2 * pad, h = size * 1.5;
const lx = right ? x - w : x, ly = bottom ? y - h : y;
ctx.fillStyle = "rgba(0,0,0,.65)"; ctx.fillRect(lx, ly, w, h);
ctx.fillStyle = "#fff"; ctx.textBaseline = "middle"; ctx.fillText(text, lx + pad, ly + h / 2);
}
const last = new Map(); // video -> its last whole frame, drawn while it seeks (a seeking video draws blank)
function pic(v) {
if (v.readyState < 2 || v.seeking) return last.get(v) || v;
let c = last.get(v);
if (!c) last.set(v, c = el("canvas"));
if (c.width !== v.videoWidth || c.height !== v.videoHeight) { c.width = v.videoWidth; c.height = v.videoHeight; }
c.getContext("2d").drawImage(v, 0, 0);
return c;
}
const refImages = () => (st.refs ? [...refBox.querySelectorAll("img")].filter(i => i.complete && i.naturalWidth) : []);
// save / rec: the pictures in a strip below the videos, as in the player (22% of the video's height)
const stripHeight = h => (refImages().length ? Math.round(h * 0.22) & ~1 : 0);
function drawRefs(ctx, top) {
const ims = refImages();
if (!ims.length) return;
const cw = ctx.canvas.width, sh = ctx.canvas.height - top, gap = Math.max(2, Math.round(sh * 0.05));
let h = sh - 2 * gap;
const widths = () => ims.map(im => Math.round(im.naturalWidth * h / im.naturalHeight));
let total = widths().reduce((a, b) => a + b, 0) + gap * (ims.length - 1);
if (total > cw - 2 * gap) { h = Math.floor(h * (cw - 2 * gap) / total); total = widths().reduce((a, b) => a + b, 0) + gap * (ims.length - 1); }
ctx.fillStyle = "#0e0e11"; ctx.fillRect(0, top, cw, sh);
let x = Math.round((cw - total) / 2);
const y = top + Math.round((sh - h) / 2), fs = Math.max(10, Math.round(h * 0.12));
ims.forEach((im, k) => {
const w = Math.round(im.naturalWidth * h / im.naturalHeight);
ctx.drawImage(im, x, y, w, h);
if (st.labels) label(ctx, `Picture ${k + 1}`, x + 3, y + 3, fs);
x += w + gap;
});
}
function compose(ctx) {
const sh = ctx.canvas.height - (ctx.canvas.videoHeight || ctx.canvas.height);
drawVideos(ctx, ctx.canvas.height - sh);
drawRefs(ctx, ctx.canvas.height - sh);
}
function drawVideos(ctx, area) {
const cw = ctx.canvas.width, ch = area ?? ctx.canvas.height, m = st.mode, p = st.pos;
const B = pic(vBot), T = pic(vTop);
const aw = vBot.videoWidth || st.data.width, ah = vBot.videoHeight || st.data.height;
const name = s => s === "guide" ? (st.data?.labels?.[0] || "guide") : s === "target" ? (st.data?.labels?.[1] || "target") : "difference";
ctx.fillStyle = "#000"; ctx.fillRect(0, 0, cw, ch);
if (m === "side") { // bottom left, top right, as on the stage
const gap = Math.round(cw * 0.004), half = (cw - gap) / 2;
const a = fit(aw, ah, 0, 0, half, ch), b = fit(aw, ah, half + gap, 0, half, ch);
ctx.drawImage(B, ...a); ctx.drawImage(T, ...b);
if (st.labels) {
const fs = Math.max(12, Math.round(a[3] * 0.032)), o = fs * 0.5;
label(ctx, name(st.bottom), a[0] + o, a[1] + o, fs); label(ctx, name(st.top), b[0] + o, b[1] + o, fs);
}
return;
}
const [x, y, w, h] = fit(aw, ah, 0, 0, cw, ch);
ctx.drawImage(B, x, y, w, h);
ctx.save();
if (m === "wipe" || m === "wipe-v") { ctx.beginPath(); m === "wipe" ? ctx.rect(x, y, w * p, h) : ctx.rect(x, y, w, h * p); ctx.clip(); }
if (m === "overlay") ctx.globalAlpha = p;
ctx.drawImage(T, x, y, w, h);
ctx.restore();
const lw = Math.max(2, Math.round(h / 360));
ctx.fillStyle = "#fff";
if (m === "wipe") ctx.fillRect(x + w * p - lw / 2, y, lw, h);
if (m === "wipe-v") ctx.fillRect(x, y + h * p - lw / 2, w, lw);
if (st.labels) {
const fs = Math.max(12, Math.round(h * 0.032)), o = fs * 0.5;
label(ctx, name(st.top) + (m === "overlay" ? ` ${Math.round(p * 100)}%` : ""), x + o, y + o, fs);
if (m === "wipe-v") label(ctx, name(st.bottom), x + o, y + h - o, fs, false, true);
else label(ctx, name(st.bottom), x + w - o, y + o, fs, true);
}
}
function stem() {
const d = st.data, clean = s => String(s || "").replace(/[^\w.-]+/g, "_").slice(0, 40);
const stamp = new Date().toISOString().replace(/[:T]/g, "-").slice(0, 19);
return `bfs_compare_${clean(d?.labels?.[0])}_vs_${clean(d?.labels?.[1])}_${st.mode}_${stamp}`;
}
function download(href, name) {
const a = el("a", { href, download: name });
document.body.append(a); a.click(); a.remove();
}
async function exportBlob(blob, ext, seconds) { // the browser's file -> ComfyUI converts it -> download
const name = stem();
if (format === "webm") {
const href = URL.createObjectURL(blob);
download(href, `${name}.${ext}`);
setTimeout(() => URL.revokeObjectURL(href), 60000);
return;
}
const id = [...crypto.getRandomValues(new Uint8Array(8))].map(b => b.toString(16).padStart(2, "0")).join("");
rec.busy = "uploading"; buttons();
try {
for (let off = 0; off < blob.size; off += PART) {
const r = await api.fetchApi(`/bfs/compare/part?id=${id}&offset=${off}`, { method: "POST", body: blob.slice(off, off + PART) });
if (!r.ok) throw new Error((await r.json().catch(() => ({}))).error || `upload failed (${r.status})`);
}
rec.busy = "converting"; buttons();
const r = await api.fetchApi("/bfs/compare/export", { method: "POST", headers: { "Content-Type": "application/json" },
body: JSON.stringify({ id, format, seconds, name }) });
const f = await r.json();
if (!r.ok || f.error) throw new Error(f.error || `conversion failed (${r.status})`);
download(url(f), f.filename);
} catch (e) {
alert(`BFS Video Compare: could not convert the recording (${e.message}). Choose "browser file" to download it as recorded.`);
} finally {
rec.busy = ""; buttons();
}
}
function buttons() {
if (rec.busy) { // uploading / converting the finished recording
btnRec.classList.remove("rec");
btnRec.disabled = btnSave.disabled = selFormat.disabled = true;
btnSave.textContent = rec.busy === "uploading" ? "⬆ uploading…" : "⚙ converting…";
btnRec.textContent = "● rec";
return;
}
btnRec.disabled = selFormat.disabled = false;
const busy = !!rec.mr;
btnRec.classList.toggle("rec", busy);
btnRec.textContent = busy ? (rec.full ? "■ stop" : `■ stop ${((performance.now() - rec.t0) / 1000).toFixed(0)}s`) : "● rec";
btnSave.disabled = busy;
btnSave.textContent = busy && rec.full ? `⬇ ${Math.round(100 * (vBot.currentTime / (vBot.duration || 1)))}%` : "⬇ save";
}
function startRec(full = false) {
if (!st.data || rec.mr || rec.busy) return false;
if (!window.MediaRecorder || !HTMLCanvasElement.prototype.captureStream) { alert("This browser can't record a canvas (MediaRecorder)."); return false; }
const aw = vBot.videoWidth || st.data.width, ah = vBot.videoHeight || st.data.height;
const side = st.mode === "side", canvas = el("canvas");
// the size of the layout it starts in (side by side is twice as wide); a later mode change is fitted into it
canvas.width = (side ? 2 * aw + Math.round(2 * aw * 0.004) : aw) & ~1;
canvas.videoHeight = ah & ~1; // the videos; the reference strip goes below them
canvas.height = canvas.videoHeight + stripHeight(canvas.videoHeight);
const ctx = canvas.getContext("2d");
compose(ctx);
const stream = canvas.captureStream(Math.max(30, Math.round(fps())));
// the soundtrack is on the target. Not Firefox's mozCaptureStream: it takes the element's picture away from the canvas
const vt = st.top === "target" ? vTop : st.bottom === "target" ? vBot : null;
try { for (const t of vt?.captureStream?.()?.getAudioTracks?.() || []) stream.addTrack(t); } catch { }
const type = pickType(stream.getAudioTracks().length > 0);
if (!type) { for (const t of stream.getTracks()) t.stop(); alert("This browser can't record MP4 or WebM."); return false; }
rec.type = type; rec.chunks = []; rec.full = full; rec.t0 = performance.now();
rec.mr = new MediaRecorder(stream, { mimeType: type, videoBitsPerSecond: 16e6 });
rec.mr.ondataavailable = e => { if (e.data?.size) rec.chunks.push(e.data); };
rec.mr.onstop = () => {
for (const t of stream.getTracks()) t.stop();
const chunks = rec.chunks, seconds = (performance.now() - rec.t0) / 1000;
rec.mr = null; rec.chunks = []; buttons();
if (chunks.length) exportBlob(new Blob(chunks, { type: rec.type.split(";")[0] }), rec.type.includes("mp4") ? "mp4" : "webm", seconds);
};
rec.mr.start(1000);
const frame = () => { if (!rec.mr) return; compose(ctx); buttons(); rec.raf = requestAnimationFrame(frame); };
rec.raf = requestAnimationFrame(frame);
buttons();
return true;
}
function stopRec() {
if (!rec.mr) return;
cancelAnimationFrame(rec.raf);
if (rec.onEnd) { vBot.removeEventListener("ended", rec.onEnd); rec.onEnd = null; }
if (rec.full) { for (const v of videos()) v.loop = true; if (st.playing) toggle(); }
if (rec.mr.state !== "inactive") rec.mr.stop();
}
async function saveFull() { // once from the first frame to the last, then stop and download
if (!st.data || rec.mr || rec.busy) return;
if (st.playing) toggle();
for (const v of videos()) v.loop = false;
const seeked = v => new Promise(res => { const t = setTimeout(res, 2000); v.addEventListener("seeked", () => { clearTimeout(t); res(); }, { once: true }); });
const wait = Promise.all(videos().map(seeked));
seek(0);
await wait;
if (!startRec(true)) { for (const v of videos()) v.loop = true; return; }
rec.onEnd = () => { rec.onEnd = null; stopRec(); };
vBot.addEventListener("ended", rec.onEnd, { once: true });
toggle();
}
return {
setData(d) {
stopRec();
last.clear();
refBox.replaceChildren(...(d.refs || []).map((f, k) => el("figure", { title: `Picture ${k + 1}` },
[el("img", { src: url(f), crossorigin: "anonymous" }), el("figcaption", {}, `Picture ${k + 1}`)])));
lblRefs.style.display = d.refs?.length ? "flex" : "none";
showRefs();
st.data = d;
empty.style.display = "none";
scrub.max = String(Math.max(0, d.frames - 1));
refreshSelects();
st.frame = Math.min(st.frame, d.frames - 1);
load();
},
layout, drawStrip,
};
}
app.registerExtension({
name: "BFSNodes.VideoCompare",
async beforeRegisterNodeDef(nodeType, nodeData) {
if (nodeData.name !== "BFSVideoCompare") return;
const created = nodeType.prototype.onNodeCreated;
nodeType.prototype.onNodeCreated = function () {
const r = created?.apply(this, arguments);
styles();
const host = el("div", { style: "width:100%;height:100%;min-height:320px" });
this._bfsCompare = buildPlayer(host);
wheelToCanvas(host); // the wheel zooms the canvas over the player too
this.addDOMWidget("bfs_video_compare", "div", host, { serialize: false, hideOnZoom: false, getMinHeight: () => 320 });
this.setSize([Math.max(this.size[0], 560), Math.max(this.size[1], 620)]);
new ResizeObserver(() => this._bfsCompare?.drawStrip()).observe(host);
return r;
};
const executed = nodeType.prototype.onExecuted;
nodeType.prototype.onExecuted = function (message) {
executed?.apply(this, arguments);
const d = message?.bfs_compare?.[0];
if (d) this._bfsCompare?.setData(d);
};
},
});
+33
View File
@@ -0,0 +1,33 @@
/**
* The mouse wheel over a node's HTML panel zooms / pans the canvas like anywhere else on the graph, unless the pointer
* is on something that scrolls that way and still can (a list, a text box, a tall panel): that one scrolls, and once
* it reaches its end the wheel goes back to the canvas. Ctrl / Cmd + wheel always goes to the canvas.
*/
import { app } from "../../scripts/app.js";
function scrolls(start, stop, dx, dy) {
for (let n = start; n && n.nodeType === 1; n = n.parentElement) {
if (n.tagName === "SELECT" && (n.size > 1 || n.multiple)) return true;
const cs = getComputedStyle(n);
if (dy && /(auto|scroll)/.test(cs.overflowY) && n.scrollHeight > n.clientHeight + 1
&& (dy < 0 ? n.scrollTop > 0 : n.scrollTop + n.clientHeight < n.scrollHeight - 1)) return true;
if (dx && /(auto|scroll)/.test(cs.overflowX) && n.scrollWidth > n.clientWidth + 1
&& (dx < 0 ? n.scrollLeft > 0 : n.scrollLeft + n.clientWidth < n.scrollWidth - 1)) return true;
if (n === stop) break;
}
return false;
}
export function wheelToCanvas(el) {
el.addEventListener("wheel", e => {
const canvas = app.canvas?.canvas;
if (!canvas || e.defaultPrevented || (!(e.ctrlKey || e.metaKey) && scrolls(e.target, el, e.deltaX, e.deltaY))) return;
e.preventDefault();
e.stopPropagation();
canvas.dispatchEvent(new WheelEvent("wheel", {
deltaX: e.deltaX, deltaY: e.deltaY, deltaZ: e.deltaZ, deltaMode: e.deltaMode, clientX: e.clientX, clientY: e.clientY,
screenX: e.screenX, screenY: e.screenY, ctrlKey: e.ctrlKey, shiftKey: e.shiftKey, altKey: e.altKey, metaKey: e.metaKey,
bubbles: true, cancelable: true,
}));
}, { passive: false });
}
+79 -18
View File
@@ -33,7 +33,7 @@ node's advanced settings, with defaults that work.
- **The source clip** (TSC's duet): the new character copies its motion, camera and cuts frame for
frame. Give the new character with `<Picture 1>` (and the place with `<Picture 2>`). The pinned clip
has no tag: the prompt calls it "the kept footage" (`layout_text` gives the sentence). See TSC's
has no tag: the prompt names its region by where it is and what is seen there (`{layout}` / `{panel_region}`). See TSC's
prompting guide: never describe the source performer, restate the identity in every shot.
- **A reference image**: a static picture of the person next to the video.
@@ -62,40 +62,101 @@ References work like the official Reference to Video node: up to 9 images (`<Pic
## Prompt
The node's prompt box starts with a template. The side panel has **no tag**: it is not a `<Picture n>`
or `<Video n>`, and the text encoder never sees it. Name it by its place ("the LEFT half is the kept
footage"); `{layout}` inserts that sentence for the current side and size. Describe only the generated
part: never the panel's performer, clothes or room, even to contrast them, because whatever you leave
undescribed is copied from the panel. Restate the new identity in every shot ("her face from
`<Picture 1>`" plus two or three face, hair or outfit words) and give exact times for cuts.
TSC's newer strategy (2026-10): **H3 has no idea what is pinned and what is noised.** It reads one description of
one video, so the prompt describes the finished split screen as plain footage. Never write "kept footage", "kept
exactly", "generated", "preserved", "in sync with", "seamlessly": they describe nothing on screen and crowd out the
action.
Example (character swap, source clip pinned on the left, a face picture and a full-body picture):
- **No tag and no retention line for the pinned clip.** It is not a reference input, only latents in the canvas. Name
its region by where it is plus what is seen there, in one clause: "the top quarter shows a man dancing in a small
bedroom". `{panel_region}` / `{video_region}` become the region names for the current side and size ("the top
quarter", "the bottom three quarters"), `{layout}` the whole split-screen sentence.
- **Write the other region as its own scene**, doing the same movements at the same moments, in a real place with
concrete details.
- **Spell out the motion** beat by beat, with times for the big moments (cuts, strikes, jumps, a line). The pinned
half alone does not carry the moves. Times are on the video's clock (the pinned clip starts at frame 0 here).
- **Soundscape** = the sounds of the new scene.
- Cuts: `[Shot 2] At MM:SS.mmm, both halves cut together to ...` with the new framing.
The VLM writer and the template filler follow these rules, and a scrub removes the old meta words from what they
return. Template (fill the `{...}`, or let the VLM fill them):
```
subject_definitions:
<Subject 1> is the woman whose appearance comes from <Picture 1> and <Picture 2>: fair skin, a narrow oval face, grey-green eyes, light brown hair in a low ponytail, wearing a white long-sleeved top under a navy denim apron.
<Subject 1> is {name or plain description} whose appearance comes from <Picture 1>: {concrete look words}, wearing {outfit piece by piece}.
summary:
[reference generation] The target video is a split screen: the kept footage beside <Subject 1>, who moves in sync with it, in the same bedroom.
[reference generation] The target video is a split screen: {panel_region} shows {what is seen in the source, one clause}, and {video_region} shows <Subject 1> doing the same movements at the same moments, in {place}.
retention_analysis:
<Subject 1> (appears in [Shot 1], [Shot 2]): fully_preserved - her face, ponytail, white top and navy apron are retained.
The kept footage: fully_preserved - the panel is kept exactly.
<Subject 1> (appears in [Shot 1]): fully_preserved - {key look words} are retained.
detailed_description:
The target video is in a realistic style, as handheld vertical smartphone footage under soft daylight. {layout}
The target video is in a {realistic / 3D CG / anime} style, as {the medium of the source clip}. A split screen divided by a thin straight {vertical / horizontal} line: {panel_region} shows {what is seen there, one clause}; {video_region} shows <Subject 1> performing the same movements at the same moments, in {place with concrete details}.
[Shot 1] A medium close-up at chest height, the phone steady at eye level. <Subject 1>, her face from <Picture 1> with grey-green eyes and soft pink lips, her light brown ponytail and navy apron, talks to the camera with small nods.
[Shot 2] At 00:03.708, both halves cut together to a closer framing. <Subject 1>, her face from <Picture 1>, the white sleeves and apron straps visible, raises one open hand beside her mouth and smiles.
[Shot 1] {Camera distance, angle and movement as the source shows it}. <Subject 1>, the face from <Picture 1>, {what they do, one sentence per real action}. At {00:01.500} {big beat}.
overall_soundscape:
Her voice speaking to the camera in a quiet bedroom.
{The sounds of the new scene and when they happen.}
non_diegetic_music:
None.
```
Example (character swap, source clip pinned on the left half, one face picture and one full-body picture):
```
subject_definitions:
<Subject 1> is a young woman whose appearance comes from <Picture 1> and <Picture 2>: fair skin, a narrow oval face, grey-green eyes, light brown hair in a low ponytail, wearing a white long-sleeved top under a navy denim apron.
summary:
[reference generation] The target video is a split screen: the left half shows a man talking to a phone camera in a bedroom, and the right half shows <Subject 1> doing the same movements at the same moments, in a sunny kitchen with white tiles.
retention_analysis:
<Subject 1> (appears in [Shot 1], [Shot 2]): fully_preserved - her face, ponytail, white top and navy apron are retained.
detailed_description:
The target video is in a realistic style, as handheld vertical smartphone footage under soft daylight. A split screen divided by a thin straight vertical line: the left half shows a man talking to a phone camera in a bedroom; the right half shows <Subject 1> performing the same movements at the same moments, in a sunny kitchen with white tiles and a window behind her.
[Shot 1] A medium close-up at chest height, the phone steady at eye level. <Subject 1>, her face from <Picture 1> with grey-green eyes and soft pink lips, talks to the camera with small nods. At 00:01.800 she laughs and tilts her head back.
[Shot 2] At 00:03.708, both halves cut together to a closer framing. <Subject 1>, her face from <Picture 1>, the white sleeves and apron straps visible, raises one open hand beside her mouth and smiles.
overall_soundscape:
Her voice speaking to the camera, a kettle starting to hiss in the kitchen at 00:02.500.
non_diegetic_music:
None.
```
## Same-world swaps: static and setting picture (1.84.0)
TSC's two fixes for a swap in the same room with the same clothes, without a VLM:
- **`panel_static`** (`off` / `face` / `person`): SAM 3 tracks the face (or the whole person) of the pinned clip and
covers it in black-and-white TV static on every frame. The generated half can no longer copy the source face, so the
identity comes from the reference picture; motion, framing and camera still come from the panel. The static only
exists in the panel, which is cropped off, so it never shows in the output. `face` keeps the body (best when the
outfit should stay); `person` hides the whole performer (best when the build and outfit should change).
- **`setting`** + **BFS Setting Picture**: one frame of the clip with the person in static. Plugged into `setting`, it
becomes the last `<Picture n>` and the prompt gets *"<Picture n> shows the setting, the same place as the original performance
in full detail; the noise patch in it is where <Subject 1> stands."* (or write `{setting}` to place its tag yourself).
This stops the background and framing from being taken from the character picture.
## Prompt from a VLM (optional, 1.85.0)
Plug a Qwen3-VL CLIP (the same CLIPLoader as `clip` works) into `vlm`:
- **prompt empty**: the VLM writes it from the task + instruction (TSC's six sections; with a `setting` picture the
place becomes `<Subject 2> ... from <Picture n>`).
- **prompt with `{...}` fields** (TSC's template): the VLM fills only the fields, from the clip's frames, the
reference pictures and the setting picture; every other word stays. `{layout}`, `{setting}`, `{target}` and
`{details}` are filled by the node, not the VLM.
- **prompt without fields**: used as written (the VLM is not called).
Without `vlm`, a prompt that still has `{...}` fields logs a warning. The prompt actually used is the `prompt` output
and is printed under the comparison.
## Settings
| setting | effect |
@@ -107,7 +168,7 @@ None.
| hold | pin the panel for the whole clip, or only its first latent frame |
| guide | optional aligned latent guide in the video area (what the body-swap LoRAs use) |
| rope_mode | *canvas*: panel and video share one wide grid (TSC). *shifted*: the video keeps the RoPE positions of a render without the panel, and the panel sits past its edge |
| (prompt in shifted mode) | With *shifted*, `{layout}` becomes "the whole frame shows the generated video, moving in sync with the kept footage": do not describe a split screen there, or the model splits the video itself. *canvas* keeps the split-screen wording and follows the framing more closely (recommended). |
| (prompt in shifted mode) | With *shifted*, `{layout}` becomes empty: do not describe a split screen there, or the model splits the video itself. *canvas* keeps the split-screen wording and follows the framing more closely (recommended). |
| rope_gap | *shifted* only: empty RoPE steps (2x2 patches) between video and panel; the panel moves away in position without any pixels in between. Keep it small against the video width: at 320 px wide (10 patches) a gap of 8 made the model draw its own split screen; 0 works everywhere |
## Notes from tests
+409 -47
View File
@@ -17,23 +17,28 @@ every shot inside that range.
| node | what it does |
|---|---|
| **BFS Shot Planner** | Pick a video, split it on a timeline, set a reference and prompt per shot. Outputs the shots as a ComfyUI **list**. |
| **BFS Shot Unpack** | Opens one shot into plain values: guide frames, reference, second reference, prompt, length, first frame, audio, size, index. Use it with any model. |
| **BFS Shot Unpack** | Opens one shot into plain values: guide frames, reference pictures (a list), prompt, length, first frame, audio, size, index. Use it with any model. |
| **BFS Shot Repack** | Puts edited pieces back into a shot: Unpack's `shot` output goes to Repack's `shot`, the edited piece (e.g. the reference with its background removed) to its input. Unconnected inputs keep their values; timing, cuts and audio are unchanged. |
| **BFS Shot H3 Conditioning** | Ready-made MiniMax H3 conditioning for one shot, built with the native nodes (Reference to Video + Add Guide). Optional *duet*: the shot's clip pinned in a side panel (canvas or shifted RoPE; route the model through the node for the shift); the join cuts the panel off. Optional prompt writer: pick a *task* (character swap, style, setting, appearance, lighting / weather, custom) and an *instruction*; with a VLM on its *vlm* input it writes each shot's duet prompt from the shot and its references, otherwise a template for the task (*planner prompt*, the default, keeps the planner's prompt). The written text comes out on *prompt*. Optional *setting_ref* (TSC's tip): one more reference, the shot's middle frame with the person covered in TV static, so the model sees the place in full detail; it is the last `<Picture n>` (write `{setting}` for its tag, otherwise a sentence is added to subject_definitions). |
| **BFS Shot Join** | Concatenates the decoded shots in order, trims each to its true length, cross-fades soft joins and returns the soundtrack. With *comparison* on it also returns a side-by-side video (original shot \| references \| result) with the shot's info and your *label* on top and its prompt below, ready for Create Video. |
| **BFS Shot H3 Conditioning** | The common MiniMax H3 path with the important controls only: aligned Add Guide and/or native video reference, references, optional audio conditioning, the Planner's inpaint mode and source-subject blur. |
| **BFS Shot H3 Conditioning (Advanced)** | The full conditioning node. Adds the setting reference, the first-frame anchor and the experimental mask-guide controls; its `model` input passes through to `model` (older workflows routed the model through it). The duet lives in **BFS Shot H3 Duet** (1.113 took it out of this node). |
| **BFS Shot Join** | Concatenates the decoded shots in order, trims each to its true length, cross-fades soft joins and returns the soundtrack. With *comparison* on it also returns a side-by-side video (original shot \| references \| result) with the shot's info and your *label* on top and its prompt below, ready for Create Video. *save to = mp4 file* writes the video (and the comparison) straight to MP4 files instead, one shot in memory at a time (long videos); `video_path` gives the file. |
| **BFS Shot H3 Duet** | Renders one shot with H3: the shot's clip pinned beside the video (duet, no LoRA), the shot as an aligned guide (for body-swap LoRAs), or both. |
## The panel
Use the language selector in the top bar to switch the interface between English, Portuguese, Spanish and Simplified
Chinese. The choice is saved in this browser and does not alter the workflow plan or prompt text. **Tour** opens an
optional step-by-step introduction when clicked; it never appears automatically.
The planner's panel has five tabs, in the order you work. The timeline stays on top of every tab.
| tab | what is there |
|---|---|
| 🎞 **Video** | pick or upload the video, audio, how it is split into shots (cuts or fixed length), generation size |
| 🎬 **Shots** | one card per shot (scroll sideways, **← →** move between shots), the preview player and the selected shot's editor: reference & prompt, who is replaced (mask, generation mode, `{target}`), continuity, **Copy to other shots** |
| 👥 **People & masks** | the cast found by face (a reference per person) and the global mask settings |
| 👥 **People / Filters / Masks** | the cast found by face, shot filters and the global mask settings |
| 📝 **Prompts & refs** | the global reference and prompt, the VLM and the reference descriptions (`{details}`) |
| ▶ **Run** | **Checks** (what to fix before running, with a button to each shot), auto / queue loop, filters, test limits |
| ▶ **Run** | **Checks** (what to fix before running, with a button to each shot) and how the shots run (shot by shot / queue loop / auto loop) |
Shot cards show badges for what each shot has: 🎯 mask, 🎭 / ✂ generation mode, 🧑 `{target}`, ⛓ continuity and ⚠ problems.
The header's ⚠ count opens the checks.
@@ -50,13 +55,99 @@ decode) runs once per shot, and shot *i* always gets guide *i*, reference *i* an
Single values such as the model and VAE are shared by every run. The join takes the whole list
back (it is an `INPUT_IS_LIST` node) and puts the video together.
Two run modes:
Three run modes:
- **Auto loop (one run)**: every shot in one queue run. Simple; holds all decoded shots in memory.
- **One run, shot by shot (default)**: one Queue renders every shot **in order inside the same run**. Each shot is
stored on disk; then BFS Shot Join expands the part of the graph between the planner and itself (ComfyUI node
expansion) for the next shot, so that shot is conditioned *after* the previous one exists and continuity
(*head* / *history*) works. No browser re-queue (the tab can be closed), Save Video fires once with the full video,
and after a failure a new Queue resumes from the stored shots (*Reset loop* starts over). Previews connected inside
the shot's part of the graph show every shot. Needs exactly one BFS Shot Planner before the join. Same idea as the
recursive loop of Ethanfel's MiniMax H3 Context Loop.
- **Queue loop (one shot per run)**: each run generates the next pending shot and stores it on
disk. The join blocks every node after it until the last shot, then outputs the full video,
so a Save Video node only fires once. With *Auto-queue the next shot* on, the panel re-queues
the workflow after each shot. *Reset loop* starts over. Changing the plan starts a new run.
so a Save Video node only fires once. With *Auto-queue the next shot* on (default), the panel re-queues
the workflow after each shot, so one click still renders the whole video (keep the ComfyUI tab open: the panel
does the re-queue). *Reset loop* starts over. Changing the plan starts a new run. It resumes from the stored
shots after a failure, and continuity *head* / *history* needs it: each shot reads the result of the one before.
- **Auto loop (all shots at once)**: every shot in one queue run, each node runs for every shot before the next node.
Holds all decoded shots in memory. Continuity *head* / *history* does not work here (all shots are conditioned
before any is sampled).
### Memory
- **Status at the top (🧠)** and a **Memory** card in the Run tab. BFS Shot Join records every shot it receives on
this GPU: its size (latent tokens = generated frames, with the continuity head, × resolution) and the peak VRAM of
its run. Before running, every planned shot is compared with the **biggest shot that already ran on this GPU**:
*✓* within it, *a bit bigger* (it will probably fit, maybe slower), or *⚠ may run out of memory* (also listed in
Checks). There is no fixed "GB per frame" for H3 (ComfyUI offloads weights as needed), so nothing is guessed: until
a first shot has run, the status says *no memory data yet*. The record lives in ComfyUI's `user/bfs_shotloop_vram.json`.
- **Between shots** (Run tab, shot by shot and queue loop): *keep models loaded* (fastest) or *free memory*, which
unloads every model and empties the GPU after each shot so the next one starts clean (low VRAM, or a later shot
that runs out of memory); models load again for every shot, so it is slower.
- What makes a shot lighter: lower **Megapixels**, a shorter **Max seconds / shot**, **head 5** instead of 22 (the
head adds its frames to every shot), **Crop** (the generation is the subject's box), and ComfyUI's own options
(`--lowvram`, fp8 / GGUF weights, tiled VAE decode).
- **The final join** (after the last shot) is where long videos used to run out of **RAM** (not VRAM: the process is
killed, e.g. `Terminated` under WSL). It now reads one stored shot at a time into the final video. Measured on 11
shots at 960×544 (~50 s): **+15.8 GB before, +8.2 GB now** with *save to = images* (that is the video itself, which an
IMAGE output must hold), and **+2.1 GB with *save to = mp4 file***, which streams the video to ffmpeg and stays at
about one shot whatever the length (also with the comparison, which is streamed 16 frames at a time: +2.1 GB
measured at *comparison_height* 1024, against ~32 GB as images). For long videos use *mp4 file* (BFS Shot Join), or
the panel's **Combine**.
- **RAM guard:** before an *images* join, BFS Shot Join asks ComfyUI's cache to drop earlier outputs and estimates
what the video (and the comparison) need; when that does not fit in the free RAM it writes MP4 files instead and
says so in the console, instead of the whole ComfyUI being killed.
- **The outputs are never empty:** with *mp4 file* (chosen or by the RAM guard) the `images` and `comparison` outputs
still carry the video, read back from the written files at a size that fits the free RAM (the console says the
scale; the files keep the full size), the join shows the files on its node, and `video_path` gives the path.
- **Default: save_to = images** (the result comes out inside ComfyUI, for the next nodes). The panel's Checks warn when
the video is long for this machine (as images it needs about 12 bytes per pixel per frame, ~3.5x with the
comparison, against the free RAM) and advise *mp4 file* for that video.
- **comparison_height** is the height of each column: the comparison is three columns wide plus text bars, so it
costs about 3× the video at the same height. 360–512 is plenty to review a long video.
### Stored shots (cache, redo, combine)
In the **shot by shot** and **queue loop** runs every generated shot is stored, and the Run tab's **Stored shots**
card manages them.
- **Shot cache:** *Keep* (default) stores them in ComfyUI's output folder, `output/bfs_shotloop/<run>`: they survive a
restart, so a stopped run resumes and a single shot can be redone. *Temp* uses ComfyUI's temp folder, which ComfyUI
**empties at every start** (before 1.104 every run was stored there).
- **Shot storage** (the format of each stored shot, measured on 145 frames at 1280×720):
| Storage | Size | Quality |
|---|---|---|
| **Lossless** (FFV1, default) | ~34 MB | exact: the final video is unchanged |
| Smaller (H.264 RGB, crf 8) | ~21 MB | visually lossless (PSNR ~52 dB) |
| Smallest (H.264 RGB, crf 14) | ~9 MB | light loss (PSNR ~48 dB) |
| Raw frames (`.pt`) | ~401 MB | exact; needs no ffmpeg |
The final video of a stored run is assembled from the stored shots, and the continuity head reads them: a lossy
storage reaches the output (the panel warns). Without ffmpeg a shot falls back to raw frames. With *comparison* on,
the shot's source frames for the comparison's original column are kept as `shot_NNNN_orig.mkv` (visually lossless:
they are only shown); the `.pt` itself holds only the shot's info (a few KB).
- **One folder per source video.** Each shot remembers the settings it was made with (its range, prompt, references,
mask, crop, continuity, the generation size, the global prompt / references and the references or prompt connected
to the planner). The next Queue **runs only the shots that are not stored or whose settings changed**, plus the
shots that **continue from a shot that was made again** (head, history, previous shot as reference / first
frame), so every seam matches. Shot cards show 💾 (stored, up to date) or ⏳ (runs next time).
- **Model, LoRA, seed, sampler (1.112.0):** the planner reads them from the queued workflow (every node that feeds
BFS Shot Join's `images`: loaders, LoRAs, sampler, seed, conditioning settings), so changing any of them runs every
shot again. The join's own settings (crossfade, comparison) don't. A seed set to *randomize* changes on every Queue,
so every Queue then runs every shot; fix the seed to keep the stored shots.
- **↻ Redo** a shot (Stored shots card): it runs again on the next Queue, and the shots that continue from it run
after it. Tick *↻ redoes only that shot* to keep the next shots as they are (the seam after it may then not match).
- **🎬 Combine stored shots:** puts the stored shots together into an MP4 in the output folder **now, without
generating** (one shot in memory at a time), on the plan's timeline: the soundtrack, the original video where a
shot is skipped, and also where a shot is not stored yet or out of date, so it previews a run in progress. A link
to the file appears in the card.
- When every shot is stored, a Queue still runs the last shot again so the join can output the video: use Combine to
only assemble.
- **Recovering a run outside ComfyUI** (e.g. ComfyUI was killed during the final join):
`python tools/bfs_assemble_shots.py output/bfs_shotloop/<run> video.mp4 --audio input/<source video>` (ComfyUI's
Python, ffmpeg on the PATH) does the same as Combine with one shot in memory; it reads the old raw `.pt` shots too.
## Guide frames and lengths
@@ -74,12 +165,21 @@ camera cuts.
minimum merge into a neighbour when the merge still fits; shots longer than the maximum split
into equal parts.
- **Fixed length**: equal parts no longer than the maximum.
- **One shot (whole range)**: the whole selected range as a single shot, whatever its
length; *Max seconds / shot* is not applied. For "N seconds in one piece" set *Range length, seconds* = N, right
next to the mode (it moves the end frame).
For pieces of at most X seconds use *Fixed length* with *Max seconds / shot* = X.
- **By hand**: drag the white handles on the timeline to move a boundary, double-click the
shot bar to split, *⇥ Merge* to join, or select a shot and press **Delete** to remove the cut at its
start (it merges into the previous shot). Your edits are what runs.
**Range** (🎞 Video tab, with its own player): *Start frame* / *End frame* (timeline frames; end 0 = the end of the
video) limit the plan to a part of the video, before anything is split, analysed or generated. Play or scrub the player
(or click the timeline) and press *⇤ start here* / *⇥ end here* to take its exact frame. The
timeline dims what is outside; the join and the soundtrack cover only the range.
*Max seconds / shot* is converted to frames and snapped down to the grid (4.5 s at 24 fps =
107 frames for H3). *Max shots* and *Max total seconds* cap a long source.
107 frames for H3). *Max shots* in the Video tab caps how many shots are used.
## Audio
@@ -97,7 +197,7 @@ that will not run. The panel shows the shot's start, end and current time (and f
playhead follows on the timeline, and clicking the timeline seeks. The source video streams from
the input folder, so only what you play is loaded.
## Filters
## Shot filters (People / Filters / Masks tab)
Skip shots that should not run, decided per shot from a few sampled frames:
@@ -122,37 +222,46 @@ matching audio) when *Skipped shots in the output* is set to remove them.
## Cast (people by face)
*Find people* samples the video every 0.5 s, detects faces and groups them into people by face
identity (InsightFace `buffalo_l` in `models/insightface`, on the CPU with a few threads so it stays
light and never touches the GPU; the result is cached per video). Each
person card shows the face, screen share and first/last appearance (click the face to seek).
Give a person a reference (and a second one) by clicking the slot, or pick one from the recent
references; *ignore* leaves that person out. Every shot card shows who is in it, with the main
person (most screen time) highlighted and linked people outlined in green.
identity, on the CPU (it never touches the GPU; the result is cached per video). **No extra package is needed**: by
default it uses OpenCV Zoo's **YuNet** (detection) + **SFace** (recognition), Apache-2.0 models run by OpenCV itself and
downloaded once into `models/opencv_face` (0.2 + 37 MB). If **InsightFace** is installed (`buffalo_l` in
`models/insightface`) it is used instead (a bit more accurate; mind its license). Last resort, without either: SAM 3.1
faces grouped by colours (hair, skin, clothes). Each
person card shows the face, screen share and first/last appearance (click the face to seek). The *ignore* checkbox excludes a detected identity from cast-based filters. Every shot card shows the people detected in it, with the main person highlighted.
| option | effect |
|---|---|
| Shots use their main person's reference | a shot without its own reference takes its main person's (shown with a dashed green outline) |
| Split where the main person changes | adds a boundary where the largest face switches to another person for at least a second, then re-splits |
| Only run shots with a linked person | shots where no linked, non-ignored person appears are skipped ("no linked person") |
| Only run shots with a detected person | shots with no detected, non-ignored person are skipped |
A reference set on the shot itself always wins; the global reference is the last fallback.
Face identities are used for filtering and optional shot splitting. References and marker colors are selected globally or per shot; they are not assigned to detected identities.
## References and prompts
Every shot can have its own reference, second reference and prompt; shots without them use the
global ones. Click an empty reference slot to upload an image straight from your computer; a filled
slot has **✕** to remove it; **Copy to other shots** gives a shot's references to every shot. The last ten
references you picked are shown under the shot editor for one-click reuse (click = reference,
shift+click = second reference). Connected `ref_image`, `ref_image_2` and `prompt` inputs override
the panel's global values, so an edited first frame or a prompt from another node can drive the
defaults.
Up to **12 reference pictures** (1.113.0): the global list starts with one slot and **+** adds more; they are
`<Picture 1>` to `<Picture 12>`, in order. Click a slot to upload a picture (the **+** takes several files at once),
or drop image files on a slot (replaces it) or anywhere on the box (adds them). A filled slot has **✕**: a global
picture is taken out (the next ones move up), a shot's own picture falls back to the global one.
The planner also outputs `ref_image` and `ref_image_2`: the references the shots actually use,
without repeats, so a plan where every shot shares one reference returns a single image.
Every shot can have its own pictures and prompt. A shot's empty slot uses the global picture at the same position
(shown faded), so a shot can change only `<Picture 2>` and keep the rest. **Copy to other shots** gives a shot's
pictures to every shot. The last ten pictures you picked are shown under the shot editor: click = picture 1,
shift+click = add it as the next picture, or drag it onto any slot.
Under every prompt (global and per shot) there is a chip per picture: **PictureN** inserts `<Picture N>` at the
cursor, **📝** inserts `{details@Picture N}` (that picture's description, below).
The node no longer has `ref_image` / `ref_image_2` inputs (the pictures live in the panel); a workflow saved with
them connected still runs, and the log says the images are ignored. A connected `prompt` input still overrides the
global prompt. The planner outputs `refs` (every picture the shots use, without repeats, as a list) and `ref_count`
where `ref_image` / `ref_image_2` were. **BFS Shot Unpack** outputs `refs`, every picture of the shot as a list (the next node runs once per picture,
e.g. a background remover), and `ref_count`; **BFS Shot Repack**'s `refs` input takes that list back and gives each
shot its own pictures (a workflow from before 1.113 that wired `ref_image` still works). **BFS Shot H3 Conditioning** passes every picture of the shot (the `use_ref_2` switch left in 1.114: the pictures are chosen in the planner); H3's own
node stops at 9 and LoRAs trained like OpenJutsu saw up to 9, so the log notes a shot with more.
## Mask & crop (SAM 3)
Generate only part of the frame. In a shot's editor (*Who is replaced*), type what to segment in English
Generate only part of the frame. In a shot's editor (*Select region*), type what to segment in English
(`person in white`, `woman with red hair`, `red car`; up to 32 tokens, commas for several things) or click
**🎯 Points** to pick positive / negative points on any frame of the shot, then **👁 Preview** (red = mask,
yellow = crop box). Then pick how the shot is generated:
@@ -160,17 +269,71 @@ yellow = crop box). Then pick how the shot is generated:
| mode | what is generated | crop / uncrop |
|---|---|---|
| **Full frame** | the whole frame; the mask only feeds `{target}` and the setting picture | no |
| **🧩 Frame + paste** | the whole frame (so the model follows the guide's pose freely and the new person may be bigger); BFS Shot Join then pastes only the person onto the original frame: the shot's mask plus the new subject's outline (SAM 3 on the result with the shot's *new subject* text, default `person`; `paste_new_outline`) | no crop; paste in the Join |
| **🎭 Mask only** | only the mask, on the whole frame; everything else stays exactly as it was | no |
| **✂ Crop** | a box around the mask (more pixels for a small person); BFS Shot Join pastes it back | yes |
| **✂🎭 Crop + mask** | only the mask, inside the crop: the most detail with the background kept | yes |
The two mask modes need **BFS Shot H3 Conditioning → inpaint = per shot (planner)** (the default; *only the mask*
*Frame + paste* is the safest for a different body (e.g. a woman in profile replaced by a man facing the camera):
*Mask only* binds the new person to the old outline, *Frame + paste* does not. The two mask modes need **BFS Shot H3 Conditioning → inpaint = per shot (planner)** (the default; *only the mask*
forces it on every shot, *off* never). The crop is one box around the mask (the union over all its frames, so it
does not shake), at the source resolution, sent to the model at the generation size; **BFS Shot Join** pastes the
result back, feathered by the mask (or the whole box). **BFS Shot Unpack** also outputs the shot's mask (`mask`),
e.g. for an inpainting model. In **BFS Shot Join**'s comparison video, cropped shots show their mask (red) and crop
box (yellow) over the original column (`comparison_mask`, on by default).
**Mask opacity** (mask modes, per shot, slider under the modes): the value of the white inside the generation mask.
1 regenerates the masked area completely. A grey mask (e.g. 0.85) keeps part of the original there: H3 puts those
rows at *opacity* × the noise level, so they start from the original partly visible. 0.8-0.9 keeps pose, outline and lighting while still swapping; too low copies the original
person. With `guide_mode = aligned guide` the model also sees the whole original shot as its guide, so it follows
the motion either way.
**🎨 Colored Mark** is an optional translucent overlay on the source video: each marked person is tinted with the
color of the picture they become, and the prompt gets the mapping ("the person marked in red becomes the character
from `<Picture 3>`"), for any model or LoRA. Every picture has its **Mark color** (under it in **Prompts & refs**:
green, purple, red, blue, yellow, magenta, cyan, orange, white, brown, pink, olive by default); change it when it is
too close to a color in the video. In a shot, turn on Colored Mark and pick which picture the shot's selection (its
SAM 3 text or points) becomes; **+ Mark another person** adds a person with their own picture and text or 🎯 points
(the points window has one point set per person). This is only a visual marker: it does not create an inpainting mask
or change **Full frame**, **Mask only** or crop behavior. More than one marked person needs Crop off, and the
shot must have every picture the marks use. The overlay keeps movement and facial
expression visible. Plans before 1.113 (a second region for ref 2) open with that region as `<Picture 2>`.
```text
body_swap: subject_definitions: <Subject 1> is the character shown in <Picture 1>, whose identity, face, head, hair or fur, skin, body shape, proportions and clothing define the green-marked replacement. <Subject 2> is the character shown in <Picture 2>, whose identity, face, head, hair or fur, skin, body shape, proportions and clothing define the purple-marked replacement. In the guide video, the translucent green and purple tints are temporary editing markers, not character appearance. Replace the green-tinted person only with <Subject 1> and the purple-tinted person only with <Subject 2>. The guide video provides the scene and the complete performance of both people, including their body poses, gestures, head motion, gaze, speech and facial expressions.
summary: [video generation + references] Replace the green-tinted guide person with <Subject 1> from <Picture 1> and the purple-tinted guide person with <Subject 2> from <Picture 2>, preserving the scene, camera and complete motion and expression of both people.
detailed_description: [Shot 1] Replace the person marked by the translucent green tint with <Subject 1> from <Picture 1>, and the person marked by the translucent purple tint with <Subject 2> from <Picture 2>. Each character follows that person's exact position, timing, body pose, gestures, head motion, gaze, speech and facial expressions in the guide. Remove both temporary color markers completely. Keep the two reference identities distinct and stable in every frame. Preserve their interaction, the background, lighting, camera and all unselected content from the guide video. Prevent identity drift, clothing changes, texture crawling and flicker. The result is a real, photographic video.
```
The VLM's shot analysis suggests a SAM 3 text selection from the original frames; it does not assign guide colors or
decide that two detections are the same identity. **Describe target** only describes the first shot selection. For
two-color swaps, choose both selections and identities yourself, and use the explicit mapping above (or let
🤖 Auto prompt write it: it is told which tinted person becomes which picture).
**Mask guide** (experimental, *Mask only* shots, BFS Shot H3 Conditioning → `mask_guide`): the shot with only the
masked region visible, the rest grey, so the model looks at the subject's pose and outline on its own while the mask
limits where it generates. *+ extra guide* adds it as a second aligned guide next to the normal one (the model sees
both); *instead of the full guide* replaces the normal one; *+ reference video* gives it as a native reference video
(`<Video n>`, after the shot's own one when guide_mode uses it; write `{mask_video}` in the prompt where its tag goes, otherwise a sentence is added). `mask_guide_look` sets what it shows of the subject: *grey blurred* (default: volume, light and head direction, no colours or face, so the model follows the motion without copying the old subject), *colour* (as it is; can make the model copy the old subject), *silhouette* (flat shape), *edges* (outlines) or *pose (people)* (the skeleton of the people in the mask, OpenPose colours; needs ultralytics and a YOLO pose model in `models/ultralytics`, e.g. `pose/yolov8m-pose.pt`). `mask_ref_size` makes that reference video smaller (default 1/2, ~1/4 of the tokens): it only has to show the pose and outline. Compare with
*off*: the LoRAs were trained with one full guide, and it can also pull the old subject's look.
### Masks from a video (rotoscoping) instead of SAM 3
A mask made elsewhere (After Effects Roto Brush, DaVinci Resolve Magic Mask, another ComfyUI workflow…) can replace
SAM 3. It is a black and white video, **white = the subject**, and it works with every mode (Mask only, Crop,
Crop + mask), `{target}`, the setting picture and the comparison overlay. Three places, the most specific wins:
| where | covers | wins over |
|---|---|---|
| shot editor → **🎞 mask video** | only that shot (its first frame = the shot's first frame) | everything |
| the shot's own SAM 3 text / points | that shot | the two below |
| the planner's optional **`mask`** input (MASK or IMAGE batch, e.g. Load Video + Convert Image to Mask) | the whole video, one mask per source frame | the panel's mask video |
| People / Filters / Masks → **🎞 Mask video (whole video)** | the whole video, matched by time | SAM 3 for shots without a mask |
Any size and frame rate: it is matched to the shot by time and resized. The console says which mask each shot used.
**Stitch finishing** (BFS Shot Join, cropped shots only; adapted from Neko (Nekodificador)'s *NKD Inpaint Stitch*, MIT, built on AbleJones's workflow and nodes):
| Option | What it does |
@@ -192,8 +355,8 @@ Credit to **Neko (Nekodificador)** and **AbleJones**, whose workflow and nodes t
ComfyUI builds without native H3 generation masks need a per-row mask patch on the model
(e.g. ComfyUI-MiniMaxH3-PerRowMasking).
The **People & masks** tab holds the global mask settings, with defaults that work as they are: fill holes on,
temporal expand 2 frames (less flicker), expand 16 px, feather 12 px, padding 15 %, paste by mask, blockify off
The **People / Filters / Masks** tab holds the global mask settings, with defaults that work as they are: fill holes on,
temporal expand 2 frames (less flicker), expand 16 px, feather 12 px, padding 15 %, paste the whole box with feathered edges, blockify off
(16 aligns the mask to H3's latent grid), threshold 0.5, 4 objects. *Show masks on shots* overlays the preview
on the shot cards. SAM 3 uses the official `sam3.1_multiplex_fp16.safetensors` in `models/checkpoints`,
downloaded from Comfy-Org/sam3.1 the first time.
@@ -209,12 +372,77 @@ editor shows its suggestion with buttons to apply it. The summary output lists t
## Describe references ({details})
The VLM can describe the references, for any task (not tied to swaps). In the **Prompts & refs** tab pick an instruction
preset: *full body* (face, hair, skin, age, build, clothing piece by piece), *head / face*, *face attributes* (a
The VLM can describe the references, for any task (not tied to swaps), and any subject: a person, a creature, a robot,
an animal or a cartoon character (it says what the subject is first). In the **Prompts & refs** tab pick an instruction
preset: *object* (a noun phrase for a prop, product, vehicle...: what it is, shape, materials, colors, text), *short* (the default: one noun phrase of at most ~20 words that fits inside a sentence, e.g. `green goblin with
long pointed ears in a torn brown leather vest`, for `<Subject 1> is the {details} shown in <Picture 1>`), *full body* (face, hair, skin, age, build, clothing piece by piece), *head / face*, *face attributes* (a
short comma-separated list), *outfit*, or *custom* (your own instruction). **📝 Describe refs** writes one
description per reference set: the global references, every shot's and every cast person's. They are saved in
the plan and editable. Write **`{details}`** anywhere in a prompt and every shot gets the description of its own
references; a set without a description is described by the VLM at run time.
description per picture (see below), saved in the plan and editable. Write **`{details}`** anywhere in a prompt and
every shot gets the description of its own first picture; a picture without a description is described by the VLM
at run time (no click needed).
**Image max side (VLM)** controls the longest side of reference and shot images prepared for VLM analysis (1024 px by
default). Aspect ratio is preserved; the model's own image processor may resize them further. Increase it when fine
reference details matter, or lower it to reduce analysis cost. This does not change the generated video size.
When a shot has two references, the planner expands a one-reference template at run time to include `<Subject 2>` and
`{details2}` for `<Picture 2>` (not when the prompt already names `<Picture 2>` or uses `{details@Picture n}`).
Existing one-reference templates remain unchanged in shots with only one reference. You can also use `{details2}`
explicitly in any template.
### 🤖 Auto prompt (1.113.0)
In **Prompts & refs → 🤖 Auto prompt**, *Write the prompts with the VLM* turns the prompt boxes (global and per shot)
into **requests** ("replace the bottle with the red can from Picture 2", "Picture 1 dances in the rain"), and the VLM
writes each shot's MiniMax H3 prompt in the official full-reference format (MiniMaxAI/MiniMax-H3
`docs/VIDEO_PROMPT_WRITING_GUIDE_ref_en.md`): `subject_definitions`, `summary` with its task type, `retention_analysis`,
`detailed_description` (`[Shot 1]`, the official camera vocabulary, sounds in time), `overall_soundscape` and
`non_diegetic_music`. It looks at:
- **key frames of the shot** with their times (*Key frames*: auto = about 1.5 per second, 3 to 8);
- **the shot's pictures** and their descriptions (Describe references);
- **the request**, the shot's **Colored Marks** (which tinted person becomes which picture; the mapping is added if the
writer leaves a color out), its selection / `{target}` and its mode (*Mask only*: only the subject is regenerated;
*Crop*: the frames are a crop);
- **how the source is given**: *guide video* (aligned guide, no tag) or `<Video 1>` (native reference video) — match the
conditioning node's `guide_mode`.
**Template:** with Auto prompt on, the planner's `prompt` input (e.g. the template a LoRA was trained with,
`body_swap: subject_definitions: ...`) is the **template** and the panel's text is the **request** applied to it
("Troque a mulher no vídeo pela mulher da <Picture 1>"). The template is kept as it is: same sentences, same order, its
trigger word first; the writer applies the request, adds the pictures' visible features and, after `[Shot 1]`, this
shot's camera, place, light and actions. The panel's 🤖 Write uses the template of the last run (shown in the card).
A request that is itself a whole prompt (H3 sections) is used as the template the same way. **Style** picks how a prompt without template is written: *H3
official* (the six sections) or *OpenJutsu captions*, the short style of the OpenJutsu training captions ("Replace the
dog in the video with the gaunt goblin … from <Picture 1>. Replace the cat … from <Picture 2>. Keep the same background
and location as in the video, and maintain the motion, camera angle, and timing."). Without a template, a
short request gets the full six-section prompt. Without a VLM, the template (else the request) is the prompt.
**Cuts:** the writer gets the shot's real camera cuts (the planner's cut detector) and writes `[Shot N] At MM:SS.mmm`
exactly there, with at least one key frame in each piece between cuts; a shot with no cut gets no timestamp at all.
Labels always keep their `< >`.
**🤖 Write all prompts** (or *only the missing*) writes them now; each shot shows its written prompt in the Shots tab,
editable, with **🤖 Write / Write again**. A shot without one is written when the workflow runs (with the VLM on the
planner, before any SAM 3 work) and the panel keeps it, so the next run uses the same text. When the request,
pictures, marks, selection or mode change after a prompt was written, the shot says it is *older*; it is still used
until you write it again. A written prompt is part of the stored shot's signature, so editing it re-renders the shot.
Without a VLM and without a written prompt, the request is used as the prompt.
### One description per picture (1.113.0)
Descriptions are per picture: **📝 Describe refs** describes every picture the plan uses by itself, once per image.
`{details}` is picture 1's description, `{details2}` picture 2's and **`{details@Picture n}`** (or
`{details@Picture3}`) picture n's, so each shot gets the descriptions of *its* pictures. The dropdown next to the button
sets what is described for every picture; a picture can have its own (e.g. *full body* for a character sheet, *head /
face* for a close-up, *outfit*, or *custom* with its own instruction), and **🔄** describes only that picture again. A
picture without a description is described at run time by the planner's VLM, else by the conditioning node's VLM, else
it becomes "character". Descriptions saved by older plans (per reference set) are read as those pictures'.
**Fit the prompt** (on by default, next to 📝 Describe refs): the VLM gets the sentence of the prompt that holds the
marker, with a blank in its place (e.g. `<Subject 1> is the ___ shown in <Picture 1>`), and writes the description for
it, so the description adapts to the prompt and not the other way round (no "the A goblin", no paragraph inside a
sentence). The panel buttons use the global prompt (or the shot's own); at run time the prompt the shot really uses.
The VLM buttons in the panel work after the workflow has run once with the VLM connected (ComfyUI only hands
models to nodes when they run). Tested with Qwen3-VL 8B; some smaller or modified models return empty answers
@@ -227,18 +455,93 @@ Every setting in the panel has a hover tooltip (ⓘ). The main ones:
| setting | what it does |
|---|---|
| Mode | Camera cuts (shots start at cuts, long ones split evenly) or fixed length |
| Mode | Camera cuts (shots start at cuts, long ones split evenly), fixed length, or one shot (the whole range) |
| Detector / Sensitivity | PySceneDetect adaptive or content, or the built-in one; higher sensitivity finds more cuts |
| Frame grid | frame counts the model accepts (H3 17n+5, LTX/Wan 8n+1, Wan 4n+1) |
| Max / Min seconds per shot | longest shot sent to the model; shorter shots merge into a neighbour |
| Timeline fps | the frame rate the planner works on (generation and audio use it) |
| Max shots / Max total seconds | test on the first shots or seconds only |
| Megapixels / Size multiple | generation size at the source's aspect ratio, snapped to the multiple |
| Run | auto loop (all shots in one run) or queue loop (one shot per run, stored on disk) |
| Max shots | test on the first shots only |
| Generation size | Megapixels (default), source video resolution, or custom width and height; dimensions snap to the size multiple |
| Run | one run, shot by shot (default) / queue loop (one shot per run) / auto loop (all shots at once) |
| Skipped shots in the output | keep the original video there or remove those shots |
| Mask: padding / expand / feather / temporal expand / blockify | crop context, paste-back growth and softness, flicker hold, square blocks |
| Mask: threshold / max objects / paste back | SAM 3 text matching, objects tracked, paste only the mask or the whole box |
| VLM: frames per shot / max tokens | how much the VLM sees and how long it may answer |
| VLM: frames per shot / max tokens / image max side | how much the VLM sees, its answer limit, and the longest side of images sent for analysis |
## One seed per shot: BFS Shot Seed (1.94.0)
A seed widget on the sampler gives every shot the SAME seed (the node runs once per shot with the same value).
**BFS Shot Seed** (shot in, seed out) makes it a choice: *same for every shot*, *increment per shot* (seed, seed+1 ...)
or *random per shot* (unrelated but repeatable seeds from the base seed). `overrides` pins single shots, e.g.
`3:12345, 5:777`, to keep the shots that came out well and re-roll only the others.
## Structured starting noise: BFS Shot Noise (1.94.0, experimental)
Optional. **BFS Shot Noise** gives the NOISE for SamplerCustomAdvanced; with every amount at 0 it is exactly RandomNoise.
| Input | What it does |
|---|---|
| `shot_seed` (+ `shot`, `overrides`) | the seed of each shot: same / increment / random per shot, as BFS Shot Seed |
| `shared_base` | the coarse pattern of the noise comes from `base_seed`: the same for every shot, at any resolution and any length (it is a tape along the timeline); the fine detail stays per shot. Put the seed of a result that worked in `base_seed` to keep its coarse noise |
| `distribution` | gaussian (default), laplacian, student-t or uniform, all at unit variance |
| `spectrum_tilt` | 1/f^alpha spectrum: > 0 puts more energy in the large blobs ("fractal" noise), < 0 in the fine grain |
| `pyramid` | coarser layers of noise added on top |
| `static_share` | a part of the noise identical in every frame |
| `guide_structure` (`vae`, `guide_frames`, `mask`) | outside the mask, the guide's latent is mixed into the noise |
| `ref_tint` (`vae`, `ref_image`, `mask`) | inside the mask, the noise leans toward the reference's average latent |
All mixes keep unit variance. The model was trained on plain noise: use small amounts and compare with 0.
## {details} filled automatically (1.92.0)
A `{details}` in a shot's prompt is filled without pressing 📝 Describe refs: the planner's VLM describes the shot's
references before any SAM 3 work; with no VLM on the planner, the slot is left for **BFS Shot H3 Conditioning** or
**BFS Shot H3 Duet**, whose `vlm` input (the same Qwen3-VL CLIPLoader) describes them (the *short* preset, fitted to
the sentence). Edited descriptions always win. With no VLM anywhere it becomes "character" and the log says so.
## Source blur (1.91.0, finer in 1.110.0)
`source_blur` on **BFS Shot H3 Conditioning** (and its Simple version), **BFS Shot H3 Duet**, **BFS H3 Duet** and
**BFS H3 Duet Conditioning** blurs the source person in what the model *sees* of the source (aligned guide, native
reference video, duet panel): the model still reads the pose and the motion but has no face to copy, so the identity
comes from the reference. The kept area of *Mask only* and the output are never blurred. Trick shared by JalenBrunson
(Sapiens2 there) for reliable ref2v swaps.
Modes read `<region> (<what stays sharp>)`:
| region | what is blurred |
|---|---|
| `face` | the face (and neck) |
| `head + hands` | also the hair and the hands |
| `person` | the whole body: only for big changes, the pose then comes from the blurred shape |
| `masked area` | the shot's own mask (the person being replaced, with its expand); no extra segmentation. Without a mask on the shot (or in the plain Duet nodes) it blurs the whole person |
| sharp | |
|---|---|
| `keep mouth` | lips, teeth and tongue stay sharp (the lips carry a lot of the likeness and the lip sync) |
| `keep lips + tongue` | the teeth are blurred too (always uses Sapiens2 for the lips) |
| `mouth too` | nothing stays sharp |
`source_blur_segmenter`: **SAM 3.1** (text prompts, already used by the planner; the 1.91 behaviour) or **Sapiens2**
(Meta's body-part segmentation, 28 classes with lips, teeth and tongue apart from the face; one pass for every part,
finer around the mouth). Sapiens2 marks every person in the frame, like the SAM prompts here. Its weights
(`facebook/sapiens2-seg-0.4b`, 1.6 GB) download to `models/sapiens2/` the first time; it runs on the GPU in half
precision and goes back to RAM after each shot. Sapiens2 License.
`source_blur_effect`: **blur** (no detail left) or **invert colours** (keeps the shapes and the expression but not the
skin tone or the colours; some users get better likeness with it).
## Big videos (upload)
ComfyUI refuses any single request bigger than `--max-upload-size` (100 MB by default), which is why big videos fail
to upload in most nodes. The planner's ⬆ Upload (source video, mask videos, pictures) sends files of 32 MB or more in
8 MB pieces that are written straight to disk and assembled in the input folder, so any size works without changing
how ComfyUI is started (a dropped piece is sent again). For other nodes, start ComfyUI with a higher limit, e.g.
`--max-upload-size 4096`, or copy the file into `ComfyUI/input` and pick it from the list.
A newly selected or uploaded source starts as **one shot covering at most its first 15 seconds**. This is only the
initial plan: widen the range or choose *Camera cuts* / *Fixed length* and press *Auto split* when you want multiple
shots. Existing saved plans keep their own range and split mode.
## Continuity between shots
@@ -248,11 +551,63 @@ references (good across camera cuts, keeps the person consistent), and *previous
at frame 0 of the shot (for continuous action without a cut). Pick which frame: the previous result's first,
middle or last. **Copy to other shots** with *continuity* ticked applies it to every shot after the first.
It needs the previous result to exist: it works in the **queue loop** (the planner reads the stored result) and in
It needs the previous result to exist: it works **shot by shot** and in the **queue loop** (the planner reads the stored result) and in
the **auto loop with BFS Shot H3 Duet** (it renders shot by shot and keeps the last result). BFS Shot Unpack also
outputs it as `previous_result`. This is different from H3 Conditioning's *first_frame*, which anchors the shot's
own frame from the source video.
### History: a long take, window by window (1.87.0)
*Continuity* = **previous shot's end as history** carries the **last 5, 22, 39 or 56 frames of the previous shot's
result** into the next shot as clean frames placed **just before frame 0 on H3's timeline**: the model sees them as
the immediate past of the video it is generating, so motion, camera, light and the new look continue across the join
(instead of one still frame). Idea from Akatz's [H3 Relay](https://github.com/akatz-ai/h3-relay) windowed edit and
Ethanfel's MiniMax H3 Context Loop; this is BFS's own implementation (a process-local patch of H3's packed layout
that moves the video, its audio and the other guides past the history; ComfyUI files are not edited).
- Plans saved before keep what they had; *head* (below) is now the default for new shots.
- **Same take only.** A shot that starts at a camera cut skips it (nothing to continue). For one long take, split it
with *Mode = fixed length* (or let *Camera cuts* split long shots) and set history on every shot after the first
with **Copy to other shots → continuity**. Shots still exist: they are the windows, and cuts reset the history.
- **Every mode:** full frame, *Frame + paste*, *Mask only* (the mask regenerates the target; the history is a
condition outside it), *Crop* (the history is cut with this shot's box) and the **duet canvas** (the source frames
under the history go into the panel strip). Not with duet *shifted RoPE*. The aligned guide still drives this shot's
own frames, so sync stays frame-exact.
- **BFS Shot H3 Conditioning and BFS Shot H3 Duet** both use it (duet canvas or aligned guide).
- **Shot by shot / queue loop:** shots run in order and each reads the stored result of the one before. Auto loop: taken
from the previous render when it matches.
- Video only for now (the previous audio is not carried). History can use 5, 22, 39 or 56 frames; longer windows use more memory.
- **Only 17k+5 frames work.** H3's VAE encodes 17-frame clips into 5 latent steps and drops the last 3 steps of the
whole video, so 17 frames come back as just their *first 5*: the old 17 / 34 / 51 options left a 12-frame (0.5 s)
hole between the history and the shot, seen as a jump or drift at every join. Plans saved with 17 / 34 / 51 are
read as 22 / 39 / 56.
### Head: the shot starts on the previous result (default)
*Continuity* = **previous shot's end as this shot's start** generates the shot from **5, 22 or 39 frames earlier**:
those first frames are the previous shot's result, encoded **into this shot's latent** and kept by the sampler
(noise mask 0), so they are **not generated or swapped again**, and the new frames continue straight from them.
BFS Shot Join cuts them off, so the timing is unchanged. Same masked-prefix continuation as the Context Loop's
*masked AV* mode, whose tested default is 22.
- **On by default, 22 frames**, for every shot after the first. Set *Continuity = off* (or *history*) to change it.
- **Same take only:** skipped at a camera cut, after a skipped shot, and on grids other than H3's.
- **Shot by shot or queue loop.** In the auto loop the previous result does not exist yet when the shot is conditioned,
so the planner generates the shot without a head (and says so in the console).
- Works with full frame, *Frame + paste*, *Mask only* / *Crop + mask* (the head is kept whatever the mask says), *Crop*
(the head is cut with this shot's box) and the duet canvas. Not with duet *shifted RoPE*. Pinned in
**BFS Shot H3 Conditioning**; other workflows still generate and cut the extra frames.
- Costs those frames of generation per shot (a 60-frame shot is generated at 90 frames instead of 73).
- **Keep colors** (BFS Shot Join, on by default). A shot that starts on the previous result comes out a little more
saturated right after it, and the next shot inherits that: measured on a 5-shot take, the new subject's saturation
went 75 → 88 → 105 → 124, with almost no change *inside* each shot (the jump is at every join). With *keep_colors*
the first shot of each take records its subject's colours, and every shot that continues from the one before has
its subject's chroma (CIELAB a, b: one transform per shot, so no flicker) matched to them before it is stored; the
next shot then starts from the anchored colours and nothing accumulates (same take: 69–73 on every shot). The
subject is where the result differs from the source video (soft edge); the background and the lightness are not
touched. A camera cut or a shot without continuity starts a new take (new anchor). *keep_colors_strength* blends it
(1 = fully). It applies to stored runs (shot by shot, queue loop), where continuity exists.
## MiniMax H3 with an aligned guide (example)
```
@@ -262,7 +617,7 @@ BFS Shot Join ◄── images ────────────────
└─► Create Video (fps, audio) ─► Save Video
```
For other models, use **BFS Shot Unpack** and wire `guide_frames`, `ref_image`, `prompt` and
For other models, use **BFS Shot Unpack** and wire `guide_frames`, `refs`, `prompt` and
`length` into that model's own guide and reference nodes; the list mapping works the same way.
## Choosing who is replaced (`{target}`), step by step
@@ -316,6 +671,12 @@ Then save:
**Save → all shots** is for a person who stays in the same place across shots (a sequential video, a fixed camera).
**🎯 Track → whole video** (in the points window) is for a person who moves around: the selection is tracked by SAM 3
through the whole video (or the range), forward and backward from the frame you clicked on, and every shot without
its own selection takes its part of that track. Also type a short text (e.g. `woman in pink top`): after a camera cut
the tracker uses it to find the subject again. The People / Filters / Masks tab shows the track and removes it. The first run
tracks the whole range (a minute or two per few hundred frames); the result is cached until the selection changes.
### 3. Describe the person
Click **🧑 Describe target** in the shot's editor:
@@ -335,7 +696,9 @@ Click **🧑 Describe target** in the shot's editor:
### What happens when the workflow runs
Every shot's prompt gets its own `{target}` text. A shot without a description gets "the person". The selection is
Every shot's prompt gets its own `{target}` text: the one you wrote or made with 🧑 Describe target; when the field is
empty but the shot has a selection (mask text, points or a mask video) and the VLM is connected, the subject is cut out
and described automatically (written for the prompt's `{target}` sentence). A shot with neither gets "the person". The selection is
only used to make the description: it does not crop or mask anything unless the shot's mode is *Mask only*, *Crop*
or *Crop + mask*.
@@ -358,4 +721,3 @@ or *Crop + mask*.
| Only the face or the hair is described | the click was on the head: click the body |
| The other person is described | add an exclude point (right-click) on the other person, **👁 Segment**, save, describe again |
| The swap still hits the wrong person | make the description more specific (position: "on the left", clothing colours) |
+486 -13
View File
@@ -1,23 +1,392 @@
// Shot Planner: constants and small helpers shared by the panel and its views (no side effects: ComfyUI loads every
// .js under web/js as an extension, so this module only exports).
import { h } from "../vendor/vue.esm-browser.prod.mjs";
import { h as vueH, reactive, ref } from "../vendor/vue.esm-browser.prod.mjs";
// UI language is a browser preference, not part of the workflow or generation prompts.
const LANG_KEY = "bfs.shotloop.language";
const savedLanguage = (() => { try { return localStorage.getItem(LANG_KEY) || "en"; } catch { return "en"; } })();
export const uiLanguage = ref(["en", "pt", "es", "zh"].includes(savedLanguage) ? savedLanguage : "en");
export function setUiLanguage(value) {
if (!["en", "pt", "es", "zh"].includes(value)) return;
uiLanguage.value = value;
try { localStorage.setItem(LANG_KEY, value); } catch { /* private mode */ }
}
const TEXT = {
"🎞 Video": ["🎞 Vídeo", "🎞 Vídeo", "🎞 视频"], "🎬 Shots": ["🎬 Shots", "🎬 Planos", "🎬 镜头"],
"👥 People & masks": ["👥 Pessoas e máscaras", "👥 Personas y máscaras", "👥 人物和蒙版"],
"📝 Prompts & refs": ["📝 Prompts e referências", "📝 Prompts y referencias", "📝 提示词和参考图"],
"▶ Run": ["▶ Executar", "▶ Ejecutar", "▶ 运行"], "📖 Guide": ["📖 Guia", "📖 Guía", "📖 指南"],
"🧭 Tour": ["🧭 Tour", "🧭 Recorrido", "🧭 引导"], "Language": ["Idioma", "Idioma", "语言"],
"Source video": ["Vídeo de origem", "Vídeo de origen", "源视频"], "Range": ["Trecho", "Rango", "范围"],
"Timeline": ["Linha do tempo", "Línea de tiempo", "时间轴"], "zoom": ["zoom", "zoom", "缩放"],
"Click a shot to select it (← → move between shots) · drag the white handles to move a boundary · double-click the shot bar to split · Delete removes the selected shot's cut": [
"Clique em um plano para selecioná-lo (← → navega entre planos) · arraste as alças brancas para mover uma divisão · dê dois cliques na barra para dividir · Delete remove o corte do plano selecionado",
"Haz clic en un plano para seleccionarlo (← → cambia de plano) · arrastra los controles blancos para mover un límite · haz doble clic en la barra para dividir · Delete elimina el corte del plano seleccionado",
"点击镜头以选中(← → 切换镜头)· 拖动白色控点移动分界 · 双击镜头条进行拆分 · 按 Delete 删除所选镜头的切点",
],
"camera cuts or fixed length, on the model's frame grid": ["cortes de câmera ou duração fixa, na grade de frames do modelo", "cortes de cámara o duración fija, según la cuadrícula de fotogramas del modelo", "按镜头切换或固定时长,并符合模型帧步长"],
"Range length, seconds (0 = to the end)": ["Duração do trecho, em segundos (0 = até o fim)", "Duración del tramo, en segundos (0 = hasta el final)", "片段时长(秒,0 = 到结尾)"],
"refresh the input folder": ["atualizar a pasta de entrada", "actualizar la carpeta de entrada", "刷新输入文件夹"],
"Pick or upload a video to start": ["Escolha ou envie um vídeo para começar", "Elige o sube un vídeo para empezar", "选择或上传视频以开始"],
"use the video’s audio": ["usar o áudio do vídeo", "usar el audio del vídeo", "使用视频音频"],
"The outputs carry a silent track of the right length, so Create Video works": ["As saídas incluem uma faixa silenciosa com a duração correta para funcionar em Create Video", "Las salidas incluyen una pista silenciosa con la duración correcta para que funcione Create Video", "输出会带有时长正确的静音音轨,以便 Create Video 正常工作"],
"Audio of the planner's and the join's outputs": ["Áudio das saídas do Planner e do Join", "Audio de las salidas del Planner y del Join", "Planner 和 Join 输出的音频"],
"the range ends after the player's frame": ["o trecho termina depois do frame atual do player", "el tramo termina después del fotograma actual del reproductor", "片段在播放器当前帧之后结束"],
"Play or scrub the video (or click the timeline), then ⇤ / ⇥. {…} frames ({…} s) of {…}: only this part is split, analysed, generated and joined.": ["Reproduza ou avance o vídeo (ou clique na linha do tempo) e use ⇤ / ⇥. {…} frames ({…} s) de {…}: somente este trecho será dividido, analisado, gerado e unido.", "Reproduce o desplaza el vídeo (o haz clic en la línea de tiempo) y usa ⇤ / ⇥. {…} fotogramas ({…} s) de {…}: solo se dividirá, analizará, generará y unirá este tramo.", "播放或拖动视频(也可点击时间轴),然后按 ⇤ / ⇥。共 {…}/{…} 帧({…} 秒):仅拆分、分析、生成并合并此片段。"],
"15 s in one shot": ["15 s em um único plano", "15 s en un solo plano", "单镜头 15 秒"],
"from the start frame · {…} frames": ["a partir do frame inicial · {…} frames", "desde el fotograma inicial · {…} fotogramas", "从起始帧开始 · {…} 帧"],
"limit how many detected shots are used": ["limita quantos planos detectados serão usados", "limita cuántos planos detectados se usan", "限制使用的检测镜头数量"],
"Auto split replaces the boundaries; per-shot settings stay with the shot at the same position.": ["A divisão automática substitui os limites; as configurações por plano permanecem na mesma posição.", "La división automática reemplaza los límites; los ajustes por plano se mantienen en la misma posición.", "自动拆分会替换分界;逐镜头设置保留在相同位置。"],
"before the start: not planned": ["antes do início: fora do plano", "antes del inicio: no incluido", "起点之前:不处理"],
"after the end: not planned": ["depois do fim: fora do plano", "después del final: no incluido", "终点之后:不处理"],
"Split into shots": ["Dividir em planos", "Dividir en planos", "拆分镜头"], "Generation size": ["Tamanho da geração", "Tamaño de generación", "生成尺寸"],
"Camera cuts": ["Cortes de câmera", "Cortes de cámara", "镜头切换"], "Fixed length": ["Duração fixa", "Duración fija", "固定时长"],
"One shot (whole range)": ["Um plano (trecho inteiro)", "Un plano (rango completo)", "单个镜头(整个范围)"],
"Mode": ["Modo", "Modo", "模式"], "Detector": ["Detector", "Detector", "检测器"], "Sensitivity": ["Sensibilidade", "Sensibilidad", "灵敏度"],
"Frame grid": ["Grade de frames", "Múltiplo de fotogramas", "帧步长"], "Max seconds / shot": ["Máx. segundos por plano", "Máx. segundos por plano", "每个镜头最长秒数"],
"Min seconds / shot": ["Mín. segundos por plano", "Mín. segundos por plano", "每个镜头最短秒数"], "Timeline fps": ["FPS da linha do tempo", "FPS de la línea de tiempo", "时间轴帧率"],
"Megapixels": ["Megapixels", "Megapíxeles", "百万像素"], "Size multiple": ["Múltiplo do tamanho", "Múltiplo del tamaño", "尺寸倍数"],
"Size mode": ["Modo de tamanho", "Modo de tamaño", "尺寸模式"], "Megapixel target": ["Alvo em megapixels", "Objetivo en megapíxeles", "百万像素目标"],
"Original video": ["Vídeo original", "Vídeo original", "原视频"], "Custom dimensions": ["Dimensões personalizadas", "Dimensiones personalizadas", "自定义尺寸"],
"Output size": ["Tamanho de saída", "Tamaño de salida", "输出尺寸"],
"Custom width": ["Largura personalizada", "Ancho personalizado", "自定义宽度"], "Custom height": ["Altura personalizada", "Alto personalizado", "自定义高度"],
"Image max side (VLM)": ["Lado máximo da imagem (VLM)", "Lado máximo de imagen (VLM)", "VLM 图像最长边"],
"VLM image sizing": ["Redimensiona as imagens enviadas ao VLM, mantendo a proporção. O processador do modelo ainda pode redimensioná-las.", "Ajusta las imágenes enviadas al VLM manteniendo la proporción. El procesador del modelo puede volver a ajustarlas.", "调整发送给 VLM 的图像并保持比例。模型处理器仍可能再次缩放。"],
"Original resolution is rounded to the size multiple for model compatibility.": ["A resolução original é arredondada ao múltiplo definido para compatibilidade com o modelo.", "La resolución original se redondea al múltiplo indicado para compatibilidad con el modelo.", "原始分辨率会取整到尺寸倍数,以兼容模型。"],
"✂ Auto split": ["✂ Dividir automaticamente", "✂ División automática", "✂ 自动拆分"], "↻ Re-analyse": ["↻ Analisar novamente", "↻ Volver a analizar", "↻ 重新分析"],
"Reference & prompt": ["Referência e prompt", "Referencia y prompt", "参考图和提示词"], "Who is replaced": ["Quem será substituído", "Quién se reemplaza", "替换对象"],
"Continuity": ["Continuidade", "Continuidad", "连续性"], "Copy to other shots": ["Copiar para outros planos", "Copiar a otros planos", "复制到其他镜头"],
"Global reference & prompt": ["Referência e prompt globais", "Referencia y prompt globales", "全局参考图和提示词"],
"Describe references ({details})": ["Descrever referências ({details})", "Describir referencias ({details})", "描述参考图({details})"],
"Ref 1 description ({details})": ["Descrição da ref. 1 ({details})", "Descripción de ref. 1 ({details})", "参考图 1 描述({details})"],
"Ref 2 description ({details2})": ["Descrição da ref. 2 ({details2})", "Descripción de ref. 2 ({details2})", "参考图 2 描述({details2})"],
"no description yet: Describe refs, write your own, or leave it to the VLM at run time": ["sem descrição: use Descrever referências, escreva uma descrição ou deixe o VLM gerar durante a execução", "sin descripción: pulsa Describir referencias, escribe una o deja que el VLM la genere al ejecutar", "暂无描述:点击“描述参考图”、自行填写,或在运行时由 VLM 生成"],
"VLM": ["VLM", "VLM", "视觉语言模型"], "Use the VLM when the workflow runs": ["Usar o VLM ao executar o workflow", "Usar el VLM al ejecutar el flujo", "运行工作流时使用 VLM"],
"Fill the mask text": ["Preencher o texto da máscara", "Completar el texto de la máscara", "填写蒙版描述"],
"Fill {shot} in the prompt": ["Preencher {shot} no prompt", "Completar {shot} en el prompt", "在提示词中填入 {shot}"],
"Frames per shot": ["Frames por plano", "Fotogramas por plano", "每个镜头的帧数"], "Max tokens": ["Máx. tokens", "Máx. tokens", "最大 token 数"],
"🤖 Analyse shots": ["🤖 Analisar planos", "🤖 Analizar planos", "🤖 分析镜头"], "📝 Describe refs": ["📝 Descrever referências", "📝 Describir referencias", "📝 描述参考图"],
"🎨 Colored Mark": ["🎨 Marcação colorida", "🎨 Marca de color", "🎨 彩色标记"], "Colored Mark": ["Marcação colorida", "Marca de color", "彩色标记"], "Ref 1 color": ["Cor da ref. 1", "Color de ref. 1", "参考图 1 的颜色"],
"Ref 2": ["Ref. 2", "Ref. 2", "参考图 2"], "+ Add a second person (optional)": ["+ Adicionar segunda pessoa (opcional)", "+ Añadir segunda persona (opcional)", "+ 添加第二个人(可选)"],
"Second person · Ref 2": ["Segunda pessoa · Ref. 2", "Segunda persona · Ref. 2", "第二个人 · 参考图 2"],
"Describe the second person for SAM 3": ["Descreva a segunda pessoa para o SAM 3", "Describe la segunda persona para SAM 3", "为 SAM 3 描述第二个人"],
"👁 Preview": ["👁 Prévia", "👁 Vista previa", "👁 预览"], "More options": ["Mais opções", "Más opciones", "更多选项"],
"Keep ref 2 color consistent across shots (optional)": ["Manter a cor da ref. 2 nos outros planos (opcional)", "Mantener el color de ref. 2 en otros planos (opcional)", "在其他镜头中保持参考图 2 的颜色(可选)"],
"Ref 1 identity": ["Identidade da ref. 1", "Identidad de ref. 1", "参考图 1 的身份"], "Ref 2 identity": ["Identidade da ref. 2", "Identidad de ref. 2", "参考图 2 的身份"],
"Choose manually": ["Escolher manualmente", "Elegir manualmente", "手动选择"], "↩ cast color": ["↩ cor do elenco", "↩ color del reparto", "↩ 使用角色颜色"],
"Opacity": ["Opacidade", "Opacidad", "不透明度"], "The default keeps movement and expression visible.": ["O padrão mantém o movimento e a expressão visíveis.", "El valor predeterminado mantiene visibles el movimiento y la expresión.", "默认值会保留动作和表情。"],
"Full frame": ["Quadro inteiro", "Fotograma completo", "整帧"], "🧩 Frame + paste": ["🧩 Quadro + colar", "🧩 Fotograma + pegar", "🧩 整帧生成并贴回"],
"🎭 Mask only": ["🎭 Somente máscara", "🎭 Solo máscara", "🎭 仅蒙版"], "✂ Crop": ["✂ Recorte", "✂ Recorte", "✂ 裁剪"], "✂🎭 Crop + mask": ["✂🎭 Recorte + máscara", "✂🎭 Recorte + máscara", "✂🎭 裁剪并蒙版"],
"People & masks": ["Pessoas e máscaras", "Personas y máscaras", "人物和蒙版"], "Cast": ["Elenco", "Reparto", "角色"],
"Mask & crop settings": ["Configurações de máscara e recorte", "Ajustes de máscara y recorte", "蒙版和裁剪设置"],
"Green": ["Verde", "Verde", "绿色"], "Purple": ["Roxo", "Morado", "紫色"],
"Red": ["Vermelho", "Rojo", "红色"], "Blue": ["Azul", "Azul", "蓝色"], "Yellow": ["Amarelo", "Amarillo", "黄色"], "Magenta": ["Magenta", "Magenta", "品红"], "Cyan": ["Ciano", "Cian", "青色"],
"Checks": ["Verificações", "Comprobaciones", "检查"], "Run": ["Executar", "Ejecutar", "运行"], "Filters": ["Filtros", "Filtros", "筛选"],
"Auto loop (one run)": ["Loop automático (uma execução)", "Bucle automático (una ejecución)", "自动循环(一次运行)"],
"Queue loop (one shot per run)": ["Fila (um plano por execução)", "Cola (un plano por ejecución)", "队列循环(每次运行一个镜头)"],
"Skipped shots in the output": ["Planos ignorados na saída", "Planos omitidos en la salida", "输出中跳过的镜头"], "Keep original video": ["Manter vídeo original", "Mantener vídeo original", "保留原视频"],
"Remove them": ["Removê-los", "Quitarlos", "移除"], "Max shots (0 = all)": ["Máx. planos (0 = todos)", "Máx. planos (0 = todos)", "镜头上限(0 = 全部)"],
"Max total seconds (0 = all)": ["Máx. duração total (0 = tudo)", "Duración total máx. (0 = todo)", "总时长上限(0 = 全部)"],
"Auto-queue the next shot": ["Enfileirar próximo plano automaticamente", "Añadir automáticamente el siguiente plano a la cola", "自动将下一个镜头加入队列"],
"Needs a person": ["Precisa ter uma pessoa", "Debe haber una persona", "需要有人物"], "Needs a face": ["Precisa ter um rosto", "Debe haber un rostro", "需要有人脸"],
"Skip dark / fades": ["Ignorar cenas escuras / fades", "Omitir escenas oscuras / fundidos", "跳过暗场/淡入淡出"], "Skip static shots": ["Ignorar planos estáticos", "Omitir planos estáticos", "跳过静止镜头"],
"👤 Analyse people & faces": ["👤 Analisar pessoas e rostos", "👤 Analizar personas y rostros", "👤 分析人物和人脸"],
"Video": ["Vídeo", "Vídeo", "视频"], "Shots": ["Planos", "Planos", "镜头"], "People": ["Pessoas", "Personas", "人物"], "Prompts": ["Prompts", "Prompts", "提示词"],
"Back": ["Voltar", "Atrás", "上一步"], "Next": ["Avançar", "Siguiente", "下一步"], "Close tour": ["Fechar tour", "Cerrar recorrido", "关闭引导"],
"Shot Planner tour": ["Tour do Shot Planner", "Recorrido de Shot Planner", "Shot Planner 引导"], "Shot": ["Plano", "Plano", "镜头"],
"This tour is optional. Use Next to visit each main tab; close it whenever you like.": ["Este tour é opcional. Use Avançar para conhecer cada aba; feche quando quiser.", "Este recorrido es opcional. Pulsa Siguiente para visitar cada pestaña; puedes cerrarlo cuando quieras.", "此引导为可选。点击下一步浏览各个主选项卡,也可随时关闭。"],
"Choose a video, set the frame range, split at camera cuts, then set the generation size.": ["Escolha um vídeo e o trecho, divida nos cortes de câmera e defina o tamanho da geração.", "Elige un vídeo y un rango, divídelo en cortes de cámara y define el tamaño de generación.", "选择视频和范围,按镜头切换拆分,然后设置生成尺寸。"],
"Review each shot, assign its references and prompt, and choose how the result is applied.": ["Revise cada plano, associe referências e prompt e escolha como aplicar o resultado.", "Revisa cada plano, asigna referencias y prompt, y elige cómo aplicar el resultado.", "检查每个镜头,设置参考图和提示词,并选择结果的应用方式。"],
"Track detected face identities to filter or split shots. Set masks here or per shot.": ["Rastreie identidades faciais para filtrar ou dividir planos. Configure as máscaras aqui ou por plano.", "Sigue identidades faciales para filtrar o dividir planos. Configura máscaras aquí o por plano.", "跟踪人脸身份以筛选或拆分镜头。可在此处或逐镜头设置蒙版。"],
"Set shared references, prompts and optional VLM assistance. Written prompts are kept exactly as authored.": ["Defina referências, prompts e assistência opcional do VLM. Os prompts escritos são preservados como estão.", "Configura referencias, prompts y asistencia opcional del VLM. Los prompts escritos se conservan tal cual.", "设置共享参考图、提示词和可选的 VLM 辅助。提示词会保持原样。"],
"Check warnings, choose auto or queue loop, and set filters or quick-run limits.": ["Confira avisos, escolha loop automático ou fila e configure filtros e limites de teste.", "Revisa avisos, elige bucle automático o cola y configura filtros y límites de prueba.", "查看警告,选择自动循环或队列,并设置筛选条件和快速运行限制。"],
"Color the selected region in the source video as a translucent visual marker. It does not create or enable an inpainting mask.": ["Colorir a região selecionada no vídeo de origem como uma marca translúcida. Isso não cria nem ativa uma máscara de inpainting.", "Colorea la región seleccionada en el vídeo de origen como una marca translúcida. No crea ni activa una máscara de inpainting.", "在源视频的选定区域叠加半透明颜色标记,不会创建或启用修补蒙版。"],
"🎬 Shot Planner": ["🎬 Planejador de planos", "🎬 Planificador de planos", "🎬 镜头规划器"],
"— choose a video from the input folder —": ["— escolha um vídeo da pasta de entrada —", "— elige un vídeo de la carpeta de entrada —", "— 从输入文件夹选择视频 —"],
"⬆ Upload": ["⬆ Enviar", "⬆ Subir", "⬆ 上传"], "Pick or upload a video to start": ["Escolha ou envie um vídeo para começar", "Elige o sube un vídeo para empezar", "选择或上传视频以开始"],
"It is split into shots the model can generate; every shot then gets its own reference, prompt and mask.": ["O vídeo será dividido em planos que o modelo consegue gerar; cada plano terá sua própria referência, prompt e máscara.", "El vídeo se divide en planos que el modelo puede generar; cada uno tendrá su referencia, prompt y máscara.", "视频会拆分为模型可生成的镜头,每个镜头都有自己的参考图、提示词和蒙版。"],
"work on a part of the video": ["trabalhe em um trecho do vídeo", "trabaja en una parte del vídeo", "处理视频的一部分"],
"Start frame": ["Frame inicial", "Fotograma inicial", "起始帧"], "End frame (0 = end)": ["Frame final (0 = fim)", "Fotograma final (0 = final)", "结束帧(0 = 末尾)"],
"⇤ start here": ["⇤ começar aqui", "⇤ empezar aquí", "⇤ 从此处开始"], "⇥ end here": ["⇥ terminar aqui", "⇥ terminar aquí", "⇥ 在此处结束"], "✕ whole video": ["✕ vídeo inteiro", "✕ vídeo completo", "✕ 整个视频"],
"Camera cuts or fixed length, on the model's frame grid": ["Cortes de câmera ou duração fixa, na grade de frames do modelo", "Cortes de cámara o duración fija, según la cuadrícula de fotogramas del modelo", "按镜头切换或固定时长,并符合模型帧步长"],
"re-analyses": ["reanalisar", "vuelve a analizar", "重新分析"], "ignores Max seconds / shot": ["ignora Máx. segundos por plano", "ignora Máx. segundos por plano", "忽略每个镜头的最长秒数"],
"cropped shots use it for the crop": ["planos recortados usam esse valor no recorte", "los planos recortados usan este valor", "裁剪镜头会使用此尺寸"],
"Split where the main person changes": ["Dividir quando a pessoa principal mudar", "Dividir cuando cambie la persona principal", "主要人物变化时拆分"],
"Only run shots with a detected person": ["Executar apenas planos com pessoa detectada", "Ejecutar solo planos con una persona detectada", "仅运行检测到人物的镜头"],
"adds a boundary when the biggest face switches to someone else (re-splits the plan)": ["cria uma divisão quando o maior rosto muda para outra pessoa (refaz a divisão)", "añade un corte cuando el rostro más grande cambia de persona (vuelve a dividir)", "最大人脸切换到其他人时添加分界(重新拆分计划)"],
"shots with no detected, non-ignored person are skipped": ["planos sem pessoa detectada e não ignorada são pulados", "se omiten los planos sin una persona detectada y no ignorada", "跳过没有检测到未忽略人物的镜头"],
"Nobody found yet.": ["Nenhuma pessoa encontrada.", "Aún no se encontraron personas.", "尚未检测到人物。"], "↻ Re-analyse people": ["↻ Analisar pessoas novamente", "↻ Volver a analizar personas", "↻ 重新分析人物"], "👥 Find people": ["👥 Encontrar pessoas", "👥 Buscar personas", "👥 查找人物"],
"faces sampled every 0.5 s and grouped by identity (InsightFace, CPU)": ["rostos amostrados a cada 0,5 s e agrupados por identidade (InsightFace, CPU)", "rostros muestreados cada 0,5 s y agrupados por identidad (InsightFace, CPU)", "每 0.5 秒采样人脸并按身份分组(InsightFace,CPU)"],
"detects faces across the video and groups them into people": ["detecta rostos no vídeo e os agrupa por pessoa", "detecta rostros en el vídeo y los agrupa por persona", "检测视频中的人脸并按人物分组"],
"track face identities to filter or split shots": ["rastreie rostos, associe cada pessoa a uma referência e divida os planos por quem aparece", "rastrea rostros, vincula cada persona a una referencia y divide los planos según quién aparece", "跟踪人脸、关联参考图,并按出镜人物拆分镜头"],
"Each shot picks what it marks (🎯 text or points, or 🎞 its own mask video) and how it is generated (": ["Cada plano define a seleção (🎯 texto, pontos ou 🎞 vídeo de máscara) e o modo de geração (", "Cada plano define la selección (🎯 texto, puntos o 🎞 vídeo de máscara) y el modo de generación (", "每个镜头设置选区(🎯 文本、点或 🎞 蒙版视频)和生成方式("],
"🎞 Mask video (whole video)": ["🎞 Vídeo de máscara (vídeo inteiro)", "🎞 Vídeo de máscara (vídeo completo)", "🎞 蒙版视频(整个视频)"],
"— none: SAM 3 per shot —": ["— nenhum: SAM 3 por plano —", "— ninguno: SAM 3 por plano —", "— 无:每个镜头使用 SAM 3 —"], "Fill holes": ["Preencher buracos", "Rellenar huecos", "填充孔洞"], "Invert": ["Inverter", "Invertir", "反转"],
"Show masks on shots": ["Mostrar máscaras nos planos", "Mostrar máscaras en los planos", "在镜头卡片上显示蒙版"], "Expand (px)": ["Expandir (px)", "Expandir (px)", "扩展(像素)"],
"Temporal expand (frames)": ["Expansão temporal (frames)", "Expansión temporal (fotogramas)", "时间扩展(帧)"], "Detection threshold": ["Limiar de detecção", "Umbral de detección", "检测阈值"], "Max objects": ["Máx. objetos", "Máx. objetos", "最大对象数"],
"Crop padding (%)": ["Margem do recorte (%)", "Margen del recorte (%)", "裁剪边距(%)"], "Feather (px)": ["Suavização da borda (px)", "Suavizado del borde (px)", "羽化(像素)"], "Paste back": ["Colar de volta", "Pegar de vuelta", "贴回方式"],
"only the mask (feathered)": ["somente a máscara (suavizada)", "solo la máscara (suavizada)", "仅蒙版(羽化边缘)"], "the whole box (feathered)": ["a caixa inteira (suavizada)", "todo el cuadro (suavizado)", "整个框(羽化边缘)"], "Blockify (px, 0 = off)": ["Blocos (px, 0 = desligado)", "Bloques (px, 0 = desactivado)", "块化(像素,0 = 关闭)"],
"Uses the official sam3.1_multiplex_fp16 checkpoint (models/checkpoints), downloaded from Comfy-Org/sam3.1 the first time.": ["Usa o checkpoint oficial sam3.1_multiplex_fp16 (models/checkpoints), baixado de Comfy-Org/sam3.1 na primeira utilização.", "Usa el checkpoint oficial sam3.1_multiplex_fp16 (models/checkpoints), descargado de Comfy-Org/sam3.1 la primera vez.", "使用官方 sam3.1_multiplex_fp16 检查点(models/checkpoints),首次运行时从 Comfy-Org/sam3.1 下载。"],
"SAM 3": ["SAM 3", "SAM 3", "SAM 3"], "track face identities to filter or split shots": ["rastreie rostos, associe cada pessoa a uma referência e organize os planos por quem aparece", "rastrea rostros, vincula cada persona a una referencia y organiza los planos según quién aparece", "跟踪人脸、关联参考图,并按出镜人物分配镜头"],
"Global reference & prompt": ["Referência e prompt globais", "Referencia y prompt globales", "全局参考图和提示词"], "defaults for every shot": ["padrão para todos os planos", "valor predeterminado para cada plano", "所有镜头的默认设置"],
"who the shot becomes": ["em quem o plano será transformado", "en quién se transforma el plano", "镜头中的人物将替换为谁"], "Prompt for this shot (empty = global prompt). Placeholders: {target} {details} {shot} {setting}": ["Prompt deste plano (vazio = prompt global). Variáveis: {target} {details} {shot} {setting}", "Prompt de este plano (vacío = prompt global). Variables: {target} {details} {shot} {setting}", "此镜头的提示词(留空则使用全局提示词)。占位符:{target} {details} {shot} {setting}"],
"recent (click = ref, shift+click = ref 2):": ["recentes (clique = ref., Shift+clique = ref. 2):", "recientes (clic = ref., Mayús+clic = ref. 2):", "最近使用(点击 = 参考图,Shift+点击 = 参考图 2):"], "✕ clear": ["✕ limpar", "✕ borrar", "✕ 清除"], "✕ clear all": ["✕ limpar tudo", "✕ borrar todo", "✕ 全部清除"], "Use global": ["Usar global", "Usar global", "使用全局设置"],
"empty slots use the global references (Prompts & refs tab)": ["campos vazios usam a referência da pessoa (Elenco) ou a global (aba Prompts e referências)", "los campos vacíos usan la referencia de la persona (Reparto) o la global (Prompts y referencias)", "空白项使用人物参考图(角色)或全局参考图(提示词和参考图选项卡)"],
"Full frame": ["Quadro inteiro", "Fotograma completo", "整帧"], "The whole frame is regenerated. The mask, if any, only feeds {target} / the setting picture.": ["O quadro inteiro é gerado novamente. A máscara, se houver, só fornece {target} / a imagem de cenário.", "Se vuelve a generar todo el fotograma. La máscara, si existe, solo aporta {target} / la imagen del escenario.", "重新生成整帧。蒙版(如有)仅用于提供 {target} / 场景参考图。"],
"Only the mask is regenerated, on the whole frame: the rest stays exactly as it was. No crop / uncrop.": ["Somente a área da máscara é regenerada; o restante permanece igual. Sem recorte.", "Solo se regenera el área de la máscara; el resto permanece igual. Sin recorte.", "仅重新生成蒙版区域,其余部分保持不变。不进行裁剪。"],
"A box around the mask is regenerated (more pixels for a small subject); BFS Shot Join pastes it back.": ["A caixa ao redor da máscara é regenerada (mais pixels para pessoas pequenas); o BFS Shot Join cola o resultado de volta.", "Se regenera el cuadro alrededor de la máscara (más píxeles para sujetos pequeños); BFS Shot Join lo pega de vuelta.", "重新生成蒙版周围的区域(为较小主体提供更多像素);BFS Shot Join 会将其贴回。"],
"Inside the crop, only the mask is regenerated: the most detail with the background kept.": ["Dentro do recorte, somente a máscara é regenerada: mais detalhes e fundo preservado.", "Dentro del recorte, solo se regenera la máscara: más detalle y fondo conservado.", "仅重新生成裁剪区域内的蒙版:细节更多,同时保留背景。"],
"The whole frame is generated (the pose follows the guide, the new subject may be bigger); BFS Shot Join pastes only the subject onto the original: background exact.": ["O quadro inteiro é gerado (a pose segue o guia e o novo sujeito pode ser maior); o BFS Shot Join cola apenas o sujeito no original e preserva o fundo.", "Se genera todo el fotograma (la pose sigue la guía y el sujeto puede ser más grande); BFS Shot Join pega solo al sujeto sobre el original y conserva el fondo.", "生成整帧(姿势遵循引导,主体可更大);BFS Shot Join 仅将主体贴回原画面,背景保持原样。"],
"🎯 Points…": ["🎯 Pontos…", "🎯 Puntos…", "🎯 点…"], "👁 Preview": ["👁 Prévia", "👁 Vista previa", "👁 预览"], "✕ Clear": ["✕ Limpar", "✕ Borrar", "✕ 清除"],
"🎞 mask video": ["🎞 vídeo de máscara", "🎞 vídeo de máscara", "🎞 蒙版视频"], "↳ global prompt": ["↳ prompt global", "↳ prompt global", "↳ 全局提示词"], "— no prompt —": ["— sem prompt —", "— sin prompt —", "— 无提示词 —"],
"▶ Play all": ["▶ Reproduzir todos", "▶ Reproducir todos", "▶ 播放全部"], "■ Stop": ["■ Parar", "■ Detener", "■ 停止"], "loop shot": ["repetir plano", "repetir plano", "循环播放镜头"],
"Click the timeline to seek. Play all skips shots that will not run.": ["Clique na linha do tempo para buscar. Reproduzir todos ignora planos que não serão executados.", "Haz clic en la línea de tiempo para buscar. Reproducir todos omite los planos que no se ejecutarán.", "点击时间轴进行定位。播放全部会跳过不运行的镜头。"],
"Reference & prompt": ["Referência e prompt", "Referencia y prompt", "参考图和提示词"], "Who is replaced": ["Quem será substituído", "Quién se reemplaza", "替换对象"], "Continuity": ["Continuidade", "Continuidad", "连续性"],
"Global reference & prompt": ["Referência e prompt globais", "Referencia y prompt globales", "全局参考图和提示词"], "Describe references ({details})": ["Descrever referências ({details})", "Describir referencias ({details})", "描述参考图({details})"],
"Custom": ["Personalizado", "Personalizado", "自定义"], "short": ["curto", "breve", "简短"], "full body": ["corpo inteiro", "cuerpo entero", "全身"], "head / face": ["cabeça / rosto", "cabeza / rostro", "头部/面部"], "face attributes": ["características do rosto", "rasgos faciales", "面部特征"], "outfit": ["roupa", "ropa", "服装"],
"Apply suggestions → all": ["Aplicar sugestões → todos", "Aplicar sugerencias → todos", "应用建议 → 全部"], "fit the prompt": ["ajustar ao prompt", "ajustar al prompt", "适配提示词"],
"Connect a Qwen3-VL (CLIPLoader) to the planner's vlm input. The VLM buttons work after the workflow has run once with it connected (ComfyUI only hands models to nodes when they run). Each shot's suggestion shows in its editor.": ["Conecte um Qwen3-VL (CLIPLoader) à entrada vlm do planner. Os botões do VLM funcionam depois que o workflow executar uma vez com ele conectado. A sugestão de cada plano aparece no editor.", "Conecta un Qwen3-VL (CLIPLoader) a la entrada vlm del planificador. Los botones del VLM funcionan después de ejecutar el flujo una vez con él conectado. La sugerencia de cada plano aparece en su editor.", "将 Qwen3-VL(CLIPLoader)连接到规划器的 vlm 输入。工作流连接后运行一次,VLM 按钮即可使用。每个镜头的建议会显示在其编辑器中。"],
"look at every shot and suggests the mask, a description and whether to run it": ["analisa cada plano e sugere máscara, descrição e se deve ser executado", "analiza cada plano y sugiere una máscara, una descripción y si debe ejecutarse", "分析每个镜头并建议蒙版、描述及是否运行"],
"🎯 Track → whole video": ["🎯 Rastrear → vídeo inteiro", "🎯 Seguir → vídeo completo", "🎯 跟踪 → 整个视频"], "Save (this shot)": ["Salvar (este plano)", "Guardar (este plano)", "保存(此镜头)"], "Save → all shots": ["Salvar → todos os planos", "Guardar → todos los planos", "保存 → 所有镜头"], "Cancel": ["Cancelar", "Cancelar", "取消"], "Clear points": ["Limpar pontos", "Borrar puntos", "清除点"], "👁 Segment": ["👁 Segmentar", "👁 Segmentar", "👁 分割"],
"Apply these points to EVERY shot, each on its frame at the same relative position (a subject that stays in place)": ["Aplicar estes pontos a TODOS os planos, na mesma posição relativa (para um sujeito que não muda de lugar)", "Aplicar estos puntos a TODOS los planos, en la misma posición relativa (si el sujeto no se mueve)", "将这些点应用到每个镜头的相同相对位置(适用于位置不变的主体)"],
"the range starts at the player's frame": ["o trecho começa no frame atual do player", "el rango empieza en el fotograma actual", "范围从播放器当前帧开始"], "the range ends after the player's frame": ["o trecho termina após o frame atual do player", "el rango termina después del fotograma actual", "范围在播放器当前帧之后结束"],
"Nothing to fix in the shots that run.": ["Nenhum problema nos planos que serão executados.", "No hay nada que corregir en los planos que se ejecutarán.", "将运行的镜头没有需要修复的问题。"], "all good": ["tudo certo", "todo bien", "一切正常"], "warning(s)": ["aviso(s)", "aviso(s)", "条警告"], "note(s)": ["observação(ões)", "nota(s)", "条备注"],
"quick tests": ["testes rápidos", "pruebas rápidas", "快速测试"], "from the start of the range": ["a partir do início do trecho", "desde el inicio del rango", "从范围起点开始"], "Each run generates one shot and stores it. Nodes after BFS Shot Join only run on the last shot, with the full video.": ["Cada execução gera e salva um plano. Os nodes após BFS Shot Join só executam no último plano, com o vídeo completo.", "Cada ejecución genera y guarda un plano. Los nodos posteriores a BFS Shot Join solo se ejecutan en el último plano, con el vídeo completo.", "每次运行生成并保存一个镜头。BFS Shot Join 后的节点仅在最后一个镜头运行,并接收完整视频。"],
"skipped shots do not run": ["planos ignorados não são executados", "los planos omitidos no se ejecutan", "跳过的镜头不会运行"], "only shots that run": ["somente planos executados", "solo los planos que se ejecutan", "仅统计将运行的镜头"],
"off": ["desligado", "desactivado", "关闭"], "on": ["ligado", "activado", "开启"], "run": ["executar", "ejecutar", "运行"], "skip": ["ignorar", "omitir", "跳过"], "filters decide": ["filtros decidem", "deciden los filtros", "由筛选条件决定"], "always run": ["sempre executar", "ejecutar siempre", "始终运行"], "always skip": ["sempre ignorar", "omitir siempre", "始终跳过"], "⏸ Disable": ["⏸ Desativar", "⏸ Desactivar", "⏸ 禁用"], "▶ Enable": ["▶ Ativar", "▶ Activar", "▶ 启用"], "✂ Split": ["✂ Dividir", "✂ Dividir", "✂ 拆分"], "⇥ Merge": ["⇥ Unir", "⇥ Unir", "⇥ 合并"],
"Advanced": ["Avançado", "Avanzado", "高级"], "English": ["Inglês", "Inglés", "英语"], "Português": ["Português", "Portugués", "葡萄牙语"], "Español": ["Espanhol", "Español", "西班牙语"], "中文": ["Chinês", "Chino", "中文"],
"Video": ["Vídeo", "Vídeo", "视频"], "Shots": ["Planos", "Planos", "镜头"], "Prompts": ["Prompts", "Prompts", "提示词"],
"Marker color · Ref 1": ["Cor do marcador · Ref. 1", "Color de marca · Ref. 1", "标记颜色 · 参考图 1"], "Marker color · Ref 2": ["Cor do marcador · Ref. 2", "Color de marca · Ref. 2", "标记颜色 · 参考图 2"],
"These are the default marker colors for the global references. Turn on Colored Mark on a shot to apply them; shot-level colors can override them.": ["Estas são as cores padrão das referências globais. Ative Marcação colorida no plano para usá-las; cada plano pode substituí-las.", "Estos son los colores predeterminados de las referencias globales. Activa Marca de color en un plano para aplicarlos; cada plano puede cambiarlos.", "这些是全局参考图的默认标记颜色。启用镜头的彩色标记即可应用;镜头也可以单独覆盖。"],
"Audio": ["Áudio", "Audio", "音频"], "use the video's audio": ["usar o áudio do vídeo", "usar el audio del vídeo", "使用视频音频"], "silent track": ["faixa silenciosa", "pista silenciosa", "静音音轨"],
"auto": ["automático", "automático", "自动"], "fixed": ["fixo", "fijo", "固定"], "single": ["único", "único", "单个"], "adaptive": ["adaptativo", "adaptativo", "自适应"], "content": ["conteúdo", "contenido", "内容"], "builtin": ["integrado", "integrado", "内置"],
"PySceneDetect adaptive": ["PySceneDetect adaptativo", "PySceneDetect adaptativo", "PySceneDetect 自适应"], "PySceneDetect content": ["PySceneDetect por conteúdo", "PySceneDetect por contenido", "PySceneDetect 内容检测"], "Built-in": ["Integrado", "Integrado", "内置"],
"Auto loop": ["Loop automático", "Bucle automático", "自动循环"], "Queue loop": ["Loop em fila", "Bucle en cola", "队列循环"], "queue loop": ["loop em fila", "bucle en cola", "队列循环"], "auto loop": ["loop automático", "bucle automático", "自动循环"],
"no audio": ["sem áudio", "sin audio", "无音频"], "silent track": ["faixa silenciosa", "pista silenciosa", "静音音轨"], "timeline": ["linha do tempo", "línea de tiempo", "时间轴"], "generate at": ["gerar em", "generar a", "生成尺寸"],
"Max person size (0-1 of frame)": ["Tamanho mínimo da pessoa (0–1 do quadro)", "Tamaño mínimo de persona (0–1 del fotograma)", "人物最小尺寸(占画面 0–1)"], "Min person size (0-1 of frame)": ["Tamanho mínimo da pessoa (0–1 do quadro)", "Tamaño mínimo de persona (0–1 del fotograma)", "人物最小尺寸(占画面 0–1)"],
"Max people (0 = any)": ["Máx. pessoas (0 = qualquer)", "Máx. personas (0 = cualquiera)", "人物上限(0 = 不限)"], "Min frames (0 = off)": ["Mín. frames (0 = desligado)", "Mín. fotogramas (0 = desactivado)", "最少帧数(0 = 关闭)"], "Frames sampled / shot": ["Frames amostrados por plano", "Fotogramas muestreados por plano", "每个镜头采样帧数"], "Dark below (0-1)": ["Escuro abaixo de (0–1)", "Oscuro por debajo de (0–1)", "暗度低于(0–1)"], "Static below": ["Estático abaixo de", "Estático por debajo de", "静止度低于"],
"e.g. 0.03 skips wide shots": ["ex.: 0,03 ignora planos abertos", "p. ej., 0,03 omite planos amplios", "例如 0.03 会跳过远景镜头"], "skip crowds": ["ignorar multidões", "omitir multitudes", "跳过人群"], "re-analyses": ["reanalisar", "vuelve a analizar", "重新分析"],
"Describe target": ["Descrever alvo", "Describir objetivo", "描述目标"], "🧑 Describe target": ["🧑 Descrever alvo", "🧑 Describir objetivo", "🧑 描述目标"], "new subject": ["novo sujeito", "nuevo sujeto", "新主体"], "person (or dog, cat, car… what the result shows)": ["pessoa (ou cão, gato, carro… o que aparecerá no resultado)", "persona (o perro, gato, coche… lo que aparecerá en el resultado)", "人物(或狗、猫、汽车等结果中的主体)"],
"The second person (optional)": ["Segunda pessoa (opcional)", "Segunda persona (opcional)", "第二个人(可选)"], "Follow cast": ["Seguir identidade", "Seguir reparto", "跟随角色"], "manual identity": ["identidade manual", "identidad manual", "手动身份"],
"Guide video": ["Vídeo guia", "Vídeo guía", "引导视频"], "Reference": ["Referência", "Referencia", "参考图"], "Prompt": ["Prompt", "Prompt", "提示词"], "reference": ["referência", "referencia", "参考图"],
"on": ["ativado", "activado", "开启"], "off": ["desativado", "desactivado", "关闭"], "not analysed": ["não analisado", "sin analizar", "未分析"], "not analyzed": ["não analisado", "sin analizar", "未分析"],
"No shots yet": ["Nenhum plano ainda", "Aún no hay planos", "尚无镜头"], "No video": ["Nenhum vídeo", "Sin vídeo", "没有视频"], "No faces": ["Nenhum rosto", "Sin rostros", "没有人脸"], "no faces": ["sem rostos", "sin rostros", "没有人脸"],
"cut": ["corte", "corte", "切换"], "run": ["executar", "ejecutar", "运行"], "skip": ["ignorar", "omitir", "跳过"], "skip this shot": ["ignorar este plano", "omitir este plano", "跳过此镜头"],
"people": ["pessoas", "personas", "人物"], "person": ["pessoa", "persona", "人物"], "shots": ["planos", "planos", "镜头"], "shot(s)": ["plano(s)", "plano(s)", "个镜头"], "linked": ["vinculada", "vinculada", "已关联"], "main": ["principal", "principal", "主要"],
"No VLM yet": ["Nenhum VLM carregado", "Aún no hay un VLM", "尚未加载 VLM"], "Connect a Qwen3-VL": ["Conecte um Qwen3-VL", "Conecta un Qwen3-VL", "连接 Qwen3-VL"],
"↻ Status": ["↻ Status", "↻ Estado", "↻ 状态"], "⟲ Reset loop": ["⟲ Reiniciar loop", "⟲ Reiniciar bucle", "⟲ 重置循环"], "Queue loop": ["Loop em fila", "Bucle en cola", "队列循环"],
"Start here": ["Começar aqui", "Empezar aquí", "从此处开始"], "End here": ["Terminar aqui", "Terminar aquí", "在此处结束"], "reset": ["redefinir", "restablecer", "重置"], "all shots": ["todos os planos", "todos los planos", "所有镜头"], "this shot": ["este plano", "este plano", "此镜头"],
"Describe target": ["Descrever alvo", "Describir objetivo", "描述目标"], "Settings": ["Configurações", "Ajustes", "设置"], "mask text": ["texto da máscara", "texto de máscara", "蒙版描述"],
"The global reference and prompt are used when a shot has no values of its own.": ["A referência e o prompt globais são usados quando um plano não tem valores próprios.", "La referencia y el prompt globales se usan cuando un plano no tiene valores propios.", "镜头没有单独设置时会使用全局参考图和提示词。"],
"A VLM is optional": ["O VLM é opcional", "El VLM es opcional", "VLM 为可选项"], "Optional": ["Opcional", "Opcional", "可选"], "optional": ["opcional", "opcional", "可选"],
"person in white": ["pessoa de branco", "persona de blanco", "穿白衣的人"], "woman in pink top": ["mulher de blusa rosa", "mujer con blusa rosa", "穿粉色上衣的女人"],
"nothing found": ["nada encontrado", "no se encontró nada", "未找到内容"], "not found": ["não encontrado", "no encontrado", "未找到"], "Preview": ["Prévia", "Vista previa", "预览"],
"Mask only / Crop + mask": ["Somente máscara / Recorte + máscara", "Solo máscara / Recorte + máscara", "仅蒙版 / 裁剪并蒙版"], "per shot (planner)": ["por plano (planner)", "por plano (planificador)", "逐镜头(规划器)"],
"Person": ["Pessoa", "Persona", "人物"], "shot": ["plano", "plano", "镜头"], "shot(s)": ["plano(s)", "plano(s)", "个镜头"],
"mask covers": ["máscara cobre", "la máscara cubre", "蒙版覆盖"], "nothing to fix": ["nada a corrigir", "nada que corregir", "无需修复"],
"Camera cuts: shots start at the detected cuts, long ones are split evenly. Fixed length: equal parts no longer than the maximum.": ["Cortes de câmera: os planos começam nos cortes detectados; planos longos são divididos igualmente. Duração fixa: partes iguais limitadas pelo máximo.", "Cortes de cámara: los planos empiezan en los cortes detectados; los largos se dividen por igual. Duración fija: partes iguales con el máximo indicado.", "镜头切换:镜头从检测到的切点开始,较长镜头会均匀拆分。固定时长:拆分为不超过上限的等长片段。"],
"Cut detector. PySceneDetect adaptive works on most footage (fast camera moves don't trigger it); content is stricter on hard cuts; built-in needs no package.": ["Detector de cortes. O modo adaptativo do PySceneDetect funciona na maioria dos vídeos (movimentos rápidos de câmera não acionam cortes); o modo conteúdo é mais rigoroso; o integrado não exige pacote.", "Detector de cortes. El modo adaptativo de PySceneDetect funciona en la mayoría de vídeos (los movimientos rápidos no activan cortes); contenido es más estricto; el integrado no requiere paquetes.", "镜头切换检测器。PySceneDetect 自适应模式适用于大多数素材(快速运镜不会触发切点);内容模式对硬切更严格;内置模式无需额外软件包。"],
"How easily a change counts as a cut. Higher finds more cuts (and more false ones).": ["O quanto uma mudança é considerada corte. Um valor maior encontra mais cortes e também mais falsos positivos.", "Facilidad con la que un cambio cuenta como corte. Un valor mayor detecta más cortes y también más falsos positivos.", "变化被判定为切点的难易程度。数值越高,检测到的切点越多,误报也越多。"],
"Frame counts the model accepts. H3: 17n+5, LTX/Wan: 8n+1, Wan: 4n+1. Each shot is generated at the next valid length and trimmed back.": ["Contagens de frames aceitas pelo modelo. H3: 17n+5; LTX/Wan: 8n+1; Wan: 4n+1. Cada plano é gerado no próximo comprimento válido e depois aparado.", "Cantidad de fotogramas que acepta el modelo. H3: 17n+5; LTX/Wan: 8n+1; Wan: 4n+1. Cada plano se genera con la siguiente duración válida y luego se recorta.", "模型接受的帧数规则。H3:17n+5;LTX/Wan:8n+1;Wan:4n+1。每个镜头按下一个有效长度生成后再裁短。"],
"Longest shot sent to the model. Keep it within what the model / LoRA was trained on (H3 body swap: about 4.5 s).": ["Duração máxima enviada ao modelo. Mantenha dentro do que o modelo/LoRA aprendeu (body swap H3: cerca de 4,5 s).", "Duración máxima enviada al modelo. Respeta lo usado en el entrenamiento del modelo/LoRA (body swap H3: unos 4,5 s).", "发送给模型的镜头最长时长。应符合模型/LoRA 的训练长度(H3 换人约 4.5 秒)。"],
"Shorter shots merge into a neighbour when the merge still fits the maximum.": ["Planos mais curtos são unidos ao vizinho quando a junção ainda respeita o máximo.", "Los planos más cortos se unen al vecino si la unión respeta el máximo.", "较短镜头会在合并后仍未超过上限时与相邻镜头合并。"],
"The planner works on this frame rate (the source is resampled). Generated video and audio use it too.": ["O planner usa esta taxa de frames (o vídeo de origem é reamostrado). O vídeo e áudio gerados também a usam.", "El planificador usa esta frecuencia de fotogramas (el origen se remuestrea). El vídeo y el audio generados también la usan.", "规划器使用此帧率(源视频会重新采样)。生成的视频和音频也会使用此帧率。"],
"Only the first N shots (0 = all). Handy for quick tests.": ["Somente os primeiros N planos (0 = todos). Útil para testes rápidos.", "Solo los primeros N planos (0 = todos). Útil para pruebas rápidas.", "仅处理前 N 个镜头(0 = 全部)。适合快速测试。"],
"Only the first N seconds of the video (0 = all).": ["Somente os primeiros N segundos do vídeo (0 = todos).", "Solo los primeros N segundos del vídeo (0 = todos).", "仅处理视频前 N 秒(0 = 全部)。"],
"Generation size: the source aspect ratio at this pixel area (0.15 ≈ 512×288). Cropped shots use it for the crop.": ["Tamanho da geração: mantém a proporção do vídeo nessa área de pixels (0,15 ≈ 512×288). Planos recortados usam esse valor no recorte.", "Tamaño de generación: mantiene la proporción de origen con esta área de píxeles (0,15 ≈ 512×288). Los planos recortados usan este valor.", "生成尺寸:按此像素面积保持源视频宽高比(0.15 ≈ 512×288)。裁剪镜头使用此尺寸。"],
"Width and height snap to this multiple (32 for H3 and most video models).": ["Largura e altura são ajustadas a este múltiplo (32 para H3 e a maioria dos modelos de vídeo).", "El ancho y alto se ajustan a este múltiplo (32 para H3 y la mayoría de modelos de vídeo).", "宽度和高度会对齐到此倍数(H3 和多数视频模型使用 32)。"],
"Auto loop: every shot in one queue run. Queue loop: one shot per run, stored on disk, the join outputs the video after the last shot.": ["Loop automático: todos os planos em uma execução da fila. Loop em fila: um plano por execução, salvo em disco; o join gera o vídeo após o último plano.", "Bucle automático: todos los planos en una ejecución. Bucle en cola: un plano por ejecución, guardado en disco; el join genera el vídeo al terminar el último.", "自动循环:一次队列运行所有镜头。队列循环:每次运行一个镜头并保存到磁盘;最后一个镜头完成后由 Join 输出视频。"],
"Shots that do not run: keep the original video there, or remove them (with their audio).": ["Planos não executados: manter o vídeo original nesses trechos ou removê-los junto com o áudio.", "Planos que no se ejecutan: conserva el vídeo original en ese tramo o elimínalo junto con su audio.", "未运行的镜头:保留对应原视频片段,或连同音频一起移除。"],
"Context kept around the mask's box, as a % of the box. More context blends better; less gives the model more pixels on the subject.": ["Contexto mantido ao redor da caixa da máscara, como porcentagem da caixa. Mais contexto mistura melhor; menos dá mais pixels ao sujeito.", "Contexto alrededor del cuadro de la máscara, como porcentaje. Más contexto mezcla mejor; menos ofrece más píxeles al sujeto.", "蒙版边框周围保留的上下文比例。上下文越多越容易融合;越少则主体像素越多。"],
"Grow the mask by this many pixels: before pasting a cropped result back, and for the generation mask (inpaint 'only the mask'). Raise it when the new person is bigger than the old one (longer hair, wider body).": ["Expande a máscara por esta quantidade de pixels antes de colar o recorte e na máscara de geração. Aumente se a nova pessoa for maior (cabelo mais comprido, corpo mais largo).", "Amplía la máscara esta cantidad de píxeles antes de pegar el recorte y para la máscara de generación. Auméntala si la persona nueva es más grande.", "粘贴裁剪结果前以及生成蒙版时,将蒙版扩展指定像素。新人物更大时(头发更长、身体更宽)可调高。"],
"Soft edge of the paste-back, in pixels.": ["Suavização da borda ao colar de volta, em pixels.", "Suavizado del borde al pegar de vuelta, en píxeles.", "贴回时边缘的柔化像素数。"],
"Hold the mask this many frames before and after: removes flicker where SAM 3 drops a frame.": ["Estende a máscara por esta quantidade de frames antes e depois; reduz falhas quando o SAM 3 perde um frame.", "Mantiene la máscara estos fotogramas antes y después; reduce parpadeos cuando SAM 3 pierde uno.", "蒙版前后延续指定帧数,减少 SAM 3 漏掉某帧时的闪烁。"],
"Square blocks of this size (0 = off). 16 matches H3's latent grid, useful as an inpainting mask.": ["Blocos quadrados deste tamanho (0 = desligado). 16 corresponde à grade latente do H3 e pode ser útil como máscara de inpainting.", "Bloques cuadrados de este tamaño (0 = desactivado). 16 coincide con la cuadrícula latente de H3 y sirve para inpainting.", "使用此尺寸的方块(0 = 关闭)。16 与 H3 潜空间网格匹配,可用于修补蒙版。"],
"SAM 3 score needed to keep a text match. Lower finds more (and more wrong) objects.": ["Pontuação mínima do SAM 3 para manter uma correspondência por texto. Valores menores encontram mais objetos, inclusive incorretos.", "Puntuación mínima de SAM 3 para conservar una coincidencia de texto. Un valor menor encuentra más objetos, también erróneos.", "保留文本匹配所需的 SAM 3 分数。调低会找到更多对象,也会增加误检。"],
"How many matches of the text prompt are tracked together.": ["Quantas correspondências do texto serão rastreadas juntas.", "Cuántas coincidencias del texto se siguen a la vez.", "同时跟踪多少个符合文本的对象。"],
"Only the mask: the result replaces the masked pixels. The whole box: the full crop is pasted with feathered borders.": ["Somente máscara: o resultado substitui os pixels mascarados. Caixa inteira: todo o recorte é colado com bordas suaves.", "Solo máscara: el resultado sustituye los píxeles enmascarados. Cuadro completo: se pega todo el recorte con bordes suavizados.", "仅蒙版:结果替换蒙版像素。整个框:粘贴完整裁剪区域并柔化边缘。"],
"How many frames of each shot the VLM looks at.": ["Quantos frames de cada plano o VLM analisa.", "Cuántos fotogramas de cada plano analiza el VLM.", "VLM 分析每个镜头的帧数。"], "Longest VLM answer. Raise it for long descriptions.": ["Tamanho máximo da resposta do VLM. Aumente para descrições longas.", "Longitud máxima de respuesta del VLM. Auméntala para descripciones largas.", "VLM 回复的最大长度。长描述可调高。"],
"skipped by hand": ["ignorado manualmente", "omitido manualmente", "手动跳过"], "no detected person": ["nenhuma pessoa detectada", "no se detectó ninguna persona", "未检测到人物"],
"longer than {…} frames: split it": ["mais longo que {…} frames: divida o plano", "supera {…} fotogramas: divide el plano", "超过 {…} 帧:请拆分镜头"],
"{…} needs a mask (text, points or a mask video; or the planner's mask input)": ["{…} precisa de uma máscara (texto, pontos, vídeo de máscara ou entrada de máscara do Planner)", "{…} necesita una máscara (texto, puntos, vídeo de máscara o entrada de máscara del planificador)", "{…} 需要蒙版(文本、点、蒙版视频或 Planner 的蒙版输入)"],
"the mask preview found nothing": ["a prévia da máscara não encontrou nada", "la vista previa de la máscara no encontró nada", "蒙版预览未找到目标"],
"the prompt uses {target} but the shot has no target description": ["o prompt usa {target}, mas o plano não tem uma descrição do alvo", "el prompt usa {target}, pero el plano no tiene una descripción del objetivo", "提示词使用了 {target},但镜头没有目标描述"],
"the prompt uses {details} but its references have no description yet": ["o prompt usa {details}, mas as referências ainda não têm descrição", "el prompt usa {details}, pero las referencias aún no tienen descripción", "提示词使用了 {details},但参考图还没有描述"],
"{details} sits inside a sentence of the prompt: use the 'short' describe preset (Prompts & refs), the other presets write a paragraph": ["{details} está dentro de uma frase do prompt: use a descrição 'short' (Prompts e referências); os outros modos escrevem um parágrafo", "{details} está dentro de una frase del prompt: usa la opción 'short' (Prompts y referencias); las demás opciones escriben un párrafo", "{details} 位于提示词句子中:请使用“short”描述模式(Prompts & refs),其他模式会生成段落"],
"no prompt (fine when the node's prompt input is connected)": ["sem prompt (tudo certo se a entrada prompt do node estiver conectada)", "sin prompt (no pasa nada si la entrada prompt del nodo está conectada)", "没有提示词(若节点的 prompt 输入已连接则无需处理)"],
"no reference (fine when the node's ref_image input is connected)": ["sem referência (tudo certo se a entrada ref_image do node estiver conectada)", "sin referencia (no pasa nada si la entrada ref_image del nodo está conectada)", "没有参考图(若节点的 ref_image 输入已连接则无需处理)"],
"Analysing video…": ["Analisando o vídeo…", "Analizando el vídeo…", "正在分析视频…"], "Detecting cuts…": ["Detectando cortes…", "Detectando cortes…", "正在检测切点…"], "Detecting people and faces…": ["Detectando pessoas e rostos…", "Detectando personas y rostros…", "正在检测人物和人脸…"],
"Finding people by face…": ["Localizando pessoas pelos rostos…", "Buscando personas por sus rostros…", "正在通过人脸查找人物…"], "Selecting the person (SAM 3) and describing them (VLM)…": ["Selecionando a pessoa (SAM 3) e descrevendo-a (VLM)…", "Seleccionando a la persona (SAM 3) y describiéndola (VLM)…", "正在选择人物(SAM 3)并生成描述(VLM)…"],
"The SAM 3 target cutout was created (shown below), but no VLM is loaded to describe it. Connect Qwen3-VL to the Shot Planner's vlm input and queue the workflow once; then click Describe target again. You can also type the target description yourself.": ["O recorte do alvo pelo SAM 3 foi criado abaixo, mas nenhum VLM está carregado para descrevê-lo. Conecte o Qwen3-VL à entrada vlm do Shot Planner e execute a fila uma vez; depois clique novamente em Descrever alvo. Você também pode escrever a descrição manualmente.", "SAM 3 creó el recorte del objetivo (se muestra abajo), pero no hay ningún VLM cargado para describirlo. Conecta Qwen3-VL a la entrada vlm de Shot Planner y ejecuta el flujo una vez; después vuelve a pulsar Describir objetivo. También puedes escribir la descripción.", "SAM 3 已生成目标抠图(见下方),但尚未加载 VLM 来描述它。将 Qwen3-VL 连接到 Shot Planner 的 vlm 输入并运行一次队列,然后再次点击“描述目标”。也可以手动填写描述。"],
"Segmenting with SAM 3… (the first time downloads/loads the model)": ["Segmentando com SAM 3… (na primeira vez, o modelo será baixado/carregado)", "Segmentando con SAM 3… (la primera vez se descarga/carga el modelo)", "正在使用 SAM 3 分割…(首次运行会下载并加载模型)"],
"open the checks (Run tab)": ["abrir as verificações (aba Executar)", "abrir las comprobaciones (pestaña Ejecutar)", "打开检查项(Run 标签页)"], "Open the full guide (every setting explained, with examples)": ["Abrir o guia completo (com explicações e exemplos de cada opção)", "Abrir la guía completa (explicaciones y ejemplos de cada opción)", "打开完整指南(包含各设置的说明和示例)"],
"click = keep (green) · right-click or shift+click = exclude (red) · click a point to remove it": ["clique = manter (verde) · clique direito ou Shift+clique = excluir (vermelho) · clique em um ponto para removê-lo", "clic = conservar (verde) · clic derecho o Mayús+clic = excluir (rojo) · clic en un punto para quitarlo", "单击 = 保留(绿色)· 右键或 Shift+单击 = 排除(红色)· 单击已有点可将其删除"],
"click to remove": ["clique para remover", "haz clic para quitar", "点击删除"], "text prompt (used when there are no points)": ["texto de seleção (usado quando não há pontos)", "texto de selección (se usa cuando no hay puntos)", "文本提示(没有标记点时使用)"],
"Save → all shots": ["Salvar → todos os planos", "Guardar → todos los planos", "保存 → 所有镜头"], "Save (this shot)": ["Salvar (este plano)", "Guardar (este plano)", "保存(此镜头)"],
"Track this selection with SAM 3 through the whole video (or the range), forward and backward from this frame; every shot without its own selection takes its part of the track. Also type a short text (e.g. 'woman in pink top') so the subject is found again after camera cuts.": ["Rastreie esta seleção com SAM 3 pelo vídeo inteiro (ou trecho), avançando e retrocedendo a partir deste frame. Cada plano sem seleção própria usa sua parte do rastreamento. Digite também uma descrição curta (por exemplo, 'mulher de blusa rosa') para reencontrar o sujeito após os cortes.", "Sigue esta selección con SAM 3 por todo el vídeo (o tramo), hacia delante y atrás desde este fotograma. Cada plano sin selección propia usa su parte del seguimiento. Añade también una descripción breve (por ejemplo, 'mujer con blusa rosa') para volver a encontrar al sujeto después de los cortes.", "使用 SAM 3 从此帧开始向前和向后跟踪所选对象,覆盖整个视频(或片段)。没有单独选区的镜头会使用对应的跟踪结果。再输入简短描述(如“穿粉色上衣的女性”),以便在切镜后重新找到主体。"],
"Colored Mark adds a translucent color overlay to a selected region in the source video. It does not enable inpainting. Choose the color before marking the region with points or a description.": ["Marcação colorida aplica uma sobreposição translúcida à região selecionada no vídeo de origem. Ela não ativa inpainting. Escolha a cor antes de marcar a região com pontos ou descrição.", "Marca de color aplica una superposición translúcida a la región seleccionada del vídeo original. No activa el inpainting. Elige el color antes de marcar con puntos o una descripción.", "彩色标记会在源视频的选定区域上叠加半透明颜色,不会启用修补。请先选颜色,再用点或描述标记区域。"],
"choose which shots run": ["escolher quais planos serão executados", "elegir qué planos se ejecutan", "选择要运行的镜头"],
"Fill enclosed gaps so each region is solid": ["Preencher espaços fechados para deixar cada região sólida", "Rellenar huecos cerrados para que cada región sea sólida", "填充封闭空隙,使区域完整"], "Use everything except the segmented object": ["Usar tudo, exceto o objeto segmentado", "Usar todo excepto el objeto segmentado", "使用除已分割对象以外的所有区域"], "Overlay the mask preview on the shot cards": ["Sobrepor a prévia da máscara nos cartões dos planos", "Superponer la vista previa de la máscara en las tarjetas de los planos", "在镜头卡片上叠加蒙版预览"],
"runs the detectors on a few frames of every shot": ["executa os detectores em alguns frames de cada plano", "ejecuta los detectores en algunos fotogramas de cada plano", "在每个镜头的若干帧上运行检测器"],
"Global prompt (a connected `prompt` input overrides it)": ["Prompt global (a entrada `prompt` conectada tem prioridade)", "Prompt global (la entrada `prompt` conectada tiene prioridad)", "全局提示词(若连接了 `prompt` 输入,则以输入为准)"],
"Used by shots without their own. Connected ref_image / ref_image_2 / prompt inputs on the node override these. Placeholders:": ["Usado nos planos sem valores próprios. As entradas ref_image / ref_image_2 / prompt conectadas ao node têm prioridade. Marcadores:", "Se usa en los planos sin valores propios. Las entradas ref_image / ref_image_2 / prompt conectadas al nodo tienen prioridad. Marcadores:", "镜头没有单独设置时使用。节点上连接的 ref_image / ref_image_2 / prompt 输入优先。占位符:"],
"description of the shot's references ·": ["descrição das referências do plano ·", "descripción de las referencias del plano ·", "镜头参考图描述 ·"], "who is replaced (per shot) ·": ["quem será substituído (por plano) ·", "quién se sustituye (por plano) ·", "被替换对象(逐镜头)·"], "the VLM's description of the shot ·": ["descrição do plano pelo VLM ·", "descripción del plano generada por el VLM ·", "VLM 对镜头的描述 ·"], "the setting picture (H3 Conditioning).": ["imagem de cenário (H3 Conditioning)。", "imagen del entorno (H3 Conditioning).", "场景参考图(H3 Conditioning)。"],
"At run time the planner asks the VLM about the shots it runs and applies the options below": ["Durante a execução, o Planner consulta o VLM sobre os planos processados e aplica as opções abaixo", "Durante la ejecución, el planificador consulta al VLM sobre los planos y aplica las opciones siguientes", "运行时,Planner 会让 VLM 分析要处理的镜头并应用以下选项"],
"Shots without a mask text or points get the VLM's segment suggestion": ["Planos sem texto de máscara ou pontos recebem uma sugestão de segmentação do VLM", "Los planos sin texto de máscara ni puntos reciben una sugerencia de segmentación del VLM", "没有蒙版文本或标记点的镜头会使用 VLM 的分割建议"],
"Write {shot} in a prompt: it becomes the VLM's description of that shot (camera, framing, action)": ["Use {shot} no prompt para inserir a descrição do plano pelo VLM (câmera, enquadramento e ação)", "Escribe {shot} en el prompt para insertar la descripción del plano generada por el VLM (cámara, encuadre y acción)", "在提示词中写入 {shot},即可插入 VLM 对镜头的描述(摄影机、构图和动作)"],
"Mask text for shots without one, skip where the VLM says skip": ["Texto de máscara para planos sem seleção; ignora os que o VLM indicar", "Texto de máscara para planos sin selección; omite los que el VLM marque", "为未设置选区的镜头填入蒙版文本,并跳过 VLM 建议跳过的镜头"],
"looks at every shot and suggests the mask, a description and whether to run it": ["analisa cada plano e sugere a máscara, uma descrição e se deve executá-lo", "analiza cada plano y sugiere la máscara, una descripción y si debe ejecutarse", "分析每个镜头并建议蒙版、描述以及是否运行"],
"Try another description: a new seed (and some temperature if it is 0), then describe again": ["Tentar outra descrição: usar uma nova seed (e aumentar a temperatura se estiver em 0) e descrever novamente", "Probar otra descripción: usar una nueva seed (y subir la temperatura si está en 0) y volver a describir", "重新生成描述:更换 seed(若温度为 0 则适当提高),再分析一次"],
"0 = always the same answer (greedy); 0.5-1 = varied answers, picked by the seed": ["0 = resposta sempre igual (greedy); 0,5–1 = respostas variadas, controladas pela seed", "0 = respuesta siempre igual (greedy); 0,5–1 = respuestas variadas según la seed", "0 = 始终生成相同答案(贪心);0.5–1 = 由 seed 控制的多样答案"],
"Used when the temperature is above 0: another seed, another description": ["Usada quando a temperatura é maior que 0: outra seed gera outra descrição", "Se usa cuando la temperatura es mayor que 0: otra seed genera otra descripción", "温度大于 0 时使用:更换 seed 可生成不同描述"],
"The VLM gets the global prompt's sentence that holds {details} (and the shot prompt's {target} sentence for 🧑 Describe target) with a blank in its place, and writes the description to fit it: the description adapts to the prompt, not the other way round.": ["O VLM recebe a frase do prompt global que contém {details} (ou a frase com {target} ao usar 🧑 Descrever alvo), com o marcador em branco, e escreve uma descrição que se encaixa nela.", "El VLM recibe la frase del prompt global que contiene {details} (o la frase con {target} al usar 🧑 Describir objetivo), con el marcador en blanco, y escribe una descripción que encaje.", "VLM 会读取包含 {details} 的全局提示词句子(使用 🧑 描述目标时则读取含 {target} 的镜头提示词),并在占位处生成适配的描述。"],
"Add a reference (global, per shot or per shot) first.": ["Adicione uma referência global ou por plano primeiro.", "Añade primero una referencia global o por plano.", "请先添加全局参考图或镜头参考图。"],
"✓ Nothing to fix in the shots that run.": ["✓ Nenhum ajuste necessário nos planos que serão executados.", "✓ No hay nada que corregir en los planos que se ejecutarán.", "✓ 将运行的镜头无需修正。"],
"Keep original video": ["Manter o vídeo original", "Conservar el vídeo original", "保留原视频"],
"Each run generates one shot and stores it. Nodes after BFS Shot Join only run on the last shot, with the full video.": ["Cada execução gera e salva um plano. Os nodes após BFS Shot Join só executam no último, com o vídeo completo.", "Cada ejecución genera y guarda un plano. Los nodos posteriores a BFS Shot Join solo se ejecutan en el último, con el vídeo completo.", "每次运行生成并保存一个镜头。BFS Shot Join 后的节点仅在最后一个镜头运行,并接收完整视频。"],
"go to this shot": ["ir para este plano", "ir a este plano", "跳转到此镜头"], "Pick a video in the Video tab.": ["Escolha um vídeo na aba Vídeo.", "Elige un vídeo en la pestaña Vídeo.", "请在 Video 标签页中选择视频。"],
"split this shot in two": ["dividir este plano em dois", "dividir este plano en dos", "将此镜头一分为二"], "merge with the next shot": ["unir ao próximo plano", "unir con el siguiente plano", "与下一个镜头合并"],
"Forget the recent pictures (the ones this plan still uses stay)": ["Esquecer as imagens recentes (as usadas neste plano serão mantidas)", "Olvidar las imágenes recientes (se conservan las que usa este plan)", "清除最近使用的图片(本计划仍在使用的图片会保留)"],
"Remove EVERY reference from this plan (global and all shots) and the recent list": ["Remover TODAS as referências deste plano (global e de todos os planos) e da lista recente", "Eliminar TODAS las referencias de este plan (globales y de todos los planos) y de la lista reciente", "从此计划(全局和所有镜头)及最近列表中移除所有参考图"],
"clear this shot's references and prompt: it uses the global ones": ["limpar as referências e o prompt deste plano para usar os globais", "borrar las referencias y el prompt de este plano para usar los globales", "清除此镜头的参考图和提示词,改用全局设置"],
"+ Mark a second region (optional)": ["+ Marcar uma segunda região (opcional)", "+ Marcar una segunda región (opcional)", "+ 标记第二个区域(可选)"], "Mark a second region · Ref 2": ["Marcar segunda região · Ref. 2", "Marcar segunda región · Ref. 2", "标记第二个区域 · 参考图 2"], "Describe second region": ["Descrever segunda região", "Describir la segunda región", "描述第二个区域"], "e.g. the person on the left": ["ex.: a pessoa à esquerda", "p. ej., la persona de la izquierda", "例如:左侧的人物"],
"mask: what to segment, in English (woman in pink top, black dog…) · Enter = preview": ["máscara: o que segmentar, em inglês (woman in pink top, black dog…) · Enter = prévia", "máscara: qué segmentar, en inglés (woman in pink top, black dog…) · Enter = vista previa", "蒙版:用英文描述分割对象(woman in pink top、black dog…)· Enter = 预览"],
"Pick positive / negative points on a frame of this shot (more precise than text when several people look alike)": ["Marque pontos positivos/negativos em um frame do plano (mais preciso que texto quando várias pessoas são parecidas)", "Marca puntos positivos/negativos en un fotograma del plano (más preciso que el texto si varias personas se parecen)", "在此镜头的一帧上标记正向/负向点(多人相似时比文本更准确)"],
"Segment this shot (the points win over the text)": ["Segmentar este plano (os pontos têm prioridade sobre o texto)", "Segmentar este plano (los puntos tienen prioridad sobre el texto)", "分割此镜头(标记点优先于文本)"], "Segment this shot with the text and show a few frames": ["Segmentar este plano pelo texto e mostrar alguns frames", "Segmentar este plano con el texto y mostrar algunos fotogramas", "根据文本分割此镜头并预览若干帧"],
"Colored Mark needs a region selection: enter a description, add points, or choose a mask video, then preview it.": ["A Marcação colorida precisa de uma região: digite uma descrição, adicione pontos ou escolha um vídeo de máscara e gere a prévia.", "La Marca de color necesita una región: escribe una descripción, añade puntos o elige un vídeo de máscara y genera la vista previa.", "彩色标记需要选定区域:输入描述、添加标记点或选择蒙版视频,然后预览。"],
"nothing found: the shot runs without a mask": ["nada encontrado: o plano será executado sem máscara", "no se encontró nada: el plano se ejecutará sin máscara", "未找到目标:此镜头将不使用蒙版运行"],
"The whole frame is regenerated. The mask, if any, only feeds {target} / the setting picture.": ["O frame inteiro é regenerado. Se houver máscara, ela só alimenta {target} ou a imagem de cenário.", "Se regenera todo el fotograma. Si hay máscara, solo se usa para {target} o la imagen del entorno.", "整帧都会重新生成。若有蒙版,仅用于 {target} 或场景参考图。"],
"What the NEW subject is, in English, for SAM 3 on the result: BFS Shot Join also pastes its outline (when it is bigger than the old one). E.g. person, dog, cat, car. Empty = person.": ["Descreva em inglês o NOVO sujeito no resultado para o SAM 3: o BFS Shot Join também cola seu contorno quando ele é maior que o antigo. Ex.: person, dog, cat, car. Vazio = person.", "Describe en inglés el NUEVO sujeto del resultado para SAM 3: BFS Shot Join también pega su silueta cuando supera la anterior. Ej.: person, dog, cat, car. Vacío = person.", "用英文描述结果中的新主体,供 SAM 3 使用:若新轮廓更大,BFS Shot Join 也会粘贴其轮廓。例如 person、dog、cat、car。留空 = person。"],
"Write {target} in the prompt: it becomes this description, so the model knows WHICH person to replace (useful with several people on screen).": ["Use {target} no prompt para inserir esta descrição e indicar ao modelo QUEM substituir (útil quando há várias pessoas na tela).", "Escribe {target} en el prompt para insertar esta descripción e indicar a quién debe sustituir el modelo (útil si hay varias personas).", "在提示词中使用 {target} 插入此描述,明确告诉模型要替换谁(画面中有多人时很有用)。"],
"Uses this shot's selection (🎯 Points… on the person — click the BODY, not only the face, so the outfit is described — or the mask text): SAM 3 cuts the person out and the VLM describes them": ["Usa a seleção deste plano (🎯 Pontos… no corpo, não apenas no rosto, para descrever a roupa; ou o texto da máscara): o SAM 3 recorta a pessoa e o VLM a descreve.", "Usa la selección de este plano (🎯 Puntos… en el cuerpo, no solo en la cara, para describir la ropa; o el texto de máscara): SAM 3 recorta a la persona y el VLM la describe.", "使用此镜头的选区(🎯 在人物身体上标点,不要只点脸,这样才能描述服装;或使用蒙版文本):SAM 3 抠出人物,VLM 生成描述。"],
"Skip this shot": ["Ignorar este plano", "Omitir este plano", "跳过此镜头"], "unparsed answer:": ["resposta não interpretada:", "respuesta sin procesar:", "未解析的回答:"],
"previous shot as reference": ["plano anterior como referência", "plano anterior como referencia", "将上一镜头作为参考图"], "previous shot as first frame": ["plano anterior como primeiro frame", "plano anterior como primer fotograma", "将上一镜头作为首帧"], "previous shot's end as history (long take)": ["final do plano anterior como histórico (plano contínuo)", "final del plano anterior como historial (toma continua)", "将上一镜头的结尾作为历史(连续长镜头)"],
"its first frame": ["seu primeiro frame", "su primer fotograma", "其首帧"], "its last frame": ["seu último frame", "su último fotograma", "其末帧"], "its middle frame": ["seu frame do meio", "su fotograma central", "其中间帧"],
"Pick a video in the Video tab.": ["Escolha um vídeo na aba Vídeo.", "Elige un vídeo en la pestaña Vídeo.", "请在 Video 标签页中选择视频。"],
"✓ copied to {…} shot(s)": ["✓ copiado para {…} plano(s)", "✓ copiado a {…} plano(s)", "✓ 已复制到 {…} 个镜头"], "show mask previews": ["mostrar prévias das máscaras", "mostrar vistas previas de las máscaras", "显示蒙版预览"],
"max total seconds": ["duração total máxima", "duración total máxima", "总时长上限"],
"Remove every reference picture from this plan (global, every shot) and forget the recent list?": ["Remover todas as imagens de referência deste plano (global e de todos os planos) e limpar a lista recente?", "¿Eliminar todas las imágenes de referencia de este plan (globales y de todos los planos) y borrar la lista reciente?", "移除此计划中的所有参考图(全局及各镜头)并清空最近列表?"],
"Colored Mark": ["Marcação colorida", "Marca de color", "彩色标记"], "Default Colored Mark for this global reference": ["Cor padrão da Marcação colorida para esta referência global", "Color predeterminado de Marca de color para esta referencia global", "此全局参考图的默认彩色标记颜色"],
"write {details} in any prompt: each shot gets the description of its own references": ["Use {details} em qualquer prompt para inserir a descrição das referências de cada plano", "Escribe {details} en cualquier prompt para insertar la descripción de las referencias de cada plano", "在提示词中使用 {details},即可插入各镜头对应参考图的描述"],
"e.g. the person on the left": ["ex.: a pessoa à esquerda", "p. ej., la persona de la izquierda", "例如:左侧的人物"], "Keeps movement and expression visible.": ["Mantém os movimentos e as expressões visíveis.", "Mantiene visibles el movimiento y la expresión.", "保留动作和表情的可见度。"],
"What to segment, in English: a short phrase (noun + one or two traits), e.g. 'woman in pink top'; commas for several things. Enough on its own - no points needed: SAM 3 finds and tracks it through the shot (every match, up to Max objects). Press Enter to preview. Points, when set, take priority.": ["O que segmentar, em inglês: uma frase curta (substantivo e uma ou duas características), por exemplo, 'woman in pink top'; use vírgulas para vários objetos. Não precisa de pontos: o SAM 3 encontra e rastreia o objeto no plano (até Máx. objetos). Pressione Enter para pré-visualizar. Os pontos têm prioridade.", "Qué segmentar, en inglés: una frase breve (sustantivo y uno o dos rasgos), p. ej., 'woman in pink top'; usa comas para varios objetos. No hacen falta puntos: SAM 3 lo encuentra y sigue durante el plano (hasta Máx. objetos). Pulsa Enter para previsualizar. Los puntos tienen prioridad.", "用英文描述要分割的对象:简短短语(名词加一两个特征),例如“woman in pink top”;多个对象用逗号分隔。无需标记点,SAM 3 会在镜头中查找并跟踪(最多为最大对象数)。按 Enter 预览。设置了标记点时以标记点为准。"],
"Remove this shot's mask (points, text, mask video) and mode - for shots where it picked the wrong thing": ["Remover a máscara (pontos, texto ou vídeo) e o modo deste plano — útil quando a seleção está errada", "Quitar la máscara (puntos, texto o vídeo) y el modo de este plano; útil si seleccionó el objeto equivocado", "移除此镜头的蒙版(点、文本或蒙版视频)及模式;适用于选错对象时"],
"A black/white video (white = the subject), e.g. rotoscoped in another tool. It replaces SAM 3 for this shot and covers only this shot: its first frame is the shot's first frame.": ["Vídeo em preto e branco (branco = sujeito), por exemplo, rotoscopiado em outra ferramenta. Substitui o SAM 3 apenas neste plano; o primeiro frame do vídeo corresponde ao primeiro frame do plano.", "Vídeo en blanco y negro (blanco = sujeto), por ejemplo, rotoscopiado en otra herramienta. Sustituye SAM 3 solo en este plano; su primer fotograma corresponde al primer fotograma del plano.", "黑白蒙版视频(白色表示主体),例如在其他工具中制作的 rotoscoping。它仅在此镜头替代 SAM 3,视频首帧对应镜头首帧。"],
"— none: SAM 3 (text / points) —": ["— nenhum: SAM 3 (texto / pontos) —", "— ninguno: SAM 3 (texto / puntos) —", "— 无:SAM 3(文本/标记点)—"], "— none: the global mask video (People / Filters / Masks) —": ["— nenhum: vídeo de máscara global (Pessoas / Filtros / Máscaras) —", "— ninguno: vídeo de máscara global (Personas / Filtros / Máscaras) —", "— 无:全局蒙版视频(人物/筛选/蒙版)—"],
"upload a mask video for this shot": ["enviar um vídeo de máscara para este plano", "subir un vídeo de máscara para este plano", "为此镜头上传蒙版视频"], "back to SAM 3 / the global mask": ["voltar ao SAM 3 / máscara global", "volver a SAM 3 / máscara global", "切换回 SAM 3/全局蒙版"],
"This shot uses its mask video: the text and points are ignored.": ["Este plano usa o vídeo de máscara; o texto e os pontos são ignorados.", "Este plano usa su vídeo de máscara; se ignoran el texto y los puntos.", "此镜头使用自己的蒙版视频,文本和标记点将被忽略。"],
"No mask of its own: this shot uses the global mask video (People / Filters / Masks tab).": ["Sem máscara própria: este plano usa o vídeo de máscara global (aba Pessoas / Filtros / Máscaras).", "Sin máscara propia: este plano usa el vídeo de máscara global (pestaña Personas / Filtros / Máscaras).", "没有单独蒙版:此镜头使用全局蒙版视频(People / Filters / Masks 标签页)。"],
"No mask of its own: this shot uses its part of the whole-video track (People / Filters / Masks tab).": ["Sem máscara própria: este plano usa sua parte do rastreamento do vídeo inteiro (aba Pessoas / Filtros / Máscaras).", "Sin máscara propia: este plano usa su parte del seguimiento de todo el vídeo (pestaña Personas / Filtros / Máscaras).", "没有单独蒙版:此镜头使用整段视频跟踪中对应的部分(People / Filters / Masks 标签页)。"],
"Opacity of the generation mask: the value of the white inside the mask (1 = fully white, the default). H3 reads a grey mask value as 'regenerate this much': at 0.85 the masked area starts at 85% of the noise and keeps ~15% of the original there. 0.8-0.9 keeps pose, outline and lighting while still swapping; too low copies the original person.": ["Opacidade da máscara de geração: valor do branco dentro da máscara (1 = branco total, padrão). O H3 interpreta o cinza como intensidade de regeneração: em 0,85, a região começa com 85% de ruído e preserva cerca de 15% do original. Valores de 0,8–0,9 mantêm pose, contorno e iluminação; valores muito baixos copiam a pessoa original.", "Opacidad de la máscara de generación: valor del blanco dentro de la máscara (1 = blanco total, predeterminado). H3 interpreta el gris como intensidad de regeneración: con 0,85, la zona empieza con un 85 % de ruido y conserva cerca del 15 % del original. Valores de 0,8–0,9 mantienen pose, silueta e iluminación; valores demasiado bajos copian a la persona original.", "生成蒙版的不透明度:蒙版白色区域的数值(1 = 全白,为默认值)。H3 将灰度解释为重绘强度:0.85 表示区域从 85% 噪声开始,约保留 15% 原图。0.8–0.9 可保留姿势、轮廓和光线;过低会复制原人物。"],
"keeps ~{…}% of the original inside the mask": ["preserva ~{…}% do original dentro da máscara", "conserva ~{…}% del original dentro de la máscara", "保留蒙版内约 {…}% 的原图"], "regenerates the masked area completely": ["regenera totalmente a região mascarada", "regenera por completo la zona enmascarada", "完全重绘蒙版区域"],
"set BFS Shot H3 Conditioning → inpaint to ": ["configure BFS Shot H3 Conditioning → inpaint como ", "configura BFS Shot H3 Conditioning → inpaint como ", "将 BFS Shot H3 Conditioning → inpaint 设置为 "],
"its outline on the result is pasted too, so a bigger silhouette is not cut": ["o contorno também é colado no resultado para não cortar uma silhueta maior", "también se pega su silueta en el resultado para no recortar una figura más grande", "结果中也会贴回其轮廓,避免裁掉更大的主体"],
"who is replaced, e.g. the young woman in a pink crop top (empty = the person)": ["quem será substituído, por exemplo, a jovem de top rosa (vazio = a pessoa)", "quién se sustituye, p. ej., la joven con top rosa (vacío = la persona)", "要替换谁,例如穿粉色短上衣的年轻女性(留空 = 人物)"],
"copy (the {shot} placeholder in the prompt gets it automatically at run time)": ["copiar (o marcador {shot} do prompt recebe isso durante a execução)", "copiar (el marcador {shot} del prompt lo inserta automáticamente al ejecutar)", "复制(运行时会自动填入提示词中的 {shot})"],
"SAM 3 mask (text or points), crop, {target}": ["Máscara SAM 3 (texto ou pontos), recorte, {target}", "Máscara SAM 3 (texto o puntos), recorte, {target}", "SAM 3 蒙版(文本或点)、裁剪、{target}"],
"How much of the previous result's end goes before this shot. 5 frames is a short context; longer windows carry more motion and use more memory.": ["Quantos frames finais do resultado anterior são colocados antes deste plano. 5 frames dão pouco contexto; janelas maiores carregam mais movimento e usam mais memória.", "Cuántos fotogramas finales del resultado anterior se colocan antes de este plano. 5 fotogramas dan poco contexto; ventanas más largas conservan más movimiento y usan más memoria.", "将上一结果的多少末尾帧放在此镜头之前。5 帧是较短上下文;更长窗口保留更多动作,也占用更多内存。"],
"Which frame of the previous shot's result": ["Qual frame do resultado do plano anterior", "Qué fotograma del resultado del plano anterior", "上一镜头结果中的哪一帧"],
"the first shot uses only its references": ["o primeiro plano usa apenas suas referências", "el primer plano solo usa sus referencias", "第一个镜头仅使用自己的参考图"],
"⚠ a camera cut starts this shot: history is skipped (it only carries a continuing take)": ["⚠ este plano começa após um corte de câmera: o histórico é ignorado (ele só funciona em uma tomada contínua)", "⚠ este plano empieza tras un corte de cámara: se omite el historial (solo sirve para una toma continua)", "⚠ 此镜头从镜头切换开始:跳过历史帧(历史帧只适用于连续长镜头)"],
"the end of the PREVIOUS shot's result continues into this one (queue loop: shots run in order). Works with every mask mode, crop and the duet canvas.": ["o final do resultado do plano ANTERIOR continua neste plano (loop em fila: planos executados em ordem). Funciona com todos os modos de máscara, recorte e tela duet.", "el final del resultado del plano ANTERIOR continúa en este (bucle en cola: los planos se ejecutan en orden). Funciona con todos los modos de máscara, recorte y lienzo duet.", "上一镜头结果的末尾延续到此镜头(队列循环按顺序运行)。支持所有蒙版模式、裁剪和 duet 画布。"],
"uses a frame of the PREVIOUS shot's result (queue loop, or auto loop with BFS Shot H3 Duet)": ["usa um frame do resultado do plano ANTERIOR (loop em fila ou loop automático com BFS Shot H3 Duet)", "usa un fotograma del resultado del plano ANTERIOR (bucle en cola o automático con BFS Shot H3 Duet)", "使用上一镜头结果中的一帧(队列循环,或搭配 BFS Shot H3 Duet 的自动循环)"],
"the points, on each shot's frame at the same relative position (a subject that stays in place)": ["os pontos em cada plano, na mesma posição relativa do frame (para um sujeito que permanece no lugar)", "los puntos en cada plano, en la misma posición relativa del fotograma (para un sujeto que no se desplaza)", "在每个镜头相同相对位置的帧上放置这些点(适用于位置不变的主体)"],
"the shot's mask video (it covers only its own shot: usually you want a different one per shot)": ["o vídeo de máscara do plano (cobre apenas este plano; normalmente cada plano precisa de um vídeo diferente)", "el vídeo de máscara del plano (solo cubre este plano; normalmente se usa uno distinto por plano)", "此镜头的蒙版视频(仅覆盖此镜头;通常每个镜头需要不同的视频)"],
"the generation mode (Full frame / Frame + paste / Mask only / Crop / Crop + mask) and its mask strength (shots with a mask)": ["o modo de geração (Quadro inteiro / Quadro + colar / Somente máscara / Recorte / Recorte + máscara) e a força da máscara", "el modo de generación (Fotograma completo / Fotograma + pegar / Solo máscara / Recorte / Recorte + máscara) y la intensidad de máscara", "生成模式(整帧/整帧+贴回/仅蒙版/裁剪/裁剪+蒙版)及蒙版强度"],
"the {target} description": ["a descrição de {target}", "la descripción de {target}", "{target} 的描述"], "continuity (not on the first shot)": ["continuidade (exceto no primeiro plano)", "continuidad (excepto en el primer plano)", "连续性(第一个镜头除外)"],
"shots after this": ["planos depois deste", "planos posteriores a este", "此镜头之后的镜头"], "replaces the old “→ all” buttons": ["substitui os antigos botões “→ todos”", "sustituye los antiguos botones “→ todos”", "替代旧的“→ 全部”按钮"],
"remove (use the global reference)": ["remover (usar a referência global)", "quitar (usar la referencia global)", "移除(使用全局参考图)"], "previous shot (←)": ["plano anterior (←)", "plano anterior (←)", "上一镜头(←)"],
"Override the content filters for this shot": ["Substituir os filtros de conteúdo neste plano", "Anular los filtros de contenido para este plano", "覆盖此镜头的内容筛选设置"],
"Colored Mark keeps two independent point sets; switch regions to edit each person. Both use the frame selected below.": ["Com Marcação colorida, cada pessoa tem seu próprio conjunto de pontos. Alterne as regiões para editar cada pessoa; ambas usam o frame selecionado abaixo.", "Marca de color mantiene un conjunto de puntos independiente por persona. Cambia de región para editar cada una; ambas usan el fotograma seleccionado abajo.", "彩色标记为每个人保留独立的点集。切换区域即可编辑;两组都使用下方选定的帧。"],
"Describe second region": ["Descreva a segunda região", "Describe la segunda región", "描述第二个区域"],
"🎯 Mark with points": ["🎯 Marcar com pontos", "🎯 Marcar con puntos", "🎯 用标记点选择"],
"🎯 Points 2": ["🎯 Pontos 2", "🎯 Puntos 2", "🎯 标记点 2"],
"Region 1 · color": ["Região 1 · cor", "Región 1 · color", "区域 1 · 颜色"],
"Region 2 · color": ["Região 2 · cor", "Región 2 · color", "区域 2 · 颜色"],
"click = keep (green) · right-click or shift+click = exclude (red) · click a point to remove it": ["clique = incluir (verde) · clique direito ou shift+clique = excluir (vermelho) · clique em um ponto para removê-lo", "clic = incluir (verde) · clic derecho o shift+clic = excluir (rojo) · clic en un punto para quitarlo", "单击 = 保留(绿色)· 右键或 shift+单击 = 排除(红色)· 单击标记点可删除"],
};
export function t(value) {
const s = String(value ?? ""), lang = uiLanguage.value;
if (lang === "en") return s;
const trimmed = s.trim(), entry = TEXT[s] || TEXT[trimmed];
if (entry) return `${s.slice(0, s.indexOf(trimmed))}${entry[{ pt: 0, es: 1, zh: 2 }[lang]]}${s.slice(s.indexOf(trimmed) + trimmed.length)}`;
let m;
if ((m = s.match(/^Person (\d+)$/))) return `${t("Person")} ${m[1]}`;
if ((m = s.match(/^Region (\d+) · color$/))) return `${lang === "pt" ? "Região" : lang === "es" ? "Región" : "区域"} ${m[1]} · ${lang === "zh" ? "颜色" : lang === "pt" ? "cor" : "color"}`;
if ((m = s.match(/^Region (\d+) \((\d+)\)$/))) return `${lang === "pt" ? "Região" : lang === "es" ? "Región" : "区域"} ${m[1]} (${m[2]})`;
if ((m = s.match(/^🎯 Points 2 \((\d+)\)$/))) return `🎯 ${lang === "pt" ? "Pontos" : lang === "es" ? "Puntos" : "标记点"} 2 (${m[1]})`;
if ((m = s.match(/^Shot #(\d+)( · points)?$/))) return `${t("Shot")} #${m[1]}${m[2] ? ` · ${lang === "pt" ? "pontos" : lang === "es" ? "puntos" : "点"}` : ""}`;
if ((m = s.match(/^(\d+) people · (\d+) linked$/))) return lang === "pt" ? `${m[1]} pessoas · ${m[2]} vinculadas` : lang === "es" ? `${m[1]} personas · ${m[2]} vinculadas` : `${m[1]} 人物 · 已关联 ${m[2]} 人`;
if ((m = s.match(/^(\d+) shot\(s\) will run$/))) return lang === "pt" ? `${m[1]} plano(s) serão executados` : lang === "es" ? `se ejecutarán ${m[1]} plano(s)` : `将运行 ${m[1]} 个镜头`;
if ((m = s.match(/^(\d+) suggestions$/))) return lang === "pt" ? `${m[1]} sugestões` : lang === "es" ? `${m[1]} sugerencias` : `${m[1]} 条建议`;
if ((m = s.match(/^(\d+) shot\(s\)$/))) return lang === "pt" ? `${m[1]} plano(s)` : lang === "es" ? `${m[1]} plano(s)` : `${m[1]} 个镜头`;
if ((m = s.match(/^(\d+) warning\(s\) · (\d+) note\(s\)$/))) return lang === "pt" ? `${m[1]} aviso(s) · ${m[2]} observação(ões)` : lang === "es" ? `${m[1]} aviso(s) · ${m[2]} nota(s)` : `${m[1]} 条警告 · ${m[2]} 条备注`;
if ((m = s.match(/^generate at (\d+)×(\d+)$/))) return lang === "pt" ? `gerar em ${m[1]}×${m[2]}` : lang === "es" ? `generar a ${m[1]}×${m[2]}` : `生成尺寸 ${m[1]}×${m[2]}`;
if ((m = s.match(/^timeline (\S+) fps · (\d+) frames$/))) return lang === "pt" ? `linha do tempo ${m[1]} fps · ${m[2]} frames` : lang === "es" ? `línea de tiempo ${m[1]} fps · ${m[2]} fotogramas` : `时间轴 ${m[1]} fps · ${m[2]} 帧`;
if ((m = s.match(/^from the start frame · (\d+) frames$/))) return lang === "pt" ? `a partir do frame inicial · ${m[1]} frames` : lang === "es" ? `desde el fotograma inicial · ${m[1]} fotogramas` : `从起始帧开始 · ${m[1]} 帧`;
if ((m = s.match(/^🔊 audio · (.+) · (\d+) ch$/))) return lang === "pt" ? `🔊 áudio · ${m[1]} · ${m[2]} canais` : lang === "es" ? `🔊 audio · ${m[1]} · ${m[2]} canales` : `🔊 音频 · ${m[1]} · ${m[2]} 声道`;
if ((m = s.match(/^🔇 (.+) → silent track$/))) return lang === "pt" ? `🔇 ${m[1]} → faixa silenciosa` : lang === "es" ? `🔇 ${m[1]} → pista silenciosa` : `🔇 ${m[1]} → 静音音轨`;
if ((m = s.match(/^Sensitivity ([\d.]+)$/))) return lang === "pt" ? `Sensibilidade ${m[1]}` : lang === "es" ? `Sensibilidad ${m[1]}` : `灵敏度 ${m[1]}`;
if ((m = s.match(/^= (\d+) frames \(([\d.]+) s\)$/))) return lang === "pt" ? `= ${m[1]} frames (${m[2]} s)` : lang === "es" ? `= ${m[1]} fotogramas (${m[2]} s)` : `= ${m[1]} 帧(${m[2]} 秒)`;
if ((m = s.match(/^generate (\d+) frames$/))) return lang === "pt" ? `gerar ${m[1]} frames` : lang === "es" ? `generar ${m[1]} fotogramas` : `生成 ${m[1]} 帧`;
if ((m = s.match(/^(\d+) too long \(> (\d+) f\)$/))) return lang === "pt" ? `${m[1]} acima do limite (> ${m[2]} frames)` : lang === "es" ? `${m[1]} demasiado largos (> ${m[2]} fotogramas)` : `${m[1]} 个镜头过长(> ${m[2]} 帧)`;
if ((m = s.match(/^boundary @ (\d+)$/))) return lang === "pt" ? `divisão no frame ${m[1]}` : lang === "es" ? `límite en el fotograma ${m[1]}` : `分界位于第 ${m[1]} 帧`;
if ((m = s.match(/^frame (\d+) · (.+)$/))) return lang === "pt" ? `frame ${m[1]} · ${m[2]}` : lang === "es" ? `fotograma ${m[1]} · ${m[2]}` : `第 ${m[1]} 帧 · ${m[2]}`;
if ((m = s.match(/^(\d+)\/(\d+) shots · ([\d.]+)s$/))) return lang === "pt" ? `${m[1]}/${m[2]} planos · ${m[3]} s` : lang === "es" ? `${m[1]}/${m[2]} planos · ${m[3]} s` : `${m[1]}/${m[2]} 个镜头 · ${m[3]} 秒`;
if ((m = s.match(/^(\d+) warning\(s\)$/))) return lang === "pt" ? `${m[1]} aviso(s)` : lang === "es" ? `${m[1]} aviso(s)` : `${m[1]} 条警告`;
if ((m = s.match(/^first seen (.+) \(click to seek\)$/))) return lang === "pt" ? `visto primeiro em ${m[1]} (clique para ir até lá)` : lang === "es" ? `primera aparición ${m[1]} (haz clic para ir)` : `首次出现于 ${m[1]}(点击跳转)`;
if ((m = s.match(/^(\d+) people · (\d+) included$/))) return lang === "pt" ? `${m[1]} pessoas · ${m[2]} incluídas` : lang === "es" ? `${m[1]} personas · ${m[2]} incluidas` : `${m[1]} 人物 · 已包含 ${m[2]} 人`;
if ((m = s.match(/^(\d+) point\(s\) at frame (\d+)$/))) return lang === "pt" ? `${m[1]} ponto(s) no frame ${m[2]}` : lang === "es" ? `${m[1]} punto(s) en el fotograma ${m[2]}` : `第 ${m[2]} 帧有 ${m[1]} 个标记点`;
if ((m = s.match(/^mask covers ([\d.]+)%(.+)$/))) return lang === "pt" ? `máscara cobre ${m[1]}%${m[2]}` : lang === "es" ? `la máscara cubre ${m[1]}%${m[2]}` : `蒙版覆盖 ${m[1]}%${m[2]}`;
if ((m = s.match(/^last (\d+) frames \(([\d.]+) s\)$/))) return lang === "pt" ? `últimos ${m[1]} frames (${m[2]} s)` : lang === "es" ? `últimos ${m[1]} fotogramas (${m[2]} s)` : `最后 ${m[1]} 帧(${m[2]} 秒)`;
if ((m = s.match(/^Mask opacity (.+)$/))) return lang === "pt" ? `Opacidade da máscara ${m[1]}` : lang === "es" ? `Opacidad de máscara ${m[1]}` : `蒙版不透明度 ${m[1]}`;
if ((m = s.match(/^Play or scrub the video \(or click the timeline\), then ⇤ \/ ⇥\. (\d+) frames \(([\d.]+) s\) of (\d+): only this part is split, analysed, generated and joined\.$/)))
return lang === "pt" ? `Reproduza ou avance o vídeo (ou clique na linha do tempo) e use ⇤ / ⇥. ${m[1]} frames (${m[2]} s) de ${m[3]}: somente este trecho será dividido, analisado, gerado e unido.` : lang === "es" ? `Reproduce o desplaza el vídeo (o haz clic en la línea de tiempo) y usa ⇤ / ⇥. ${m[1]} fotogramas (${m[2]} s) de ${m[3]}: solo se dividirá, analizará, generará y unirá este tramo.` : `播放或拖动视频(也可点击时间轴),然后按 ⇤ / ⇥。${m[1]} 帧(${m[2]} 秒),共 ${m[3]} 帧:仅拆分、分析、生成并合并此片段。`;
return s;
}
function localizeProps(props) {
if (!props || typeof props !== "object") return props;
let changed = false; const out = { ...props };
for (const k of ["title", "placeholder", "aria-label"]) if (typeof out[k] === "string") { const v = t(out[k]); changed ||= v !== out[k]; out[k] = v; }
return changed ? out : props;
}
function localizeChildren(value) {
if (typeof value === "string") return t(value);
if (Array.isArray(value)) return value.map(localizeChildren);
return value;
}
// All view modules use this h wrapper, so labels, placeholders, tooltips and section titles stay in sync.
export function h(type, props, children) {
const hasChildrenArg = arguments.length >= 3;
if (!hasChildrenArg && props?.__v_isVNode) return vueH(type, null, props);
const p = localizeProps(props);
if (hasChildrenArg) return vueH(type, p, localizeChildren(children));
if (typeof props === "string" || Array.isArray(props)) return vueH(type, localizeChildren(props));
return vueH(type, p);
}
// Use for literal labels that must keep their native spelling (for example language names).
export const rawH = (...args) => vueH(...args);
export const GRIDS = { "H3 (17n+5)": [17, 5], "LTX / Wan (8n+1)": [8, 1], "Wan (4n+1)": [4, 1], "any": [1, 0] };
export const DEFAULTS = {
video: "", fps: 24, grid: "H3 (17n+5)", mode: "shots", max_s: 4.5, min_s: 1.0, sensitivity: 0.5,
max_parts: 0, max_total_s: 0, bounds: [], segs: [], global_ref: "", global_ref2: "", global_prompt: "",
megapixels: 0.15, multiple: 32, detector: "adaptive", run: "auto", auto_continue: true,
skip_fill: "original", audio_mode: "auto", ref_details: {}, mask_cfg: {}, vlm_cfg: {}, cast: {}, cast_assign: true, cast_split: false, cast_only: false,
max_parts: 0, max_total_s: 0, start_s: 0, end_s: 0, start_f: null, end_f: null, track: null, bounds: [], segs: [], global_refs: [], global_ref: "", global_ref2: "", global_prompt: "", global_color_guide_ref1: "#00ff00", global_color_guide_ref2: "#9b30ff", mask_video: "",
megapixels: 0.5, multiple: 32, generation_size_mode: "megapixels", custom_width: 1024, custom_height: 1024,
second_pass: { enabled: false, mode: "scale", scale: 2, width: 1280, height: 704, megapixels: 1, align: 32 },
detector: "adaptive", run: "recursive", between_shots: "keep", shot_cache: "output", shot_storage: "lossless", unload_before_join: true, auto_continue: true,
skip_fill: "original", audio_mode: "auto", ref_details: {}, picture_details: {}, auto_prompt: {}, auto_prompts: {}, mask_cfg: {}, vlm_cfg: {}, cast: {}, cast_split: false, cast_only: false,
filters: { person: false, min_person_area: 0, max_persons: 0, face: false, skip_dark: false, dark_level: 0.06,
skip_static: false, static_level: 0.004, min_frames: 0, samples: 6 },
};
// global mask settings (same defaults as the server's DEFAULT_MASK)
export const MASK_DEFAULTS = { threshold: 0.5, max_objects: 4, invert: false, fill_holes: true, temporal_expand: 2, blockify: 0,
padding: 0.15, expand: 16, feather: 12, paste: "mask" };
padding: 0.15, expand: 16, feather: 12, paste: "box" };
export const VLM_DEFAULTS = { enabled: false, frames: 3, max_tokens: 1024, auto_segment: true, auto_shot: true, instruction: "",
describe_preset: "full body", describe_custom: "" };
describe_preset: "short", describe_custom: "", fit_prompt: true, temperature: 0, seed: 0,
image_max_side: 1024 };
// hover help for every setting (the 📖 Guide button opens the full documentation)
export const TIPS = {
@@ -29,10 +398,12 @@ export const TIPS = {
"Min seconds / shot": "Shorter shots merge into a neighbour when the merge still fits the maximum.",
"Timeline fps": "The planner works on this frame rate (the source is resampled). Generated video and audio use it too.",
"Max shots": "Only the first N shots (0 = all). Handy for quick tests.",
"Max total seconds": "Only the first N seconds of the video (0 = all).",
"Megapixels": "Generation size: the source aspect ratio at this pixel area (0.15 ≈ 512×288). Cropped shots use it for the crop.",
"Megapixels": "Generation size: the source aspect ratio at this pixel area (0.5 ≈ 960×544 for 16:9). Cropped shots use it for the crop.",
"Size multiple": "Width and height snap to this multiple (32 for H3 and most video models).",
"Run": "Auto loop: every shot in one queue run. Queue loop: one shot per run, stored on disk, the join outputs the video after the last shot.",
"Shot cache": "Where the generated shots are stored. Keep: ComfyUI's output folder (output/bfs_shotloop): they survive a restart, so a stopped run resumes and a single shot can be redone. Temp: ComfyUI empties it at every start.",
"Shot storage": "How each stored shot is saved. Lossless (FFV1): exact, ~12x smaller than raw frames. Smaller / Smallest (H.264 RGB): about 20x / 40x smaller with a slight loss, which reaches the final video and the continuity head. Raw: exact, biggest, needs no ffmpeg.",
"Between shots": "Keep models loaded: fastest. Free memory: after each shot, unload every model and empty the GPU, so the next shot starts clean (for low VRAM or when a later shot runs out of memory); models load again for every shot, so it is slower.",
"Run": "One run, shot by shot: one Queue renders every shot in order, storing each on disk (continuity works, resumes after a failure). Queue loop: one shot per run, the panel re-queues (keep the tab open). Auto loop: every shot at once in memory (no continuity).",
"Skipped shots in the output": "Shots that do not run: keep the original video there, or remove them (with their audio).",
"Crop padding": "Context kept around the mask's box, as a % of the box. More context blends better; less gives the model more pixels on the subject.",
"Expand": "Grow the mask by this many pixels: before pasting a cropped result back, and for the generation mask (inpaint 'only the mask'). Raise it when the new person is bigger than the old one (longer hair, wider body).",
@@ -57,7 +428,11 @@ export const snapDown = (n, grid) => {
export const hue = i => `hsl(${(i * 47 + 200) % 360} 55% 46%)`;
export const fmtT = (f, fps) => { const s = f / fps; return `${Math.floor(s / 60)}:${(s % 60).toFixed(2).padStart(5, "0")}`; };
export const segKey = s => `${s.start}-${s.end}`;
export const hasMask = s => !!(s.mask?.text || (s.mask?.points || []).length);
// a shot has a mask: its own mask video, SAM 3 text or points, or the plan's mask video for the whole video
// (the planner's `mask` input is only known when the workflow runs)
export const hasMask = s => !!(s.mask?.video || s.mask?.text || (s.mask?.points || []).length || s.extMask);
export const maskSource = s => s.mask?.video ? "video" : (s.mask?.text || (s.mask?.points || []).length) ? "sam"
: s.trackMask ? "track" : s.extMask ? "global" : "";
// ---- small view helpers
export const pill = (text, kind = "", title = "") => h("span", { class: ["pill", kind], title }, text);
@@ -72,9 +447,107 @@ export const check = (checked, onChange, label, title = "") =>
export const select = (value, options, onChange, attrs = {}) => h("select", { value, onChange: e => onChange(e.target.value), ...attrs },
options.map(o => h("option", { value: Array.isArray(o) ? o[0] : o }, Array.isArray(o) ? o[1] : o)));
// a titled section inside a card; `open` makes it a collapsible <details>
// what the user opened or closed wins over a section's default: re-renders (a timer, a status line) no longer snap a
// section back shut right after it was clicked open
const OPEN = reactive({});
export const section = (title, body, { sub = "", right = null, open = null, cls = "" } = {}) => {
const head = [h("span", { class: "sect" }, title), sub ? hint(sub) : null, h("span", { class: "grow" }), right];
return open === null
? h("div", { class: ["sec", cls] }, [h("div", { class: "sech" }, head), ...[body].flat()])
: h("details", { class: ["sec", cls], open }, [h("summary", { class: "sech" }, [h("span", { class: "caret" }, "▸"), ...head]), ...[body].flat()]);
if (open === null) return h("div", { class: ["sec", cls] }, [h("div", { class: "sech" }, head), ...[body].flat()]);
const key = `${cls}|${typeof title === "string" ? title : ""}`;
const isOpen = key in OPEN ? OPEN[key] : !!open;
return h("details", { class: ["sec", cls], open: isOpen, onToggle: e => { OPEN[key] = e.target.open; } },
[h("summary", { class: "sech" }, [h("span", { class: "caret" }, "▸"), ...head]), ...[body].flat()]);
};
// second pass size, the same arithmetic as bfs_shot_loop.second_pass_size (and Minimax H3 Latent Upscaler (3D)):
// target pixels, rounded to `align`, then to whole 16 px latent cells
export const secondPassSize = (w, h, sp) => {
if (!w || !h) return { w: 0, h: 0 };
let tw, th;
if (sp.mode === "dims") { tw = +sp.width; th = +sp.height; }
else if (sp.mode === "megapixels") { const px = +sp.megapixels * 1024 * 1024; th = Math.sqrt(px / (w / h)); tw = th * (w / h); }
else { tw = w * +sp.scale; th = h * +sp.scale; }
const al = Math.max(1, Math.round(+sp.align || 1)), r = x => Math.round(x / al) * al, c16 = x => Math.max(16, Math.round(x / 16) * 16);
return { w: c16(r(tw)), h: c16(r(th)) };
};
// ---- reference pictures (<Picture 1..12>)
export const MAX_REFS = 12;
// a reference list of the plan: `refs` (plans from 1.113 on), else the old ref / ref2 pair; empty slots at the end dropped
export const refList = (refs, a = "", b = "") => {
const out = (Array.isArray(refs) ? refs : [a, b]).map(x => (typeof x === "string" ? x : "")).slice(0, MAX_REFS);
while (out.length && !out[out.length - 1]) out.pop();
return out;
};
// the pictures a shot uses, in <Picture n> order: its own slot, else the global one at the same position; a slot with
// neither is left out (same rule as bfs_shot_loop.shot_ref_names)
export const shotRefNames = (own, glob) => {
const out = [];
for (let k = 0; k < Math.max(own.length, glob.length); k++) { const x = own[k] || glob[k]; if (x) out.push(x); }
return out.slice(0, MAX_REFS);
};
// one slot per picture, then "+" (up to 12). Click: upload (the "+" takes several files at once); drop image files or a
// recent thumbnail on a slot to replace it, anywhere else in the box to add them. `fallback`: the global pictures a
// shot's empty slots use (shown faded). `extra(k, name)`: more controls under slot k.
export function refBox(c, target, own, fallback = [], extra = () => null) {
const n = Math.max(own.length, fallback.length);
const drop = k => ({ onDragover: e => e.preventDefault(),
onDrop: e => { e.preventDefault(); e.stopPropagation(); c.dropRefs(target, k, e); } });
const slots = [];
for (let k = 0; k < n; k++) {
const name = own[k] || "", via = name ? "" : fallback[k] || "";
slots.push(h("div", { class: "rslot" }, [
h("div", { class: "refslot", ...drop(k),
title: name ? `<Picture ${k + 1}>: ${name} (click to replace)` : via ? `<Picture ${k + 1}>: ${via} (global; click to use another picture in this shot)` : "click to upload",
onClick: () => c.uploadRef(target, k) },
name || via ? [h("img", { src: c.viewUrl(name || via), style: via ? "opacity:.45" : "" }), h("span", { class: "rnum" }, String(k + 1))]
: [`⬆ ${k + 1}`]),
h("div", { class: "rbtns" }, [name ? h("button", { title: target === "global" || k >= fallback.length ? "remove this picture"
: "remove (use the global picture)", onClick: () => c.setRef(target, k, "") }, "✕") : null]),
extra(k, name || via),
]));
}
if (n < MAX_REFS) slots.push(h("div", { class: "rslot" }, [
h("div", { class: "refslot add", ...drop(n), title: `add pictures (up to ${MAX_REFS}): click to upload one or more, or drop image files here`,
onClick: () => c.uploadRef(target, n, true) }, "+"),
h("div", { class: "rbtns" }, []),
]));
return h("div", { class: "refbox", ...drop(null) }, slots);
}
// chips under a prompt, one per picture: PictureN inserts <Picture N> at the cursor, 📝 inserts {details@Picture N}
// (the description of that picture alone). The prompt box and its chips sit in one .pbox.
export function promptChips(c, names, onSet) {
if (!names.length) return null;
const insert = (e, tag) => {
e.preventDefault(); // mousedown: the prompt keeps its focus and cursor
const ta = e.currentTarget.closest(".pbox")?.querySelector("textarea"); if (!ta) return;
const v = ta.value, a = ta.selectionStart ?? v.length, b = ta.selectionEnd ?? v.length;
const before = v.slice(0, a) + (a && !/\s$/.test(v.slice(0, a)) ? " " : "") + tag;
onSet(before + v.slice(b));
requestAnimationFrame(() => { ta.focus(); ta.setSelectionRange(before.length, before.length); });
};
return h("div", { class: "pchips" }, names.map((name, k) => h("span", { class: "pchip" }, [
h("button", { class: "sm", title: `insert <Picture ${k + 1}> (${name})`, onMousedown: e => insert(e, `<Picture ${k + 1}>`) },
[h("img", { src: c.viewUrl(name) }), `Picture${k + 1}`]),
h("button", { class: "sm", title: `insert {details@Picture ${k + 1}}: the description of this picture (📝 Describe refs, or the VLM at run time)`,
onMousedown: e => insert(e, `{details@Picture ${k + 1}}`) }, "📝"),
])));
}
// ---- Colored Mark: one color per picture (<Picture 1> first; the first seven are the colors of the first mark sets)
export const MARK_PALETTE = [["#00ff00", "green"], ["#9b30ff", "purple"], ["#ff2020", "red"], ["#168cff", "blue"],
["#ffe600", "yellow"], ["#ff00d4", "magenta"], ["#00e5c0", "cyan"], ["#ff8000", "orange"], ["#ffffff", "white"],
["#8b4513", "brown"], ["#ff9ec4", "pink"], ["#808000", "olive"]];
export const markName = color => (MARK_PALETTE.find(([v]) => v === String(color).toLowerCase()) || [0, color])[1];
// the mark color of <Picture k> (same rule as bfs_shot_loop.picture_color)
export const pictureColor = (plan, k) => {
const c = Array.isArray(plan.picture_colors) ? plan.picture_colors[k - 1] : "";
return /^#[0-9a-f]{6}$/i.test(c || "") ? c.toLowerCase() : MARK_PALETTE[(k - 1) % MARK_PALETTE.length][0];
};
// a shot's extra Colored Mark regions (the second person on): plans before 1.113 had one (text2 / points2 = picture 2)
export const markRegions = m => Array.isArray(m.color_guide_regions) ? m.color_guide_regions
: (m.color_guide_text2 || (m.color_guide_points2 || []).length)
? [{ picture: 2, text: m.color_guide_text2 || "", points: m.color_guide_points2 || [], key: m.color_guide_key2 || 0 }] : [];
// ---- 🤖 Auto prompt: the VLM writes each shot's H3 prompt from the request in the prompt box
export const AUTO_DEFAULTS = { enabled: false, frames: 0, source: "guide", style: "h3" };
+35 -4
View File
@@ -97,6 +97,10 @@ const CSS = `
.bsl .hdl{position:absolute;top:-70px;bottom:0;width:9px;margin-left:-4px;cursor:ew-resize;z-index:3}
.bsl .hdl::after{content:"";position:absolute;left:3px;top:0;bottom:0;width:3px;background:#fff;opacity:.85;border-radius:2px;box-shadow:0 0 4px #000}
.bsl .hdl:hover::after{background:#ffd34d}
.bsl .rgrid{display:grid;grid-template-columns:auto 130px auto 130px auto;gap:6px 10px;align-items:start;justify-content:start}
.bsl .rgrid>button{margin-top:15px}
.bsl .rgrid .fld .hint{margin-top:2px}
.bsl .outr{position:absolute;top:0;bottom:0;background:repeating-linear-gradient(45deg,#000a 0 6px,#0007 6px 12px);z-index:2;pointer-events:none}
.bsl .cut{position:absolute;top:0;height:64px;width:2px;background:#ff4d6d;opacity:.9;pointer-events:none}
.bsl .ph{position:absolute;top:0;bottom:0;width:1px;background:#ffd34d;pointer-events:none;z-index:4}
.bsl .playhead{position:absolute;top:0;bottom:0;width:2px;background:#ffd34d;box-shadow:0 0 6px #ffd34d;pointer-events:none;z-index:5}
@@ -136,14 +140,24 @@ const CSS = `
.bsl .refslot:hover{border-color:var(--acc)}
.bsl .refslot.sm{width:56px;height:56px;font-size:10px}
.bsl .refslot.via{border-style:solid;border-color:#3b5b3b}
.bsl .refslot{position:relative}
.bsl .refslot .rnum{position:absolute;left:3px;top:2px;font-size:10px;padding:0 4px;border-radius:4px;background:#000a;color:#fff}
.bsl .refslot.add{font-size:22px;color:#8a8a99}
.bsl .refbox{flex-wrap:wrap;max-width:100%;align-items:flex-start}
.bsl .pchips{display:flex;gap:4px;flex-wrap:wrap;margin-top:4px}
.bsl .pchip{display:inline-flex;gap:1px}
.bsl .pchip button{display:inline-flex;align-items:center;gap:3px;padding:1px 5px;font-size:10px}
.bsl .pchip img{width:16px;height:16px;border-radius:3px;object-fit:cover}
.bsl .rslot{display:flex;flex-direction:column;gap:3px;align-items:center}
.bsl .guidecolor{display:flex;flex-direction:column;align-items:center;gap:2px;max-width:112px}
.bsl .guidecolor select{width:100%;font-size:10px;padding:2px 4px}
.bsl .rbtns{display:flex;gap:3px;min-height:20px}
.bsl .rbtns button{padding:1px 6px;font-size:10px}
.bsl .recent{display:flex;gap:4px;flex-wrap:wrap;margin-top:6px;align-items:center}
.bsl .recent img{width:30px;height:30px;border-radius:5px;object-fit:cover;cursor:pointer;border:1px solid #3a3a45}
.bsl .recent img:hover{border-color:var(--acc)}
.bsl .recent.sm img{width:22px;height:22px}
.bsl .modes{display:grid;grid-template-columns:repeat(4,minmax(0,1fr));gap:6px;margin-bottom:6px}
.bsl .modes{display:grid;grid-template-columns:repeat(5,minmax(0,1fr));gap:6px;margin-bottom:6px}
.bsl .mode{border:1px solid var(--line2);border-radius:7px;padding:6px 8px;cursor:pointer;background:#18181d}
.bsl .mode:hover{border-color:#55556a}
.bsl .mode.on{border-color:var(--acc);background:#1a2034}
@@ -176,14 +190,31 @@ const CSS = `
.bsl .ck .lvl{font-size:12px}
.bsl .prog{height:7px;border-radius:4px;background:#26262e;overflow:hidden}
.bsl .prog>div{height:100%;background:linear-gradient(90deg,#3a63e0,#7fe0a8)}
.bsl .langpick{display:flex;gap:5px;align-items:center}
.bsl .langpick select{width:auto;min-width:100px;padding:4px 6px}
.bsl .tourback{position:fixed;inset:0;z-index:10001;background:rgba(0,0,0,.68);display:flex;align-items:center;justify-content:center;padding:16px}
.bsl .tourbox{width:min(480px,94vw);background:#1d1d23;border:1px solid #4a4a58;border-radius:12px;padding:16px;box-shadow:0 12px 48px #0009}
.bsl .tourbox h3{margin:18px 0 8px;color:var(--hi)}
.bsl .tourbox p{line-height:1.55}
/* points window (teleported to <body>) */
.bsl .mmodal{position:fixed;inset:0;background:rgba(0,0,0,.72);z-index:10000;display:flex;align-items:center;justify-content:center}
.bsl .mbox{background:#1d1d23;border:1px solid #3a3a45;border-radius:10px;padding:12px;width:min(960px,94vw);max-height:94vh;overflow:auto}
.bsl .mimg{position:relative;cursor:crosshair;user-select:none;line-height:0}
.bsl .mimg img{width:100%;border-radius:6px}
.bsl .mbox{background:#1d1d23;border:1px solid #3a3a45;border-radius:10px;padding:12px;width:min(960px,94vw);height:min(900px,94vh);overflow:auto;box-sizing:border-box}
/* the frame fits a fixed stage whatever the video's size or orientation; points are stored relative to the frame */
.bsl .mstage{display:flex;justify-content:center;align-items:center;height:min(540px,56vh);background:#0d0d10;border-radius:6px}
.bsl .mimg{position:relative;cursor:crosshair;user-select:none;line-height:0;display:inline-block}
.bsl .mimg img{display:block;max-width:100%;max-height:min(540px,56vh);border-radius:6px}
.bsl .mthumb{width:22px;height:22px;border-radius:4px;object-fit:cover;vertical-align:middle;border:2px solid var(--mark,#555)}
.bsl .pt{position:absolute;width:14px;height:14px;margin:-7px 0 0 -7px;border-radius:50%;border:2px solid #fff;cursor:pointer}
.bsl .pt.pos{background:#3ccf6b}.bsl .pt.neg{background:#e8455a}
.bsl .pt.cg.pos{background:var(--point-color,#00ff00)}
.bsl .pt.other{opacity:.5}
.bsl .kslider{position:relative;flex:1;min-width:160px;padding-bottom:8px}
.bsl .kmarks{position:absolute;left:8px;right:8px;bottom:0;height:8px}
.bsl .kmark{position:absolute;width:8px;height:8px;margin-left:-4px;border-radius:50%;background:var(--mark,#3ccf6b);opacity:.55;cursor:pointer;border:1px solid #000}
.bsl .kmark.on{opacity:1}
.bsl .kchip{display:inline-flex;gap:1px}.bsl .kchip.on button:first-child{border-color:var(--acc)}
.bsl .mreg{display:flex;gap:6px;align-items:center;flex-wrap:wrap;margin-top:6px}
`;
export function styles() {
+119 -21
View File
@@ -1,6 +1,6 @@
// The busy bar and the points window (🎯 Points…: click on a frame to pick the subject for SAM 3).
import { h, Teleport } from "../vendor/vue.esm-browser.prod.mjs";
import { hint } from "./common.js";
import { Teleport } from "../vendor/vue.esm-browser.prod.mjs";
import { h, hint, t, pictureColor, markName } from "./common.js";
export function busyBar(c, msg) {
const { busySince, now, upPct, status } = c;
@@ -18,42 +18,140 @@ export function pointsModal(c) {
const { modal, segs, plan, api } = c;
const ms = segs.value[modal.idx];
if (!modal.open || !ms) return null;
// Colored Mark: region 1 = the shot's own selection, 2.. = its extra regions; each one is a picture, in its color
const regionPicture = r => (r === 1 ? ms.color_guide_picture : ms.regions[r - 2]?.picture) || 1;
const regionColor = r => (ms.color_guide ? pictureColor(plan, regionPicture(r)) : "#3ccf6b");
const regionsN = ms.color_guide ? 1 + ms.regions.length : 1;
const regions = Array.from({ length: regionsN }, (_, q) => q + 1);
// the points of a region on the frame shown (the one being edited are in modal.points)
const pointsAt = r => (r === modal.region ? modal.points : modal.groups[r]?.keys.find(k => k.key === modal.key)?.points || []);
const keysOf = r => {
const ks = new Set((modal.groups[r]?.keys || []).map(k => k.key));
if (r === modal.region) { if (modal.points.length) ks.add(modal.key); else ks.delete(modal.key); }
return [...ks].sort((a, b) => a - b);
};
const edited = () => { modal.stale = !!modal.prev; };
const pointOverlays = regions.flatMap(r => pointsAt(r).map((p, k) => h("div", {
class: ["pt", ms.color_guide && "cg", r !== modal.region && "other", p.label ? "pos" : "neg"],
style: `left:${p.x * 100}%;top:${p.y * 100}%;--point-color:${regionColor(r)}`,
title: r === modal.region ? "click to remove" : `Picture ${regionPicture(r)}`,
onPointerdown: e => { if (r !== modal.region) return; e.stopPropagation(); modal.points = modal.points.filter((_, j) => j !== k); edited(); },
})));
const regionButton = r => {
const k = regionPicture(r), name = ms.names[k - 1];
return h("button", { class: modal.region === r ? "pri" : "", onClick: () => c.switchPointRegion(r),
title: `Region ${r}: becomes <Picture ${k}> (${markName(regionColor(r))})` }, [
name ? h("img", { class: "mthumb", src: c.viewUrl(name), style: `--mark:${regionColor(r)}` }) : null,
` Picture ${k} · ${markName(regionColor(r))} (${keysOf(r).length} key frame${keysOf(r).length === 1 ? "" : "s"})`]);
};
// after Segment: the mask over every frame of the shot (modal.prev.frames, one per frame)
const masked = modal.prev && modal.showMask ? modal.prev.frames.find(f => f.f === ms.start + modal.key) : null;
const play = () => {
if (modal.playing) { clearInterval(modal.playing); modal.playing = 0; return; }
modal.playing = setInterval(() => {
if (modal.key >= ms.len - 1 || !modal.open) { clearInterval(modal.playing); modal.playing = 0; return; }
c.showFrame(modal.key + 1);
}, 1000 / Math.min(24, plan.fps || 24));
};
const marks = h("div", { class: "kmarks" }, regions.flatMap(r => keysOf(r).map(f => h("span", {
class: ["kmark", r === modal.region && "on"], style: `left:${(f / Math.max(1, ms.len - 1)) * 100}%;--mark:${regionColor(r)}`,
title: `key frame ${ms.start + f}`, onClick: () => c.showFrame(f) }))));
return h(Teleport, { to: "body" }, h("div", { class: "bsl", style: "background:none;padding:0" }, h("div", { class: "mmodal",
onKeydown: e => e.stopPropagation(), onPointerdown: e => { if (e.target === e.currentTarget) modal.open = false; } }, [
h("div", { class: "mbox" }, [
onKeydown: e => {
e.stopPropagation();
if (e.target.tagName === "INPUT" && e.target.type === "text") return;
if (e.key === "ArrowLeft") c.showFrame(modal.key - 1);
if (e.key === "ArrowRight") c.showFrame(modal.key + 1);
}, onPointerdown: e => { if (e.target === e.currentTarget) modal.open = false; } }, [
h("div", { class: "mbox", tabindex: 0 }, [
h("div", { class: "row", style: "margin-bottom:6px" }, [h("span", { class: "sect" }, `Shot #${modal.idx + 1} · points`),
hint("click = keep (green) · right-click or shift+click = exclude (red) · click a point to remove it")]),
ms.color_guide ? h("div", { class: "row" }, regions.map(regionButton)) : null,
hint("click = keep · right-click or shift+click = exclude · click a point to remove it · points on another frame = a new key frame: the mask is tracked from each key frame up to the next one")]),
h("div", { class: "mstage" }, [
h("div", { class: "mimg", onContextmenu: e => e.preventDefault(), onPointerdown: e => {
const box = e.currentTarget.getBoundingClientRect();
const x = (e.clientX - box.left) / box.width, y = (e.clientY - box.top) / box.height;
modal.points = [...modal.points, { x: +x.toFixed(4), y: +y.toFixed(4), label: (e.button === 2 || e.shiftKey) ? 0 : 1 }];
modal.prev = null;
edited();
} }, [
h("img", { draggable: false, src: api.apiURL(`/bfs/shotloop/frame?video=${encodeURIComponent(plan.video)}&fps=${plan.fps}&f=${ms.start + modal.key}&w=900`) }),
...modal.points.map((p, k) => h("div", { class: ["pt", p.label ? "pos" : "neg"], style: `left:${p.x * 100}%;top:${p.y * 100}%`,
title: "click to remove", onPointerdown: e => { e.stopPropagation(); modal.points = modal.points.filter((_, j) => j !== k); modal.prev = null; } })),
]),
h("div", { class: "row", style: "margin-top:6px" }, [
h("img", { draggable: false, src: masked ? masked.src : api.apiURL(`/bfs/shotloop/frame?video=${encodeURIComponent(plan.video)}&fps=${plan.fps}&f=${ms.start + modal.key}&w=900`) }),
...pointOverlays,
])]),
h("div", { class: "row", style: "margin-top:6px;align-items:center" }, [
h("button", { title: "play the shot (with the mask after Segment)", onClick: play }, modal.playing ? "⏸" : "▶"),
h("button", { title: "previous frame (←)", onClick: () => c.showFrame(modal.key - 1) }, "◀"),
h("button", { title: "next frame (→)", onClick: () => c.showFrame(modal.key + 1) }, "▶|"),
hint(`frame ${ms.start + modal.key} (${modal.key + 1}/${ms.len})`),
h("input", { type: "range", min: 0, max: ms.len - 1, step: 1, value: modal.key, style: "flex:1",
onInput: e => { modal.key = parseInt(e.target.value); modal.prev = null; } }),
h("div", { class: "kslider" }, [
h("input", { type: "range", min: 0, max: ms.len - 1, step: 1, value: modal.key, style: "width:100%",
onInput: e => c.showFrame(parseInt(e.target.value)) }),
marks,
]),
modal.prev ? h("label", { class: "row", style: "gap:4px" }, [h("input", { type: "checkbox", checked: modal.showMask,
onChange: e => { modal.showMask = e.target.checked; } }), "mask"]) : null,
]),
h("div", { class: "row", style: "margin-top:4px;gap:4px;flex-wrap:wrap" }, [
hint("key frames:"),
...(keysOf(modal.region).length ? keysOf(modal.region).map(f => h("span", { class: ["kchip", f === modal.key && "on"] }, [
h("button", { class: "sm", onClick: () => c.showFrame(f) }, `frame ${ms.start + f}`),
h("button", { class: "sm dng", title: "remove this key frame's points", onClick: () => { c.dropKeyFrame(f); edited(); } }, "✕")]))
: [hint("none yet: click on the frame to add points")]),
]),
h("div", { class: "row", style: "margin-top:6px" }, [
h("input", { type: "text", value: modal.text, style: "flex:1", placeholder: "text prompt (used when there are no points)",
onChange: e => { modal.text = e.target.value; } }),
h("button", { onClick: () => { modal.points = []; modal.prev = null; } }, "Clear points"),
h("button", { class: "pri", disabled: !!modal.busy || !(modal.points.length || modal.text),
onClick: () => c.previewMask(modal.idx, { points: modal.points, key: modal.key, text: modal.text }, true) }, "👁 Segment"),
onChange: e => { modal.text = e.target.value; edited(); } }),
h("button", { title: "remove the points of this frame", onClick: () => { modal.points = []; edited(); } }, "Clear frame"),
h("button", { class: "pri", disabled: !!modal.busy || !(keysOf(modal.region).length || modal.text),
onClick: () => c.previewMask(modal.idx, { ...c.pointSpec(), ...(modal.region > 1 ? { max_objects: 1 } : {}),
preview_color: regionColor(modal.region) }, true) }, modal.prev && modal.stale ? "👁 Segment again" : "👁 Segment"),
]),
modal.busy ? busyBar(c, modal.busy) : null,
modal.prev ? h("div", { class: "mstrip" }, [...modal.prev.frames.map(f => h("img", { src: f.src, title: `frame ${f.f}` })),
hint(modal.prev.empty ? "nothing found" : `covers ${(modal.prev.coverage * 100).toFixed(1)}%`)]) : null,
modal.prev ? hint(modal.prev.empty ? "nothing found" : `covers ${(modal.prev.coverage * 100).toFixed(1)}% of the frames`
+ (modal.stale ? " · the points changed: Segment again to see the new mask" : " · play or scrub to check every frame; where it slips, add points on that frame"),
"display:block;margin-top:4px") : null,
h("div", { class: "row", style: "margin-top:8px;justify-content:flex-end" }, [
h("button", { onClick: () => { modal.open = false; } }, "Cancel"),
h("button", { title: "Apply these points to EVERY shot, each on its frame at the same relative position (a subject that stays in place)",
onClick: c.savePointsAll }, "Save → all shots"),
!ms.color_guide ? h("button", { title: "Apply this frame's points to EVERY shot, each on its frame at the same relative position (a subject that stays in place)",
onClick: c.savePointsAll }, "Save → all shots") : null,
h("button", { disabled: !modal.points.length || ms.color_guide,
title: "Track this frame's selection with SAM 3 through the whole video (or the range), forward and backward from this frame; every shot without its own selection takes its part of the track. Also type a short text (e.g. 'woman in pink top') so the subject is found again after camera cuts.",
onClick: c.saveTrack }, "🎯 Track → whole video"),
h("button", { class: "pri", onClick: c.savePoints }, "Save (this shot)"),
]),
]),
])));
}
const TOUR_STEPS = [
["video", "Video", "Choose a video, set the frame range, split at camera cuts, then set the generation size."],
["shots", "Shots", "Review each shot, assign its references and prompt, and choose how the result is applied."],
["people", "People", "Track detected face identities to filter or split shots. Set masks here or per shot."],
["prompts", "Prompts", "Set shared references, prompts and optional VLM assistance. Written prompts are kept exactly as authored."],
["shots", "Colored Mark", "Colored Mark adds a translucent color overlay to a selected region in the source video. It does not enable inpainting. Choose the color before marking the region with points or a description."],
["run", "Run", "Check warnings, choose auto or queue loop, and set filters or quick-run limits."],
];
export function tourModal(c) {
const { tour, tab } = c;
const step = tour.value - 1;
if (step < 0 || step >= TOUR_STEPS.length) return null;
const [targetTab, title, body] = TOUR_STEPS[step];
return h(Teleport, { to: "body" }, h("div", { class: "bsl", style: "background:none;padding:0" },
h("div", { class: "tourback", role: "dialog", "aria-modal": "true", "aria-label": t("Shot Planner tour"),
onKeydown: e => { if (e.key === "Escape") tour.value = 0; }, onPointerdown: e => { if (e.target === e.currentTarget) tour.value = 0; } }, [
h("div", { class: "tourbox" }, [
h("div", { class: "row" }, [h("b", t("Shot Planner tour")), h("span", { class: "grow" }), h("span", { class: "hint" }, `${step + 1} / ${TOUR_STEPS.length}`),
h("button", { class: "ghost", title: t("Close tour"), "aria-label": t("Close tour"), onClick: () => { tour.value = 0; } }, "✕")]),
h("h3", t(title)),
h("p", t(body)),
h("p", { class: "hint" }, t("This tour is optional. Use Next to visit each main tab; close it whenever you like.")),
h("div", { class: "row", style: "justify-content:flex-end;margin-top:12px" }, [
step > 0 ? h("button", { onClick: () => { tour.value--; tab.value = TOUR_STEPS[step - 1][0]; } }, t("Back")) : null,
h("button", { class: "pri", onClick: () => {
if (step === TOUR_STEPS.length - 1) tour.value = 0;
else { tour.value++; tab.value = TOUR_STEPS[step + 1][0]; }
} }, t(step === TOUR_STEPS.length - 1 ? "Close tour" : "Next")),
]),
]),
])));
}
+52 -22
View File
@@ -1,17 +1,15 @@
// 👥 People & masks tab: the cast found by face (each person linked to a reference) and the global SAM 3 mask settings.
import { h } from "../vendor/vue.esm-browser.prod.mjs";
import { fmtT, fld, check, section, pill, hint } from "./common.js";
// 👥 People / Filters / Masks tab: cast, shot filters, and global SAM 3 mask settings.
import { h, fmtT, fld, check, section, pill, hint } from "./common.js";
function castSection(c) {
const { plan, people, busy, usedRefs } = c;
const { plan, people, busy } = c;
const fps = plan.fps;
const castN = people.value.filter(p => c.linked(p.id)).length;
const castN = people.value.filter(p => c.included(p.id)).length;
const opt = (k, label, title) => check(plan[k], v => c.setCastOpt(k, v), label, title);
return section("Cast", [
h("div", { class: "row", style: "gap:14px;margin-bottom:6px" }, [
opt("cast_assign", "Shots use their main person's reference", "a shot without its own reference takes the reference of the person with the most screen time in it"),
opt("cast_split", "Split where the main person changes", "adds a boundary when the biggest face switches to someone else (re-splits the plan)"),
opt("cast_only", "Only run shots with a linked person", "shots where no linked person appears are skipped"),
opt("cast_only", "Only run shots with a detected person", "shots with no detected, non-ignored person are skipped"),
]),
people.value.length ? h("div", { class: "cast" }, people.value.map(p => {
const cc = c.castOf(p.id);
@@ -19,13 +17,6 @@ function castSection(c) {
h("img", { class: "face", src: p.thumb, title: `first seen ${fmtT(p.first, fps)} (click to seek)`, onClick: () => c.seek(p.first) }),
h("div", { class: "t" }, [h("b", `Person ${p.id}`), ` · ${(p.share * 100).toFixed(1)}%`]),
h("div", { class: "t" }, `${fmtT(p.first, fps)} → ${fmtT(p.last, fps)}`),
h("div", { class: "refbox" }, ["ref", "ref2"].map(k => h("div", { class: "rslot" }, [
h("div", { class: "refslot sm", title: cc[k] ? cc[k] + " (click to replace)" : "click to upload",
onClick: () => c.pickFile("image/*", name => c.setCast(p.id, k, name)) }, cc[k] ? [h("img", { src: c.viewUrl(cc[k]) })] : [k === "ref" ? "⬆ ref" : "⬆ ref 2"]),
cc[k] ? h("div", { class: "rbtns" }, [h("button", { title: "remove", onClick: () => c.setCast(p.id, k, "") }, "✕")]) : null,
]))),
usedRefs.value.length ? h("div", { class: "recent sm" }, usedRefs.value.map(n => h("img", { src: c.viewUrl(n), title: `${n} (click = ref, shift+click = ref 2)`,
onClick: e => c.setCast(p.id, e.shiftKey ? "ref2" : "ref", n) }))) : null,
check(cc.ignore, v => c.setCast(p.id, "ignore", v), "ignore"),
]);
})) : h("div", { class: "hint" }, "Nobody found yet."),
@@ -33,8 +24,8 @@ function castSection(c) {
h("button", { class: "pri", disabled: !!busy.value, onClick: () => c.findPeople(true) }, people.value.length ? "↻ Re-analyse people" : "👥 Find people"),
hint(people.value.length ? "faces sampled every 0.5 s and grouped by identity (InsightFace, CPU)" : "detects faces across the video and groups them into people"),
]),
], { sub: "track faces, link each person to a reference, split and assign shots by who is on screen",
right: people.value.length ? pill(`${people.value.length} people · ${castN} linked`) : pill("not analysed") });
], { sub: "track face identities to filter or split shots",
right: people.value.length ? pill(`${people.value.length} people · ${castN} included`) : pill("not analysed") });
}
function maskSection(c) {
@@ -43,13 +34,26 @@ function maskSection(c) {
const mnum = (k, step, label, help, scale = 1) => fld(label, h("input", { type: "number", step, min: 0, value: +(MC[k] * scale).toFixed(3),
onChange: e => c.setMaskCfg(k, (parseFloat(e.target.value) || 0) / scale) }), help);
const S = segs.value;
const counts = { mask: 0, crop: 0, cropmask: 0 };
const counts = { mask: 0, crop: 0, cropmask: 0, paste: 0 };
S.forEach(s => { const m = c.modeOf(s); if (m in counts) counts[m]++; });
const masked = S.filter(s => s.mask?.text || (s.mask?.points || []).length).length;
const masked = S.filter(s => s.mask?.text || (s.mask?.points || []).length || s.mask?.video || s.extMask).length;
return section("Mask & crop settings", [
h("div", { class: "note", style: "margin-bottom:8px" }, [
"Each shot picks what it marks (🎯 text or points) and how it is generated (", h("b", "Full frame · Mask only · Crop · Crop + mask"),
") in the Shots tab. These settings apply to every shot."]),
"Each shot picks what it marks (🎯 text or points, or 🎞 its own mask video) and how it is generated (", h("b", "Full frame · Frame + paste · Mask only · Crop · Crop + mask"),
") in the Shots tab. Masks from a video (here, per shot, or the planner's ", h("b", "mask"), " input) replace SAM 3. These settings apply to every shot."]),
h("div", { class: "row", style: "margin-bottom:8px;align-items:center" }, [
h("span", { style: "white-space:nowrap", title: "A black/white video of the WHOLE source video (white = the subject), e.g. rotoscoped in another tool. Shots without their own mask (video, text or points) use it instead of SAM 3. The planner's optional mask input does the same from the graph and wins over this." }, "🎞 Mask video (whole video)"),
h("select", { value: plan.mask_video || "", style: "flex:1;min-width:180px", onChange: e => c.setGlobalMaskVideo(e.target.value) },
[h("option", { value: "" }, "— none: SAM 3 per shot —"), ...c.files.videos.map(v => h("option", { value: v }, v))]),
h("button", { onClick: () => c.pickFile("video/*", c.setGlobalMaskVideo) }, "⬆ Upload"),
plan.mask_video ? h("button", { onClick: () => c.setGlobalMaskVideo("") }, "✕") : null,
]),
plan.track?.points?.length ? h("div", { class: "note", style: "margin-bottom:8px;display:flex;gap:8px;align-items:center" }, [
h("span", { class: "grow" }, [h("b", "🎯 Whole-video track: "), `${plan.track.points.length} point(s) at frame ${plan.track.frame}`,
plan.track.text ? ` · re-found by "${plan.track.text}" after cuts` : " · no text: after a cut the subject may be lost (add one in 🎯 Points…)",
". Shots without their own selection use it."]),
h("button", { class: "ghost", onClick: c.clearTrack }, "✕ remove"),
]) : null,
h("div", { class: "row", style: "gap:14px;margin-bottom:6px" }, [
check(MC.fill_holes, v => c.setMaskCfg("fill_holes", v), "Fill holes", "Fill enclosed gaps so each region is solid"),
check(MC.invert, v => c.setMaskCfg("invert", v), "Invert", "Use everything except the segmented object"),
@@ -68,10 +72,36 @@ function maskSection(c) {
]),
hint("Uses the official sam3.1_multiplex_fp16 checkpoint (models/checkpoints), downloaded from Comfy-Org/sam3.1 the first time.", "display:block;margin-top:6px"),
], { sub: "SAM 3",
right: h("span", { class: "row" }, [pill(`${masked} masked`), counts.mask ? pill(`${counts.mask} mask only`, "acc") : null,
right: h("span", { class: "row" }, [pill(`${masked} masked`), counts.mask ? pill(`${counts.mask} mask only`, "acc") : null, counts.paste ? pill(`${counts.paste} frame+paste`, "acc") : null,
counts.crop + counts.cropmask ? pill(`${counts.crop + counts.cropmask} cropped`, "warn") : null]) });
}
function filtersSection(c) {
const { plan, stats, segs, busy, filtersOn } = c;
const F = plan.filters;
const chk = (k, label) => check(F[k], v => c.setFilter(k, v), label);
const fnum = (k, step, label, help) => fld(label, h("input", { type: "number", step, min: 0, value: F[k], onChange: e => c.setFilter(k, parseFloat(e.target.value) || 0) }), help);
const skippedN = segs.value.filter(s => c.skipWhy(s)).length;
return section("Shot filters", [
h("div", { class: "row", style: "gap:14px;margin-bottom:6px" }, [
chk("person", "Needs a person"), chk("face", "Needs a face"), chk("skip_dark", "Skip dark / fades"), chk("skip_static", "Skip static shots"),
]),
h("div", { class: "grid" }, [
fnum("min_person_area", 0.01, "Min person size (0-1 of frame)", "e.g. 0.03 skips wide shots"),
fnum("max_persons", 1, "Max people (0 = any)", "skip crowds"),
fnum("min_frames", 1, "Min frames (0 = off)"),
fnum("samples", 1, "Frames sampled / shot"),
fnum("dark_level", 0.01, "Dark below (0-1)"),
fnum("static_level", 0.001, "Static below"),
]),
h("div", { class: "row", style: "margin-top:8px" }, [
h("button", { class: "pri", disabled: !!busy.value, onClick: c.analyzeContent }, "👤 Analyse people & faces"),
hint(stats.value.length ? `stats for ${stats.value.length} shots · YOLO person/face from models/ultralytics` : "runs the detectors on a few frames of every shot"),
]),
], { open: filtersOn.value || stats.value.length > 0, sub: "choose which shots run",
right: filtersOn.value ? pill(`${skippedN} skipped`, "warn") : pill("off") });
}
export function peopleTab(c) {
return h("div", { class: "card" }, [castSection(c), maskSection(c)]);
return h("div", { class: "card" }, [castSection(c), filtersSection(c), maskSection(c)]);
}
+90 -28
View File
@@ -1,31 +1,37 @@
// 📝 Prompts & refs tab: the global reference / prompt, the VLM (shot suggestions) and the reference descriptions ({details}).
import { h } from "../vendor/vue.esm-browser.prod.mjs";
import { fld, check, section, pill, hint, select } from "./common.js";
import { h, fld, check, section, pill, hint, select, refBox, promptChips, MAX_REFS, MARK_PALETTE, pictureColor } from "./common.js";
const PRESETS = ["short", "object", "full body", "head / face", "face attributes", "outfit", "custom"];
function globalSection(c) {
const { plan } = c;
const slot = (name, label, field) => h("div", { class: "rslot" }, [
h("div", { class: "refslot", title: name ? name + " (click to replace)" : "click to upload", onClick: () => c.uploadRef("global", field) },
name ? [h("img", { src: c.viewUrl(name) })] : [label]),
h("div", { class: "rbtns" }, [name ? h("button", { title: "remove", onClick: () => c.setRef("global", field, "") }, "✕") : null]),
// Colored Mark: every picture has its mark color (a person marked in it becomes that picture)
const marker = k => h("label", { class: "guidecolor" }, [
h("span", { class: "hint" }, "Mark color"),
h("select", { value: pictureColor(plan, k + 1), title: `Colored Mark color of <Picture ${k + 1}>`,
onChange: e => c.setPictureColor(k + 1, e.target.value) },
MARK_PALETTE.map(([v, name]) => h("option", { value: v }, name))),
]);
return section("Global reference & prompt", [
h("div", { class: "row", style: "align-items:flex-start;gap:10px" }, [
h("div", { class: "refbox" }, [slot(plan.global_ref, "⬆ global\nreference", "global_ref"), slot(plan.global_ref2, "⬆ global\nref 2", "global_ref2")]),
h("div", { style: "flex:1;min-width:200px" }, [
h("textarea", { style: "min-height:78px", placeholder: "Global prompt (a connected `prompt` input overrides it)",
value: plan.global_prompt, onChange: e => c.setPlan("global_prompt", e.target.value) }),
]),
refBox(c, "global", plan.global_refs, [], marker),
h("div", { class: "pbox", style: "margin-top:6px" }, [
h("textarea", { style: "min-height:78px", placeholder: plan.auto_prompt?.enabled
? "Request for every shot (🤖 Auto prompt writes the H3 prompt from it), e.g. 'replace the bottle with the red can from Picture 2'"
: "Global prompt (a connected `prompt` input overrides it)",
value: plan.global_prompt, onChange: e => c.setPlan("global_prompt", e.target.value) }),
promptChips(c, plan.global_refs.filter(Boolean), v => c.setPlan("global_prompt", v)),
]),
h("div", { class: "hint", style: "margin-top:4px" }, [
"Used by shots without their own. Connected ref_image / ref_image_2 / prompt inputs on the node override these. Placeholders: ",
`Used by shots without their own (up to ${MAX_REFS} pictures, <Picture 1> first; click + or drop image files on the box). A connected prompt input on the node overrides the global prompt. Placeholders: `,
h("b", "{target}"), " who is replaced (per shot) · ", h("b", "{details}"), " description of the shot's references · ",
h("b", "{details@Picture n}"), " description of picture n alone · ",
h("b", "{shot}"), " the VLM's description of the shot · ", h("b", "{setting}"), " the setting picture (H3 Conditioning)."]),
h("div", { class: "hint", style: "margin-top:4px" }, "Mark color: the Colored Mark color of each picture. A person marked for <Picture n> in a shot is drawn in that color, and the prompt says which picture the color becomes. Change it when it is too close to a color in the video."),
], { sub: "defaults for every shot" });
}
function vlmSection(c) {
const { plan, vlmSug, busy, refSets } = c;
const { plan, vlmSug, busy, pictures } = c;
const VC = plan.vlm_cfg || {};
const opt = (k, label, title) => check(VC[k], v => c.setVlmCfg(k, v), label, title);
const sugN = Object.keys(vlmSug.value).length;
@@ -39,6 +45,9 @@ function vlmSection(c) {
h("div", { class: "grid" }, [
fld("Frames per shot", h("input", { type: "number", min: 1, max: 8, step: 1, value: VC.frames, onChange: e => c.setVlmCfg("frames", parseInt(e.target.value) || 3) })),
fld("Max tokens", h("input", { type: "number", min: 64, max: 2048, step: 32, value: VC.max_tokens, onChange: e => c.setVlmCfg("max_tokens", parseInt(e.target.value) || 1024) })),
fld("Image max side (VLM)", select(String(VC.image_max_side || 1024),
[448, 672, 896, 1024, 1280, 1536, 2048].map(n => [String(n), `${n} px`]),
v => c.setVlmCfg("image_max_side", parseInt(v) || 1024)), "VLM image sizing"),
]),
h("textarea", { style: "margin-top:6px;min-height:44px", placeholder: "Extra instruction for the VLM (optional), e.g. 'segment the woman, not the man'",
value: VC.instruction || "", onChange: e => c.setVlmCfg("instruction", e.target.value) }),
@@ -51,26 +60,79 @@ function vlmSection(c) {
right: h("span", { class: "row" }, [VC.enabled ? pill("on", "ok") : pill("off"), sugN ? pill(`${sugN} suggestions`) : null]) }),
section("Describe references ({details})", [
h("div", { class: "row", style: "gap:8px" }, [
select(VC.describe_preset, ["full body", "head / face", "face attributes", "outfit", "custom"], v => c.setVlmCfg("describe_preset", v), { style: "width:auto" }),
h("button", { class: "pri", disabled: !!busy.value || !refSets.value.length, onClick: c.describeRefs }, "📝 Describe refs"),
hint(`${refSets.value.length} reference set(s) · editable below`),
fld("Describe", select(VC.describe_preset, PRESETS, v => c.setVlmCfg("describe_preset", v), { style: "width:auto" }),
"what every picture gets, unless it has its own below"),
h("button", { class: "pri", disabled: !!busy.value || !pictures.value.length, onClick: () => c.describePictures() }, "📝 Describe refs"),
h("button", { disabled: !!busy.value || !pictures.value.length,
title: "Try other descriptions: a new seed (and some temperature if it is 0), then describe again",
onClick: () => { if (!(+VC.temperature > 0)) c.setVlmCfg("temperature", 0.7); c.setVlmCfg("seed", (+VC.seed || 0) + 1); c.describePictures(); } }, "🎲"),
fld("Temperature", h("input", { type: "number", min: 0, max: 1.5, step: 0.1, value: VC.temperature ?? 0, style: "width:70px",
title: "0 = always the same answer (greedy); 0.5-1 = varied answers, picked by the seed",
onChange: e => c.setVlmCfg("temperature", Math.max(0, parseFloat(e.target.value) || 0)) })),
fld("Seed", h("input", { type: "number", min: 0, step: 1, value: VC.seed ?? 0, style: "width:80px",
title: "Used when the temperature is above 0: another seed, another description",
onChange: e => c.setVlmCfg("seed", Math.max(0, parseInt(e.target.value) || 0)) })),
hint(`${pictures.value.length} picture(s) · editable below`),
check(VC.fit_prompt !== false, v => c.setVlmCfg("fit_prompt", v), "fit the prompt",
"The VLM gets the global prompt's sentence about that picture ({details}, {details2} or {details@Picture n}; and the shot prompt's {target} sentence for 🧑 Describe target) with a blank in its place, and writes the description to fit it: the description adapts to the prompt, not the other way round."),
]),
VC.describe_preset === "custom" ? h("textarea", { style: "margin-top:6px;min-height:52px", value: VC.describe_custom || "",
placeholder: "Your instruction for the VLM, e.g. 'Describe the character's costume and props piece by piece…'",
onChange: e => c.setVlmCfg("describe_custom", e.target.value) }) : null,
refSets.value.length ? h("div", { class: "dlist" }, refSets.value.map(x => h("div", { class: "drow" }, [
h("div", { class: "dthumbs" }, [x.a ? h("img", { src: c.viewUrl(x.a) }) : null, x.b ? h("img", { src: c.viewUrl(x.b) }) : null]),
h("div", { style: "flex:1" }, [
hint(x.where.length > 4 ? `${x.where.slice(0, 4).join(", ")} +${x.where.length - 4}` : x.where.join(", ")),
h("textarea", { style: "min-height:44px", value: plan.ref_details?.[x.key] || "",
placeholder: "no description yet: Describe refs, write your own, or leave it to the VLM at run time",
onChange: e => c.setDetail(x.key, e.target.value) }),
]),
]))) : hint("Add a reference (global, per shot or per person) first."),
], { sub: "write {details} in any prompt: each shot gets the description of its own references" }),
pictures.value.length ? h("div", { class: "dlist" }, pictures.value.map(x => {
const own = plan.picture_cfg?.[x.name] || {};
return h("div", { class: "drow" }, [
h("div", { class: "dthumbs" }, [h("img", { src: c.viewUrl(x.name) })]),
h("div", { style: "flex:1" }, [
h("div", { class: "row", style: "gap:6px;margin-bottom:3px" }, [
hint(`Picture n in ${x.where.length > 4 ? `${x.where.slice(0, 4).join(", ")} +${x.where.length - 4}` : x.where.join(", ")}`),
h("span", { class: "grow" }),
select(own.preset || "", [["", `default (${VC.describe_preset || "short"})`], ...PRESETS],
v => c.setPictureCfg(x.name, "preset", v), { style: "width:auto", title: "What to describe in this picture" }),
h("button", { class: "sm", disabled: !!busy.value, title: "Describe only this picture again",
onClick: () => c.describePictures([x.name]) }, "🔄"),
]),
own.preset === "custom" ? h("textarea", { style: "min-height:36px;margin-bottom:3px", value: own.custom || "",
placeholder: "Instruction for this picture, e.g. 'Describe only the outfit, piece by piece'",
onChange: e => c.setPictureCfg(x.name, "custom", e.target.value) }) : null,
h("textarea", { style: "min-height:44px", value: plan.picture_details?.[x.name] || "",
placeholder: "no description yet: 📝 Describe refs, 🔄, write your own, or leave it to the VLM at run time",
onChange: e => c.setPictureDetail(x.name, e.target.value) }),
]),
]);
})) : hint("Add a reference (global or per shot) first."),
], { sub: "{details} = picture 1, {details2} = picture 2, {details@Picture n} = picture n: each shot gets the descriptions of its own pictures" }),
];
}
// 🤖 Auto prompt: the VLM writes each shot's prompt in H3's full-reference format from key frames of the shot, its
// pictures and their descriptions, the request in the prompt box and the shot's marks and mode
function autoSection(c) {
const { plan, busy, segs } = c;
const A = plan.auto_prompt || {};
const S = segs.value, missing = S.filter(s => !c.writtenOf(s)?.text).length;
const stale = S.filter(s => c.writtenOf(s)?.text && c.writtenOf(s).for !== c.autoFor(s)).length;
return section("🤖 Auto prompt (H3)", [
h("div", { class: "row", style: "gap:12px;align-items:center" }, [
check(A.enabled, v => c.setAuto("enabled", v), "Write the prompts with the VLM",
"On: the prompt boxes (global and per shot) hold your request ('change the bottle into a red can', 'Picture 1 dances in the rain'); the VLM writes each shot's MiniMax H3 prompt from key frames of the shot, the pictures and their descriptions, the request and the shot's Colored Marks, selection and mode, in H3's official full-reference format."),
fld("Style", select(A.style || "h3", [["h3", "H3 official (six sections)"], ["captions", "OpenJutsu captions (short)"]],
v => c.setAuto("style", v), { style: "width:auto" }), "captions: 'Replace the ... in the video with ... from <Picture 1>. Keep ...', the training captions' style (OpenJutsu); a template on the prompt input wins over both"),
fld("Key frames", select(String(A.frames || 0), [["0", "auto (by length)"], ...[3, 4, 5, 6, 8, 10, 12].map(n => [String(n), String(n)])],
v => c.setAuto("frames", parseInt(v) || 0), { style: "width:auto" }), "auto: about 1.5 per second, 3 to 8"),
fld("Source video in the prompt", select(A.source || "guide", [["guide", "guide video (aligned guide)"], ["video", "<Video 1> (native reference video)"]],
v => c.setAuto("source", v), { style: "width:auto" }), "match the conditioning node's guide_mode"),
h("button", { class: "pri", disabled: !!busy.value || !S.length, onClick: () => c.writePrompts() }, "🤖 Write all prompts"),
h("button", { disabled: !!busy.value || !missing, onClick: () => c.writePrompts(S.map((s, i) => (c.writtenOf(s)?.text ? -1 : i)).filter(i => i >= 0)) },
`only the missing (${missing})`),
]),
A.enabled && A.template ? h("div", { class: "hint", style: "display:block;margin-top:4px", title: A.template },
`Template (the node's prompt input, last run): ${A.template.slice(0, 110)}${A.template.length > 110 ? "…" : ""} · the prompt boxes are the request applied to it`) : null,
hint(A.enabled ? `${S.length - missing}/${S.length} shots have a written prompt${stale ? ` · ${stale} older than their request, pictures or marks` : ""} · shots without one are written when the workflow runs (with the VLM on the planner) · edit them in each shot (Shots tab)`
: "Off: the prompt boxes are the prompts.", "display:block;margin-top:4px"),
], { sub: "request + video + pictures -> MiniMax H3 prompt", right: A.enabled ? pill("on", "ok") : pill("off") });
}
export function promptsTab(c) {
return h("div", { class: "card" }, [globalSection(c), ...vlmSection(c)]);
return h("div", { class: "card" }, [globalSection(c), autoSection(c), ...vlmSection(c)]);
}
+102 -33
View File
@@ -1,6 +1,5 @@
// ▶ Run tab: checks before running, how the loop runs (auto / queue), which shots run (filters) and test limits.
import { h } from "../vendor/vue.esm-browser.prod.mjs";
import { fld, check, section, pill, hint, select } from "./common.js";
// ▶ Run tab: checks before running and how the loop runs (auto / queue).
import { h, fld, check, section, pill, hint, select, DEFAULTS, secondPassSize } from "./common.js";
function checksSection(c) {
const { checks, tab } = c;
@@ -23,53 +22,123 @@ function checksSection(c) {
function loopSection(c) {
const { plan, prog, active } = c;
const num = (k, step, min) => h("input", { type: "number", step, min, value: plan[k], onChange: e => c.setPlan(k, parseFloat(e.target.value) || 0) });
return section("Run", [
h("div", { class: "grid" }, [
fld("Run", select(plan.run, [["auto", "Auto loop (one run)"], ["queue", "Queue loop (one shot per run)"]], v => c.setPlan("run", v, c.progress))),
fld("Run", select(plan.run, [["recursive", "One run, shot by shot (recommended)"], ["queue", "Queue loop (one shot per run)"],
["auto", "Auto loop (all shots at once)"]], v => c.setPlan("run", v, c.progress))),
plan.run !== "auto" ? fld("Between shots", select(plan.between_shots || "keep", [["keep", "Keep models loaded (fastest)"],
["free", "Free memory: unload models (low VRAM)"]], v => c.setPlan("between_shots", v))) : null,
fld("Skipped shots in the output", select(plan.skip_fill, [["original", "Keep original video"], ["drop", "Remove them"]], v => c.setPlan("skip_fill", v))),
fld("Max shots (0 = all)", num("max_parts", 1, 0), "quick tests"),
fld("Max total seconds (0 = all)", num("max_total_s", 0.5, 0), "quick tests"),
]),
plan.run === "queue" ? h("div", { style: "margin-top:8px" }, [
h("div", { class: "row", style: "margin-bottom:4px" }, [h("b", "Queue loop"), h("span", { class: "grow" }),
h("div", { class: "row", style: "margin-top:6px" }, [
check(plan.unload_before_join !== false, v => c.setPlan("unload_before_join", v), "Unload models before the join",
"Once every shot is rendered, unload the models so the join (and the nodes after it) get the VRAM and RAM. Turn it off if a model runs again after BFS Shot Join (e.g. an upscale pass on the joined video): it would load again."),
]),
plan.run !== "auto" ? h("div", { style: "margin-top:8px" }, [
h("div", { class: "row", style: "margin-bottom:4px" }, [h("b", plan.run === "queue" ? "Queue loop" : "Shot by shot"), h("span", { class: "grow" }),
hint(`${prog.done}/${prog.count || active.value.length} shots done`)]),
h("div", { class: "prog" }, [h("div", { style: `width:${(100 * prog.done / Math.max(1, prog.count || active.value.length)).toFixed(1)}%` })]),
h("div", { class: "row", style: "margin-top:6px" }, [
check(plan.auto_continue, v => c.setPlan("auto_continue", v), "Auto-queue the next shot"),
plan.run === "queue" ? check(plan.auto_continue, v => c.setPlan("auto_continue", v), "Auto-queue the next shot") : null,
h("button", { onClick: () => c.progress(false) }, "↻ Status"), h("button", { class: "dng", onClick: () => c.progress(true) }, "⟲ Reset loop"),
]),
hint("Each run generates one shot and stores it. Nodes after BFS Shot Join only run on the last shot, with the full video.", "display:block;margin-top:4px"),
hint(plan.run === "queue"
? "Each run generates one shot and stores it. Nodes after BFS Shot Join only run on the last shot, with the full video."
: "One Queue renders every shot in order inside the same run: each shot is stored, then BFS Shot Join starts the next one, so continuity can read the shot before. After a failure, Queue again to resume from the stored shots.",
"display:block;margin-top:4px"),
]) : null,
], { sub: `${active.value.length} shot(s) will run` });
}
function filtersSection(c) {
const { plan, stats, segs, busy, filtersOn } = c;
const F = plan.filters;
const chk = (k, label) => check(F[k], v => c.setFilter(k, v), label);
const fnum = (k, step, label, help) => fld(label, h("input", { type: "number", step, min: 0, value: F[k], onChange: e => c.setFilter(k, parseFloat(e.target.value) || 0) }), help);
const skippedN = segs.value.filter(s => c.skipWhy(s)).length;
return section("Filters", [
h("div", { class: "row", style: "gap:14px;margin-bottom:6px" }, [
chk("person", "Needs a person"), chk("face", "Needs a face"), chk("skip_dark", "Skip dark / fades"), chk("skip_static", "Skip static shots"),
]),
function memorySection(c) {
const { memory, vram, size } = c;
const m = memory.value;
if (!m.level) return null;
const k = n => (n >= 1000 ? `${(n / 1000).toFixed(1)}k` : `${n}`);
const pct = vram.max_tokens ? Math.min(100, 100 * m.top.tok / (1.3 * vram.max_tokens)) : 0;
return section("Memory", [
h("div", { class: "grid" }, [
fnum("min_person_area", 0.01, "Min person size (0-1 of frame)", "e.g. 0.03 skips wide shots"),
fnum("max_persons", 1, "Max people (0 = any)", "skip crowds"),
fnum("min_frames", 1, "Min frames (0 = off)"),
fnum("samples", 1, "Frames sampled / shot"),
fnum("dark_level", 0.01, "Dark below (0-1)"),
fnum("static_level", 0.001, "Static below"),
fld("GPU", vram.gpu ? `${vram.gpu} · ${vram.total_gb} GB` : "—"),
fld("Biggest shot in this plan", m.top.i >= 0 ? `#${m.top.i + 1} · ${m.top.gen} frames · ${size.w}×${size.h} · ${k(m.top.tok)} tokens` : "—"),
fld("Biggest that ran here", vram.max_tokens ? `${k(vram.max_tokens)} tokens · peak ${vram.max_peak_gb} GB` : "nothing recorded yet"),
]),
h("div", { class: "row", style: "margin-top:8px" }, [
h("button", { class: "pri", disabled: !!busy.value, onClick: c.analyzeContent }, "👤 Analyse people & faces"),
hint(stats.value.length ? `stats for ${stats.value.length} shots · YOLO person/face from models/ultralytics` : "runs the detectors on a few frames of every shot"),
vram.max_tokens ? h("div", { class: "prog", style: "margin-top:6px", title: "100% = 1.3× the biggest shot that ran on this GPU" },
[h("div", { style: `width:${pct.toFixed(1)}%;${m.level === "warn" ? "background:var(--warn,#d97706)" : ""}` })]) : null,
hint({ ok: "✓ Every shot is within what already ran on this GPU.",
note: "A bit bigger than anything that ran here: it will probably fit, maybe slower.",
warn: "⚠ Much bigger than anything that ran here: it may run out of memory. Lower Megapixels or Max seconds / shot, use head 5, or set Between shots = free memory.",
info: "No shot has run on this GPU yet: after the first one, every shot is compared with what already ran here." }[m.level],
"display:block;margin-top:4px"),
], { sub: { ok: "within what ran here", note: "a bit bigger", warn: "may run out of memory", info: "no data yet" }[m.level] });
}
function cacheSection(c) {
const { plan, cache, active } = c;
if (plan.run === "auto") return null;
const n = active.value.length, ok = cache.ok || [];
const done = ok.filter(Boolean).length;
const lossy = plan.shot_storage === "smaller" || plan.shot_storage === "smallest";
return section("Stored shots", [
h("div", { class: "grid" }, [
fld("Shot cache", select(plan.shot_cache || "output", [["output", "Keep (output folder)"], ["temp", "Temp (lost when ComfyUI restarts)"]],
v => c.setPlan("shot_cache", v, c.progress))),
fld("Shot storage", select(plan.shot_storage || "lossless", [["lossless", "Lossless (FFV1, exact)"], ["smaller", "Smaller (H.264 RGB, visually lossless)"],
["smallest", "Smallest (H.264 RGB, light loss)"], ["raw", "Raw frames (.pt, biggest)"]], v => c.setPlan("shot_storage", v))),
]),
], { open: filtersOn.value || stats.value.length > 0, sub: "skipped shots do not run",
right: filtersOn.value ? pill(`${skippedN} skipped`, "warn") : pill("off") });
lossy ? hint("⚠ The final video is assembled from the stored shots (and the continuity head reads them): a lossy storage "
+ "slightly changes the output. Lossless keeps it exact.", "display:block;margin-top:4px;color:var(--warn,#d97706)") : null,
h("div", { class: "row", style: "margin-top:8px;flex-wrap:wrap;gap:4px" }, ok.length ? ok.map((g, k) => {
const i = c.segs.value.indexOf(active.value[k]);
return h("span", { class: ["pill", g ? "ok" : ""], title: g ? "stored and up to date" : "runs on the next Queue" }, [
`#${(i >= 0 ? i : k) + 1} ${g ? "💾" : "⏳"}`,
g ? h("button", { class: "chip", style: "margin-left:4px", title: "generate this shot again on the next Queue"
+ (plan.redo_only ? " (only this shot)" : " (and the shots that continue from it)"),
onClick: () => c.redo([k], !plan.redo_only) }, "↻") : null]);
}) : [hint(n ? "Nothing stored yet for this video: Queue the workflow." : "")]),
h("div", { class: "row", style: "margin-top:6px;flex-wrap:wrap;gap:6px" }, [
hint(ok.length ? `${done}/${ok.length} stored and up to date · ${cache.mb || 0} MB` : ""),
h("span", { class: "grow" }),
check(!!plan.redo_only, v => c.setPlan("redo_only", v), "↻ redoes only that shot"),
ok.length ? h("button", { title: "every stored shot runs again on the next Queue", onClick: () => c.redo(ok.map((_, k) => k), true) }, "↻ Redo all") : null,
h("button", { disabled: !done, title: "Put the stored shots together into an MP4 in the output folder now, without generating: "
+ "one shot in memory at a time, the original video where a shot is skipped or not stored yet, with the soundtrack.",
onClick: () => c.combine() }, "🎬 Combine stored shots"),
]),
cache.combined ? h("div", { class: "row", style: "margin-top:6px" }, [
hint(`✓ ${cache.combined.shots}/${cache.combined.of} shots → `),
h("a", { href: cache.combined.url, target: "_blank", style: "margin-left:4px" }, cache.combined.filename)]) : null,
hint("Each shot remembers the settings it was made with: the next Queue runs only the shots whose settings changed "
+ "(and the ones that continue from them). Changing what renders them (model, LoRA, seed, sampler, conditioning) runs every shot again; the join's own settings do not. "
+ (cache.folder ? `Folder: ${cache.folder}` : ""), "display:block;margin-top:6px"),
], { sub: ok.length ? `${done}/${ok.length} stored` : "nothing stored yet" });
}
// The planner only works out the second pass size; the graph upscales the latent (e.g. Minimax H3 Latent Upscaler
// (3D), target dimensions) and samples it again, reading the planner's second_pass / upscale_width / upscale_height
function secondPassSection(c) {
const { plan, size } = c;
const sp = { ...DEFAULTS.second_pass, ...(plan.second_pass || {}) };
const set = (k, v) => c.setPlan("second_pass", { ...sp, [k]: v });
const num = (k, step, min, max) => h("input", { type: "number", step, min, max, value: sp[k],
onChange: e => { const v = parseFloat(e.target.value); if (Number.isFinite(v)) set(k, v); } });
const out = secondPassSize(size.w, size.h, sp);
return section("Second pass (upscale)", [
h("div", { class: "row" }, [check(sp.enabled, v => set("enabled", v), "Enable the second pass")]),
h("div", { class: "grid", style: sp.enabled ? "" : "opacity:.55" }, [
fld("Size", select(sp.mode, [["scale", "Scale by multiplier"], ["dims", "Target dimensions"], ["megapixels", "Megapixels"]], v => set("mode", v))),
sp.mode === "scale" ? fld("Multiplier", num("scale", 0.05, 1, 4)) : null,
sp.mode === "dims" ? fld("Width", num("width", 8, 64, 8192)) : null,
sp.mode === "dims" ? fld("Height", num("height", 8, 64, 8192)) : null,
sp.mode === "megapixels" ? fld("Megapixels", num("megapixels", 0.1, 0.1, 16)) : null,
fld("Align (px)", num("align", 1, 1, 512), "32 recommended"),
fld("Result", size.w ? `${size.w}×${size.h} → ${out.w}×${out.h}` : "—"),
]),
hint("The planner outputs second_pass (on/off), upscale_width and upscale_height. Upscale the denoised latent of each shot "
+ "to that size (e.g. Minimax H3 Latent Upscaler (3D) in target dimensions mode) and sample it again before decoding; "
+ "use second_pass to switch that branch.", "display:block;margin-top:6px"),
], { sub: sp.enabled ? `on · ${out.w}×${out.h}` : "off" });
}
export function runTab(c) {
return h("div", { class: "card" }, [checksSection(c), loopSection(c), filtersSection(c)]);
return h("div", { class: "card" }, [checksSection(c), memorySection(c), loopSection(c), secondPassSection(c), cacheSection(c)]);
}
+144 -52
View File
@@ -1,7 +1,8 @@
// 🎬 Shots tab: one card per shot, then the selected shot's editor (preview, reference & prompt, who is replaced,
// continuity, copy to other shots).
import { h } from "../vendor/vue.esm-browser.prod.mjs";
import { hue, fmtT, segKey, hasMask, pill, hint, check, section } from "./common.js";
import { h, hue, fmtT, segKey, hasMask, maskSource, pill, hint, check, section, refBox, promptChips, pictureColor, markName,
MAX_REFS } from "./common.js";
import { h as rawH } from "../vendor/vue.esm-browser.prod.mjs";
const bdg = (text, kind = "", title = "") => h("span", { class: ["bdg", kind], title }, text);
@@ -16,29 +17,35 @@ function shotCard(c, s, i) {
h("div", { class: "row" }, [h("b", `#${i + 1}`), s.cut ? pill("cut") : null,
why ? pill("skip", "warn", why) : pill("run", "ok"), s.force !== "auto" ? pill(s.force) : null]),
h("div", { class: "badges" }, [
hasMask(s) ? bdg("🎯 " + (s.mask.text ? s.mask.text.slice(0, 14) : `${(s.mask.points || []).length} pts`), "", s.mask.text || `${(s.mask.points || []).length} points`) : null,
s.crop || s.inpaint ? bdg({ mask: "🎭 mask only", crop: "✂ crop", cropmask: "✂🎭 crop+mask" }[c.modeOf(s)], "on", c.modeName(s)) : null,
{ video: bdg("🎞 mask video", "", s.mask.video), global: bdg("🎞 global mask", "", "the plan's mask video for the whole video"),
track: bdg("🎯 track", "", "its part of the whole-video SAM 3 track"),
sam: bdg("🎯 " + (s.mask.text ? s.mask.text.slice(0, 14) : `${(s.mask.points || []).length} pts`), "", s.mask.text || `${(s.mask.points || []).length} points`) }[maskSource(s)] || null,
s.crop || s.inpaint || s.paste ? bdg({ paste: "🧩 frame+paste", mask: "🎭 mask only", crop: "✂ crop", cropmask: "✂🎭 crop+mask" }[c.modeOf(s)] + (s.inpaint && s.strength < 1 ? ` ${Number(s.strength).toFixed(2)}` : ""), "on", c.modeName(s)) : null,
s.color_guide ? bdg(`🎨 ${1 + s.regions.length}`, "on", [s.color_guide_picture, ...s.regions.map(r => r.picture)]
.map(k => `Picture ${k} · ${markName(pictureColor(c.plan, k))}`).join("\n") + ` · opacity ${s.color_guide_opacity ?? 0.35}`) : null,
s.target ? bdg("🧑 target", "on", s.target) : null,
i > 0 && s.chain !== "off" ? bdg("⛓", "on", `continues from the previous shot's ${s.chainFrame} frame as ${s.chain}`) : null,
i > 0 && s.chain !== "off" ? bdg("⛓", "on", s.chain === "history" ? `continues the previous shot's last ${s.chainHistory} frames (history)` : s.chain === "head" ? `starts with the previous shot's last ${Math.min(s.chainHistory, 39)} frames, kept as they are (head)` : `continues from the previous shot's ${s.chainFrame} frame as ${s.chain}`) : null,
{ ok: bdg("💾", "on", "stored and up to date: the next Queue does not generate it again (Run tab → Stored shots to redo it)"),
run: bdg("⏳", "", "not stored yet, or its settings changed: the next Queue generates it") }[c.cacheOf(s)] || null,
warnN ? bdg(`⚠ ${warnN}`, "warn", issues.map(x => x.text).join("\n")) : null,
]),
who ? h("div", { class: "who" }, who.people.length ? who.people.map(id => h("img", {
src: c.personOf(id)?.thumb || "", title: `Person ${id}${id === who.main ? " (main)" : ""}${c.linked(id) ? " · linked" : ""}`,
class: [id === who.main && "main", c.linked(id) && "lk"] })) : [h("span", { class: "t" }, "no faces")]) : null,
src: c.personOf(id)?.thumb || "", title: `Person ${id}${id === who.main ? " (main)" : ""}${c.included(id) ? " · included" : ""}`,
class: [id === who.main && "main", c.included(id) && "lk"] })) : [h("span", { class: "t" }, "no faces")]) : null,
st ? h("div", { class: "t", style: "margin-top:2px" },
`👤 ${st.persons} · ${(st.person_area * 100).toFixed(1)}% · 🙂 ${st.faces} · ☀ ${(st.brightness * 100).toFixed(0)}%`) : null,
why ? h("div", { class: "t", style: "color:#ffc46b" }, why) : null,
h("div", { class: "t" }, `${fmtT(s.start, fps)} → ${fmtT(s.end, fps)} · ${s.len}f → ${s.gen}f`),
h("div", { class: "thumbs" }, [
...["ref", "ref2"].map(k => {
const own = s[k], via = c.castRef(s, k), name = own || via || plan["global_" + k];
return name ? h("img", { class: "rt", src: c.viewUrl(name), title: own ? name : via ? `${name} (from the person)` : `${name} (global)`,
style: own ? "" : via ? "outline:1px dashed #8fd18f" : "opacity:.45" }) : h("div", { class: "rt ph2" }, k);
}),
...(s.names.length ? s.names.slice(0, 3).map((name, k) => h("img", { class: "rt", src: c.viewUrl(name),
title: s.refs.includes(name) ? name : `${name} (global)`, style: s.refs.includes(name) ? "" : "opacity:.45" }))
: [h("div", { class: "rt ph2" }, "ref")]),
s.names.length > 3 ? h("div", { class: "rt ph2", title: s.names.slice(3).join("\n") }, `+${s.names.length - 3}`) : null,
h("img", { class: "rt", style: "width:56px", src: showMasks.value && mp?.frames?.length
? mp.frames[Math.floor(mp.frames.length / 2)].src : c.thumbFor(s.start + Math.floor(s.len / 2)) }),
]),
h("div", { class: "p" }, s.prompt ? s.prompt : (plan.global_prompt ? "↳ global prompt" : "— no prompt —")),
s.prompt ? rawH("div", { class: "p" }, s.prompt)
: h("div", { class: "p" }, plan.global_prompt ? "↳ global prompt" : "— no prompt —"),
]);
}
@@ -67,20 +74,10 @@ function player(c, cur) {
]);
}
function refSlot(c, name, label, target, field, via = "") {
return h("div", { class: "rslot" }, [
h("div", { class: ["refslot", !name && via && "via"], title: name ? name + " (click to replace)" : via ? `uses ${via}; click to upload this shot's own` : "click to upload",
onClick: () => c.uploadRef(target, field) },
name ? [h("img", { src: c.viewUrl(name) })] : via ? [h("img", { src: c.viewUrl(via), style: "opacity:.4" })] : [label]),
h("div", { class: "rbtns" }, [name ? h("button", { title: "remove (use the person's / global one)", onClick: () => c.setRef(target, field, "") }, "✕") : null]),
]);
}
function editor(c, cur) {
const { plan, sel, segs, maskPrev, targetCrop, vlmSug, busy, usedRefs, copyOpts, COPY_FIELDS } = c;
const i = sel.value, S = segs.value, k = segKey(cur), mp = maskPrev.value[k], g = vlmSug.value[k];
const issues = c.checksFor(i);
const viaRef = f => cur[f] ? "" : (c.castRef(cur, f) || plan["global_" + f] || "");
const head = h("div", { class: "edhd" }, [
h("button", { class: "ghost", disabled: i === 0, title: "previous shot (←)", onClick: () => c.select(i - 1) }, "◀"),
@@ -101,28 +98,81 @@ function editor(c, cur) {
h("div", { class: "ck" }, [h("span", { class: "lvl" }, x.lvl === "warn" ? "⚠" : "ℹ"), h("span", x.text)]))) : null;
const refPrompt = section("Reference & prompt", [
h("div", { class: "row", style: "align-items:flex-start;gap:10px" }, [
h("div", { class: "refbox" }, [
refSlot(c, cur.ref, "⬆ reference", i, "ref", viaRef("ref")),
refSlot(c, cur.ref2, "⬆ ref 2", i, "ref2", viaRef("ref2")),
]),
h("div", { style: "flex:1;min-width:200px" }, [
h("textarea", { placeholder: "Prompt for this shot (empty = global prompt). Placeholders: {target} {details} {shot} {setting}",
value: cur.prompt, onChange: e => c.setMeta(i, "prompt", e.target.value) }),
!cur.prompt && plan.global_prompt ? hint(`↳ uses the global prompt: ${plan.global_prompt.slice(0, 120)}${plan.global_prompt.length > 120 ? "…" : ""}`) : null,
]),
refBox(c, i, cur.refs, plan.global_refs),
h("div", { class: "pbox", style: "margin-top:6px" }, [
h("textarea", { placeholder: plan.auto_prompt?.enabled
? "Request for this shot (empty = the global request); 🤖 Auto prompt writes the H3 prompt below from it"
: "Prompt for this shot (empty = global prompt). Placeholders: {target} {details} {details@Picture n} {shot} {setting}",
value: cur.prompt, onChange: e => c.setMeta(i, "prompt", e.target.value) }),
promptChips(c, cur.names, v => c.setMeta(i, "prompt", v)),
!cur.prompt && plan.global_prompt ? rawH("span", { class: "hint" }, `↳ uses the global prompt: ${plan.global_prompt.slice(0, 120)}${plan.global_prompt.length > 120 ? "…" : ""}`) : null,
]),
usedRefs.value.length ? h("div", { class: "recent" }, [hint("recent (click = ref, shift+click = ref 2):"),
...usedRefs.value.map(n => h("img", { src: c.viewUrl(n), title: n, onClick: e => c.setRef(i, e.shiftKey ? "ref2" : "ref", n) }))]) : null,
plan.auto_prompt?.enabled ? (() => {
const w = c.writtenOf(cur), old = w?.text && w.for !== c.autoFor(cur);
return h("div", { class: "note", style: "margin-top:6px;padding:6px 8px" }, [
h("div", { class: "row", style: "gap:6px;align-items:center;margin-bottom:4px" }, [
h("b", "🤖 Written prompt"), old ? pill("older than the request / pictures / marks", "warn") : null, h("span", { class: "grow" }),
h("button", { class: w?.text ? "" : "pri", disabled: !!busy.value, onClick: () => c.writePrompts([i]) }, w?.text ? "🤖 Write again" : "🤖 Write"),
w?.text ? h("button", { class: "dng", title: "forget it: the next run writes a new one", onClick: () => c.setWritten(cur, "") }, "✕") : null,
]),
h("textarea", { style: "min-height:140px;font-size:11px", value: w?.text || "",
placeholder: "not written yet: 🤖 Write, or the planner's VLM writes it when the workflow runs",
onChange: e => c.setWritten(cur, e.target.value) }),
]);
})() : null,
usedRefs.value.length ? h("div", { class: "recent" }, [hint("recent (click = picture 1, shift+click = add, or drag onto a slot):"),
...usedRefs.value.map(n => h("img", { src: c.viewUrl(n), title: n, draggable: true,
onDragstart: e => e.dataTransfer.setData("text/bfs-ref", n),
onClick: e => (e.shiftKey ? c.putRefs(i, cur.refs.length, [n]) : c.setRef(i, 0, n)) })),
h("button", { class: "sm", title: "Forget the recent pictures (the ones this plan still uses stay)", onClick: () => c.clearRecent() }, "✕ clear"),
h("button", { class: "sm dng", title: "Remove EVERY reference from this plan (global and all shots) and the recent list", onClick: () => c.clearAllRefs() }, "✕ clear all")]) : null,
h("div", { class: "row", style: "margin-top:6px" }, [
h("button", { class: "dng", title: "clear this shot's references and prompt: it uses its person's / the global ones",
onClick: () => { ["ref", "ref2", "prompt"].forEach(f => { c.meta(i)[f] = ""; }); c.save(); } }, "Use global"),
hint("empty slots use the person's reference (Cast) or the global one (Prompts & refs tab)"),
h("button", { class: "dng", title: "clear this shot's references and prompt: it uses the global ones",
onClick: () => { Object.assign(c.meta(i), { refs: [], prompt: "" }); c.save(); } }, "Use global"),
hint("empty slots use the global references (Prompts & refs tab)"),
]),
], { sub: "who the shot becomes" });
const mode = (on, title, text, onClick) => h("div", { class: ["mode", on && "on"], onClick }, [h("b", title), h("span", text)]);
const who = section("Who is replaced", [
// which picture a marked person becomes: "Picture 3 · red" with its thumbnail (the color is the picture's)
const picturePick = (k, onPick) => {
const n = Math.max(cur.names.length, k), color = pictureColor(plan, k);
return h("span", { class: "row", style: "gap:4px;align-items:center" }, [
cur.names[k - 1] ? h("img", { class: "mthumb", src: c.viewUrl(cur.names[k - 1]), style: `--mark:${color}`, title: cur.names[k - 1] })
: h("span", { class: "mthumb", style: `--mark:${color};display:inline-block;background:${color}` }),
h("select", { value: String(k), style: "width:auto", title: "The picture this person becomes; the mark is drawn in its color",
onChange: e => onPick(+e.target.value) },
Array.from({ length: Math.min(MAX_REFS, n) }, (_, q) => h("option", { value: String(q + 1) },
`Picture ${q + 1} · ${markName(pictureColor(plan, q + 1))}${cur.names[q] ? "" : " (no picture)"}`))),
]);
};
const who = section("Select region", [
h("div", { class: "row", style: "margin-top:4px;align-items:center;gap:8px" }, [
check(!!cur.color_guide, v => c.setMeta(i, "color_guide", v), "🎨 Colored Mark",
"Color each selected person in the source video with the color of the picture they become, as a translucent marker; the prompt gets 'the person marked in <color> becomes the character from <Picture n>'. It does not create or enable an inpainting mask."),
cur.color_guide ? h("span", { class: "hint" }, "this selection becomes") : null,
cur.color_guide ? picturePick(cur.color_guide_picture, k => { c.setMarkPicture(i, 1, k); if (maskPrev.value[segKey(cur)]) c.previewMask(i); }) : null,
]),
cur.color_guide ? h("div", { class: "note", style: "margin:4px 0;padding:6px 8px" }, [
...cur.regions.map((r, j) => h("div", { class: "mreg" }, [
h("span", { class: "hint", style: "white-space:nowrap" }, `Person ${j + 2}`),
picturePick(r.picture, k => c.setMarkRegion(i, j, "picture", k)),
h("input", { type: "text", value: r.text || "", style: "flex:1;min-width:160px", placeholder: "who, in English: e.g. the man on the left",
onChange: e => c.setMarkRegion(i, j, "text", e.target.value) }),
h("button", { disabled: !!busy.value || !(r.text || (r.points || []).length),
onClick: () => c.previewMask(i, { text: r.text || "", points: r.points || [], key: r.key || 0, max_objects: 1,
preview_color: pictureColor(plan, r.picture) }) }, "👁 Preview"),
h("button", { onClick: () => c.openPoints(i, j + 2) }, c.keyFramesOf(r).length ? `🎯 Points (${c.keyFramesOf(r).length} key frame${c.keyFramesOf(r).length > 1 ? "s" : ""})` : "🎯 Points…"),
h("button", { class: "dng", title: "remove this person's mark", onClick: () => c.removeMarkRegion(i)(j) }, "✕"),
])),
h("div", { class: "row", style: "margin-top:6px;align-items:center;gap:8px" }, [
h("button", { disabled: 1 + cur.regions.length >= MAX_REFS, onClick: () => c.addMarkRegion(i) }, "+ Mark another person"),
hint(cur.names.length > 1 + cur.regions.length ? "each person: a picture (its color) and a text or points"
: "add more pictures (Reference & prompt) to give each person their own"),
]),
]) : null,
cur.color_guide ? h("details", { style: "margin:4px 0" }, [h("summary", { class: "hint", style: "cursor:pointer" }, `Opacity · ${Number(cur.color_guide_opacity ?? 0.35).toFixed(2)}`),
h("div", { class: "row", style: "margin-top:6px;gap:8px;align-items:center" }, [h("input", { type: "range", min: 0.05, max: 0.8, step: 0.01, value: cur.color_guide_opacity ?? 0.35, style: "flex:1;max-width:320px", onInput: e => { c.meta(i).color_guide_opacity = parseFloat(e.target.value); }, onChange: c.save }), hint("Keeps movement and expression visible.")])]) : null,
h("div", { class: "row", style: "align-items:center" }, [
h("input", { type: "text", value: cur.mask.text || "", style: "flex:1;min-width:160px",
placeholder: "mask: what to segment, in English (woman in pink top, black dog…) · Enter = preview",
@@ -130,26 +180,55 @@ function editor(c, cur) {
onInput: e => c.setMask(i, "text", e.target.value),
onKeydown: e => { if (e.key === "Enter") { e.preventDefault(); c.setMask(i, "text", e.target.value); if (e.target.value || (cur.mask.points || []).length) c.previewMask(i); } } }),
h("button", { title: "Pick positive / negative points on a frame of this shot (more precise than text when several people look alike)",
onClick: () => c.openPoints(i) }, (cur.mask.points || []).length ? `🎯 Points (${cur.mask.points.length})` : "🎯 Points…"),
onClick: () => c.openPoints(i) }, c.keyFramesOf(cur.mask).length ? `🎯 Points (${c.keyFramesOf(cur.mask).length} key frame${c.keyFramesOf(cur.mask).length > 1 ? "s" : ""})` : "🎯 Points…"),
h("button", { class: hasMask(cur) && !mp ? "pri" : "", disabled: !!busy.value || !hasMask(cur),
title: (cur.mask.points || []).length ? "Segment this shot (the points win over the text)" : "Segment this shot with the text and show a few frames",
onClick: () => c.previewMask(i) }, "👁 Preview"),
h("button", { class: "dng", disabled: !(hasMask(cur) || cur.crop), title: "Remove this shot's segmentation (points, text, crop) - for shots where it picked the wrong thing",
h("button", { class: "dng", disabled: !(cur.mask.text || (cur.mask.points || []).length || cur.mask.video || cur.crop || cur.inpaint), title: "Remove this shot's mask (points, text, mask video) and mode - for shots where it picked the wrong thing",
onClick: () => c.clearMask(i) }, "✕ Clear"),
]),
h("div", { class: "row", style: "margin-top:6px;align-items:center" }, [
h("span", { class: "hint", style: "white-space:nowrap", title: "A black/white video (white = the subject), e.g. rotoscoped in another tool. It replaces SAM 3 for this shot and covers only this shot: its first frame is the shot's first frame." }, "🎞 mask video"),
h("select", { value: cur.mask.video || "", style: "flex:1;min-width:160px",
onChange: e => c.setMask(i, "video", e.target.value) },
[h("option", { value: "" }, cur.extMask ? "— none: the global mask video (People / Filters / Masks) —" : "— none: SAM 3 (text / points) —"),
...c.files.videos.map(v => h("option", { value: v }, v))]),
h("button", { title: "upload a mask video for this shot", onClick: () => c.pickFile("video/*", name => c.setMask(i, "video", name)) }, "⬆ Upload"),
cur.mask.video ? h("button", { title: "back to SAM 3 / the global mask", onClick: () => c.setMask(i, "video", "") }, "✕") : null,
]),
maskSource(cur) === "video" ? hint("This shot uses its mask video: the text and points are ignored.", "display:block;margin-top:2px")
: maskSource(cur) === "global" ? hint("No mask of its own: this shot uses the global mask video (People / Filters / Masks tab).", "display:block;margin-top:2px")
: maskSource(cur) === "track" ? hint("No mask of its own: this shot uses its part of the whole-video track (People / Filters / Masks tab).", "display:block;margin-top:2px") : null,
mp ? h("div", { class: "mstrip" }, [
...mp.frames.map(f => h("img", { src: f.src, title: `frame ${f.f}` })),
hint(mp.empty ? "nothing found: the shot runs without a mask" : `mask covers ${(mp.coverage * 100).toFixed(1)}%${cur.crop ? " · yellow = crop box" : ""}`),
]) : null,
h("div", { class: "modes", style: "margin-top:8px" }, [
["frame", "Full frame", "The whole frame is regenerated. The mask, if any, only feeds {target} / the setting picture."],
["paste", "🧩 Frame + paste", "The whole frame is generated (the pose follows the guide, the new subject may be bigger); BFS Shot Join pastes only the subject onto the original: background exact."],
["mask", "🎭 Mask only", "Only the mask is regenerated, on the whole frame: the rest stays exactly as it was. No crop / uncrop."],
["crop", "✂ Crop", "A box around the mask is regenerated (more pixels for a small person); BFS Shot Join pastes it back."],
["crop", "✂ Crop", "A box around the mask is regenerated (more pixels for a small subject); BFS Shot Join pastes it back."],
["cropmask", "✂🎭 Crop + mask", "Inside the crop, only the mask is regenerated: the most detail with the background kept."],
].map(([k, t, d]) => mode(c.modeOf(cur) === k, t, d, () => c.setMode(i, k)))),
(cur.crop || cur.inpaint) && !hasMask(cur) ? h("div", { class: "err" }, `${c.modeName(cur)} needs a mask on this shot: add a mask text or points.`) : null,
cur.color_guide && !hasMask(cur) ? h("div", { class: "err" }, "Colored Mark needs a region selection: enter a description, add points, or choose a mask video, then preview it.") : null,
(cur.crop || cur.inpaint || cur.paste) && !hasMask(cur) ? h("div", { class: "err" }, `${c.modeName(cur)} needs a mask on this shot: a mask text, points or a mask video (or the planner's mask input).`) : null,
cur.inpaint ? h("div", { class: "row", style: "margin-bottom:6px;align-items:center",
title: "Opacity of the generation mask: the value of the white inside the mask (1 = fully white, the default). H3 reads a grey mask value as 'regenerate this much': at 0.85 the masked area starts at 85% of the noise and keeps ~15% of the original there. 0.8-0.9 keeps pose, outline and lighting while still swapping; too low copies the original person." }, [
h("span", { style: "white-space:nowrap" }, `Mask opacity ${Number(cur.strength).toFixed(2)}`),
h("input", { type: "range", min: 0.3, max: 1, step: 0.01, value: cur.strength, style: "flex:1;max-width:320px",
onInput: e => { c.meta(i).strength = parseFloat(e.target.value); }, onChange: c.save }),
h("button", { class: "ghost", disabled: cur.strength >= 1, onClick: () => c.setMeta(i, "strength", 1) }, "reset"),
hint(cur.strength >= 1 ? "regenerates the masked area completely" : `keeps ~${Math.round(100 - cur.strength * 100)}% of the original inside the mask`),
]) : null,
cur.inpaint ? h("div", { class: "note", style: "margin-bottom:6px" }, [h("b", "Mask only / Crop + mask: "),
"set BFS Shot H3 Conditioning → inpaint to ", h("b", "per shot (planner)"), " (the default). Raise Expand (People & masks tab) when the new person is bigger than the old one."]) : null,
"set BFS Shot H3 Conditioning → inpaint to ", h("b", "per shot (planner)"), " (the default). Raise Expand (People / Filters / Masks tab) when the new subject is bigger than the old one."]) : null,
cur.paste ? h("div", { class: "row", style: "margin-bottom:6px;align-items:center",
title: "What the NEW subject is, in English, for SAM 3 on the result: BFS Shot Join also pastes its outline (when it is bigger than the old one). E.g. person, dog, cat, car. Empty = person." }, [
h("span", { style: "white-space:nowrap" }, "new subject"),
h("input", { type: "text", value: cur.pasteText || "", placeholder: "person (or dog, cat, car… what the result shows)", style: "flex:1;max-width:420px",
onChange: e => c.setMeta(i, "paste_text", e.target.value) }),
hint("its outline on the result is pasted too, so a bigger silhouette is not cut"),
]) : null,
h("div", { class: "row", style: "margin-top:4px;align-items:center" }, [
targetCrop.value[k] ? h("img", { src: targetCrop.value[k], style: "height:56px;border-radius:6px" }) : null,
h("span", { class: "hint", style: "white-space:nowrap" }, "{target}"),
@@ -173,22 +252,35 @@ function editor(c, cur) {
const cont = section("Continuity", h("div", { class: "row" }, [
h("select", { value: cur.chain, style: "width:auto", disabled: i === 0,
title: "reference: the previous result's frame becomes one more <Picture n> after this shot's own references. first frame: it is anchored at frame 0 of this shot.",
title: "reference: the previous result's frame becomes one more <Picture n> after this shot's own references. first frame: it is anchored at frame 0 of this shot. history: the END of the previous result (5/22/39/56 frames) goes just before this shot on H3's timeline, so a long take continues its motion and look (same take only, not across a cut). head: the END of the previous result (5/22/39 frames) becomes the first frames of this shot, kept exactly as they are (not swapped again), and the new frames continue from them; the join cuts them off (same take only, H3 grid).",
onChange: e => c.setMeta(i, "chain", e.target.value) },
[h("option", { value: "off" }, "off"), h("option", { value: "reference" }, "previous shot as reference"),
h("option", { value: "first frame" }, "previous shot as first frame")]),
h("select", { value: cur.chainFrame, style: "width:auto", disabled: i === 0 || cur.chain === "off",
title: "Which frame of the previous shot's result", onChange: e => c.setMeta(i, "chain_frame", e.target.value) },
[h("option", { value: "first" }, "its first frame"), h("option", { value: "middle" }, "its middle frame"), h("option", { value: "last" }, "its last frame")]),
hint(i === 0 ? "the first shot uses only its references" : "uses a frame of the PREVIOUS shot's result (queue loop, or auto loop with BFS Shot H3 Duet)"),
h("option", { value: "first frame" }, "previous shot as first frame"),
h("option", { value: "history" }, "previous shot's end as history (long take)"),
h("option", { value: "head" }, "previous shot's end as this shot's start (kept, long take)")]),
cur.chain === "history" || cur.chain === "head"
? h("select", { value: String(cur.chainHistory), style: "width:auto", disabled: i === 0,
title: "How much of the previous result's end goes before this shot. 5 frames is a short context; longer windows carry more motion and use more memory.",
onChange: e => c.setMeta(i, "chain_history", Number(e.target.value)) },
(cur.chain === "head" ? [5, 22, 39] : [5, 22, 39, 56]).map(n => h("option", { value: String(n) }, `last ${n} frames (${(n / 24).toFixed(1)} s)`)))
: h("select", { value: cur.chainFrame, style: "width:auto", disabled: i === 0 || cur.chain === "off",
title: "Which frame of the previous shot's result", onChange: e => c.setMeta(i, "chain_frame", e.target.value) },
[h("option", { value: "first" }, "its first frame"), h("option", { value: "middle" }, "its middle frame"), h("option", { value: "last" }, "its last frame")]),
hint(i === 0 ? "the first shot uses only its references"
: cur.chain === "head" ? (cur.cut ? "⚠ a camera cut starts this shot: head is skipped (it only carries a continuing take)"
: "this shot is generated from earlier: its first frames are the PREVIOUS shot's result, pinned in the latent, and the join cuts them (queue loop: shots run in order). Costs those frames of generation per shot.")
: cur.chain === "history" ? (cur.cut ? "⚠ a camera cut starts this shot: history is skipped (it only carries a continuing take)"
: "the end of the PREVIOUS shot's result continues into this one (queue loop: shots run in order). Works with every mask mode, crop and the duet canvas.")
: "uses a frame of the PREVIOUS shot's result (queue loop, or auto loop with BFS Shot H3 Duet)"),
]), { open: cur.chain !== "off", sub: cur.chain !== "off" ? cur.chain : "off" });
const copy = section("Copy to other shots", [
h("div", { class: "copy" }, Object.entries(COPY_FIELDS).map(([f, label]) =>
check(copyOpts[f], v => { copyOpts[f] = v; }, label, {
ref: "reference + ref 2", prompt: "this shot's prompt", mask_text: "the mask text (shots keep their own points)",
ref: "every reference picture", prompt: "this shot's prompt", mask_text: "the mask text (shots keep their own points)",
points: "the points, on each shot's frame at the same relative position (a subject that stays in place)",
crop: "the generation mode: Full frame / Mask only / Crop / Crop + mask (shots with a mask)", target: "the {target} description",
mask_video: "the shot's mask video (it covers only its own shot: usually you want a different one per shot)",
crop: "the generation mode (Full frame / Frame + paste / Mask only / Crop / Crop + mask) and its mask strength (shots with a mask)", target: "the {target} description",
chain: "continuity (not on the first shot)" }[f]))),
h("div", { class: "row", style: "margin-top:6px" }, [
h("div", { class: "seg-btns" }, [["all", "all shots"], ["after", "shots after this"]].map(([v, l]) =>
+4 -2
View File
@@ -1,6 +1,5 @@
// Timeline: thumbnails, cut marks, cut score, shot bars (click = select, drag the white handles, double-click = split).
import { h } from "../vendor/vue.esm-browser.prod.mjs";
import { hue, fmtT, pill } from "./common.js";
import { h, hue, fmtT, pill } from "./common.js";
export function timelineCard(c) {
const { plan, an, segs, sel, n, pxPerFrame, tlWidth, maxLen, cuts, hover, tlEl, zoom, play, vid, active } = c;
@@ -26,6 +25,9 @@ export function timelineCard(c) {
onClick: e => { if (e.target.classList.contains("hdl")) return; c.seek(c.frameAt(e).frame); } }, [
h("div", { class: "tlin", style: `width:${tlWidth.value}px` }, [
h("div", { class: "strip" }, thumbs.map(t => h("img", { src: t.src, style: `width:${thumbW}px` }))),
// outside the start / end range: dimmed, not planned
c.range.value.lo > 0 ? h("div", { class: "outr", style: `left:0;width:${c.range.value.lo * ppf}px`, title: "before the start: not planned" }) : null,
c.range.value.hi < N ? h("div", { class: "outr", style: `left:${c.range.value.hi * ppf}px;width:${(N - c.range.value.hi) * ppf}px`, title: "after the end: not planned" }) : null,
...cuts.value.filter(x => x < N).map(x => h("div", { class: "cut", style: `left:${x * ppf}px`, title: `cut @ ${x}` })),
h("svg", { class: "spark", viewBox: `0 0 ${tlWidth.value} 26`, preserveAspectRatio: "none" },
[h("polyline", { points: sparkPts, fill: "none", stroke: "#ff7a90", "stroke-width": 1, "vector-effect": "non-scaling-stroke" })]),
+66 -12
View File
@@ -1,12 +1,14 @@
// 🎞 Video tab: the source video, how it is split into shots, and the generation size.
import { h } from "../vendor/vue.esm-browser.prod.mjs";
import { GRIDS, fld, select, section, pill, hint } from "./common.js";
import { h, GRIDS, fld, select, section, pill, hint, fmtT } from "./common.js";
export function videoTab(c) {
const { plan, an, files, size, busy, detectorUsed, maxLen } = c;
const fps = plan.fps;
const num = (k, step = 0.1, min = 0) => h("input", { type: "number", step, min, value: plan[k], onChange: e => c.setPlan(k, parseFloat(e.target.value) || 0) });
const sel_ = (k, opts, after) => select(plan[k], opts, v => c.setPlan(k, v, after));
const num = (k, step = 0.1, min = 0) => h("input", { type: "number", step, min, value: plan[k],
onChange: e => c.setPlan(k, parseFloat(e.target.value) || 0,
["max_s", "min_s"].includes(k) ? () => c.autoSplit(true) : undefined) });
const sel_ = (k, opts, after) => select(plan[k], opts, v =>
c.setPlan(k, v, after || (k === "mode" ? () => c.autoSplit(true) : undefined)));
const au = an.value?.audio;
const source = section("Source video", [
@@ -29,29 +31,81 @@ export function videoTab(c) {
"It is split into shots the model can generate; every shot then gets its own reference, prompt and mask."]),
]);
const R = c.range.value, N = c.n.value;
const playF = Math.min(N - 1, Math.max(0, c.play.frame || 0));
// a player right here, so the range is picked by watching (scrub it, or click the timeline above)
const rng = section("Range", [
h("div", { class: "player" }, [
h("video", { ref: c.vid, src: c.viewUrl(plan.video), preload: "metadata", controls: true, playsinline: true,
style: "max-height:240px" }),
h("div", { class: "pinfo" }, [
h("div", { class: "tc" }, [
h("div", ["now ", h("b", fmtT(playF, fps)), ` (frame ${playF})`]),
h("div", ["start ", h("b", fmtT(R.lo, fps)), ` (frame ${R.lo})`]),
h("div", ["end ", h("b", fmtT(R.hi, fps)), ` (frame ${R.hi})`]),
]),
h("div", { class: "rgrid" }, [
h("button", { class: "pri", title: "the range starts at the player's frame", onClick: () => c.setRange("start_f", playF) }, "⇤ start here"),
fld("Start frame", h("input", { type: "number", step: 1, min: 0, max: N - 1, value: R.lo,
onChange: e => c.setRange("start_f", e.target.value) }), fmtT(R.lo, fps)),
h("button", { class: "pri", title: "the range ends after the player's frame", onClick: () => c.setRange("end_f", playF + 1) }, "⇥ end here"),
fld("End frame (0 = end)", h("input", { type: "number", step: 1, min: 0, max: N, value: R.hi >= N ? 0 : R.hi,
onChange: e => c.setRange("end_f", e.target.value) }), fmtT(R.hi, fps)),
(R.lo > 0 || R.hi < N) ? h("button", { class: "ghost", onClick: () => { plan.start_f = 0; c.setRange("end_f", 0); } }, "✕ whole video") : h("span"),
]),
hint(`Play or scrub the video (or click the timeline), then ⇤ / ⇥. ${R.hi - R.lo} frames (${((R.hi - R.lo) / fps).toFixed(2)} s) of ${N}: only this part is split, analysed, generated and joined.`),
]),
]),
], { sub: "work on a part of the video" });
const split = section("Split into shots", [
h("div", { class: "grid" }, [
fld("Mode", sel_("mode", [["shots", "Camera cuts"], ["fixed", "Fixed length"]])),
fld("Detector", sel_("detector", [["adaptive", "PySceneDetect adaptive"], ["content", "PySceneDetect content"], ["builtin", "Built-in"]])),
fld("Mode", sel_("mode", [["shots", "Camera cuts"], ["fixed", "Fixed length"], ["single", "One shot (whole range)"]]),
plan.mode === "single" ? "ignores Max seconds / shot" : ""),
// the range in seconds, here next to the mode: "15 s in one shot" is set without leaving this tab
fld("Range length, seconds (0 = to the end)", h("input", { type: "number", step: 0.5, min: 0,
value: c.range.value.hi < c.n.value ? +((c.range.value.hi - c.range.value.lo) / plan.fps).toFixed(2) : 0,
onChange: e => { const v = parseFloat(e.target.value) || 0;
c.setRange("end_f", v > 0 ? c.range.value.lo + Math.round(v * plan.fps) : 0); } }),
`from the start frame · ${c.range.value.hi - c.range.value.lo} frames`),
fld("Detector", sel_("detector", [["adaptive", "PySceneDetect adaptive"], ["content", "PySceneDetect content"], ["builtin", "Built-in"]], () => c.autoSplit(true))),
fld(`Sensitivity ${Number(plan.sensitivity).toFixed(2)}`, h("input", { type: "range", min: 0, max: 1, step: 0.05, value: plan.sensitivity,
onInput: e => { plan.sensitivity = parseFloat(e.target.value); }, onChange: c.save })),
onInput: e => { plan.sensitivity = parseFloat(e.target.value); }, onChange: () => { c.save(); c.autoSplit(true); } })),
fld("Frame grid", sel_("grid", Object.keys(GRIDS))),
fld("Max seconds / shot", num("max_s", 0.1, 0.2), `= ${maxLen.value} frames (${(maxLen.value / fps).toFixed(2)} s)`),
fld("Min seconds / shot", num("min_s", 0.1, 0)),
fld("Max shots (0 = all)", num("max_parts", 1, 0), "limit how many detected shots are used"),
fld("Timeline fps", num("fps", 1, 1), "re-analyses"),
]),
h("div", { class: "row", style: "margin-top:8px" }, [
h("button", { class: "pri", disabled: !plan.video || !!busy.value, onClick: () => c.autoSplit(true) }, "✂ Auto split"),
h("button", { disabled: !plan.video, onClick: c.analyze }, "↻ Re-analyse"),
detectorUsed.value ? hint(`detector: ${detectorUsed.value}`) : null,
c.allSegCount.value > c.segs.value.length ? hint(`Max shots = ${plan.max_parts}: ${c.allSegCount.value - c.segs.value.length} `
+ `more shot(s) on the timeline are left out. Set Max shots to 0 to use them all.`, "color:#f0b429") : null,
hint("Auto split replaces the boundaries; per-shot settings stay with the shot at the same position."),
]),
], { sub: "camera cuts or fixed length, on the model's frame grid" });
const gen = section("Generation size", h("div", { class: "grid" }, [
fld("Megapixels", num("megapixels", 0.01, 0.02), size.w ? `${size.w}×${size.h}` : ""),
fld("Size multiple", num("multiple", 8, 8)),
]), { sub: "cropped shots use it for the crop" });
const updateSize = () => c.autoSplit(false);
const genFields = [
fld("Size mode", select(plan.generation_size_mode || "megapixels", [
["megapixels", "Megapixel target"], ["original", "Original video"], ["custom", "Custom dimensions"],
], v => c.setPlan("generation_size_mode", v, updateSize))),
];
if ((plan.generation_size_mode || "megapixels") === "megapixels")
genFields.push(fld("Megapixels", h("input", { type: "number", step: 0.01, min: 0.02, value: plan.megapixels,
onChange: e => c.setPlan("megapixels", parseFloat(e.target.value) || 0.5, updateSize) }), size.w ? `${size.w}×${size.h}` : ""));
if (plan.generation_size_mode === "custom") {
for (const [key, label] of [["custom_width", "Custom width"], ["custom_height", "Custom height"]])
genFields.push(fld(label, h("input", { type: "number", step: 32, min: 32, value: plan[key],
onChange: e => c.setPlan(key, Math.max(32, parseInt(e.target.value) || 1024), updateSize) })));
if (size.w) genFields.push(fld("Output size", `${size.w}×${size.h}`));
}
if (plan.generation_size_mode === "original") genFields.push(h("div", { class: "hint" }, "Original resolution is rounded to the size multiple for model compatibility."));
genFields.push(fld("Size multiple", h("input", { type: "number", step: 8, min: 8, value: plan.multiple,
onChange: e => c.setPlan("multiple", Math.max(8, parseInt(e.target.value) || 32), updateSize) })));
const gen = section("Generation size", h("div", { class: "grid" }, genFields), { sub: "cropped shots use it for the crop" });
return h("div", { class: "card" }, [source, an.value ? split : null, an.value ? gen : null]);
return h("div", { class: "card" }, [source, an.value ? rng : null, an.value ? split : null, an.value ? gen : null]);
}