# BFS Shot Planner / Shot Loop Turn a long video into shots a video model can actually follow, give every shot its own reference image and prompt, run the rest of the workflow once per shot, and join the results back into one video with the original timing and soundtrack. ## Why A video model only follows a guide reliably inside the clip length it was trained on. An H3 body-swap LoRA trained on 107-frame clips keeps the scene, framing and motion at 4.5 s and loses them at 10 s, where it starts summarising the clip instead of following it frame by frame. Splitting the source into model-sized shots, at camera cuts when there are any, keeps every shot inside that range. ## Nodes | node | what it does | |---|---| | **BFS Shot Planner** | Pick a video, split it on a timeline, set a reference and prompt per shot. Outputs the shots as a ComfyUI **list**. | | **BFS Shot Unpack** | Opens one shot into plain values: guide frames, reference pictures (a list), prompt, length, first frame, audio, size, index. Use it with any model. | | **BFS Shot Repack** | Puts edited pieces back into a shot: Unpack's `shot` output goes to Repack's `shot`, the edited piece (e.g. the reference with its background removed) to its input. Unconnected inputs keep their values; timing, cuts and audio are unchanged. | | **BFS Shot H3 Conditioning** | The common MiniMax H3 path with the important controls only: aligned Add Guide and/or native video reference, references, optional audio conditioning, the Planner's inpaint mode and source-subject blur. | | **BFS Shot H3 Conditioning (Advanced)** | The full conditioning node. Adds the setting reference, the first-frame anchor and the experimental mask-guide controls; its `model` input passes through to `model` (older workflows routed the model through it). The duet lives in **BFS Shot H3 Duet** (1.113 took it out of this node). | | **BFS Shot Join** | Concatenates the decoded shots in order, trims each to its true length, cross-fades soft joins and returns the soundtrack. With *comparison* on it also returns a side-by-side video (original shot \| references \| result) with the shot's info and your *label* on top and its prompt below, ready for Create Video. *save to = mp4 file* writes the video (and the comparison) straight to MP4 files instead, one shot in memory at a time (long videos); `video_path` gives the file. | | **BFS Shot H3 Duet** | Renders one shot with H3: the shot's clip pinned beside the video (duet, no LoRA), the shot as an aligned guide (for body-swap LoRAs), or both. | ## The panel Use the language selector in the top bar to switch the interface between English, Portuguese, Spanish and Simplified Chinese. The choice is saved in this browser and does not alter the workflow plan or prompt text. **Tour** opens an optional step-by-step introduction when clicked; it never appears automatically. The planner's panel has five tabs, in the order you work. The timeline stays on top of every tab. | tab | what is there | |---|---| | 🎞 **Video** | pick or upload the video, audio, how it is split into shots (cuts or fixed length), generation size | | 🎬 **Shots** | one card per shot (scroll sideways, **← β†’** move between shots), the preview player and the selected shot's editor: reference & prompt, who is replaced (mask, generation mode, `{target}`), continuity, **Copy to other shots** | | πŸ‘₯ **People / Filters / Masks** | the cast found by face, shot filters and the global mask settings | | πŸ“ **Prompts & refs** | the global reference and prompt, the VLM and the reference descriptions (`{details}`) | | β–Ά **Run** | **Checks** (what to fix before running, with a button to each shot) and how the shots run (shot by shot / queue loop / auto loop) | Shot cards show badges for what each shot has: 🎯 mask, 🎭 / βœ‚ generation mode, πŸ§‘ `{target}`, β›“ continuity and ⚠ problems. The header's ⚠ count opens the checks. **Copy to other shots** (in the shot editor) copies the chosen parts of the selected shot to all shots, or to the shots after it: references, prompt, mask text, mask points (each shot gets them on its frame at the same relative position), generation mode, `{target}`, continuity. It replaces the old "β†’ all" buttons. ## How the loop works ComfyUI runs any node that receives a list once per item and pairs several lists by index. The planner's `shots` output is a list, so everything downstream of it (conditioning, sampler, decode) runs once per shot, and shot *i* always gets guide *i*, reference *i* and prompt *i*. Single values such as the model and VAE are shared by every run. The join takes the whole list back (it is an `INPUT_IS_LIST` node) and puts the video together. Three run modes: - **One run, shot by shot (default)**: one Queue renders every shot **in order inside the same run**. Each shot is stored on disk; then BFS Shot Join expands the part of the graph between the planner and itself (ComfyUI node expansion) for the next shot, so that shot is conditioned *after* the previous one exists and continuity (*head* / *history*) works. No browser re-queue (the tab can be closed), Save Video fires once with the full video, and after a failure a new Queue resumes from the stored shots (*Redo all* regenerates every shot). Previews connected inside the shot's part of the graph show every shot. Needs exactly one BFS Shot Planner before the join. Same idea as the recursive loop of Ethanfel's MiniMax H3 Context Loop. - **Queue loop (one shot per run)**: each run generates the next pending shot and stores it on disk. The join blocks every node after it until the last shot, then outputs the full video, so a Save Video node only fires once. With *Auto-queue the next shot* on (default), the panel re-queues the workflow after each shot, so one click still renders the whole video (keep the ComfyUI tab open: the panel does the re-queue). *Redo all* regenerates every shot. Changing the plan starts a new run. It resumes from the stored shots after a failure, and continuity *head* / *history* needs it: each shot reads the result of the one before. - **Auto loop (all shots at once)**: every shot in one queue run, each node runs for every shot before the next node. Holds all decoded shots in memory. Continuity *head* / *history* does not work here (all shots are conditioned before any is sampled). ### Memory - **Status at the top (🧠)** and a **Memory** card in the Run tab. BFS Shot Join records every shot it receives on this GPU: its size (latent tokens = generated frames, with the continuity head, Γ— resolution) and the peak VRAM of its run. Before running, every planned shot is compared with the **biggest shot that already ran on this GPU**: *βœ“* within it, *a bit bigger* (it will probably fit, maybe slower), or *⚠ may run out of memory* (also listed in Checks). There is no fixed "GB per frame" for H3 (ComfyUI offloads weights as needed), so nothing is guessed: until a first shot has run, the status says *no memory data yet*. The record lives in ComfyUI's `user/bfs_shotloop_vram.json`. - **Between shots** (Run tab, shot by shot and queue loop): *keep models loaded* (fastest) or *free memory*, which unloads every model and empties the GPU after each shot so the next one starts clean (low VRAM, or a later shot that runs out of memory); models load again for every shot, so it is slower. - What makes a shot lighter: lower **Megapixels**, a shorter **Max seconds / shot**, **head 5** instead of 22 (the head adds its frames to every shot), **Crop** (the generation is the subject's box), and ComfyUI's own options (`--lowvram`, fp8 / GGUF weights, tiled VAE decode). - **The final join** (after the last shot) is where long videos used to run out of **RAM** (not VRAM: the process is killed, e.g. `Terminated` under WSL). It now reads one stored shot at a time into the final video. Measured on 11 shots at 960Γ—544 (~50 s): **+15.8 GB before, +8.2 GB now** with *save to = images* (that is the video itself, which an IMAGE output must hold), and **+2.1 GB with *save to = mp4 file***, which streams the video to ffmpeg and stays at about one shot whatever the length (also with the comparison, which is streamed 16 frames at a time: +2.1 GB measured at *comparison_height* 1024, against ~32 GB as images). For long videos use *mp4 file* (BFS Shot Join), or the panel's **Combine**. - **RAM guard:** before an *images* join, BFS Shot Join asks ComfyUI's cache to drop earlier outputs and estimates what the video (and the comparison) need; when that does not fit in the free RAM it writes MP4 files instead and says so in the console, instead of the whole ComfyUI being killed. - **The outputs are never empty:** with *mp4 file* (chosen or by the RAM guard) the `images` and `comparison` outputs still carry the video, read back from the written files at a size that fits the free RAM (the console says the scale; the files keep the full size), the join shows the files on its node, and `video_path` gives the path. - **Default: save_to = images** (the result comes out inside ComfyUI, for the next nodes). The panel's Checks warn when the video is long for this machine (as images it needs about 12 bytes per pixel per frame, ~3.5x with the comparison, against the free RAM) and advise *mp4 file* for that video. - **comparison_height** is the height of each column: the comparison is three columns wide plus text bars, so it costs about 3Γ— the video at the same height. 360–512 is plenty to review a long video. ### Stored shots (cache, redo, combine) In the **shot by shot** and **queue loop** runs every generated shot is stored, and the Run tab's **Stored shots** card manages them. - **Shot cache:** *Keep* (default) stores them in ComfyUI's output folder, `output/bfs_shotloop/`: they survive a restart, so a stopped run resumes and a single shot can be redone. *Temp* uses ComfyUI's temp folder, which ComfyUI **empties at every start** (before 1.104 every run was stored there). - **Shot storage** (the format of each stored shot, measured on 145 frames at 1280Γ—720): | Storage | Size | Quality | |---|---|---| | **Lossless** (FFV1, default) | ~34 MB | exact: the final video is unchanged | | Smaller (H.264 RGB, crf 8) | ~21 MB | visually lossless (PSNR ~52 dB) | | Smallest (H.264 RGB, crf 14) | ~9 MB | light loss (PSNR ~48 dB) | | Raw frames (`.pt`) | ~401 MB | exact; needs no ffmpeg | The final video of a stored run is assembled from the stored shots, and the continuity head reads them: a lossy storage reaches the output (the panel warns). Without ffmpeg a shot falls back to raw frames. With *comparison* on, the shot's source frames for the comparison's original column are kept as `shot_NNNN_orig.mkv` (visually lossless: they are only shown); the `.pt` itself holds only the shot's info (a few KB). - **One folder per source video.** Each shot remembers the settings it was made with (its range, prompt, references, mask, crop, continuity, the generation size, the global prompt / references and the references or prompt connected to the planner). The next Queue **runs only the shots that are not stored or whose settings changed**, plus the shots that **continue from a shot that was made again** (head, history, previous shot as reference / first frame), so every seam matches. Shot cards show πŸ’Ύ (stored, up to date) or ⏳ (runs next time). - **Model, LoRA, seed, sampler (1.112.0):** the planner reads them from the queued workflow (every node that feeds BFS Shot Join's `images`: loaders, LoRAs, sampler, seed, conditioning settings), so changing any of them runs every shot again. The join's own settings (crossfade, comparison) don't. A seed set to *randomize* changes on every Queue, so every Queue then runs every shot; fix the seed to keep the stored shots. - **↻ Redo** a shot (Stored shots card): it runs again on the next Queue, and the shots that continue from it run after it. Tick *↻ redoes only that shot* to keep the next shots as they are (the seam after it may then not match). - **🎬 Combine stored shots:** puts the stored shots together into an MP4 in the output folder **now, without generating** (one shot in memory at a time), on the plan's timeline: the soundtrack, the original video where a shot is skipped, and also where a shot is not stored yet or out of date, so it previews a run in progress. A link to the file appears in the card. - When every shot is stored, a Queue still runs the last shot again so the join can output the video: use Combine to only assemble. - **Recovering a run outside ComfyUI** (e.g. ComfyUI was killed during the final join): `python tools/bfs_assemble_shots.py output/bfs_shotloop/ video.mp4 --audio input/` (ComfyUI's Python, ffmpeg on the PATH) does the same as Combine with one shot in memory; it reads the old raw `.pt` shots too. ## Guide frames and lengths Each shot is generated at a length the model accepts (frame grid: H3 `17n+5`, LTX/Wan `8n+1`, Wan `4n+1`, or any). When a shot is shorter than that length, the extra guide frames come from the video that follows it, so the guide never repeats or stretches. The join cuts each result back to the shot's own frames, which keeps the timing identical to the source. Those extra frames are also a real overlap, which the join uses to cross-fade boundaries that are not camera cuts. ## Splitting - **Camera cuts** (default): cuts from **PySceneDetect** (`adaptive` or `content`; install `scenedetect`), or the built-in detector when the package is missing. Shots shorter than the minimum merge into a neighbour when the merge still fits; shots longer than the maximum split into equal parts. - **Fixed length**: equal parts no longer than the maximum. - **One shot (whole range)**: the whole selected range as a single shot, whatever its length; *Max seconds / shot* is not applied. For "N seconds in one piece" set *Range length, seconds* = N, right next to the mode (it moves the end frame). For pieces of at most X seconds use *Fixed length* with *Max seconds / shot* = X. - **By hand**: drag the white handles on the timeline to move a boundary, double-click the shot bar to split, *β‡₯ Merge* to join, or select a shot and press **Delete** to remove the cut at its start (it merges into the previous shot). Your edits are what runs. **Range** (🎞 Video tab, with its own player): *Start frame* / *End frame* (timeline frames; end 0 = the end of the video) limit the plan to a part of the video, before anything is split, analysed or generated. Play or scrub the player (or click the timeline) and press *⇀ start here* / *β‡₯ end here* to take its exact frame. The timeline dims what is outside; the join and the soundtrack cover only the range. *Max seconds / shot* is converted to frames and snapped down to the grid (4.5 s at 24 fps = 107 frames for H3). *Max shots* in the Video tab caps how many shots are used. ## Audio The source card shows whether the video has a usable audio track (rate, channels) or why not. *Use the video's audio* or *silent track*. Without a usable track (none, unreadable, or silent chosen) the planner's `audio`, BFS Shot Join's `audio` and the H3 Duet nodes return a **silent track of the right length** instead of nothing, so Create Video / Save Video always work. Audio is read with ffmpeg, or with PyAV when the ffmpeg binary is missing. ## Preview *Play shot* plays only the selected shot, from its first to its last frame, and stops (or loops with *loop shot*); every shot card has its own β–Ά. *Play all* plays the whole plan and jumps over shots that will not run. The panel shows the shot's start, end and current time (and frame), a yellow playhead follows on the timeline, and clicking the timeline seeks. The source video streams from the input folder, so only what you play is loaded. ## Shot filters (People / Filters / Masks tab) Skip shots that should not run, decided per shot from a few sampled frames: | filter | skips a shot when | |---|---| | Needs a person | no person is detected | | Min person size | the largest person covers less than that fraction of the frame (wide shots) | | Max people | more people than that (crowds) | | Needs a face | no face is detected | | Skip dark / fades | the mean brightness is below the level | | Skip static | there is almost no motion (title cards, freeze frames) | | Min frames | the shot is shorter than that | People and faces use YOLO from `models/ultralytics` (`person_yolov8m-seg.pt`, `face_yolov8m.pt`, the Impact Pack files) with OpenCV fallbacks. *Analyse people & faces* shows the numbers on every shot card. Each shot can override the filters (always run / always skip). Skipped shots do not run. Connect the planner's `timeline` output to BFS Shot Join and they are filled with the original video, keeping the full duration and soundtrack, or removed (with the matching audio) when *Skipped shots in the output* is set to remove them. ## Cast (people by face) *Find people* samples the video every 0.5 s, detects faces and groups them into people by face identity, on the CPU (it never touches the GPU; the result is cached per video). **No extra package is needed**: by default it uses OpenCV Zoo's **YuNet** (detection) + **SFace** (recognition), Apache-2.0 models run by OpenCV itself and downloaded once into `models/opencv_face` (0.2 + 37 MB). If **InsightFace** is installed (`buffalo_l` in `models/insightface`) it is used instead (a bit more accurate; mind its license). Last resort, without either: SAM 3.1 faces grouped by colours (hair, skin, clothes). Each person card shows the face, screen share and first/last appearance (click the face to seek). The *ignore* checkbox excludes a detected identity from cast-based filters. Every shot card shows the people detected in it, with the main person highlighted. | option | effect | |---|---| | Split where the main person changes | adds a boundary where the largest face switches to another person for at least a second, then re-splits | | Only run shots with a detected person | shots with no detected, non-ignored person are skipped | Face identities are used for filtering and optional shot splitting. References and marker colors are selected globally or per shot; they are not assigned to detected identities. ## References and prompts Up to **12 reference pictures** (1.113.0): the global list starts with one slot and **+** adds more; they are `` to ``, in order. Click a slot to upload a picture (the **+** takes several files at once), or drop image files on a slot (replaces it) or anywhere on the box (adds them). A filled slot has **βœ•**: a global picture is taken out (the next ones move up), a shot's own picture falls back to the global one. Every shot can have its own pictures and prompt. A shot's empty slot uses the global picture at the same position (shown faded), so a shot can change only `` and keep the rest. **Copy to other shots** gives a shot's pictures to every shot. The last ten pictures you picked are shown under the shot editor: click = picture 1, shift+click = add it as the next picture, or drag it onto any slot. Under every prompt (global and per shot) there is a chip per picture: **PictureN** inserts `` at the cursor, **πŸ“** inserts `{details@Picture N}` (that picture's description, below). The node no longer has `ref_image` / `ref_image_2` inputs (the pictures live in the panel); a workflow saved with them connected still runs, and the log says the images are ignored. A connected `prompt` input still overrides the global prompt. The planner outputs `refs` (every picture the shots use, without repeats, as a list) and `ref_count` where `ref_image` / `ref_image_2` were. **BFS Shot Unpack** outputs `refs`, every picture of the shot as a list (the next node runs once per picture, e.g. a background remover), and `ref_count`; **BFS Shot Repack**'s `refs` input takes that list back and gives each shot its own pictures (a workflow from before 1.113 that wired `ref_image` still works). **BFS Shot H3 Conditioning** passes every picture of the shot (the `use_ref_2` switch left in 1.114: the pictures are chosen in the planner); H3's own node stops at 9 and LoRAs trained like OpenJutsu saw up to 9, so the log notes a shot with more. ## Mask & crop (SAM 3) Generate only part of the frame. In a shot's editor (*Select region*), type what to segment in English (`person in white`, `woman with red hair`, `red car`; up to 32 tokens, commas for several things) or click **🎯 Points** to pick positive / negative points on any frame of the shot, then **πŸ‘ Preview** (red = mask, yellow = crop box). Then pick how the shot is generated: | mode | what is generated | crop / uncrop | |---|---|---| | **Full frame** | the whole frame; the mask only feeds `{target}` and the setting picture | no | | **🧩 Frame + paste** | the whole frame (so the model follows the guide's pose freely and the new person may be bigger); BFS Shot Join then pastes only the person onto the original frame: the shot's mask plus the new subject's outline (SAM 3 on the result with the shot's *new subject* text, default `person`; `paste_new_outline`) | no crop; paste in the Join | | **🎭 Mask only** | only the mask, on the whole frame; everything else stays exactly as it was | no | | **βœ‚ Crop** | a box around the mask (more pixels for a small person); BFS Shot Join pastes it back | yes | | **βœ‚πŸŽ­ Crop + mask** | only the mask, inside the crop: the most detail with the background kept | yes | *Frame + paste* is the safest for a different body (e.g. a woman in profile replaced by a man facing the camera): *Mask only* binds the new person to the old outline, *Frame + paste* does not. The two mask modes need **BFS Shot H3 Conditioning β†’ inpaint = per shot (planner)** (the default; *only the mask* forces it on every shot, *off* never). The crop is one box around the mask (the union over all its frames, so it does not shake), at the source resolution, sent to the model at the generation size; **BFS Shot Join** pastes the result back, feathered by the mask (or the whole box). **BFS Shot Unpack** also outputs the shot's mask (`mask`), e.g. for an inpainting model. In **BFS Shot Join**'s comparison video, cropped shots show their mask (red) and crop box (yellow) over the original column (`comparison_mask`, on by default). **Mask opacity** (mask modes, per shot, slider under the modes): the value of the white inside the generation mask. 1 regenerates the masked area completely. A grey mask (e.g. 0.85) keeps part of the original there: H3 puts those rows at *opacity* Γ— the noise level, so they start from the original partly visible. 0.8-0.9 keeps pose, outline and lighting while still swapping; too low copies the original person. With `guide_mode = aligned guide` the model also sees the whole original shot as its guide, so it follows the motion either way. **🎨 Colored Mark** is an optional translucent overlay on the source video: each marked person is tinted with the color of the picture they become, and the prompt gets the mapping ("the person marked in red becomes the character from ``"), for any model or LoRA. Every picture has its **Mark color** (under it in **Prompts & refs**: green, purple, red, blue, yellow, magenta, cyan, orange, white, brown, pink, olive by default); change it when it is too close to a color in the video. In a shot, turn on Colored Mark and pick which picture the shot's selection (its SAM 3 text or points) becomes; **+ Mark another person** adds a person with their own picture and text or 🎯 points (the points window has one point set per person). This is only a visual marker: it does not create an inpainting mask or change **Full frame**, **Mask only** or crop behavior. More than one marked person needs Crop off, and the shot must have every picture the marks use. The overlay keeps movement and facial expression visible. Plans before 1.113 (a second region for ref 2) open with that region as ``. ```text body_swap: subject_definitions: is the character shown in , whose identity, face, head, hair or fur, skin, body shape, proportions and clothing define the green-marked replacement. is the character shown in , whose identity, face, head, hair or fur, skin, body shape, proportions and clothing define the purple-marked replacement. In the guide video, the translucent green and purple tints are temporary editing markers, not character appearance. Replace the green-tinted person only with and the purple-tinted person only with . The guide video provides the scene and the complete performance of both people, including their body poses, gestures, head motion, gaze, speech and facial expressions. summary: [video generation + references] Replace the green-tinted guide person with from and the purple-tinted guide person with from , preserving the scene, camera and complete motion and expression of both people. detailed_description: [Shot 1] Replace the person marked by the translucent green tint with from , and the person marked by the translucent purple tint with from . Each character follows that person's exact position, timing, body pose, gestures, head motion, gaze, speech and facial expressions in the guide. Remove both temporary color markers completely. Keep the two reference identities distinct and stable in every frame. Preserve their interaction, the background, lighting, camera and all unselected content from the guide video. Prevent identity drift, clothing changes, texture crawling and flicker. The result is a real, photographic video. ``` The VLM's shot analysis suggests a SAM 3 text selection from the original frames; it does not assign guide colors or decide that two detections are the same identity. **Describe target** only describes the first shot selection. For two-color swaps, choose both selections and identities yourself, and use the explicit mapping above (or let πŸ€– Auto prompt write it: it is told which tinted person becomes which picture). **Mask guide** (experimental, *Mask only* shots, BFS Shot H3 Conditioning β†’ `mask_guide`): the shot with only the masked region visible, the rest grey, so the model looks at the subject's pose and outline on its own while the mask limits where it generates. *+ extra guide* adds it as a second aligned guide next to the normal one (the model sees both); *instead of the full guide* replaces the normal one; *+ reference video* gives it as a native reference video (`