Author SHA1 Message Date
jetthuangai 3c053e09df Bump version 1.7.0 2026-10-18 12:10:29 +07:00
jetthuangai 5a617560a3 docs: SAM 3 and NH_SEGS nodes in the EN/VI guides
- vision guide (EN/VI): requirements and install, how the two passes work
  and their limits, the NH_SEGS type, the six nodes, three workflows and
  troubleshooting
- READMEs (75 nodes), CHANGELOG 1.7.0, THIRD_PARTY_NOTICES (SAM License,
  user-installed sam3 and checkpoint)
- six node cards, sam3-segment.jpg example from the public-domain astronaut
  photo; build_examples.py builds it with --gpu
2026-10-18 12:10:18 +07:00
jetthuangai 5389e93994 feat: SAM 3 segmentation and NH_SEGS instance nodes (Vision v2 P0)
Six new Vision nodes, 69 -> 75:
- SAM3 Segment: every instance of the concepts you name, with Meta's SAM 3.
  The whole image is encoded once for all concepts (pass 1); in balanced mode
  each instance is segmented again on a full-resolution close-up with the
  box-prompt decoder (pass 2), so small and medium objects keep crisp edges
  on 4K-8K photos. A close-up that does not show the pass-1 instance is
  dropped (tolerant check that keeps thin poles and cables).
- Segs Select, Segs Compose (label formulas: | - & ~ all image none),
  Segs To Masks, Segs To Impact SEGS, Segs Preview.

NH_SEGS keeps per-instance masks only inside each box, so dozens of
instances on an 8K image stay small; full-size masks are built only on
output (with a RAM guard in Segs To Masks).

Self-contained: no other custom node pack is used. Meta's sam3 package is
imported lazily with an API check (install hint pinned to the tested
commit), TF32 switches changed by its import are restored, and the gated
sam3.pt checkpoint in models/sam3 is checked before loading and never
downloaded. CUDA only. Weights and sam3's GPU-side tables go back to RAM
after each run unless keep_model_loaded.
2026-10-18 12:10:18 +07:00
jetthuangai f4843d30d3 docs: bilingual EN/VI guide, node cards, examples and new README
- README rewritten: banner, highlights with real outputs, install incl.
  ONNX Runtime and extra_model_paths keys, quick start, full catalog,
  licence summary, PDF guide links
- docs/en and docs/vi: guide home + 13 pages covering all 69 nodes (card,
  inputs/outputs from the live definitions, behaviour, recipes, tips)
- docs/assets: 69 node cards generated from INPUT_TYPES, 14 example images
  and a banner rendered from real node outputs on public-domain/CC0 photos
- docs/_tools: card, example, spec and PDF builders (reportlab) so the docs
  can be regenerated from the code
2026-10-18 09:17:48 +07:00
26 changed files with 3320 additions and 47 deletions
+18 -7
View File
@@ -11,10 +11,10 @@
# NH-Nodes for ComfyUI
**69 production-focused nodes** for ComfyUI: load and save image sets with their metadata, tile and composite large
**75 production-focused nodes** for ComfyUI: load and save image sets with their metadata, tile and composite large
images, pick the right resolution for current models and put results back into the original frame, refine masks,
segment garments and faces, build VTON masks, route workflows with lazy switches, generate prompts, and test many
LoRAs or models from one node.
segment garments and faces, find every person, window or sky by name with SAM 3, build VTON masks, route workflows
with lazy switches, generate prompts, and test many LoRAs or models from one node.
Every node is documented with a visual card, its inputs and outputs, and examples — in
[English](docs/en/README.md) and [Vietnamese](docs/vi/README.md).
@@ -32,8 +32,9 @@ Every node is documented with a visual card, its inputs and outputs, and example
</tr>
<tr>
<td><a href="docs/en/vision.md"><img src="docs/assets/examples/garment-segment.jpg" alt="Garment segmentation"></a><br>
<b>Segmentation</b> — garment and face-part masks from SegFormer models, plus VTON garment masks with pose and
parsing.</td>
<b>Segmentation</b> — name what you want (people, windows, the sky) and SAM 3 returns every instance as its own
mask, with crisp edges on small and medium objects even in 4K–8K photos; garment and face-part masks from
SegFormer models; VTON garment masks with pose and parsing.</td>
<td><a href="docs/en/mask.md"><img src="docs/assets/examples/mask-morphology.jpg" alt="Mask morphology"></a><br>
<b>Mask tools</b> — fast grow / shrink on the CPU or the GPU, hole filling, soft edges, bounding boxes and
aspect-ratio matching.</td>
@@ -66,7 +67,8 @@ cd NH-Nodes
python -m pip install -r requirements.txt
```
Restart ComfyUI afterwards. Requirements: a recent ComfyUI (tested with 0.41) and Python 3.10+.
Restart ComfyUI afterwards. Requirements: a recent ComfyUI (tested with 0.41) and Python 3.10+. **SAM3 Segment** has
more requirements, listed below.
### Optional: VTON and segmentation models
@@ -81,7 +83,12 @@ Restart ComfyUI afterwards. Requirements: a recent ComfyUI (tested with 0.41) an
base_path: D:/ai-models/
nh_nodes: NH-Nodes/ # SegFormer models (garment, face parsing)
vton_mask: ComfyUI-Vton-Mask/ # VTON pose + parsing ONNX models
sam3: sam3/ # SAM 3 checkpoint (you download sam3.pt yourself)
```
- **Requirements for SAM 3** (SAM3 Segment only): an NVIDIA GPU with CUDA, Meta's `sam3` package and the `sam3.pt`
checkpoint. NH-Nodes installs and downloads none of them: you install the package yourself and request the weights
from Meta. The steps, and what to do if another SAM 3 pack is already installed, are in
[Before you start with SAM 3](docs/en/vision.md#before-you-start-with-sam-3).
## Quick start
@@ -90,6 +97,7 @@ Restart ComfyUI afterwards. Requirements: a recent ComfyUI (tested with 0.41) an
| Upscale or refine a huge image piece by piece | Load Image → **Image Tile** → your sampler/upscaler per tile → **Image Untile** | [image-compose](docs/en/image-compose.md#image-tile-nh) |
| Generate at the model's native size, then restore the original framing | Load Image → **NH Smart Ratio Image Resize** → edit → **NH Restore Image Geometry** | [resolution](docs/en/resolution.md) |
| Inpaint only the clothes | Load Image → **Garment Segment** → **Mask Morphology** (grow) → inpaint | [vision](docs/en/vision.md) |
| Mask every person, window or the sky in a large photo | Load Image → **SAM3 Segment** → **Segs Select** or **Segs Compose** → mask → edit or inpaint | [vision](docs/en/vision.md#sam-3-workflows) |
| Compare many LoRAs, one per queue | **Counter** → **Load LoRA Model Index** (index) → sampler → **Image Label** (prints the LoRA name) → Save Image | [loaders](docs/en/loaders.md) |
| Run one of two branches without executing the other | condition → **If/Else** (lazy) → … | [logic](docs/en/logic.md#ifelse-nh) |
@@ -103,7 +111,7 @@ Restart ComfyUI afterwards. Requirements: a recent ComfyUI (tested with 0.41) an
| Image — mask-based retouch | NH Mask-Aware Resize, NH Agnostic Image Generator, NH Simple Face Paste | [EN](docs/en/image-retouch.md) · [VI](docs/vi/image-retouch.md) |
| Resolution | NH Smart Resolution Picker, NH Smart Ratio Image Resize, NH Ratio Preset Image Resize, NH Remove Resize Padding, NH Restore Image Geometry | [EN](docs/en/resolution.md) · [VI](docs/vi/resolution.md) |
| Mask | Mask Morphology, Mask Properties, NH Create Box Mask, NH Mask Aspect Ratio Match | [EN](docs/en/mask.md) · [VI](docs/vi/mask.md) |
| Vision | Garment Segment, Portrait Segment | [EN](docs/en/vision.md) · [VI](docs/vi/vision.md) |
| Vision | Garment Segment, Portrait Segment, SAM3 Segment, Segs Select, Segs Compose, Segs To Masks, Segs To Impact SEGS, Segs Preview | [EN](docs/en/vision.md) · [VI](docs/vi/vision.md) |
| VTON | VTON Ultimate Processor | [EN](docs/en/vton.md) · [VI](docs/vi/vton.md) |
| Logic | Compare, Logic Gate, If/Else, Switch N, Any Switch, Any Branch Switch, Gate Switch, Value Match Index, Math Eval, Random Choice | [EN](docs/en/logic.md) · [VI](docs/vi/logic.md) |
| Text | Prompt Template, Prompt Scheduler, String Operations, Prompt Join, Text Split, Regex Extract, Text Index, Text Concatenate, Text Split Lines, Text Random Line | [EN](docs/en/text.md) · [VI](docs/vi/text.md) |
@@ -128,6 +136,9 @@ NH-Nodes' own code is [MIT](LICENSE). Some features rely on third-party code or
- **VTON Ultimate Processor** and **Portrait Segment** are for **non-commercial use only** (CC BY-NC-SA 4.0 code and
parsing models; non-commercial face-parsing model).
- **Garment Segment**'s model does not declare a licence.
- **SAM3 Segment** uses Meta's SAM 3 under the SAM License: royalty-free, with no non-commercial clause, but it bars
military or warfare, nuclear, espionage, and gun or illegal-weapon uses. The weights and the `sam3` package are not
included; you get them from Meta.
## Development
+32 -1
View File
@@ -16,6 +16,7 @@ MIT license.
| `VTON Ultimate Processor (NH)` | **No** | Its code and parsing models are CC BY-NC-SA 4.0 (FitDiT / OOTDiffusion) |
| `Portrait Segment (NH)` | **No** | The face-parsing model card restricts use to non-commercial research and education |
| `Garment Segment (NH)` | Unclear | The model uploader does not declare a license |
| `SAM3 Segment (NH)` | Yes, within trade-control limits | Meta's SAM License is royalty-free and has no non-commercial clause, but it forbids ITAR-regulated activities and uses that trade controls prohibit (military or warfare, nuclear, espionage, guns and illegal weapons). The model and the `sam3` package are not part of NH-Nodes: you get them from Meta |
## Source code bundled in this repository
@@ -55,8 +56,38 @@ They are looked up first in the folders registered as `vton_mask` and `nh_nodes`
| Portrait Segment | [`jonathandinu/face-parsing`](https://huggingface.co/jonathandinu/face-parsing) @ `758b82e` | SegFormer fine-tuned on CelebAMask-HQ | Model card: "non-commercial research and educational purposes" |
| Garment Segment | [`1038lab/segformer_fashion`](https://huggingface.co/1038lab/segformer_fashion) @ `e5bae2d` | SegFormer trained for Fashionpedia classes | No license declared by the uploader (Fashionpedia annotations are CC BY 4.0) |
## Software and models you install yourself
`SAM3 Segment (NH)` runs Meta's SAM 3. NH-Nodes neither bundles nor downloads it, and the
nodes only load it when you use them. You get both parts from Meta and accept Meta's terms
yourself.
| What | Where it comes from | Terms |
|------|---------------------|-------|
| `sam3` package (code) | [`facebookresearch/sam3`](https://github.com/facebookresearch/sam3), installed by the user with pip | SAM License |
| SAM 3 checkpoint `sam3.pt` | [`facebook/sam3`](https://huggingface.co/facebook/sam3) on Hugging Face, a gated repository: access is granted after the user accepts the license. It goes in the `sam3` model folder (`models/sam3`, or any location added under `sam3` in `extra_model_paths.yaml`), which other SAM 3 packs share | SAM License |
The [SAM License](https://github.com/facebookresearch/sam3/blob/main/LICENSE) (Meta, last
updated 19 November 2025) is a non-exclusive, worldwide, non-transferable, royalty-free
limited license to use, reproduce, distribute, copy, modify and create derivative works of
the SAM Materials. It does not mention commercial use. What matters to a user of NH-Nodes:
- **Trade controls.** You must not be a target of trade controls and must comply with them.
The materials may not be used for activities subject to the International Traffic in Arms
Regulations (ITAR), or for end uses that trade controls prohibit, including military or
warfare purposes, nuclear industries or applications, espionage, and the development or
use of guns or illegal weapons.
- Whoever redistributes the SAM Materials must include a copy of the license. NH-Nodes does
not redistribute them.
- Research publications that use the SAM Materials must acknowledge it.
- The materials may not be used to reverse engineer or decompile them, nor to encourage
others to.
- They are provided "as is", and Meta may terminate the license if you breach it.
## Python dependencies
Dependencies listed in `requirements.txt` are installed from PyPI and are not
redistributed here; each keeps its own license. ONNX Runtime (MIT) is optional
and only needed by the VTON node.
and only needed by the VTON node. The `sam3` package (SAM License) and its extras
(`timm`, `ftfy`, `regex`, `iopath`) are optional too: only `SAM3 Segment (NH)` needs
them, and you install them yourself.
+626
View File
@@ -0,0 +1,626 @@
"""SAM 3 for the NH vision nodes: finding the model, loading it, and segmenting concepts in two passes.
NH-Nodes does not ship SAM 3. It uses Meta's ``sam3`` package, installed separately (see ``INSTALL_HINT``), and the
``sam3.pt`` checkpoint, which Hugging Face hands out only after Meta's license is accepted, so it is never
downloaded automatically. Checkpoints are looked up in the ``sam3`` model folder (``models/sam3`` by default, more
locations in ``extra_model_paths.yaml``); other SAM 3 packs use the same folder, so one copy serves all of them.
Pass 1 looks at the whole image once: the image is resized to the model's square input, encoded once and asked for
every concept. Each instance's mask is sampled back to full resolution inside its own box only.
Pass 2 (``detail_level="balanced"``) cuts each instance out of the full-resolution image with a margin and segments it
again with SAM 3's box-prompt decoder, the pass-1 mask given as a hint. Objects that are small or medium in the frame
then get their edges from the original pixels instead of a ~1000 px view of the whole frame. SAM 3 draws masks on a
grid of about 288 cells across whatever it looks at, so a subject that fills most of an 8K frame still has edges at
about 1/300 of its size; hair-level edges need matting. A close-up that does not show the pass-1 instance is dropped
and the pass-1 mask kept.
The official sam3 code builds some of its lookup tables on the GPU, so SAM 3 runs on CUDA only.
"""
import contextlib
import importlib
import importlib.util
import inspect
import itertools
import math
import os
import cv2
import numpy as np
import torch
import torch.nn.functional as F
from .model_files import make_room_on_gpu, models_dir, register_model_folder
from .segs import SegsFrame, make_segment
try:
import folder_paths
except Exception: # running outside ComfyUI (tests)
folder_paths = None
try:
import comfy.model_management as _mm
except Exception:
_mm = None
FOLDER_NAME = "sam3" # shared with the other SAM 3 packs
# Listed in ComfyUI for the folder: SAM 3 checkpoints in the formats SAM 3 packs use, not Hugging Face cache files.
FOLDER_EXTENSIONS = (".pt", ".pth", ".safetensors")
CHECKPOINT_EXTENSIONS = (".pt", ".pth") # what sam3's own loader reads (torch.load)
DEFAULT_CHECKPOINT = "sam3.pt"
DETAIL_LEVELS = ("balanced", "fast")
SAM3_COMMIT = "b26a5f330e05d321afb39d01d3d4881f258f65ff" # the sam3 build NH-Nodes is tested with
INSTALL_HINT = (
"SAM 3 needs Meta's sam3 package, which NH-Nodes does not install for you. In ComfyUI's Python environment run:\n"
f" pip install --no-deps git+https://github.com/facebookresearch/sam3@{SAM3_COMMIT}\n"
" pip install timm ftfy regex iopath\n"
"--no-deps keeps your numpy: sam3 asks for numpy<2, which ComfyUI does not need. "
"Restart ComfyUI afterwards."
)
CHECKPOINT_HINT = (
"Ask for access at https://huggingface.co/facebook/sam3 (Meta has you accept the SAM License first), "
"download sam3.pt from that page and put it in {folder}. Then refresh ComfyUI (press R) and pick it."
)
DEDUP_IOU = 0.9 # a second instance of the same concept overlapping a better one this much is a duplicate
# A close-up replaces the pass-1 mask when it shows the same object: at least REFINE_MIN_INSIDE of it lies within the
# pass-1 blur of the pass-1 mask, and it covers at least REFINE_MIN_CORE of the pass-1 core (see close_up_agrees).
REFINE_MIN_INSIDE = 0.8
REFINE_MIN_CORE = 0.6
_PASS1_GRID = 288 # SAM 3's detector draws masks on a 288 x 288 grid of its square view of the whole image
_AGREE_PIXELS = 1 << 21 # masks larger than this are compared at a reduced scale
_WIDE_ASPECT = 4.0 # beyond this aspect ratio SAM 3's square view squeezes the picture enough to miss small things
_CROP_MARGIN = 0.2 # pass-2 crop margin on each side, as a fraction of the instance's longer side
_CROP_MIN_MARGIN = 16 # px
_CROP_MAX_ASPECT = 2.0 # pass-2 crops are widened to at most 2:1, the model squeezes them into a square
_SAMPLE_CHUNK = 1 << 22 # output pixels per grid_sample call when resampling masks
_WORKING_BYTES = 3 << 30 # VRAM for activations and the full-resolution image, on top of the weights
_API = None
_CACHE = {}
# ---------------------------------------------------------------------------
# Checkpoints
# ---------------------------------------------------------------------------
def default_folder():
"""<models dir>/sam3, the folder ComfyUI and the other SAM 3 packs use."""
return os.path.join(models_dir(), FOLDER_NAME)
def checkpoint_folders():
"""Every folder searched for checkpoints, best first (registers the folder name with ComfyUI)."""
return register_model_folder(FOLDER_NAME, default_folder(), FOLDER_EXTENSIONS)
def checkpoint_choices():
"""Checkpoint names for the node's list: .pt/.pth files in the sam3 folders, or the default name if none yet."""
names = set()
folders = checkpoint_folders()
if folder_paths is not None:
try:
names.update(folder_paths.get_filename_list(FOLDER_NAME))
except Exception as exc:
print(f"[NH-Nodes] Could not list the {FOLDER_NAME} model folder: {exc}")
else:
for folder in folders:
if os.path.isdir(folder):
names.update(os.listdir(folder))
names = sorted(name for name in names if name.lower().endswith(CHECKPOINT_EXTENSIONS))
return names or [DEFAULT_CHECKPOINT]
def resolve_checkpoint(name):
"""Full path of checkpoint ``name`` in the sam3 folders; a RuntimeError says where to get it when it is missing."""
name = str(name or DEFAULT_CHECKPOINT)
parts = name.replace("\\", "/").split("/")
if os.path.isabs(name) or ".." in parts:
raise RuntimeError(f"[NH-Nodes] SAM 3 checkpoint name must be a file in the {FOLDER_NAME} folder: {name!r}")
if not name.lower().endswith(CHECKPOINT_EXTENSIONS):
raise RuntimeError(
f"[NH-Nodes] {name} is not a SAM 3 .pt checkpoint. The safetensors export on Hugging Face is for the "
"transformers library and uses other weight names; download sam3.pt instead."
)
folders = checkpoint_folders()
path = folder_paths.get_full_path(FOLDER_NAME, name) if folder_paths is not None else None
if path is None:
path = next((os.path.join(folder, *parts) for folder in folders
if os.path.isfile(os.path.join(folder, *parts))), None)
if path is None or not os.path.isfile(path):
raise RuntimeError(
f"[NH-Nodes] SAM 3 checkpoint {name} not found in: {', '.join(folders)}.\n"
+ CHECKPOINT_HINT.format(folder=folders[0] if folders else default_folder())
)
return path
# Make the folder name known to ComfyUI as soon as the pack loads.
checkpoint_folders()
# ---------------------------------------------------------------------------
# The sam3 package and the model
# ---------------------------------------------------------------------------
def _tf32_flags():
"""(matmul, cudnn) TF32 switches, or None when torch refuses to report them (its newer precision API in use)."""
try:
return torch.backends.cuda.matmul.allow_tf32, torch.backends.cudnn.allow_tf32
except Exception:
return None
def _restore_tf32(flags):
if flags is not None:
try:
torch.backends.cuda.matmul.allow_tf32, torch.backends.cudnn.allow_tf32 = flags
except Exception:
pass
def _missing_parameters(func, names):
"""Names from ``names`` that ``func`` does not accept (none when its signature cannot be read or takes **kwargs)."""
try:
parameters = inspect.signature(func).parameters
except (TypeError, ValueError):
return []
if any(p.kind == p.VAR_KEYWORD for p in parameters.values()):
return []
return [name for name in names if name not in parameters]
def ensure_sam3():
"""(build_sam3_image_model, Sam3Processor) from the installed sam3 package.
A RuntimeError explains what to do when the package is missing, fails to import or lacks the image API
NH-Nodes uses. Importing sam3's model builder switches TF32 on for the whole process; the previous setting is
put back, so ComfyUI's own models keep theirs.
"""
global _API
if _API is not None:
return _API
if importlib.util.find_spec("sam3") is None:
raise RuntimeError("[NH-Nodes] " + INSTALL_HINT)
flags = _tf32_flags()
try:
builder = importlib.import_module("sam3.model_builder")
processors = importlib.import_module("sam3.model.sam3_image_processor")
except Exception as exc:
raise RuntimeError(f"[NH-Nodes] The sam3 package is installed but failed to load: {exc}\n{INSTALL_HINT}") from exc
finally:
_restore_tf32(flags)
build = getattr(builder, "build_sam3_image_model", None)
processor_cls = getattr(processors, "Sam3Processor", None)
missing = []
if not callable(build):
missing.append("build_sam3_image_model")
else:
missing += [f"build_sam3_image_model({name}=)" for name in _missing_parameters(
build, ("checkpoint_path", "load_from_HF", "enable_inst_interactivity", "device", "eval_mode"))]
if processor_cls is None:
missing.append("Sam3Processor")
else:
missing += [f"Sam3Processor({name}=)" for name in
_missing_parameters(processor_cls, ("device", "confidence_threshold"))]
missing += [f"Sam3Processor.{name}" for name in ("set_image", "set_text_prompt", "reset_all_prompts")
if not callable(getattr(processor_cls, name, None))]
if missing:
raise RuntimeError(
f"[NH-Nodes] The installed sam3 package lacks {', '.join(missing)}; it is older or newer than the "
f"versions NH-Nodes supports. Reinstall it:\n{INSTALL_HINT}"
)
_API = (build, processor_cls)
return _API
def sam3_device():
"""ComfyUI's torch device, which must be a CUDA GPU for SAM 3."""
if _mm is not None:
device = _mm.get_torch_device()
else:
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
if device.type != "cuda":
raise RuntimeError(
f"[NH-Nodes] SAM 3 needs an NVIDIA GPU with CUDA (ComfyUI runs on {device}): the official sam3 code "
"builds part of the model on the GPU."
)
return device
def check_checkpoint(path):
"""Fail early, with the download advice, when ``path`` is not a readable SAM 3 checkpoint.
sam3 loads weights non-strictly and only prints the names it misses, so another model's .pt in the shared folder
would run with random weights, and a truncated download fails deep inside torch. Reading the names is quick:
the file is memory-mapped, not loaded.
"""
advice = CHECKPOINT_HINT.format(folder=os.path.dirname(path))
try:
state = torch.load(path, map_location="cpu", weights_only=True, mmap=True)
except Exception as exc:
raise RuntimeError(f"[NH-Nodes] {path} cannot be read as a SAM 3 checkpoint ({exc}); it may be an "
f"incomplete download.\n{advice}") from exc
if isinstance(state, dict) and isinstance(state.get("model"), dict):
state = state["model"]
names = list(state.keys()) if isinstance(state, dict) else []
del state
if not any(name.startswith("detector.") for name in names) or not any(name.startswith("tracker.") for name in names):
raise RuntimeError(f"[NH-Nodes] {path} is not a SAM 3 checkpoint (it has no SAM 3 detector and tracker "
f"weights).\n{advice}")
def load_model(checkpoint):
"""The SAM 3 image model for ``checkpoint`` (a name in the sam3 folders); kept in RAM between runs."""
path = resolve_checkpoint(checkpoint)
stamp = (os.path.normcase(os.path.abspath(path)), os.path.getmtime(path), os.path.getsize(path))
if _CACHE.get("model") is not None and _CACHE.get("stamp") == stamp:
return _CACHE["model"]
unload()
build, _ = ensure_sam3()
device = sam3_device()
check_checkpoint(path)
print(f"[NH-Nodes] Loading SAM 3 from {path}")
with torch.cuda.device(device): # the tables sam3 builds on "cuda" go to ComfyUI's GPU
model = build(checkpoint_path=path, load_from_HF=False, enable_inst_interactivity=True,
device="cpu", eval_mode=True)
if not callable(getattr(model, "predict_inst", None)) or getattr(model, "inst_interactive_predictor", None) is None:
raise RuntimeError("[NH-Nodes] This sam3 build has no box-prompt decoder (predict_inst). " + INSTALL_HINT)
transforms = getattr(model.inst_interactive_predictor, "_transforms", None)
if getattr(transforms, "max_hole_area", 0):
# sam3 fills holes of close-up masks by default; keep real ones (between an arm and the body), as pass 1 does
transforms.max_hole_area = 0.0
model.requires_grad_(False)
_CACHE.update(model=model, stamp=stamp)
return model
def _model_device(model):
return next(model.parameters()).device
def _is_on(current, wanted):
"""True when ``current`` satisfies ``wanted``; an index-less "cuda" accepts any cuda:N."""
return current.type == wanted.type and (wanted.index is None or current.index == wanted.index)
_TABLE_ATTRIBUTES = ("freqs_cis", "freqs_cis_real", "freqs_cis_imag") # sam3's rotary tables, plain attributes
def _all_on(model, device):
return all(_is_on(p.device, device) for p in model.parameters())
def _move_tables(model, device):
"""Move sam3's rotary tables, which model.to() leaves behind because they are not buffers."""
for module in model.modules():
for name in _TABLE_ATTRIBUTES:
value = getattr(module, name, None)
if isinstance(value, torch.Tensor) and not _is_on(value.device, device):
setattr(module, name, value.to(device))
def move_model(model, device):
"""Put the weights on ``device``, asking ComfyUI to free VRAM first; back to the CPU if the move fails."""
if device.type == "cuda":
weights = 0 if _all_on(model, device) else sum(
t.numel() * t.element_size() for t in itertools.chain(model.parameters(), model.buffers()))
make_room_on_gpu(weights + _WORKING_BYTES)
try:
model.to(device)
_move_tables(model, device)
except Exception:
offload(model) # a half-moved model would fail every later run
raise
def offload(model):
"""Move the weights and sam3's GPU-side tables to the CPU, so nothing holds VRAM between runs.
The position-encoding caches sam3 fills at build time (about 200 MB on the GPU) are emptied; sam3 rebuilds them
on the GPU when it needs them again.
"""
cpu = torch.device("cpu")
if not _all_on(model, cpu):
model.to(cpu)
_move_tables(model, cpu)
for module in model.modules():
if type(module).__name__ == "PositionEmbeddingSine" and isinstance(getattr(module, "cache", None), dict):
module.cache.clear()
if torch.cuda.is_available():
torch.cuda.empty_cache()
def unload():
"""Forget the cached model (another checkpoint was chosen)."""
model = _CACHE.pop("model", None)
_CACHE.pop("stamp", None)
if model is not None:
offload(model)
def _native_bf16():
try:
return torch.cuda.is_bf16_supported(including_emulation=False)
except TypeError: # torch without the including_emulation switch
return torch.cuda.is_bf16_supported()
def _autocast(device):
"""bfloat16 autocast on GPUs with native bfloat16 (Ampere and newer), as Meta runs SAM 3; full precision elsewhere."""
if device.type == "cuda" and _native_bf16():
return torch.autocast("cuda", dtype=torch.bfloat16)
return contextlib.nullcontext()
# ---------------------------------------------------------------------------
# Geometry helpers (model-free, unit tested)
# ---------------------------------------------------------------------------
def to_logits(masks):
"""sam3 returns its masks after a sigmoid (under the key "masks_logits"); turn them back into logits.
Values outside [0, 1] are taken as logits already, in case a later sam3 returns them as the key says.
"""
masks = masks.float()
if masks.numel() and (float(masks.min()) < 0.0 or float(masks.max()) > 1.0):
return masks
masks = masks.clamp(1e-6, 1.0 - 1e-6)
return torch.log(masks) - torch.log1p(-masks)
def sample_region(logits, image_size, region, out_size):
"""Resample the square model-space map ``logits`` [S, S] over ``region`` of the image, bilinearly.
``region`` is (x1, y1, x2, y2) in pixels of an image of ``image_size`` (height, width), which the model saw
squeezed into S x S; the result is [h, w] for ``out_size`` (h, w), each value taken at the centre of its output
pixel. Runs in row chunks, so a full 8K region needs a few hundred MB at most.
"""
height, width = image_size
size = logits.shape[-1]
x1, y1, x2, y2 = region
out_h, out_w = int(out_size[0]), int(out_size[1])
device = logits.device
xs = x1 + (torch.arange(out_w, device=device, dtype=torch.float32) + 0.5) * ((x2 - x1) / out_w)
ys = y1 + (torch.arange(out_h, device=device, dtype=torch.float32) + 0.5) * ((y2 - y1) / out_h)
# model-map pixel index of each centre, as grid_sample coordinates (align_corners=True: -1/+1 = first/last centre)
grid_x = (xs * (size / width) - 0.5) * (2.0 / max(1, size - 1)) - 1.0
grid_y = (ys * (size / height) - 0.5) * (2.0 / max(1, size - 1)) - 1.0
source = logits.float()[None, None]
result = torch.empty((out_h, out_w), device=device, dtype=torch.float32)
rows = max(1, _SAMPLE_CHUNK // max(1, out_w))
for start in range(0, out_h, rows):
gy = grid_y[start:start + rows]
grid = torch.stack(torch.broadcast_tensors(grid_x[None, :], gy[:, None]), dim=-1)[None]
result[start:start + rows] = F.grid_sample(source, grid, mode="bilinear", padding_mode="border",
align_corners=True)[0, 0]
return result
def mask_box(mask):
"""Tight (x1, y1, x2, y2) box of a bool [H, W] tensor with exclusive ends; None when empty."""
rows = torch.nonzero(mask.any(dim=1)).flatten()
if rows.numel() == 0:
return None
cols = torch.nonzero(mask.any(dim=0)).flatten()
return int(cols[0]), int(rows[0]), int(cols[-1]) + 1, int(rows[-1]) + 1
def model_box_to_image(box, size, image_size, margin=1):
"""An S x S model-space box as an image-space region, ``margin`` model pixels wider on each side, in the image."""
height, width = image_size
sx, sy = width / size, height / size
x1, y1, x2, y2 = box
return (max(0, int((x1 - margin) * sx)), max(0, int((y1 - margin) * sy)),
min(width, int(math.ceil((x2 + margin) * sx))), min(height, int(math.ceil((y2 + margin) * sy))))
def _fit_span(center, size, limit):
size = int(round(min(size, limit)))
start = int(round(center - size / 2.0))
start = min(max(0, start), limit - size)
return start, start + size
def crop_region(box, image_size):
"""Pass-2 crop around an image-space box: a margin on each side, widened to at most 2:1, kept inside the image."""
height, width = image_size
x1, y1, x2, y2 = box
margin = max(_CROP_MIN_MARGIN, _CROP_MARGIN * max(x2 - x1, y2 - y1))
crop_w, crop_h = (x2 - x1) + 2 * margin, (y2 - y1) + 2 * margin
crop_w, crop_h = max(crop_w, crop_h / _CROP_MAX_ASPECT), max(crop_h, crop_w / _CROP_MAX_ASPECT)
left, right = _fit_span((x1 + x2) / 2.0, crop_w, width)
top, bottom = _fit_span((y1 + y2) / 2.0, crop_h, height)
# the box itself always stays inside, whatever the rounding did
return min(left, x1), min(top, y1), max(right, x2), max(bottom, y2)
def drop_stray_parts(refined, coarse):
"""``refined`` without the connected parts that do not touch ``coarse`` at all (specks around the object)."""
count, labels = cv2.connectedComponents(refined.astype(np.uint8), connectivity=8)
if count <= 2:
return refined
touching = np.bincount(labels[coarse], minlength=count) > 0
touching[0] = False
return touching[labels]
def close_up_agrees(refined, coarse, tolerance):
"""True when the close-up mask ``refined`` shows the same object as the pass-1 mask ``coarse`` (bool arrays).
The pass-1 mask is blurred by up to ``tolerance`` px, so plain IoU would turn down a sharp mask of a thin pole or
cable. Instead most of the close-up must lie within ``tolerance`` of the pass-1 mask, and it must cover most of
the pass-1 core (pixels deeper than twice ``tolerance`` inside it); thin objects have no core and need only the
first. Large masks are compared at a reduced scale.
"""
if not refined.any() or not coarse.any():
return False
step = max(1, int(math.sqrt(refined.size / _AGREE_PIXELS)))
if step > 1 and not refined[::step, ::step].any():
step = 1 # something thinner than the step: compare at full scale
refined, coarse = refined[::step, ::step], coarse[::step, ::step]
tolerance = max(1.0, tolerance / step)
to_coarse = cv2.distanceTransform((~coarse).astype(np.uint8), cv2.DIST_L2, 3)
inside = np.count_nonzero(refined & (to_coarse <= tolerance)) / np.count_nonzero(refined)
core = cv2.distanceTransform(coarse.astype(np.uint8), cv2.DIST_L2, 3) > 2.0 * tolerance
covered = np.count_nonzero(refined & core) / np.count_nonzero(core) if core.any() else 1.0
return inside >= REFINE_MIN_INSIDE and covered >= REFINE_MIN_CORE
def _box_overlap(a, b):
"""Intersection box of two (x1, y1, x2, y2) boxes, or None."""
x1, y1, x2, y2 = max(a[0], b[0]), max(a[1], b[1]), min(a[2], b[2]), min(a[3], b[3])
return (x1, y1, x2, y2) if x1 < x2 and y1 < y2 else None
def find_instances(processor, state, concept, max_instances=0):
"""[(score, logits [S, S], model-space box)] for one concept on an encoded image, best first, duplicates dropped."""
processor.reset_all_prompts(state)
output = processor.set_text_prompt(concept, state)
masks, scores = output["masks_logits"], output["scores"]
if masks is None or len(scores) == 0:
return []
scores = scores.float().flatten()
found, kept = [], [] # kept: (bool mask, box, area) of accepted instances
for i in torch.argsort(scores, descending=True).tolist():
logits = to_logits(masks[i].reshape(masks.shape[-2], masks.shape[-1]))
inside = logits > 0
box = mask_box(inside)
if box is None:
continue
area = int(inside.sum())
duplicate = False
for other, other_box, other_area in kept:
overlap = _box_overlap(box, other_box)
if overlap is None:
continue
ox1, oy1, ox2, oy2 = overlap
inter = int((inside[oy1:oy2, ox1:ox2] & other[oy1:oy2, ox1:ox2]).sum())
if inter / max(1, area + other_area - inter) >= DEDUP_IOU:
duplicate = True
break
if duplicate:
continue
kept.append((inside, box, area))
found.append((float(scores[i]), logits, box))
if max_instances and len(found) >= max_instances:
break
return found
# ---------------------------------------------------------------------------
# Segmenting
# ---------------------------------------------------------------------------
def _hint_size(model):
"""Side of the low-resolution mask the box-prompt decoder takes as a hint (288 for SAM 3)."""
encoder = getattr(getattr(getattr(model, "inst_interactive_predictor", None), "model", None),
"sam_prompt_encoder", None)
size = getattr(encoder, "mask_input_size", (288, 288))
return int(size[0]), int(size[1])
def _upsample(concept, score, logits, box, image_size):
"""Pass 1 only: the instance's mask at full resolution, inside its own region."""
region = model_box_to_image(box, logits.shape[-1], image_size)
x1, y1, x2, y2 = region
mask = sample_region(logits, image_size, region, (y2 - y1, x2 - x1)) > 0
return make_segment(concept, score, mask, origin=(x1, y1)), False
def _refine(model, processor, image, concept, score, logits, box, image_size, hint_size):
"""Pass 2: segment the instance again on a full-resolution crop, the pass-1 mask as a hint.
Returns (segment, refined) where ``refined`` is False when the pass-1 mask was kept.
"""
target = model_box_to_image(box, logits.shape[-1], image_size)
cx1, cy1, cx2, cy2 = crop_region(target, image_size)
coarse = (sample_region(logits, image_size, (cx1, cy1, cx2, cy2), (cy2 - cy1, cx2 - cx1)) > 0).cpu().numpy()
hint = sample_region(logits, image_size, (cx1, cy1, cx2, cy2), hint_size).clamp(-20.0, 20.0)
state = processor.set_image(image[:, cy1:cy2, cx1:cx2])
x1, y1, x2, y2 = target
prompt_box = np.array([x1 - cx1, y1 - cy1, x2 - cx1, y2 - cy1], dtype=np.float32)
masks, _, _ = model.predict_inst(state, box=prompt_box, mask_input=hint[None], multimask_output=False,
return_logits=True)
refined = np.asarray(masks).reshape(-1, cy2 - cy1, cx2 - cx1)[0] > 0
refined = drop_stray_parts(refined, coarse)
accepted = close_up_agrees(refined, coarse, max(image_size) / _PASS1_GRID + 2.0)
return make_segment(concept, score, refined if accepted else coarse, origin=(cx1, cy1)), accepted
def _image_chw(image):
"""One ComfyUI image [H, W, C] -> float [3, H, W] in [0, 1]."""
channels = image.shape[-1]
if channels == 1:
image = image.expand(-1, -1, 3)
elif channels not in (3, 4):
raise ValueError(f"[NH-Nodes] SAM 3 expects an IMAGE with 1, 3 or 4 channels, got {channels}")
return image[..., :3].permute(2, 0, 1).float().clamp(0.0, 1.0)
def segment_images(model, make_processor, images, concepts, threshold=0.5, detail_level="balanced",
max_per_concept=0, device=None, interrupt=None, progress=None):
"""NH_SEGS (one SegsFrame per image) for ``concepts`` found in the ComfyUI IMAGE batch ``images``.
``make_processor(threshold)`` returns a Sam3Processor for ``model`` on ``device``; the model must already be
there. Instances come in concept order, best score first within a concept. ``interrupt()`` runs before each
concept and each close-up (it raises to stop), ``progress(done, total)`` after each concept of each image.
"""
device = device or _model_device(model)
refine = detail_level != "fast"
hint_size = _hint_size(model) if refine else None
frames, kept_pass1, warned_wide = [], 0, False
done, total = 0, len(images) * len(concepts)
with torch.inference_mode(), _autocast(device):
for image in images:
height, width = int(image.shape[0]), int(image.shape[1])
if not warned_wide and max(height, width) > _WIDE_ASPECT * min(height, width):
warned_wide = True
print(f"[NH-Nodes] SAM 3 sees the whole {width}x{height} picture squeezed into a square, so on images "
"wider than 4:1 it can miss small things; split such a picture into tiles for better results")
pixels = _image_chw(image).to(device)
processor = make_processor(threshold)
size = int(getattr(processor, "resolution", 1008))
square = F.interpolate(pixels[None], size=(size, size), mode="bilinear", antialias=True,
align_corners=False)[0].clamp_(0.0, 1.0)
state = processor.set_image(square)
segments = []
for concept in concepts:
if interrupt is not None:
interrupt()
for score, logits, box in find_instances(processor, state, concept, max_per_concept):
if refine:
if interrupt is not None:
interrupt()
seg, refined = _refine(model, processor, pixels, concept, score, logits, box,
(height, width), hint_size)
kept_pass1 += 0 if refined else 1
else:
seg, _ = _upsample(concept, score, logits, box, (height, width))
if seg is not None:
segments.append(seg)
done += 1
if progress is not None:
progress(done, total)
frames.append(SegsFrame(height, width, tuple(segments)))
del pixels, square, state
if kept_pass1:
print(f"[NH-Nodes] SAM 3: kept the first-pass mask for {kept_pass1} instance(s) whose close-up disagreed")
return tuple(frames)
def segment(images, concepts, threshold=0.5, detail_level="balanced", max_per_concept=0, checkpoint=DEFAULT_CHECKPOINT,
keep_model_loaded=False, interrupt=None, progress=None):
"""Load SAM 3 and run ``segment_images`` on ComfyUI's GPU; the weights go back to RAM afterwards unless kept."""
if images.shape[0] == 0:
raise ValueError("[NH-Nodes] SAM 3 got an empty image batch")
model = load_model(checkpoint)
_, processor_cls = ensure_sam3()
device = sam3_device()
try:
move_model(model, device)
return segment_images(
model, lambda value: processor_cls(model, device=device, confidence_threshold=value), images, concepts,
threshold, detail_level, max_per_concept, device, interrupt, progress,
)
finally:
if not keep_model_loaded:
offload(model)
+515
View File
@@ -0,0 +1,515 @@
"""NH_SEGS: the instances the NH vision nodes pass between each other.
An ``NH_SEGS`` value is a tuple with one ``SegsFrame`` per image of a batch. A frame holds the image size and the
``Segment`` instances found in that image. A segment keeps its label, score and box, and only the mask pixels inside
its box, so dozens of instances on an 8K image stay small. Full-size masks are built only when a node outputs a MASK.
Plain torch/numpy, no model: the select, compose, preview and convert nodes and their tests run anywhere.
"""
import fnmatch
import functools
from collections import namedtuple
from dataclasses import dataclass
import cv2
import numpy as np
import torch
from PIL import Image, ImageDraw, ImageFont
SEGS_TYPE = "NH_SEGS"
@dataclass(frozen=True, eq=False)
class Segment:
"""One instance found in an image.
``box`` is (x1, y1, x2, y2) in image pixels with exclusive ends, so the instance covers ``image[y1:y2, x1:x2]``;
it is the tight box around the mask. ``mask`` is a bool [y2 - y1, x2 - x1] CPU tensor over that box.
"""
label: str
score: float
box: tuple
mask: torch.Tensor
source: str = "sam3"
@property
def area(self):
"""Number of mask pixels."""
return int(self.mask.sum())
@property
def center(self):
x1, y1, x2, y2 = self.box
return (x1 + x2) / 2.0, (y1 + y2) / 2.0
@dataclass(frozen=True, eq=False)
class SegsFrame:
"""The instances found in one image of ``height`` x ``width`` pixels, in the order they were found."""
height: int
width: int
segments: tuple = ()
def with_segments(self, segments):
return SegsFrame(self.height, self.width, tuple(segments))
def _comfy_cache_tensors(self):
"""The tensors this frame holds, so ComfyUI's RAM-pressure cache can size it."""
return [seg.mask for seg in self.segments]
def make_segment(label, score, mask, origin=(0, 0), source="sam3"):
"""Segment for the True pixels of the bool [h, w] ``mask`` whose top-left pixel sits at ``origin`` (x, y) of the image.
The stored box is the tight box around those pixels and the stored mask a compact CPU copy of that box. Works on
any device, cropping before the copy to the CPU. Returns None when the mask is empty.
"""
mask = torch.as_tensor(mask)
if mask.dtype != torch.bool:
mask = mask > 0.5
rows = torch.nonzero(mask.any(dim=1)).flatten()
if rows.numel() == 0:
return None
cols = torch.nonzero(mask.any(dim=0)).flatten()
y1, y2 = int(rows[0]), int(rows[-1]) + 1
x1, x2 = int(cols[0]), int(cols[-1]) + 1
crop = mask[y1:y2, x1:x2].to("cpu").clone(memory_format=torch.contiguous_format)
ox, oy = int(origin[0]), int(origin[1])
return Segment(str(label), float(score), (ox + x1, oy + y1, ox + x2, oy + y2), crop, source)
def frame_mask(frame, segments=None):
"""Bool [H, W] union of ``segments`` (every segment of ``frame`` by default)."""
canvas = torch.zeros((frame.height, frame.width), dtype=torch.bool)
for seg in frame.segments if segments is None else segments:
x1, y1, x2, y2 = seg.box
canvas[y1:y2, x1:x2] |= seg.mask
return canvas
def segs_union(segs):
"""MASK [B, H, W] float: the union of every instance, one mask per image."""
return torch.stack([frame_mask(frame) for frame in segs]).float()
def check_memory(nbytes, what):
"""Raise MemoryError with advice when ``nbytes`` would take more than half of the RAM that is free now."""
try:
import psutil
available = psutil.virtual_memory().available
except Exception: # no psutil: nothing to compare with
return
if nbytes > available // 2:
raise MemoryError(
f"[NH-Nodes] {what} would need {nbytes / 2**30:.1f} GiB but only {available / 2**30:.1f} GiB of RAM is "
"free. Keep fewer instances with Segs Select, or combine them into one mask with Segs Compose."
)
def instance_masks(segs):
"""(MASK [N, H, W] float with one full-size mask per instance of every image, in order; the labels).
With no instance at all the MASK holds a single empty mask, as nodes downstream may not accept an empty batch.
Each full-size mask takes H x W x 4 bytes (about 145 MB on an 8K image), so a MemoryError explains what to do
when they would not fit in RAM.
"""
labels = [seg.label for frame in segs for seg in frame.segments]
height, width = segs[0].height, segs[0].width
check_memory(max(1, len(labels)) * height * width * 4, f"{len(labels)} full-size instance masks")
masks = torch.zeros((max(1, len(labels)), height, width))
index = 0
for frame in segs:
for seg in frame.segments:
x1, y1, x2, y2 = seg.box
masks[index, y1:y2, x1:x2] = seg.mask.float()
index += 1
return masks, labels
def count_segments(segs):
return sum(len(frame.segments) for frame in segs)
# ---------------------------------------------------------------------------
# Labels
# ---------------------------------------------------------------------------
def normalize_label(text):
"""Lower case, '_' as a space, single spaces: "Red_Car " and "red car" both become "red car"."""
return " ".join(str(text).replace("_", " ").lower().split())
def label_matches(label, pattern):
"""True when ``label`` matches ``pattern``, ignoring case; the pattern may use * and ? wildcards."""
return fnmatch.fnmatchcase(normalize_label(label), normalize_label(pattern))
def parse_label_list(text):
"""Commas and new lines separate entries: "person, red car" -> ["person", "red car"].
Blank entries and lines starting with '#' are dropped; a repeated entry (ignoring case and spacing) is kept once,
with its first spelling.
"""
entries, seen = [], set()
for line in str(text or "").splitlines():
if line.strip().startswith("#"):
continue
for part in line.split(","):
entry = " ".join(part.split())
key = normalize_label(entry)
if entry and key not in seen:
seen.add(key)
entries.append(entry)
return entries
# ---------------------------------------------------------------------------
# Select
# ---------------------------------------------------------------------------
SORT_ORDERS = ("score", "area", "left_to_right", "right_to_left", "top_to_bottom", "bottom_to_top",
"center_first", "as_found")
def sort_segments(segments, order, frame):
"""``segments`` in ``order`` (one of SORT_ORDERS); positions use each box's centre. Ties keep their found order."""
segments = list(segments)
if order == "score":
return sorted(segments, key=lambda s: -s.score)
if order == "area":
return sorted(segments, key=lambda s: -s.area)
if order == "left_to_right":
return sorted(segments, key=lambda s: s.center[0])
if order == "right_to_left":
return sorted(segments, key=lambda s: -s.center[0])
if order == "top_to_bottom":
return sorted(segments, key=lambda s: s.center[1])
if order == "bottom_to_top":
return sorted(segments, key=lambda s: -s.center[1])
if order == "center_first":
cx, cy = frame.width / 2.0, frame.height / 2.0
return sorted(segments, key=lambda s: (s.center[0] - cx) ** 2 + (s.center[1] - cy) ** 2)
return segments
def select_segments(frame, labels=(), min_score=0.0, min_area=0.0, max_area=1.0, order="score", start=0, count=0):
"""Frame with the segments that pass the filters, sorted by ``order``, from position ``start`` (0 = first).
``labels`` are patterns (any one must match; none = every label). Areas are fractions of the image. ``count`` = 0
keeps every segment from ``start`` on.
"""
total = max(1, frame.height * frame.width)
kept = []
for seg in frame.segments:
if labels and not any(label_matches(seg.label, pattern) for pattern in labels):
continue
if seg.score < min_score:
continue
if not min_area <= seg.area / total <= max_area:
continue
kept.append(seg)
kept = sort_segments(kept, order, frame)
start = max(0, int(start))
kept = kept[start:] if count <= 0 else kept[start:start + int(count)]
return frame.with_segments(kept)
# ---------------------------------------------------------------------------
# Compose: a small boolean language over labels
# ---------------------------------------------------------------------------
class ExpressionError(ValueError):
"""An NH_SEGS mask expression that cannot be read."""
_OPERATORS = {"+": "|", "|": "|", "-": "-", "&": "&", "~": "~", "!": "~", "(": "(", ")": ")"}
_WORD_OPERATORS = {"or": "|", "and": "&", "not": "~"}
_KEYWORDS = ("all", "image", "none")
_QUOTES = "\"'"
def _is_word_char(ch):
return ch.isalnum() or ch == "_"
def _tokenize(text):
"""[(kind, value, position, written)] with kinds "op", "keyword", "label"; positions count from 1.
A quote opens a quoted label only where a token starts, and a '-' between two word characters is part of the
word, so ``women's`` and ``t-shirt`` are plain labels while ``person - hair`` subtracts.
"""
tokens, i, n = [], 0, len(text)
while i < n:
ch = text[i]
if ch.isspace():
i += 1
elif ch in _OPERATORS:
tokens.append(("op", _OPERATORS[ch], i + 1, ch))
i += 1
elif ch in _QUOTES:
end = text.find(ch, i + 1)
if end < 0:
raise ExpressionError(f"quote opened at position {i + 1} is never closed")
tokens.append(("label", text[i + 1:end], i + 1, text[i:end + 1]))
i = end + 1
else:
j = i
while j < n and not text[j].isspace():
hyphen_in_word = (text[j] == "-" and _is_word_char(text[j - 1])
and j + 1 < n and _is_word_char(text[j + 1]))
if text[j] in _OPERATORS and not hyphen_in_word:
break
j += 1
word = text[i:j]
lowered = word.lower()
if lowered in _WORD_OPERATORS:
tokens.append(("op", _WORD_OPERATORS[lowered], i + 1, word))
elif lowered in _KEYWORDS:
tokens.append(("keyword", lowered, i + 1, word))
else:
tokens.append(("label", word, i + 1, word))
i = j
return tokens
class _Parser:
"""expr := term (('|' | '-') term)* ; term := unary ('&' unary)* ; unary := '~' unary | atom ;
atom := label | keyword | '(' expr ')'. Builds a tuple tree evaluated by ``evaluate``."""
def __init__(self, text):
self.tokens = _tokenize(text)
self.pos = 0
def _peek(self):
return self.tokens[self.pos] if self.pos < len(self.tokens) else None
def _where(self):
token = self._peek()
return f"'{token[3]}' at position {token[2]}" if token else "the end"
def parse(self):
if not self.tokens:
raise ExpressionError("the expression is empty; use 'all' for every instance")
tree = self._expr()
if self._peek() is not None:
raise ExpressionError(f"unexpected {self._where()}")
return tree
def _expr(self):
tree = self._term()
while (token := self._peek()) is not None and token[0] == "op" and token[1] in ("|", "-"):
self.pos += 1
tree = (token[1], tree, self._term())
return tree
def _term(self):
tree = self._unary()
while (token := self._peek()) is not None and token[0] == "op" and token[1] == "&":
self.pos += 1
tree = ("&", tree, self._unary())
return tree
def _unary(self):
token = self._peek()
if token is not None and token[0] == "op" and token[1] == "~":
self.pos += 1
return ("~", self._unary())
return self._atom()
def _atom(self):
token = self._peek()
if token is None:
raise ExpressionError("the expression ends where a label, 'all', 'image', 'none' or '(' was expected")
kind, value, position, written = token
self.pos += 1
if kind == "label":
return ("label", value)
if kind == "keyword":
return ("keyword", value)
if value == "(":
tree = self._expr()
closing = self._peek()
if closing is None or closing[:2] != ("op", ")"):
raise ExpressionError(f"'(' at position {position} is not closed (found {self._where()})")
self.pos += 1
return tree
raise ExpressionError(f"unexpected '{written}' at position {position}")
@functools.lru_cache(maxsize=64)
def parse_expression(text):
"""Parse a mask expression (see ``evaluate``) into a tree; raises ExpressionError with the position on mistakes."""
return _Parser(str(text)).parse()
def expression_labels(tree):
"""Label patterns used in a parsed expression, in order."""
if tree[0] == "label":
return [tree[1]]
if tree[0] == "keyword":
return []
return [label for child in tree[1:] for label in expression_labels(child)]
def evaluate(tree, frame):
"""Bool [H, W] mask of a parsed expression for one frame.
A label (wildcards allowed; quotes or '_' for spaces; a '-' inside a word is part of it) is the union of the
matching instances,
``all`` every instance, ``image`` the whole frame, ``none`` nothing. ``a | b`` (also ``+``, ``or``) is the union,
``a - b`` the difference, ``a & b`` (``and``) the intersection, ``~a`` (``!``, ``not``) everything outside ``a``.
"""
kind = tree[0]
if kind == "label":
return frame_mask(frame, [seg for seg in frame.segments if label_matches(seg.label, tree[1])])
if kind == "keyword":
if tree[1] == "all":
return frame_mask(frame)
if tree[1] == "image":
return torch.ones((frame.height, frame.width), dtype=torch.bool)
return torch.zeros((frame.height, frame.width), dtype=torch.bool)
if kind == "~":
return ~evaluate(tree[1], frame)
left, right = evaluate(tree[1], frame), evaluate(tree[2], frame)
if kind == "|":
return left | right
if kind == "&":
return left & right
return left & ~right
def expression_masks(tree, segs):
"""MASK [B, H, W] float of a parsed expression, one mask per image of ``segs``."""
return torch.stack([evaluate(tree, frame) for frame in segs]).float()
# ---------------------------------------------------------------------------
# Impact Pack SEGS
# ---------------------------------------------------------------------------
# The field names of Impact Pack's SEG, so its detailers read these items; NH-Nodes does not import Impact Pack.
IMPACT_SEG = namedtuple("SEG", ["cropped_image", "cropped_mask", "confidence", "crop_region", "bbox", "label",
"control_net_wrapper"], defaults=[None])
def _centred_span(lo, hi, factor, limit):
"""[start, end) of ``factor`` times the length of [lo, hi), centred on it and slid inside [0, limit).
It is never shorter than [lo, hi) and only shorter than asked when the image is, so it always contains [lo, hi).
"""
size = min(limit, max(hi - lo, round((hi - lo) * factor)))
start = min(max(0, round((lo + hi - size) / 2)), limit - size)
return start, start + size
def detail_crop_region(width, height, box, crop_factor):
"""[x1, y1, x2, y2] a detailer works on around ``box``: ``crop_factor`` times its size, centred, inside the image."""
rx1, rx2 = _centred_span(box[0], box[2], crop_factor, width)
ry1, ry2 = _centred_span(box[1], box[3], crop_factor, height)
return [rx1, ry1, rx2, ry2]
def to_impact_segs(frame, crop_factor=3.0):
"""Impact Pack SEGS ``((height, width), [SEG, ...])`` for one frame; each mask is placed in its crop region."""
items = []
for seg in frame.segments:
region = detail_crop_region(frame.width, frame.height, seg.box, crop_factor)
rx1, ry1, rx2, ry2 = region
x1, y1, x2, y2 = seg.box
cropped = np.zeros((ry2 - ry1, rx2 - rx1), dtype=np.float32)
cropped[y1 - ry1:y2 - ry1, x1 - rx1:x2 - rx1] = seg.mask.numpy()
items.append(IMPACT_SEG(None, cropped, float(seg.score), region, [x1, y1, x2, y2], seg.label, None))
return (frame.height, frame.width), items
# ---------------------------------------------------------------------------
# Preview
# ---------------------------------------------------------------------------
_PALETTE = (
(230, 25, 75), (60, 180, 75), (255, 225, 25), (0, 130, 200), (245, 130, 48), (145, 30, 180),
(70, 240, 240), (240, 50, 230), (210, 245, 60), (250, 190, 212), (0, 128, 128), (170, 110, 40),
)
@functools.lru_cache(maxsize=16)
def _label_font(size):
for name in ("arial.ttf", "DejaVuSans.ttf"): # both cover Vietnamese
try:
return ImageFont.truetype(name, size)
except Exception:
continue
try:
return ImageFont.load_default(size=size)
except TypeError: # Pillow < 10.1
return ImageFont.load_default()
def _free_spot(x, y, width, height, placed, limit):
"""``y`` moved down (or up near the bottom ``limit``) until a ``width`` x ``height`` label at ``x`` overlaps none
of the ``placed`` (x1, y1, x2, y2) rectangles; labels of instances sharing a top edge (person, hair) then stack."""
for _ in range(len(placed)):
hit = next((r for r in placed if x < r[2] and r[0] < x + width and y < r[3] and r[1] < y + height), None)
if hit is None:
break
y = hit[3] + 1 if hit[3] + 1 + height <= limit else max(0, hit[1] - height - 1)
return y
def render_preview(image, frame, fill=0.45):
"""uint8 HxWx3 copy of ``image`` with each instance tinted, outlined and labelled "label score"."""
canvas = np.ascontiguousarray(image[:, :, :3]).copy()
height, width = canvas.shape[:2]
if not frame.segments:
return canvas
thickness = max(1, round(min(height, width) / 700))
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (2 * thickness + 1, 2 * thickness + 1))
for index, seg in enumerate(frame.segments):
color = np.array(_PALETTE[index % len(_PALETTE)], dtype=np.float32)
x1, y1, x2, y2 = seg.box
region = canvas[y1:y2, x1:x2]
inside = seg.mask.numpy()
region[inside] = (region[inside] * (1.0 - fill) + color * fill).astype(np.uint8)
padded = np.pad(inside.astype(np.uint8), thickness) # so a box edge counts as an outline too
edge = (padded > 0) & (cv2.erode(padded, kernel) == 0)
region[edge[thickness:-thickness, thickness:-thickness]] = color.astype(np.uint8)
font = _label_font(max(12, round(min(height, width) / 45)))
picture = Image.fromarray(canvas)
draw = ImageDraw.Draw(picture)
placed = [] # label rectangles drawn so far, so a label never hides another one
for index, seg in enumerate(frame.segments):
color = _PALETTE[index % len(_PALETTE)]
text = f"{seg.label} {seg.score:.2f}"
left, top, right, bottom = draw.textbbox((0, 0), text, font=font)
pad = max(2, (bottom - top) // 6)
tw, th = right - left + 2 * pad, bottom - top + 2 * pad
# centred over the instance's topmost pixels (a head, a roof), not at a box corner that may be far from it
top_columns = torch.nonzero(seg.mask[0]).flatten()
anchor = seg.box[0] + int(top_columns[top_columns.numel() // 2]) if top_columns.numel() else seg.box[0]
x = min(max(0, anchor - tw // 2), max(0, width - tw))
y = _free_spot(x, seg.box[1] - th if seg.box[1] - th >= 0 else seg.box[1], tw, th, placed, height)
placed.append((x, y, x + tw, y + th))
draw.rectangle((x, y, x + tw, y + th), fill=color)
dark_text = (0.299 * color[0] + 0.587 * color[1] + 0.114 * color[2]) > 150
draw.text((x + pad - left, y + pad - top), text, font=font, fill=(0, 0, 0) if dark_text else (255, 255, 255))
return np.asarray(picture)
def preview_batch(image, segs):
"""IMAGE batch with every instance of ``segs`` drawn on its image (tinted, outlined, labelled)."""
if len(segs) != image.shape[0]:
raise ValueError(
f"[NH-Nodes] The image batch has {image.shape[0]} image(s) but the segs belong to {len(segs)}; "
"connect the image the segs were found in."
)
pictures = []
for picture, frame in zip(image, segs):
if (picture.shape[0], picture.shape[1]) != (frame.height, frame.width):
raise ValueError(
f"[NH-Nodes] The image is {picture.shape[1]}x{picture.shape[0]} but the segs were found on a "
f"{frame.width}x{frame.height} image; connect the image the segs were found in."
)
rgb = picture[..., :3] if picture.shape[-1] >= 3 else picture[..., :1].expand(-1, -1, 3)
pixels = (rgb.float().clamp(0.0, 1.0) * 255.0).round().to(torch.uint8).cpu().numpy()
pictures.append(torch.from_numpy(render_preview(pixels, frame).astype(np.float32) / 255.0))
return torch.stack(pictures)
+24
View File
@@ -3,6 +3,30 @@
All notable changes to NH-Nodes. Versions follow the Comfy Registry releases.
"Behaviour change" marks fixes that can change the output of an existing workflow.
## 1.7.0 — SAM 3 segmentation and instance nodes (2026-10-18)
### Added
Six nodes under **NH-Nodes → Vision** (75 nodes in total) and a new data type, `NH_SEGS`. Nothing in the existing nodes changes.
- `SAM3 Segment (NH)`: finds every instance of the concepts you name (short English noun phrases, one per line or separated by commas) with Meta's SAM 3. Outputs the instances (`segs`), their union `mask` and the `count`; `Segs Preview (NH)` shows what it found. `detail_level` `balanced` (default) segments each instance a second time on a close-up cut from the full-resolution image, so small and medium objects get crisp edges even on 4K to 8K photos; `fast` keeps the whole-image pass only. Other inputs: `threshold`, `max_per_concept`, `checkpoint` and `keep_model_loaded`.
- `NH_SEGS`: for every image of a batch, a list of instances with a label, a score, a tight box and the mask inside that box only, so dozens of instances on an 8K photo stay light. Full-size masks are built only when a node outputs a `MASK`.
- `Segs Select (NH)`: keeps instances by label (`*` and `?` wildcards), score and size, sorts them (score, area, position, centre first, as found) and takes a slice, for example the second person from the left.
- `Segs Compose (NH)`: one mask from a formula over labels: `|` `+` `or` (union), `-` (minus), `&` `and` (overlap), `~` `!` `not` (everything except), parentheses, and `all`, `image`, `none`. `~person` is the background, `(wall | ceiling) - window` the walls without the windows. A hyphen inside a word belongs to the label (`t-shirt` needs no quotes), so put spaces around a minus: `person - hair`.
- `Segs To Masks (NH)`: one mask per instance, the union per image, the labels and the count. It stops with a message when the masks would need more than about half of the free RAM, for example hundreds of instances on an 8K photo.
- `Segs To Impact SEGS (NH)`: Impact Pack's `SEGS` for one image of the batch (`crop_factor` 3.0 like Impact Pack). Impact Pack is not needed for anything else in NH-Nodes.
- `Segs Preview (NH)`: draws the instances (tint, outline, label and score) on the picture they were found in.
### Notes
- SAM3 Segment needs an NVIDIA GPU with CUDA: the official sam3 code builds part of the model on the GPU, so there is no CPU mode.
- NH-Nodes does not install or download anything for it. You install Meta's `sam3` package yourself (the Vision guide gives the command, pinned to the commit NH-Nodes was tested with; newer commits with the same image API also work), and the weights are gated on Hugging Face: request access, accept Meta's SAM License and put `sam3.pt` in `models/sam3` (or a folder registered as `sam3` in `extra_model_paths.yaml`). The folder is shared with other SAM 3 packs, so an existing copy is found. The licence is in `THIRD_PARTY_NOTICES.md`.
- The model stays in system memory between runs and moves to the GPU only while the node runs, unless `keep_model_loaded` is on.
- `detail_level` has two levels for now, `balanced` and `fast`.
- SAM 3 describes whatever it looks at on a grid of about 288 cells. A subject that fills most of an 8K frame therefore gets edges at about 1/300 of its size: fine for the outline of a person, soft on hair, lace and fur. Hair-level edges are not part of this release.
- SAM 3 first sees the whole picture squeezed into a square, so on a picture wider or taller than 4:1 it can miss small things. The console says so; split such a picture into tiles with `Image Tile (NH)`.
- A file in the `sam3` folder that is not a SAM 3 checkpoint, or an incomplete download, is refused with the download advice before anything runs.
### Planned
Not part of this release, and the names may change: scene parsing that labels the whole picture (sky, wall, floor…), automatic concept tagging with ready presets and a report, matte refinement for hair and fur edges (it will add a `max` detail level), and parsing of each person's parts.
## 1.6.2 — Load images matching picks the right group
- `Load images matching` (behaviour change): when the source and the matching library share parent folders (for example
+54 -5
View File
@@ -1,14 +1,15 @@
"""Build the example pictures of the NH-Nodes guides by running the real node code on open-licensed sample photos.
Run with ComfyUI's Python from the ComfyUI root:
python custom_nodes/NH-Nodes/docs/_tools/build_examples.py [assets_dir] [--only a.jpg,b.jpg]
python custom_nodes/NH-Nodes/docs/_tools/build_examples.py [assets_dir] [--only a.jpg,b.jpg] [--gpu]
It writes <assets_dir>/examples/*.jpg and <assets_dir>/banner.jpg (default assets_dir: docs/assets of the pack).
Every picture is made from the live node classes, called the way ComfyUI calls them, so the examples cannot drift
from the code: rerun the script after a node changes. The photos are the public-domain / CC0 sample images that ship
with scikit-image (astronaut, coffee, chelsea, rocket). The script is deterministic and runs on the CPU only: the GPU
is hidden before torch loads, so a ComfyUI server that is running at the same time is never disturbed. An example whose
models are missing is skipped (and reported); nothing is downloaded.
is hidden before torch loads, so a ComfyUI server that is running at the same time is never disturbed. The one
exception is the SAM 3 example, which needs a CUDA GPU: it is built only with --gpu, which leaves the GPU visible (SAM 3
goes back to RAM after its run). An example whose models are missing is skipped (and reported); nothing is downloaded.
"""
import argparse
@@ -23,7 +24,9 @@ import warnings
from dataclasses import dataclass
from pathlib import Path
os.environ["CUDA_VISIBLE_DEVICES"] = "-1" # CPU only. (An empty value would be dropped by Windows, so use -1.)
GPU = "--gpu" in sys.argv[1:] # read before torch loads: only the SAM 3 example needs the GPU
if not GPU:
os.environ["CUDA_VISIBLE_DEVICES"] = "-1" # CPU only. (An empty value would be dropped by Windows, so use -1.)
warnings.filterwarnings("ignore")
if hasattr(sys.stdout, "reconfigure"):
sys.stdout.reconfigure(encoding="utf-8", errors="replace")
@@ -46,7 +49,7 @@ from PIL import Image, ImageChops, ImageDraw, ImageFilter, ImageFont # noqa: E4
from comfy.cli_args import args # noqa: E402
args.cpu = True # before the pack imports comfy.model_management
args.cpu = not GPU # before the pack imports comfy.model_management
torch.set_num_threads(max(1, min(8, os.cpu_count() or 1))) # leave the rest of the machine to its other users
PACK_DIR = COMFY_ROOT / "custom_nodes" / "NH-Nodes"
@@ -1056,6 +1059,51 @@ def ex_vton_masks():
"pose, body parsing and the try-on mask of a person photo")
SAM3_CONCEPTS = "person, helmet, face"
@example("sam3-segment.jpg")
def ex_sam3_segment():
if not GPU or not torch.cuda.is_available():
raise Skip("SAM 3 needs a CUDA GPU: run with --gpu")
if importlib.util.find_spec("sam3") is None:
raise Skip("the sam3 package is not installed")
try:
importlib.import_module("NH-Nodes.core.sam3_backend").resolve_checkpoint("sam3.pt")
except RuntimeError:
raise Skip("models/sam3/sam3.pt is missing") from None
image = photo("astronaut")
segs, _mask, count = run("NH_SAM3Segment", image=to_tensor(image), concepts=SAM3_CONCEPTS, threshold=0.5,
detail_level="balanced", max_per_concept=0, checkpoint="sam3.pt",
keep_model_loaded=False)
(preview,) = run("NH_SegsPreview", image=to_tensor(image), segs=segs)
def compose(expression):
(mask,) = run("NH_SegsCompose", segs=segs, expression=expression)
return mask_array(mask)
_biggest, biggest_mask, biggest_count = run("NH_SegsSelect", segs=segs, labels="helmet", min_score=0.0,
min_area=0.0, max_area=1.0, sort_by="area", start=0, count=1)
if biggest_count != 1:
raise RuntimeError("SAM 3 found no helmet in the astronaut photo")
top = Row(
[Tile(image, "image", "input photo"), ARROW,
Tile(to_pil(preview), "Segs Preview", f"concepts: {SAM3_CONCEPTS}")],
"SAM3 Segment (NH) → Segs Preview (NH)", f"{count} instances · threshold 0.5 · detail_level balanced",
max_h=470,
)
bottom = Row(
[Tile(masked(image, Layer(compose("person - face"), TEAL, 0.45, 2.0)), "person - face",
"Segs Compose: the person without the face", aspect=1.0),
Tile(masked(image, Layer(compose("~person"), TEAL, 0.45, 2.0)), "~person",
"Segs Compose: everything except the person", aspect=1.0),
Tile(masked(image, Layer(compose("helmet"), TEAL, 0.0, 1.6), Layer(mask_array(biggest_mask), AMBER, 0.5, 2.2)),
"Segs Select", "labels helmet · sort_by area · count 1: the biggest helmet", aspect=1.0)],
"Segs Compose and Segs Select (NH)", "masks from the labels", max_h=330,
)
return render_panel([top, bottom])
# ---------------------------------------------------------------------------------------------------------------
# Banner
# ---------------------------------------------------------------------------------------------------------------
@@ -1204,6 +1252,7 @@ def main():
parser = argparse.ArgumentParser(description=__doc__.split("\n")[0])
parser.add_argument("assets_dir", nargs="?", default=str(PACK_DIR / "docs" / "assets"))
parser.add_argument("--only", default="", help="comma separated file names to build (default: all)")
parser.add_argument("--gpu", action="store_true", help="leave the GPU visible so the SAM 3 example is built too")
options = parser.parse_args()
assets = Path(options.assets_dir)
wanted = {name.strip() for name in options.only.split(",") if name.strip()}
Binary file not shown.

After

Width:  |  Height:  |  Size: 193 KiB

+42
View File
@@ -0,0 +1,42 @@
<svg xmlns="http://www.w3.org/2000/svg" width="360" height="338" viewBox="0 0 360 338" font-family="Inter, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif" role="img" aria-label="SAM3 Segment (NH) node">
<rect x="0.5" y="0.5" width="359" height="337" rx="10" fill="#26282c" stroke="#45484f"/>
<path d="M0.5 10.5 a10 10 0 0 1 10 -10 h339 a10 10 0 0 1 10 10 v36 h-359 z" fill="#34373d"/>
<circle cx="17" cy="17" r="4.5" fill="#8FD18F"/>
<text x="28" y="22" font-size="14" font-weight="600" fill="#EEF0F3">SAM3 Segment (NH)</text>
<text x="28" y="38" font-size="10.5" fill="#9AA0A8" font-family="'JetBrains Mono', Consolas, 'Courier New', monospace">NH_SAM3Segment · Vision</text>
<circle cx="12" cy="67.0" r="5" fill="#64B5F6" stroke="#1b1c1f" stroke-width="1.5"/>
<text x="24" y="71.0" font-size="12" fill="#D7DADF">image</text>
<text x="336" y="71.0" font-size="12" fill="#D7DADF" text-anchor="end">segs</text>
<circle cx="348" cy="67.0" r="5" fill="#E0B272" stroke="#1b1c1f" stroke-width="1.5"/>
<text x="336" y="93.0" font-size="12" fill="#D7DADF" text-anchor="end">mask</text>
<circle cx="348" cy="89.0" r="5" fill="#81C784" stroke="#1b1c1f" stroke-width="1.5"/>
<text x="336" y="115.0" font-size="12" fill="#D7DADF" text-anchor="end">count</text>
<circle cx="348" cy="111.0" r="5" fill="#9CC2E5" stroke="#1b1c1f" stroke-width="1.5"/>
<rect x="12" y="130" width="336" height="46" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="22" y="146" font-size="11.5" fill="#9AA0A8">concepts</text>
<text x="22" y="166" font-size="11.5" fill="#E6E8EB" font-family="'JetBrains Mono', Consolas, 'Courier New', monospace">person</text>
<rect x="12" y="181" width="336" height="24" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="20" y="197" font-size="10" fill="#6f747c">◀</text>
<text x="332" y="197" font-size="10" fill="#6f747c">▶</text>
<text x="34" y="197" font-size="11.5" fill="#9AA0A8">threshold</text>
<text x="326" y="197" font-size="11.5" fill="#E6E8EB" text-anchor="end">0.50</text>
<rect x="12" y="210" width="336" height="24" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="20" y="226" font-size="10" fill="#6f747c">◀</text>
<text x="332" y="226" font-size="10" fill="#6f747c">▶</text>
<text x="34" y="226" font-size="11.5" fill="#9AA0A8">detail_level</text>
<text x="326" y="226" font-size="11.5" fill="#E6E8EB" text-anchor="end">balanced</text>
<rect x="12" y="239" width="336" height="24" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="20" y="255" font-size="10" fill="#6f747c">◀</text>
<text x="332" y="255" font-size="10" fill="#6f747c">▶</text>
<text x="34" y="255" font-size="11.5" fill="#9AA0A8">max_per_concept</text>
<text x="326" y="255" font-size="11.5" fill="#E6E8EB" text-anchor="end">0</text>
<rect x="12" y="268" width="336" height="24" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="20" y="284" font-size="10" fill="#6f747c">◀</text>
<text x="332" y="284" font-size="10" fill="#6f747c">▶</text>
<text x="34" y="284" font-size="11.5" fill="#9AA0A8">checkpoint</text>
<text x="326" y="284" font-size="11.5" fill="#E6E8EB" text-anchor="end">(choose a file)</text>
<rect x="12" y="297" width="336" height="24" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="22" y="313" font-size="11.5" fill="#9AA0A8">keep_model_loaded</text>
<rect x="308" y="303" width="30" height="13" rx="6.5" fill="#4a4d53"/>
<circle cx="315" cy="309.5" r="5" fill="#F5F6F7"/>
</svg>

After

Width:  |  Height:  |  Size: 3.3 KiB

+14
View File
@@ -0,0 +1,14 @@
<svg xmlns="http://www.w3.org/2000/svg" width="360" height="149" viewBox="0 0 360 149" font-family="Inter, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif" role="img" aria-label="Segs Compose (NH) node">
<rect x="0.5" y="0.5" width="359" height="148" rx="10" fill="#26282c" stroke="#45484f"/>
<path d="M0.5 10.5 a10 10 0 0 1 10 -10 h339 a10 10 0 0 1 10 10 v36 h-359 z" fill="#34373d"/>
<circle cx="17" cy="17" r="4.5" fill="#8FD18F"/>
<text x="28" y="22" font-size="14" font-weight="600" fill="#EEF0F3">Segs Compose (NH)</text>
<text x="28" y="38" font-size="10.5" fill="#9AA0A8" font-family="'JetBrains Mono', Consolas, 'Courier New', monospace">NH_SegsCompose · Vision</text>
<circle cx="12" cy="67.0" r="5" fill="#E0B272" stroke="#1b1c1f" stroke-width="1.5"/>
<text x="24" y="71.0" font-size="12" fill="#D7DADF">segs</text>
<text x="336" y="71.0" font-size="12" fill="#D7DADF" text-anchor="end">mask</text>
<circle cx="348" cy="67.0" r="5" fill="#81C784" stroke="#1b1c1f" stroke-width="1.5"/>
<rect x="12" y="86" width="336" height="46" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="22" y="102" font-size="11.5" fill="#9AA0A8">expression</text>
<text x="22" y="122" font-size="11.5" fill="#E6E8EB" font-family="'JetBrains Mono', Consolas, 'Courier New', monospace">all</text>
</svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

+13
View File
@@ -0,0 +1,13 @@
<svg xmlns="http://www.w3.org/2000/svg" width="360" height="112" viewBox="0 0 360 112" font-family="Inter, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif" role="img" aria-label="Segs Preview (NH) node">
<rect x="0.5" y="0.5" width="359" height="111" rx="10" fill="#26282c" stroke="#45484f"/>
<path d="M0.5 10.5 a10 10 0 0 1 10 -10 h339 a10 10 0 0 1 10 10 v36 h-359 z" fill="#34373d"/>
<circle cx="17" cy="17" r="4.5" fill="#8FD18F"/>
<text x="28" y="22" font-size="14" font-weight="600" fill="#EEF0F3">Segs Preview (NH)</text>
<text x="28" y="38" font-size="10.5" fill="#9AA0A8" font-family="'JetBrains Mono', Consolas, 'Courier New', monospace">NH_SegsPreview · Vision</text>
<circle cx="12" cy="67.0" r="5" fill="#64B5F6" stroke="#1b1c1f" stroke-width="1.5"/>
<text x="24" y="71.0" font-size="12" fill="#D7DADF">image</text>
<text x="336" y="71.0" font-size="12" fill="#D7DADF" text-anchor="end">preview</text>
<circle cx="348" cy="67.0" r="5" fill="#64B5F6" stroke="#1b1c1f" stroke-width="1.5"/>
<circle cx="12" cy="89.0" r="5" fill="#E0B272" stroke="#1b1c1f" stroke-width="1.5"/>
<text x="24" y="93.0" font-size="12" fill="#D7DADF">segs</text>
</svg>

After

Width:  |  Height:  |  Size: 1.1 KiB

+48
View File
@@ -0,0 +1,48 @@
<svg xmlns="http://www.w3.org/2000/svg" width="360" height="345" viewBox="0 0 360 345" font-family="Inter, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif" role="img" aria-label="Segs Select (NH) node">
<rect x="0.5" y="0.5" width="359" height="344" rx="10" fill="#26282c" stroke="#45484f"/>
<path d="M0.5 10.5 a10 10 0 0 1 10 -10 h339 a10 10 0 0 1 10 10 v36 h-359 z" fill="#34373d"/>
<circle cx="17" cy="17" r="4.5" fill="#8FD18F"/>
<text x="28" y="22" font-size="14" font-weight="600" fill="#EEF0F3">Segs Select (NH)</text>
<text x="28" y="38" font-size="10.5" fill="#9AA0A8" font-family="'JetBrains Mono', Consolas, 'Courier New', monospace">NH_SegsSelect · Vision</text>
<circle cx="12" cy="67.0" r="5" fill="#E0B272" stroke="#1b1c1f" stroke-width="1.5"/>
<text x="24" y="71.0" font-size="12" fill="#D7DADF">segs</text>
<text x="336" y="71.0" font-size="12" fill="#D7DADF" text-anchor="end">segs</text>
<circle cx="348" cy="67.0" r="5" fill="#E0B272" stroke="#1b1c1f" stroke-width="1.5"/>
<text x="336" y="93.0" font-size="12" fill="#D7DADF" text-anchor="end">mask</text>
<circle cx="348" cy="89.0" r="5" fill="#81C784" stroke="#1b1c1f" stroke-width="1.5"/>
<text x="336" y="115.0" font-size="12" fill="#D7DADF" text-anchor="end">count</text>
<circle cx="348" cy="111.0" r="5" fill="#9CC2E5" stroke="#1b1c1f" stroke-width="1.5"/>
<rect x="12" y="130" width="336" height="24" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="22" y="146" font-size="11.5" fill="#9AA0A8">labels</text>
<text x="338" y="146" font-size="11.5" fill="#E6E8EB" text-anchor="end"></text>
<rect x="12" y="159" width="336" height="24" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="20" y="175" font-size="10" fill="#6f747c">◀</text>
<text x="332" y="175" font-size="10" fill="#6f747c">▶</text>
<text x="34" y="175" font-size="11.5" fill="#9AA0A8">min_score</text>
<text x="326" y="175" font-size="11.5" fill="#E6E8EB" text-anchor="end">0.00</text>
<rect x="12" y="188" width="336" height="24" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="20" y="204" font-size="10" fill="#6f747c">◀</text>
<text x="332" y="204" font-size="10" fill="#6f747c">▶</text>
<text x="34" y="204" font-size="11.5" fill="#9AA0A8">min_area</text>
<text x="326" y="204" font-size="11.5" fill="#E6E8EB" text-anchor="end">0.0000</text>
<rect x="12" y="217" width="336" height="24" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="20" y="233" font-size="10" fill="#6f747c">◀</text>
<text x="332" y="233" font-size="10" fill="#6f747c">▶</text>
<text x="34" y="233" font-size="11.5" fill="#9AA0A8">max_area</text>
<text x="326" y="233" font-size="11.5" fill="#E6E8EB" text-anchor="end">1.0000</text>
<rect x="12" y="246" width="336" height="24" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="20" y="262" font-size="10" fill="#6f747c">◀</text>
<text x="332" y="262" font-size="10" fill="#6f747c">▶</text>
<text x="34" y="262" font-size="11.5" fill="#9AA0A8">sort_by</text>
<text x="326" y="262" font-size="11.5" fill="#E6E8EB" text-anchor="end">score</text>
<rect x="12" y="275" width="336" height="24" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="20" y="291" font-size="10" fill="#6f747c">◀</text>
<text x="332" y="291" font-size="10" fill="#6f747c">▶</text>
<text x="34" y="291" font-size="11.5" fill="#9AA0A8">start</text>
<text x="326" y="291" font-size="11.5" fill="#E6E8EB" text-anchor="end">0</text>
<rect x="12" y="304" width="336" height="24" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="20" y="320" font-size="10" fill="#6f747c">◀</text>
<text x="332" y="320" font-size="10" fill="#6f747c">▶</text>
<text x="34" y="320" font-size="11.5" fill="#9AA0A8">count</text>
<text x="326" y="320" font-size="11.5" fill="#E6E8EB" text-anchor="end">0</text>
</svg>

After

Width:  |  Height:  |  Size: 3.7 KiB

+21
View File
@@ -0,0 +1,21 @@
<svg xmlns="http://www.w3.org/2000/svg" width="360" height="156" viewBox="0 0 360 156" font-family="Inter, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif" role="img" aria-label="Segs To Impact SEGS (NH) node">
<rect x="0.5" y="0.5" width="359" height="155" rx="10" fill="#26282c" stroke="#45484f"/>
<path d="M0.5 10.5 a10 10 0 0 1 10 -10 h339 a10 10 0 0 1 10 10 v36 h-359 z" fill="#34373d"/>
<circle cx="17" cy="17" r="4.5" fill="#8FD18F"/>
<text x="28" y="22" font-size="14" font-weight="600" fill="#EEF0F3">Segs To Impact SEGS (NH)</text>
<text x="28" y="38" font-size="10.5" fill="#9AA0A8" font-family="'JetBrains Mono', Consolas, 'Courier New', monospace">NH_SegsToImpactSEGS · Vision</text>
<circle cx="12" cy="67.0" r="5" fill="#E0B272" stroke="#1b1c1f" stroke-width="1.5"/>
<text x="24" y="71.0" font-size="12" fill="#D7DADF">segs</text>
<text x="336" y="71.0" font-size="12" fill="#D7DADF" text-anchor="end">segs</text>
<circle cx="348" cy="67.0" r="5" fill="#E0B272" stroke="#1b1c1f" stroke-width="1.5"/>
<rect x="12" y="86" width="336" height="24" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="20" y="102" font-size="10" fill="#6f747c">◀</text>
<text x="332" y="102" font-size="10" fill="#6f747c">▶</text>
<text x="34" y="102" font-size="11.5" fill="#9AA0A8">crop_factor</text>
<text x="326" y="102" font-size="11.5" fill="#E6E8EB" text-anchor="end">3.0</text>
<rect x="12" y="115" width="336" height="24" rx="7" fill="#1d1f22" stroke="#3a3d43"/>
<text x="20" y="131" font-size="10" fill="#6f747c">◀</text>
<text x="332" y="131" font-size="10" fill="#6f747c">▶</text>
<text x="34" y="131" font-size="11.5" fill="#9AA0A8">batch_index</text>
<text x="326" y="131" font-size="11.5" fill="#E6E8EB" text-anchor="end">0</text>
</svg>

After

Width:  |  Height:  |  Size: 1.7 KiB

+17
View File
@@ -0,0 +1,17 @@
<svg xmlns="http://www.w3.org/2000/svg" width="360" height="156" viewBox="0 0 360 156" font-family="Inter, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif" role="img" aria-label="Segs To Masks (NH) node">
<rect x="0.5" y="0.5" width="359" height="155" rx="10" fill="#26282c" stroke="#45484f"/>
<path d="M0.5 10.5 a10 10 0 0 1 10 -10 h339 a10 10 0 0 1 10 10 v36 h-359 z" fill="#34373d"/>
<circle cx="17" cy="17" r="4.5" fill="#8FD18F"/>
<text x="28" y="22" font-size="14" font-weight="600" fill="#EEF0F3">Segs To Masks (NH)</text>
<text x="28" y="38" font-size="10.5" fill="#9AA0A8" font-family="'JetBrains Mono', Consolas, 'Courier New', monospace">NH_SegsToMasks · Vision</text>
<circle cx="12" cy="67.0" r="5" fill="#E0B272" stroke="#1b1c1f" stroke-width="1.5"/>
<text x="24" y="71.0" font-size="12" fill="#D7DADF">segs</text>
<text x="336" y="71.0" font-size="12" fill="#D7DADF" text-anchor="end">masks</text>
<circle cx="348" cy="67.0" r="5" fill="#81C784" stroke="#1b1c1f" stroke-width="1.5"/>
<text x="336" y="93.0" font-size="12" fill="#D7DADF" text-anchor="end">union</text>
<circle cx="348" cy="89.0" r="5" fill="#81C784" stroke="#1b1c1f" stroke-width="1.5"/>
<text x="336" y="115.0" font-size="12" fill="#D7DADF" text-anchor="end">labels</text>
<circle cx="348" cy="111.0" r="5" fill="#C5E1A5" stroke="#1b1c1f" stroke-width="1.5"/>
<text x="336" y="137.0" font-size="12" fill="#D7DADF" text-anchor="end">count</text>
<circle cx="348" cy="133.0" r="5" fill="#9CC2E5" stroke="#1b1c1f" stroke-width="1.5"/>
</svg>

After

Width:  |  Height:  |  Size: 1.5 KiB

+36
View File
@@ -311,6 +311,12 @@
"category": "NH-Nodes/Resolution",
"deprecated": false
},
{
"id": "NH_SAM3Segment",
"name": "SAM3 Segment (NH)",
"category": "NH-Nodes/Vision",
"deprecated": false
},
{
"id": "NH_SaveImagePath",
"name": "Save Image Path (NH)",
@@ -323,6 +329,36 @@
"category": "NH-Nodes/Image",
"deprecated": false
},
{
"id": "NH_SegsCompose",
"name": "Segs Compose (NH)",
"category": "NH-Nodes/Vision",
"deprecated": false
},
{
"id": "NH_SegsPreview",
"name": "Segs Preview (NH)",
"category": "NH-Nodes/Vision",
"deprecated": false
},
{
"id": "NH_SegsSelect",
"name": "Segs Select (NH)",
"category": "NH-Nodes/Vision",
"deprecated": false
},
{
"id": "NH_SegsToImpactSEGS",
"name": "Segs To Impact SEGS (NH)",
"category": "NH-Nodes/Vision",
"deprecated": false
},
{
"id": "NH_SegsToMasks",
"name": "Segs To Masks (NH)",
"category": "NH-Nodes/Vision",
"deprecated": false
},
{
"id": "NH_SimpleFacePaste",
"name": "NH Simple Face Paste",
+15 -7
View File
@@ -4,13 +4,17 @@
# NH-Nodes guide
NH-Nodes adds **69 nodes** to ComfyUI for photo and production work: loading and saving image sets, tiling and compositing,
resolution management for current models, mask tools, garment and portrait segmentation, VTON masks, logic routing,
prompt and text tools, batch helpers and indexed model loaders.
NH-Nodes adds **75 nodes** to ComfyUI for photo and production work: loading and saving image sets, tiling and compositing,
resolution management for current models, mask tools, garment and portrait segmentation, SAM 3 instance masks for
anything you can name, VTON masks, logic routing, prompt and text tools, batch helpers and indexed model loaders.
This guide explains every node — what it is for, each input and output, and how it behaves in the details that matter.
Installation and a quick start are on the [project page](../../README.md#installation).
> **Requirements for SAM 3:** SAM3 Segment needs an NVIDIA GPU with CUDA, Meta's `sam3` package and the `sam3.pt`
> checkpoint, which you install and download yourself. Every other node works without them. Setup:
> [Before you start with SAM 3](vision.md#before-you-start-with-sam-3).
## Reading a node card
Every node section starts with a card drawn from the node's real definition:
@@ -20,7 +24,7 @@ Every node section starts with a card drawn from the node's real definition:
- **Title bar** — the display name, then the node id (type it in the node search) and its menu group under `NH-Nodes`.
- **Dots on the left** are inputs you connect, **dots on the right** are outputs. The colour shows the data type:
IMAGE blue, MASK green, LATENT pink, MODEL purple, CLIP yellow, VAE red, numbers (INT, FLOAT) light blue,
text (STRING) light green, BOOLEAN rose, any type (`*`) grey, and NH-specific data (tile data, resize data…) amber.
text (STRING) light green, BOOLEAN rose, any type (`*`) grey, and NH-specific data (tile data, resize data, instances…) amber.
- **Rounded rows** are widgets with their default value. `opt` marks an optional input; `≡` after an output means it
returns a list.
- A **green** dot in the title is an ordinary node; **orange** marks an output node, which runs even when nothing is
@@ -36,7 +40,7 @@ Every node section starts with a card drawn from the node's real definition:
| Image — mask-based retouch | [image-retouch](image-retouch.md) | NH Mask-Aware Resize · NH Agnostic Image Generator · NH Simple Face Paste |
| Resolution | [resolution](resolution.md) | NH Smart Resolution Picker · NH Smart Ratio Image Resize · NH Ratio Preset Image Resize · NH Remove Resize Padding · NH Restore Image Geometry |
| Mask | [mask](mask.md) | Mask Morphology · Mask Properties · NH Create Box Mask · NH Mask Aspect Ratio Match |
| Vision | [vision](vision.md) | Garment Segment · Portrait Segment |
| Vision | [vision](vision.md) | Garment Segment · Portrait Segment · SAM3 Segment · Segs Select · Segs Compose · Segs To Masks · Segs To Impact SEGS · Segs Preview |
| VTON | [vton](vton.md) | VTON Ultimate Processor |
| Logic | [logic](logic.md) | Compare · Logic Gate · If/Else · Switch N · Any Switch · Any Branch Switch · Gate Switch · Value Match Index · Math Eval · Random Choice |
| Text | [text](text.md) | Prompt Template · Prompt Scheduler · String Operations · Prompt Join · Text Split · Regex Extract · Text Index · Text Concatenate · Text Split Lines · Text Random Line |
@@ -58,13 +62,17 @@ Every node section starts with a card drawn from the node's real definition:
[image-io](image-io.md#paths-and-file-types).
- **Resize data.** The resolution nodes return a `resize_data` record of how an image was cropped or padded, so a result
can be put back into the original frame with [NH Restore Image Geometry](resolution.md#nh-restore-image-geometry).
- **Instances.** SAM3 Segment returns every thing it finds as its own instance, with a label, a score and a mask, in an
`NH_SEGS` value. The Segs nodes pick instances, combine them with a formula and turn them into masks; see
[Instances: the NH_SEGS type](vision.md#instances-the-nh_segs-type).
- **Licences.** All nodes are MIT except the parts listed in [THIRD_PARTY_NOTICES](../../THIRD_PARTY_NOTICES.md):
VTON Ultimate Processor and Portrait Segment are for **non-commercial** use, and Garment Segment's model declares no
licence.
licence. SAM3 Segment uses Meta's SAM 3 under the SAM License, which has no non-commercial clause but bars military,
nuclear, espionage, and gun or illegal-weapon uses; you install the package and download the checkpoint yourself.
## More
- [Changelog](../CHANGELOG.md) — what changed in each release, including behaviour changes.
- [Logic gate diagram](../logic_gate_diagram.html) — a visual truth-table sheet for the Logic Gate node (download and open
it in a browser).
- [Third-party notices](../../THIRD_PARTY_NOTICES.md) — licences of bundled code and downloaded models.
- [Third-party notices](../../THIRD_PARTY_NOTICES.md) — licences of bundled code, downloaded models and SAM 3.
+1 -1
View File
@@ -12,7 +12,7 @@ Three nodes that work with a mask. One resizes an image to an exact size and kee
| [NH Agnostic Image Generator](#nh-agnostic-image-generator) | Erase the masked area with grey, noise or blur |
| [NH Simple Face Paste](#nh-simple-face-paste) | Paste the masked face of one image onto the masked area of another |
A `MASK` is white (`1`) where the node should work and black (`0`) elsewhere. Get one from a segmentation node such as [Garment Segment (NH)](./vision.md#garment-segment-nh) or [Portrait Segment (NH)](./vision.md#portrait-segment-nh), or from any mask tool in ComfyUI.
A `MASK` is white (`1`) where the node should work and black (`0`) elsewhere. Get one from a segmentation node such as [SAM3 Segment (NH)](./vision.md#sam3-segment-nh) (any object you can name, refined with [Segs Compose (NH)](./vision.md#segs-compose-nh)), [Garment Segment (NH)](./vision.md#garment-segment-nh) or [Portrait Segment (NH)](./vision.md#portrait-segment-nh), or from any mask tool in ComfyUI.
---
+369 -6
View File
@@ -2,17 +2,43 @@
# Vision
Two nodes that cut parts out of a photo of a person with SegFormer models: clothing with Garment Segment, and face, hair and neck with Portrait Segment. Each returns a mask at the size of the input image, a cutout with transparency and a mask preview. Find them under **NH-Nodes → Vision**.
Eight nodes that cut parts out of a photo. Two use SegFormer models that know people: clothing with Garment Segment, and face, hair and neck with Portrait Segment. Each returns a mask at the size of the input image, a cutout with transparency and a mask preview. The other six work with SAM 3, Meta's segmentation model: SAM3 Segment finds every instance of anything you can name in plain English (people, a wine glass, windows, the sky), and five Segs nodes pick, combine, preview and convert what it found. Find them under **NH-Nodes → Vision**.
**On this page**
- [Before you start](#before-you-start) — models, first run, licences, memory
- [Refining the mask](#refining-the-mask) — the options both nodes share
- [Garment Segment (NH)](#garment-segment-nh) — mask upper garments, lower garments, footwear or headwear
- [Portrait Segment (NH)](#portrait-segment-nh) — mask face, hair, neck and facial features
- [Which node do I need?](#which-node-do-i-need) — the SegFormer nodes or SAM 3
- **SegFormer nodes: garments and portraits**
- [Before you start](#before-you-start) — models, first run, licences, memory
- [Refining the mask](#refining-the-mask) — the options both nodes share
- [Garment Segment (NH)](#garment-segment-nh) — mask upper garments, lower garments, footwear or headwear
- [Portrait Segment (NH)](#portrait-segment-nh) — mask face, hair, neck and facial features
- **SAM 3 nodes: anything you can name**
- [Before you start with SAM 3](#before-you-start-with-sam-3) — GPU, package, checkpoint, memory, licence
- [How the two passes work](#how-the-two-passes-work) — what `balanced` does on 4K to 8K photos, and where it stops
- [Instances: the NH_SEGS type](#instances-the-nh_segs-type) — what flows between the nodes
- [SAM3 Segment (NH)](#sam3-segment-nh) — find every instance of the concepts you name
- [Segs Select (NH)](#segs-select-nh) — keep instances by label, score, size and position
- [Segs Compose (NH)](#segs-compose-nh) — build a mask from labels with `|`, `-`, `&` and `~`
- [Segs To Masks (NH)](#segs-to-masks-nh) — one mask per instance, the union and the labels
- [Segs To Impact SEGS (NH)](#segs-to-impact-segs-nh) — hand the instances to Impact Pack's detailers
- [Segs Preview (NH)](#segs-preview-nh) — draw instances on their picture
- [SAM 3 workflows](#sam-3-workflows) — the background behind people, the biggest object, a facade without windows
- [If something goes wrong with SAM 3](#if-something-goes-wrong-with-sam-3)
## Which node do I need?
| You want | Use | What it needs |
|---|---|---|
| The clothing of one person: shirt, trousers, shoes, hat | [Garment Segment](#garment-segment-nh) | A SegFormer model that downloads itself on the first run. Runs on the CPU or the GPU. |
| The face, hair, neck and facial features of a portrait | [Portrait Segment](#portrait-segment-nh) | The same. |
| Anything you can name (people, objects, sky, walls, windows), every instance on its own, also on 4K to 8K photos | [SAM3 Segment](#sam3-segment-nh) and the Segs nodes | An NVIDIA GPU, Meta's `sam3` package and a checkpoint that you install and download yourself. See [Before you start with SAM 3](#before-you-start-with-sam-3). |
You can ask SAM3 Segment for `dress` or `hair` too. The SegFormer nodes stay the easier choice for their fixed sets of clothing and face parts: they set themselves up and also run without a GPU.
## Before you start
This section is about Garment Segment and Portrait Segment. SAM3 Segment and the Segs nodes have their own: [Before you start with SAM 3](#before-you-start-with-sam-3).
| | Garment Segment (NH) | Portrait Segment (NH) |
|---|---|---|
| Model | SegFormer trained for Fashionpedia clothing classes | SegFormer fine-tuned on CelebAMask-HQ face parts |
@@ -30,7 +56,7 @@ Two nodes that cut parts out of a photo of a person with SegFormer models: cloth
## Refining the mask
Both nodes turn the model's label map into a mask, then refine it in this order: `fill_holes`, `mask_expand`, `mask_blur`. The mask always has the size of the input image.
Garment Segment and Portrait Segment turn the model's label map into a mask, then refine it in this order: `fill_holes`, `mask_expand`, `mask_blur`. The mask always has the size of the input image.
| Option | Range | Garment Segment | Portrait Segment | What it does |
|---|---|---|---|---|
@@ -158,4 +184,341 @@ Switch off every region except `hair`. With the defaults you get face, ears, nec
---
## Before you start with SAM 3
SAM 3 is Meta's model for finding things by name: you write what you want (`person`, `window`, `wine glass`) and it returns every instance of it, each with a score and its own mask. NH-Nodes does not include the model and never downloads it, so SAM3 Segment needs three things from you, once. If another SAM 3 pack already works in your ComfyUI, steps 2 and 3 are usually done: that pack has installed the package and put `sam3.pt` where this node looks.
**1. An NVIDIA GPU with CUDA.** SAM 3 runs only on CUDA: the official sam3 code builds part of the model on the GPU, so there is no CPU mode and no `device` input. Plan for about 5 GB of VRAM at the peak.
**2. Meta's `sam3` package.** NH-Nodes does not install it for you. In the Python that runs ComfyUI, run these two lines, then restart ComfyUI:
```text
python -m pip install --no-deps git+https://github.com/facebookresearch/sam3@b26a5f330e05d321afb39d01d3d4881f258f65ff
python -m pip install timm ftfy regex iopath
```
The first line installs the sam3 commit that NH-Nodes was tested with; newer commits with the same image API also work. It needs Git on your computer. `--no-deps` keeps your numpy: sam3 asks for numpy below 2, which ComfyUI does not need. Meta's page for the package lists Python 3.12 or newer, PyTorch 2.7 or newer and CUDA 12.6 or newer. If another SAM 3 pack already provides `sam3`, skip the first line: installing it again could replace the copy that pack uses.
**3. The checkpoint `sam3.pt`.** Meta hands out the weights only after you accept its SAM License, so NH-Nodes cannot download them for you. Request access at [huggingface.co/facebook/sam3](https://huggingface.co/facebook/sam3), download `sam3.pt` from that page and put it in `ComfyUI/models/sam3/`. Press R in ComfyUI to refresh the lists, then pick the file in `checkpoint`. To keep it somewhere else, add the folder under the `sam3` key in `extra_model_paths.yaml` (see [Installation](../../README.md#installation) for the file):
```yaml
my_models:
base_path: D:/ai-models/
sam3: sam3/ # SAM 3 checkpoints, shared with other SAM 3 packs
```
Only the PyTorch checkpoint `sam3.pt` works. The safetensors export on Hugging Face is made for the transformers library and uses other weight names.
**Memory.** The weights are read from disk once and kept in system memory (about 3.6 GB). They move to the GPU while SAM3 Segment runs and go back afterwards, so ComfyUI gets its VRAM back. Before they move, the node asks ComfyUI to unload some of its own models if VRAM is tight. Turn on `keep_model_loaded` to leave them on the GPU when you run the node again and again.
> **Licence:** SAM 3 is released under Meta's SAM License. It is royalty-free and has no non-commercial clause, but it forbids uses that fall under ITAR or that trade controls prohibit, including military or warfare, nuclear, espionage, and guns or illegal weapons. NH-Nodes' own code is MIT, and it contains neither the model nor the `sam3` package. Read [THIRD_PARTY_NOTICES.md](../../THIRD_PARTY_NOTICES.md) before you use the node commercially.
---
## How the two passes work
SAM 3 looks at a picture of about 1000 px. On a 6000 px photo that is a sixth of the width, so small objects and thin edges would come out soft. SAM3 Segment therefore looks twice.
1. **First pass: the whole image.** The picture is shrunk to the model's square input (1008 px) and encoded once. Every concept you listed is asked against that one encoding, which is cheap. This gives each instance a score, a box and a first mask, which is resampled to full resolution inside its own box only.
2. **Second pass: a close-up of each instance** (`balanced` only). The instance is cut out of the full-resolution image with a margin (20% of its longer side, at least 16 px) and segmented again with SAM 3's box-prompt decoder, with the first mask as a hint. Its edges now come from the original pixels.
3. **A safety net.** A close-up is kept when it matches the first-pass instance to within the blur of the first pass, so thin things such as poles and cables survive. When it does not match, the first-pass mask stays, and the console says for how many instances. Specks of the new mask that do not touch the first-pass mask are dropped.
| `detail_level` | Passes | Use it for |
|---|---|---|
| `balanced` (default) | Both | Crisp edges on small and medium objects, also on 4K to 8K photos. About 0.1 s per instance on a fast GPU. |
| `fast` | The first only | Small pictures, and quick tries while you look for the right words for the concepts. |
**What `balanced` can and cannot do.** Small and medium objects (a window, a glass, one person in a group photo) get crisp edges even on a 4K to 8K photo, because each one is judged on a close-up of the original pixels. The limit is the model: SAM 3 describes whatever it looks at on a grid of about 288 cells. A subject that fills most of an 8K frame is its own close-up, so its edges are accurate to about 1/300 of its size, around 25 px for a subject 7000 px across. That is fine for the outline of a person or a building, and soft on hair, lace and fur. Hair-level edges need a matting step that NH-Nodes does not have yet; a Matte Refine node is planned.
**Measured on one machine** (an NVIDIA RTX 5090): the first run loads the model in about 9 s. A 7360×4912 photo with the concepts `person, wine glass, dress` gave 13 instances in about 3.6 s with `balanced`. A 6720×4480 interior with 8 concepts gave 29 instances in about 6 s. The VRAM peak was about 5 GB. Other GPUs will differ.
---
## Instances: the NH_SEGS type
`NH_SEGS` is the data that SAM3 Segment hands to the Segs nodes. For every image of a batch it holds a list of instances. Each instance keeps its **label** (the concept you wrote), its **score**, a tight **box** and the mask **only inside that box**. A full-size mask is built only when a node outputs a `MASK`, so dozens of instances on an 8K photo stay light.
- Instances come in the order of your concepts, and by score within a concept.
- The Segs nodes treat every image of a batch separately.
- ComfyUI's own nodes do not read `NH_SEGS`. To get masks out, use the `mask` outputs, [Segs Compose](#segs-compose-nh) or [Segs To Masks](#segs-to-masks-nh). To use Impact Pack's detailers, convert with [Segs To Impact SEGS](#segs-to-impact-segs-nh).
---
## SAM3 Segment (NH)
<p align="center"><img src="../assets/nodes/NH_SAM3Segment.svg" alt="SAM3 Segment (NH) node" width="360"></p>
Finds every instance of the things you name and returns them as `NH_SEGS`, together with their combined mask and a count. To see what it found, connect [Segs Preview](#segs-preview-nh).
**Use it when** you want masks of "all the people", "every window", "the sky" or "the wine glasses" in a photo, also a 4K to 8K one, without painting anything. Then use the [Segs nodes](#segs-select-nh) to pick instances, subtract one from another, or turn them into plain masks.
Combo inputs without a stated default start on their first option.
| Input | Type | Default | Description |
|---|---|---|---|
| `image` | IMAGE | — | The picture. Colour (RGB) is used and any alpha is dropped. Every image of a batch is processed in turn. |
| `concepts` | STRING | `person` | What to find: short English noun phrases, one per line or separated by commas (`person, red car, window, sky`). Lines that start with `#` are ignored. Each phrase becomes the label of the instances it finds. Write at least one. |
| `threshold` | FLOAT | `0.5` | Range 0.05 to 0.95. The minimum confidence. Lower finds more instances and more mistakes; higher keeps only the sure ones. |
| `detail_level` | COMBO | `balanced` | `balanced` or `fast`. `balanced` segments every instance a second time on a close-up of the full-resolution image, for crisp edges on small and medium objects. `fast` keeps the first, whole-image pass. See [How the two passes work](#how-the-two-passes-work). |
| `max_per_concept` | INT | `0` | Range 0 to 200. Keep at most this many instances of each concept, best score first. `0` keeps all. |
| `checkpoint` | COMBO | — | A SAM 3 `.pt` or `.pth` file from `models/sam3` or another folder registered as `sam3`. Other SAM 3 packs use the same folder. When it holds no file, the list shows `sam3.pt` and the node tells you where to get it when you run it. |
| `keep_model_loaded` | BOOLEAN | `false` | Off: after the run the weights go back to system memory and ComfyUI gets the VRAM back. On: they stay on the GPU (about 3.6 GB), which is faster when the node runs again, but ComfyUI cannot use that VRAM meanwhile. |
| Output | Type | Description |
|---|---|---|
| `segs` | NH_SEGS | The instances of every image, each with its label, score, box and mask. Connect it to the Segs nodes. |
| `mask` | MASK | The union of every instance, one mask per image, at the size of the image. |
| `count` | INT | The number of instances found, over the whole batch. |
<p align="center"><img src="../assets/examples/sam3-segment.jpg" alt="SAM3 Segment instances drawn by Segs Preview, and three masks built from them on a sample photo" width="720"><br><em>SAM3 Segment on a sample photo with the concepts "person, helmet, face", drawn by Segs Preview, then three masks made from the instances with Segs Compose and Segs Select.</em></p>
**How it works**
- **Labels.** The concept you wrote is the label of its instances. The Segs nodes ignore case when they match labels, so spelling is not a problem.
- **Order and duplicates.** Instances come in the order of your concepts, best score first within a concept. Two instances of the same concept whose masks overlap almost completely (90% or more) count as one, and the better score stays. `max_per_concept` then keeps the best of what is left.
- **Seeing the result.** Connect `segs` and the same picture to [Segs Preview](#segs-preview-nh): every instance is tinted, outlined and labelled with its label and score.
- **Nothing found** is not an error: you get no instances, an empty mask and a `count` of 0.
- **Batches.** Every image is processed in turn and has its own list of instances. A progress bar counts the concepts, and you can cancel the run between concepts and between close-ups.
- **Very wide or tall pictures.** SAM 3 first sees the whole picture squeezed into a square. On a picture wider or taller than 4:1 (a panorama), it can miss small things, and the console says so. Split such a picture into tiles with [Image Tile (NH)](./image-compose.md#image-tile-nh) and run the tiles.
- **Setup problems** stop the node with a message that says what to do: a missing package, a missing checkpoint, no CUDA GPU. See [If something goes wrong with SAM 3](#if-something-goes-wrong-with-sam-3).
> **Tip:** Write simple English nouns: `window`, `wine glass`, `red car`. If a concept finds nothing, try another word (an unusual building may answer to `structure` or `monument` rather than `building`) or lower `threshold`.
> **Tip:** Ask for a whole and its parts together (`person, hair, face, dress`), then take them apart with [Segs Compose](#segs-compose-nh): `person - hair`.
> **Licence:** SAM 3 is under Meta's SAM License. See [Before you start with SAM 3](#before-you-start-with-sam-3).
---
## Segs Select (NH)
<p align="center"><img src="../assets/nodes/NH_SegsSelect.svg" alt="Segs Select (NH) node" width="360"></p>
Keeps the instances you want, by label, score, size and position, in the order you choose.
**Use it when** you need only some of what SAM3 Segment found: the biggest object, the second person from the left, only the sure detections, or only the large windows.
| Input | Type | Default | Description |
|---|---|---|---|
| `segs` | NH_SEGS | — | The instances, from SAM3 Segment or another Segs node. |
| `labels` | STRING | empty | The labels to keep, separated by commas. `*` stands for any run of characters and `?` for one character (`*shirt`). Case does not matter, and `_` counts as a space (`red_car` finds `red car`). Labels with spaces need no quotes here: `wine glass, red car`. Empty keeps every label. |
| `min_score` | FLOAT | `0.0` | Range 0 to 1. Drops instances with a lower score. |
| `min_area` | FLOAT | `0.0` | Range 0 to 1. Drops instances smaller than this fraction of the image (`0.01` is 1%). |
| `max_area` | FLOAT | `1.0` | Range 0 to 1. Drops instances larger than this fraction of the image. |
| `sort_by` | COMBO | `score` | The order of the kept instances. See the table below. |
| `start` | INT | `0` | Range 0 to 4096. Skips this many instances of the sorted list. `0` starts at the first. |
| `count` | INT | `0` | Range 0 to 4096. Keeps this many instances from `start` on. `0` keeps all of them. |
| Output | Type | Description |
|---|---|---|
| `segs` | NH_SEGS | The kept instances, in the sorted order. |
| `mask` | MASK | The union of the kept instances, one mask per image. |
| `count` | INT | The number of instances kept, over the whole batch. |
| `sort_by` | Order |
|---|---|
| `score` | Highest score first. |
| `area` | Largest first. |
| `left_to_right`, `right_to_left` | By the horizontal centre of the box. |
| `top_to_bottom`, `bottom_to_top` | By the vertical centre of the box. |
| `center_first` | The instance whose box centre is nearest the centre of the image first. |
| `as_found` | The order SAM3 Segment found them in: your concepts in turn, best score first within each. |
**How it works**
- **Three steps, in this order:** the filters (`labels`, `min_score`, `min_area`, `max_area`), then the sorting, then the slice (`start` and `count`).
- **Per image.** Every image of a batch is handled on its own, so `start` `1` skips the first instance of each image.
- **Area** is the share of the image's pixels that the mask covers, not the size of its box. Positions use the centre of the box. Ties keep the order the instances were found in.
- The kept instances stay sorted, so [Segs To Masks](#segs-to-masks-nh) after this node returns its masks in that order.
| To get | Set |
|---|---|
| The second person from the left | `labels` `person`, `sort_by` `left_to_right`, `start` `1`, `count` `1` |
| The biggest object | `sort_by` `area`, `count` `1` |
| The three surest instances | `sort_by` `score`, `count` `3` |
| Nothing under 1% of the picture | `min_area` `0.01` |
| The instance nearest the middle | `sort_by` `center_first`, `count` `1` |
---
## Segs Compose (NH)
<p align="center"><img src="../assets/nodes/NH_SegsCompose.svg" alt="Segs Compose (NH) node" width="360"></p>
Builds one mask from the labels of the instances with a small formula: union, minus, overlap and "everything except".
**Use it when** you want "the background", "the person without the hair", "the wall without the windows", or any other combination of the things SAM3 Segment found.
| Input | Type | Default | Description |
|---|---|---|---|
| `segs` | NH_SEGS | — | The instances, from SAM3 Segment or Segs Select. |
| `expression` | STRING | `all` | The formula. See below. The field has several lines, and a line break counts as a space. |
| Output | Type | Description |
|---|---|---|
| `mask` | MASK | The result of the formula, one mask per image, at the size of the image. |
**What you can write**
| Write | Meaning |
|---|---|
| `person` | A label: the union of every instance with that label. Case does not matter, `_` counts as a space, and `*` and `?` are wildcards (`*shirt`). |
| `"red car"` | Put a label that contains spaces or an operator character in quotes, `"` or `'`, or write the space as `_` (`red_car`). Without quotes, `red car` is an error. Quotes also turn a word such as `all` or `image` into an ordinary label. |
| `t-shirt` | A hyphen between two letters or digits belongs to the label, so `t-shirt` and `x-ray` need no quotes. An apostrophe inside a word is fine too (`women's`). A label with a space needs quotes, and double quotes when it also has an apostrophe: `"women's dress"`. |
| `a \| b`, `a + b`, `a or b` | Union: everything in a or in b. |
| `a - b` | a without b. Put spaces around the minus: `person-hair` is read as one label. When both `person` and `hair` were found, the node stops with an error that shows `person - hair`; otherwise the unknown label gives an empty part and the console names it. |
| `a & b`, `a and b` | Overlap: only where a and b both are. |
| `~a`, `!a`, `not a` | Everything except a: the whole picture minus a. |
| `( … )` | Grouping. |
| `all` | Every instance in the segs. |
| `image` | The whole picture. |
| `none` | An empty mask. |
`~` binds first, then `&`, then `|` and `-` together, left to right: `a - b | c` is `(a - b) | c`. When in doubt, add parentheses.
| Expression | You get |
|---|---|
| `person - hair` | The people without their hair (ask SAM3 Segment for `person, hair`). |
| `~person` | Everything except the people: the background. |
| `(wall \| ceiling) - window` | The walls and the ceiling with the window openings cut out. |
| `"wine glass" \| person` | The glasses and the people together. |
| `all - person` | Everything that was found except the people. Unlike `~person`, it leaves out the background that nothing found. |
**How it works**
- **A label that no instance has** gives an empty part, and the console says so (`no instance is labelled 'x'; those parts are empty`). A typo in `~persn` therefore gives the whole picture: check the console when a mask looks wrong.
- **Every image** of a batch gets its own mask, built from its own instances.
- **A formula that cannot be read** stops the node with a message that gives the position of the problem, for example `unexpected 'car' at position 5`.
---
## Segs To Masks (NH)
<p align="center"><img src="../assets/nodes/NH_SegsToMasks.svg" alt="Segs To Masks (NH) node" width="360"></p>
Turns the instances into ordinary masks: one mask per instance, the union per image, and the list of labels.
**Use it when** you need each instance as a mask of its own: to process them one by one, to save them, or to read their labels.
| Input | Type | Default | Description |
|---|---|---|---|
| `segs` | NH_SEGS | — | The instances, from SAM3 Segment or Segs Select. |
| Output | Type | Description |
|---|---|---|
| `masks` | MASK | One mask per instance, image after image, in the order of `labels`. When there is no instance at all, a single empty mask, so that nodes after it do not fail. |
| `union` | MASK | One mask per image: the union of its instances. |
| `labels` | STRING | The label of every instance, one per line, in the order of `masks`. |
| `count` | INT | The number of instances. It is 0 when there are none, although `masks` then holds one empty mask. |
> **Note:** Every mask has the size of the picture and 4 bytes per pixel: about 145 MB on a 7360×4912 photo. When the masks would need more than about half of your free RAM (for example hundreds of instances on an 8K photo), the node stops with a message instead of filling the memory. Use [Segs Select](#segs-select-nh) first to keep only the instances you need, or [Segs Compose](#segs-compose-nh) to get one mask.
---
## Segs To Impact SEGS (NH)
<p align="center"><img src="../assets/nodes/NH_SegsToImpactSEGS.svg" alt="Segs To Impact SEGS (NH) node" width="360"></p>
Converts the instances of one image to Impact Pack's `SEGS`, so that its detailers and SEGS tools can work on them.
**Use it when** you already use Impact Pack and want its detailers to refine what SAM3 Segment found. NH-Nodes needs Impact Pack for nothing else, so skip this node if you do not use it.
| Input | Type | Default | Description |
|---|---|---|---|
| `segs` | NH_SEGS | — | The instances, from SAM3 Segment or Segs Select. |
| `crop_factor` | FLOAT | `3.0` | Range 1 to 10. The size of the region a detailer works on around each instance, as a multiple of its box (Impact Pack's `crop_factor`). The region is moved to stay inside the image and always contains the box. |
| `batch_index` | INT | `0` | Range 0 to 4095. Impact Pack's SEGS hold one image, so this picks which image of the batch to convert. A number past the last image stops the node with a message. |
| Output | Type | Description |
|---|---|---|
| `segs` | SEGS | Impact Pack's `SEGS` for that image. Each instance has its box, crop region, mask, label and score (as `confidence`). |
**How it works**
- The output is Impact Pack's own type, not `NH_SEGS`. NH-Nodes builds it with the same fields and does not import Impact Pack, so the conversion needs no Impact Pack. Using the result does.
- To convert several images of a batch, use one node per `batch_index`.
---
## Segs Preview (NH)
<p align="center"><img src="../assets/nodes/NH_SegsPreview.svg" alt="Segs Preview (NH) node" width="360"></p>
Draws the instances on their picture: each one tinted in its own colour, outlined and labelled with its label and score.
**Use it when** you want to see what SAM3 Segment found, or what Segs Select or Segs Compose works with. Put it after SAM3 Segment to see everything, or after Segs Select to see only the selection.
| Input | Type | Default | Description |
|---|---|---|---|
| `image` | IMAGE | — | The picture the segs were found in: the same number of images and the same size. Otherwise the node stops and says so. |
| `segs` | NH_SEGS | — | The instances to draw. |
| Output | Type | Description |
|---|---|---|
| `preview` | IMAGE | The picture with the instances drawn on it, at the same size. |
The colours follow the order of the instances, each label sits above the highest point of its instance, and the size of the labels follows the size of the picture. An 8K picture gives an 8K preview.
---
## SAM 3 workflows
In each of these, check what SAM3 Segment found before you edit anything. Connect the same picture and the `segs` to [Segs Preview (NH)](#segs-preview-nh): every instance you want should have an outline and a label. If one is missing, lower `threshold` (for example to 0.3) or try another word for the concept.
### The background behind people
```text
Load Image → SAM3 Segment (concepts: person)
→ Segs Compose (expression: ~person)
→ Mask Morphology (NH)
→ edit the background
```
`~person` is the whole picture except the people. Add what people hold or sit on to both fields, for example the concepts `person, wine glass, chair` and the expression `~(person | "wine glass" | chair)`. Move the edge of the mask a few pixels with [Mask Morphology (NH)](./mask.md#mask-morphology-nh): grow it so that the edit covers the old outline, or shrink it if the edit eats into the people. Then use the mask to blur, relight, recolour or inpaint the background.
### The biggest object, inpainted
```text
Load Image → SAM3 Segment (concepts: person, dog, car)
→ Segs Select (sort_by: area, count: 1)
→ Mask Morphology (NH, grow)
→ inpaint
```
With `labels` empty, Segs Select picks the biggest of everything you asked for. Set `labels` (for example `car`) to pick the biggest of one kind, `start` `1` for the second biggest, and raise `min_score` so that a doubtful detection cannot be picked. Grow the mask a few pixels so that the inpainted area covers the old outline.
### Architecture: a facade without windows
```text
Load Image → SAM3 Segment (concepts: building, window, door)
→ Segs Compose (expression: building - (window | door))
→ recolour, restyle or inpaint the wall
```
`building` covers the whole facade with its windows and doors; the formula cuts them out and leaves the wall. For a room, ask for `wall, ceiling, window` and use `(wall | ceiling) - window`. If `building` finds nothing on an unusual building, try `structure` or `monument` and change the formula to match. On a 4K to 8K photo, keep `balanced`: every window is segmented again on a close-up of the original pixels. The outline of a facade that fills the whole frame is accurate to about 1/300 of its width (see [How the two passes work](#how-the-two-passes-work)), so check the roofline in Segs Preview.
---
## If something goes wrong with SAM 3
| What you see | What to do |
|---|---|
| An error that SAM 3 needs Meta's sam3 package | Install it as in [Before you start with SAM 3](#before-you-start-with-sam-3), then restart ComfyUI. |
| An error that the sam3 package is installed but failed to load, or lacks something | The installed version is not one NH-Nodes supports, or one of its extras is missing. Reinstall with the two lines in [Before you start with SAM 3](#before-you-start-with-sam-3). The message shows the cause. |
| An error that the checkpoint was not found | Download `sam3.pt` as in [Before you start with SAM 3](#before-you-start-with-sam-3), press R in ComfyUI and pick the file. The message lists the folders that were searched. |
| An error that a file is not a SAM 3 `.pt` checkpoint | Use `sam3.pt`. The safetensors files on Hugging Face do not work with this node. |
| An error that a file cannot be read as a SAM 3 checkpoint, or is not one | The file in the `sam3` folder is an incomplete download or another model's `.pt`. Download `sam3.pt` again as in [Before you start with SAM 3](#before-you-start-with-sam-3). |
| An error that SAM 3 needs an NVIDIA GPU with CUDA | ComfyUI is running on the CPU or on another kind of GPU. SAM3 Segment has no CPU mode. |
| An error that you must write at least one concept | `concepts` is empty or holds only comments. Write a word such as `person`. |
| `count` is 0, or an instance is missing | Lower `threshold`, try another word for the concept, and look at the result with Segs Preview. |
| The console says it kept the first-pass mask for some instances | Their close-up did not match the first pass, so those instances keep the first pass's edges. Nothing is wrong. |
| An error from Segs Preview about sizes or the number of images | Connect the picture that the segs were found in. |
| An error from Segs Compose that gives a position | Check the formula at that position. Put labels with spaces in quotes: `"red car"`. |
| A Segs Compose mask is empty, and the console says no instance has a label | Check the spelling of the label. Put spaces around a minus: `person - hair`, not `person-hair`. |
| An error from Segs To Masks that the full-size instance masks would need more memory than is free | There are too many instances for the size of the picture. Use Segs Select to keep only the ones you need, or Segs Compose to get one mask. |
| The console says that SAM 3 sees the picture squeezed into a square and can miss small things | The picture is wider or taller than 4:1. Split it into tiles with Image Tile (NH) and run the tiles. |
| ComfyUI runs out of VRAM after the node | Leave `keep_model_loaded` off, so the weights leave the GPU after each run. |
---
[← Back to guide](./README.md)
+16 -7
View File
@@ -4,13 +4,18 @@
# Hướng dẫn NH-Nodes
NH-Nodes bổ sung **69 node** cho ComfyUI, phục vụ công việc ảnh và sản xuất: nạp và lưu bộ ảnh, chia ô (tile) và ghép ảnh,
quản lý độ phân giải cho các model hiện nay, công cụ mask, phân đoạn trang phục và khuôn mặt, tạo mask VTON, rẽ nhánh
logic, xử lý prompt và văn bản, tiện ích batch và bộ nạp model theo chỉ số.
NH-Nodes bổ sung **75 node** cho ComfyUI, phục vụ công việc ảnh và sản xuất: nạp và lưu bộ ảnh, chia ô (tile) và ghép ảnh,
quản lý độ phân giải cho các model hiện nay, công cụ mask, phân đoạn trang phục và khuôn mặt, mask từng instance của bất cứ
thứ gì bạn gọi tên bằng SAM 3, tạo mask VTON, rẽ nhánh logic, xử lý prompt và văn bản, tiện ích batch và bộ nạp model theo
chỉ số.
Tài liệu này giải thích từng node: dùng để làm gì, từng đầu vào và đầu ra, và cách node xử lý ở những chi tiết quan trọng.
Phần cài đặt và hướng dẫn nhanh nằm ở [trang dự án](../../README.md#installation).
> **Yêu cầu cho SAM 3:** SAM3 Segment cần GPU NVIDIA có CUDA, gói `sam3` của Meta và checkpoint `sam3.pt`, những thứ bạn
> tự cài và tự tải. Mọi node khác chạy được mà không cần chúng. Cách thiết lập:
> [Trước khi bắt đầu với SAM 3](vision.md#trước-khi-bắt-đầu-với-sam-3).
## Cách đọc thẻ node
Mỗi phần node bắt đầu bằng một thẻ được vẽ trực tiếp từ định nghĩa thật của node:
@@ -20,7 +25,7 @@ Mỗi phần node bắt đầu bằng một thẻ được vẽ trực tiếp t
- **Thanh tiêu đề** — tên hiển thị, bên dưới là mã node (gõ mã này vào ô tìm node) và nhóm menu trong `NH-Nodes`.
- **Chấm bên trái** là đầu vào để nối dây, **chấm bên phải** là đầu ra. Màu chấm cho biết kiểu dữ liệu: IMAGE xanh dương,
MASK xanh lá, LATENT hồng, MODEL tím, CLIP vàng, VAE đỏ, số (INT, FLOAT) xanh dương nhạt, chữ (STRING) xanh lá nhạt,
BOOLEAN hồng phấn, kiểu bất kỳ (`*`) màu xám, và dữ liệu riêng của NH (thông tin tile, thông tin resize…) màu hổ phách.
BOOLEAN hồng phấn, kiểu bất kỳ (`*`) màu xám, và dữ liệu riêng của NH (thông tin tile, thông tin resize, instance…) màu hổ phách.
- **Các hàng bo tròn** là widget kèm giá trị mặc định. `opt` đánh dấu đầu vào tùy chọn; `≡` sau một đầu ra nghĩa là đầu ra
đó trả về danh sách.
- Chấm **xanh lá** trên tiêu đề là node thường; chấm **cam** là node đầu ra (output node), luôn chạy kể cả khi không có gì
@@ -36,7 +41,7 @@ Mỗi phần node bắt đầu bằng một thẻ được vẽ trực tiếp t
| Ảnh — chỉnh sửa theo mask | [image-retouch](image-retouch.md) | NH Mask-Aware Resize · NH Agnostic Image Generator · NH Simple Face Paste |
| Độ phân giải | [resolution](resolution.md) | NH Smart Resolution Picker · NH Smart Ratio Image Resize · NH Ratio Preset Image Resize · NH Remove Resize Padding · NH Restore Image Geometry |
| Mask | [mask](mask.md) | Mask Morphology · Mask Properties · NH Create Box Mask · NH Mask Aspect Ratio Match |
| Vision | [vision](vision.md) | Garment Segment · Portrait Segment |
| Vision | [vision](vision.md) | Garment Segment · Portrait Segment · SAM3 Segment · Segs Select · Segs Compose · Segs To Masks · Segs To Impact SEGS · Segs Preview |
| VTON | [vton](vton.md) | VTON Ultimate Processor |
| Logic | [logic](logic.md) | Compare · Logic Gate · If/Else · Switch N · Any Switch · Any Branch Switch · Gate Switch · Value Match Index · Math Eval · Random Choice |
| Văn bản | [text](text.md) | Prompt Template · Prompt Scheduler · String Operations · Prompt Join · Text Split · Regex Extract · Text Index · Text Concatenate · Text Split Lines · Text Random Line |
@@ -58,13 +63,17 @@ Mỗi phần node bắt đầu bằng một thẻ được vẽ trực tiếp t
ComfyUI; chi tiết ở [image-io](image-io.md#đường-dẫn-và-loại-tệp).
- **Resize data.** Các node độ phân giải trả về `resize_data` ghi lại ảnh đã được cắt hay thêm viền thế nào, để đưa kết quả
về đúng khung ảnh gốc bằng [NH Restore Image Geometry](resolution.md#nh-restore-image-geometry).
- **Instance.** SAM3 Segment trả về mỗi thứ nó tìm thấy như một instance riêng, có nhãn, điểm và mask, trong một giá trị
`NH_SEGS`. Các node Segs chọn instance, ghép chúng bằng công thức và đổi chúng thành mask; xem
[Các instance: kiểu NH_SEGS](vision.md#các-instance-kiểu-nh_segs).
- **Giấy phép.** Mọi node theo giấy phép MIT, trừ các phần ghi trong [THIRD_PARTY_NOTICES](../../THIRD_PARTY_NOTICES.md):
VTON Ultimate Processor và Portrait Segment chỉ dùng cho mục đích **phi thương mại**, còn model của Garment Segment
không công bố giấy phép.
không công bố giấy phép. SAM3 Segment dùng SAM 3 của Meta theo SAM License, giấy phép không có điều khoản cấm thương mại
nhưng cấm các mục đích quân sự, hạt nhân, gián điệp, súng và vũ khí bất hợp pháp; bạn tự cài gói và tự tải checkpoint.
## Xem thêm
- [Changelog](../CHANGELOG.md) — thay đổi của từng phiên bản, gồm cả các thay đổi về hành vi.
- [Sơ đồ cổng logic](../logic_gate_diagram.html) — bảng chân trị trực quan cho node Logic Gate (tải về và mở bằng trình
duyệt).
- [Third-party notices](../../THIRD_PARTY_NOTICES.md) — giấy phép của mã nguồn đi kèm và các model được tải về.
- [Third-party notices](../../THIRD_PARTY_NOTICES.md) — giấy phép của mã nguồn đi kèm, các model được tải về và SAM 3.
+1 -1
View File
@@ -12,7 +12,7 @@ Ba node làm việc với mask. Một node đổi kích thước ảnh về đú
| [NH Agnostic Image Generator](#nh-agnostic-image-generator) | Xóa vùng mask bằng màu xám, nhiễu hoặc làm mờ |
| [NH Simple Face Paste](#nh-simple-face-paste) | Dán khuôn mặt trong mask của ảnh này lên vùng mask của ảnh khác |
Một `MASK` có màu trắng (`1`) ở nơi node cần làm việc và màu đen (`0`) ở những chỗ còn lại. Bạn lấy mask từ một node phân đoạn như [Garment Segment (NH)](./vision.md#garment-segment-nh) hoặc [Portrait Segment (NH)](./vision.md#portrait-segment-nh), hoặc từ bất kỳ công cụ mask nào trong ComfyUI.
Một `MASK` có màu trắng (`1`) ở nơi node cần làm việc và màu đen (`0`) ở những chỗ còn lại. Bạn lấy mask từ một node phân đoạn như [SAM3 Segment (NH)](./vision.md#sam3-segment-nh) (mọi vật thể gọi được tên, ghép lại bằng [Segs Compose (NH)](./vision.md#segs-compose-nh)), [Garment Segment (NH)](./vision.md#garment-segment-nh) hoặc [Portrait Segment (NH)](./vision.md#portrait-segment-nh), hoặc từ bất kỳ công cụ mask nào trong ComfyUI.
---
+369 -6
View File
@@ -2,17 +2,43 @@
# Vision (phân đoạn ảnh)
Hai node cắt từng phần trên ảnh chụp một người bằng các model SegFormer: trang phục với Garment Segment, và khuôn mặt, tóc, cổ với Portrait Segment. Mỗi node trả về một mask có cùng kích thước với ảnh đầu vào, một ảnh cắt nền có độ trong suốt và một ảnh xem trước mask. Bạn tìm thấy chúng trong **NH-Nodes → Vision**.
Tám node cắt từng phần trên ảnh chụp. Hai node dùng các model SegFormer hiểu về con người: trang phục với Garment Segment, và khuôn mặt, tóc, cổ với Portrait Segment. Mỗi node trả về một mask có cùng kích thước với ảnh đầu vào, một ảnh cắt nền có độ trong suốt và một ảnh xem trước mask. Sáu node còn lại dùng SAM 3, model phân đoạn của Meta: SAM3 Segment tìm mọi instance (từng đối tượng riêng lẻ) của bất cứ thứ gì bạn gọi tên bằng tiếng Anh thông thường (người, ly rượu, cửa sổ, bầu trời), và năm node Segs giúp chọn, ghép, xem trước và chuyển đổi những gì nó tìm được. Bạn tìm thấy chúng trong **NH-Nodes → Vision**.
**Trên trang này**
- [Trước khi bắt đầu](#trước-khi-bắt-đầu) — model, lần chạy đầu tiên, giấy phép, bộ nhớ
- [Tinh chỉnh mask](#tinh-chỉnh-mask) — các tùy chọn dùng chung của hai node
- [Garment Segment (NH)](#garment-segment-nh) — mask trang phục thân trên, thân dưới, giày dép hoặc mũ nón
- [Portrait Segment (NH)](#portrait-segment-nh) — mask khuôn mặt, tóc, cổ và các bộ phận trên mặt
- [Nên dùng node nào?](#nên-dùng-node-nào) — các node SegFormer hay SAM 3
- **Các node SegFormer: trang phục và chân dung**
- [Trước khi bắt đầu](#trước-khi-bắt-đầu) — model, lần chạy đầu tiên, giấy phép, bộ nhớ
- [Tinh chỉnh mask](#tinh-chỉnh-mask) — các tùy chọn dùng chung của hai node
- [Garment Segment (NH)](#garment-segment-nh) — mask trang phục thân trên, thân dưới, giày dép hoặc mũ nón
- [Portrait Segment (NH)](#portrait-segment-nh) — mask khuôn mặt, tóc, cổ và các bộ phận trên mặt
- **Các node SAM 3: bất cứ thứ gì bạn gọi tên được**
- [Trước khi bắt đầu với SAM 3](#trước-khi-bắt-đầu-với-sam-3) — GPU, gói cài đặt, checkpoint, bộ nhớ, giấy phép
- [Cách hai lượt phân tích hoạt động](#cách-hai-lượt-phân-tích-hoạt-động) — `balanced` làm được gì trên ảnh 4K đến 8K, và giới hạn ở đâu
- [Các instance: kiểu NH_SEGS](#các-instance-kiểu-nh_segs) — thứ được chuyển qua lại giữa các node
- [SAM3 Segment (NH)](#sam3-segment-nh) — tìm mọi instance của các concept bạn gọi tên
- [Segs Select (NH)](#segs-select-nh) — giữ instance theo nhãn, điểm, kích thước và vị trí
- [Segs Compose (NH)](#segs-compose-nh) — dựng mask từ nhãn bằng `|`, `-`, `&` và `~`
- [Segs To Masks (NH)](#segs-to-masks-nh) — mỗi instance một mask, cùng mask hợp và danh sách nhãn
- [Segs To Impact SEGS (NH)](#segs-to-impact-segs-nh) — đưa các instance cho detailer của Impact Pack
- [Segs Preview (NH)](#segs-preview-nh) — vẽ các instance lên ảnh của chúng
- [Workflow với SAM 3](#workflow-với-sam-3) — nền phía sau người, vật lớn nhất, mặt tiền không có cửa sổ
- [Khi có sự cố với SAM 3](#khi-có-sự-cố-với-sam-3)
## Nên dùng node nào?
| Bạn muốn | Dùng | Cần gì |
|---|---|---|
| Trang phục của một người: áo, quần, giày, mũ | [Garment Segment](#garment-segment-nh) | Một model SegFormer tự tải ở lần chạy đầu. Chạy được trên CPU hoặc GPU. |
| Khuôn mặt, tóc, cổ và các bộ phận trên mặt của một ảnh chân dung | [Portrait Segment](#portrait-segment-nh) | Giống như trên. |
| Bất cứ thứ gì bạn gọi tên được (người, đồ vật, bầu trời, tường, cửa sổ), từng instance riêng, kể cả trên ảnh 4K đến 8K | [SAM3 Segment](#sam3-segment-nh) và các node Segs | GPU NVIDIA, gói `sam3` của Meta và một checkpoint mà bạn tự cài và tự tải. Xem [Trước khi bắt đầu với SAM 3](#trước-khi-bắt-đầu-với-sam-3). |
Bạn cũng có thể yêu cầu SAM3 Segment tìm `dress` hay `hair`. Các node SegFormer vẫn là lựa chọn dễ hơn cho các bộ phận trang phục và khuôn mặt cố định của chúng: chúng tự thiết lập và cũng chạy được không cần GPU.
## Trước khi bắt đầu
Phần này nói về Garment Segment và Portrait Segment. SAM3 Segment và các node Segs có phần riêng: [Trước khi bắt đầu với SAM 3](#trước-khi-bắt-đầu-với-sam-3).
| | Garment Segment (NH) | Portrait Segment (NH) |
|---|---|---|
| Model | SegFormer huấn luyện cho các lớp trang phục của Fashionpedia | SegFormer tinh chỉnh (fine-tune) trên CelebAMask-HQ cho các bộ phận khuôn mặt |
@@ -30,7 +56,7 @@ Hai node cắt từng phần trên ảnh chụp một người bằng các model
## Tinh chỉnh mask
Cả hai node biến bản đồ nhãn của model thành mask, rồi tinh chỉnh theo thứ tự: `fill_holes`, `mask_expand`, `mask_blur`. Mask luôn có kích thước bằng ảnh đầu vào.
Garment Segment và Portrait Segment biến bản đồ nhãn của model thành mask, rồi tinh chỉnh theo thứ tự: `fill_holes`, `mask_expand`, `mask_blur`. Mask luôn có kích thước bằng ảnh đầu vào.
| Tùy chọn | Khoảng | Garment Segment | Portrait Segment | Tác dụng |
|---|---|---|---|---|
@@ -158,4 +184,341 @@ Tắt mọi vùng trừ `hair`. Với các giá trị mặc định, bạn có k
---
## Trước khi bắt đầu với SAM 3
SAM 3 là model của Meta để tìm vật theo tên: bạn viết thứ mình muốn (`person`, `window`, `wine glass`) và nó trả về mọi instance của thứ đó, mỗi instance có điểm tin cậy (score) và mask riêng. NH-Nodes không kèm model và không bao giờ tự tải nó, nên SAM3 Segment cần bạn chuẩn bị ba thứ, chỉ một lần. Nếu một pack SAM 3 khác đã chạy tốt trong ComfyUI của bạn, bước 2 và 3 thường đã xong: pack đó đã cài gói và đặt `sam3.pt` đúng nơi node này tìm.
**1. GPU NVIDIA có CUDA.** SAM 3 chỉ chạy trên CUDA: mã sam3 chính thức dựng một phần model trên GPU, nên không có chế độ CPU và không có đầu vào `device`. Hãy dự trù khoảng 5 GB VRAM ở mức cao nhất.
**2. Gói `sam3` của Meta.** NH-Nodes không tự cài nó cho bạn. Trong Python đang chạy ComfyUI, hãy chạy hai dòng sau, rồi khởi động lại ComfyUI:
```text
python -m pip install --no-deps git+https://github.com/facebookresearch/sam3@b26a5f330e05d321afb39d01d3d4881f258f65ff
python -m pip install timm ftfy regex iopath
```
Dòng đầu cài đúng commit sam3 mà NH-Nodes đã được thử nghiệm; các commit mới hơn có cùng API ảnh cũng dùng được. Dòng này cần máy bạn có Git. `--no-deps` giữ nguyên numpy của bạn: sam3 yêu cầu numpy dưới 2, điều mà ComfyUI không cần. Trang của Meta cho gói này ghi Python 3.12 trở lên, PyTorch 2.7 trở lên và CUDA 12.6 trở lên. Nếu một pack SAM 3 khác đã cung cấp `sam3`, hãy bỏ qua dòng đầu: cài lại có thể thay thế bản mà pack đó đang dùng.
**3. Checkpoint `sam3.pt`.** Meta chỉ phát trọng số sau khi bạn chấp nhận SAM License của họ, nên NH-Nodes không thể tải giúp bạn. Hãy xin quyền truy cập tại [huggingface.co/facebook/sam3](https://huggingface.co/facebook/sam3), tải `sam3.pt` từ trang đó và đặt vào `ComfyUI/models/sam3/`. Nhấn R trong ComfyUI để làm mới danh sách, rồi chọn tệp trong `checkpoint`. Muốn để ở nơi khác, hãy thêm thư mục đó vào khóa `sam3` trong `extra_model_paths.yaml` (xem tệp này ở mục [Installation](../../README.md#installation)):
```yaml
my_models:
base_path: D:/ai-models/
sam3: sam3/ # checkpoint SAM 3, dùng chung với các pack SAM 3 khác
```
Chỉ checkpoint PyTorch `sam3.pt` dùng được. Bản safetensors trên Hugging Face dành cho thư viện transformers và dùng tên trọng số khác.
**Bộ nhớ.** Trọng số được đọc từ đĩa một lần và giữ trong bộ nhớ hệ thống (khoảng 3.6 GB). Chúng chuyển lên GPU khi SAM3 Segment chạy và trở về sau đó, nên ComfyUI nhận lại VRAM. Trước khi chuyển, node nhờ ComfyUI gỡ bớt các model của chính nó nếu VRAM đang eo hẹp. Hãy bật `keep_model_loaded` để trọng số ở lại trên GPU nếu bạn chạy node liên tục.
> **Giấy phép:** SAM 3 được phát hành theo SAM License của Meta. Giấy phép này miễn phí bản quyền (royalty-free) và không có điều khoản cấm thương mại, nhưng cấm những mục đích thuộc ITAR hoặc bị kiểm soát thương mại (trade controls) cấm, gồm quân sự hay chiến tranh, hạt nhân, gián điệp, súng và vũ khí bất hợp pháp. Mã nguồn của NH-Nodes thuộc giấy phép MIT và không chứa model lẫn gói `sam3`. Hãy đọc [THIRD_PARTY_NOTICES.md](../../THIRD_PARTY_NOTICES.md) trước khi dùng node cho mục đích thương mại.
---
## Cách hai lượt phân tích hoạt động
SAM 3 nhìn một bức ảnh cỡ 1000 px. Trên ảnh 6000 px, đó chỉ bằng một phần sáu bề ngang, nên vật nhỏ và các cạnh mảnh sẽ bị nhòe. Vì vậy SAM3 Segment nhìn hai lượt.
1. **Lượt đầu: cả bức ảnh.** Ảnh được thu nhỏ về đầu vào vuông của model (1008 px) và mã hóa một lần. Mọi concept bạn liệt kê được hỏi trên đúng lần mã hóa đó, rất nhẹ. Kết quả cho mỗi instance một điểm, một khung (box) và một mask đầu tiên, được lấy mẫu lại lên độ phân giải đầy đủ, chỉ trong phạm vi khung của chính nó.
2. **Lượt hai: ảnh cận cảnh của từng instance** (chỉ với `balanced`). Instance được cắt khỏi ảnh độ phân giải đầy đủ kèm một lề (20% cạnh dài của nó, tối thiểu 16 px) và được phân đoạn lại bằng bộ giải mã dùng khung gợi ý (box-prompt decoder) của SAM 3, lấy mask lượt đầu làm gợi ý. Lúc này cạnh của nó đến từ các điểm ảnh gốc.
3. **Cơ chế dự phòng.** Mask cận cảnh được giữ khi nó khớp với instance của lượt đầu trong phạm vi độ nhòe của lượt đầu, nên những thứ mảnh như cột và dây cáp vẫn còn. Khi không khớp, mask lượt đầu được giữ lại, và console cho biết số instance bị như vậy. Các mảnh vụn của mask mới không chạm vào mask lượt đầu bị bỏ đi.
| `detail_level` | Lượt | Dùng khi |
|---|---|---|
| `balanced` (mặc định) | Cả hai | Cạnh sắc nét cho vật nhỏ và vừa, kể cả trên ảnh 4K đến 8K. Khoảng 0.1 giây mỗi instance trên GPU nhanh. |
| `fast` | Chỉ lượt đầu | Ảnh nhỏ, và khi thử nhanh để tìm từ đúng cho concept. |
**`balanced` làm được và không làm được gì.** Vật nhỏ và vừa (một ô cửa sổ, một chiếc ly, một người trong ảnh nhóm) có cạnh sắc nét ngay cả trên ảnh 4K đến 8K, vì mỗi vật được xét trên một ảnh cận cảnh của các điểm ảnh gốc. Giới hạn nằm ở model: SAM 3 mô tả bất cứ thứ gì nó nhìn bằng một lưới khoảng 288 ô. Một đối tượng chiếm gần hết khung hình 8K tự nó là ảnh cận cảnh, nên cạnh của nó chỉ chính xác đến khoảng 1/300 kích thước của nó, tức cỡ 25 px với đối tượng rộng 7000 px. Mức đó ổn cho đường viền của một người hay một tòa nhà, và còn mềm ở tóc, ren và lông. Cạnh ở mức sợi tóc cần một bước tách nền mịn (matting) mà NH-Nodes chưa có; một node Matte Refine đang được dự kiến.
**Đo trên một máy** (NVIDIA RTX 5090): lần chạy đầu nạp model mất khoảng 9 giây. Một ảnh 7360×4912 với các concept `person, wine glass, dress` cho 13 instance trong khoảng 3.6 giây với `balanced`. Một ảnh nội thất 6720×4480 với 8 concept cho 29 instance trong khoảng 6 giây. Mức VRAM cao nhất khoảng 5 GB. GPU khác sẽ cho số khác.
---
## Các instance: kiểu NH_SEGS
`NH_SEGS` là dữ liệu mà SAM3 Segment chuyển cho các node Segs. Với mỗi ảnh trong batch (lô ảnh), nó chứa một danh sách instance. Mỗi instance giữ **nhãn** (concept bạn đã viết), **điểm tin cậy**, một **khung** ôm sát và mask **chỉ trong phạm vi khung đó**. Mask kích thước đầy đủ chỉ được dựng khi một node xuất `MASK`, nên hàng chục instance trên ảnh 8K vẫn nhẹ.
- Các instance xếp theo thứ tự concept bạn viết, và theo điểm trong cùng một concept.
- Các node Segs xử lý từng ảnh trong batch riêng biệt.
- Các node gốc của ComfyUI không đọc được `NH_SEGS`. Để lấy mask ra, hãy dùng các đầu ra `mask`, [Segs Compose](#segs-compose-nh) hoặc [Segs To Masks](#segs-to-masks-nh). Để dùng detailer của Impact Pack, hãy chuyển đổi bằng [Segs To Impact SEGS](#segs-to-impact-segs-nh).
---
## SAM3 Segment (NH)
<p align="center"><img src="../assets/nodes/NH_SAM3Segment.svg" alt="SAM3 Segment (NH) node" width="360"></p>
Tìm mọi instance của những thứ bạn gọi tên và trả về chúng dưới dạng `NH_SEGS`, kèm mask gộp của chúng và số lượng. Muốn xem nó tìm được gì, hãy nối [Segs Preview](#segs-preview-nh).
**Dùng khi** bạn muốn có mask của "mọi người", "mọi cửa sổ", "bầu trời" hay "các ly rượu" trong một bức ảnh, kể cả ảnh 4K đến 8K, mà không phải vẽ gì. Sau đó dùng các [node Segs](#segs-select-nh) để chọn instance, trừ cái này khỏi cái kia, hoặc đổi chúng thành mask thông thường.
Các đầu vào COMBO không nêu mặc định sẽ bắt đầu ở lựa chọn đầu tiên.
| Đầu vào | Kiểu | Mặc định | Mô tả |
|---|---|---|---|
| `image` | IMAGE | — | Bức ảnh. Node dùng màu (RGB), kênh alpha nếu có sẽ bị bỏ. Mọi ảnh trong batch được xử lý lần lượt. |
| `concepts` | STRING | `person` | Thứ cần tìm: các cụm danh từ tiếng Anh ngắn, mỗi dòng một cụm hoặc cách nhau bằng dấu phẩy (`person, red car, window, sky`). Dòng bắt đầu bằng `#` bị bỏ qua. Mỗi cụm trở thành nhãn của các instance nó tìm được. Hãy viết ít nhất một cụm. |
| `threshold` | FLOAT | `0.5` | Khoảng 0.05 đến 0.95. Độ tin cậy tối thiểu. Thấp hơn thì tìm được nhiều instance hơn và cũng nhiều nhầm lẫn hơn; cao hơn chỉ giữ những cái chắc chắn. |
| `detail_level` | COMBO | `balanced` | `balanced` hoặc `fast`. `balanced` phân đoạn lại mỗi instance lần thứ hai trên ảnh cận cảnh cắt từ ảnh độ phân giải đầy đủ, để vật nhỏ và vừa có cạnh sắc nét. `fast` giữ lượt đầu trên cả bức ảnh. Xem [Cách hai lượt phân tích hoạt động](#cách-hai-lượt-phân-tích-hoạt-động). |
| `max_per_concept` | INT | `0` | Khoảng 0 đến 200. Giữ tối đa bấy nhiêu instance cho mỗi concept, điểm cao nhất trước. `0` giữ tất cả. |
| `checkpoint` | COMBO | — | Một tệp `.pt` hoặc `.pth` của SAM 3 trong `models/sam3` hoặc thư mục khác đã đăng ký với tên `sam3`. Các pack SAM 3 khác dùng chung thư mục này. Khi thư mục chưa có tệp nào, danh sách hiện `sam3.pt` và node cho bạn biết lấy nó ở đâu khi bạn chạy. |
| `keep_model_loaded` | BOOLEAN | `false` | Tắt: sau lần chạy, trọng số trở về bộ nhớ hệ thống và ComfyUI nhận lại VRAM. Bật: chúng ở lại trên GPU (khoảng 3.6 GB), giúp lần chạy sau nhanh hơn, nhưng ComfyUI không dùng được phần VRAM đó trong lúc ấy. |
| Đầu ra | Kiểu | Mô tả |
|---|---|---|
| `segs` | NH_SEGS | Các instance của từng ảnh, mỗi instance có nhãn, điểm, khung và mask. Hãy nối nó vào các node Segs. |
| `mask` | MASK | Hợp của mọi instance, một mask cho mỗi ảnh, có kích thước bằng ảnh. |
| `count` | INT | Số instance tìm được, tính trên cả batch. |
<p align="center"><img src="../assets/examples/sam3-segment.jpg" alt="Các instance của SAM3 Segment do Segs Preview vẽ, và ba mask tạo từ chúng trên một ảnh mẫu" width="720"><br><em>SAM3 Segment trên một ảnh mẫu với các concept "person, helmet, face", do Segs Preview vẽ, rồi ba mask tạo từ các instance bằng Segs Compose và Segs Select.</em></p>
**Cách hoạt động**
- **Nhãn.** Concept bạn viết là nhãn của các instance của nó. Các node Segs bỏ qua chữ hoa/thường khi so khớp nhãn, nên cách viết hoa không thành vấn đề.
- **Thứ tự và bản trùng.** Các instance xếp theo thứ tự concept của bạn, điểm cao nhất trước trong mỗi concept. Hai instance của cùng một concept mà mask gần như trùng hoàn toàn (từ 90% trở lên) được tính là một, giữ lại cái điểm cao hơn. Sau đó `max_per_concept` giữ những cái tốt nhất trong phần còn lại.
- **Xem kết quả.** Nối `segs` và đúng bức ảnh đó vào [Segs Preview](#segs-preview-nh): mỗi instance được tô màu, có viền và gắn nhãn cùng điểm của nó.
- **Không tìm thấy gì** không phải lỗi: bạn nhận được không instance nào, một mask rỗng và `count` bằng 0.
- **Batch.** Mỗi ảnh được xử lý lần lượt và có danh sách instance riêng. Thanh tiến trình đếm theo concept, và bạn có thể hủy lần chạy giữa các concept và giữa các ảnh cận cảnh.
- **Ảnh rất rộng hoặc rất cao.** Trước hết SAM 3 nhìn cả bức ảnh bị nén vào một hình vuông. Với ảnh rộng hoặc cao hơn 4:1 (ảnh toàn cảnh), nó có thể bỏ sót vật nhỏ, và console cho biết điều đó. Hãy chia ảnh như vậy thành các ô bằng [Image Tile (NH)](./image-compose.md#image-tile-nh) rồi chạy từng ô.
- **Lỗi cài đặt** làm node dừng lại với thông báo nói rõ cần làm gì: thiếu gói, thiếu checkpoint, không có GPU CUDA. Xem [Khi có sự cố với SAM 3](#khi-có-sự-cố-với-sam-3).
> **Mẹo:** Hãy viết danh từ tiếng Anh đơn giản: `window`, `wine glass`, `red car`. Nếu một concept không tìm thấy gì, hãy thử từ khác (một tòa nhà có hình dạng lạ có thể đáp ứng `structure` hoặc `monument` thay vì `building`) hoặc hạ `threshold`.
> **Mẹo:** Hãy yêu cầu cả tổng thể lẫn các phần của nó (`person, hair, face, dress`), rồi tách chúng ra bằng [Segs Compose](#segs-compose-nh): `person - hair`.
> **Giấy phép:** SAM 3 thuộc SAM License của Meta. Xem [Trước khi bắt đầu với SAM 3](#trước-khi-bắt-đầu-với-sam-3).
---
## Segs Select (NH)
<p align="center"><img src="../assets/nodes/NH_SegsSelect.svg" alt="Segs Select (NH) node" width="360"></p>
Giữ lại những instance bạn muốn, theo nhãn, điểm, kích thước và vị trí, theo thứ tự bạn chọn.
**Dùng khi** bạn chỉ cần một phần những gì SAM3 Segment tìm được: vật lớn nhất, người thứ hai từ trái sang, chỉ những kết quả chắc chắn, hay chỉ những ô cửa lớn.
| Đầu vào | Kiểu | Mặc định | Mô tả |
|---|---|---|---|
| `segs` | NH_SEGS | — | Các instance, từ SAM3 Segment hoặc một node Segs khác. |
| `labels` | STRING | rỗng | Các nhãn cần giữ, cách nhau bằng dấu phẩy. `*` thay cho một chuỗi ký tự bất kỳ và `?` thay cho một ký tự (`*shirt`). Không phân biệt chữ hoa/thường, và `_` được coi là dấu cách (`red_car` tìm `red car`). Nhãn có dấu cách không cần ngoặc kép ở đây: `wine glass, red car`. Để trống thì giữ mọi nhãn. |
| `min_score` | FLOAT | `0.0` | Khoảng 0 đến 1. Loại các instance có điểm thấp hơn. |
| `min_area` | FLOAT | `0.0` | Khoảng 0 đến 1. Loại các instance nhỏ hơn tỉ lệ này của bức ảnh (`0.01` là 1%). |
| `max_area` | FLOAT | `1.0` | Khoảng 0 đến 1. Loại các instance lớn hơn tỉ lệ này của bức ảnh. |
| `sort_by` | COMBO | `score` | Thứ tự của các instance được giữ. Xem bảng bên dưới. |
| `start` | INT | `0` | Khoảng 0 đến 4096. Bỏ qua bấy nhiêu instance đầu của danh sách đã sắp xếp. `0` bắt đầu từ instance đầu tiên. |
| `count` | INT | `0` | Khoảng 0 đến 4096. Giữ bấy nhiêu instance kể từ `start`. `0` giữ tất cả. |
| Đầu ra | Kiểu | Mô tả |
|---|---|---|
| `segs` | NH_SEGS | Các instance được giữ, theo thứ tự đã sắp xếp. |
| `mask` | MASK | Hợp của các instance được giữ, một mask cho mỗi ảnh. |
| `count` | INT | Số instance được giữ, tính trên cả batch. |
| `sort_by` | Thứ tự |
|---|---|
| `score` | Điểm cao nhất trước. |
| `area` | Lớn nhất trước. |
| `left_to_right`, `right_to_left` | Theo tâm ngang của khung. |
| `top_to_bottom`, `bottom_to_top` | Theo tâm dọc của khung. |
| `center_first` | Instance có tâm khung gần tâm bức ảnh nhất đứng trước. |
| `as_found` | Theo thứ tự SAM3 Segment tìm ra: lần lượt từng concept của bạn, điểm cao nhất trước trong mỗi concept. |
**Cách hoạt động**
- **Ba bước, theo thứ tự này:** các bộ lọc (`labels`, `min_score`, `min_area`, `max_area`), rồi sắp xếp, rồi cắt lấy một đoạn (`start` và `count`).
- **Theo từng ảnh.** Mỗi ảnh trong batch được xử lý riêng, nên `start` `1` bỏ qua instance đầu tiên của từng ảnh.
- **Diện tích** là tỉ lệ số điểm ảnh của bức ảnh mà mask phủ, không phải kích thước khung của nó. Vị trí dùng tâm của khung. Khi bằng nhau, thứ tự lúc tìm ra được giữ nguyên.
- Các instance được giữ vẫn theo thứ tự đã sắp, nên [Segs To Masks](#segs-to-masks-nh) đặt sau node này trả về các mask theo đúng thứ tự đó.
| Muốn có | Đặt |
|---|---|
| Người thứ hai từ trái sang | `labels` `person`, `sort_by` `left_to_right`, `start` `1`, `count` `1` |
| Vật lớn nhất | `sort_by` `area`, `count` `1` |
| Ba instance chắc chắn nhất | `sort_by` `score`, `count` `3` |
| Không có gì nhỏ hơn 1% bức ảnh | `min_area` `0.01` |
| Instance gần giữa ảnh nhất | `sort_by` `center_first`, `count` `1` |
---
## Segs Compose (NH)
<p align="center"><img src="../assets/nodes/NH_SegsCompose.svg" alt="Segs Compose (NH) node" width="360"></p>
Tạo một mask từ nhãn của các instance bằng một công thức nhỏ: hợp, trừ, giao và "mọi thứ trừ...".
**Dùng khi** bạn muốn có "phần nền", "người nhưng không có tóc", "bức tường nhưng không có cửa sổ", hoặc bất kỳ tổ hợp nào khác của những thứ SAM3 Segment tìm được.
| Đầu vào | Kiểu | Mặc định | Mô tả |
|---|---|---|---|
| `segs` | NH_SEGS | — | Các instance, từ SAM3 Segment hoặc Segs Select. |
| `expression` | STRING | `all` | Công thức. Xem bên dưới. Ô này có nhiều dòng, và xuống dòng được coi là dấu cách. |
| Đầu ra | Kiểu | Mô tả |
|---|---|---|
| `mask` | MASK | Kết quả của công thức, một mask cho mỗi ảnh, có kích thước bằng ảnh. |
**Những gì bạn có thể viết**
| Viết | Nghĩa |
|---|---|
| `person` | Một nhãn: hợp của mọi instance mang nhãn đó. Không phân biệt chữ hoa/thường, `_` được coi là dấu cách, và `*`, `?` là ký tự đại diện (wildcard) (`*shirt`). |
| `"red car"` | Đặt nhãn có dấu cách hoặc ký tự toán tử trong ngoặc kép, `"` hoặc `'`, hoặc viết dấu cách thành `_` (`red_car`). Không có ngoặc kép, `red car` là lỗi. Ngoặc kép cũng biến một từ như `all` hay `image` thành nhãn thông thường. |
| `t-shirt` | Dấu gạch nối nằm giữa hai chữ cái hoặc chữ số thuộc về nhãn, nên `t-shirt` và `x-ray` không cần ngoặc kép. Dấu nháy đơn nằm giữa một từ cũng được (`women's`). Nhãn có dấu cách cần ngoặc kép, và dùng ngoặc kép đôi khi nhãn còn có dấu nháy đơn: `"women's dress"`. |
| `a \| b`, `a + b`, `a or b` | Hợp: mọi thứ thuộc a hoặc thuộc b. |
| `a - b` | a bỏ đi phần của b. Hãy đặt dấu cách hai bên dấu trừ: `person-hair` được đọc là một nhãn duy nhất. Khi cả `person` và `hair` đều có, node dừng với lỗi gợi ý `person - hair`; nếu không, nhãn lạ cho phần rỗng và console ghi tên nhãn đó. |
| `a & b`, `a and b` | Giao: chỉ nơi cả a và b cùng có. |
| `~a`, `!a`, `not a` | Mọi thứ trừ a: cả bức ảnh bỏ đi a. |
| `( … )` | Nhóm. |
| `all` | Mọi instance trong segs. |
| `image` | Cả bức ảnh. |
| `none` | Một mask rỗng. |
`~` được tính trước, rồi `&`, rồi `|` và `-` cùng cấp, từ trái sang phải: `a - b | c` nghĩa là `(a - b) | c`. Khi còn phân vân, hãy thêm dấu ngoặc.
| Biểu thức | Bạn nhận được |
|---|---|
| `person - hair` | Người nhưng bỏ phần tóc (hãy yêu cầu SAM3 Segment `person, hair`). |
| `~person` | Mọi thứ trừ người: phần nền. |
| `(wall \| ceiling) - window` | Tường và trần, khoét bỏ các ô cửa sổ. |
| `"wine glass" \| person` | Các ly rượu và người cùng lúc. |
| `all - person` | Mọi thứ đã tìm được trừ người. Khác với `~person`, nó không gồm phần nền mà không instance nào phủ tới. |
**Cách hoạt động**
- **Nhãn mà không instance nào có** cho một phần rỗng, và console báo điều đó (`no instance is labelled 'x'; those parts are empty`). Vì thế gõ sai `~persn` sẽ cho cả bức ảnh: hãy xem console khi mask trông lạ.
- **Mỗi ảnh** trong batch có mask riêng, dựng từ chính các instance của nó.
- **Công thức không đọc được** làm node dừng lại với thông báo nêu vị trí của chỗ sai, ví dụ `unexpected 'car' at position 5`.
---
## Segs To Masks (NH)
<p align="center"><img src="../assets/nodes/NH_SegsToMasks.svg" alt="Segs To Masks (NH) node" width="360"></p>
Đổi các instance thành mask thông thường: một mask cho mỗi instance, mask hợp của từng ảnh, và danh sách nhãn.
**Dùng khi** bạn cần mỗi instance là một mask riêng: để xử lý từng cái một, để lưu lại, hoặc để đọc nhãn của chúng.
| Đầu vào | Kiểu | Mặc định | Mô tả |
|---|---|---|---|
| `segs` | NH_SEGS | — | Các instance, từ SAM3 Segment hoặc Segs Select. |
| Đầu ra | Kiểu | Mô tả |
|---|---|---|
| `masks` | MASK | Một mask cho mỗi instance, ảnh này sang ảnh khác, theo thứ tự của `labels`. Khi hoàn toàn không có instance nào, đầu ra là một mask rỗng duy nhất, để các node phía sau không bị lỗi. |
| `union` | MASK | Một mask cho mỗi ảnh: hợp các instance của ảnh đó. |
| `labels` | STRING | Nhãn của từng instance, mỗi dòng một nhãn, theo thứ tự của `masks`. |
| `count` | INT | Số instance. Bằng 0 khi không có instance nào, dù lúc đó `masks` chứa một mask rỗng. |
> **Lưu ý:** Mỗi mask có kích thước bằng bức ảnh và 4 byte cho mỗi điểm ảnh: khoảng 145 MB trên ảnh 7360×4912. Khi các mask cần hơn khoảng một nửa dung lượng RAM còn trống của bạn (ví dụ hàng trăm instance trên ảnh 8K), node dừng lại với một thông báo thay vì làm đầy bộ nhớ. Hãy dùng [Segs Select](#segs-select-nh) trước để chỉ giữ những instance bạn cần, hoặc [Segs Compose](#segs-compose-nh) để lấy một mask duy nhất.
---
## Segs To Impact SEGS (NH)
<p align="center"><img src="../assets/nodes/NH_SegsToImpactSEGS.svg" alt="Segs To Impact SEGS (NH) node" width="360"></p>
Chuyển các instance của một ảnh sang `SEGS` của Impact Pack, để các detailer và công cụ SEGS của nó làm việc được với chúng.
**Dùng khi** bạn đã dùng Impact Pack và muốn các detailer của nó tinh chỉnh những gì SAM3 Segment tìm được. NH-Nodes không cần Impact Pack cho bất kỳ việc nào khác, nên hãy bỏ qua node này nếu bạn không dùng nó.
| Đầu vào | Kiểu | Mặc định | Mô tả |
|---|---|---|---|
| `segs` | NH_SEGS | — | Các instance, từ SAM3 Segment hoặc Segs Select. |
| `crop_factor` | FLOAT | `3.0` | Khoảng 1 đến 10. Kích thước vùng mà detailer xử lý quanh mỗi instance, tính bằng bội số của khung của nó (cũng là `crop_factor` của Impact Pack). Vùng được dịch để nằm trong ảnh và luôn chứa trọn khung. |
| `batch_index` | INT | `0` | Khoảng 0 đến 4095. SEGS của Impact Pack chỉ chứa một ảnh, nên đầu vào này chọn ảnh nào trong batch được chuyển. Số vượt quá ảnh cuối làm node dừng lại với một thông báo. |
| Đầu ra | Kiểu | Mô tả |
|---|---|---|
| `segs` | SEGS | `SEGS` của Impact Pack cho ảnh đó. Mỗi instance có khung, vùng cắt (crop region), mask, nhãn và điểm (là `confidence`). |
**Cách hoạt động**
- Đầu ra là kiểu riêng của Impact Pack, không phải `NH_SEGS`. NH-Nodes dựng nó với cùng các trường và không import Impact Pack, nên việc chuyển đổi không cần Impact Pack. Còn dùng kết quả thì cần.
- Muốn chuyển nhiều ảnh trong batch, hãy dùng mỗi `batch_index` một node.
---
## Segs Preview (NH)
<p align="center"><img src="../assets/nodes/NH_SegsPreview.svg" alt="Segs Preview (NH) node" width="360"></p>
Vẽ các instance lên ảnh của chúng: mỗi instance được tô màu riêng, có viền và gắn nhãn cùng điểm của nó.
**Dùng khi** bạn muốn xem SAM3 Segment đã tìm được gì, hoặc Segs Select và Segs Compose đang làm việc với cái gì. Hãy đặt node này sau SAM3 Segment để xem tất cả, hoặc sau Segs Select để chỉ xem phần đã chọn.
| Đầu vào | Kiểu | Mặc định | Mô tả |
|---|---|---|---|
| `image` | IMAGE | — | Bức ảnh mà segs được tìm ra từ đó: cùng số ảnh và cùng kích thước. Nếu không, node dừng lại và nói rõ. |
| `segs` | NH_SEGS | — | Các instance cần vẽ. |
| Đầu ra | Kiểu | Mô tả |
|---|---|---|
| `preview` | IMAGE | Bức ảnh với các instance được vẽ lên, cùng kích thước. |
Màu sắc theo thứ tự của các instance, mỗi nhãn nằm phía trên điểm cao nhất của instance của nó, và cỡ chữ của nhãn theo kích thước bức ảnh. Ảnh 8K cho ra bản xem trước 8K.
---
## Workflow với SAM 3
Trong mỗi workflow dưới đây, hãy kiểm tra những gì SAM3 Segment tìm được trước khi chỉnh sửa bất cứ thứ gì. Nối đúng bức ảnh đó và `segs` vào [Segs Preview (NH)](#segs-preview-nh): mọi instance bạn muốn đều phải có viền và nhãn. Nếu thiếu một cái, hãy hạ `threshold` (ví dụ xuống 0.3) hoặc thử từ khác cho concept.
### Nền phía sau người
```text
Load Image → SAM3 Segment (concepts: person)
→ Segs Compose (expression: ~person)
→ Mask Morphology (NH)
→ chỉnh sửa phần nền
```
`~person` là cả bức ảnh trừ những người trong đó. Hãy thêm những thứ người đang cầm hoặc ngồi lên vào cả hai ô, ví dụ concept `person, wine glass, chair` và biểu thức `~(person | "wine glass" | chair)`. Dịch cạnh của mask vài pixel bằng [Mask Morphology (NH)](./mask.md#mask-morphology-nh): mở rộng để phần chỉnh sửa phủ lên đường viền cũ, hoặc thu hẹp nếu phần chỉnh sửa lấn vào người. Sau đó dùng mask này để làm mờ, đổi ánh sáng, đổi màu hoặc inpaint phần nền.
### Vật lớn nhất, rồi inpaint
```text
Load Image → SAM3 Segment (concepts: person, dog, car)
→ Segs Select (sort_by: area, count: 1)
→ Mask Morphology (NH, mở rộng)
→ inpaint
```
Khi `labels` để trống, Segs Select chọn cái lớn nhất trong mọi thứ bạn yêu cầu. Hãy đặt `labels` (ví dụ `car`) để chọn cái lớn nhất của một loại, `start` `1` để lấy cái lớn thứ hai, và tăng `min_score` để một kết quả đáng ngờ không thể bị chọn. Hãy mở rộng mask thêm vài pixel để vùng inpaint phủ lên đường viền cũ.
### Kiến trúc: mặt tiền không có cửa sổ
```text
Load Image → SAM3 Segment (concepts: building, window, door)
→ Segs Compose (expression: building - (window | door))
→ đổi màu, đổi phong cách hoặc inpaint bức tường
```
`building` phủ cả mặt tiền cùng cửa sổ và cửa ra vào; công thức khoét chúng ra và chừa lại bức tường. Với một căn phòng, hãy yêu cầu `wall, ceiling, window` và dùng `(wall | ceiling) - window`. Nếu `building` không tìm thấy gì trên một tòa nhà có hình dạng lạ, hãy thử `structure` hoặc `monument` và sửa công thức cho khớp. Trên ảnh 4K đến 8K, hãy giữ `balanced`: mỗi cửa sổ được phân đoạn lại trên một ảnh cận cảnh của các điểm ảnh gốc. Đường viền của một mặt tiền chiếm trọn khung hình chỉ chính xác đến khoảng 1/300 bề ngang của nó (xem [Cách hai lượt phân tích hoạt động](#cách-hai-lượt-phân-tích-hoạt-động)), nên hãy kiểm tra đường mái trong Segs Preview.
---
## Khi có sự cố với SAM 3
| Bạn thấy | Cần làm |
|---|---|
| Lỗi báo SAM 3 cần gói sam3 của Meta | Cài nó như trong [Trước khi bắt đầu với SAM 3](#trước-khi-bắt-đầu-với-sam-3), rồi khởi động lại ComfyUI. |
| Lỗi báo gói sam3 đã cài nhưng không nạp được, hoặc thiếu một thành phần | Phiên bản đã cài không phải bản NH-Nodes hỗ trợ, hoặc thiếu một gói đi kèm. Hãy cài lại bằng hai dòng trong [Trước khi bắt đầu với SAM 3](#trước-khi-bắt-đầu-với-sam-3). Thông báo cho biết nguyên nhân. |
| Lỗi báo không tìm thấy checkpoint | Tải `sam3.pt` như trong [Trước khi bắt đầu với SAM 3](#trước-khi-bắt-đầu-với-sam-3), nhấn R trong ComfyUI và chọn tệp. Thông báo liệt kê các thư mục đã tìm. |
| Lỗi báo một tệp không phải checkpoint `.pt` của SAM 3 | Hãy dùng `sam3.pt`. Các tệp safetensors trên Hugging Face không dùng được với node này. |
| Lỗi báo không đọc được một tệp như checkpoint SAM 3, hoặc tệp đó không phải checkpoint SAM 3 | Tệp trong thư mục `sam3` là bản tải dở hoặc là `.pt` của model khác. Hãy tải lại `sam3.pt` như trong [Trước khi bắt đầu với SAM 3](#trước-khi-bắt-đầu-với-sam-3). |
| Lỗi báo SAM 3 cần GPU NVIDIA có CUDA | ComfyUI đang chạy trên CPU hoặc trên loại GPU khác. SAM3 Segment không có chế độ CPU. |
| Lỗi báo bạn phải viết ít nhất một concept | `concepts` đang trống hoặc chỉ có dòng ghi chú. Hãy viết một từ như `person`. |
| `count` bằng 0, hoặc thiếu một instance | Hạ `threshold`, thử từ khác cho concept, và xem kết quả bằng Segs Preview. |
| Console báo đã giữ mask lượt đầu cho một số instance | Ảnh cận cảnh của chúng không khớp với lượt đầu, nên các instance đó giữ cạnh của lượt đầu. Không có gì sai. |
| Lỗi từ Segs Preview về kích thước hoặc số ảnh | Hãy nối đúng bức ảnh mà segs được tìm ra từ đó. |
| Lỗi từ Segs Compose có nêu một vị trí | Kiểm tra công thức tại vị trí đó. Đặt nhãn có dấu cách trong ngoặc kép: `"red car"`. |
| Mask của Segs Compose rỗng, và console báo không instance nào có một nhãn | Kiểm tra chính tả của nhãn. Đặt dấu cách quanh dấu trừ: `person - hair`, không phải `person-hair`. |
| Lỗi từ Segs To Masks báo các mask instance kích thước đầy đủ cần nhiều bộ nhớ hơn mức còn trống | Có quá nhiều instance so với kích thước bức ảnh. Dùng Segs Select để chỉ giữ những instance bạn cần, hoặc Segs Compose để lấy một mask duy nhất. |
| Console báo SAM 3 nhìn bức ảnh bị nén vào hình vuông và có thể bỏ sót vật nhỏ | Bức ảnh rộng hoặc cao hơn 4:1. Hãy chia nó thành các ô bằng Image Tile (NH) rồi chạy từng ô. |
| ComfyUI hết VRAM sau khi node chạy | Để `keep_model_loaded` tắt, để trọng số rời GPU sau mỗi lần chạy. |
---
[← Về trang hướng dẫn](./README.md)
+92
View File
@@ -0,0 +1,92 @@
"""SAM3 Segment (NH): every instance of the concepts you name, as NH_SEGS plus their union mask.
The model work lives in core/sam3_backend.py; this module only maps the node's inputs and outputs.
"""
from ...core import sam3_backend as sam3
from ...core.segs import SEGS_TYPE, count_segments, parse_label_list, segs_union
try:
from comfy.model_management import throw_exception_if_processing_interrupted
except Exception: # outside ComfyUI (tests)
def throw_exception_if_processing_interrupted():
return None
try:
from comfy.utils import ProgressBar
except Exception: # outside ComfyUI (tests)
ProgressBar = None
class NH_SAM3Segment:
"""Find every instance of each concept with SAM 3 (Meta), at the full resolution of the image."""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"image": ("IMAGE",),
"concepts": ("STRING", {
"default": "person", "multiline": True,
"tooltip": "What to find: short English noun phrases, one per line or separated by commas "
"(person, red car, window, sky). Each phrase becomes the label of its instances.",
}),
"threshold": ("FLOAT", {
"default": 0.5, "min": 0.05, "max": 0.95, "step": 0.01,
"tooltip": "Minimum confidence. Lower finds more (and more mistakes), higher keeps only sure ones.",
}),
"detail_level": (list(sam3.DETAIL_LEVELS), {
"default": "balanced",
"tooltip": "balanced: each instance is segmented again on a close-up cut from the full-resolution "
"image, so small and medium objects get crisp edges even on 4K-8K pictures (about "
"0.1 s per instance on a fast GPU). A subject filling most of the frame keeps edges "
"at about 1/300 of its size. fast: one look at the whole image only.",
}),
"max_per_concept": ("INT", {
"default": 0, "min": 0, "max": 200, "step": 1,
"tooltip": "Keep at most this many instances of each concept, best first. 0 keeps all.",
}),
"checkpoint": (sam3.checkpoint_choices(), {
"tooltip": "SAM 3 checkpoint in models/sam3 (shared with other SAM 3 packs). Download sam3.pt from "
"huggingface.co/facebook/sam3 after accepting Meta's license.",
}),
"keep_model_loaded": ("BOOLEAN", {
"default": False,
"tooltip": "Keep SAM 3 on the GPU after the run (about 3.6 GB): faster when it runs again, but "
"ComfyUI cannot use that VRAM meanwhile.",
}),
}
}
RETURN_TYPES = (SEGS_TYPE, "MASK", "INT")
RETURN_NAMES = ("segs", "mask", "count")
FUNCTION = "segment"
CATEGORY = "NH-Nodes/Vision"
DESCRIPTION = (
"Find every instance of the concepts you name (people, garments, objects, sky, walls, windows...) with "
"Meta's SAM 3. Outputs the instances (segs) for Segs Select / Compose / To Masks / Preview, their union mask "
"and their number. Needs the sam3 package, models/sam3/sam3.pt and a CUDA GPU."
)
SEARCH_ALIASES = ["sam3", "sam 3", "segment anything", "text segment", "concept segment", "object mask",
"detect objects", "instance segmentation"]
def segment(self, image, concepts, threshold, detail_level, max_per_concept, checkpoint, keep_model_loaded):
names = parse_label_list(concepts)
if not names:
raise ValueError("[NH-Nodes] SAM3 Segment: write at least one concept, e.g. 'person' or 'window'.")
bar = ProgressBar(image.shape[0] * len(names)) if ProgressBar is not None else None
segs = sam3.segment(
image, names, threshold, detail_level, max_per_concept, checkpoint, keep_model_loaded,
interrupt=throw_exception_if_processing_interrupted,
progress=(lambda done, total: bar.update_absolute(done, total)) if bar is not None else None,
)
return segs, segs_union(segs), count_segments(segs)
NODE_CLASS_MAPPINGS = {
"NH_SAM3Segment": NH_SAM3Segment,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"NH_SAM3Segment": "SAM3 Segment (NH)",
}
+213
View File
@@ -0,0 +1,213 @@
"""NH_SEGS nodes: pick instances, combine them into masks, preview them and hand them to Impact Pack.
All of them work on the instances SAM3 Segment (NH) finds; the logic lives in core/segs.py.
"""
from ...core.segs import (
SEGS_TYPE,
SORT_ORDERS,
count_segments,
expression_labels,
expression_masks,
instance_masks,
label_matches,
parse_expression,
parse_label_list,
preview_batch,
segs_union,
select_segments,
to_impact_segs,
)
CATEGORY = "NH-Nodes/Vision"
class NH_SegsSelect:
"""Keep the instances that match labels, score and size, in a chosen order."""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"segs": (SEGS_TYPE,),
"labels": ("STRING", {
"default": "",
"tooltip": "Labels to keep, separated by commas; * and ? are wildcards (\"*shirt\"). "
"Empty keeps every label.",
}),
"min_score": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.01,
"tooltip": "Drop instances with a lower confidence."}),
"min_area": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.0001,
"tooltip": "Drop instances smaller than this fraction of the image (0.01 = 1%)."}),
"max_area": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.0001,
"tooltip": "Drop instances larger than this fraction of the image."}),
"sort_by": (list(SORT_ORDERS), {
"default": "score",
"tooltip": "Order of the kept instances: highest score, largest area, by position (box centre), "
"nearest the image centre first, or as found.",
}),
"start": ("INT", {"default": 0, "min": 0, "max": 4096, "step": 1,
"tooltip": "Skip this many instances of the sorted list (0 starts at the first)."}),
"count": ("INT", {"default": 0, "min": 0, "max": 4096, "step": 1,
"tooltip": "Keep this many instances from start on; 0 keeps all of them."}),
}
}
RETURN_TYPES = (SEGS_TYPE, "MASK", "INT")
RETURN_NAMES = ("segs", "mask", "count")
FUNCTION = "select"
CATEGORY = CATEGORY
DESCRIPTION = (
"Filter and order instances, per image: by label (wildcards allowed), score and size, then sort and take a "
"slice. Example: the second person from the left = labels 'person', sort_by left_to_right, start 1, count 1."
)
SEARCH_ALIASES = ["segs select", "filter segs", "pick instance", "largest object", "leftmost person"]
def select(self, segs, labels, min_score, min_area, max_area, sort_by, start, count):
patterns = parse_label_list(labels)
picked = tuple(
select_segments(frame, patterns, min_score, min_area, max_area, sort_by, start, count) for frame in segs
)
return picked, segs_union(picked), count_segments(picked)
class NH_SegsCompose:
"""Build a mask from instance labels with + - & ~ and parentheses."""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"segs": (SEGS_TYPE,),
"expression": ("STRING", {
"default": "all", "multiline": True,
"tooltip": "Labels combined with | or + (union), - (minus), & (overlap), ~ (everything except) "
"and parentheses. all = every instance, image = whole frame, none = nothing. Quote "
"labels with spaces (\"red car\") or use _ (red_car); t-shirt needs no quotes. Put "
"spaces around a minus to subtract. Examples: person - hair · ~person · "
"(wall | ceiling) - window.",
}),
}
}
RETURN_TYPES = ("MASK",)
RETURN_NAMES = ("mask",)
FUNCTION = "compose"
CATEGORY = CATEGORY
DESCRIPTION = (
"Turn instances into one mask with a small formula over their labels: | or + joins, - removes, & keeps the "
"overlap, ~ inverts. ~person is the background around people; (building | roof) - window keeps the facade."
)
SEARCH_ALIASES = ["segs compose", "mask formula", "combine masks", "subtract mask", "background mask"]
def compose(self, segs, expression):
tree = parse_expression(expression)
labels = [seg.label for frame in segs for seg in frame.segments]
def found(pattern):
return any(label_matches(label, pattern) for label in labels)
unmatched = [pattern for pattern in expression_labels(tree) if not found(pattern)]
for pattern in unmatched:
parts = pattern.split("-")
if len(parts) > 1 and all(part and found(part) for part in parts):
# "person-hair" is one label; with person and hair both found, a subtraction was meant
raise ValueError(f"[NH-Nodes] Segs Compose: no instance is labelled '{pattern}', but "
f"{' and '.join(repr(part) for part in parts)} are. To subtract, put spaces around "
f"the minus: {' - '.join(parts)}")
if unmatched:
print(f"[NH-Nodes] Segs Compose: no instance is labelled {', '.join(repr(p) for p in unmatched)}; "
"those parts are empty")
return (expression_masks(tree, segs),)
class NH_SegsToMasks:
"""One full-size mask per instance, plus the union and the labels."""
@classmethod
def INPUT_TYPES(cls):
return {"required": {"segs": (SEGS_TYPE,)}}
RETURN_TYPES = ("MASK", "MASK", "STRING", "INT")
RETURN_NAMES = ("masks", "union", "labels", "count")
FUNCTION = "convert"
CATEGORY = CATEGORY
DESCRIPTION = (
"masks: one mask per instance, image after image (a single empty mask when there is none). union: one mask "
"per image. labels: the instance labels, one per line, in the order of masks."
)
SEARCH_ALIASES = ["segs to masks", "instance masks", "segs to mask batch"]
def convert(self, segs):
masks, labels = instance_masks(segs)
return masks, segs_union(segs), "\n".join(labels), len(labels)
class NH_SegsToImpactSEGS:
"""Hand the instances of one image to Impact Pack detailers as its SEGS type."""
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"segs": (SEGS_TYPE,),
"crop_factor": ("FLOAT", {
"default": 3.0, "min": 1.0, "max": 10.0, "step": 0.1,
"tooltip": "Size of the region a detailer crops around each instance, as a multiple of its box "
"(Impact Pack's crop_factor).",
}),
"batch_index": ("INT", {"default": 0, "min": 0, "max": 4095, "step": 1,
"tooltip": "Impact SEGS hold one image: which image of the batch to convert."}),
}
}
RETURN_TYPES = ("SEGS",)
RETURN_NAMES = ("segs",)
FUNCTION = "convert"
CATEGORY = CATEGORY
DESCRIPTION = (
"Convert the instances of one image to Impact Pack's SEGS, for its detailers and SEGS tools. Impact Pack is "
"not needed for anything else in NH-Nodes."
)
SEARCH_ALIASES = ["segs to impact", "impact segs", "detailer segs"]
def convert(self, segs, crop_factor, batch_index):
if batch_index >= len(segs):
raise ValueError(f"[NH-Nodes] Segs To Impact SEGS: batch_index {batch_index} but the segs hold "
f"{len(segs)} image(s)")
return (to_impact_segs(segs[batch_index], crop_factor),)
class NH_SegsPreview:
"""Draw the instances on their image."""
@classmethod
def INPUT_TYPES(cls):
return {"required": {"image": ("IMAGE",), "segs": (SEGS_TYPE,)}}
RETURN_TYPES = ("IMAGE",)
RETURN_NAMES = ("preview",)
FUNCTION = "preview"
CATEGORY = CATEGORY
DESCRIPTION = "Each instance tinted in its own colour, outlined and labelled with its label and score."
SEARCH_ALIASES = ["segs preview", "show segs", "draw instances"]
def preview(self, image, segs):
return (preview_batch(image, segs),)
NODE_CLASS_MAPPINGS = {
"NH_SegsSelect": NH_SegsSelect,
"NH_SegsCompose": NH_SegsCompose,
"NH_SegsToMasks": NH_SegsToMasks,
"NH_SegsToImpactSEGS": NH_SegsToImpactSEGS,
"NH_SegsPreview": NH_SegsPreview,
}
NODE_DISPLAY_NAME_MAPPINGS = {
"NH_SegsSelect": "Segs Select (NH)",
"NH_SegsCompose": "Segs Compose (NH)",
"NH_SegsToMasks": "Segs To Masks (NH)",
"NH_SegsToImpactSEGS": "Segs To Impact SEGS (NH)",
"NH_SegsPreview": "Segs Preview (NH)",
}
+2 -2
View File
@@ -1,7 +1,7 @@
[project]
name = "nh-nodes"
description = "ComfyUI custom nodes for image workflows, tiling, masks, smart resize, logic routing, prompt and text tools, batch automation, garment/portrait segmentation, and VTON preprocessing."
version = "1.6.2"
description = "ComfyUI custom nodes for image workflows, tiling, masks, smart resize, logic routing, prompt and text tools, batch automation, SAM 3 object segmentation with instance selection and mask formulas, garment/portrait segmentation, and VTON preprocessing."
version = "1.7.0"
license = { file = "LICENSE" }
dependencies = [
"numpy",
+5 -4
View File
@@ -13,9 +13,10 @@ for path in (COMFY_ROOT, os.path.dirname(PACK_DIR)):
pack = importlib.import_module("NH-Nodes")
EXPECTED_NODE_COUNT = 69
# Native libraries that only VTON needs; importing the pack must not load them.
LAZY_ONLY = ("onnxruntime", "skimage")
EXPECTED_NODE_COUNT = 75
# Heavy libraries only some nodes need (VTON: onnxruntime, skimage; SAM 3: sam3, whose import also switches TF32
# on process-wide); importing the pack must not load them.
LAZY_ONLY = ("onnxruntime", "skimage", "sam3")
def test_registry_is_complete_and_consistent():
@@ -24,7 +25,7 @@ def test_registry_is_complete_and_consistent():
assert pack.WEB_DIRECTORY == "./web"
def test_pack_import_does_not_load_vton_native_libraries():
def test_pack_import_does_not_load_heavy_optional_libraries():
# Fresh interpreter: other tests in this session may import these libraries legitimately.
code = (
"import importlib, sys\n"
+499
View File
@@ -0,0 +1,499 @@
"""SAM 3 backend: geometry helpers, the two-pass pipeline with a fake sam3, package/checkpoint checks.
Everything but the last test runs without sam3, a checkpoint or a GPU. The last one runs the real model and is
skipped unless all three are present.
"""
import contextlib
import importlib
import os
import sys
import types
import numpy as np
import pytest
import torch
backend = importlib.import_module("NH-Nodes.core.sam3_backend")
sam3_nodes = importlib.import_module("NH-Nodes.nodes.vision.sam3_nodes")
segs = importlib.import_module("NH-Nodes.core.segs")
# --- geometry ----------------------------------------------------------------
def test_sample_region_maps_pixel_centres_onto_the_model_view():
size, height, width = 16, 30, 60
u = torch.arange(size, dtype=torch.float32).repeat(size, 1) # value = model column index
out = backend.sample_region(u, (height, width), (0, 0, width, height), (height, width))
expected = ((torch.arange(width) + 0.5) * size / width - 0.5).clamp(0, size - 1)
assert torch.allclose(out[height // 2], expected, atol=1e-4)
# a sub-region samples the same values as the full grid
sub = backend.sample_region(u, (height, width), (10, 5, 40, 20), (15, 30))
assert torch.allclose(sub, out[5:20, 10:40], atol=1e-4)
def test_sample_region_chunks_give_the_same_result(monkeypatch):
logits = torch.randn(32, 32)
whole = backend.sample_region(logits, (90, 70), (3, 4, 66, 88), (84, 63))
monkeypatch.setattr(backend, "_SAMPLE_CHUNK", 100)
assert torch.equal(backend.sample_region(logits, (90, 70), (3, 4, 66, 88), (84, 63)), whole)
def test_model_box_to_image_adds_a_margin_inside_the_image():
assert backend.model_box_to_image((16, 16, 48, 32), 64, (300, 600)) == (140, 70, 460, 155)
assert backend.model_box_to_image((0, 0, 64, 64), 64, (300, 600)) == (0, 0, 600, 300)
@pytest.mark.parametrize("box", [(100, 100, 120, 400), (0, 0, 30, 30), (1900, 950, 2000, 1000), (10, 500, 1990, 520),
(0, 0, 2000, 1000)])
def test_crop_region_contains_the_box_with_margin_and_bounded_aspect(box):
x1, y1, x2, y2 = box
cx1, cy1, cx2, cy2 = backend.crop_region(box, (1000, 2000))
assert 0 <= cx1 <= x1 and 0 <= cy1 <= y1 and x2 <= cx2 <= 2000 and y2 <= cy2 <= 1000
width, height = cx2 - cx1, cy2 - cy1
if width < 2000 and height < 1000: # not clamped by the image
assert max(width, height) / min(width, height) <= 2.0 + 1e-6
assert cx1 < x1 or cx1 == 0
def test_to_logits_inverts_probabilities_and_passes_logits_through():
logits = torch.tensor([-5.0, -0.5, 0.0, 2.0, 6.0])
assert torch.allclose(backend.to_logits(torch.sigmoid(logits)), logits, atol=1e-3)
assert torch.equal(backend.to_logits(logits), logits)
def test_drop_stray_parts():
coarse = np.zeros((20, 20), bool)
coarse[5:15, 5:15] = True
refined = np.zeros((20, 20), bool)
refined[4:14, 4:14] = True
refined[18:20, 0:2] = True # speck far from the object
cleaned = backend.drop_stray_parts(refined, coarse)
assert cleaned[4:14, 4:14].all() and not cleaned[18:20, 0:2].any()
def _blob(shape, box):
mask = np.zeros(shape, bool)
x1, y1, x2, y2 = box
mask[y1:y2, x1:x2] = True
return mask
def test_close_up_agrees_keeps_sharp_thin_objects_and_rejects_other_objects():
coarse_pole = _blob((400, 300), (130, 20, 170, 380)) # pass 1: a 40 px wide blur around a pole
sharp_pole = _blob((400, 300), (145, 25, 155, 375)) # close-up: the 10 px pole, IoU only 0.24
assert backend.close_up_agrees(sharp_pole, coarse_pole, tolerance=16)
coarse_person = _blob((400, 300), (60, 40, 240, 390))
assert backend.close_up_agrees(_blob((400, 300), (55, 35, 245, 395)), coarse_person, tolerance=16)
assert not backend.close_up_agrees(_blob((400, 300), (0, 0, 40, 40)), coarse_person, 16) # something else
assert not backend.close_up_agrees(_blob((400, 300), (100, 40, 140, 80)), coarse_person, 16) # only a part
assert not backend.close_up_agrees(_blob((400, 300), (0, 0, 300, 400)), coarse_person, 16) # far too much
assert not backend.close_up_agrees(np.zeros((400, 300), bool), coarse_person, 16)
def test_close_up_agrees_on_large_masks_at_a_reduced_scale(monkeypatch):
monkeypatch.setattr(backend, "_AGREE_PIXELS", 1000)
coarse = _blob((400, 300), (130, 20, 170, 380))
assert backend.close_up_agrees(_blob((400, 300), (148, 25, 151, 375)), coarse, tolerance=16) # 3 px pole
assert backend.close_up_agrees(_blob((400, 300), (149, 25, 150, 375)), coarse, tolerance=16) # thinner than a step
# --- fake sam3 ---------------------------------------------------------------
SQUARE = (16, 16, 48, 32) # model-space box (x1, y1, x2, y2) of the fake "square" instance
def _prob_map(box, size=64, inside=0.99):
prob = torch.full((size, size), 0.01)
x1, y1, x2, y2 = box
prob[y1:y2, x1:x2] = inside
return prob
class FakeProcessor:
resolution = 64
def __init__(self, threshold=0.5):
self.threshold = threshold
self.calls = []
def set_image(self, image, state=None):
self.calls.append(("image", tuple(image.shape)))
return {"hw": tuple(image.shape[-2:])}
def reset_all_prompts(self, state):
self.calls.append(("reset",))
state.pop("prompt", None)
def set_text_prompt(self, prompt, state):
self.calls.append(("text", prompt))
state["prompt"] = prompt
if prompt == "square":
# best instance, an exact duplicate with a lower score, and a second distinct one
maps = [_prob_map(SQUARE), _prob_map(SQUARE), _prob_map((50, 40, 60, 60))]
scores = [0.9, 0.6, 0.7]
else:
maps, scores = [], []
masks = torch.stack(maps)[:, None] if maps else torch.zeros((0, 1, 64, 64))
return {"masks_logits": masks, "scores": torch.tensor(scores), "boxes": torch.zeros((len(scores), 4))}
class FakeModel(torch.nn.Module):
"""predict_inst answers with a mask filling the box prompt (or the opposite corner when ``stray``)."""
def __init__(self, stray=False):
super().__init__()
self.weight = torch.nn.Parameter(torch.zeros(1))
self.stray = stray
self.prompts = []
def predict_inst(self, state, box=None, mask_input=None, multimask_output=True, return_logits=False):
height, width = state["hw"]
self.prompts.append((tuple(np.round(box).astype(int)), tuple(mask_input.shape)))
logits = np.full((1, height, width), -10.0, np.float32)
if self.stray:
logits[0, :3, :3] = 10.0
else:
x1, y1, x2, y2 = np.round(box).astype(int)
logits[0, y1:y2, x1:x2] = 10.0
return logits, np.array([0.9]), np.zeros((1, 288, 288), np.float32)
def _run(detail, model=None, concepts=("square", "nothing"), **kwargs):
model = model or FakeModel()
processors = []
def make_processor(threshold):
processors.append(FakeProcessor(threshold))
return processors[-1]
images = torch.rand(2, 300, 600, 3)
frames = backend.segment_images(model, make_processor, images, list(concepts), 0.4, detail, device=torch.device("cpu"),
**kwargs)
return frames, processors, model
def test_find_instances_drops_duplicates_and_honours_the_cap():
processor = FakeProcessor()
state = processor.set_image(torch.zeros(3, 64, 64))
found = backend.find_instances(processor, state, "square")
assert [round(score, 2) for score, _, _ in found] == [0.9, 0.7]
assert found[0][2] == SQUARE
assert len(backend.find_instances(processor, state, "square", max_instances=1)) == 1
assert backend.find_instances(processor, state, "nothing") == []
assert processor.calls.count(("reset",)) == 3 # every prompt starts from a clean state
def test_fast_pass_maps_instances_to_full_resolution():
frames, processors, model = _run("fast")
assert len(frames) == 2 and (frames[0].height, frames[0].width) == (300, 600)
first = frames[0].segments[0]
assert first.label == "square" and first.score == pytest.approx(0.9)
# model box (16..48, 16..32) of a 64 px square view -> (150..450, 75..150) in the 600 x 300 image
assert all(abs(a - b) <= 2 for a, b in zip(first.box, (150, 75, 450, 150)))
assert len(frames[0].segments) == 2 and model.prompts == []
assert processors[0].threshold == 0.4 and processors[0].calls[0] == ("image", (3, 64, 64))
def test_balanced_pass_replaces_masks_with_the_close_up_result():
frames, processors, model = _run("balanced")
target = backend.model_box_to_image(SQUARE, 64, (300, 600))
first = frames[0].segments[0]
assert first.box == target # the fake close-up fills the box prompt exactly
assert len(model.prompts) == 4 # 2 instances x 2 images
assert all(shape == (1, 288, 288) for _, shape in model.prompts)
crops = [call[1] for call in processors[0].calls if call[0] == "image"][1:]
assert all(shape[0] == 3 and shape[1] < 300 for shape in crops) # close-ups are crops of the original
def test_balanced_pass_keeps_the_first_pass_mask_when_the_close_up_strays(capsys):
fast, _, _ = _run("fast")
kept, _, _ = _run("balanced", model=FakeModel(stray=True))
assert kept[0].segments[0].box == fast[0].segments[0].box
assert "kept the first-pass mask for 4 instance(s)" in capsys.readouterr().out
def test_interrupt_and_progress_hooks():
seen, steps = [], []
_run("balanced", interrupt=lambda: seen.append(1), progress=lambda done, total: steps.append((done, total)))
assert steps == [(1, 4), (2, 4), (3, 4), (4, 4)]
assert len(seen) == 4 + 4 # each concept of each image + each close-up
def stop():
raise RuntimeError("interrupted")
with pytest.raises(RuntimeError, match="interrupted"):
_run("fast", interrupt=stop)
def test_segment_offloads_unless_asked_to_keep(monkeypatch):
model = FakeModel()
moves = []
monkeypatch.setattr(backend, "load_model", lambda checkpoint: model)
monkeypatch.setattr(backend, "ensure_sam3", lambda: (None, lambda m, device, confidence_threshold: FakeProcessor()))
monkeypatch.setattr(backend, "sam3_device", lambda: torch.device("cpu"))
monkeypatch.setattr(backend, "move_model", lambda m, device: moves.append("to"))
monkeypatch.setattr(backend, "offload", lambda m: moves.append("off"))
frames = backend.segment(torch.rand(1, 64, 64, 3), ["square"], keep_model_loaded=False)
assert len(frames) == 1 and moves == ["to", "off"]
moves.clear()
backend.segment(torch.rand(1, 64, 64, 3), ["square"], keep_model_loaded=True)
assert moves == ["to"]
# --- package, device and checkpoint checks -----------------------------------
def _install_fake_sam3(monkeypatch, build=None, processor=None, fail=None, flip_tf32=False):
monkeypatch.setattr(backend, "_API", None)
monkeypatch.setattr(backend.importlib.util, "find_spec", lambda name, *a: object() if name == "sam3" else None)
def import_module(name):
if fail:
raise ImportError(fail)
if name == "sam3.model_builder":
if flip_tf32: # like sam3's _setup_tf32() at import
torch.backends.cuda.matmul.allow_tf32 = not torch.backends.cuda.matmul.allow_tf32
torch.backends.cudnn.allow_tf32 = not torch.backends.cudnn.allow_tf32
return types.SimpleNamespace(build_sam3_image_model=build)
return types.SimpleNamespace(Sam3Processor=processor)
monkeypatch.setattr(backend.importlib, "import_module", import_module)
def _good_build(checkpoint_path=None, load_from_HF=True, enable_inst_interactivity=False, device="cuda",
eval_mode=True):
return None
class _GoodProcessor:
def __init__(self, model, resolution=1008, device="cuda", confidence_threshold=0.5):
pass
def set_image(self):
pass
def set_text_prompt(self):
pass
def reset_all_prompts(self):
pass
def test_missing_sam3_explains_how_to_install(monkeypatch):
monkeypatch.setattr(backend, "_API", None)
monkeypatch.setattr(backend.importlib.util, "find_spec", lambda name, *a: None)
with pytest.raises(RuntimeError, match="pip install --no-deps git\\+https://github.com/facebookresearch/sam3"):
backend.ensure_sam3()
def test_broken_sam3_import_is_reported(monkeypatch):
_install_fake_sam3(monkeypatch, fail="No module named 'pkg_resources'")
with pytest.raises(RuntimeError, match="failed to load: No module named 'pkg_resources'"):
backend.ensure_sam3()
def test_sam3_without_the_needed_api_is_reported(monkeypatch):
_install_fake_sam3(monkeypatch, build=lambda checkpoint_path=None: None, processor=types.SimpleNamespace())
with pytest.raises(RuntimeError, match="load_from_HF.*eval_mode.*Sam3Processor.set_image"):
backend.ensure_sam3()
class OldProcessor(_GoodProcessor):
def __init__(self, model):
pass
_install_fake_sam3(monkeypatch, build=_good_build, processor=OldProcessor)
with pytest.raises(RuntimeError, match=r"Sam3Processor\(device=\), Sam3Processor\(confidence_threshold=\)"):
backend.ensure_sam3()
def test_importing_sam3_does_not_change_tf32(monkeypatch):
before = (torch.backends.cuda.matmul.allow_tf32, torch.backends.cudnn.allow_tf32)
_install_fake_sam3(monkeypatch, build=_good_build, processor=_GoodProcessor, flip_tf32=True)
assert backend.ensure_sam3() == (_good_build, _GoodProcessor)
assert (torch.backends.cuda.matmul.allow_tf32, torch.backends.cudnn.allow_tf32) == before
def test_cpu_only_comfyui_gets_a_clear_error(monkeypatch):
monkeypatch.setattr(backend, "_mm", types.SimpleNamespace(get_torch_device=lambda: torch.device("cpu")))
with pytest.raises(RuntimeError, match="needs an NVIDIA GPU"):
backend.sam3_device()
def test_missing_checkpoint_says_where_to_get_it(monkeypatch, tmp_path):
monkeypatch.setattr(backend, "folder_paths", None)
monkeypatch.setattr(backend, "checkpoint_folders", lambda: [str(tmp_path)])
with pytest.raises(RuntimeError, match="huggingface.co/facebook/sam3"):
backend.resolve_checkpoint("sam3.pt")
(tmp_path / "sam3.pt").write_bytes(b"x")
assert backend.resolve_checkpoint("sam3.pt") == os.path.join(str(tmp_path), "sam3.pt")
with pytest.raises(RuntimeError, match="not a SAM 3 .pt checkpoint"):
backend.resolve_checkpoint("model.safetensors")
with pytest.raises(RuntimeError, match="must be a file"):
backend.resolve_checkpoint("../secret.pt")
def test_checkpoint_list_always_has_an_entry(monkeypatch, tmp_path):
monkeypatch.setattr(backend, "folder_paths", None)
monkeypatch.setattr(backend, "checkpoint_folders", lambda: [str(tmp_path)])
assert backend.checkpoint_choices() == ["sam3.pt"]
(tmp_path / "sam3.1.pt").write_bytes(b"x")
(tmp_path / "config.json").write_text("{}")
assert backend.checkpoint_choices() == ["sam3.1.pt"]
# --- node ----------------------------------------------------------------------
def test_segment_node_outputs(monkeypatch):
frame = segs.SegsFrame(8, 10, (segs.make_segment("cat", 0.8, torch.ones((2, 3), dtype=torch.bool), (1, 1)),))
calls = []
def fake_segment(images, concepts, *args, **kwargs):
calls.append(concepts)
return (frame,)
monkeypatch.setattr(sam3_nodes.sam3, "segment", fake_segment)
node = sam3_nodes.NH_SAM3Segment()
result, mask, count = node.segment(torch.rand(1, 8, 10, 3), "cat, dog\ncat", 0.5, "fast", 0, "sam3.pt", False)
assert calls == [["cat", "dog"]] and result == (frame,) and count == 1
assert mask.shape == (1, 8, 10) and int(mask.sum()) == 6
assert sam3_nodes.NH_SAM3Segment.RETURN_NAMES == ("segs", "mask", "count")
with pytest.raises(ValueError, match="at least one concept"):
node.segment(torch.rand(1, 8, 10, 3), " ,\n# none", 0.5, "fast", 0, "sam3.pt", False)
assert sam3_nodes.NH_SAM3Segment.INPUT_TYPES()["required"]["checkpoint"][0]
# --- real model ----------------------------------------------------------------
def _real_sam3_ready():
if not torch.cuda.is_available() or importlib.util.find_spec("sam3") is None:
return False
try:
backend.resolve_checkpoint(backend.DEFAULT_CHECKPOINT)
except RuntimeError:
return False
return True
@pytest.mark.skipif(not _real_sam3_ready(), reason="needs CUDA, the sam3 package and models/sam3/sam3.pt")
def test_real_sam3_finds_the_astronaut_and_releases_vram():
from skimage import data
image = torch.from_numpy(data.astronaut().astype(np.float32) / 255.0)[None]
frames = backend.segment(image, ["person"], 0.5, "balanced", 0, backend.DEFAULT_CHECKPOINT, False)
people = frames[0].segments
assert len(people) >= 1 and people[0].label == "person" and people[0].score > 0.5
x1, y1, x2, y2 = people[0].box
assert 0 <= x1 < x2 <= 512 and 0 <= y1 < y2 <= 512 and people[0].area > 0.1 * 512 * 512
assert next(backend.load_model(backend.DEFAULT_CHECKPOINT).parameters()).device.type == "cpu"
assert "sam3" in sys.modules
def test_unreadable_tf32_switches_are_left_alone(monkeypatch):
class Refusing:
@property
def allow_tf32(self):
raise RuntimeError("mixing legacy and new precision APIs")
monkeypatch.setattr(torch.backends.cuda, "matmul", Refusing())
assert backend._tf32_flags() is None
backend._restore_tf32(None) # nothing to put back, no error
def test_checkpoint_check_rejects_foreign_and_broken_files(tmp_path):
good = tmp_path / "sam3.pt"
torch.save({"model": {"detector.backbone.w": torch.zeros(1), "tracker.decoder.w": torch.zeros(1)}}, good)
backend.check_checkpoint(str(good))
other = tmp_path / "other.pt"
torch.save({"encoder.w": torch.zeros(1)}, other)
with pytest.raises(RuntimeError, match="not a SAM 3 checkpoint"):
backend.check_checkpoint(str(other))
broken = tmp_path / "broken.pt"
broken.write_bytes(good.read_bytes()[:100])
with pytest.raises(RuntimeError, match="(?s)incomplete download.*huggingface.co/facebook/sam3"):
backend.check_checkpoint(str(broken))
def test_load_model_builds_once_and_keeps_real_holes(monkeypatch, tmp_path):
path = tmp_path / "sam3.pt"
path.write_bytes(b"x")
builds = []
def build(**kwargs):
builds.append(kwargs)
model = FakeModel()
model.inst_interactive_predictor = types.SimpleNamespace(_transforms=types.SimpleNamespace(max_hole_area=256.0))
return model
monkeypatch.setattr(backend, "_CACHE", {})
monkeypatch.setattr(backend, "resolve_checkpoint", lambda name: str(path))
monkeypatch.setattr(backend, "ensure_sam3", lambda: (build, None))
monkeypatch.setattr(backend, "sam3_device", lambda: torch.device("cpu"))
monkeypatch.setattr(backend, "check_checkpoint", lambda p: None)
monkeypatch.setattr(backend.torch.cuda, "device", lambda device: contextlib.nullcontext())
model = backend.load_model("sam3.pt")
assert backend.load_model("sam3.pt") is model and len(builds) == 1
assert builds[0]["enable_inst_interactivity"] is True and builds[0]["load_from_HF"] is False
assert builds[0]["device"] == "cpu" and builds[0]["checkpoint_path"] == str(path)
assert model.inst_interactive_predictor._transforms.max_hole_area == 0.0
assert not any(p.requires_grad for p in model.parameters())
def test_failed_gpu_move_puts_the_model_back(monkeypatch):
class Unmovable(FakeModel):
def to(self, device, *args, **kwargs):
if torch.device(device).type != "cpu":
raise RuntimeError("CUDA out of memory")
return super().to(device, *args, **kwargs)
offloaded = []
monkeypatch.setattr(backend, "make_room_on_gpu", lambda nbytes: None)
monkeypatch.setattr(backend, "offload", lambda model: offloaded.append(model))
model = Unmovable()
with pytest.raises(RuntimeError, match="out of memory"):
backend.move_model(model, torch.device("cuda", 0))
assert offloaded == [model]
def test_segment_offloads_when_the_run_fails(monkeypatch):
model = FakeModel()
offloaded = []
monkeypatch.setattr(backend, "load_model", lambda checkpoint: model)
monkeypatch.setattr(backend, "ensure_sam3", lambda: (None, FakeProcessor))
monkeypatch.setattr(backend, "sam3_device", lambda: torch.device("cpu"))
monkeypatch.setattr(backend, "move_model", lambda m, device: None)
monkeypatch.setattr(backend, "offload", lambda m: offloaded.append(m))
def broken(*args, **kwargs):
raise RuntimeError("boom")
monkeypatch.setattr(backend, "segment_images", broken)
with pytest.raises(RuntimeError, match="boom"):
backend.segment(torch.rand(1, 8, 8, 3), ["x"])
assert offloaded == [model]
with pytest.raises(ValueError, match="empty image batch"):
backend.segment(torch.rand(0, 8, 8, 3), ["x"])
def test_offload_clears_position_caches_and_moves_rotary_tables():
PositionEmbeddingSine = type("PositionEmbeddingSine", (torch.nn.Module,), {})
model = FakeModel()
model.pos = PositionEmbeddingSine()
model.pos.cache = {(72, 72): torch.zeros(4)}
model.rope = torch.nn.Module()
model.rope.freqs_cis = torch.zeros(3)
backend._move_tables(model, torch.device("meta"))
assert model.rope.freqs_cis.device.type == "meta"
model.rope.freqs_cis = torch.zeros(3) # meta tensors hold no data to bring back
backend.offload(model)
assert model.pos.cache == {} and model.rope.freqs_cis.device.type == "cpu"
def test_wide_pictures_get_a_note(capsys):
model = FakeModel()
backend.segment_images(model, lambda threshold: FakeProcessor(threshold), torch.rand(1, 100, 600, 3), ["nothing"],
detail_level="fast", device=torch.device("cpu"))
assert "wider than 4:1" in capsys.readouterr().out
def test_one_and_four_channel_images_become_rgb():
gray = torch.rand(6, 8, 1)
assert torch.equal(backend._image_chw(gray), gray.permute(2, 0, 1).expand(3, -1, -1))
rgba = torch.rand(6, 8, 4)
assert torch.equal(backend._image_chw(rgba), rgba[..., :3].permute(2, 0, 1))
with pytest.raises(ValueError, match="1, 3 or 4 channels"):
backend._image_chw(torch.rand(6, 8, 2))
+278
View File
@@ -0,0 +1,278 @@
"""NH_SEGS data model and the select / compose / convert / preview helpers (no model needed)."""
import importlib
import re
import numpy as np
import pytest
import torch
segs = importlib.import_module("NH-Nodes.core.segs")
segs_nodes = importlib.import_module("NH-Nodes.nodes.vision.segs_nodes")
def _seg(label, score, box, height=40, width=60):
"""Segment filling ``box`` (x1, y1, x2, y2) of a height x width image."""
mask = torch.zeros((height, width), dtype=torch.bool)
x1, y1, x2, y2 = box
mask[y1:y2, x1:x2] = True
return segs.make_segment(label, score, mask)
def _frame(*segments, height=40, width=60):
return segs.SegsFrame(height, width, tuple(segments))
# --- data model --------------------------------------------------------------
def test_make_segment_keeps_a_compact_tight_crop_with_origin_offset():
mask = torch.zeros((10, 12), dtype=torch.bool)
mask[2:5, 3:7] = True
seg = segs.make_segment("car", 0.9, mask, origin=(100, 50))
assert seg.box == (103, 52, 107, 55)
assert tuple(seg.mask.shape) == (3, 4) and seg.mask.all()
assert seg.area == 12 and seg.center == (105.0, 53.5)
# a compact copy, not a view keeping the big mask alive
assert seg.mask.untyped_storage().nbytes() == seg.mask.numel()
def test_make_segment_thresholds_float_masks_and_returns_none_when_empty():
assert segs.make_segment("x", 1.0, torch.zeros((4, 4))) is None
soft = torch.tensor([[0.2, 0.6], [0.5, 0.9]])
seg = segs.make_segment("x", 1.0, soft)
assert seg.box == (1, 0, 2, 2) and seg.mask.tolist() == [[True], [True]] # values above 0.5, tight box
assert segs.make_segment("x", 1.0, np.array([[0, 1]], dtype=bool)).box == (1, 0, 2, 1)
def test_union_and_instance_masks():
frame = _frame(_seg("a", 0.9, (0, 0, 10, 10)), _seg("b", 0.8, (5, 5, 20, 15)))
union = segs.segs_union((frame, frame))
assert union.shape == (2, 40, 60) and union.dtype == torch.float32
assert int(union[0].sum()) == 100 + 150 - 25
masks, labels = segs.instance_masks((frame,))
assert masks.shape == (2, 40, 60) and labels == ["a", "b"]
empty, none = segs.instance_masks((_frame(),))
assert empty.shape == (1, 40, 60) and float(empty.sum()) == 0 and none == []
def test_parse_label_list_splits_trims_dedupes_and_skips_comments():
text = "person, Red car\n# a comment\n\nwindow,person ,red_car"
assert segs.parse_label_list(text) == ["person", "Red car", "window"]
assert segs.parse_label_list("") == [] and segs.parse_label_list(None) == []
def test_label_matching_ignores_case_spacing_and_supports_wildcards():
assert segs.label_matches("Red Car", "red_car")
assert segs.label_matches("t-shirt", "*shirt")
assert segs.label_matches("person", "pers?n")
assert not segs.label_matches("person", "persons")
# --- select ------------------------------------------------------------------
def test_select_filters_by_label_score_and_area():
frame = _frame(_seg("person", 0.9, (0, 0, 10, 10)), _seg("person", 0.4, (20, 0, 30, 10)),
_seg("car", 0.95, (0, 20, 60, 40)))
picked = segs.select_segments(frame, ["person"], min_score=0.5)
assert [s.score for s in picked.segments] == [0.9]
big = segs.select_segments(frame, min_area=0.2) # car = 1200 / 2400 px
assert [s.label for s in big.segments] == ["car"]
small = segs.select_segments(frame, max_area=0.1)
assert sorted(s.score for s in small.segments) == [0.4, 0.9]
@pytest.mark.parametrize("order, expected", [
("score", ["c", "a", "b"]), ("area", ["b", "a", "c"]), ("left_to_right", ["a", "c", "b"]),
("right_to_left", ["b", "c", "a"]), ("top_to_bottom", ["c", "a", "b"]), ("bottom_to_top", ["b", "a", "c"]),
("center_first", ["c", "b", "a"]), ("as_found", ["a", "b", "c"]),
])
def test_sort_orders(order, expected):
frame = _frame(_seg("a", 0.5, (0, 10, 6, 16)), _seg("b", 0.2, (40, 20, 60, 40)), _seg("c", 0.9, (28, 0, 32, 4)))
assert [s.label for s in segs.select_segments(frame, order=order).segments] == expected
def test_select_start_and_count_slice_after_sorting():
frame = _frame(*[_seg(f"p{i}", 0.5, (i * 10, 0, i * 10 + 5, 5)) for i in range(5)])
second = segs.select_segments(frame, order="left_to_right", start=1, count=1)
assert [s.label for s in second.segments] == ["p1"]
rest = segs.select_segments(frame, order="right_to_left", start=3, count=0)
assert [s.label for s in rest.segments] == ["p1", "p0"]
assert segs.select_segments(frame, start=9).segments == ()
# --- compose -----------------------------------------------------------------
def _compose_frame():
return _frame(_seg("person", 0.9, (0, 0, 30, 40)), _seg("hair", 0.8, (0, 0, 30, 10)),
_seg("red car", 0.7, (40, 30, 60, 40)), _seg("t-shirt", 0.6, (0, 10, 30, 25)))
def _px(expression, frame=None):
return int(segs.evaluate(segs.parse_expression(expression), frame or _compose_frame()).sum())
def test_compose_operators_and_keywords():
assert _px("person") == 1200
assert _px("person - hair") == 900
assert _px("person | 'red car'") == _px("person + \"red car\"") == _px("person or 'red car'") == 1400
assert _px("person & hair") == _px("person and hair") == 300
assert _px("~person") == _px("!person") == _px("not person") == 2400 - 1200
assert _px("all") == 1400 and _px("image") == 2400 and _px("none") == 0
assert _px("'t-shirt'") == 450 and _px("*shirt") == 450 and _px("red_car") == 200
assert _px("unknown") == 0
def test_compose_hyphens_and_apostrophes_inside_words_belong_to_the_label():
assert _px("t-shirt") == 450 # no quotes needed
assert _px("person - hair") == _px("person -hair") == _px("person- hair") == 900
assert _px("person-hair") == 0 # one (unknown) label: subtracting needs spaces
frame = _frame(_seg("women's dress", 0.9, (0, 0, 10, 10)))
assert _px("\"women's dress\"", frame) == 100 and _px("women's_dress", frame) == 100
def test_compose_precedence_and_associativity():
# & binds tighter than | and -, which read left to right
assert _px("hair | person & 'red car'") == _px("hair | (person & 'red car')") == 300
assert _px("person - hair - 't-shirt'") == 1200 - 300 - 450
assert _px("person - (hair - 't-shirt')") == 900
assert _px("~(person | 'red car')") == 1000
@pytest.mark.parametrize("expression, message", [
("", "empty"), (" ", "empty"), ("'red car", "never closed"), ("(person | hair", "not closed"),
("person |", "ends where"), ("person hair", "unexpected 'hair' at position 8"), (")", "unexpected ')'"),
("person !", "unexpected '!' at position 8"), ("(person or)", "unexpected ')' at position 11"),
])
def test_compose_errors_explain_where(expression, message):
with pytest.raises(segs.ExpressionError, match=re.escape(message)):
segs.parse_expression(expression)
def test_compose_node_reports_unmatched_labels(capsys):
(mask,) = segs_nodes.NH_SegsCompose().compose((_compose_frame(),), "person - ghost")
assert mask.shape == (1, 40, 60) and int(mask.sum()) == 1200
assert "'ghost'" in capsys.readouterr().out
def test_compose_node_catches_a_minus_without_spaces():
with pytest.raises(ValueError, match="person - hair"):
segs_nodes.NH_SegsCompose().compose((_compose_frame(),), "person-hair")
# a real hyphenated label, or parts that are not both labels, stay plain labels
assert int(segs_nodes.NH_SegsCompose().compose((_compose_frame(),), "t-shirt")[0].sum()) == 450
assert int(segs_nodes.NH_SegsCompose().compose((_compose_frame(),), "person-ghost")[0].sum()) == 0
# --- Impact Pack SEGS --------------------------------------------------------
@pytest.mark.parametrize("box, factor, expected", [
((20, 10, 30, 20), 3.0, [10, 0, 40, 30]), # 30 x 30 centred on the box
((0, 0, 10, 10), 3.0, [0, 0, 30, 30]), # slid right/down to stay inside
((50, 30, 60, 40), 2.0, [40, 20, 60, 40]), # slid left/up
((5, 5, 55, 35), 3.0, [0, 0, 60, 40]), # larger than the image: the whole image
((11, 13, 12, 14), 1.0, [11, 13, 12, 14]), # factor 1: the box itself
])
def test_detail_crop_region(box, factor, expected):
assert segs.detail_crop_region(60, 40, box, factor) == expected
def test_detail_crop_region_always_contains_the_box_and_stays_inside():
rng = np.random.default_rng(0)
for _ in range(2000):
width, height = int(rng.integers(1, 300)), int(rng.integers(1, 300))
x1, y1 = int(rng.integers(0, width)), int(rng.integers(0, height))
x2, y2 = int(rng.integers(x1 + 1, width + 1)), int(rng.integers(y1 + 1, height + 1))
factor = float(rng.uniform(1.0, 6.0))
rx1, ry1, rx2, ry2 = segs.detail_crop_region(width, height, (x1, y1, x2, y2), factor)
assert 0 <= rx1 <= x1 and 0 <= ry1 <= y1 and x2 <= rx2 <= width and y2 <= ry2 <= height
def test_to_impact_segs_places_each_mask_in_its_crop_region():
frame = _frame(_seg("person", 0.9, (20, 10, 30, 20)))
(height, width), items = segs.to_impact_segs(frame, 3.0)
assert (height, width) == (40, 60) and len(items) == 1
item = items[0]
assert item._fields == ("cropped_image", "cropped_mask", "confidence", "crop_region", "bbox", "label",
"control_net_wrapper")
rx1, ry1, rx2, ry2 = item.crop_region
assert item.cropped_mask.dtype == np.float32 and item.cropped_mask.shape == (ry2 - ry1, rx2 - rx1)
full = np.zeros((40, 60), np.float32)
full[ry1:ry2, rx1:rx2] = item.cropped_mask
assert full[10:20, 20:30].all() and full.sum() == 100
assert item.bbox == [20, 10, 30, 20] and item.label == "person" and item.confidence == pytest.approx(0.9)
assert item.cropped_image is None
def test_preview_labels_stack_instead_of_hiding_each_other():
placed = [(30, 10, 100, 30)]
assert segs._free_spot(40, 10, 70, 20, placed, 200) == 31 # moved below the label already there
assert segs._free_spot(40, 185, 70, 20, [(30, 180, 100, 200)], 200) == 159 # near the bottom: moved above
assert segs._free_spot(200, 10, 70, 20, placed, 200) == 10 # no overlap: stays
frame = _frame(_seg("person", 0.9, (10, 10, 50, 40)), _seg("hair", 0.8, (10, 10, 50, 20)), height=200, width=200)
assert segs.render_preview(np.full((200, 200, 3), 128, np.uint8), frame).shape == (200, 200, 3)
def test_impact_node_checks_batch_index():
node = segs_nodes.NH_SegsToImpactSEGS()
frame = _frame(_seg("a", 0.9, (0, 0, 5, 5)))
assert node.convert((frame,), 3.0, 0)[0][0] == (40, 60)
with pytest.raises(ValueError, match="batch_index 1"):
node.convert((frame,), 3.0, 1)
# --- preview and nodes -------------------------------------------------------
def test_render_preview_tints_only_instances_and_handles_any_label_text():
image = np.full((40, 60, 3), 128, np.uint8)
frame = _frame(_seg("áo dài", 0.91, (10, 10, 30, 30)))
out = segs.render_preview(image, frame)
assert out.shape == image.shape and out.dtype == np.uint8
assert (out[20, 20] != 128).any() # inside: tinted
assert (out[38, 58] == 128).all() # far outside: untouched
assert (image == 128).all() # the input is not modified
assert (segs.render_preview(image, _frame()) == image).all()
def test_preview_batch_rejects_segs_from_another_image():
frame = _frame(_seg("a", 0.9, (0, 0, 5, 5)))
assert segs.preview_batch(torch.rand(1, 40, 60, 3), (frame,)).shape == (1, 40, 60, 3)
assert segs.preview_batch(torch.rand(1, 40, 60, 1), (frame,)).shape == (1, 40, 60, 3)
with pytest.raises(ValueError, match="batch"):
segs.preview_batch(torch.rand(2, 40, 60, 3), (frame,))
with pytest.raises(ValueError, match="60x40"):
segs.preview_batch(torch.rand(1, 20, 30, 3), (frame,))
def test_select_and_to_masks_nodes():
frame = _frame(_seg("person", 0.9, (0, 0, 10, 10)), _seg("person", 0.8, (40, 0, 50, 10)),
_seg("dog", 0.7, (20, 20, 30, 30)))
picked, mask, count = segs_nodes.NH_SegsSelect().select((frame,), "person", 0.0, 0.0, 1.0, "right_to_left", 0, 1)
assert count == 1 and picked[0].segments[0].box == (40, 0, 50, 10)
assert mask.shape == (1, 40, 60) and int(mask.sum()) == 100
masks, union, labels, n = segs_nodes.NH_SegsToMasks().convert((frame,))
assert masks.shape == (3, 40, 60) and union.shape == (1, 40, 60)
assert labels == "person\nperson\ndog" and n == 3
def test_to_masks_refuses_when_ram_is_short(monkeypatch):
import psutil
frame = _frame(*[_seg("window", 0.9, (i, 0, i + 2, 2)) for i in range(0, 20, 4)])
monkeypatch.setattr(psutil, "virtual_memory", lambda: type("Memory", (), {"available": 4 * 40 * 60 * 4})())
with pytest.raises(MemoryError, match="Segs Select"):
segs_nodes.NH_SegsToMasks().convert((frame,))
monkeypatch.setattr(psutil, "virtual_memory", lambda: type("Memory", (), {"available": 1 << 30})())
assert segs_nodes.NH_SegsToMasks().convert((frame,))[0].shape == (5, 40, 60)
def test_frames_tell_comfyui_cache_what_they_hold():
frame = _frame(_seg("a", 0.9, (0, 0, 10, 10)), _seg("b", 0.8, (5, 5, 20, 15)))
tensors = frame._comfy_cache_tensors()
assert len(tensors) == 2 and all(t is s.mask for t, s in zip(tensors, frame.segments))
def test_segs_preview_node_and_every_node_declares_matching_outputs():
frame = _frame(_seg("a", 0.9, (0, 0, 5, 5)))
(preview,) = segs_nodes.NH_SegsPreview().preview(torch.rand(1, 40, 60, 3), (frame,))
assert preview.shape == (1, 40, 60, 3) and preview.dtype == torch.float32
sam3_nodes = importlib.import_module("NH-Nodes.nodes.vision.sam3_nodes")
classes = [*segs_nodes.NODE_CLASS_MAPPINGS.values(), *sam3_nodes.NODE_CLASS_MAPPINGS.values()]
for cls in classes:
required = cls.INPUT_TYPES()["required"]
assert len(cls.RETURN_TYPES) == len(cls.RETURN_NAMES) and cls.CATEGORY == "NH-Nodes/Vision"
assert callable(getattr(cls(), cls.FUNCTION)) and required
assert all(isinstance(spec, tuple) and spec for spec in required.values())