70 Commits
Author SHA1 Message Date
Salvador E. Tropea 1a9e3092b3 [Added] Experimental nodes that process one mask/metric at a time
They aren't usable. ComfyUI doesn't move execution to the next node
until all inputs are processed. So we end loading all images to memory.
2025-11-28 08:57:29 -03:00
Salvador E. Tropea 726cbdba51 [Added] Helper to load one mask 2025-11-28 08:56:57 -03:00
Salvador E. Tropea 70e753333f [Added][ConsolidateMetrics] Resume consolidation 2025-11-24 19:26:20 -03:00
Salvador E. Tropea a33906ede8 [Added] Example that computes all the SOD metrics in one step 2025-11-23 20:01:48 -03:00
Salvador E. Tropea b24e9687c0 [Added] Temporal examples for the dataset comparisson 2025-11-23 19:22:21 -03:00
Salvador E. Tropea d5ecc8b32d [CartesianProduct][Added] To create all dataset/method combinations 2025-11-23 19:19:53 -03:00
Salvador E. Tropea 23831bc8af [PlotMetricCurvesPIL] Support plotting more than one dataset
So we can plot multiple images
2025-11-23 19:18:10 -03:00
Salvador E. Tropea 45cb1339ce [ConsolidateMetrics] Support consolidating to more than one target
So we can run tests in parallel
2025-11-23 19:16:51 -03:00
Salvador E. Tropea d02c46e15c [pil_to_tensor] Unified
- Removed the one from PlotMetricCurvesPIL
- Created one that returns a 4D tensor
2025-11-20 13:21:34 -03:00
Salvador E. Tropea aa545d16f9 [PlotMetricCurvesPIL] Multiple curves and more
- Multiple traces: colors and legend_labels
- auto_scale option
- Better grid division
- Margin, font, trace size auto-adjust
2025-11-20 13:15:05 -03:00
Salvador E. Tropea 31debcce47 [ImageResize][Fixed] Call to upscale_comfy
Partially migrated
2025-11-20 13:09:38 -03:00
Salvador E. Tropea 8cc5abf0fd [ImageResize][Added] Comment about the original code
Was in the docstring, but now this is the description
2025-11-20 07:45:59 -03:00
Salvador E. Tropea 2393f347d4 [PlotMetricCurvesPIL] Removed unused test code 2025-11-20 07:45:19 -03:00
Salvador E. Tropea f3a1105d8d [ConsolidateMetrics] Return the data from disk when blocked 2025-11-20 07:44:05 -03:00
Salvador E. Tropea 4b5e5ff766 [Added] Option to make the block message optional
So we can choose to continue silently blocking
2025-11-20 07:41:45 -03:00
Salvador E. Tropea 636a5e8bbd Moved the DESCRIPTIONs to docstrings
Now SeCoNoHe can get them from docstrings
2025-11-20 07:39:01 -03:00
Salvador E. Tropea eb69bacdea [Image loader] Better log when loading masks 2025-11-19 10:24:50 -03:00
Salvador E. Tropea 8b1b6431ef [Fixed] upscale/upscale_comfy 2025-11-19 10:23:31 -03:00
Salvador E. Tropea 6bc0a90de4 Code clean-up
- Moved some conditional ComfyUI imports to SeCoNoHe
- Now using cleaner IO.TYPE and ABC (Abstract Base Class)
2025-11-17 09:22:21 -03:00
Salvador E. Tropea 3476ef8d14 [Removed] InputDir/OutputDir, contributed to basic_data_handling nodes 2025-11-17 08:31:17 -03:00
Salvador E. Tropea 202fe0d7ab [Added] Test examples for the Dataset stuff 2025-11-14 13:48:35 -03:00
Salvador E. Tropea 3053c89b25 [PlotMetricCurvesPIL][Added] Precision vs Recall and F vs threshold plots
- Also modified to store F, Fr, Fp and E tensors as tensors, not individual
  values, at least in memory. They become individual values on disk.
- Also added output with consolidated values, so we can plot them
2025-11-14 13:44:26 -03:00
Salvador E. Tropea 4cb9bf3d81 [SaliencyEvaluationMetrics] More detailed parameters
- Including the consolidation of F/E using all the curve
2025-11-12 11:23:50 -03:00
Salvador E. Tropea d4ba71ae5a [SaliencyEvaluationMetrics][Added] Normalization option
Disable by default
2025-11-12 11:22:15 -03:00
Salvador E. Tropea 5c2230b078 [ImageDataset] Better stop mechanism using ExecutionBlocker 2025-11-12 11:07:30 -03:00
Salvador E. Tropea f59f7b7d9f Trying to have some tolerance to None as argument 2025-11-12 07:28:31 -03:00
Salvador E. Tropea a34689b27c [Added] Nodes to get the ComfyUI input/output path
Might be contributed to ComfyUI-basic_data_handling
2025-11-11 09:56:15 -03:00
Salvador E. Tropea 4ab32569c6 [Saliency Evaluation Metrics][Fixed] Should be output
So we can use it to see the results
2025-11-11 07:35:14 -03:00
Salvador E. Tropea 83264cccab [Added][Load Image Mask] To directly load masks
- Also added the used file name as output
2025-11-10 13:59:41 -03:00
Salvador E. Tropea 64eafc6402 [Added] First draft for the consolidation node 2025-11-09 17:12:40 -03:00
Salvador E. Tropea 295681fbcb [Metrics][Added] Support for lists and save individual results
- We can process a list of batches
- We can now save the metrics for each image
2025-11-09 16:20:24 -03:00
Salvador E. Tropea 650646a5f2 [Requirements][Added] SciPy
- Used for S-measure
- ComfyUI dependency
2025-11-09 16:18:47 -03:00
Salvador E. Tropea a57fe0cba3 [SaliencyEvaluationMetrics] Better computations
- All moved outside the main source
- Documented
- Most replaced by proven versions
2025-11-07 13:45:29 -03:00
Salvador E. Tropea a2ca2fe876 [SaliencyEvaluationMetrics][Added] Conditional computation 2025-11-06 20:43:06 -03:00
Salvador E. Tropea 4d03617fc6 [SaliencyEvaluationMetrics][Removed] Redundant operations 2025-11-06 14:06:28 -03:00
Salvador E. Tropea 50cd6c67e9 [Added][WIP] SaliencyEvaluationMetrics 2025-11-06 13:58:37 -03:00
Salvador E. Tropea 31577cb58c [Added] MaskCompare node 2025-11-06 11:01:16 -03:00
Salvador E. Tropea afeda6db47 [Modified][ImageDataset] Progress indication
More detailed
2025-11-06 10:41:36 -03:00
Salvador E. Tropea a58773147c [Added] Support for batched load/save
- Also control over preview
- And control over metadata
2025-11-06 09:57:15 -03:00
Salvador E. Tropea e40b1f76a6 [Added] Node to save a mask to any path 2025-11-05 20:52:15 -03:00
Salvador E. Tropea 81bf74ec1f [Added] Random sort for the dataset loader 2025-11-05 20:39:44 -03:00
Salvador E. Tropea 9d411bd074 [ImageDataset] Made it "stateless"
- Our state is the file system, which files are processed.
- We now can process N images in RAM and use the queue to process
  the whole dataset
2025-11-05 18:18:55 -03:00
Salvador E. Tropea 5265d27b93 [Added] Node to save images to arbitrary paths 2025-11-05 16:47:55 -03:00
Salvador E. Tropea a3a9872bf3 [Added][WIP] Node to load a dataset
- Also made more robust the LoadImage dependency, if absent we have a
  faal-back
2025-11-05 14:03:29 -03:00
Salvador E. Tropea 1078a8a9e0 Bumped version to 1.4.0 2025-10-22 13:23:05 -03:00
Salvador E. Tropea e939a863c2 [Added] "Image with text label" node 2025-10-20 10:50:03 -03:00
Salvador E. Tropea 1d1b99ac49 [Added] Arbitrary normalization and a node to input the parameters 2025-10-14 11:40:16 -03:00
Salvador E. Tropea c5060bafe6 [Added] Icon for the ComfyUI registry 2025-10-14 09:31:35 -03:00
Salvador E. Tropea 055236953d [Added] Icon image 2025-10-14 09:30:54 -03:00
Salvador E. Tropea 346fc8590b [DOCs] Updated the nodes description 2025-10-14 09:16:53 -03:00
Salvador E. Tropea 2c732ce7cb Bumped version to 1.3.0 2025-10-14 09:11:11 -03:00
Salvador E. Tropea e7d124d786 [ImagePad][Added] Copy input size when target_* is 0 2025-10-14 09:04:32 -03:00
Salvador E. Tropea c976f3d057 [DOCs] And another way to display the examples 2025-10-14 08:15:32 -03:00
Salvador E. Tropea c74e859aaa [DOCs] Trying a different way to show the examples 2025-10-14 08:14:28 -03:00
Salvador E. Tropea 684d3fcb6b [DOCs][Added] Image for the examples 2025-10-14 08:12:22 -03:00
Salvador E. Tropea 0f0157ed4f [DOCs] Moved node docs to a separated file
To make the README.md more clear
2025-10-14 08:07:31 -03:00
Salvador E. Tropea b306bd37a7 [DOCs] Separated nodes in categories (in the ToC) 2025-10-14 07:57:47 -03:00
Salvador E. Tropea ea94753216 [DOCs] Made the example links to point to raw files, easier to download 2025-10-14 07:52:54 -03:00
Salvador E. Tropea 3844dd20b5 [DOCs][Added] Documentation about the new nodes 2025-10-14 07:48:32 -03:00
Salvador E. Tropea 79f1003873 [Added] Resize example 2025-10-13 20:11:08 -03:00
Salvador E. Tropea e159265648 [ImagePad/Resize][Added] Control over padding transparency
By default is transparent
2025-10-13 20:00:58 -03:00
Salvador E. Tropea aa8dc7b2ce [Download][Added] Option to create RGBA images 2025-10-13 19:17:13 -03:00
Salvador E. Tropea 684afdf5e6 [ImageResize][Fixed] Mask processing for modes others than edge_pixel
Avoid replicating the edge for other modes.
2025-10-13 18:24:30 -03:00
Salvador E. Tropea 314c6bf609 [ResizeImage][Fixed] Comment about the difference when w/h==0 2025-10-13 17:50:04 -03:00
Salvador E. Tropea 529756d415 [ImagePad][Fixed] Error when padding an RGBA image 2025-10-13 17:47:54 -03:00
Salvador E. Tropea 79b14305d5 [ImageResize][Fixed] Size computation when w/h are 0 2025-10-13 13:24:30 -03:00
Salvador E. Tropea 968859fc5a [ImageResize][Fixed] Color must be [0,1] 2025-10-13 13:23:23 -03:00
Salvador E. Tropea f05c36f520 [Nodes][Added] ResizeMask also synced with Kijai 1.1.7 2025-10-06 13:36:24 -03:00
Salvador E. Tropea d99c165f3d Merge branch 'resize_pad'
Some operations are needed and Comfy-Core lacks them
2025-10-05 14:42:14 -03:00
Salvador E. Tropea 015f8be8c9 [Added] Resize and Pad image from KJNodes
Just to keep dependencies low
But current Comfy-Core can handle it
2025-09-16 08:36:55 -03:00
24 changed files with 3992 additions and 388 deletions
+39 -187
View File
@@ -17,16 +17,27 @@ Currently we just have a few nodes used by other nodes I maintain.
- 📦 [Dependencies](#-dependencies)
- 🖼️ [Examples](#️-examples)
- ✨ [Nodes](#-extra-nodes)
- [Image Download and Load](#1-image-download-and-load)
- [Face Composite](#2-face-composite)
- [Face Composite (frame by frame)](#3-face-composite-frame-by-frame)
- [Normalize Image to ImageNet](#4-normalize-image-to-imagenet)
- [Normalize Image to [-0.5, 0.5]](#5-normalize-image-to-05-05)
- [Normalize Image to [-1, 1]](#6-normalize-image-to-1-1)
- [Apply Mask using AFFCE](#7-apply-mask-using-affce)
- [Estimate foreground (AFFCE)](#8-estimate-foreground-affce)
- [Estimate foreground (FMLFE)](#9-estimate-foreground-fmlfe)
- [Create Empty Image](#10-create-empty-image)
- Foreground: (Foreground estimation)
- [Estimate foreground (AFFCE)](docs/nodes_img.md#8-estimate-foreground-affce)
- [Estimate foreground (FMLFE)](docs/nodes_img.md#9-estimate-foreground-fmlfe)
- Generation:
- [Create Empty Image](docs/nodes_img.md#10-create-empty-image)
- Input/Output:
- [Image Download and Load](docs/nodes_img.md#1-image-download-and-load)
- Manipulation:
- [Face Composite](docs/nodes_img.md#2-face-composite)
- [Face Composite (frame by frame)](docs/nodes_img.md#3-face-composite-frame-by-frame)
- [Apply Mask using AFFCE](docs/nodes_img.md#7-apply-mask-using-affce)
- [Pad Image](docs/nodes_img.md#11-pad-image-kjset)
- [Resize Image](docs/nodes_img.md#12-resize-image-kjset)
- [Resize Mask](docs/nodes_img.md#13-resize-mask-kjset)
- [Image with text label](docs/nodes_img.md#16-image-with-text-label)
- Normalization:
- [Normalize Image to ImageNet](docs/nodes_img.md#4-normalize-image-to-imagenet)
- [Normalize Image to [-0.5, 0.5]](docs/nodes_img.md#5-normalize-image-to-05-05)
- [Normalize Image to [-1, 1]](docs/nodes_img.md#6-normalize-image-to-1-1)
- [Arbitrary Normalize](docs/nodes_img.md#14-arbitrary-normalize)
- [Normalize Parameters](docs/nodes_img.md#15-normalize-parameters)
- 📝 [Usage Notes](#-usage-notes)
- 📜 [Project History](#-project-history)
- ⚖️ [License](#️-license)
@@ -34,180 +45,7 @@ Currently we just have a few nodes used by other nodes I maintain.
## ✨ Nodes
### 1. Image Download and Load
- **Display Name:** `Image Download and Load`
- **Internal Name:** `SET_ImageDownload`
- **Category:** `image/io`
- **Description:** Downloads an image file from a URL into the `ComfyUI/input/` directory if it's not already there, and then loads it as an image and mask. This is perfect for creating self-contained, shareable workflows with example image.
- **Inputs:**
- `image_bypass` (IMAGE, Optional): If an image is provided here it will be used for the output. You can connect a `Load Image` node here, if the connected node is muted (bypassed) we download the file, otherwise we use the image from the `Load Image` node.
- `mask_bypass` (MASK, Optional): This input complements `image_bypass`.
- `base_url` (STRING): The URL of the directory containing the image file.
- `filename` (STRING): The name of the image file to download (e.g., photo.jpg, art.png).
- `local_name` (STRING, optional): The name used locally. Leave blank to use `filename`.
- **Output:**
- `image` (IMAGE): The loaded image.
- `alpha_mask` (MASK): The alpha mask for the loaded image.
- **Behavior Details:**
- **Caching:** The node checks the `ComfyUI/input/` folder first. If the file with the specified `filename` already exists, the download is skipped.
- **Bypass:** If only one of `image_bypass` and `mask_bypass` is connected the other will be assumed to be empty. You should connect both or avoid using the output corresponding to the unconnected input.
### 2. Face Composite
- **Display Name:** `Face Composite`
- **Internal Name:** `SET_CompositeFace`
- **Category:** `image/manipulation`
- **Description:** This node is designed for complex batch operations where a single reference image generates multiple animated or processed frames. It handles an **M-to-N** relationship.
- **Purpose:** The primary use case is for animation workflows (e.g., using AnimateDiff or SVD) where you:
1. Extract a face from **one** source image.
2. Generate **N** animated frames of that face.
3. Paste each of the **N** animated frames back onto the original source image to create an animated sequence.
- **Inputs:**
- `animated` (`IMAGE`): A batch of `M*N` cropped/processed images (e.g., the animated faces).
- `reference` (`IMAGE`): A batch of `M` original context images that the faces were extracted from.
- `bboxes` (`BBOX`): A list of `M` bounding boxes. Each box in the list must correspond to a `reference` image, defining the location for the paste operation. Format: (X, Y, W, H)
- **Output:**
- `images` (`IMAGE`): The final output batch of `M*N` full-size images. Each image consists of a `reference` image with one of the `animated` frames pasted onto it.
- **How it Works:**
- For each of the `M` reference images and its corresponding bounding box, the node takes the next `N` frames from the `animated` batch. It then pastes each of these `N` frames onto a *copy* of the reference image, generating `N` final output frames. This process is repeated for all `M` reference images.
- The animated frame is automatically resized to fit the bounding box dimensions before pasting. The pasting logic safely handles cases where the bbox is partially outside the image boundaries.
### 3. Face Composite (frame by frame)
- **Display Name:** `"Face Composite (frame by frame)`
- **Internal Name:** `SET_CompositeFaceFrameByFrame`
- **Category:** `image/manipulation`
- **Description:** This node is a simplified variant for **1-to-1** compositing. It is perfect for video processing workflows where you need to paste a sequence of processed frames back into the original video sequence at a static location.
- **Purpose:** Ideal for tasks like video face restoration, stylization, or simple lip-sync, where:
1. A video is loaded as a sequence of `N` frames.
2. A corresponding sequence of `N` processed/animated frames is generated.
3. Each processed frame needs to be pasted back into its corresponding original frame at the same location.
- **Inputs:**
- `animated` (`IMAGE`): A batch of `N` processed frames.
- `reference` (`IMAGE`): A batch of `N` original frames. **Must have the same batch size as `animated`**.
- `bboxes` (`BBOX`): A list of bounding boxes. **Only the first bbox in the list is used** as the static paste location for all frames.
- **Output:**
- `images` (`IMAGE`): The final batch of `N` composited frames.
- **How it Works:** The node iterates from frame 0 to N-1. In each step, it takes the i-th `animated` frame and the i-th `reference` frame. It then pastes the animated frame onto the reference frame using the coordinates from the single, static bounding box.
### 4. Normalize Image to ImageNet
- **Display Name:** `"Normalize Image to ImageNet`
- **Internal Name:** `SET_NormalizeToImageNetDataset`
- **Category:** `image/normalization`
- **Description:** Normalizes an image tensor using the mean and standard deviation of the ImageNet dataset.
- **Purpose:** Essential for pre-processing images before feeding them into models that were pre-trained on ImageNet (e.g., most ResNet, VGG, EfficientNet models).
- **Inputs:**
- `image` (`IMAGE`): A standard ComfyUI image tensor in the `[0, 1]` range.
- **Output:**
- `image` (`IMAGE`): The normalized image tensor. The value range will be altered significantly.
- **How it Works:** For each channel, it performs the operation `output = (input - mean) / std`, using the standard ImageNet values:
- **Mean:** `[0.485, 0.456, 0.406]`
- **Std Dev:** `[0.229, 0.224, 0.225]`
### 5. Normalize Image to [-0.5, 0.5]
- **Display Name:** `Normalize Image to [-0.5, 0.5]`
- **Internal Name:** `SET_NormalizeToMinus05_05`
- **Category:** `image/normalization`
- **Description:** Normalizes an image tensor by centering its values around zero.
- **Purpose:** Useful for models trained from scratch or those that expect input data in the `[-0.5, 0.5]` range. This can help stabilize training.
- **Inputs:**
- `image` (`IMAGE`): A standard ComfyUI image tensor in the `[0, 1]` range.
- **Output:**
- `image` (`IMAGE`): The normalized image tensor, with values in the `[-0.5, 0.5]` range.
- **How it Works:** For each channel, it performs the operation `output = (input - mean) / std`, using:
- **Mean:** `[0.5, 0.5, 0.5]`
- **Std Dev:** `[1.0, 1.0, 1.0]`
### 6. Normalize Image to [-1, 1]
- **Display Name:** `Normalize Image to [-1, 1]`
- **Internal Name:** `SET_NormalizeToMinus1_1`
- **Category:** `image/normalization`
- **Description:** Normalizes an image tensor to the `[-1, 1]` range.
- **Purpose:** A common requirement for certain model architectures, particularly Generative Adversarial Networks (GANs) and models using the `tanh` activation function in their output layer.
- **Inputs:**
- `image` (`IMAGE`): A standard ComfyUI image tensor in the `[0, 1]` range.
- **Output:**
- `image` (`IMAGE`): The normalized image tensor, with values in the `[-1, 1]` range.
- **How it Works:** For each channel, it performs the operation `output = (input - mean) / std`, using:
- **Mean:** `[0.5, 0.5, 0.5]`
- **Std Dev:** `[0.5, 0.5, 0.5]`
### 7. Apply Mask using AFFCE
- **Display Name:** `Apply Mask using AFFCE`
- **Internal Name:** `SET_ApplyMaskAFFCE`
- **Category:** `image/manipulation`
- **Description:** Applies a mask to an image using [Approximate Fast Foreground Colour Estimation](https://github.com/Photoroom/fast-foreground-estimation). This blends the image contour in a better way.
- **Purpose:** Used to apply the mask of a background removal model.
- **Inputs:**
- `images` (`IMAGE`): One ore more ComfyUI images
- `masks` (`MASK`): Masks to apply
- `blur_size` (`INT`): Diameter for the coarse gaussian blur
- `blur_size_two` (`INT`): Diameter for the fine gaussian blur
- `fill_color` (`BOOLEAN`): When enabled the removed image is replaced by a color, the output is an RGB image. Otherwise the removed part becomes transparent and the output is an RGBA image.
- `color` (`STRING`): A string representing a color to be used when `fill_color` is enabled. Can be an hexadecimal RGB (i.e. `#AABBCC`) or comma separated RGB components. The components can be in the [0-255] or [0-1.0] range.
- `batched` (`BOOLEAN`): Process all the images at once. Otherwise do it one at a time.
- **Output:**
- `image` (`IMAGE`): The image after applying the mask.
- `mask` (`MASK`): The input mask
### 8. Estimate foreground (AFFCE)
- **Display Name:** `Estimate foreground (AFFCE)`
- **Internal Name:** `SET_AFFCE`
- **Category:** `image/foreground`
- **Description:** Estimates the foreground of an image using [Approximate Fast Foreground Colour Estimation](https://github.com/Photoroom/fast-foreground-estimation). The result is suitable for background replacement.
- **Purpose:** Used to get a better foreground for background removal.
- **Inputs:**
- `images` (`IMAGE`): One ore more ComfyUI images
- `masks` (`MASK`): Masks to apply
- `blur_size` (`INT`): Diameter for the coarse gaussian blur
- `blur_size_two` (`INT`): Diameter for the fine gaussian blur
- `batched` (`BOOLEAN`): Process all the images at once. Otherwise do it one at a time.
- **Output:**
- `image` (`IMAGE`): The image after foreground estimation.
- `mask` (`MASK`): The input mask
### 9. Estimate foreground (FMLFE)
- **Display Name:** `Estimate foreground (FMLFE)`
- **Internal Name:** `SET_FMLFE`
- **Category:** `image/foreground`
- **Description:** Estimates the foreground of an image using [Fast Multi-Level Foreground Estimation](https://arxiv.org/abs/2006.14970). The result is suitable for background replacement.
- **Purpose:** Used to get a better foreground for background removal.
- **Inputs:**
- `images` (`IMAGE`): The source image(s) from which to estimate the foreground and background.
- `masks` (`MASK`): The alpha matte that guides the estimation. White areas are treated as known foreground, black as known background, and gray areas are the semi-transparent regions the algorithm will solve for.
- `implementation" (`auto`, `cupy`, `opencl`, `numba`, `torch`): Which implementation to use. The `auto` will use the fastest available. The `torch` implementation is slow and approximated, but doesn't need extra dependencies. The `numba` implementation is good and just needs [Numba](https://numba.pydata.org/). The `cupy` implementation is the fastest, but needs [CuPy](https://cupy.dev/) and a full CUDA environment.
- `regularization` (`FLOAT`): The regularization strength (epsilon). This acts as a smoothness prior. Higher values result in smoother, more blended foreground and background colors, but may lose very fine details. Lower values preserve more detail but can be noisier.
- `n_small_iterations` (`INT`): The number of solver iterations to perform on the lower-resolution levels of the image pyramid. More iterations can improve quality at the cost of speed.
- `n_big_iterations` (`INT`): The number of solver iterations to perform on the higher-resolution (larger) levels of the image pyramid. Fewer iterations are typically needed at high resolution as the details are propagated up from the smaller levels.
- `small_size` (`INT`): The pixel dimension threshold. Image pyramid levels smaller than this size will use the higher 'n_small_iterations' count, while larger levels will use 'n_big_iterations'.
- `gradient_weight` (`FLOAT`): Controls how strongly the edges in the alpha matte influence color blending. A higher value makes the algorithm respect the mask's edges more, leading to sharper color boundaries. A lower value allows more color bleeding, an effect similar to increasing regularization.
- **Output:**
- `image` (`IMAGE`): The estimated foreground image (F). This is a full-color image where the algorithm has estimated the true, un-blended color of the foreground object.
- `mask` (`MASK`): The estimated background image (B). The algorithm has effectively "inpainted" the area behind the foreground object, creating a clean background plate.
### 10. Create Empty Image
- **Display Name:** `Create Empty Image`
- **Internal Name:** `SET_CreateEmptyImage`
- **Category:** `image/generation`
- **Description:** Creates an image filled with a solid color. Similar to standard `EmptyImage`, but you can use an image as reference and you have more options to select the color.
- **Purpose:** Create an empty image
- **Inputs:**
- `width` (`INT`): The width of the new image in pixels. This value is ignored if a `reference` is provided.
- `height` (`INT`): The height of the new image in pixels. This value is ignored if a `reference` is provided.
- `batch_size` (`INT`): The number of images to create in the batch. This value is ignored if a `reference` is provided.
- `color` (`STRING`): The solid color to fill the image with. Can be a named color (e.g., "black", "red"), a hex string (e.g., "#FF0000", "00ff00"), or comma-separated components in the [0, 255] or [0.0, 1.0] range.
- `reference` (`IMAGE`, optional): If an image is connected here, its dimensions (batch size, height, and width) will be used for the new image, overriding the manual width, height, and batch_size inputs.
- **Output:**
- `image` (`IMAGE`): A new image tensor of the specified size and color.
The nodes are documented [here](docs/nodes_img.md). Use the above ToC to access them by category.
## 🚀 Installation
@@ -221,7 +59,7 @@ You can install the nodes from the ComfyUI nodes manager, the name is *Image Mis
2. Install dependencies: `pip install -r ComfyUI/custom_nodes/ComfyUI-ImageMisc/requirements.txt`
3. Restart ComfyUI.
The nodes should then appear under the "image/io" category in the "Add Node" menu.
The nodes should then appear under the "image/io", "image/manipulation", "image/normalization" and "image/foreground" categories in the "Add Node" menu.
## 📦 Dependencies
@@ -238,7 +76,18 @@ The nodes should then appear under the "image/io" category in the "Add Node" men
Once installed the examples are available in the ComfyUI workflow templates, in the *Image Misc* section (or ComfyUI-ImageMisc).
- [image_download.json](example_workflows/image_download.json): Shows how to use the image downloader node.
### [image_download.json](https://raw.githubusercontent.com/set-soft/ComfyUI-ImageMisc/refs/heads/main/example_workflows/image_download.json)
Shows how to use the image downloader node.
[![image_download.json](example_workflows/image_download.jpg)](https://raw.githubusercontent.com/set-soft/ComfyUI-ImageMisc/refs/heads/main/example_workflows/image_download.json)
### [resize.json](https://raw.githubusercontent.com/set-soft/ComfyUI-ImageMisc/refs/heads/main/example_workflows/resize.json)
Shows the most common uses of the resize and padding nodes. You can see how various modes and options affects the result and how the different interpolation methods works. It shows how masks and RGBA images are handled.
[![resize.json](example_workflows/resize.jpg)](https://raw.githubusercontent.com/set-soft/ComfyUI-ImageMisc/refs/heads/main/example_workflows/resize.json)
## 📝 Usage Notes
@@ -261,7 +110,9 @@ Once installed the examples are available in the ComfyUI workflow templates, in
- Apply mask nodes, including support for:
- "Fast Multi-Level Foreground Estimation" (from [PyMatting](https://github.com/pymatting/pymatting))
- "Approximate Fast Foreground Colour Estimation" (from [Photoroom](https://github.com/Photoroom/fast-foreground-estimation))
- Create empty image node, with reference image
- Create empty image node, from reference image
- 1.3.0 2025-10-14: Pad Image, Resize Image and Resize Mask nodes
- 1.4.0 2025-10-20: "Image with text label" and "Arbitrary Normalize"/"Normalize Parameters"
## ⚖️ License
@@ -271,3 +122,4 @@ Once installed the examples are available in the ComfyUI workflow templates, in
## 🙏 Attributions
- Good part of the initial code and this README was generated using Gemini 2.5 Pro.
- The *Pad Image*, *Resize Image* and *Resize Mask* are from [KJNodes](https://github.com/kijai/ComfyUI-KJNodes) by [Kijai](https://github.com/kijai) (Jukka Seppänen)
Binary file not shown.

After

Width:  |  Height:  |  Size: 51 KiB

+307
View File
@@ -0,0 +1,307 @@
# 1. Image Download and Load
- **Display Name:** `Image Download and Load`
- **Internal Name:** `SET_ImageDownload`
- **Category:** `image/io`
- **Description:** Downloads an image file from a URL into the `ComfyUI/input/` directory if it's not already there, and then loads it as an image and mask. This is perfect for creating self-contained, shareable workflows with example image.
- **Inputs:**
- `image_bypass` (IMAGE, Optional): If an image is provided here it will be used for the output. You can connect a `Load Image` node here, if the connected node is muted (bypassed) we download the file, otherwise we use the image from the `Load Image` node.
- `mask_bypass` (MASK, Optional): This input complements `image_bypass`.
- `base_url` (STRING): The URL of the directory containing the image file.
- `filename` (STRING): The name of the image file to download (e.g., photo.jpg, art.png).
- `local_name` (STRING, optional): The name used locally. Leave blank to use `filename`.
- `embed_transparency` (BOOLEAN, optional): Create RGBA images when they have transparency. Not all nodes supports them.
- **Output:**
- `image` (IMAGE): The loaded image.
- `alpha_mask` (MASK): The alpha mask for the loaded image.
- **Behavior Details:**
- **Caching:** The node checks the `ComfyUI/input/` folder first. If the file with the specified `filename` already exists, the download is skipped.
- **Bypass:** If only one of `image_bypass` and `mask_bypass` is connected the other will be assumed to be empty. You should connect both or avoid using the output corresponding to the unconnected input.
# 2. Face Composite
- **Display Name:** `Face Composite`
- **Internal Name:** `SET_CompositeFace`
- **Category:** `image/manipulation`
- **Description:** This node is designed for complex batch operations where a single reference image generates multiple animated or processed frames. It handles an **M-to-N** relationship.
- **Purpose:** The primary use case is for animation workflows (e.g., using AnimateDiff or SVD) where you:
1. Extract a face from **one** source image.
2. Generate **N** animated frames of that face.
3. Paste each of the **N** animated frames back onto the original source image to create an animated sequence.
- **Inputs:**
- `animated` (`IMAGE`): A batch of `M*N` cropped/processed images (e.g., the animated faces).
- `reference` (`IMAGE`): A batch of `M` original context images that the faces were extracted from.
- `bboxes` (`BBOX`): A list of `M` bounding boxes. Each box in the list must correspond to a `reference` image, defining the location for the paste operation. Format: (X, Y, W, H)
- **Output:**
- `images` (`IMAGE`): The final output batch of `M*N` full-size images. Each image consists of a `reference` image with one of the `animated` frames pasted onto it.
- **How it Works:**
- For each of the `M` reference images and its corresponding bounding box, the node takes the next `N` frames from the `animated` batch. It then pastes each of these `N` frames onto a *copy* of the reference image, generating `N` final output frames. This process is repeated for all `M` reference images.
- The animated frame is automatically resized to fit the bounding box dimensions before pasting. The pasting logic safely handles cases where the bbox is partially outside the image boundaries.
# 3. Face Composite (frame by frame)
- **Display Name:** `Face Composite (frame by frame)`
- **Internal Name:** `SET_CompositeFaceFrameByFrame`
- **Category:** `image/manipulation`
- **Description:** This node is a simplified variant for **1-to-1** compositing. It is perfect for video processing workflows where you need to paste a sequence of processed frames back into the original video sequence at a static location.
- **Purpose:** Ideal for tasks like video face restoration, stylization, or simple lip-sync, where:
1. A video is loaded as a sequence of `N` frames.
2. A corresponding sequence of `N` processed/animated frames is generated.
3. Each processed frame needs to be pasted back into its corresponding original frame at the same location.
- **Inputs:**
- `animated` (`IMAGE`): A batch of `N` processed frames.
- `reference` (`IMAGE`): A batch of `N` original frames. **Must have the same batch size as `animated`**.
- `bboxes` (`BBOX`): A list of bounding boxes. **Only the first bbox in the list is used** as the static paste location for all frames.
- **Output:**
- `images` (`IMAGE`): The final batch of `N` composited frames.
- **How it Works:** The node iterates from frame 0 to N-1. In each step, it takes the i-th `animated` frame and the i-th `reference` frame. It then pastes the animated frame onto the reference frame using the coordinates from the single, static bounding box.
# 4. Normalize Image to ImageNet
- **Display Name:** `Normalize Image to ImageNet`
- **Internal Name:** `SET_NormalizeToImageNetDataset`
- **Category:** `image/normalization`
- **Description:** Normalizes an image tensor using the mean and standard deviation of the ImageNet dataset.
- **Purpose:** Essential for pre-processing images before feeding them into models that were pre-trained on ImageNet (e.g., most ResNet, VGG, EfficientNet models).
- **Inputs:**
- `image` (`IMAGE`): A standard ComfyUI image tensor in the `[0, 1]` range.
- **Output:**
- `image` (`IMAGE`): The normalized image tensor. The value range will be altered significantly.
- **How it Works:** For each channel, it performs the operation `output = (input - mean) / std`, using the standard ImageNet values:
- **Mean:** `[0.485, 0.456, 0.406]`
- **Std Dev:** `[0.229, 0.224, 0.225]`
# 5. Normalize Image to [-0.5, 0.5]
- **Display Name:** `Normalize Image to [-0.5, 0.5]`
- **Internal Name:** `SET_NormalizeToMinus05_05`
- **Category:** `image/normalization`
- **Description:** Normalizes an image tensor by centering its values around zero.
- **Purpose:** Useful for models trained from scratch or those that expect input data in the `[-0.5, 0.5]` range. This can help stabilize training.
- **Inputs:**
- `image` (`IMAGE`): A standard ComfyUI image tensor in the `[0, 1]` range.
- **Output:**
- `image` (`IMAGE`): The normalized image tensor, with values in the `[-0.5, 0.5]` range.
- **How it Works:** For each channel, it performs the operation `output = (input - mean) / std`, using:
- **Mean:** `[0.5, 0.5, 0.5]`
- **Std Dev:** `[1.0, 1.0, 1.0]`
# 6. Normalize Image to [-1, 1]
- **Display Name:** `Normalize Image to [-1, 1]`
- **Internal Name:** `SET_NormalizeToMinus1_1`
- **Category:** `image/normalization`
- **Description:** Normalizes an image tensor to the `[-1, 1]` range.
- **Purpose:** A common requirement for certain model architectures, particularly Generative Adversarial Networks (GANs) and models using the `tanh` activation function in their output layer.
- **Inputs:**
- `image` (`IMAGE`): A standard ComfyUI image tensor in the `[0, 1]` range.
- **Output:**
- `image` (`IMAGE`): The normalized image tensor, with values in the `[-1, 1]` range.
- **How it Works:** For each channel, it performs the operation `output = (input - mean) / std`, using:
- **Mean:** `[0.5, 0.5, 0.5]`
- **Std Dev:** `[0.5, 0.5, 0.5]`
# 7. Apply Mask using AFFCE
- **Display Name:** `Apply Mask using AFFCE`
- **Internal Name:** `SET_ApplyMaskAFFCE`
- **Category:** `image/manipulation`
- **Description:** Applies a mask to an image using [Approximate Fast Foreground Colour Estimation](https://github.com/Photoroom/fast-foreground-estimation). This blends the image contour in a better way.
- **Purpose:** Used to apply the mask of a background removal model.
- **Inputs:**
- `images` (`IMAGE`): One ore more ComfyUI images
- `masks` (`MASK`): Masks to apply
- `blur_size` (`INT`): Diameter for the coarse gaussian blur
- `blur_size_two` (`INT`): Diameter for the fine gaussian blur
- `fill_color` (`BOOLEAN`): When enabled the removed image is replaced by a color, the output is an RGB image. Otherwise the removed part becomes transparent and the output is an RGBA image.
- `color` (`STRING`): A string representing a color to be used when `fill_color` is enabled. Can be an hexadecimal RGB (i.e. `#AABBCC`) or comma separated RGB components. The components can be in the [0-255] or [0-1.0] range.
- `batched` (`BOOLEAN`): Process all the images at once. Otherwise do it one at a time.
- **Output:**
- `image` (`IMAGE`): The image after applying the mask.
- `mask` (`MASK`): The input mask
# 8. Estimate foreground (AFFCE)
- **Display Name:** `Estimate foreground (AFFCE)`
- **Internal Name:** `SET_AFFCE`
- **Category:** `image/foreground`
- **Description:** Estimates the foreground of an image using [Approximate Fast Foreground Colour Estimation](https://github.com/Photoroom/fast-foreground-estimation). The result is suitable for background replacement.
- **Purpose:** Used to get a better foreground for background removal.
- **Inputs:**
- `images` (`IMAGE`): One ore more ComfyUI images
- `masks` (`MASK`): Masks to apply
- `blur_size` (`INT`): Diameter for the coarse gaussian blur
- `blur_size_two` (`INT`): Diameter for the fine gaussian blur
- `batched` (`BOOLEAN`): Process all the images at once. Otherwise do it one at a time.
- **Output:**
- `image` (`IMAGE`): The image after foreground estimation.
- `mask` (`MASK`): The input mask
# 9. Estimate foreground (FMLFE)
- **Display Name:** `Estimate foreground (FMLFE)`
- **Internal Name:** `SET_FMLFE`
- **Category:** `image/foreground`
- **Description:** Estimates the foreground of an image using [Fast Multi-Level Foreground Estimation](https://arxiv.org/abs/2006.14970). The result is suitable for background replacement.
- **Purpose:** Used to get a better foreground for background removal.
- **Inputs:**
- `images` (`IMAGE`): The source image(s) from which to estimate the foreground and background.
- `masks` (`MASK`): The alpha matte that guides the estimation. White areas are treated as known foreground, black as known background, and gray areas are the semi-transparent regions the algorithm will solve for.
- `implementation" (`auto`, `cupy`, `opencl`, `numba`, `torch`): Which implementation to use. The `auto` will use the fastest available. The `torch` implementation is slow and approximated, but doesn't need extra dependencies. The `numba` implementation is good and just needs [Numba](https://numba.pydata.org/). The `cupy` implementation is the fastest, but needs [CuPy](https://cupy.dev/) and a full CUDA environment.
- `regularization` (`FLOAT`): The regularization strength (epsilon). This acts as a smoothness prior. Higher values result in smoother, more blended foreground and background colors, but may lose very fine details. Lower values preserve more detail but can be noisier.
- `n_small_iterations` (`INT`): The number of solver iterations to perform on the lower-resolution levels of the image pyramid. More iterations can improve quality at the cost of speed.
- `n_big_iterations` (`INT`): The number of solver iterations to perform on the higher-resolution (larger) levels of the image pyramid. Fewer iterations are typically needed at high resolution as the details are propagated up from the smaller levels.
- `small_size` (`INT`): The pixel dimension threshold. Image pyramid levels smaller than this size will use the higher 'n_small_iterations' count, while larger levels will use 'n_big_iterations'.
- `gradient_weight` (`FLOAT`): Controls how strongly the edges in the alpha matte influence color blending. A higher value makes the algorithm respect the mask's edges more, leading to sharper color boundaries. A lower value allows more color bleeding, an effect similar to increasing regularization.
- **Output:**
- `image` (`IMAGE`): The estimated foreground image (F). This is a full-color image where the algorithm has estimated the true, un-blended color of the foreground object.
- `mask` (`MASK`): The estimated background image (B). The algorithm has effectively "inpainted" the area behind the foreground object, creating a clean background plate.
# 10. Create Empty Image
- **Display Name:** `Create Empty Image`
- **Internal Name:** `SET_CreateEmptyImage`
- **Category:** `image/generation`
- **Description:** Creates an image filled with a solid color. Similar to standard `EmptyImage`, but you can use an image as reference and you have more options to select the color.
- **Purpose:** Create an empty image
- **Inputs:**
- `width` (`INT`): The width of the new image in pixels. This value is ignored if a `reference` is provided.
- `height` (`INT`): The height of the new image in pixels. This value is ignored if a `reference` is provided.
- `batch_size` (`INT`): The number of images to create in the batch. This value is ignored if a `reference` is provided.
- `color` (`STRING`): The solid color to fill the image with. Can be a named color (e.g., "black", "red"), a hex string (e.g., "#FF0000", "00ff00"), or comma-separated components in the [0, 255] or [0.0, 1.0] range.
- `reference` (`IMAGE`, optional): If an image is connected here, its dimensions (batch size, height, and width) will be used for the new image, overriding the manual width, height, and batch_size inputs.
- **Output:**
- `image` (`IMAGE`): A new image tensor of the specified size and color.
# 11. Pad Image (KJ/SET)
- **Display Name:** `Pad Image (KJ/SET)`
- **Internal Name:** `SET_ImagePad`
- **Category:** `image/manipulation`
- **Description:** Pads the input image and an optional mask with specified padding on each side. It offers several modes to fill the padded area.
- **Purpose:** To add borders or expand the canvas of an image using various fill methods, such as solid color, edge pixel replication, or a blurred background effect.
- **Inputs:**
- `image` (`IMAGE`): The image to be padded.
- `left` (`INT`): The number of pixels of padding to add to the left side.
- `right` (`INT`): The number of pixels of padding to add to the right side.
- `top` (`INT`): The number of pixels of padding to add to the top side.
- `bottom` (`INT`): The number of pixels of padding to add to the bottom side.
- `extra_padding` (`INT`): Additional padding applied to all sides. Note: If `target_width` and `target_height` are used, this value will shrink the source image before padding.
- `pad_mode` (`STRING`): The method used to fill the padded area.
- `edge`: Fills with the average color of the nearest edge.
- `edge_pixel`: Repeats the outermost line of pixels.
- `color`: Fills with the specified solid `color`.
- `pillarbox_blur`: Fills with a blurred, desaturated, and dimmed version of the original image, scaled to fit the new dimensions.
- `color` (`STRING`): The solid color to use when `pad_mode` is "color". Accepts hex codes, color names, and RGB values. This is more flexible than in KJNodes.
- `mask` (`MASK`, optional): An optional mask to be padded along with the image.
- `target_width` (`INT`, optional): If provided with `target_height`, pads the image to this exact width, centering the original content. This overrides the `left` and `right` inputs. If you set it to 0 the width of the image is used. Note that you must also connect `target_height`.
- `target_height` (`INT`, optional): If provided with `target_width`, pads the image to this exact height, centering the original content. This overrides the `top` and `bottom` inputs. If you set it to 0 the heigth of the image is used. Note that you must also connect `target_width`.
- `pad_transparency` (`FLOAT`, optional): Controls the opacity of the padded area in the output mask. 1.0 is fully transparent (no mask), and 0.0 is fully opaque (masked). Not in KJNodes.
- **Outputs:**
- `images` (`IMAGE`): The padded image tensor.
- `masks` (`MASK`): The padded mask tensor.
# 12. Resize Image (KJ/SET)
- **Display Name:** `Resize Image (KJ/SET)`
- **Internal Name:** `SET_ImageResize`
- **Category:** `image/manipulation`
- **Description:** A comprehensive node to resize, crop, and pad images. It provides multiple methods for handling aspect ratio and can optionally process large batches in smaller chunks to conserve memory. This node uses [Pad Image](#11-pad-image-kjset) so please also read its documentation.
- **Purpose:** To change the dimensions of an image with fine-grained control over how the aspect ratio is preserved, including stretching, cropping, or padding to fit the target size.
- **Inputs:**
- `image` (`IMAGE`): The source image to resize.
- `width` (`INT`): The target width in pixels. If set to 0, it uses the original width or the width from `get_image_size`.
- `height` (`INT`): The target height in pixels. If set to 0, it uses the original height or the height from `get_image_size`.
- `upscale_method` (`STRING`): The interpolation algorithm to use for resizing (e.g., `bicubic`, `lanczos`, `nearest-exact`).
- `keep_proportion` (`STRING`): Defines how to handle the aspect ratio when resizing.
- `stretch`: Ignores aspect ratio and forces the image to the exact target dimensions.
- `resize`: Maintains aspect ratio, fitting the image within the target dimensions. The final output size may be smaller than the target on one axis.
- `pad`, `pad_edge`, `pad_edge_pixel`: Resizes the image to fit within the target dimensions while maintaining aspect ratio, then pads the remaining space to meet the exact target dimensions.
- `crop`: Resizes and crops the image to match the target aspect ratio and size.
- `pillarbox_blur`: Resizes the image to fit and fills the remaining space with a blurred background.
- `pad_color` (`STRING`): The solid color to use for padding when `keep_proportion` is set to `pad`. Accepts hex codes, color names, and RGB values. This is more flexible than in KJNodes.
- `crop_position` (`STRING`): The anchor point (`center`, `top`, `left`, etc.) for `crop` and `pad` operations.
- `divisible_by` (`INT`): Ensures the final width and height are multiples of this number, which is often required for models like VAEs.
- `mask` (`MASK`, optional): An optional mask to be transformed along with the image.
- `device` (`STRING`, optional): The computation device (`cpu` or `gpu`) to use for the operation. Note that `lanczos` is not supported on the GPU.
- `get_image_size` (`IMAGE`, optional): If an image is connected, its dimensions will be used as the target width and height, overriding the manual inputs. Not in KJNodes.
- `per_batch` (`INT`, optional): If greater than 0, processes the input batch in smaller sub-batches of this size to reduce memory usage.
- `pad_transparency` (`FLOAT`, optional): Controls the opacity of the padded area in the output mask. Not in KJNodes.
- **Outputs:**
- `IMAGE` (`IMAGE`): The transformed image.
- `width` (`INT`): The final width of the output image.
- `height` (`INT`): The final height of the output image.
- `mask` (`MASK`): The transformed mask.
# 13. Resize Mask (KJ/SET)
- **Display Name:** `Resize Mask (KJ/SET)`
- **Internal Name:** `SET_ResizeMask`
- **Category:** `image/manipulation`
- **Description:** Resizes a mask or a batch of masks to the specified width and height, with options to preserve proportions.
- **Purpose:** A dedicated and simplified node for resizing masks, which often require different interpolation methods (like `nearest-exact`) than images.
- **Inputs:**
- `mask` (`MASK`): The mask to be resized.
- `width` (`INT`): The target width for the new mask.
- `height` (`INT`): The target height for the new mask.
- `keep_proportions` (`BOOLEAN`): If `True`, maintains the original aspect ratio, fitting the mask within the target dimensions.
- `upscale_method` (`STRING`): The interpolation algorithm to use for resizing. Defaults to `nearest-exact`, which is ideal for masks to avoid creating grayscale values.
- `crop` (`STRING`): If and how to crop the mask (`disabled` or `center`).
- `get_image_size` (`IMAGE`, optional): If an image is connected here, its dimensions will be used as the target width and height.
- **Outputs:**
- `mask` (`MASK`): The resized mask tensor.
- `width` (`INT`): The final width of the output mask.
- `height` (`INT`): The final height of the output mask.
# 14. Arbitrary Normalize
- **Display Name:** `Arbitrary Normalize`
- **Internal Name:** `SET_NormalizeArbitrary`
- **Category:** `image/normalization`
- **Description:** Normalizes an image tensor using the mean and standard deviation provided in the `parameters` input.
- **Purpose:** Essential for pre-processing images before feeding them into models
- **Inputs:**
- `image` (`IMAGE`): A standard ComfyUI image tensor in the `[0, 1]` range.
- `parameters` (`NORM_PARAMS`): The parameters for the normalization. This is a dict with two keys, `mean` and `std`, which are lists with the R, G, B parameters.
- **Output:**
- `image` (`IMAGE`): The normalized image tensor. The value range will be altered significantly.
- **How it Works:** For each channel, it performs the operation `output = (input - mean) / std`, using the values from NORM_PARAMS
# 15. Normalize Parameters
- **Display Name:** `Normalize Parameters`
- **Internal Name:** `SET_NormalizeParameters`
- **Category:** `image/normalization`
- **Description:** Provides the `NORM_PARAMS` for the [Arbitrary Normalize](#14-arbitrary-normalize) node
- **Purpose:** This is a user interface to the [Arbitrary Normalize](#14-arbitrary-normalize) node
- **Inputs:**
- `mean_red` (`FLOAT`): Mean value for the red channel
- `mean_green` (`FLOAT`): Mean value for the green channel
- `mean_blue` (`FLOAT`): Mean value for the blue channel
- `std_red` (`FLOAT`): Standard deviation for the red channel
- `std_green` (`FLOAT`): Standard deviation for the green channel
- `std_blue` (`FLOAT`): Standard deviation for the blue channel
- **Output:**
- `parameters` (`NORM_PARAMS`): The parameters for the normalization. This is a dict with two keys, `mean` and `std`, which are lists with the R, G, B parameters.
# 16. Image with text label
- **Display Name:** `Image with text label`
- **Internal Name:** `SET_ImageWithTextLabel`
- **Category:** `image/manipulation`
- **Description:** Adds a border and a label to an image or a mask. This can be used to create comparison grids. Use the Comfy-Core `Image Stitch` node to concatenate the images.
- **Purpose:** Intended to create labeled images
- **Inputs:**
- `image` (`IMAGE`): Image to use, leave unconnected when using a mask.
- `mask` ("MASK`): Mask to be used as image, leave unconnected when using an image.
- `text` (`STRING`): Text to use as label. Can contain multiple lines.
- `side` ("top", "bottom", "left", "right"): The side where the label will be placed.
- `label_size` (`STRING`): The width (left/right) or height (top/bottom) of the area for the label. Can be expressed as a percentage (i.e. 10%) or absolute number of pixels.
- `separation` (`STRING`): The size of the border around the image. Can be expressed as a percentage (i.e. 1%) or absolute number of pixels.
- `background_color` (`STRING`): Color to fill the border and the label area. Can be an hexadecimal RGB (i.e. `#AABBCC`) or comma separated RGB components. The components can be in the [0-255] or [0-1.0] range. Some color names are also supported.
- `foreground_color` (`STRING`): Color the label font. Can be an hexadecimal RGB (i.e. `#AABBCC`) or comma separated RGB components. The components can be in the [0-255] or [0-1.0] range. Some color names are also supported.
- `font_name` (`STRING`): The name of the font to use for the text.
- **Output:**
- `IMAGE` (`IMAGE`): The generated RGB image.
+90
View File
@@ -0,0 +1,90 @@
### Documentation of Saliency Evaluation Metrics
This document details the quantitative metrics used to evaluate the performance of Salient Object Detection (SOD) and Dichotomous Image Segmentation (DIS) models. These metrics compare the model's continuous prediction map (`prediction`) with a binary ground truth map (`ground_truth`).
---
### 1. Mean Absolute Error (MAE)
* **What it Measures**: MAE computes the average pixel-wise absolute difference between the normalized prediction mask and the ground truth mask. It provides a direct, straightforward measure of the overall error.
* **Formula**:
$$ \text{MAE} = \frac{1}{W \times H} \sum_{x=1}^{W} \sum_{y=1}^{H} | P(x,y) - G(x,y) | $$
where `P` is the prediction, `G` is the ground truth, and `W, H` are the dimensions of the mask.
* **Interpretation**:
* **Lower is better**. A score of **0** indicates a perfect pixel-for-pixel match. A higher score indicates a greater average error across the entire image.
* **Relevance and Justification**: MAE is a fundamental metric that penalizes all errors equally, regardless of their location (e.g., at the center of an object vs. its boundary). While its simplicity is a strength, this lack of perceptual awareness is also its primary limitation. It serves as an excellent baseline for measuring raw prediction accuracy. Its use in saliency evaluation was solidified in early benchmark papers that sought a simple way to quantify prediction error.
* **Bibliographic Citation**:
> Perazzi, F., Krähenbühl, P., Pritch, Y., & Hornung, A. (2012). "Saliency filters: Contrast based filtering for salient region detection." In *Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)*, pp. 733-740.
---
### 2. Maximum F-measure (Max Fβ)
* **What it Measures**: The F-measure is the harmonic mean of Precision and Recall, providing a score that balances the two. In saliency evaluation, the continuous prediction map is converted to a binary map using a series of thresholds (from 0 to 255). The F-measure is calculated for each threshold, and the **maximum** value obtained across all thresholds is reported. This adaptive thresholding makes the metric robust to models that produce well-shaped but poorly-calibrated (e.g., generally too dark or bright) saliency maps.
* **Interpretation**:
* **Higher is better**. A score of **1** represents a perfect balance of precision and recall at the optimal threshold.
* **Relevance and Justification**: Unlike the pixel-level MAE, the F-measure is region-based. It evaluates how well the *shape* of the predicted salient region aligns with the ground truth. By finding the optimal threshold for a given prediction, it fairly assesses the quality of the saliency map's structure, forgiving issues with overall intensity. The standard beta-squared value ($β^2$) is set to **0.3** to weigh precision more heavily than recall, as proposed by the authors of the foundational paper below.
* **Bibliographic Citation**:
> Achanta, R., Hemami, S., Estrada, F., & Süsstrunk, S. (2009). "Frequency-tuned salient region detection." In *Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)*, pp. 1597-1604.
---
### 3. S-measure (Structure-measure, Sα)
* **What it Measures**: The S-measure is a more advanced metric designed to overcome the limitations of pixel- and region-based metrics by evaluating structural similarity. It combines two components: an **object-aware structural similarity** (evaluating the similarity between the predicted foreground and the ground truth object) and a **region-aware structural similarity** (evaluating the similarity between sub-regions of the maps).
* **Interpretation**:
* **Range**:.
* **Higher is better**. A score of **1** indicates a perfect structural match, meaning the prediction captures the true form of the salient object much better than a simple "blob" would.
* **Relevance and Justification**: The S-measure was introduced because previous metrics could give high scores to predictions that correctly located a salient object but failed to capture its detailed structure. This metric is less sensitive to minor pixel errors and more attuned to the geometric and structural correctness of the prediction, making it highly aligned with human visual perception.
* **Bibliographic Citation**:
> Fan, D. P., Cheng, M. M., Liu, Y., Li, T., & Borji, A. (2017). "Structure-measure: A new way to evaluate saliency maps." In *Proceedings of the IEEE international conference on computer vision (ICCV)*, pp. 4548-4557.
---
### 4. E-measure (Enhanced-alignment measure, Eξ)
* **What it Measures**: The E-measure simultaneously captures both image-level statistics (the global mean) and local pixel-matching information. It computes an "enhanced alignment matrix" that quantifies the relationship between the prediction and ground truth, considering both their values and their statistical context.
* **Interpretation**:
* **Range**:.
* **Higher is better**. A high score indicates strong alignment in both the fine-grained pixel details and the overall statistical distribution of saliency values.
* **Relevance and Justification**: The E-measure provides a more holistic evaluation than F-measure or MAE. It was designed to be robust and to capture a wider range of similarities and differences that might be missed by other metrics. Its ability to combine local and global aspects makes it a powerful tool for comprehensively assessing a model's performance.
* **Bibliographic Citation**:
> Fan, D. P., Gong, C., Cao, Y., Ren, B., Cheng, M. M., & Borji, A. (2018). "Enhanced-alignment measure for binary foreground map evaluation." In *Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI)*, pp. 698-704.
---
### 5. Weighted F-measure (Fβw)
* **What it Measures**: This is a modification of the F-measure that introduces a weighting scheme to address a key flaw: not all pixels are equally important. It assigns higher importance to pixels near the center of a salient region and less importance to those near the boundary. Errors made on perceptually important pixels are therefore penalized more heavily.
* **Interpretation**:
* **Range**:.
* **Higher is better**. A high score indicates high precision and recall, especially in the most critical areas of the salient object.
* **Relevance and Justification**: The motivation behind the Weighted F-measure is to create a metric that better correlates with human perception of error. A small error in the middle of an object is often more jarring than an error along its noisy edge. By using a weighting map (often based on centrality or distance from the boundary), this metric provides a more perceptually accurate evaluation of a saliency map's quality.
* **Bibliographic Citation**:
> Margolin, R., Zelnik-Manor, L., & Tal, A. (2014). "How to evaluate foreground maps?" In *Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)*, pp. 248-255.
### Summary Table
| Metric | What it Measures | Range | Goal | Key Strength |
| :--- | :--- | :--- | :--- | :--- |
| **MAE** | Average pixel-wise error | | **Minimize** | Simple, direct error measurement. |
| **Max F-measure** | Optimal balance of precision and recall | | **Maximize** | Robust to intensity calibration issues; region-based. |
| **S-measure** | Structural and object-level similarity | | **Maximize** | Evaluates the geometric correctness of the prediction. |
| **E-measure** | Global and local pixel alignment | | **Maximize** | Provides a holistic score combining local and global stats. |
| **Weighted F-measure** | Perceptually weighted precision and recall | | **Maximize** | Aligns better with human perception by weighting errors. |
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

File diff suppressed because one or more lines are too long
+7 -2
View File
@@ -2,14 +2,18 @@
name = "image-misc"
description = """
Miscellaneous nodes for image manipulation.
Currently just download image with bypass, so you can create workflows including image examples.
Download image with bypass, so you can create workflows including image examples.
Foreground estimation and mask apply.
Face composition nodes (to reinsert a processed face).
Advanced image/mask pad/resize (from KJNodes with some additions).
Normalization nodes for low level image pre-processing.
No extra dependencies, just an internal module.
"""
# This is the modern Python mechanism, but not supported
# dynamic = ["version"]
# license = "GPL-3.0-or-later"
# Only this is supported :-(
version = "1.2.0"
version = "1.4.0"
license = { file = "LICENSE" }
# The following is just ignored
dependencies = ["seconohe"]
@@ -20,6 +24,7 @@ Repository = "https://github.com/set-soft/ComfyUI-ImageMisc"
[tool.comfy]
PublisherId = "set-soft"
DisplayName = "Image Misc"
Icon = "https://raw.githubusercontent.com/set-soft/ComfyUI-ImageMisc/main/assets/Icon_400x400.jpg"
[tool.setuptools.dynamic]
version = {attr = "nodes.__version__"}
+2 -1
View File
@@ -1,3 +1,4 @@
Pillow
numpy
seconohe
scipy
seconohe >= 1.0.7
+5 -1
View File
@@ -4,6 +4,10 @@
# Project: ComfyUI-ImageMisc
from seconohe.logger import initialize_logger
__version__ = "1.2.0"
__version__ = "1.4.0"
NODES_NAME = "ImageMisc"
main_logger = initialize_logger(NODES_NAME)
# Epsilon: small value to avoid "divide by 0" errors
EPS = 1e-8
F_POINTS = 255
+123
View File
@@ -0,0 +1,123 @@
import torch
from typing import Tuple
from . import EPS, F_POINTS
def get_e_measure(
pred: torch.Tensor,
gt: torch.Tensor,
num_thresholds: int = F_POINTS,
chunk_size: int = 16
) -> Tuple[float, float, float, torch.Tensor, torch.Tensor]:
"""
Calculates the E-measure scores using a memory-efficient chunking strategy.
This implementation is fully vectorized within chunks to maintain high performance
while ensuring that memory usage remains low and predictable, making it suitable
for high-resolution images.
Args:
pred (torch.Tensor): The continuous prediction mask (normalized to [0, 1]).
gt (torch.Tensor): The binary ground truth mask (values are 0 or 1).
num_thresholds (int): The number of thresholds to evaluate. Defaults to F_POINTS.
chunk_size (int): The number of thresholds to process in a single batch.
Lower this value if you encounter VRAM issues. Defaults to 16.
Returns:
A tuple containing:
- float: The mean E-measure score across all thresholds.
- float: The maximum E-measure score across all thresholds.
- float: The adaptive E-measure score.
- torch.Tensor: A 1D tensor with the E-measure score for each threshold.
- torch.Tensor: A 1D tensor with the thresholds.
"""
# 1. --- Calculate scores for all thresholds using chunking ---
# Create a 1D tensor of thresholds from 0 to almost 1
thlist = torch.linspace(0, 1 - 1e-10, num_thresholds, device=pred.device)
all_scores = []
# Process thresholds in memory-efficient chunks
for i in range(0, num_thresholds, chunk_size):
# Get the current chunk of thresholds
th_chunk = thlist[i:i + chunk_size]
# Vectorized operation on the smaller chunk.
# Reshape pred to (1, H, W) and thlist to (N, 1, 1).
# Broadcasting (>=) creates a binarized prediction for each threshold.
# The result `binarized_preds` has a shape of (chunk_size, H, W).
binarized_preds_chunk = (pred.unsqueeze(0) >= th_chunk.view(-1, 1, 1)).to(pred.dtype)
# Calculate scores for the current chunk
scores_chunk = e_calculate_enhanced_scores(binarized_preds_chunk, gt)
all_scores.append(scores_chunk)
# Combine the scores from all chunks
scores = torch.cat(all_scores)
# 2. --- Calculate the single adaptive score ---
# Calculate the adaptive threshold, clamping at 1.0 to be safe.
adaptive_th = torch.clamp(2 * pred.mean(), max=1.0)
# Binarize the prediction with this single threshold
adaptive_pred_binarized = (pred >= adaptive_th).to(pred.dtype)
# Reuse the same helper function by adding a temporary batch dimension
adaptive_score_tensor = e_calculate_enhanced_scores(adaptive_pred_binarized.unsqueeze(0), gt)
# 3. --- Return the final results ---
return (
scores.mean().item(),
scores.max().item(),
adaptive_score_tensor.item(),
scores.cpu(), thlist
)
def e_calculate_enhanced_scores(binarized_preds: torch.Tensor, gt: torch.Tensor) -> torch.Tensor:
"""
Helper function to compute E-measure scores for a batch of binarized predictions.
This function is fully vectorized.
Args:
binarized_preds (torch.Tensor): A tensor of binarized predictions, with shape
(N, H, W), where N is the number of thresholds.
gt (torch.Tensor): The single binary ground truth mask, with shape (H, W).
Returns:
torch.Tensor: A 1D tensor of shape (N,) containing the E-measure score
for each binarized prediction.
"""
# Handle the edge case where the ground truth is all black
if torch.mean(gt) == 0.0:
# The score is based on how much of the prediction is also black.
# Original formula: sum(1 - y_pred_th) / (numel - 1)
# This is equivalent to numel * (1 - mean) / (numel - 1)
enhanced = 1 - binarized_preds
# Handle the edge case where the ground truth is all white
elif torch.mean(gt) == 1.0:
# The score is based on how much of the prediction is also white.
# Original formula: sum(y_pred_th) / (numel - 1)
enhanced = binarized_preds
# Normal case with a mixed ground truth
else:
# Demean the ground truth. `gt_demeaned` has shape (H, W).
gt_demeaned = gt - gt.mean()
# Demean the binarized predictions.
# `mean` is calculated over spatial dims (H, W), keeping the threshold dim.
# `fm` (foreground map) will have shape (N, H, W).
fm = binarized_preds - binarized_preds.mean(dim=[-2, -1], keepdim=True)
# The demeaned GT will be broadcasted to match the shape of `fm`.
align_matrix = 2 * gt_demeaned * fm / (gt_demeaned.square() + fm.square() + EPS)
enhanced = (align_matrix + 1).square() / 4
# Calculate the final score for each threshold by summing over the spatial dimensions.
# The denominator (y.numel() - 1) is a quirk from the original paper's code.
scores = torch.sum(enhanced, dim=[-2, -1]) / (gt.numel() - 1 + EPS)
return scores
+195
View File
@@ -0,0 +1,195 @@
import numpy as np
import scipy
import torch
from typing import Tuple
from . import EPS, F_POINTS
def _f_measure(pred: torch.Tensor, gt: torch.Tensor, threshold: torch.Tensor, beta2: float = 0.3) -> float:
# Binarize the continuous prediction map using the current threshold.
# Pixels >= threshold become 1.0 (positive), and others become 0.0 (negative).
pred_binary = (pred >= threshold).float()
# Calculate True Positives (TP): pixels that are positive in both the prediction and ground truth.
# Element-wise multiplication results in 1 only where both are 1.
tp = (pred_binary * gt).sum()
# Optimization: If there are no true positives, the F-measure will be 0.
# We can skip the rest of the calculations for this threshold.
if tp == 0:
return 0.0, 0.0, 0.0
# Calculate Precision = TP / (TP + FP).
# The sum of `pred_binary` gives the total number of predicted positives (TP + FP).
precision = tp / (pred_binary.sum() + EPS)
# Calculate Recall = TP / (TP + FN).
# The sum of `gt` gives the total number of actual positives (TP + FN).
recall = tp / (gt.sum() + EPS)
# Calculate the F-beta score using the computed precision and recall.
# The beta^2=0.3 value is standard in saliency detection literature.
f_beta = (1 + beta2) * precision * recall / (beta2 * precision + recall + EPS)
return f_beta.item(), precision.item(), recall.item()
def get_f_measure(pred: torch.Tensor, gt: torch.Tensor, beta2: float = 0.3) -> Tuple[float, Tuple[float, float], float]:
"""
Calculates the maximum F-measure score for a continuous prediction against a binary ground truth.
The F-measure evaluates the balance between precision and recall. Since the prediction
is a continuous map (0.0 to 1.0), this function iterates through F_POINTS possible
thresholds to binarize the prediction. It calculates the F-measure for each
threshold and returns the highest (max) score found. This provides a fair
evaluation of the prediction's structural quality, independent of its overall brightness.
Args:
pred (torch.Tensor): The continuous prediction mask (normalized to [0, 1]).
gt (torch.Tensor): The binary ground truth mask (values are 0 or 1).
beta2 (float): The beta-squared value for the F-measure. The standard value of 0.3
is used to weigh precision more heavily than recall. Defaults to 0.3.
Returns:
float: The maximum F-measure score found across all thresholds.
tuple: The F-measure for all the 255 thresholds we tried, (threshold, F)
float: The F-measure for an adaptative threshold
"""
# Initialize f_max to store the highest F-measure score found so far.
f_max = 0.0
f4 = torch.Tensor(size=(4, F_POINTS), device="cpu")
f = f4.movedim(1, 0) # Easier here, not for computation
# Iterate through F_POINTS evenly spaced thresholds from 0.0 to 1.0.
# This corresponds to testing every possible 8-bit grayscale value as the cutoff.
# The thresholds tensor is created on the same device as the input for efficiency.
for c, threshold in enumerate(torch.linspace(0, 1, F_POINTS, device=gt.device)):
f_beta, prec, rec = _f_measure(pred, gt, threshold, beta2)
f[c] = torch.Tensor((threshold.item(), f_beta, prec, rec))
# Update f_max if the F-beta score for the current threshold is the highest yet.
# .item() extracts the single float value from the 0-dimensional tensor.
if f_beta > f_max:
f_max = f_beta
# Also compute F for an "adaptative" threshold
threshold = min(pred.mean() * 2, 1)
f_adp, _, _ = _f_measure(pred, gt, threshold, beta2)
# After checking all thresholds, return the maximum score found.
return f_max, f4, f_adp
def get_weighted_f_measure(pred: torch.Tensor, gt: torch.Tensor, beta2: float = 0.3) -> float:
"""
Calculates the true research-grade Weighted F-measure (F_beta^w).
This is a faithful and optimized port of the reference algorithm from the
"How to Evaluate Foreground Maps?" paper, ensuring verifiable results.
The implementation performs pre-checks on the GPU for efficiency before
transferring data to the CPU for SciPy-based calculations.
Args:
pred (torch.Tensor): The continuous prediction mask (normalized to [0, 1]).
gt (torch.Tensor): The binary ground truth mask (values are 0 or 1).
beta2 (float): The beta-squared value for the F-measure. Defaults to 0.3.
Returns:
float: The final Weighted F-measure score.
"""
# --- Step 1: GPU-side Pre-checks for Efficiency ---
# Handle the edge case of an all-black ground truth on the GPU.
# If the ground truth is empty, the score is 1 minus the mean of the prediction.
# A perfect prediction (all black) would yield a score of 1.
# This avoids the expensive CPU transfer and SciPy calculations entirely.
if torch.mean(gt) == 0.0:
return (1.0 - pred.mean()).item()
# --- Step 2: Data Transfer to CPU for NumPy/SciPy Processing ---
# Move tensors to the CPU and convert to NumPy arrays. SciPy functions require this.
# Squeezing removes any singleton channel dimensions (e.g., from [1, H, W] to [H, W]).
gt_np = gt.squeeze().cpu().numpy()
pred_np = pred.squeeze().cpu().numpy()
# --- Step 3: Creation of Core Error and Dependency Maps ---
# Create boolean masks for foreground (gt_mask) and background (not_gt_mask) regions.
# np.isclose is used for safe floating-point comparison.
gt_mask = np.isclose(gt_np, 1)
not_gt_mask = np.logical_not(gt_mask)
# Calculate the initial, simple absolute pixel-wise error map.
E = np.abs(pred_np - gt_np)
# Calculate the Euclidean Distance Transform on the INVERTED mask.
# For each background pixel, `dist` will be its distance to the nearest foreground pixel.
# `idx` will store the coordinates of that nearest foreground pixel. This is key for the next step.
# Note: The original implementation uses scipy.ndimage.morphology.distance_transform_edt
# which is an alias for scipy.ndimage.distance_transform_edt.
dist, idx = scipy.ndimage.morphology.distance_transform_edt(not_gt_mask, return_indices=True)
# --- Step 4: Pixel Dependency Map (Et -> EA -> min_E_EA) ---
# Create the "Pixel Dependency" map, starting with a copy of the original error.
Et = np.array(E)
# This is the crucial step for pixel dependency. For every background pixel,
# its error value is replaced with the error value of its NEAREST foreground pixel.
# This propagates the error from the foreground edges into the background.
Et[not_gt_mask] = E[idx[0, not_gt_mask], idx[1, not_gt_mask]]
# Smooth the dependency-aware error map with a Gaussian filter.
# This creates a soft, blurred error field around the object.
sigma = 5.0
EA = scipy.ndimage.gaussian_filter(Et, sigma=sigma, truncate=3 / sigma, mode='constant', cval=0.0)
# The final error for any FOREGROUND pixel is the MINIMUM of its original error (E)
# and the new smoothed, dependency-aware error (EA).
# This prevents unfairly penalizing small errors right at the boundary of a correct prediction.
# The `where=gt_mask` argument ensures this operation only applies to the foreground.
min_E_EA = np.minimum(E, EA, where=gt_mask, out=np.array(E))
# --- Step 5: Pixel Importance Map (B) ---
# Create the "Pixel Importance" map, which starts as a uniform map of ones.
B = np.ones(gt_np.shape)
# For each BACKGROUND pixel, assign an importance weight based on its distance
# from the foreground. The weight is calculated using a non-linear function
# that decreases as the distance increases. This makes errors near the object more important.
B[not_gt_mask] = 2 - np.exp(np.log(1 - 0.5) / 5 * dist[not_gt_mask])
# The final Weighted Error Map is the element-wise product of the dependency-aware
# error map and the pixel importance map.
Ew = min_E_EA * B
# --- Step 6: Final Metric Computation ---
# Get a small machine epsilon value for numerically stable division.
eps = np.spacing(1)
# Calculate Weighted True Positives (TPw) and False Positives (FPw) from the Ew map.
# TPw = Sum of foreground weights - Sum of weighted errors in the foreground.
# FPw = Sum of weighted errors in the background.
TPw = np.sum(gt_np) - np.sum(Ew[gt_mask])
FPw = np.sum(Ew[not_gt_mask])
# Calculate Weighted Recall (R) and Weighted Precision (P).
# These definitions are specific to this metric's formulation.
R = 1 - np.mean(Ew[gt_mask]) # Weighed Recall
P = TPw / (eps + TPw + FPw) # Weighted Precision
# The final Weighted F-measure (Q) is calculated using the standard formula
# with the newly computed weighted Precision and Recall.
# beta2 = 0.3 is standard, weighing precision more heavily than recall.
# Q = 2 * (R * P) / (eps + R + P) # Beta=1
Q = (1 + beta2) * (R * P) / (eps + R + (beta2 * P))
# Raise an error if the result is Not a Number (NaN), indicating a potential issue.
if np.isnan(Q):
raise ValueError("Weighted F-measure resulted in NaN")
return Q
+359
View File
@@ -0,0 +1,359 @@
import json
import numpy as np
import os
from PIL import Image, ImageOps, ImageSequence, ImageFile, UnidentifiedImageError
from PIL.PngImagePlugin import PngInfo
import torch
from . import main_logger
try:
# We need to import the built-in LoadImage class for ImageDownload
from nodes import LoadImage, LoadImageMask
from folder_paths import get_input_directory, get_output_directory
has_load_image = hasattr(LoadImage, "load_image") and hasattr(LoadImageMask, "load_image")
from comfy.utils import common_upscale
except Exception:
has_load_image = False
logger = main_logger
def empty_image(b=0, h=64, w=64, c=1):
if b == 0 and c == 1:
dims = (h, w)
elif c == 1:
dims = (b, h, w)
else:
dims = (b, h, w, c)
return torch.zeros(dims, dtype=torch.float32, device="cpu")
def upscale(image, width, height, upscale_method, crop="disabled"):
# return F.interpolate(image, size=(height, width), mode=upscale_method)
return common_upscale(image, width, height, upscale_method, crop=crop)
def upscale_comfy(image, width, height, upscale_method, crop="disabled"):
if image.dim() == 3:
# A mask
if upscale_method == "lanczos":
# Lanczos needs an RGB image
return common_upscale(image.unsqueeze(1).repeat(1, 3, 1, 1), width, height, upscale_method,
crop=crop).movedim(1, -1)[:, :, :, 0]
else:
return common_upscale(image.unsqueeze(1), width, height, upscale_method, crop=crop).squeeze(1)
return common_upscale(image.movedim(-1, 1), width, height, upscale_method, crop=crop).movedim(1, -1)
def pillow(fn, arg):
prev_value = None
try:
x = fn(arg)
except (OSError, UnidentifiedImageError, ValueError): # PIL issues #4472 and #2445, also fixes ComfyUI issue #3416
prev_value = ImageFile.LOAD_TRUNCATED_IMAGES
ImageFile.LOAD_TRUNCATED_IMAGES = True
x = fn(arg)
finally:
if prev_value is not None:
ImageFile.LOAD_TRUNCATED_IMAGES = prev_value
return x
class CustomLoadImage(object):
def load_image(self, image):
""" ComfyUI 0.3.59 loader """
image_path = image
img = pillow(Image.open, image_path)
output_images = []
output_masks = []
w, h = None, None
excluded_formats = ['MPO']
for i in ImageSequence.Iterator(img):
i = pillow(ImageOps.exif_transpose, i)
if i.mode == 'I':
i = i.point(lambda i: i * (1 / 255))
image = i.convert("RGB")
if len(output_images) == 0:
w = image.size[0]
h = image.size[1]
if image.size[0] != w or image.size[1] != h:
continue
image = np.array(image).astype(np.float32) / 255.0
image = torch.from_numpy(image)[None,]
if 'A' in i.getbands():
mask = np.array(i.getchannel('A')).astype(np.float32) / 255.0
mask = 1. - torch.from_numpy(mask)
elif i.mode == 'P' and 'transparency' in i.info:
mask = np.array(i.convert('RGBA').getchannel('A')).astype(np.float32) / 255.0
mask = 1. - torch.from_numpy(mask)
else:
mask = empty_image()
output_images.append(image)
output_masks.append(mask.unsqueeze(0))
if len(output_images) > 1 and img.format not in excluded_formats:
output_image = torch.cat(output_images, dim=0)
output_mask = torch.cat(output_masks, dim=0)
else:
output_image = output_images[0]
output_mask = output_masks[0]
return (output_image, output_mask)
class CustomLoadMask(object):
def load_image(self, image, channel):
image_path = image
i = pillow(Image.open, image_path)
i = pillow(ImageOps.exif_transpose, i)
if i.getbands() != ("R", "G", "B", "A"):
if i.mode == 'I':
i = i.point(lambda i: i * (1 / 255))
i = i.convert("RGBA")
mask = None
c = channel[0].upper()
if c in i.getbands():
mask = np.array(i.getchannel(c)).astype(np.float32) / 255.0
mask = torch.from_numpy(mask)
if c == 'A':
mask = 1. - mask
else:
mask = empty_image()
return (mask.unsqueeze(0),)
def get_image_preview_info(file_name, where="input"):
if file_name is None:
return {}
# This information is for the preview, as we are an output node and we return images
# they will be displayed in our node. Quite simple.
if os.path.isabs(file_name):
base = get_input_directory() if where == "input" else get_output_directory()
ff_name = os.path.relpath(file_name, base)
fname = os.path.basename(ff_name)
dname = os.path.dirname(ff_name)
else:
fname = file_name
dname = ""
return {"filename": fname, "subfolder": dname, "type": where}
def load_one_image(file_name, disp_name, embed_transparency):
if file_name is None:
return (empty_image(b=1, c=3), empty_image(b=1), file_name)
if not os.path.isabs(file_name):
file_name = os.path.join(get_input_directory(), file_name)
if not os.path.exists(file_name):
raise ValueError(f"File '{file_name}' not found")
try:
if has_load_image:
# --- REUSE ComfyUI's LoadImage LOGIC ---
# Instantiate the built-in LoadImage node
loader_instance = LoadImage()
# The LoadImage node's `load_image` method expects the filename as passed
# by the ComfyUI widget, which is just the filename. It internally
# resolves the path using folder_paths.
logger.debug(f"Calling built-in LoadImage.load_image() with filename: '{file_name}'")
else:
# Instantiate the built-in LoadImage node
loader_instance = CustomLoadImage()
logger.debug(f"Calling our CustomLoadImage.load_image() with filename: '{file_name}'")
# Call the method and return its result directly
result = loader_instance.load_image(file_name)
result = (result[0], result[1], file_name)
# Create an RGBA image if needed
if embed_transparency:
image, mask = result
# Expand the mask to (b, h, w, 1)
mask = mask[..., None]
# Concatenate image and mask into (b, h, w, 4)
image_with_alpha = torch.cat([image, 1.0 - mask], dim=-1)
result = (image_with_alpha, mask, file_name)
return result
except Exception as e:
logger.error(f"Failed to load image '{disp_name}': {e}", exc_info=True)
# Re-raise to make the error visible in ComfyUI
raise IOError(f"Could not load the image file '{disp_name}'. "
"It may be corrupt or in an unsupported format.") from e
def load_one_mask(file_name, disp_name, channel='red'):
if file_name is None:
return (empty_image(b=1), file_name)
if not os.path.isabs(file_name):
file_name = os.path.join(get_input_directory(), file_name)
if not os.path.exists(file_name):
raise ValueError(f"File '{file_name}' not found")
try:
if has_load_image:
loader_instance = LoadImageMask()
logger.debug(f"Calling built-in LoadImageMask.load_image() with filename: '{file_name}'")
else:
# Instantiate the built-in LoadImage node
loader_instance = CustomLoadMask()
logger.debug(f"Calling our CustomLoadMask.load_image() with filename: '{file_name}'")
# Call the method and return its result directly
result = loader_instance.load_image(file_name, channel)
return (result[0], file_name)
except Exception as e:
logger.error(f"Failed to load image '{disp_name}': {e}", exc_info=True)
# Re-raise to make the error visible in ComfyUI
raise IOError(f"Could not load the image file '{disp_name}'. "
"It may be corrupt or in an unsupported format.") from e
def load_image_wrapper(file_name, embed_transparency=False, disp_name=None, show_preview=True, channel=None):
disp_name = disp_name or file_name
if channel is not None:
result = load_one_mask(file_name, disp_name, channel)
else:
result = load_one_image(file_name, disp_name, embed_transparency)
if not show_preview:
return result
return {"ui": {"images": [get_image_preview_info(file_name)]}, "result": result}
def load_images_wrapper(file_names, embed_transparency=False, disp_names=None, show_preview=True, batch_size=1,
channel=None):
# We work with lists
if isinstance(file_names, str):
file_names = [file_names]
disp_names = disp_names or file_names
if isinstance(disp_names, str):
disp_names = [disp_names]
imgs = []
masks = []
is_mask = channel is not None
used_file_names = []
all_preview_imgs = []
total = len(file_names)
for i in range(0, total, batch_size):
if batch_size > 1:
# Add upto batch_size images in a batch
imgs_batch = []
masks_batch = []
max_w = max_h = 0
max_mw = max_mh = 0
for j in range(batch_size):
index = i+j
if index >= total:
continue
file_name = file_names[i+j]
disp_name = disp_names[i+j]
if is_mask:
# A mask
mask, file_name = load_one_mask(file_name, disp_name, channel)
else:
img, mask, file_name = load_one_image(file_name, disp_name, embed_transparency)
max_w = max(max_w, img.shape[2])
max_h = max(max_h, img.shape[1])
imgs_batch.append(img)
max_mw = max(max_mw, mask.shape[2])
max_mh = max(max_mh, mask.shape[1])
masks_batch.append(mask)
used_file_names.append(file_name)
if show_preview:
all_preview_imgs.append(get_image_preview_info(file_name))
for j in range(len(imgs_batch)):
if channel is None:
img = imgs_batch[j]
H, W = img.shape[1:3]
if H != max_h or W != max_w:
logger.debug(f"Upscaling image to fit batch: {W}x{H} -> {max_w}x{max_h}")
imgs_batch[j] = upscale_comfy(img, max_w, max_h, "bicubic")
mask = masks_batch[j]
H, W = mask.shape[1:3]
if H != max_mh or W != max_mw:
logger.debug(f"Upscaling mask to fit batch: {W}x{H} -> {max_mw}x{max_mh}")
masks_batch[j] = upscale_comfy(mask, max_mw, max_mh, "bicubic")
if channel is None:
imgs.append(torch.cat(imgs_batch))
masks.append(torch.cat(masks_batch))
else:
# Add a single image
file_name = file_names[i]
if is_mask:
# A mask
mask, file_name = load_one_mask(file_name, disp_names[i], channel)
else:
img, mask, file_name = load_one_image(file_name, disp_names[i], embed_transparency)
imgs.append(img)
masks.append(mask)
used_file_names.append(file_name)
if show_preview:
all_preview_imgs.append(get_image_preview_info(file_name))
logger.debug(f"Loaded {len(masks)} batches:")
for n, i in enumerate(masks):
logger.debug(f"{n}) {i.shape}")
result = (imgs, masks, used_file_names) if channel is None else (masks, used_file_names)
if not show_preview:
return result
return {"ui": {"images": all_preview_imgs}, "result": result}
def save_image(images, filenames, prompt=None, extra_pnginfo=None, compress_level=4, show_preview=True):
if isinstance(filenames, str):
if not filenames:
raise ValueError("You must provide a file name")
filenames = [filenames]
# Make a list with all the images, unroll the batches
imgs = []
for img in images:
imgs.extend([i for i in img])
if len(filenames) != len(imgs):
raise ValueError(f"{len(imgs)} images provided but only {len(filenames)} file names")
all_preview_imgs = []
for index, (image, filename) in enumerate(zip(imgs, filenames)):
i = 255. * image.cpu().numpy()
img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
metadata = PngInfo()
if prompt is not None:
metadata.add_text("prompt", json.dumps(prompt))
if extra_pnginfo is not None:
for x in extra_pnginfo:
metadata.add_text(x, json.dumps(extra_pnginfo[x]))
logger.debug(f"Saving {image.shape[1]}x{image.shape[0]} image to {filename}")
img.save(filename, pnginfo=metadata, compress_level=compress_level)
if show_preview:
all_preview_imgs.append(get_image_preview_info(filename, where="output"))
if not show_preview:
return ()
return {"ui": {"images": all_preview_imgs}, "result": ()}
+2659 -197
View File
File diff suppressed because it is too large Load Diff
+195
View File
@@ -0,0 +1,195 @@
import torch
from . import EPS
def get_s_measure(pred: torch.Tensor, gt: torch.Tensor, alpha: float = 0.5) -> float:
"""
Calculates the S-measure for a given prediction and ground truth mask.
The S-measure evaluates structural similarity, combining object-aware and
region-aware metrics.
Args:
pred (torch.Tensor): The continuous prediction mask (normalized to [0, 1]).
gt (torch.Tensor): The binary ground truth mask (values are 0 or 1).
alpha (float): The weight for balancing object-aware vs. region-aware scores.
Defaults to 0.5.
Returns:
float: The final S-measure score.
"""
# gt is assumed to be binary
y = gt.mean()
if y == 0:
# If the ground truth is all black, the score is the inverse of the prediction's average.
# A perfect prediction would also be all black (mean=0), yielding a score of 1.
x = pred.mean()
q = 1.0 - x
elif y == 1:
# If the ground truth is all white, the score is simply the prediction's average.
# A perfect prediction would be all white (mean=1), yielding a score of 1.
x = pred.mean()
q = x
else:
# For a mixed ground truth, balance the object and region scores.
q = alpha * s_object(pred, gt) + (1 - alpha) * s_region(pred, gt)
# Ensure the score is non-negative.
if q < 0:
return 0
return q.item()
def s_object(pred: torch.Tensor, gt: torch.Tensor) -> torch.Tensor:
"""
Calculates the object-aware structural similarity score.
This measures the similarity for foreground and background regions separately
and combines them based on the foreground's size.
"""
# Create a map of foreground-only predictions
fg = torch.where(gt == 0, torch.zeros_like(pred), pred)
# Create a map of background-only predictions (inverted)
bg = torch.where(gt == 1, torch.zeros_like(pred), 1 - pred)
# Calculate scores for each region
o_fg = s_object_calc(fg, gt)
o_bg = s_object_calc(bg, 1 - gt)
# Combine scores based on foreground area
u = gt.mean() # The ratio of foreground pixels
q = u * o_fg + (1 - u) * o_bg
return q
def s_object_calc(pred: torch.Tensor, gt: torch.Tensor) -> torch.Tensor:
"""
Helper function to compute the similarity score for a given region (FG or BG).
"""
# Select the prediction pixels corresponding to the region of interest in the ground truth
region_pixels = pred[gt == 1]
# If the region is empty, the score is undefined, but 0 is a safe return.
if region_pixels.numel() == 0:
return torch.tensor(0.0, device=pred.device)
x = region_pixels.mean()
# Note: The original paper's formula uses variance (sigma**2), but many public
# implementations use std dev (sigma). We follow this implementation's logic.
sigma_x = region_pixels.std()
# A score that rewards high mean (x) but penalizes high variance (sigma_x).
# It is maximized when x=1 and sigma_x=0.
score = 2.0 * x / (x**2 + 1.0 + sigma_x + EPS)
return score
def s_region(pred: torch.Tensor, gt: torch.Tensor) -> torch.Tensor:
"""
Calculates the region-aware structural similarity score.
This divides the image into four quadrants based on the ground truth's
centroid and computes a weighted SSIM score.
"""
# Find the centroid (center of mass) of the ground truth mask
X, Y = s_centroid(gt)
# Divide the ground truth and prediction into four quadrants based on the centroid
gt_parts = s_divide_tensor(gt, X, Y)
pred_parts = s_divide_tensor(pred, X, Y)
# Calculate the area weights for each quadrant
w = s_calculate_weights(gt.shape, X, Y)
# Calculate SSIM for each quadrant and combine the quadrant scores using their area weights
return sum([w[i] * s_ssim(pred_parts[i], gt_parts[i]) for i in range(4)])
def s_centroid(gt: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
"""
Calculates the centroid (center of mass) of a binary mask.
"""
rows, cols = gt.shape[-2:]
total = gt.sum()
# Handle the edge case of an all-black mask
if total == 0:
X = torch.tensor(round(cols / 2), device=gt.device)
Y = torch.tensor(round(rows / 2), device=gt.device)
else:
# Create coordinate ranges directly on the target device, avoiding CPU-GPU transfer.
i_coords = torch.arange(cols, device=gt.device, dtype=gt.dtype)
j_coords = torch.arange(rows, device=gt.device, dtype=gt.dtype)
# Calculate weighted average of coordinates
X = torch.round((gt.sum(dim=0) * i_coords).sum() / total)
Y = torch.round((gt.sum(dim=1) * j_coords).sum() / total)
return X.long(), Y.long()
def s_divide_tensor(tensor: torch.Tensor, X: torch.Tensor, Y: torch.Tensor) -> tuple:
"""
Divides a tensor into four quadrants based on a pivot point (X, Y).
This is memory-efficient as it returns views, not copies.
"""
h, w = tensor.shape[-2:]
LT = tensor[..., :Y, :X]
RT = tensor[..., :Y, X:]
LB = tensor[..., Y:, :X]
RB = tensor[..., Y:, X:]
return LT, RT, LB, RB
def s_calculate_weights(shape: tuple, X: torch.Tensor, Y: torch.Tensor) -> tuple:
"""Calculates the proportional area of the four quadrants."""
h, w = shape[-2:]
area = h * w
# Ensure coordinates are float for division
Xf = X.float()
Yf = Y.float()
w1 = Xf * Yf / area
w2 = (w - Xf) * Yf / area
w3 = Xf * (h - Yf) / area
w4 = 1.0 - w1 - w2 - w3 # More stable calculation for the last weight
return (w1, w2, w3, w4)
def s_ssim(pred: torch.Tensor, gt: torch.Tensor) -> float:
"""
Computes a custom structural similarity (SSIM-like) score between two tensors.
"""
# If a quadrant is empty, its contribution to similarity is ambiguous.
# Returning 0 is a safe choice, but 1 could also be argued if gt is also empty.
if pred.numel() == 0 or gt.numel() == 0:
return 0.0
h, w = pred.shape[-2:]
N = h * w
# Means
x = pred.mean()
y = gt.mean()
# Variances and Covariance (using unbiased estimator N-1)
# This is numerically safer than calculating std dev separately.
sigma_x2 = ((pred - x) * (pred - x)).sum() / (N - 1 + EPS)
sigma_y2 = ((gt - y) * (gt - y)).sum() / (N - 1 + EPS)
sigma_xy = ((pred - x) * (gt - y)).sum() / (N - 1 + EPS)
# Numerator and denominator of the SSIM formula
alpha = 4 * x * y * sigma_xy
beta = (x*x + y*y) * (sigma_x2 + sigma_y2)
# Handle special cases for stability, as defined in the original code
if alpha != 0:
Q = alpha / (beta + EPS)
elif alpha == 0 and beta == 0:
# If both inputs are flat and identical, similarity is perfect.
Q = 1.0
else:
# If numerator is 0 but denominator isn't, similarity is 0.
Q = 0
return Q