diff --git a/docs/E2E_TEST_STRATEGY.md b/docs/E2E_TEST_STRATEGY.md new file mode 100644 index 0000000..327728a --- /dev/null +++ b/docs/E2E_TEST_STRATEGY.md @@ -0,0 +1,433 @@ +# E2E Test Strategy + +End-to-end test strategy for ComfyUI Impact Pack, covering isolated server launch, baseline smoke verification, real workflow execution, and operational notes for reliable test runs. + +--- + +## Table of Contents + +1. [Purpose & Scope](#purpose--scope) +2. [Prerequisites](#prerequisites) +3. [Server Launch](#server-launch) +4. [Baseline Smoke Test](#baseline-smoke-test) +5. [Workflow Execution E2E](#workflow-execution-e2e) +6. [Verification Criteria](#verification-criteria) +7. [Operational Notes](#operational-notes) +8. [Extension Points](#extension-points) + +--- + +## Purpose & Scope + +End-to-end (E2E) testing validates ComfyUI Impact Pack as it behaves in a real ComfyUI runtime, not only through unit-level assertions. The goal is to catch regressions that surface only when the full stack is loaded: server startup, node registration, REST API responses, frontend page loading, and actual workflow execution through the `/prompt` API. + +**Why isolated E2E** +- A developer's ComfyUI installation typically has many custom nodes. Any one of them can fail to import, log warnings, bind frontend routes, or shadow Impact Pack behavior. +- Running Impact Pack against a clean environment removes noise and produces reproducible results. +- `--disable-all-custom-nodes` combined with `--whitelist-custom-nodes` isolates Impact Pack (and its companion subpack) from every other installed custom node while still letting them load normally. + +**What this strategy validates** +- ComfyUI server starts successfully with Impact Pack and Impact Subpack loaded. +- The frontend page is served and renders the ComfyUI application shell. +- The `/object_info` REST endpoint returns the full node catalog for both packs. +- A baseline count and a sample of expected node names are present, guarding against silent node-registration regressions. +- Detailer workflows execute through the `/prompt` API and produce non-degenerate outputs, proving the full inference path (detector → SEGSDetailer → SEGSPaste) runs end to end. + +**Out of scope** (covered by [Extension Points](#extension-points)) +- UI-driven node creation and connection +- Visual regression testing beyond pixel-delta sanity checks +- Performance benchmarking + +--- + +## Prerequisites + +### Environment + +| Component | Requirement | +|-----------|-------------| +| Python | >= 3.12 | +| Playwright (Python) | >= 1.58.0 | +| Chromium runtime | Installed via `playwright install chromium` | +| ComfyUI repository | Checked out at the parent directory of `custom_nodes/comfyui-impact-pack` | +| Impact Pack | Installed as a custom node (this repository) | +| Impact Subpack | Installed as a custom node alongside Impact Pack (delivers Ultralytics / SAM node types) | + +### Directory Layout + +E2E tests assume the standard ComfyUI custom-node layout: + +``` +ComfyUI/ +├── main.py +├── custom_nodes/ +│ ├── comfyui-impact-pack/ ← this repository +│ └── comfyui-impact-subpack/ ← detector / SAM node provider +└── ... +``` + +### Install Commands + +```bash +# From the ComfyUI repository root +pip install playwright +playwright install chromium +``` + +Impact Pack's own Python dependencies are expected to be installed already (see `pyproject.toml` / `requirements.txt`). Impact Subpack contributes its own dependency list — install it with its documented procedure. + +### Model Assets + +Detailer-dependent workflows require model files on disk. Place them at the ComfyUI-relative paths below: + +| Path | Purpose | +|------|---------| +| `models/checkpoints/SD1.5/realcartoonPixar_v8.safetensors` | SD1.5 checkpoint used by `CheckpointLoaderSimple` (any compatible SD1.5 checkpoint works; adjust the workflow's `ckpt_name` accordingly) | +| `models/ultralytics/bbox/face_yolov8m.pt` | Face bbox detector used by `UltralyticsDetectorProvider` | +| `models/sams/sam_vit_b_01ec64.pth` | SAM weights used when a workflow includes `SAMLoader` | +| `input/ComfyUI_00156_.png` | Portrait with a clearly visible face, used by `LoadImage` in the reference workflow | + +The Ultralytics and SAM node classes themselves ship with Impact Subpack — installing the subpack is the delivery vehicle for those node types, separate from the model weights above. + +Pure [baseline smoke](#baseline-smoke-test) testing does not need any of these assets; they are required only once a workflow hits a detector or loader node. + +--- + +## Server Launch + +### Default Launch + +Launch an isolated ComfyUI instance with Impact Pack and Impact Subpack as the only active custom nodes. This is the default for every test beyond the pure smoke layer: + +```bash +# Working directory: the ComfyUI repository root (parent of custom_nodes/) +python main.py \ + --disable-all-custom-nodes \ + --whitelist-custom-nodes comfyui-impact-pack comfyui-impact-subpack \ + --port 18188 +``` + +Most detailer workflows depend on `UltralyticsDetectorProvider`, `SAMLoader`, and related detector/segmenter node types shipped by Impact Subpack. Running without the subpack leaves those node classes unregistered, and any workflow referencing them will fail at prompt validation. + +### Minimal Launch (smoke only) + +For the pure API smoke path — page load + `/object_info` inspection, no workflow execution — Impact Pack alone is sufficient: + +```bash +python main.py \ + --disable-all-custom-nodes \ + --whitelist-custom-nodes comfyui-impact-pack \ + --port 18188 +``` + +This minimal launch is **insufficient for detection-dependent tests**. Use it only when the test consists of startup + `/object_info` inspection. + +### Flag Explanation + +| Flag | Purpose | +|------|---------| +| `--disable-all-custom-nodes` | Skip import of every custom node in `custom_nodes/`. Eliminates side effects from unrelated packs. | +| `--whitelist-custom-nodes comfyui-impact-pack comfyui-impact-subpack` | Re-enable only the listed folder names. Accepts multiple values separated by spaces. | +| `--port 18188` | Bind to a non-default port so the test server does not collide with a developer's regular ComfyUI instance on 8188. | + +### Expected Startup Log Markers + +After launch, the server log should include lines similar to: + +``` +### Loading: ComfyUI-Impact-Pack (V8.28.2) +### Loading: ComfyUI-Impact-Subpack (V) +Skipping due to disable_all_custom_nodes and whitelist_custom_nodes +... +To see the GUI go to: http://127.0.0.1:18188 +``` + +The `Skipping ...` lines confirm isolation: other custom nodes are present on disk but were not loaded. The final `To see the GUI ...` line confirms the server is ready to accept connections. + +--- + +## Baseline Smoke Test + +The smoke test confirms that an isolated server serves both the frontend page and the node catalog. It is intentionally small and has no external dependencies beyond Playwright. + +### Script + +```python +# e2e_smoke.py +# Usage: python e2e_smoke.py +# Preconditions: +# - ComfyUI is running at http://127.0.0.1:18188 with Impact Pack AND +# Impact Subpack whitelisted (default launch). +# - Playwright Python + chromium runtime are installed. + +from playwright.sync_api import sync_playwright + +BASE_URL = "http://127.0.0.1:18188" +REQUIRED_SUBPACK_NODES = { + "UltralyticsDetectorProvider", + "SAMLoader", + "SAMDetectorCombined", + "SAMDetectorSegmented", +} + + +def main() -> None: + with sync_playwright() as p: + browser = p.chromium.launch(headless=True) + page = browser.new_context().new_page() + + # 1. Frontend page loads + page.goto(f"{BASE_URL}/", wait_until="networkidle") + title = page.title() + assert "ComfyUI" in title, f"Unexpected page title: {title!r}" + + # 2. /object_info returns the node catalog + resp = page.request.get(f"{BASE_URL}/object_info") + assert resp.status == 200, f"/object_info HTTP {resp.status}" + object_info = resp.json() + + # 3. Impact Pack nodes are registered + impact_nodes = [name for name in object_info if name.startswith("Impact")] + assert len(impact_nodes) >= 85, ( + f"Impact node count regression: got {len(impact_nodes)}, expected >= 85" + ) + + # 4. Impact Subpack nodes are registered + missing = REQUIRED_SUBPACK_NODES - set(object_info.keys()) + assert not missing, f"Subpack nodes missing: {sorted(missing)}" + + print(f"Title: {title}") + print(f"Total nodes: {len(object_info)}") + print(f"Impact nodes: {len(impact_nodes)}") + print(f"Sample: {impact_nodes[:3]}") + + browser.close() + + +if __name__ == "__main__": + main() +``` + +### Expected Output + +Observed baseline against Impact Pack v8.28.2 + Impact Subpack on an isolated server (default launch with both packs whitelisted): + +``` +Title: *Unsaved Workflow - ComfyUI +Total nodes: 858 +Impact nodes: 87 +Sample: ['ImpactSegsAndMask', 'ImpactSegsAndMaskForEach', 'ImpactFlattenMask'] +``` + +Additional Subpack-contributed node names verified present: `UltralyticsDetectorProvider`, `SAMLoader`, `SAMDetectorCombined`, `SAMDetectorSegmented`. + +Exact values will drift as the codebase evolves; the assertions above validate the minimum contract, not literal equality. + +--- + +## Workflow Execution E2E + +The baseline smoke test confirms node registration, but it does not exercise execution logic. Workflow execution E2E posts a concrete graph to `/prompt`, polls `/history/{prompt_id}` until completion, and downloads output artifacts from `/view` for inspection. This catches regressions in the inference path that are invisible to catalog-level checks. + +### API Pattern + +``` +POST /prompt — submit {prompt, client_id}, receive {prompt_id} +GET /queue — running + pending prompts (progress monitoring) +GET /history/{prompt_id} — status + outputs once execution completes +GET /view?filename=... — download an individual output artifact (PNG, etc.) +``` + +Polling cadence of ~3s on `/history` is sufficient; the endpoint returns an empty object while the prompt is still in flight and populates fully on completion. + +### Flat Prompt Format + +The `/prompt` endpoint expects a **flat** graph: a dict keyed by node ID, where each value is `{class_type, inputs}`. Socket connections are expressed as two-element lists `[upstream_node_id, output_slot_index]`. + +```python +PROMPT = { + "ckpt": { + "class_type": "CheckpointLoaderSimple", + "inputs": {"ckpt_name": "SD1.5/realcartoonPixar_v8.safetensors"}, + }, + "pos": { + "class_type": "CLIPTextEncode", + "inputs": {"clip": ["ckpt", 1], "text": "a detailed face, sharp focus"}, + }, + "img": { + "class_type": "LoadImage", + "inputs": {"image": "ComfyUI_00156_.png"}, + }, + "detector": { + "class_type": "UltralyticsDetectorProvider", + "inputs": {"model_name": "bbox/face_yolov8m.pt"}, + }, + "detail": { + "class_type": "SEGSDetailer", + "inputs": { + "image": ["img", 0], + "segs": ["bbox_segs", 0], + # ... sampler knobs elided ... + "noise_mask_feather": 20, # non-zero triggers DifferentialDiffusion path + }, + }, +} +``` + +### Pitfall: Subgraph Blueprints + +Workflow JSON exported from the ComfyUI UI may contain high-level blueprint nodes such as `workflow/Impact::MAKE_BASIC_PIPE`. These are template/subgraph references that the UI expands client-side; they are not valid `class_type` values for direct `/prompt` submission. + +For programmatic E2E, **flatten blueprints into their concrete constituent nodes** before posting. For example, a `MAKE_BASIC_PIPE` blueprint flattens into: + +``` +CheckpointLoaderSimple → (model, clip, vae) +CLIPTextEncode (positive prompt) +CLIPTextEncode (negative prompt) +ToBasicPipe (model, clip, vae, positive, negative) +``` + +The reference implementation below demonstrates this flattening. + +### Reference Implementation + +`tests/e2e_dd_compat.py` is a validated reference workflow covering the critical detailer path: + +``` +LoadImage + → UltralyticsDetectorProvider + → BboxDetectorSEGS + → SEGSDetailer(noise_mask_feather=20) # activates DifferentialDiffusion compat + → SEGSPaste + → PreviewImage (paste output) + → PreviewImage (untouched input) +``` + +Pass criteria: +- Submission returns HTTP 200 with a `prompt_id`. +- `/history/{prompt_id}` eventually reports `status.status_str == "success"` with no `execution_error` messages. +- Both expected `PreviewImage` outputs are present in `history[prompt_id].outputs`. +- Input preview and paste preview have matching dimensions. +- Paste preview has non-degenerate statistics (`std >= 1.0`). +- Paste preview differs from input (`abs(mean_delta) >= 0.005` or `abs(std_delta) >= 0.005`) — equality would indicate the detailer path, and therefore the DifferentialDiffusion compat shim, was bypassed. + +Typical observed deltas for the reference image: `mean_delta ≈ 0.01`, `std_delta ≈ 0.03`. These are small because `SEGSPaste` only rewrites the cropped face region; the majority of the frame is untouched and subtracts out. + +--- + +## Verification Criteria + +The smoke test is considered to PASS when **all** of the following hold: + +| # | Criterion | How to verify | +|---|-----------|---------------| +| 1 | Server startup log contains `Loading: ComfyUI-Impact-Pack` | Inspect server stdout / log tail | +| 2 | Server startup log contains `Loading: ComfyUI-Impact-Subpack` (default launch) | Inspect server stdout / log tail | +| 3 | Log contains `Skipping ... due to disable_all_custom_nodes and whitelist_custom_nodes` for at least one other custom node (when other nodes are installed) | Inspect server log | +| 4 | `http://127.0.0.1:18188/` returns HTTP 200 and a page title containing `ComfyUI` | `page.goto(...)` + `page.title()` | +| 5 | `GET /object_info` returns HTTP 200 with a JSON body | `page.request.get(...).status` and `.json()` | +| 6 | `/object_info` contains at least 85 Impact-prefixed node names | Count keys where `name.startswith("Impact")` | +| 7 | Required Subpack nodes present: `UltralyticsDetectorProvider`, `SAMLoader`, `SAMDetectorCombined`, `SAMDetectorSegmented` | Set membership against `object_info` keys | + +The baseline threshold of 85 was chosen below the observed value of 87 to tolerate minor refactors that rename or remove a handful of nodes without triggering a false failure. Raise the threshold deliberately when new nodes ship; lower it only with a reviewed explanation. + +For workflow execution tests, pass criteria are scenario-specific; see the per-test criteria listed alongside each reference implementation. + +--- + +## Operational Notes + +### Cache Busting + +ComfyUI caches sampler outputs across runs when node inputs are identical. This can mask regressions — a workflow may appear to "pass" because it is replaying a cached success. + +| Strategy | When to use | +|----------|-------------| +| Per-run seed randomization (`seed = int(time.time()) & 0xFFFFFFFF`) | Default; cheapest invalidation for sampler-bearing nodes | +| Full server restart | After changing Python source inside `modules/impact/` or loaded packages | +| Clear `modules/impact/__pycache__/` | When `.pyc` files may be stale relative to edited `.py` files | + +When restarting the server, kill any prior instance first and confirm the port is free before relaunching: + +```bash +pkill -9 -f 'python main.py' +# wait a moment, then verify nothing is still listening on the test port +curl -fsS http://127.0.0.1:18188/system_stats && echo "still up" || echo "port free" +``` + +Only relaunch once the probe reports the port is free. Launching while a dying process still holds the socket produces confusing `address already in use` errors downstream. + +### Verifying Internal Code Paths Executed + +Workflow-level pass criteria (`status_str == "success"`, no exception, non-zero pixel delta) prove the graph ran end to end. They do **not** prove that a specific internal function was reached. A bug that silently bypasses a compat shim can still return `success`. + +To confirm a specific branch executed, temporarily instrument the target: + +```python +# modules/impact/utils.py (temporary) +import logging +def apply_differential_diffusion(...): + logging.warning("[E2E-MARKER] apply_differential_diffusion:execute") + ... +``` + +Run the workflow, then grep the server log for the marker: + +```bash +grep 'E2E-MARKER' /tmp/server.log +``` + +Absence of the marker despite `status_str == "success"` is a signal that the code path was skipped — typically because an upstream dispatch picked a different branch. Remove the marker before committing. + +### Log File Decoding + +Progress bars emitted by `tqdm` (used by samplers, detectors, SAM) write carriage returns (`\r`) rather than newlines, collapsing a long run onto a single physical line in the log file. Naive `grep` on that file may report only the final progress state. + +Normalize before grepping: + +```bash +tr '\r' '\n' < /tmp/server.log | grep -F '[E2E-MARKER]' +``` + +Or in Python: + +```python +with open("/tmp/server.log", "r", errors="replace") as f: + text = f.read().replace("\r", "\n") +``` + +--- + +## Extension Points + +The smoke and reference workflow tests are the baseline verification layers. Future E2E scenarios should build on the same isolated-launch foundation: + +### Reference Test Implementation + +`tests/e2e_dd_compat.py` — validated workflow covering LoadImage → UltralyticsDetectorProvider → BboxDetectorSEGS → SEGSDetailer → SEGSPaste → PreviewImage. Demonstrates the full `/prompt` + `/history` + `/view` lifecycle and the pixel-delta assertion pattern. Treat it as the canonical template for new workflow-execution tests. + +### Additional Workflow Scenarios + +Beyond the detailer compat path, useful workflow-level tests include: FaceDetailer end-to-end (KSampler-generated face pipe), SEGSDetailer with `cycle > 1` (iterative refinement), MASK_TO_SEGS + SEGSPaste (mask-driven editing), and Impact Switch / Pipe nodes (control-flow regressions). + +### UI-Driven Node Creation + +Use Playwright to open the frontend, drag an Impact node from the node library onto the canvas, and connect inputs/outputs. Validates that frontend metadata (category, display name, input schema) stays synchronized with backend definitions. + +### Node Signature Regression Detection + +Snapshot the full `/object_info` payload for a known-good release, then diff against the current response. Flag any change in input type, input name, output type, or output count. Useful as a pre-release guard against accidental public API breakage. + +### Headed Mode for Debugging + +For interactive debugging, launch Playwright with `headless=False` and optionally `slow_mo=500`. Pair with `page.pause()` at the point of failure to inspect the live browser state. + +```python +browser = p.chromium.launch(headless=False, slow_mo=500) +# ... later ... +page.pause() # opens Playwright Inspector +``` + +### Cross-Browser Coverage + +Extend beyond chromium by parameterizing the browser launcher over `p.chromium`, `p.firefox`, and `p.webkit`. Impact Pack's frontend surface is thin, but cross-browser validation guards against regressions introduced by future frontend-facing features. diff --git a/modules/impact/animatediff_nodes.py b/modules/impact/animatediff_nodes.py index 5b45e82..0a0b933 100644 --- a/modules/impact/animatediff_nodes.py +++ b/modules/impact/animatediff_nodes.py @@ -66,7 +66,7 @@ class SEGSDetailerForAnimateDiff: cnet_image_list = [] if not (isinstance(model, str) and model == "DUMMY") and noise_mask_feather > 0 and 'denoise_mask_function' not in model.model_options: - model = nodes_differential_diffusion.DifferentialDiffusion().execute(model)[0] + model = utils.apply_differential_diffusion(model) for seg in segs[1]: cropped_image_frames = None diff --git a/modules/impact/core.py b/modules/impact/core.py index 801b71f..a75a5bf 100644 --- a/modules/impact/core.py +++ b/modules/impact/core.py @@ -262,7 +262,7 @@ def enhance_detail(image, model, clip, vae, guide_size, guide_size_for_bbox, max noise_mask = noise_mask.squeeze(3) if noise_mask_feather > 0 and 'denoise_mask_function' not in model.model_options: - model = nodes_differential_diffusion.DifferentialDiffusion().execute(model)[0] + model = utils.apply_differential_diffusion(model) if wildcard_opt is not None and wildcard_opt != "": model, _, wildcard_positive = wildcards.process_with_loras(wildcard_opt, model, clip) @@ -435,7 +435,7 @@ def enhance_detail_for_animatediff(image_frames, model, clip, vae, guide_size, g noise_mask = noise_mask.squeeze(3) if noise_mask_feather > 0 and 'denoise_mask_function' not in model.model_options: - model = nodes_differential_diffusion.DifferentialDiffusion().execute(model)[0] + model = utils.apply_differential_diffusion(model) if wildcard_opt is not None and wildcard_opt != "": model, _, wildcard_positive = wildcards.process_with_loras(wildcard_opt, model, clip) diff --git a/modules/impact/impact_pack.py b/modules/impact/impact_pack.py index 9e25c0b..c776c03 100644 --- a/modules/impact/impact_pack.py +++ b/modules/impact/impact_pack.py @@ -301,7 +301,7 @@ class DetailerForEach: ordered_segs = segs[1] if not (isinstance(model, str) and model == "DUMMY") and noise_mask_feather > 0 and 'denoise_mask_function' not in model.model_options: - model = nodes_differential_diffusion.DifferentialDiffusion().execute(model)[0] + model = utils.apply_differential_diffusion(model) for i, seg in enumerate(ordered_segs): cropped_image = utils.crop_ndarray4(image.cpu().numpy(), seg.crop_region) # Never use seg.cropped_image to handle overlapping area @@ -518,7 +518,7 @@ class DetailerForEachAutoRetry: ordered_segs = segs[1] if not (isinstance(model, str) and model == "DUMMY") and noise_mask_feather > 0 and 'denoise_mask_function' not in model.model_options: - model = nodes_differential_diffusion.DifferentialDiffusion().execute(model)[0] + model = utils.apply_differential_diffusion(model) for i, seg in enumerate(ordered_segs): cropped_image = utils.crop_ndarray4(image.cpu().numpy(), seg.crop_region) # Never use seg.cropped_image to handle overlapping area diff --git a/modules/impact/segs_nodes.py b/modules/impact/segs_nodes.py index ee46df1..3edf9ab 100644 --- a/modules/impact/segs_nodes.py +++ b/modules/impact/segs_nodes.py @@ -86,7 +86,7 @@ class SEGSDetailer: cnet_pil_list = [] if not (isinstance(model, str) and model == "DUMMY") and noise_mask_feather > 0 and 'denoise_mask_function' not in model.model_options: - model = nodes_differential_diffusion.DifferentialDiffusion().execute(model)[0] + model = utils.apply_differential_diffusion(model) for i in range(batch_size): seed += 1 diff --git a/modules/impact/segs_upscaler.py b/modules/impact/segs_upscaler.py index fb3cd30..741633a 100644 --- a/modules/impact/segs_upscaler.py +++ b/modules/impact/segs_upscaler.py @@ -102,7 +102,7 @@ def img2img_segs(image, model, clip, vae, seed, steps, cfg, sampler_name, schedu noise_mask = noise_mask.squeeze(3) if noise_mask_feather > 0 and 'denoise_mask_function' not in model.model_options: - model = nodes_differential_diffusion.DifferentialDiffusion().execute(model)[0] + model = utils.apply_differential_diffusion(model) if control_net_wrapper is not None: positive, negative, _ = control_net_wrapper.apply(positive, negative, image, noise_mask) diff --git a/modules/impact/utils.py b/modules/impact/utils.py index bd438c0..d8fb40c 100644 --- a/modules/impact/utils.py +++ b/modules/impact/utils.py @@ -690,6 +690,22 @@ def try_install_custom_node(custom_node_url, msg): logging.info("[Impact Pack] ComfyUI-Manager is outdated. The custom node installation feature is not available.") +def apply_differential_diffusion(model): + # ComfyUI ≥0.3.63 exposes V3 schema (classmethod `execute`); older versions use instance method `apply`. + # Import is deferred so callers with guarded imports (e.g. segs_upscaler.py) still work when the + # comfy_extras module is absent on very old ComfyUI — the ImportError propagates as before. + from comfy_extras import nodes_differential_diffusion + dd = nodes_differential_diffusion.DifferentialDiffusion() + if hasattr(dd, 'execute'): + return dd.execute(model)[0] + if hasattr(dd, 'apply'): + return dd.apply(model)[0] + raise AttributeError( + "DifferentialDiffusion has neither 'execute' nor 'apply'. " + "Update ComfyUI (≥0.3.63 for V3) or reinstall Impact Pack." + ) + + # author: Trung0246 ---> class TautologyStr(str): def __ne__(self, other): diff --git a/tests/e2e_dd_compat.py b/tests/e2e_dd_compat.py new file mode 100644 index 0000000..26940d4 --- /dev/null +++ b/tests/e2e_dd_compat.py @@ -0,0 +1,267 @@ +"""E2E test for the DifferentialDiffusion cross-version compat shim. + +Exercises the `utils.apply_differential_diffusion` helper end-to-end through +a real SEGSDetailer inference with `noise_mask_feather > 0`. This is the +smallest graph that deterministically triggers the helper without relying on +FaceDetailer's KSampler-generated face pipeline. + +Prerequisites: +- ComfyUI running on http://127.0.0.1:18188 with impact-pack AND + impact-subpack whitelisted: + python main.py --disable-all-custom-nodes \ + --whitelist-custom-nodes comfyui-impact-pack comfyui-impact-subpack \ + --port 18188 +- Models available: + models/checkpoints/SD1.5/realcartoonPixar_v8.safetensors + models/ultralytics/bbox/face_yolov8m.pt +- Input image with a visible face at input/ComfyUI_00156_.png +- Python Playwright 1.58+ (`pip install playwright && playwright install chromium`) + +What it verifies: +1. SEGSDetailer inference runs without AttributeError on DifferentialDiffusion. +2. Output image differs slightly from input (detailer actually edited the face). +3. Execution returns status=success. + +To verify the helper's .execute branch is actually taken on the current +ComfyUI version, temporarily add a `logging.warning(...)` inside +`modules/impact/utils.py::apply_differential_diffusion` and grep the server +log for the marker. See +`.claude/livecontext/analysis/2026-04-20_glarybot_v3_migration_proposal_review.md` +for broader context on the DD compat story. +""" +from playwright.sync_api import sync_playwright +import json +import time +import sys +import urllib.parse + +BASE_URL = "http://127.0.0.1:18188" +TIMEOUT_S = 600 + +_SEED = int(time.time()) & 0xFFFFFFFF + +PROMPT = { + "ckpt": { + "class_type": "CheckpointLoaderSimple", + "inputs": {"ckpt_name": "SD1.5/realcartoonPixar_v8.safetensors"}, + }, + "pos": { + "class_type": "CLIPTextEncode", + "inputs": {"clip": ["ckpt", 1], "text": "a detailed face, high quality, sharp focus"}, + }, + "neg": { + "class_type": "CLIPTextEncode", + "inputs": {"clip": ["ckpt", 1], "text": "blurry, low quality"}, + }, + "pipe": { + "class_type": "ToBasicPipe", + "inputs": { + "model": ["ckpt", 0], + "clip": ["ckpt", 1], + "vae": ["ckpt", 2], + "positive": ["pos", 0], + "negative": ["neg", 0], + }, + }, + "img": { + "class_type": "LoadImage", + "inputs": {"image": "ComfyUI_00156_.png"}, + }, + "detector": { + "class_type": "UltralyticsDetectorProvider", + "inputs": {"model_name": "bbox/face_yolov8m.pt"}, + }, + "bbox_segs": { + "class_type": "BboxDetectorSEGS", + "inputs": { + "bbox_detector": ["detector", 0], + "image": ["img", 0], + "threshold": 0.30, + "dilation": 10, + "crop_factor": 3.0, + "drop_size": 10, + "labels": "all", + }, + }, + # Non-zero noise_mask_feather is the critical knob — this is what + # activates the DifferentialDiffusion path inside enhance_detail and + # SEGSDetailer.do_detail. + "detail": { + "class_type": "SEGSDetailer", + "inputs": { + "image": ["img", 0], + "segs": ["bbox_segs", 0], + "guide_size": 512, + "guide_size_for": True, + "max_size": 1024, + "seed": _SEED, + "steps": 10, + "cfg": 7.0, + "sampler_name": "euler", + "scheduler": "normal", + "denoise": 0.5, + "noise_mask": True, + "force_inpaint": True, + "basic_pipe": ["pipe", 0], + "refiner_ratio": 0.2, + "batch_size": 1, + "cycle": 1, + "noise_mask_feather": 20, + }, + }, + "paste": { + "class_type": "SEGSPaste", + "inputs": { + "image": ["img", 0], + "segs": ["detail", 0], + "feather": 5, + "alpha": 255, + }, + }, + "preview_paste": { + "class_type": "PreviewImage", + "inputs": {"images": ["paste", 0]}, + }, + "preview_input": { + "class_type": "PreviewImage", + "inputs": {"images": ["img", 0]}, + }, +} + + +def fail(msg: str, code: int = 1) -> None: + print(f"FAIL: {msg}") + sys.exit(code) + + +def main() -> None: + with sync_playwright() as p: + browser = p.chromium.launch(headless=True) + ctx = browser.new_context(viewport={"width": 1280, "height": 800}) + page = ctx.new_page() + page.goto(f"{BASE_URL}/", wait_until="domcontentloaded", timeout=30000) + + submit = page.evaluate( + """async (prompt) => { + const client_id = (window.api && window.api.clientId) || crypto.randomUUID(); + const resp = await fetch('/prompt', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ prompt, client_id }) + }); + const text = await resp.text(); + let parsed = null; try { parsed = JSON.parse(text); } catch {} + return { status: resp.status, body: parsed || text }; + }""", + PROMPT, + ) + if submit.get("status") != 200 or not isinstance(submit.get("body"), dict): + fail(f"submission failed: {submit}", 2) + prompt_id = submit["body"].get("prompt_id") + if not prompt_id: + fail("no prompt_id returned", 2) + print(f"prompt_id: {prompt_id}") + + deadline = time.time() + TIMEOUT_S + last_sig = None + history_entry = None + while time.time() < deadline: + state = page.evaluate( + f"""async () => {{ + const h = await fetch('/history/{prompt_id}').then(r => r.json()); + const q = await fetch('/queue').then(r => r.json()); + return {{ h, q }}; + }}""" + ) + q = state["q"] + sig = f"running={len(q.get('queue_running', []))} pending={len(q.get('queue_pending', []))}" + if sig != last_sig: + print(f"[{int(TIMEOUT_S - (deadline - time.time())):>3}s] {sig}") + last_sig = sig + if prompt_id in state["h"]: + history_entry = state["h"][prompt_id] + break + time.sleep(3) + if history_entry is None: + fail(f"prompt did not complete within {TIMEOUT_S}s", 2) + + status = history_entry.get("status", {}) + messages = status.get("messages", []) + status_str = status.get("status_str") + print(f"status_str: {status_str}") + + exec_errors = [m[1] for m in messages if m[0] == "execution_error"] + for err in exec_errors: + print(f"[ERROR] node={err.get('node_id')} {err.get('exception_type')}: {err.get('exception_message')}") + for t in err.get("traceback", [])[-5:]: + print(f" {t.strip()}") + if exec_errors: + fail("execution_error present (DD compat shim or unrelated)", 1) + if status_str != "success": + fail(f"status_str={status_str!r}") + + outputs = history_entry.get("outputs", {}) + if "preview_input" not in outputs or "preview_paste" not in outputs: + fail(f"expected previews missing from outputs: {list(outputs)}") + + def fetch_png_stats(meta): + qs = urllib.parse.urlencode( + { + "filename": meta.get("filename", ""), + "subfolder": meta.get("subfolder", ""), + "type": meta.get("type", "output"), + } + ) + raw_list = page.evaluate( + f"""async () => {{ + const r = await fetch('/view?{qs}'); + if (!r.ok) return null; + const ab = await r.arrayBuffer(); + return Array.from(new Uint8Array(ab)); + }}""" + ) + if not raw_list: + return None + import io as _io + import numpy as np + from PIL import Image as PILImage + + pim = PILImage.open(_io.BytesIO(bytes(raw_list))) + arr = np.array(pim) + return { + "size": pim.size, + "mean": float(arr.mean()), + "std": float(arr.std()), + } + + in_stats = fetch_png_stats(outputs["preview_input"]["images"][0]) + out_stats = fetch_png_stats(outputs["preview_paste"]["images"][0]) + if in_stats is None or out_stats is None: + fail("could not fetch one or both preview images") + + print(f"input : {in_stats}") + print(f"paste : {out_stats}") + + if out_stats["std"] < 1.0: + fail("paste output is degenerate (flat image)") + + if in_stats["size"] != out_stats["size"]: + fail(f"size mismatch: {in_stats['size']} vs {out_stats['size']}") + + # SEGSPaste with a detailed face should produce a slightly different mean/std + # vs the untouched input. Exact equality would indicate the detailer path + # (and thus the DD compat shim) was bypassed. + mean_delta = abs(out_stats["mean"] - in_stats["mean"]) + std_delta = abs(out_stats["std"] - in_stats["std"]) + if mean_delta < 0.005 and std_delta < 0.005: + fail( + f"output identical to input (mean_delta={mean_delta:.4f}, " + f"std_delta={std_delta:.4f}) — detailer likely didn't run" + ) + + print(f"PASS: detailer ran, mean_delta={mean_delta:.4f}, std_delta={std_delta:.4f}") + browser.close() + + +if __name__ == "__main__": + main()