Files

T

Peyton DeNiro ae8d5a2f2a fix(hardening): address all PR #157 review findings

Critical:
  - _load_module(): wrap exec_module() in try/except, pop from sys.modules
    on failure so broken modules aren't cached
  - get_comfyui_path(): add _is_comfyui_directory() check to third (CWD walk)
    strategy so a non-ComfyUI directory named "ComfyUI" isn't accepted
  - get_node_class_mappings(): return cached sys.modules["nodes"] if already
    loaded to avoid re-executing nodes.py and losing init_extra_nodes() state

Suggestions:
  - import_custom_nodes(): wrap sys.path filtering in try/finally so path is
    always restored even if _load_module("server") raises mid-execution
  - render.py: auto-discover helpers from generated_helpers.__all__ instead of
    a manually maintained list
  - pyproject.toml: remove duplicate ruff dev dependency (kept standard
    [project.optional-dependencies] form)

Nitpicks:
  - Remove dead imports (types, StringIO) from test_import_path_resolution.py
  - Trim trailing blank lines in pyproject.toml
  - Tone down _load_module docstring: "eliminates ALL" -> "significantly reduces"

Tests (7 new cases):
  - TestLoadModuleFailureCleanup: broken module removed on exec failure, retry
    starts fresh
  - TestLoadModuleTemp: temp modules purged from sys.modules (success + failure)
  - TestRepeatedNodeLoad: repeated _load_module("nodes") returns cached copy
  - TestGetComfyuiPathThirdStrategy: fake "ComfyUI" dir rejected
  - TestImportCustomNodesSysPathRestoration: sys.path restored after crash

2026-05-10 10:43:12 -05:00

fixtures

Make generator codegen regressions explicit and fix string seed randomization

2026-03-29 17:09:30 -05:00

runtime

fix: filter comfy/ subdirectory from sys.path before loading server.py

2026-05-10 09:58:06 -05:00

README.md

Strengthen test suite robustness

2026-03-29 14:08:18 -05:00

test_generator_codegen_issue_regressions.py

Make generator codegen regressions explicit and fix string seed randomization

2026-03-29 17:09:30 -05:00

test_import_path_resolution.py

fix(hardening): address all PR #157 review findings

2026-05-10 10:43:12 -05:00

test_node_runtime_cleanup.py

Guard ComfyUI cleanup teardown failures

2026-03-29 19:02:21 -05:00

test_project_contracts.py

Align Cluster 2 compatibility boundaries

2026-03-29 17:34:30 -05:00

test_runtime_validation_harness.py

Strengthen test suite robustness

2026-03-29 14:08:18 -05:00

test_upscale_model_loader_export.py

fix: use _load_module_temp for bootstrap modules to avoid sys.modules conflicts

2026-05-10 00:31:02 -05:00

README.md

Tests

This directory contains two test paths:

unit tests for exporter behavior
runtime validation for committed workflow fixtures

Use the lightweight path for routine contributor validation, and use the runtime path when you need end-to-end confidence against a real ComfyUI checkout.

Prerequisites

Run commands from the repo root.
Use uv for all project commands.
Install the repo environment first:

uv sync

Runtime validation also needs a working ComfyUI checkout. Set COMFYUI_PATH to that checkout, or run the tests from a location where a parent directory contains ComfyUI.
Runtime-tier tests do not download missing models automatically. If a required model is missing, the fixture fails with model provisioning failure. Use --print-download-plan to print the manual download commands.

Runtime Model Requirements

The current runtime-capable fixtures require these model files in the target ComfyUI checkout:

text-to-image: models/checkpoints/v1-5-pruned-emaonly-fp16.safetensors
upscale-model-loader: models/upscale_models/RealESRGAN_x4plus.safetensors

Unit Tests

Run the exporter-focused unit test module:

uv run python -m unittest tests.test_upscale_model_loader_export

Run the runtime-harness unit test module:

uv run python -m unittest tests.test_runtime_validation_harness

Run all unittest-discoverable tests under tests:

uv run python -m unittest discover -s tests

Runtime Tests

The runtime harness lives at tests/runtime/run_runtime_validation.py.

Fast Tier

Fast tier validates export behavior against committed fixtures without requiring a full ComfyUI runtime for every fixture. This is the recommended default validation lane for routine changes.

Run all fast-tier compatible fixtures:

uv run python tests/runtime/run_runtime_validation.py --tier fast --fixture all

Run one fixture:

uv run python tests/runtime/run_runtime_validation.py --tier fast --fixture unsafe-kwargs

Runtime Tier

Runtime tier exports inside a real ComfyUI checkout and executes generated Python for runtime-capable fixtures. This is the heavier validation lane for changes that need end-to-end runtime confidence.

Run all runtime-capable fixtures:

uv run python tests/runtime/run_runtime_validation.py --tier runtime --fixture all

Run one runtime-capable fixture:

uv run python tests/runtime/run_runtime_validation.py --tier runtime --fixture text-to-image

Print download commands for missing models instead of failing immediately:

uv run python tests/runtime/run_runtime_validation.py --tier runtime --fixture all --print-download-plan

Runtime Fixture Names

Current committed fixtures:

upscale-model-loader
text-to-image
unsafe-kwargs
subgraph-identifiers
reused-node-class-branches
secondary-output-selection

Notes:

--tier runtime only runs fixtures marked runtime-capable.
--tier fast only runs fixtures with local test mappings.
reused-node-class-branches protects repeated node-class usage and branch wiring in the fast tier.
secondary-output-selection protects non-zero output index wiring in the fast tier.

Troubleshooting

No module named ...: run uv sync in this repo, and for runtime-tier failures also make sure the target ComfyUI checkout has its own dependencies installed.
Could not find a valid ComfyUI checkout for runtime validation.: set COMFYUI_PATH to your ComfyUI checkout, or run the tests from a directory layout where a parent folder contains ComfyUI.
Missing models for ...: runtime-tier tests do not fetch models for you. Rerun with --print-download-plan to print the expected curl commands and target model directories, then install the files manually.
No selected fixtures are runtime-capable for this tier.: choose a runtime-capable fixture such as text-to-image or upscale-model-loader.
Generated script execution fails because files or models are missing: confirm the required models exist under the target ComfyUI checkout and that any staged inputs can be copied into its input/ directory.