Five review findings of the P3 phase, each fixed at its root:
1. (medium) The P3.7 metadata test imported `tomllib` at module level: the
**3.10 CI job** (the declared `requires-python >=3.10` minimum) died
during collection, so the registry-metadata assertions never ran on 3.10.
The parser import is now guarded (`tomllib` -> `tomli`, the package the
3.10 job installs); without any parser the module *skips* instead of
dying. `pyproject.toml` gained the test extra
`tomli>=2.0; python_version < '3.11'`.
Proof: a 3.10 simulation (the stdlib `tomllib` refused for the test
module, `tomli` on sys.path) -> 6 passed; the control without `tomli`
-> 6 skipped (no collection error).
2. (medium) The P3.5 "primary-root-first multi-root" resolution was not
wired into the production load path: `resolve_model_file` (the only
`get_full_path_or_raise` caller) was used by tests only, so a weight in
a secondary model root was downloaded into the primary root again.
`src/keep_paths.py` now has `find_model_file()` (the multi-root lookup
through `folder_paths.get_full_path_or_raise`) and `resolve_model_root()`
(the root the weights really live in); `load_file_from_url_comfy()` - the
production path of `src/keep_model_loader.py:107` - uses the lookup, and
the facelib root follows the weights (the facelib helper builds its
paths from a single root). New tests prove that the production path
uses a weight of an extra model root and that the primary root wins.
The `models_dir` grep guard is no longer a blanket exemption: only the
registration in `src/keep_paths.py` may use the attribute.
3. (low) `[tool.comfy].Icon` was an empty string and the test only checked
that the key exists. The preview asset **is** committed
(`git cat-file -s HEAD:assets/keep-preview.png` -> 93279 bytes); only the
working copy was missing (an uncommitted local deletion, restored from
HEAD - no asset change is in this commit). The key now points at
`assets/keep-preview.png` and the test asserts that the value is not
empty and names a file that is really in the repository.
4. (low) `tests/test_package_entrypoint.py` hard-coded the V1 state and
would have gone red (not xfail) when P4.4 flips the root `__init__.py`
to `comfy_entrypoint`. The assertions now follow the entry point that is
really there: exactly **one** mapping style (V1 XOR V3). The mocked
smoke test of `tests/test_package_import.py` got the same treatment; the
headless variant is P4.4's job (its test list names it).
5. (low) The plan's §3 rule ("`comfy.*` imports inside `src/` are lazy
(function-level) so the pack imports without a ComfyUI server") was
violated and unguarded. The `comfy` imports of `src/keep_model_loader.py`
and `src/keep_processor.py` are function-level now; the vendored facelib
helper (which imports `comfy` itself) is imported lazily through a module
`__getattr__` (PEP 562), so `loader.FaceRestoreHelper` still resolves to
the vendored class. Structure **guard (f)** (`tests/test_structure.py`)
forbids a module-level `comfy`/`comfy_api` import in `src/**` (with the
self-test the guards file demands), and the new
`tests/test_lazy_imports.py` proves at runtime that the engines import
while every `comfy` import is blocked.
Gate after the commit:
python.exe -m pytest -q tests -m "unit or mock_integration"
-> 153 passed, 1 skipped, 5 deselected, 3 xfailed
COMFYUI_ROOT=C:/_Dev/ComfyUI_dev/ComfyUI-source/ComfyUI python.exe
-m pytest -q tests -m "unit or mock_integration or headless"
-> 158 passed, 1 skipped, 3 xfailed
Details: docs/2026-10-23-p3-review-findings.md
Five review findings of the P2 phase, each fixed at its root:
1. (medium) The P2.6 guard could not see an undeclared third-party import:
the closure's imports were filtered through a hard-coded 10-name map, so
every module outside it was dropped silently (a module-level `import av`
in a copy of the repo still gave "1 passed"). The map is now a *rename*
map (cv2 -> opencv-python, PIL -> Pillow, yaml -> pyyaml); every other
module is mapped to its own name, so an unknown module shows up as an
undeclared dependency. The guard is split by import time: every
third-party module imported at import time must be a declared
distribution, every lazy (function-level) one must be allowlisted with a
reason (dlib, gdown, mc, wandb - all unreachable from the node).
2. (low) The P2.1 (c) progression test was vacuous: it returned before its
assertions and probed a second, top-level copy of the vendored tree
(`sys.path.insert(modules/deps)`). It now imports the packaged
architecture, asserts that FeedForward / AdaLayerNorm come from the
vendored _diffusers_compat and that no top-level `basicsr` shadows an
installed one.
3. (low) Two swallow-and-stub fallbacks remained in the vendored tree
(facelib/detection, facelib/parsing - their stubs called an unimported
`urlparse` and would have raised NameError). Both now import the vendored
symbol directly; a guard forbids a fallback stub in the vendored tree.
4. (low) `basicsr/utils/file_client.py` still appended to `sys.path` (the
optional `sys_path` of MemcachedBackend), which the plan's §3 rule
forbids anywhere in the package. The mutation and the parameter are gone
and guard (d) now covers the whole package (the vendored tree included).
5. (low) The unreachable body of `KEEPFaceProcessor.process_video` still
called the deleted VideoReader / VideoWriter. The 141 dead lines are
removed; an AST guard keeps the references out.
Found while fixing the findings: the CI job did not install `requests`,
which the vendored `basicsr/utils/download_util.py` imports at module level
(P2.6 added it to requirements.txt but not to the workflow) - a clean CI
runner died with "No module named 'requests'". The install list is now
completed and a test derives it from requirements.txt.
Gate: pytest -m "unit or mock_integration" -> 109 passed, 1 skipped,
4 xfailed; plan's local gate (+ headless) -> 109 passed, 3 skipped,
4 xfailed; tools/scan_compat.py -> finding_count 0 (exit 0); ruff check . ->
All checks passed. A mutation check re-introduces six defects into a copy of
the repository: the matching guard fails in 6/6 cases.
Seven review findings of the P1 phase, each fixed at its root:
1. (high) The CI install list omitted PyYAML, but the vendored tree imports
yaml eagerly (modules/deps/basicsr/utils/options.py:5), so the CI job
failed on its first run. pyyaml is now installed and asserted by
tests/test_ci_workflow.py.
2. (medium) tests/test_ci_workflow.py skipped the whole module without
PyYAML. The module never skips now: a minimal indentation parser is the
fallback, and it is cross-checked against the PyYAML parse.
3. (medium) Guard (e)'s self-test asserted that the tree still contains
breakage (it would break exactly at P2.1). It now proves the guard
instead; P2.1's work list is the scanner output and
test_guard_e_flip_owner_covers_every_current_finding keeps plan and tree
in sync.
4. (medium) The mock-fidelity tripwire skipped the comfy_api.latest
symbols. Every mocked symbol is now checked against the real source
(re-export / definition / instance attribute), including io.ComfyNode,
ComfyExtension, ComfyAPI and ComfyAPISync, plus the reverse check that
the mock invents nothing.
5. (low) The phase review listed committed work as outstanding and
misstated the ruff ignore list; both are corrected.
6. (low) The reported whole-suite count was impossible; the phase review
now carries the measured numbers.
7. (low) The root conftest installed the mocks unconditionally. The
KEEP_NO_COMFY_MOCKS=1 opt-out lets the headless tier import the real
comfy/comfy_api modules in-process.
tools/ci_env_sim.py runs the gate in an environment without the packages
the CI install list omits.
.github/workflows/tests.yml:
- lint job: pip install 'ruff==0.8.*' (pinned major) + ruff check .
- test job: matrix python 3.10/3.12/3.13, CPU torch + torchvision from the
pytorch CPU index, numpy<2.3 and tomli on 3.10 only, opencv-python-headless,
einops, scipy, Pillow, tqdm, pytest; runs
python -m pytest -q -m 'unit or mock_integration or headless' with
COMFYUI_ROOT empty so the env-dependent tests skip
- publish.yml untouched
pyproject.toml: [tool.ruff] scope (E9+F, F401/F541 ignored) and the vendored
tree modules/deps excluded - with the default rules the repo has 127 findings.
tests/test_ci_workflow.py: parses the workflow and asserts the jobs, the
matrix, the install steps, COMFYUI_ROOT='' and that the CI command equals the
local gate command.
tests/_headless.py resolves the ComfyUI tree from COMFYUI_ROOT and the weight
directory from KEEP_MODEL_DIR (no hardcoded paths, actionable errors), loads
extra_model_paths.yaml through utils.extra_config like the server does,
registers a package under an importable alias with __path__ set and can stub
only the ui preview helpers.
tests/test_headless_bootstrap.py proves the helper on a dummy package: alias
resolution, submodule import, the missing-environment-variable errors and the
absence of machine-specific paths.
tests/test_structure.py implements the five plan guards as pure functions over
a repository root, each xfail(strict=False) with the step that flips it
strict in the reason:
(a) nodes/ classes exported by nodes/__init__.py -> P3.1
(b) no io.Schema(/define_schema in src/ -> P3.2
(c) __init__.py exposes exactly comfy_entrypoint -> P4.4
(d) no sys.path mutation outside modules/deps/ -> P2.5
(e) no pdb/profiler/pip in the vendored closure -> P2.1
The self-tests copy the repository into tmp_path and prove that each guard
fails on a deliberately violating file.
- conftest.py: comfy_api.latest (io/ui as siblings, ComfyExtension,
ComfyAPI/ComfyAPISync with an instance-only progress recorder),
comfy.model_management, comfy.model_patcher, comfy.utils and folder_paths
are installed before collection; the package alias ComfyUI_KEEP keeps
__path__ + submodule_search_locations and provides the logger the
modules/* code imports
- fixtures: deterministic sample_image and an autouse per-test
models_dir/folder-registry reset
- tests/test_conftest_mocks.py: dummy V3 node end-to-end through the mock
io, folder_paths isolation, mock-fidelity tripwire against the real
ComfyUI source (skips cleanly without COMFYUI_ROOT)
- pyproject: [tool.pytest.ini_options] with testpaths=tests, the four plan
markers (unit/mock_integration/headless/gpu_acceptance), addopts
'-v --tb=short --import-mode=importlib', consider_namespace_packages and
pythonpath=['.']
- tests/test_sanity.py::test_python_and_torch pins the supported interpreter
and the torch import
- 'python -m pytest --collect-only -q' exits 0 (40 tests collected)
All 7 P0 review findings were verified by execution and fixed.
F1 (medium) - the compat scanner modelled only the basicsr/archs eager scan.
The vendored basicsr also scans data/, losses/ and models/ eagerly
(basicsr/__init__.py:3-11 imports all of them), so the closure missed 32
files that the repaired package really imports (incl. models/keep_model.py and
models/keep_gan_model.py, both with 'import pdb' at line 4). The scanner now
models all four scans plus 'from <package> import <submodule>': closure
100 -> 132 files, findings 8 -> 10, and 0 runtime-loaded vendored files are
missing from the closure (verified on a temp copy of modules/deps).
The audit doc's opposite claim is corrected and P2.4 in the plan now has to
rewrite basicsr/__init__.py and prove 'import basicsr' still succeeds after
pruning (otherwise ModuleNotFoundError is swallowed into DummyArchRegistry).
F2 (medium) - the Asian manifest entry is now decided (P0.4 section 4.1):
url 404, sha256 null, sha256_prefix '4765ebe0', unverified true, available
false, required FALSE (a required entry would break the default KEEP model
through P5.3's fail-fast rule). Plan P5.0/P5.1/P5.3 and the weights scanner
were aligned.
F3 (low) - the plan's section 8 decision 1, the B3 baseline row, Goal 5, the
risk table and P5.5 carried the superseded GMFlow premise; they now match the
P0.4 decision (the shipped checkpoint carries the flownet weights; the GMFlow
file is defensive and resolved only when the checkpoint has no flownet.* keys).
F4 (low) - the P0.1 fixture asserted rc == 0 before checking the probe's fatal
report, so the documented skip was unreachable. It now handles the probe's
exit code 2 + fatal report and skips with a reason.
F5 (low) - the baseline-import doc claimed all eight stub symbols fire in the
ffmpeg-blocked run; it is 7 of 8 (loader_face_restore_helper_is_dummy does not
fire - the real facelib imports fine).
F6 (low) - the scanner's exit-code contract (0 = no findings, 1 = findings) is
documented in the tool docstring and in the audit doc.
F7 (low) - the root conftest registered '__init__' and the invalid directory
basename; it now registers the plan's alias 'ComfyUI_KEEP' and provides
execute_package_init() so the V3 entrypoint is executed rather than bypassed.
P1.2 is specified to replace that bootstrap.
Gates: pytest tests/ -q -> 37 passed, 1 xfailed; scan_compat.py --quiet ->
closure 132, 10 findings, exit 1; baseline_probe.py -> import ok / model build
fails (unchanged baseline); scan_weights.py -> 0 of 8 present.
WildAi