olivv-cs-ComfyUI-FunPack/docs/testing.md

Testing and verification

Every rule in these guides is held by a test or says who checks it. This page shows how to run the tests, how to write a good one, and what counts as proof.

Python tests (core and modules)

pip install -r requirements.txt pytest
COMFYUI_ROOT=/path/to/ComfyUI python -m pytest

• pyproject.toml sets --import-mode=importlib and testpaths = ["core/tests", "modules"]. The root __init__.py is ComfyUI's entry point and cannot be imported on its own, which is why the mode matters.
• Tests sit beside what they test: core/tests/ and modules/<group>/<name>/tests/.
• COMFYUI_ROOT points at a ComfyUI checkout (it must contain comfy/supported_models.py). Without it, the default is ~/Documents/ComfyUI. With none found, tests that need ComfyUI skip, and a module that imports comfy at load is simply absent.
• On a machine with no GPU (a cloud session), sh tools/test_env.sh sets all of this up (CPU torch, pytest, a ComfyUI checkout) and prints the exact command; tools/cpuforce.py is the plugin that forces the CPU. Run it once in the environment's setup script and a session starts ready to test.
• ComfyUI needs torch. A CPU build is enough for the tests. If torch complains about CUDA, force the CPU device for the test run rather than editing a test.
• Tests that need aiohttp skip themselves with importorskip when it is missing.

Fixtures (conftest.py at the repo root)

Fixture	Gives you
comfyui	The ComfyUI path on sys.path, or skips the test.
tiny_h3	The real H3 model class, built tiny (random weights, CPU) with a patcher. Use it to prove a modifier or hook runs on the real forward pass. It says nothing about picture quality.
tiny_ltx	The real LTX model class, built tiny (random weights, CPU), once per run. It proves wiring only; LTX stays UNTESTED on a real model. Clone its patcher; never leave anything installed on the model itself.
autouse	A private addons file, a private quarantine file, and block-influence switched off, all in tmp_path, so a developer's own settings never change a result.

packed_av and unpacked build and undo the flat audio and video packing H3 uses.

What the suite already enforces

These run with the rest and fail the build if you break them.

Test	Rule
core/tests/test_module_imports.py	A module reaches core only through _core and never imports another module.
core/tests/test_registry.py, test_retired_modules.py	Announcement, ids, absence on failure.
core/tests/test_shipped_hints.py	Every shipped setting has a plain one-sentence hint and a unique label.
core/tests/test_loads_as_comfyui_does.py	The pack loads the way ComfyUI loads it.
core/tests/test_log_call_sites.py	Log calls use the right level and shape.
core/tests/test_control.py	Quarantine and Unavailable behave as described in Architecture.
modules/training/lora/tests/test_train_h3.py	A whole LoRA training run on a tiny real H3 on CPU, with a stand-in text encoder and VAEs. It proves our side (sets, latents, repairs, loss stream, saved file), nothing about real weights or hardware.

Environment-dependent failures

A few tests depend on the machine: a real GPU, a git checkout on a named branch, or a subprocess probe. On a stripped CPU box or a detached checkout they can fail without your change being the cause. Before you blame your change, run the same test on a clean checkout of the base branch. Name any such failure in your pull request instead of hiding it.

JavaScript tests (the app)

cd app
npm install     # dev only: jsdom and Playwright. Users never run npm.
npm test        # node --test --test-timeout=30000

• Plain ES modules, no build step, no bundler.
• Tests sit beside the code (*.test.js) or in composer/tests/ and shell/tests/.
• composer/tests/_dom.js gives a jsdom window. It has a URL, because without one localStorage throws and anything that remembers a preference silently takes its fallback.
• jsdom is honest about structure and events and dishonest about layout. It has no box model. Anything that depends on measured geometry (placement, scrolling, wrapping) belongs in a real browser: check it in the catalogue, or in a Playwright pass.

Testing a UI feature

The pattern is in Writing a UI feature: build a host, a fake app, call setup, assert on the handle's node. Also test the hide paths: a missing need, an unoffered mount and a throwing setup must each leave nothing drawn.

Testing a kit element

See Adding an element. The registry-wide rule tests apply to your element the moment it has a demo.

tools/kit_lint

scan.js applies KIT_RULES to the kit and the shell: no markup parsing, no eval. rule_kit_never_interprets.test.js runs it and checks it still catches a bad example. FEATURE_RULES run over every UI feature (rule_features_use_the_kit.test.js): no document, window, style, class names, literal colours or sizes. It is a discipline check, not a security boundary.

Looking at it

python tools/devserver.py 8199

Open http://localhost:8199/fun/. The real routes are served and nothing is queued, so Generate says there is no queue. Set COMFYUI_ROOT, or most modules will be absent. The catalogue pages are in UI kit reference. Check any visual change in dark and light and at a narrow width.

How to write a test that proves something

• Use fixtures, not real projects. Build the smallest input that shows the behaviour. A test must not depend on someone's saved project, models or addons file.
• Prove the mechanism end to end. A unit test of a helper does not show the feature works. For a modifier: install it on tiny_h3 or tiny_ltx, run a step, and compare the output with the module off. For a route: call it through the real router. For a UI feature: mount it with the real kit.
• Attack your own change. Ask what a hostile or careless caller would do: a missing key, a value outside min..max, a hook that raises on step 3, a name containing markup, a module that is absent. Write the test for the one most likely to happen.
• Test the failure path as hard as the happy path. The promise of this project is that a broken piece is absent and the rest carries on. A test for a new module should show that.
• Fail first. Make sure the test fails without your change. A test that passes either way proves nothing.
• Say what was and was not checked. A pull request states what was run and what could not be (for example, no GPU, so no real render). Do not describe a probabilistic effect as certain.
• Real renders are a separate step. Judging quality needs a real model on a real GPU and a person looking at the result. The unit tests prove the wiring; they do not prove the picture is better.

Before you push

• python -m pytest and npm test pass, or the failures are shown to be unrelated.
• New code has a test that fails without it.
• No test depends on your machine's paths, projects or models.
• No long dashes in new text. No node_modules or scratch files committed.