Document test commands and remove hard-coded runtime path

This commit is contained in:
Peyton DeNiro
2026-03-28 16:22:12 -05:00
parent 6cbc383415
commit 025e58a58e
2 changed files with 108 additions and 4 deletions
+102
View File
@@ -0,0 +1,102 @@
# Tests
This directory contains two test paths:
- unit tests for exporter behavior
- runtime validation for committed workflow fixtures
## Prerequisites
- Run commands from the repo root.
- Use `uv` for all project commands.
- Install the repo environment first:
```bash
uv sync
```
- Runtime validation also needs a working ComfyUI checkout.
Set `COMFYUI_PATH` to that checkout, or run the tests from a location where a parent directory contains `ComfyUI`.
## Unit Tests
Run the current unit test module:
```bash
uv run python -m unittest tests.test_upscale_model_loader_export
```
Run all `unittest`-discoverable tests under `tests`:
```bash
uv run python -m unittest discover -s tests
```
## Runtime Tests
The runtime harness lives at `tests/runtime/run_runtime_validation.py`.
### Fast Tier
Fast tier validates export behavior against committed fixtures without requiring a full ComfyUI runtime for every fixture.
Run all fast-tier compatible fixtures:
```bash
uv run python tests/runtime/run_runtime_validation.py --tier fast --fixture all
```
Run one fixture:
```bash
uv run python tests/runtime/run_runtime_validation.py --tier fast --fixture unsafe-kwargs
```
### Runtime Tier
Runtime tier exports inside a real ComfyUI checkout and executes generated Python for runtime-capable fixtures.
Run all runtime-capable fixtures:
```bash
uv run python tests/runtime/run_runtime_validation.py --tier runtime --fixture all
```
Run one runtime-capable fixture:
```bash
uv run python tests/runtime/run_runtime_validation.py --tier runtime --fixture text-to-image
```
Print download commands for missing models instead of failing immediately:
```bash
uv run python tests/runtime/run_runtime_validation.py --tier runtime --fixture all --print-download-plan
```
## Runtime Fixture Names
Current committed fixtures:
- `upscale-model-loader`
- `text-to-image`
- `unsafe-kwargs`
- `subgraph-identifiers`
Notes:
- `--tier runtime` only runs fixtures marked runtime-capable.
- `--tier fast` only runs fixtures with local test mappings.
## Troubleshooting
- `No module named ...`:
run `uv sync` in this repo, and for runtime-tier failures also make sure the target ComfyUI checkout has its own dependencies installed.
- `Could not find a valid ComfyUI checkout for runtime validation.`:
set `COMFYUI_PATH` to your ComfyUI checkout, or run the tests from a directory layout where a parent folder contains `ComfyUI`.
- `Missing models for ...`:
rerun with `--print-download-plan` to print the expected `curl` commands and target model directories.
- `No selected fixtures are runtime-capable for this tier.`:
choose a runtime-capable fixture such as `text-to-image` or `upscale-model-loader`.
- Generated script execution fails because files or models are missing:
confirm the required models exist under the target ComfyUI checkout and that any staged inputs can be copied into its `input/` directory.
+6 -4
View File
@@ -13,11 +13,11 @@ from io import StringIO
from pathlib import Path
from typing import Callable
from comfyui_to_python_utils import get_comfyui_path
ROOT = Path(__file__).resolve().parents[2]
FIXTURE_DIR = ROOT / "tests" / "fixtures" / "runtime"
GENERATED_DIR = ROOT / "tests" / "runtime" / "generated"
BASELINE_COMFYUI_PATH = Path("/home/peyton/code/ComfyUI")
COMFYUI_OUTPUT_DIRNAME = "output"
COMFYUI_INPUT_DIRNAME = "input"
@@ -286,14 +286,16 @@ def load_fixture_names(selection: str) -> list[str]:
def ensure_runtime_path(tier: str) -> str:
if tier == "runtime":
runtime_path = BASELINE_COMFYUI_PATH
runtime_path = get_comfyui_path()
else:
return os.environ.get("COMFYUI_PATH", "")
if not runtime_path.is_dir():
if not runtime_path or not Path(runtime_path).is_dir():
raise ValidationFailure(
"environment/setup failure",
f"Expected ComfyUI checkout at {runtime_path}, but it was not found.",
"Could not find a valid ComfyUI checkout for runtime validation. "
"Set COMFYUI_PATH or run the tests from a location where a "
"parent directory contains ComfyUI.",
)
return str(runtime_path)