nodes/JasonHoku-ComfyUI-Ultimate-Auto-Sampler-Config-Grid-Testing-Suite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs: bring README, ProjectStructure, Roadmap, AI-CONTEXT and CONTRIBUTING in line with main (11/4/26)

Browse Source 
README.md
- Moved: "Pairwise Review (blind A/B compare + scoreboard)" from a stranded "##" between File
  Locations and Keyboard Shortcuts to a "###" inside Dashboard Interface (after Revision Modal,
  before Standalone Dashboard); anchor unchanged.
- Moved: "Image Metrics (Optional)" from after the Config Builder's closing rule to right after
  "Job Order & Preview Pass", before "Workflow Sweep"; the rule now follows Workflow Sweep.
- TOC rebuilt to mirror document order (Quick Start, Optional Companion Plugins, Config Builder
  as its own branch, Pairwise/Standalone/Image Metrics/Job Order/Skip Layer Guidance/MOSS/Workflow
  Sweep, API routes, Development / Tests); fixes the broken "json-bars" anchor (three hyphens).
- Key Features: one bullet each for Workflow Sweep, Job Order, Preview Pass, Skip Layer Guidance,
  16 px latent models, Examples folder, Pairwise Review, Standalone Dashboard + Run API.
- Run Settings: pointer line to Job Order & Preview Pass, Image Metrics and te_batch_size.
- API Integration: pointer sentence after the intro, "Provided by the ComfyUI-USCG-Distributed
  companion plugin" under Distribution API, new "Dashboard, Review, Metrics & Sweep Routes" table.
- Notes for AI Development: gotcha 2 rewritten (one config-to-JSON implementation), gotcha 13 loses
  the "if True" sentence, new gotcha 14 pointing at ProjectStructure.md (36 gotchas), File
  Dependencies and Server Event Types updated, Development / Tests shortened to 3 commands + link.
- Compare mode shortcut row: "see below" is now a link to the moved Pairwise section.
- Changelog: new "Update 11/4/26" entry above 11/3/26 (older entries untouched).
- Removed duplicate sentences (TE batching): in the MOSS-Image bullet "Prompts that can't be
  batched (weighted, AND/CAT/AVG()/BREAK) and encoders other than MOSS-Image / Krea 2 /
  Qwen-Image keep the per-prompt path." (already in the te_batch_size bullets) and "The summary
  line shows how many prompts were batched." (moved into the te_batch_size bullet as
  "The pre-encode summary line shows how many prompts were batched."). Dropped from
  Development / Tests: the stale counts (137 passed / 15 skipped / 152 passed), the torch and
  playwright install lines and the CI job paragraph (all kept in CONTRIBUTING.md).

ProjectStructure.md
- File map regenerated with measured line counts; adds florence2_hires, ltx_video_generation,
  civitai/distribution facades, conftest, pytest.ini, sampler_help_note.js, conf-builder-workflow-sweep.js,
  conf-builder-readme.md, tests/, tests/e2e, tests/js, examples, .github/workflows; removes
  network_utils, distribution_manager, distribution_worker, distribution_routes; resources/ in
  js_files order. Section 2b rewritten as the companion facade.
- Node Class Mappings: USCGSweepCollect, no line number.
- API Routes: one row per registered route with File and Body / query columns (Line column
  dropped); Distribution table moved under "Companion plugin routes".
- Server Event Types: update_data, next_run_seeds, metrics_progress, phase payloads.
- Gotchas: #1 now "Single Data Path" with the add-a-field checklist, #2/#26/Testing Changes/Key
  Functions/utilities.js rows stop naming convertStateToConfigs, the unnumbered Workflow Sweep
  quote became #30 (text verbatim), new #31-#36.

Roadmap.md: #23, #25, #26, #27 status notes appended, new "Open follow-ups (11/4/26)" #29-#32.
AI-CONTEXT.md: real line counts and new file rows, "One Source of Truth for Config JSON" replaces
the two-codepaths section, network_utils/convertStateToConfigs/js load order/gotcha count fixed.
CONTRIBUTING.md: measured counts, CI job table, the CI JS syntax loops and assembled-page check,
node --test, e2e flags, checklist updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
Jason Hoku
2026-11-04 01:05:30 -08:00
co-authored by Claude Fable 5.1
parent ecf28d45aa
commit 0c813d58f6
5 changed files with 388 additions and 253 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AI-CONTEXT.md
+45 -39
View File
@@ -10,10 +10,10 @@ A ComfyUI custom node extension for automated image generation grid testing. Use
## Project Location
```
Z:\comfy_v0.12.3\ComfyUI\custom_nodes\ComfyUI-Ultimate-Auto-Sampler-Config-Grid-Testing-Suite\
ComfyUI/custom_nodes/ComfyUI-Ultimate-Auto-Sampler-Config-Grid-Testing-Suite/
```
Read `ProjectStructure.md` in this directory for the full reference (file structure, all API routes with line numbers, data shapes, communication patterns, dependency graph, and 28 development gotchas).
Read `ProjectStructure.md` in this directory for the full reference (file structure, all API routes with method / body / purpose, data shapes, communication patterns, dependency graph, and 36 development gotchas).
## Architecture (3 Layers)
@@ -25,42 +25,55 @@ Read `ProjectStructure.md` in this directory for the full reference (file struct
| File | Lines | Role |
|------|-------|------|
| `__init__.py` | ~1000 | API endpoints (config CRUD, dashboard save/export/delete, scan, upscale presets, config presets, async upscale), node mappings, path security |
| `sampler_node.py` | 314 | Main node class. Unwraps configs, extracts `_distribution` + `_session_settings`, delegates to orchestrator |
| `config_builder_node.py` | ~1200 | Config builder node. `generate_config()` reads ALL state from `lora_config` widget. API endpoints for model lists, trigger lookups |
| `generation_orchestrator.py` | ~2600 | **Largest file.** Main generation loop, smart skip, upscaling pipelines, deferred upscales (`run_deferred_upscales()`), GPU cooldown, distribution, ETA tracking, Start At Job # |
| `image_generation.py` | ~780 | KSampler wrapper, VAE decode, `flush_batch_with_vae()` (saves images + updates manifest), `upscale_image()`, `create_image_metadata()`, rolling ETA |
| `network_utils.py` | 280 | **All outbound network requests.** CivitAI API, HuggingFace Remote VAE (allowlisted), Distribution LAN calls. No other file imports urllib. |
| `upscale_runner.py` | 371 | Dashboard async upscaling. Background thread loads models, runs pipeline chains, updates manifests, sends progress events. |
| `config_utils.py` | 674 | Config expansion (Cartesian products), nested prompt parsing, job preparation |
| `model_loader.py` | 805 | Checkpoint/LoRA/VAE/GGUF loading |
| `model_cache.py` | 900 | 3-tier cache: LoRA files → incremental states → patched models. Async preloading |
| `manifest_utils.py` | 120 | Load/save/merge manifest. `save_manifest()` reloads from disk first to preserve concurrent user edits |
| `__init__.py` | 1597 | API endpoints (config CRUD, presets, dashboard save/export/delete, scan, async upscale, pairwise votes, metrics jobs, standalone dashboard page, run API, workflow sweep), node mappings, path security |
| `sampler_node.py` | 401 | Main node class. Unwraps configs, extracts `_distribution` + `_session_settings`, delegates to orchestrator |
| `config_builder_node.py` | 1578 | Config builder node. `state_to_configs_json(state)` is the ONLY state-to-configs implementation (`generate_config()` and `POST /configbuilder/preview` both call it). API endpoints for model lists, trigger lookups |
| `generation_orchestrator.py` | 3693 | **Largest file.** Main generation loop over a flat job plan (`config_utils.build_job_plan`), smart skip, preview pass, upscaling pipelines, deferred upscales (`run_deferred_upscales()`), GPU cooldown, distribution, ETA tracking, Start At Job # |
| `image_generation.py` | 1321 | KSampler wrapper, VAE decode, `flush_batch_with_vae()` (saves images + updates manifest), `upscale_image()`, `create_image_metadata()`, rolling ETA |
| `batch_encoding.py` | 878 | Prompt pre-encode with combinators, `pinned_clip()` (one text-encoder load per pass), `te_batch_size` text-encoder batching |
| `civitai.py`, `distribution.py` | 40, 107 | Thin facades for the companion plugins (ComfyUI-USCG-CivitAI, ComfyUI-USCG-Distributed). No outbound network code in this package. |
| `upscale_runner.py` | 642 | Dashboard async upscaling. Background thread loads models, runs pipeline chains, updates manifests, sends progress events. |
| `config_utils.py` | 1108 | Config expansion (Cartesian products), nested prompt parsing, job preparation, job plan / ordering, `config_fingerprint()` |
| `model_loader.py` | 907 | Checkpoint/LoRA/VAE/GGUF loading, latent geometry (`get_latent_downscale_ratio`), CLIP type resolution |
| `model_cache.py` | 935 | 3-tier cache: LoRA files → incremental states → patched models. Async preloading |
| `manifest_utils.py` | 316 | Load/save/merge manifest. `save_manifest()` reloads from disk first to preserve concurrent user edits (favorites, votes, metrics) |
| `metadata_packer.py` | 561 | PNG metadata embedding for CivitAI. `pack_metadata_into_image()` accepts `workflow_data` kwarg |
| `web/conf_builder/conf-builder-main.js` | 608 | Node registration. Default state (lines 80-147) defines ALL fields including upscaling/cooldown |
| `web/conf_builder/conf-builder-utilities.js` | 958 | `convertStateToConfigs()` (line 497) — **MUST stay in sync with Python `generate_config()`** |
| `web/conf_builder/conf-builder-config-management.js` | 4975 | All config builder UI rendering. Upscaling section ~line 4437 |
| `resources/logic_utils.js` | 599 | Dashboard helpers: `exportFavorites()`, `deleteNonFavorites()`, `loadSession()`, `scanDirectory()` |
| `resources/logic_ui.js` | 1780 | Dashboard DOM: cards, modals, `buildComfyNodesWorkflow()`, analytics |
| `resources/logic_virtual.js` | 925 | Virtual scroller — renders only visible cards from thousands |
| `run_api.py` | 144 | Pure helpers behind `POST /config_tester/run` and `GET /config_tester/run_status` |
| `image_metrics.py`, `metrics_runner.py` | 668, 167 | Optional per-image metrics at save time; background job that backfills them for old sessions |
| `workflow_sweep.py`, `workflow_sweep_runner.py`, `workflow_sweep_collect.py` | 969, 912, 140 | Workflow Sweep: pure expansion core, in-process queue feeder (never blocks the executor), `USCGSweepCollect` output node |
| `web/conf_builder/conf-builder-main.js` | 779 | Node registration. Default state and its migration blocks define ALL fields including upscaling/cooldown |
| `web/conf_builder/conf-builder-utilities.js` | 866 | Model-list cache (`LS_CACHE_VERSION`), `getIterationCount()`, LoRA string helpers. No config converter: the preview calls `/configbuilder/preview` |
| `web/conf_builder/conf-builder-config-management.js` | 7736 | All config builder UI rendering. Upscaling section, preview fetch |
| `web/conf_builder/conf-builder-workflow-sweep.js` | 744 | Workflow Sweep UI (load API workflow, axes, dry run, start/pause/cancel) |
| `resources/logic_utils.js` | 1073 | Dashboard helpers: `exportFavorites()`, `deleteNonFavorites()`, `loadSession()`, `scanDirectory()` |
| `resources/logic_ui.js` | 2907 | Dashboard DOM: cards, modals, `buildComfyNodesWorkflow()`, analytics |
| `resources/logic_virtual.js` | 1092 | Virtual scroller: renders only visible cards from thousands |
| `resources/logic_ratings.js`, `resources/logic_compare.js` | 538, 824 | Pairwise review: pure Bradley-Terry / pairing / scoreboard logic, and the compare overlay that uses it |
| `resources/logic_standalone.js` | 170 | Standalone-page mode (own websocket, run-API GENERATE NEW); loaded last |
## The Two Codepaths That Must Stay In Sync
## One Source of Truth for Config JSON
When adding ANY new config field, you MUST update BOTH:
`UltimateConfigBuilder.state_to_configs_json(state)` in `config_builder_node.py` is the only implementation that turns Builder state into configs JSON. `generate_config()` (the node's runtime output) and `POST /configbuilder/preview` (what the JS preview panel calls) both run it, so they cannot drift. There is no JS converter any more; do not add one.
1. **JS:** `convertStateToConfigs()` in `conf-builder-utilities.js` (line 497)
2. **Python:** `generate_config()` in `config_builder_node.py` (line 465)
When adding ANY new config field, ALL of these are needed:
Also add default values to:
- `conf-builder-main.js` default state (line 80)
- Migration check in `onConfigure` handler (~line 363) for existing saved workflows
- default in `conf-builder-main.js` new-state AND in its three migration blocks
- default in the new-config-array object in `conf-builder-config-management.js`
- UI control in the right render function
- emission in `config_builder_node.state_to_configs_json`
- key in `_EXPECTED_CONFIG_KEYS`
- pass-through in `config_utils.expand_configs`
- consumer in the orchestrator / `generate_image`
- README bullet and unit tests (`tests/test_state_to_configs_json.py`)
Bump `LS_CACHE_VERSION` in `conf-builder-utilities.js` only if the cached model-list shape changes.
## Config Data Flow
```
JS node.state
→ JSON.stringify() → lora_config widget value
→ Python json.loads(lora_config) in generate_config()
→ Python json.loads(lora_config) in generate_config() → state_to_configs_json()
→ Output: {"configs": [...], "_distribution": {...}, "_session_settings": {...}}
→ sampler_node.py unwraps configs, extracts _distribution + _session_settings
→ generation_orchestrator.run_generation_loop() receives all as params
@@ -117,7 +130,7 @@ Images saved as WebP at `output/benchmarks/{session}/images/img_{id}.webp`
- **Model discovery:** `folder_paths.get_filename_list("key")` → API at `/configbuilder/model_lists` → JS `getModelLists()`
- **Path security:** All endpoints use `_is_path_within()` + `re.sub(r'[^\w\-]', '', name)` sanitization
- **Dashboard updates:** `PromptServer.send_sync("ultimate_grid.update", {...})` → `dashboard.js` → `postMessage()` → iframe
- **All outbound network requests go through `network_utils.py`** — no other file imports `urllib.request`. CivitAI, HuggingFace VAE, and Distribution calls are all centralized there.
- **No outbound network code in this package**: CivitAI, Remote VAE and Distribution network calls live in the companion plugins; `civitai.py`, `remote_vae.py` and `distribution.py` only detect the companion and forward to it. `network_utils.py` no longer exists.
- **No `subprocess`, `os.system`, `eval()`, `exec()`** — blocked by security scanner
- **LoRA string format:** `"name:model_str:clip_str"`, stacked with `" + "` separator
- **Prompt nesting:** Flat list = OR, nested lists = AND (Cartesian product). Recursive in `parse_prompt_input_nested()`
@@ -135,13 +148,7 @@ Images saved as WebP at `output/benchmarks/{session}/images/img_{id}.webp`
## Common Tasks
**Add a new config field:**
1. Add to default state in `conf-builder-main.js` (line 80)
2. Add migration in `onConfigure` (~line 363)
3. Add UI rendering in `conf-builder-config-management.js`
4. Add to `convertStateToConfigs()` in `conf-builder-utilities.js`
5. Add to `generate_config()` in `config_builder_node.py`
6. Consume in `generation_orchestrator.py` or relevant backend file
**Add a new config field:** follow the checklist in "One Source of Truth for Config JSON" above (defaults + migration blocks in `conf-builder-main.js`, UI in `conf-builder-config-management.js`, `state_to_configs_json()`, `_EXPECTED_CONFIG_KEYS`, `expand_configs`, then the consumer in `generation_orchestrator.py` or the relevant backend file).
**Add a new API endpoint:**
1. Add route in `__init__.py` with `@server.PromptServer.instance.routes`
@@ -151,11 +158,10 @@ Images saved as WebP at `output/benchmarks/{session}/images/img_{id}.webp`
**Add a new session setting:**
1. Add to default state in `conf-builder-main.js` under appropriate key
2. Add UI in `conf-builder-config-management.js`
3. Add to `convertStateToConfigs()` session settings block (line 689)
4. Add to `generate_config()` session settings embedding (line 706)
5. Consume in `generation_orchestrator.py` after the `session_settings` param check
3. Add to the `_session_settings` block of `state_to_configs_json()` in `config_builder_node.py`
4. Consume in `generation_orchestrator.py` after the `session_settings` param check
**Modify dashboard UI:**
1. Edit files in `resources/` (HTML in `template.html`, JS in `logic_*.js`, CSS in `report.css`)
2. Remember: changes require HTML regeneration to take effect
3. JS load order: state → utils → ui → virtual → pipeline → events → init
3. JS load order is `html_generator.js_files`: logic_state.js → logic_utils.js → logic_ratings.js → logic_ui.js → logic_upscale_modal.js → logic_virtual.js → logic_pipeline.js → logic_compare.js → logic_events.js → logic_init.js → logic_standalone.js (last). All of them share ONE inlined script scope, so top-level names must be unique
CONTRIBUTING.md
+34 -28
View File
@@ -11,7 +11,7 @@ Before diving in, read these files in the repo root:
| File | What it covers |
|---|---|
| `README.md` | What the project does, how the three nodes connect, and user-facing features |
| `ProjectStructure.md` | Every file's purpose, all API routes with line numbers, data shapes, communication patterns, dependency graph, and 28 development gotchas |
| `ProjectStructure.md` | Every file's purpose, all API routes (method, body, purpose), data shapes, communication patterns, dependency graph, and 36 development gotchas |
| `AI-CONTEXT.md` | Internal dev notes: architecture layers, critical codepaths, field naming rules, config data flow, common tasks |
---
@@ -46,50 +46,55 @@ pip install -r requirements.txt
```bash
# Python unit tests (ComfyUI + torch are stubbed): needs pytest, numpy, pillow
pip install -r tests/requirements-ci.txt
python -m pytest tests/ -q -ra --ignore=tests/e2e # 137 passed, 15 skipped without torch
python -m pytest tests/ -q -ra --ignore=tests/e2e # 499 passed, 16 skipped without torch (measured 11/4/26)
# With real torch (e.g. ComfyUI's python), the 15 `requires_torch` tests run too.
# With real torch (e.g. ComfyUI's python), the 16 `requires_torch` tests run too.
# USCG_REQUIRE_TORCH=1 makes them fail instead of skip if torch is missing:
USCG_REQUIRE_TORCH=1 python -m pytest tests/ -q -ra --ignore=tests/e2e # 152 passed
USCG_REQUIRE_TORCH=1 python -m pytest tests/ -q -ra --ignore=tests/e2e
# Dashboard JS unit tests (node:test, no browser)
node --test tests/js/
# Dashboard SPA browser smoke test (Playwright, Chromium, no ComfyUI needed)
pip install -r tests/e2e/requirements.txt
python -m playwright install chromium
python -m pytest tests/e2e -q -ra
USCG_REQUIRE_E2E=1 python -m pytest tests/e2e -q -ra --browser chromium
```
GitHub Actions (`.github/workflows/tests.yml`) runs four jobs on every push and PR: `pytest-stubs`, `pytest-torch` (CPU torch), `js-syntax` (`node --check` on `web/` and `resources/`, the assembled Dashboard page, and `node --test tests/js/` when that folder exists) and `e2e-dashboard`.
GitHub Actions (`.github/workflows/tests.yml`) runs four jobs on every push and PR:
| Job id | What it runs |
|---|---|
| `pytest-stubs` | the Python tests with ComfyUI and torch stubbed (asserts torch is NOT installed) |
| `pytest-torch` | the same tests with real CPU torch and `USCG_REQUIRE_TORCH=1` |
| `js-syntax` | JS syntax checks (see below), the assembled Dashboard page, and `node --test tests/js/` |
| `e2e-dashboard` | the Playwright smoke test (`USCG_REQUIRE_E2E=1`, Chromium) |
Tests live in `tests/` and must not contain network or process-spawning code (no `socket`, `http.server`, `subprocess`, `urllib`): `tests/` ships in the registry zip and is yara-scanned.
`tests/test_docs_consistency.py` keeps the docs honest: it fails when a route, event, source file or CI job exists in the repo but is missing from `ProjectStructure.md` / this file, when a README anchor link does not resolve, or when the TOC misses a section. Update the doc it names.
PRs that change Python logic should add tests where it's reasonable to do so.
---
## JS syntax check
The Builder UI and Dashboard are plain JavaScript (no build step). You can catch syntax errors quickly with Node:
The Builder UI and Dashboard are plain JavaScript (no build step). CI runs these checks (the `js-syntax` job); run the same loops locally from the repo root:
```bash
cd web/conf_builder
node --check conf-builder-main.js
node --check conf-builder-config-management.js
node --check conf-builder-utilities.js
node --check conf-builder-ui-components.js
node --check conf-builder-distribution.js
# web/**/*.js are ES modules
for f in $(find web -name '*.js'); do
node --input-type=module --check < "$f" || { echo "FAIL $f"; break; }
done
# resources/*.js are plain scripts
for f in resources/*.js; do
node --check "$f" || { echo "FAIL $f"; break; }
done
```
The dashboard JS files can also be checked the same way:
```bash
cd resources
node --check logic_utils.js
node --check logic_ui.js
node --check logic_virtual.js
node --check logic_pipeline.js
node --check logic_events.js
node --check logic_init.js
node --check logic_state.js
```
The Dashboard files in `resources/` are inlined into ONE script (see `js_files` in `html_generator.py`), so a duplicate top-level name only shows up in the assembled page. The `js-syntax` job therefore also builds the page with `html_generator.get_html_template()` and syntax-checks every inline script body (the snippet is in `.github/workflows/tests.yml`).
---
@@ -97,7 +102,7 @@ node --check logic_state.js
**The Builder UI is a DOM widget on the Python node.** Its entire state is serialized to a single `lora_config` STRING widget as JSON. Python's `generate_config()` reads only that widget — all other `INPUT_TYPES` entries are vestigial.
**State → configs_json transformation:** All state-to-output transformation logic lives in `state_to_configs_json` (a `@staticmethod` on `UltimateConfigBuilder` in `config_builder_node.py`, around line 524). Both runtime (`generate_config`, line ~837) and the preview endpoint (`POST /configbuilder/preview`) call this single function. **Do not add a JS-side equivalent transformer** — that pattern existed historically (`convertStateToConfigs`) and was a recurring source of drift bugs; it was deliberately removed in favor of the single source of truth in Python.
**State → configs_json transformation:** All state-to-output transformation logic lives in `state_to_configs_json` (a `@staticmethod` on `UltimateConfigBuilder` in `config_builder_node.py`). Both runtime (`generate_config`) and the preview endpoint (`POST /configbuilder/preview`) call this single function. **Do not add a JS-side equivalent transformer** — that pattern existed historically (`convertStateToConfigs`) and was a recurring source of drift bugs; it was deliberately removed in favor of the single source of truth in Python.
When adding a new state field that should appear in `configs_json`:
1. Add the field to `state_to_configs_json` in `config_builder_node.py`. Make sure tests in `tests/test_state_to_configs_json.py` cover the new field.
@@ -134,8 +139,9 @@ When adding a new state field that should appear in `configs_json`:
Before submitting:
- [ ] `python -m pytest tests/ -v` passes
- [ ] JS syntax checks pass (`node --check` on changed files)
- [ ] `python -m pytest tests/ -q --ignore=tests/e2e` passes (and `node --test tests/js/`)
- [ ] JS syntax checks pass (the two loops above, plus the assembled-page check for `resources/` changes)
- [ ] New routes, events or source files are added to `ProjectStructure.md` (`tests/test_docs_consistency.py` checks this)
- [ ] No regressions in existing features
- [ ] For new config fields: `state_to_configs_json` updated in Python, default state and migration backfill added in `conf-builder-main.js`, and UI read/write added in `conf-builder-config-management.js`
- [ ] For user-facing features: `README.md` updated
ProjectStructure.md
+155 -122
View File
@@ -30,75 +30,85 @@ Reference: https://blog.comfy.org/p/comfyui-2025-jan-security-update
```text
ComfyUI-Ultimate-Sampler-Grid/
├── __init__.py # Server Routes & Node Mappings (933 lines)
├── sampler_node.py # Main "Sampler Grid" Node (314 lines)
├── __init__.py # Server Routes & Node Mappings (1597 lines)
├── sampler_node.py # Main "Sampler Grid" Node (401 lines)
├── dashboard_node.py # "Dashboard Viewer" Node (68 lines)
├── config_builder_node.py # "Config Builder" Node + API endpoints (1167 lines)
├── config_builder_node.py # "Config Builder" Node + API endpoints + state_to_configs_json (1578 lines)
├── json_text_node.py # "Smart JSON" Node (60 lines)
│
├── generation_orchestrator.py # The "Conductor" (Loop Logic, 2281 lines)
├── image_generation.py # KSampler & VAE Wrappers (748 lines)
├── image_metrics.py # Optional per-image metrics (sharpness/entropy/...), run context, scorer-pack adapters
├── metrics_runner.py # Background job: compute metrics for an existing session (CPU, manifest lock)
├── model_loader.py # Checkpoint/LoRA/VAE/GGUF Loading Logic (805 lines)
├── model_cache.py # 3-Tier Model Caching System (900 lines)
├── remote_vae.py # Remote VAE Offloading (340 lines)
├── generation_orchestrator.py # The "Conductor" (Loop Logic, job plan, preview pass, 3693 lines)
├── image_generation.py # KSampler & VAE Wrappers (1321 lines)
├── image_metrics.py # Optional per-image metrics (sharpness/entropy/...), run context, scorer-pack adapters (668 lines)
├── metrics_runner.py # Background job: compute metrics for an existing session (CPU, manifest lock) (167 lines)
├── model_loader.py # Checkpoint/LoRA/VAE/GGUF Loading Logic, latent geometry, CLIP type (907 lines)
├── model_cache.py # 3-Tier Model Caching System (935 lines)
├── remote_vae.py # Remote VAE decode worker + facade for the RemoteVAE companion (314 lines)
├── florence2_hires.py # Florence2 Hi-Res Fix: targeted-region inpaint, soft dependency (899 lines)
├── ltx_video_generation.py # LTX 2.3 video pipeline + _call_node helper for V3/V1 nodes (716 lines)
│
├── batch_encoding.py # Batch CLIP Encoding with Combinator Support (378 lines)
├── batch_encoding.py # Batch CLIP Encoding, Combinators, pinned_clip, Text-Encoder Batching (878 lines)
├── conditioning_cache.py # Disk-based Conditioning Cache (372 lines)
├── lora_utils.py # LoRA Searching & Validation (481 lines)
├── trigger_words.py # CivitAI Trigger Word Fetching & Prompt Assembly (377 lines)
├── config_utils.py # Config Parsing, Expansion & Cartesian Products (674 lines)
├── directory_scanner.py # External Directory Session Scanner (463 lines)
├── lora_utils.py # LoRA Searching & Validation (468 lines)
├── trigger_words.py # Trigger Word Prompt Assembly (377 lines)
├── config_utils.py # Config Parsing, Expansion, Cartesian Products, job plan + fingerprints (1108 lines)
├── directory_scanner.py # External Directory Session Scanner (609 lines)
├── metadata_packer.py # PNG/WebP Metadata Embedding for Exports (561 lines)
├── manifest_utils.py # Manifest Read/Write/Merge Helpers (120 lines)
├── network_utils.py # ALL outbound network requests (CivitAI, HuggingFace VAE, Distribution LAN) (280 lines)
├── upscale_runner.py # Dashboard async upscaling background thread (371 lines)
├── workflow_sweep.py # Workflow Sweep core: parse API workflow, bindings -> jobs, prompt substitution, fingerprints (pure, no torch/server)
├── workflow_sweep_runner.py # Workflow Sweep runner: ComfyBridge (in-process post_prompt mirror), windowed feeder thread, pause/resume/cancel, route helpers
├── workflow_sweep_collect.py # USCGSweepCollect output node: saves each job's image + manifest item into the USCG session
├── html_generator.py # Reads /resources/ to build the Dashboard HTML (57 lines)
├── run_api.py # Pure helpers for POST /config_tester/run + run_status (no server imports)
│
├── distribution_manager.py # Distributed Job Queue Coordinator/Master (634 lines)
├── distribution_worker.py # Worker Thread for Remote Processing (813 lines)
├── distribution_routes.py # Distribution REST API Endpoints (640 lines)
├── manifest_utils.py # Manifest Read/Write/Merge Helpers, comparisons (316 lines)
├── upscale_runner.py # Dashboard async upscaling background thread (642 lines)
├── civitai.py # Facade for the ComfyUI-USCG-CivitAI companion, no network code here (40 lines)
├── distribution.py # Facade for the ComfyUI-USCG-Distributed companion, no network code here (107 lines)
├── workflow_sweep.py # Workflow Sweep core: parse API workflow, bindings -> jobs, prompt substitution, fingerprints (pure, no torch/server) (969 lines)
├── workflow_sweep_runner.py # Workflow Sweep runner: ComfyBridge (in-process post_prompt mirror), windowed feeder thread, pause/resume/cancel, route helpers (912 lines)
├── workflow_sweep_collect.py # USCGSweepCollect output node: saves each job's image + manifest item into the USCG session (140 lines)
├── html_generator.py # Reads /resources/ to build the Dashboard HTML (66 lines)
├── run_api.py # Pure helpers for POST /config_tester/run + run_status, no server imports (144 lines)
├── conftest.py # Registry import stub: lets pytest import this package without ComfyUI (213 lines)
├── pytest.ini # --import-mode=importlib, testpaths, markers
│
├── web/ # [ComfyUI Integration Layer]
│ ├── dashboard.js # Registers Dashboard Node & Message Handling (432 lines)
│ ├── dashboard.js # Registers Dashboard Node & Message Handling (489 lines)
│ ├── smart_json_text.js # Registers JSON Node & Syntax Highlighting (239 lines)
│ ├── sampler_help_note.js # Help note on the Sampler Grid node (48 lines)
│ └── conf_builder/ # [Config Builder UI Modules]
│ ├── conf-builder-main.js # Main entry, widget registration (608 lines)
│ ├── conf-builder-ui-components.js # UI rendering (dropdowns, sliders, cards) (944 lines)
│ ├── conf-builder-config-management.js # Save/Load/Duplicate config logic (4975 lines)
│ ├── conf-builder-utilities.js # State-to-JSON conversion, iteration counting (958 lines)
│ └── conf-builder-distribution.js # Distribution settings UI (workers, toggles) (474 lines)
│ ├── conf-builder-main.js # Main entry, widget registration, default state + migrations (779 lines)
│ ├── conf-builder-ui-components.js # UI rendering (dropdowns, sliders, cards) (972 lines)
│ ├── conf-builder-config-management.js # Save/Load/Duplicate config logic, all Builder rendering (7736 lines)
│ ├── conf-builder-utilities.js # Model-list cache, iteration counting, LoRA string helpers (866 lines)
│ ├── conf-builder-distribution.js # Distribution settings UI (workers, toggles) (620 lines)
│ ├── conf-builder-workflow-sweep.js # Workflow Sweep UI (load API workflow, axes, dry run, start/pause/cancel) (744 lines)
│ └── conf-builder-readme.md # Notes on the Builder module split (134 lines)
│
├── resources/ # [The Dashboard SPA Core]
│ ├── template.html # HTML Skeleton for the generated report (662 lines)
│ ├── report.css # Styling (Infinite Canvas, Cards, Modals) (1642 lines)
│ ├── logic_init.js # Bootstrapper (Load JSON, Init State) (104 lines)
│ ├── logic_standalone.js # Standalone-page mode: own websocket, run-API GENERATE NEW, openStandalone() (loaded last)
│ ├── logic_state.js # State Store (Redux-style: filters, favorites) (104 lines)
│ ├── logic_pipeline.js # Data Processing (Filter -> Sort -> Layout) (510 lines)
│ ├── logic_ratings.js # Pairwise review pure logic: axis normalisers, Bradley-Terry, pairing, scoreboard (no DOM, node-testable)
│ ├── logic_compare.js # Pairwise review DOM glue: compare overlay, vote queue, scoreboard modal, rating getters
│ ├── logic_virtual.js # Virtual DOM / Infinite Scroller Engine (925 lines)
│ ├── logic_ui.js # UI Renderer (DOM creation, Modal handling) (1780 lines)
│ ├── logic_upscale_modal.js # Dashboard upscale modal (inline config, presets, async upscale) (380 lines)
│ ├── logic_events.js # Interaction Handler (Hotkeys, API calls) (91 lines)
│ └── logic_utils.js # Helpers (Export, Scan, Session loading) (599 lines)
├── resources/ # [The Dashboard SPA Core] (listed in html_generator.js_files order)
│ ├── template.html # HTML Skeleton for the generated report (1005 lines)
│ ├── report.css # Styling (Infinite Canvas, Cards, Modals) (2062 lines)
│ ├── logic_state.js # State Store (Redux-style: filters, favorites) (149 lines)
│ ├── logic_utils.js # Helpers (Export, Scan, Session loading) (1073 lines)
│ ├── logic_ratings.js # Pairwise review pure logic: axis normalisers, Bradley-Terry, pairing, scoreboard (no DOM, node-testable) (538 lines)
│ ├── logic_ui.js # UI Renderer (DOM creation, Modal handling) (2907 lines)
│ ├── logic_upscale_modal.js # Dashboard upscale modal (inline config, presets, async upscale) (1163 lines)
│ ├── logic_virtual.js # Virtual DOM / Infinite Scroller Engine (1092 lines)
│ ├── logic_pipeline.js # Data Processing (Filter -> Sort -> Layout) (831 lines)
│ ├── logic_compare.js # Pairwise review DOM glue: compare overlay, vote queue, scoreboard modal, rating getters (824 lines)
│ ├── logic_events.js # Interaction Handler (Hotkeys, API calls) (101 lines)
│ ├── logic_init.js # Bootstrapper (Load JSON, Init State) (112 lines)
│ └── logic_standalone.js # Standalone-page mode: own websocket, run-API GENERATE NEW, openStandalone() (loaded last, 170 lines)
│
├── tests/ # pytest suite (conftest stubs ComfyUI + torch, fixtures/ holds API-workflow samples)
├── tests/e2e/ # Playwright smoke test of the Dashboard SPA (own requirements.txt)
├── tests/js/ # node:test unit tests for pure Dashboard logic
├── examples/ # Ready-to-load workflows per model family (+ README with the files each needs)
├── .github/workflows/ # tests.yml (CI: pytest stubbed + torch, JS syntax, e2e), publish.yml
├── docs/plans/ # Design docs and implementation plans
├── pyproject.toml # Package metadata (36 lines)
├── requirements.txt # Python deps — diffusers, piexif
├── requirements.txt # Python deps: diffusers, piexif
├── .gitignore
├── README.md
├── Roadmap.md
└── SamplrConfig-Basic-Workflow.json # Example workflow for quick start
```
Distribution, CivitAI and Remote-VAE network code lives in the companion plugins (ComfyUI-USCG-Distributed, -CivitAI, -RemoteVAE); `distribution.py` and `civitai.py` are thin facades.
**Output Directory Structure (Generated at Runtime):**
```text
ComfyUI/output/
@@ -170,7 +180,7 @@ ComfyUI/output/
### **Node Class Mappings**
Defined in `__init__.py` at line 920:
Defined at the end of `__init__.py`:
| Internal Name | Python Class | Purpose |
|---|---|---|
@@ -178,6 +188,7 @@ Defined in `__init__.py` at line 920:
| `UltimateGridDashboard` | `SamplerConfigDashboardViewer` | Dashboard viewer — renders iframe with results |
| `UltimateConfigBuilder` | `UltimateConfigBuilder` | Visual config builder — outputs `configs_json` string |
| `SmartJSONText` | `SmartJSONTextNode` | JSON text widget with syntax highlighting |
| `USCGSweepCollect` | `USCGSweepCollect` | Workflow Sweep output node (`workflow_sweep_collect.py`): saves each job's image + manifest item into the USCG session |
`WEB_DIRECTORY = "./web"` — tells ComfyUI to serve `web/` files at `/extensions/`.
@@ -185,53 +196,67 @@ Defined in `__init__.py` at line 920:
### **API Routes Reference**
All routes registered via `PromptServer.instance.routes`.
All routes registered via `PromptServer.instance.routes`. One row per route this package registers; `tests/test_docs_consistency.py` fails when a registered route is missing here.
**Config Builder APIs** (registered in `__init__.py` and `config_builder_node.py`):
| Method | Route | Handler File | Line | Purpose |
| Method | Route | File | Body / query | Purpose |
|---|---|---|---|---|
| GET | `/configbuilder/list_configs` | `__init__.py` | 46 | List saved config preset filenames |
| POST | `/configbuilder/save_config` | `__init__.py` | 58 | Save config preset `{name, data}` |
| POST | `/configbuilder/load_config` | `__init__.py` | 108 | Load config preset `{name}` |
| GET | `/configbuilder/custom_resolutions` | `__init__.py` | 86 | Get custom resolution presets |
| POST | `/configbuilder/custom_resolutions` | `__init__.py` | 97 | Save custom resolution presets |
| POST | `/configbuilder/lookup_triggers` | `config_builder_node.py` | 742 | Bulk LoRA trigger word lookup `{loras: [...]}` |
| POST | `/configbuilder/lookup_lora_metadata` | `config_builder_node.py` | 768 | Full CivitAI metadata for LoRA `{lora_name}` |
| POST | `/configbuilder/lookup_model_metadata` | `config_builder_node.py` | 916 | Full CivitAI metadata for model `{model_name, model_type}` |
| GET | `/configbuilder/model_lists` | `config_builder_node.py` | 1029 | All model/sampler/scheduler/VAE/upscale lists |
| POST | `/config_tester/compute_metrics` | `__init__.py` | - | Start the background metrics job for a session (body: `{session_name, overwrite?}`) |
| GET | `/config_tester/metrics_status` | `__init__.py` | - | Job progress (`?job_id=`) |
| POST | `/config_tester/cancel_metrics` | `__init__.py` | - | Cancel a metrics job (body: `{job_id}`) |
| GET | `/configbuilder/metrics_info` | `config_builder_node.py` | - | Built-in metric names + optional scorer-pack install status (Run Settings hint) |
| POST | `/configbuilder/refresh_models` | `config_builder_node.py` | 1093 | Signal frontend cache clear |
| GET | `/configbuilder/get_lora_triggers` | `config_builder_node.py` | 1115 | Get triggers for single LoRA |
| POST | `/configbuilder/save_lora_triggers` | `config_builder_node.py` | 1138 | Save edited triggers for LoRA |
| GET | `/configbuilder/list_configs` | `__init__.py` | none | List saved config preset filenames (JSON array) |
| POST | `/configbuilder/save_config` | `__init__.py` | `{name, data}` | Save a config preset |
| POST | `/configbuilder/load_config` | `__init__.py` | `{name}` | Load a config preset |
| GET | `/configbuilder/custom_resolutions` | `__init__.py` | none | Get custom resolution presets |
| POST | `/configbuilder/custom_resolutions` | `__init__.py` | JSON body, stored as sent | Save custom resolution presets |
| GET | `/configbuilder/upscale_presets` | `__init__.py` | none | Upscale pipeline presets (`{"presets": []}` when none), shared by Builder and Dashboard |
| POST | `/configbuilder/upscale_presets` | `__init__.py` | JSON body, stored as sent (400 when empty) | Save upscale pipeline presets |
| GET | `/configbuilder/config_section_presets` | `__init__.py` | none | Config array presets (`{"presets": []}` when none) |
| POST | `/configbuilder/config_section_presets` | `__init__.py` | JSON body, stored as sent (400 when empty) | Save config array presets |
| GET | `/configbuilder/section_presets` | `__init__.py` | `?section=` one of `models`, `loras`, `prompts` (400 otherwise) | Presets for one Builder section |
| POST | `/configbuilder/section_presets` | `__init__.py` | `?section=` as above, JSON body stored as sent | Save presets for one Builder section |
| POST | `/configbuilder/preview` | `config_builder_node.py` | `{state}` (the Builder state; 400 when missing) | Runs `state_to_configs_json(state)`, returns `{configs_json}`; what the JSON Preview panel shows |
| POST | `/configbuilder/lookup_triggers` | `config_builder_node.py` | `{loras: [...]}` | Bulk LoRA trigger word lookup |
| POST | `/configbuilder/lookup_lora_metadata` | `config_builder_node.py` | `{lora_name, force_refresh?}` | Full CivitAI metadata for a LoRA (via the companion facade) |
| POST | `/configbuilder/lookup_model_metadata` | `config_builder_node.py` | `{model_name, model_type? (default checkpoint), force_refresh?}` | Full CivitAI metadata for a model |
| GET | `/configbuilder/model_lists` | `config_builder_node.py` | none | All model/sampler/scheduler/VAE/upscale lists |
| GET | `/configbuilder/model_counts` | `config_builder_node.py` | none | File counts per model category (fast, no listing) |
| GET | `/configbuilder/metrics_info` | `config_builder_node.py` | none | Built-in metric names + optional scorer-pack install status (Run Settings hint) |
| POST | `/configbuilder/refresh_models` | `config_builder_node.py` | none | Signal frontend cache clear |
| GET | `/configbuilder/get_lora_triggers` | `config_builder_node.py` | `?lora_name=` | Get triggers for a single LoRA |
| POST | `/configbuilder/save_lora_triggers` | `config_builder_node.py` | `{lora_name, triggers}` | Save edited triggers for a LoRA |
**Dashboard/Tester APIs** (registered in `__init__.py`):
| Method | Route | Line | Purpose |
|---|---|---|---|
| POST | `/config_tester/delete_session` | 142 | Delete session folder `{session_name}` |
| GET | `/config_tester/list_sessions` | 174 | List all sessions with metadata (name, count, thumbnail, mtime) |
| POST | `/config_tester/save_changes` | 234 | Delta save — only changed items `{session_name, changed_items}` |
| POST | `/config_tester/save_comparisons` | - | Pairwise-review votes `{session_name, append, remove}`; only touches manifest `comparisons` |
| POST | `/config_tester/clear_comparisons` | - | Delete all votes of a session `{session_name}` |
| POST | `/config_tester/save_manifest` | 308 | Full merge save `{session_name, manifest}` |
| POST | `/config_tester/get_session_html` | 390 | Generate dashboard HTML on-demand `{session_name, node_id}` |
| GET | `/config_tester/dashboard` | - | Dashboard SPA as a top-level page, optional `?session=` (standalone mode, `USCG_STANDALONE`, own websocket via `resources/logic_standalone.js`) |
| POST | `/config_tester/run` | - | Headless run: `{configs_json \| config_name, session_name?, overwrite_existing?, force? (default true), front?, client_id?}` -> builds Generator+Dashboard prompt, enqueues in-process like `/prompt`. Helpers in `run_api.py` |
| GET | `/config_tester/run_status` | - | `?prompt_id=` -> `pending/running/success/error` + session/dashboard_url/item_count |
| POST | `/config_tester/export_favorites` | 435 | Export favorited images with options (metadata, workflow, organize) |
| POST | `/config_tester/delete_non_favorites` | 662 | Delete all non-favorited images from session |
| POST | `/config_tester/scan_directory` | 746 | Scan external dir, create symlinks, generate manifest |
| POST | `/config_tester/sweep/parse` | - | Workflow Sweep: describe an API-format workflow `{api_json}` (nodes, outputs, image sources, suggested bindings) |
| POST | `/config_tester/sweep/dry_run` | - | Workflow Sweep: first N concrete prompts + validation of job 1 `{state or configs_json, limit}`; queues nothing |
| POST | `/config_tester/sweep/start` | - | Workflow Sweep: start a sweep `{state or configs_json, node_id?}` (400 invalid, 409 one already active) |
| GET | `/config_tester/sweep/status` | - | Workflow Sweep: `?sweep_id=` or `?session_name=` (state, counts, ETA, last errors) |
| POST | `/config_tester/sweep/pause`, `/resume`, `/cancel` | - | Workflow Sweep controls `{sweep_id}` |
| Method | Route | File | Body / query | Purpose |
|---|---|---|---|---|
| POST | `/config_tester/delete_session` | `__init__.py` | `{session_name}` | Delete a session folder |
| GET | `/config_tester/list_sessions` | `__init__.py` | none | List all sessions with metadata (name, count, thumbnail, mtime) |
| POST | `/config_tester/save_changes` | `__init__.py` | `{session_name, changed_items}` | Delta save: only the changed items |
| POST | `/config_tester/save_comparisons` | `__init__.py` | `{session_name, append: [...], remove: ["ts:a:b", ...]}` | Pairwise-review votes; only touches manifest `comparisons` |
| POST | `/config_tester/clear_comparisons` | `__init__.py` | `{session_name}` | Delete all votes of a session |
| POST | `/config_tester/save_manifest` | `__init__.py` | `{session_name, manifest}` | Full merge save (keeps server-side `meta` and `comparisons`) |
| POST | `/config_tester/get_session_html` | `__init__.py` | `{session_name, node_id}` | Generate the dashboard HTML on demand (empty manifest for an unknown session, not a 404) |
| GET | `/config_tester/dashboard` | `__init__.py` | `?session=` (optional) | Dashboard SPA as a top-level page (standalone mode, `USCG_STANDALONE`, own websocket via `resources/logic_standalone.js`) |
| POST | `/config_tester/run` | `__init__.py` | `{configs_json or config_name, session_name?, overwrite_existing?, force? (default true), front?, client_id?}` | Headless run: builds the Generator+Dashboard prompt and enqueues it in-process like `/prompt`. Helpers in `run_api.py` |
| GET | `/config_tester/run_status` | `__init__.py` | `?prompt_id=` | `pending`, `running`, `success` or `error` + session / dashboard_url / item_count |
| POST | `/config_tester/export_favorites` | `__init__.py` | `{session_name, pack_metadata?, organize_by_prompt?, organize_by_lora?, export_prompt_txt?, copy_manifest? (default true), pack_workflow?, pack_nodes_workflow?, workflow_data?, nodes_workflows?}` | Export favorited images with options (metadata, workflow, organize) |
| POST | `/config_tester/delete_non_favorites` | `__init__.py` | `{session_name}` | Delete all non-favorited images from a session |
| POST | `/config_tester/delete_rejected` | `__init__.py` | `{session_name}` | Delete all rejected images from a session (rejected preview fingerprints are remembered in `meta.rejected_preview_fps`) |
| POST | `/config_tester/scan_directory` | `__init__.py` | `{directory_path, session_name}` | Scan an external dir, create symlinks, generate a manifest |
| POST | `/config_tester/upscale_images` | `__init__.py` | `{session_name, image_ids, upscale_config: {pipelines: [...]}, all_favorited?}` | Start an async dashboard upscale job; returns `{job_id, total_images}` |
| GET | `/config_tester/upscale_status` | `__init__.py` | `?job_id=` | Progress of an upscale job |
| POST | `/config_tester/cancel_upscale` | `__init__.py` | `{job_id}` | Cancel an upscale job; returns `{cancelled}` |
| POST | `/config_tester/compute_metrics` | `__init__.py` | `{session_name, overwrite?}` | Start the background metrics job for a session; returns `{job_id, total_images}` |
| GET | `/config_tester/metrics_status` | `__init__.py` | `?job_id=` | Metrics job progress |
| POST | `/config_tester/cancel_metrics` | `__init__.py` | `{job_id}` | Cancel a metrics job; returns `{cancelled}` |
| POST | `/config_tester/sweep/parse` | `__init__.py` | `{api_json}` | Workflow Sweep: describe an API-format workflow (nodes, outputs, image sources, suggested bindings) |
| POST | `/config_tester/sweep/dry_run` | `__init__.py` | `{state or configs_json, limit?}` | Workflow Sweep: first N concrete prompts + validation of job 1; queues nothing |
| POST | `/config_tester/sweep/start` | `__init__.py` | `{state or configs_json, node_id?, client_id?}` | Workflow Sweep: start a sweep (400 invalid, 409 one already active) |
| GET | `/config_tester/sweep/status` | `__init__.py` | `?sweep_id=` or `?session_name=` | Workflow Sweep: state, counts, ETA, last errors (404 for an unknown sweep) |
| POST | `/config_tester/sweep/pause` | `__init__.py` | `{sweep_id}` | Workflow Sweep: pause (409 when not possible) |
| POST | `/config_tester/sweep/resume` | `__init__.py` | `{sweep_id}` | Workflow Sweep: resume (409 when not possible) |
| POST | `/config_tester/sweep/cancel` | `__init__.py` | `{sweep_id}` | Workflow Sweep: cancel (409 when not possible) |
**Distribution APIs** (registered in `distribution_routes.py`):
**Companion plugin routes** (registered by ComfyUI-USCG-Distributed, not by this package):
| Method | Route | Purpose |
|---|---|---|
@@ -356,21 +381,7 @@ All routes registered via `PromptServer.instance.routes`.
### **2b. The Backend: Distribution System**
* **`distribution_manager.py`** (634 lines)
* **Class:** `DistributionManager` — thread-safe distributed job queue on master
* **Job states:** PENDING → CLAIMED → COMPLETED/FAILED
* `populate_jobs()`, `claim_job(worker_id)`, `complete_job(job_id)`, `get_stats()`, `set_encoded_conditionings()`, `get_encoded_conditionings_for_job()`
* Thread-safe with `RLock`, timeout handling (default 600s), `MAX_FAIL_COUNT = 3`
* **`distribution_worker.py`** (813 lines)
* **Class:** `WorkerThread(threading.Thread)` — daemon thread on remote instances
* Polls `/uscg-distribution/claim_job` every 2s, processes locally, uploads results
* Caches model/CLIP/VAE between jobs, supports model sync from master
* Exits after 30 empty polls or 503 response
* **`distribution_routes.py`** (640 lines)
* Registers `/uscg-distribution/*` endpoints on PromptServer
* Global state: `_distribution_manager` (master), `_worker_thread` (worker)
The distribution code (job queue, worker thread, `/uscg-distribution/*` routes, model sync) lives in the ComfyUI-USCG-Distributed companion plugin, not in this package. `distribution.py` is a thin facade: it detects whether the companion is installed (`is_distribution_available()`) and forwards `create_manager()`, `notify_workers_to_start()`, `stop_all_workers()` and `get_master_url()` to it, raising a clear install hint when it is missing. `generation_orchestrator.py` imports only the facade. The companion calls `check_if_job_completed()` positionally, so that signature must not change (new flags are keyword-only).
---
@@ -378,7 +389,7 @@ All routes registered via `PromptServer.instance.routes`.
* **`dashboard.js`** (432 lines)
* Registers `UltimateGridDashboard` node
* Listens for `ultimate_grid.update`, `ultimate_grid.progress`, `ultimate_grid.distribution_status`
* Listens for `ultimate_grid.update`, `ultimate_grid.progress`, `ultimate_grid.distribution_status` (the only three `api.addEventListener("ultimate_grid.*")` calls in `web/`; the standalone page in `resources/logic_standalone.js` also handles `ultimate_grid.update_data`)
* Forwards events to matching iframe via `postMessage()`
* **Auto-loads on node creation** with 500ms delay via `forceLoadSession()`
* **Widgets:** "RELOAD / SHOW SESSION" button, "DELETE SESSION" button
@@ -392,7 +403,7 @@ All routes registered via `PromptServer.instance.routes`.
* **Node methods:** `saveState()`, `renderUI()`, `loadSession()`, `saveConfigToBackend()`, `loadConfigFromBackend()`, `triggerAutoSave()` (2s debounce)
* **`conf-builder-utilities.js`** (958 lines)
* `convertStateToConfigs()` (line 497) — JS-side config conversion (MUST stay in sync with Python `generate_config()`)
* `convertStateToConfigs()` was removed: the Builder preview POSTs the state to `/configbuilder/preview`, which runs Python's `state_to_configs_json` (no JS converter any more)
* `getIterationCount()` (line 457) — calculates total Cartesian product count
* `parseLoraString()` / `buildLoraString()` — LoRA string format parsing
* `refreshAllConfigBuilders()`, `clearAllCaches()`
@@ -643,7 +654,7 @@ iframe postMessage({type: 'toggle_fullscreen', node_id})
→ dashboard.js listener → toggle .dashboard-fullscreen class
```
**Distribution Job Flow (Master ↔ Worker):**
**Distribution Job Flow (Master ↔ Worker)** (the manager and worker named here live in the ComfyUI-USCG-Distributed companion):
```
Master:
1. generation_orchestrator → DistributionManager.populate_jobs()
@@ -685,7 +696,7 @@ The upscaling system allows testing multiple upscale configurations per generate
```
**Data Flow:**
1. JS `convertStateToConfigs()` embeds upscaling in `_session_settings`
1. Python `state_to_configs_json()` embeds upscaling in `_session_settings` (the JS preview gets the same output from `/configbuilder/preview`)
2. Python `config_builder_node.py` line 706-715 embeds into output JSON
3. `sampler_node.py` line 277 extracts `session_settings`
4. `generation_orchestrator.py` line 1096-1204 processes after each generation
@@ -742,16 +753,19 @@ sampler_node.py
→ conditioning_cache.py (ConditioningCache)
→ remote_vae.py (RemoteVAEDecodeWorker)
→ html_generator.py (get_html_template)
→ distribution_manager.py (DistributionManager)
→ distribution_worker.py (WorkerThread)
→ distribution.py (facade for the ComfyUI-USCG-Distributed companion)
→ image_metrics.py (optional per-image metrics)
config_builder_node.py
→ lora_utils.py (trigger word lookup, CivitAI API)
__init__.py
→ all node classes (sampler, dashboard, config_builder, json_text)
→ distribution_routes.py (distribution API endpoints)
→ html_generator.py (on-demand HTML generation)
→ run_api.py (POST /config_tester/run helpers)
→ metrics_runner.py (background metrics job)
→ workflow_sweep.py, workflow_sweep_runner.py, workflow_sweep_collect.py (Workflow Sweep)
→ upscale_runner.py (dashboard upscale job)
→ metadata_packer.py (export favorites)
→ directory_scanner.py (scan external directories)
→ manifest_utils.py (save/load manifests — note: NOT directly imported, uses inline logic)
@@ -763,8 +777,11 @@ __init__.py
| Event Name | Sender | Data Fields | Purpose |
|---|---|---|---|
| `ultimate_grid.update` | `generation_orchestrator.py` / `image_generation.py` | `session_name, node, manifest, meta, new_items` | New images generated |
| `ultimate_grid.progress` | `generation_orchestrator.py` | `session_name, node, progress_pct, eta_str, current_job, total_jobs, avg_duration, last_duration, finish_time` | ETA/progress bar. Optional `phase`: `upscaling` (deferred upscales) or `sweep` (Workflow Sweep; also `sweep_id, state, errors`; label "Sweep job") |
| `ultimate_grid.update` | `generation_orchestrator.py` / `image_generation.py` / `remote_vae.py` / `workflow_sweep_collect.py` | `session_name, node, manifest, meta, new_items` | New images generated |
| `ultimate_grid.update_data` | `generation_orchestrator.py`, `upscale_runner.py` | `session_name, new_items` (+ `node` from the orchestrator) | Per-item replacement: the listed items replace the cards with the same id (deferred upscales, dashboard upscales) |
| `ultimate_grid.progress` | `generation_orchestrator.py`, `upscale_runner.py`, `workflow_sweep_runner.py` | `session_name, node, progress_pct, eta_str, current_job, total_jobs, avg_duration, last_duration, finish_time` | ETA/progress bar. Optional `phase`: `upscaling` (deferred and dashboard upscales) or `sweep` (Workflow Sweep, label "Sweep job"). The sweep payload is `{node, session_name, phase: "sweep", sweep_id, state, errors, current_job, total_jobs, progress_pct, eta_str, finish_time, avg_duration, last_duration}` plus, on completion, `complete, total_elapsed, total_generated` |
| `ultimate_grid.next_run_seeds` | `generation_orchestrator.py` | `node, seeds` | Per-config seeds the NEXT run should use after a `random_after` / `increment_after` / `decrement_after` full-run seed behavior, as `{config_index: seed}` (no listener in this package) |
| `ultimate_grid.metrics_progress` | `metrics_runner.py` | `session_name, job_id, status, completed, total` | Progress of a "compute metrics for this session" job (no listener in this package; the SPA polls `/config_tester/metrics_status`) |
| `ultimate_grid.distribution_status` | `generation_orchestrator.py` | `session_name, workers, stats` | Distribution worker status |
---
@@ -791,16 +808,14 @@ __init__.py
> **GOLDEN RULE: DO NOT REMOVE ANY CODE. DO NOT REMOVE ANY COMMENTS. ONLY CHANGE WHAT IS NECESSARY.**
1. **Dual Data Path** — Frontend `convertStateToConfigs()` (in `conf-builder-utilities.js` line 497) and Python `generate_config()` (in `config_builder_node.py` line 465) are **independent implementations** that both transform `node.state` into config JSON. Both must stay in sync when adding new config fields or the UI preview will mismatch the actual output.
1. **Single Data Path**: `state_to_configs_json(state)` in `config_builder_node.py` is the only state-to-configs implementation; the JS preview calls `/configbuilder/preview`, which runs the same function. Do not add a JS-side converter. Adding a Builder/config field means ALL of: default in `conf-builder-main.js` new-state AND in its three migration blocks; default in the new-config-array object in `conf-builder-config-management.js`; UI control in the right render function; emission in `config_builder_node.state_to_configs_json`; key in `_EXPECTED_CONFIG_KEYS`; pass-through in `config_utils.expand_configs`; consumer in the orchestrator / `generate_image`; README bullet; unit tests. Bump `LS_CACHE_VERSION` in `conf-builder-utilities.js` only if the cached model-list shape changes.
2. **Widget Bridge** — The `lora_config` widget (STRING type) holds the **entire** node state as serialized JSON. Python `generate_config()` ignores all other widget params and reads only `lora_config`. The `samplers`, `schedulers`, `steps`, `cfg` widgets in `INPUT_TYPES` are vestigial (kept for ComfyUI schema compliance only).
2. **Widget Bridge** — The `lora_config` widget (STRING type) holds the **entire** node state as serialized JSON. Python `generate_config()` (which wraps `state_to_configs_json`) ignores all other widget params and reads only `lora_config`. The `samplers`, `schedulers`, `steps`, `cfg` widgets in `INPUT_TYPES` are vestigial (kept for ComfyUI schema compliance only).
3. **onNodeCreated Must Be Synchronous** — In `conf-builder-main.js` line 51, `onNodeCreated` MUST remain synchronous. Async initialization runs inside a fire-and-forget `(async () => { ... })()` IIFE. Making `onNodeCreated` itself async breaks widget registration.
4. **Resources Are Inlined** — `html_generator.py` reads all `/resources/` files and bakes them into one HTML string. Zero external HTTP requests. Changes to dashboard JS/CSS require re-generating the HTML (re-run the sampler node or call `/config_tester/get_session_html`).
> **Workflow Sweep never blocks the executor.** `USCGSweepCollect` and the Generator in workflow mode only start/record work; the runner feeds sweep prompts through `PromptServer.instance.prompt_queue` from its own thread (an in-process mirror of `server.py post_prompt`, loop-bound calls via `ComfyBridge.run_on_loop`, never from the loop thread, no lock held across it). A node that waited on a prompt it queued would deadlock ComfyUI's single executor thread. The expansion/substitution core (`workflow_sweep.py`) is pure; only `ComfyBridge` touches ComfyUI, so tests use a fake bridge.
5. **Manifest Merge on Save** — `manifest_utils.py` `save_manifest()` calls `merge_manifest_user_changes()` which reloads from disk before saving. This preserves user favorites/rejected/notes that changed concurrently while generation was running. Also, `flush_batch_with_vae()` (image_generation.py line 657) does its own disk sync.
6. **Optional Inputs Disable Caches** — `sampler_node.py` disables conditioning cache when `optional_model`/`optional_clip`/`optional_positive`/`optional_negative` are connected. `IS_CHANGED()` returns `NaN` (forces re-execution). `check_if_job_completed()` skips model/lora/prompt matching.
@@ -843,7 +858,7 @@ __init__.py
25. **Model Discovery Pattern** — `folder_paths.get_filename_list("key")` returns available models. API endpoint `/configbuilder/model_lists` returns all lists. JS side: `conf-builder-utilities.js` `getModelLists()` fetches and stores in global vars. `conf-builder-main.js` passes to renderers as `modelLists` object.
26. **Bypass States Are UI-Only** — `vae_bypass_states` and `te_bypass_states` control filtering in `convertStateToConfigs()` but should NOT appear in config output. They're internal UI state persisted only by `node.saveState()`. `lora_bypass_states` and `model_bypass_states` DO appear in output for smart-skip matching.
26. **Bypass States Are UI-Only** — `vae_bypass_states` and `te_bypass_states` control filtering in `state_to_configs_json()` but should NOT appear in config output. They're internal UI state persisted only by `node.saveState()`. `lora_bypass_states` and `model_bypass_states` DO appear in output for smart-skip matching.
27. **Manifest Save from Dashboard** — Two save mechanisms exist:
- `save_changes` (line 234 in `__init__.py`): Delta save — only changed items by ID. Fast. Used for individual star/reject.
@@ -853,6 +868,20 @@ __init__.py
29. **Image Metrics Run Context** - `image_metrics.begin_run(spec)` (wrapped around `run_generation_loop` in `sampler_node.py`) returns a token and `end_run(token)` closes only that context, so a second run or the dashboard upscale thread can never clobber it. Generation-side hooks read the active context (`decode_latent_with_vae(..., metrics_sink=)`, `flush_batch_with_vae`, orchestrator upscale inserts via `image_metrics.finalize`); the dashboard upscale thread (`upscale_runner.py`) must NOT read generation-run state, so it passes an explicit spec derived from the source item (`spec_from_item`). Every place that copies a source item into an upscaled entry excludes `"metrics"` (never inherit); the new entry is re-measured. `metrics` is session-level (`_session_settings.metrics`), not a per-config key.
30. **Workflow Sweep never blocks the executor.** `USCGSweepCollect` and the Generator in workflow mode only start/record work; the runner feeds sweep prompts through `PromptServer.instance.prompt_queue` from its own thread (an in-process mirror of `server.py post_prompt`, loop-bound calls via `ComfyBridge.run_on_loop`, never from the loop thread, no lock held across it). A node that waited on a prompt it queued would deadlock ComfyUI's single executor thread. The expansion/substitution core (`workflow_sweep.py`) is pure; only `ComfyBridge` touches ComfyUI, so tests use a fake bridge.
31. **pytest package-name trap**: pytest runs with `--import-mode=importlib` and a root `pytest.ini`. A parent folder named `integration`, `metrics` etc. (no hyphen) is imported as a package and the stub conftest breaks. Worktree folder names need a hyphen; never add `tests/__init__.py`.
32. **Manifest writers preserve disk state**: every writer (generation saves, the remote VAE worker, the `save_manifest` route) keeps the on-disk `comparisons` (`manifest_utils.preserve_disk_comparisons`) and the per-item `metrics` written by `metrics_runner` (an item's on-disk `metrics` survives when the in-memory copy has none, in `manifest_utils.merge_manifest_user_changes`). Only `update_manifest_comparisons` writes votes; only `metrics_runner` rewrites `metrics` in place.
33. **Flat job plan + `config_fingerprint`**: `run_generation_loop` walks a flat `job_plan` (`config_utils.build_job_plan`, one entry per job, binding `conf, conf_idx, job_idx, job, w, h, batch_idx`) instead of nested resolution x config loops; `order_job_plan` reorders it for `job_order: minimize_reloads` (stable sort on first-appearance ranks). `check_if_job_completed(..., preview=False)` keeps its positional signature (the distributed companion calls it positionally); the `preview` flag is keyword-only and previews/full items never match each other. `config_fingerprint()` is the identity of a grid cell (stored as `config_fp` on every new item; preview items add `preview`, `preview_of`, `full_*`; upscaled items add `base_fp`). Bump `FP_VERSION` (currently `"fp1"`, and keep the old reader) if you ever change its payload; a golden-value test pins it. Preview/promotion/job-order features are ignored by the distributed path.
34. **`pinned_clip`**: every loop that encodes more than one prompt runs inside `batch_encoding.pinned_clip(clip, sample_texts)` (the orchestrator's pre-encode groups, the per-job encode and the deferred-upscale re-encode). Nesting is safe.
35. **Dynamic-VRAM re-prepare**: on ComfyUI 0.4x (comfy_aimdo) `load_models_gpu` re-runs `ModelPatcherDynamic.load()` on every encode; the repeated "prepared for dynamic VRAM loading" log line is the symptom. TE batching (`te_batch_size`) self-checks batch 1 against a single encode and halves on OOM; non-batchable prompts keep the per-prompt path.
36. **`html_generator.js_files` order is load-bearing** (one shared script scope): `logic_ratings.js` (pure) before `logic_ui.js`, `logic_compare.js` after `logic_pipeline.js`, `logic_standalone.js` LAST (top-level-page overrides). CI assembles the page and syntax-checks it.
---
### **Testing Changes**
@@ -861,7 +890,7 @@ __init__.py
- **`web/` JS files:** Refresh browser (Ctrl+F5) — served via `/extensions/` with cache-busting
- **`resources/` JS/CSS/HTML files:** Dashboard HTML must be regenerated — re-run the sampler node or call `/config_tester/get_session_html`
- **Config Builder state fields:** Also update migration checks in `conf-builder-main.js` `onConfigure`
- **New config fields:** Update BOTH `convertStateToConfigs()` in JS AND `generate_config()` in Python
- **New config fields:** `state_to_configs_json()` in Python is the only converter (the JS preview calls `/configbuilder/preview`); then follow the checklist in gotcha 1
---
@@ -872,7 +901,7 @@ __init__.py
| File | Key Functions | Line Numbers |
|---|---|---|
| `sampler_node.py` | `run_tests()`, `find_existing_match()`, `get_latent_channels()`, `IS_CHANGED()` | 252, 201, 169, 103 |
| `config_builder_node.py` | `generate_config()`, `process_lora_array()`, `get_available_sessions()`, `expand_lora_folders()` | 465, 424, 360, 370 |
| `config_builder_node.py` | `state_to_configs_json()`, `generate_config()`, `preview_endpoint()`, `process_lora_array()`, `get_available_sessions()`, `expand_lora_folders()` | - |
| `generation_orchestrator.py` | `run_generation_loop()`, `check_if_job_completed()`, `setup_session_directories()`, `_preencode_all_conditionings()`, `_conditioning_to_serializable()`, `_run_distributed_generation()` | -, -, 43, -, -, - |
| `image_generation.py` | `generate_image()`, `create_image_metadata()`, `flush_batch_with_vae()`, `upscale_image()`, `decode_latent_with_vae()`, `calculate_eta()` | 16, 483, 605, 265, -, 540 |
| `batch_encoding.py` | `encode_prompt_with_combinators()`, `batch_encode_prompts()` | |
@@ -883,15 +912,19 @@ __init__.py
| `manifest_utils.py` | `load_existing_manifest()`, `save_manifest()`, `merge_manifest_user_changes()` | |
| `metadata_packer.py` | `pack_metadata_into_image()`, `extract_metadata_from_image()`, `calculate_file_hash()` | 181, -, - |
| `directory_scanner.py` | `scan_directory_for_images()`, `parse_a1111_parameters()` | |
| `distribution_manager.py` | `DistributionManager.populate_jobs()`, `.claim_job()`, `.complete_job()`, `.set_encoded_conditionings()` | |
| `distribution_worker.py` | `WorkerThread.run()`, `._process_job()`, `._download_model_from_master()` | |
| `distribution.py` | `is_distribution_available()`, `create_manager()`, `notify_workers_to_start()`, `stop_all_workers()`, `get_master_url()` (facade; the manager and worker live in the companion) | |
| `run_api.py` | `build_run_prompt()`, `normalize_configs_payload()`, `resolve_saved_config()`, `summarize_history()`, `RunTracker` | |
| `image_metrics.py` | `compute_metrics_numpy()`, `compute_metrics_torch()`, `begin_run()`, `end_run()`, `finalize()`, `adapter_status()` | |
| `metrics_runner.py` | `start_metrics_job()`, `get_metrics_status()`, `cancel_metrics_job()` | |
| `workflow_sweep.py` | `parse_api_workflow()`, `suggest_bindings()`, `parse_for_ui()` (pure; no torch or server imports) | |
| `workflow_sweep_runner.py` | `dry_run()`, `api_start()`, `start_sweep()`, `get_status()`, `pause()`, `resume()`, `cancel()`, `ComfyBridge` | |
**JavaScript (by file):**
| File | Key Functions |
|---|---|
| `conf-builder-main.js` | `ensureModulesLoaded()`, `node.saveState()`, `node.renderUI()`, `node.loadSession()`, `node.triggerAutoSave()` |
| `conf-builder-utilities.js` | `convertStateToConfigs()`, `getIterationCount()`, `parseLoraString()`, `buildLoraString()`, `refreshAllConfigBuilders()`, `getModelLists()` |
| `conf-builder-utilities.js` | `getIterationCount()`, `parseLoraString()`, `buildLoraString()`, `refreshAllConfigBuilders()`, `getModelLists()` |
| `conf-builder-config-management.js` | `renderUI()`, `updatePreview()`, `createConfigArrayElement()`, `createModelElement()`, `renderUpscalingSection()` |
| `conf-builder-ui-components.js` | `createSearchableSelect()`, `createSlider()`, `getStyles()` |
| `conf-builder-distribution.js` | `renderDistributionSection()`, `renderDistributionSettingsSection()` |
README.md
+130 -63
View File
@@ -62,25 +62,33 @@ If you'd rather build the graph yourself:
## 📑 Table of Contents
- [Quick Start](#-quick-start--the-3-nodes-you-need)
- [Key Features](#-key-features)
- [Powerful Grid Generation](#-powerful-grid-generation)
- [Distributed Processing (Multi-Machine)](#-distributed-processing-multi-machine)
- [Upscaling & Post-Processing](#-upscaling--post-processing)
- [Favorites Export & Workflow Packing](#-favorites-export--workflow-packing)
- [Interactive Dashboard (The "IDE")](#-interactive-dashboard-the-ide)
- [The "Revise & Generate" Workflow](#-the-revise--generate-workflow)
- [Curation & JSON Export](#-curation--json-export)
- [Config Builder Node (Visual UI For Creating Runs)](#-config-builder-node-visual-ui)
- [Basic Setup](#basic-setup)
- [Understanding the Interface](#understanding-the-interface)
- [Session Management](#session-management)
- [Config Management (Save/Load Presets)](#config-management-saveload-presets)
- [Config Arrays](#config-arrays)
- [Fast CivitAI Info Lookup](#fast-civitai-info-lookup)
- [Optional Companion Plugins](#optional-companion-plugins)
- [Installation](#-installation)
- [Getting Started](#getting-started)
- [The Nodes](#1-the-nodes)
- [Generator Node Parameters](#2-generator-node-parameters)
- [Core Settings](#core-settings)
- [Performance Settings (incl. Text-Encoder Batching)](#performance-settings)
- [Advanced Settings](#advanced-settings)
- [The JSON Configuration](#3-the-json-configuration)
- [Hybrid Inputs (Optional)](#4-hybrid-inputs-optional)
- [Config Builder Node (Visual UI For Creating Runs)](#-config-builder-node-visual-ui)
- [Run Settings](#run-settings)
- [Basic Setup](#basic-setup)
- [Understanding the Interface](#understanding-the-interface)
- [Session Management](#session-management)
- [Config Management (Save/Load Presets)](#config-management-saveload-presets)
- [Config Arrays](#config-arrays)
- [Fast CivitAI Info Lookup](#fast-civitai-info-lookup)
- [Basic Parameters](#basic-parameters)
- [Iteration Count Display](#iteration-count-display)
- [Config Array Controls](#config-array-controls)
@@ -95,15 +103,22 @@ If you'd rather build the graph yourself:
- [LoRA Stacking](#lora-stacking)
- [VAEs Section](#vaes-section)
- [Attention Modes](#attention-modes)
- [Skip Layer Guidance](#skip-layer-guidance-dit-models)
- [MOSS-Image, Flux.2, Krea 2 (16 px latent models)](#moss-image-flux2-krea-2-16-px-latent-models)
- [Model-Specific Prompts](#model-specific-prompts)
- [Recursive Cartesian Prompt Builder](#recursive-cartesian-prompt-builder)
- [Job Order & Preview Pass](#job-order--preview-pass-run-settings)
- [Image Metrics (Optional)](#image-metrics-optional)
- [Workflow Sweep](#workflow-sweep-sweep-any-comfyui-workflow)
- [Dashboard Interface](#-dashboard-interface)
- [Header Bar](#header-bar)
- [Toolbar](#toolbar)
- [Navigation & Controls](#navigation--controls)
- [Card Overlays](#card-overlays)
- [JSON Bars](#json-bars-bottom---horizontal-layout)
- [JSON Bars](#json-bars-bottom--horizontal-layout)
- [Revision Modal](#revision-modal)
- [Pairwise Review](#pairwise-review-blind-ab-compare--scoreboard)
- [Standalone Dashboard](#standalone-dashboard-phone--second-monitor)
- [Example Workflows](#-example-workflows)
- [Quick Quality Test](#quick-quality-test-40-images)
- [Multi-Model Comparison](#multi-model-comparison-9-images)
@@ -129,16 +144,22 @@ If you'd rather build the graph yourself:
- [Browser Compatibility](#browser-compatibility)
- [JSON Output Format Reference](#json-output-format-reference)
- [API Integration (Config Builder)](#api-integration-config-builder)
- [Run API (headless runs)](#run-api-headless-runs-discord-bot-session-hub-curl)
- [Dashboard, Review, Metrics & Sweep Routes](#dashboard-review-metrics--sweep-routes)
- [File Locations](#file-locations)
- [Keyboard Shortcuts](#keyboard-shortcuts)
- [Dashboard](#dashboard)
- [Compare mode](#dashboard-compare-mode-blind-pairwise-review)
- [Config Builder](#config-builder)
- [Distributed Processing](#-distributed-processing)
- [Setting Up Distribution](#setting-up-distribution)
- [Master Text Encoding](#master-text-encoding)
- [Model Sync](#model-sync)
- [Distribution Troubleshooting](#distribution-troubleshooting)
- [Upscaling & Post-Processing](#-upscaling--post-processing)
- [Favorites Export & Workflow Packing](#-favorites-export--workflow-packing)
- [Notes for AI Development](#-notes-for-ai-development)
- [Development / Tests](#development--tests)
- [Key Gotchas](#key-gotchas-for-ai-development)
- [Server Event Types](#server-event-types)
- [Changelog](#-changelog)
- [License](#-license)
- [Credits](#-credits)
@@ -187,6 +208,12 @@ If you'd rather build the graph yourself:
* **Configurable Label Mode:** Toggle overlay labels on dashboard cards showing model, sampler, scheduler, steps, CFG, and other parameters directly on each image.
* **GPU Cooldown:** Automatic pause between generations to prevent thermal throttling. Configurable interval, duration, and optional VRAM clearing.
* **Image Metrics (optional):** Run Settings checkbox that stores sharpness, entropy, colorfulness, contrast, luma and saturation for every saved image in the manifest (see [Image Metrics](#image-metrics-optional)).
* **Workflow Sweep:** sweep any ComfyUI workflow exported as API JSON. Bind any input as an axis, dry run it, then pause, resume or cancel; every image lands in a normal USCG session (see [Workflow Sweep](#workflow-sweep-sweep-any-comfyui-workflow)).
* **Job Order (Minimize reloads):** reorders the whole job list so each model and LoRA stack loads as few times as possible (see [Job Order & Preview Pass](#job-order--preview-pass-run-settings)).
* **Preview Pass (generator side):** renders the whole grid cheaply first; reject the losers and only the survivors render at full quality (see [Job Order & Preview Pass](#job-order--preview-pass-run-settings)).
* **Skip Layer Guidance:** Builder option that wraps ComfyUI's `SkipLayerGuidanceDiTSimple` for DiT models (see [Skip Layer Guidance](#skip-layer-guidance-dit-models)).
* **16 px latent models:** MOSS-Image, Flux.2 and Krea 2 render at the size you ask for, with the CLIP type picked for the detected family (see [MOSS-Image, Flux.2, Krea 2](#moss-image-flux2-krea-2-16-px-latent-models)).
* **Examples folder:** [`examples/`](examples/README.md) has one ready workflow per model family.
### 🌐 Distributed Processing (Multi-Machine)
* **Multi-Worker Distribution:** Split large grid runs across multiple ComfyUI instances. A master node coordinates job distribution while workers process images in parallel and upload results back.
@@ -232,6 +259,8 @@ If you'd rather build the graph yourself:
* **Scan External Directory:** Load image sessions from any folder (including other ComfyUI instances) into the dashboard for viewing and filtering. Reads PNG metadata for full config display.
* **Export Favorites:** Copy all favorited images to a dedicated export folder with optional CivitAI metadata packing for sharing.
* **Preview Pass support:** preview images carry an amber `PREVIEW` badge, a **Pass** filter row (Preview / Full, shown once previews exist) separates them, and a preview whose cell now has a full image is hidden by default (isolate `Preview` in the Pass row to see them). A bar under the header counts previews and offers **Full pass for survivors (N)**, which switches Preview Pass off in the matching Config Builder and queues the workflow (falls back to instructions if the ComfyUI frontend does not allow it).
* **Pairwise Review:** blind A/B compare mode (`C` key) with Bradley-Terry ratings and a per-setting scoreboard (see [Pairwise Review](#pairwise-review-blind-ab-compare--scoreboard)).
* **Standalone Dashboard + Run API:** open the dashboard as a normal web page (phone, second monitor) and start headless runs with `POST /config_tester/run` (see [Standalone Dashboard](#standalone-dashboard-phone--second-monitor) and [Run API](#run-api-headless-runs-discord-bot-session-hub-curl)).
### ⚡ The "Revise & Generate" Workflow
* **One-Click Revision:** Click "REVISE" on any image to open a detail view.
@@ -377,6 +406,7 @@ This suite consists of three nodes found under the `sampling/testing` category:
- Only MOSS-Image, Krea 2 and Qwen-Image encoders batch; every other encoder ignores it.
- Weighted prompts and `AND` / `CAT` / `AVG()` / `BREAK` prompts are still encoded one at a time.
- The first batch is self-checked against a single-prompt encode; on any mismatch the run falls back to one-by-one automatically. A CUDA OOM halves the batch and retries.
- The pre-encode summary line shows how many prompts were batched.
* **`flush_batch_every`**: Update dashboard every X images (0 = use VAE batch size).
- `0`: Wait until VAE batch is full
@@ -518,6 +548,8 @@ The Run Settings section of the Builder holds options that apply to the whole ru
* **Additional Random Seeds** (`add_random_seeds_to_gens`, 0-100): render this many extra images per config using extra seeds on top of the base seed. Handy for telling "the base model did that" from "the LoRA did that" by checking whether a quirk survives across seeds. The extra seeds are derived from the base seed, so the same base seed always gives the same extras (and resuming an interrupted run works). Contributed by DRuggeri (PR #21).
Covered in their own sections: [Job Order & Preview Pass](#job-order--preview-pass-run-settings), [Image Metrics](#image-metrics-optional), Text Encoder Batch Size ([`te_batch_size`](#performance-settings)).
### Basic Setup
1. **Add the Node:** Right-click on the canvas → `Add Node` → `sampling/testing` → `Ultimate Config Builder`
@@ -765,7 +797,7 @@ Set **Model Type = Diffusion Model**, pick the diffusion file (e.g. `moss_image_
* The CLIP Type list is read from your installed ComfyUI's `CLIPLoader`, so new model families appear without a plugin update. If you leave it at `stable_diffusion` for a diffusion model, the generator picks the CLIP type that matches the detected model family and logs it (MOSS-Image needs `moss_image` to get its chat template; with `stable_diffusion` ComfyUI builds a generic Qwen3.5 encoder).
* The empty latent follows the model's latent format: 16 px per latent cell for the Flux.2 family, 8 for SD / SDXL / Flux.1. Use sizes that are multiples of 16; a 832x1216 job renders at 832x1216. Earlier releases always used 8, which made 16x models render at double size and run ~10x slower than the stock workflows.
* Pre-encoding loads the text encoder once per pass instead of once per prompt (ComfyUI's dynamic VRAM loader re-prepares a model on every encode, which is the repeated `Model ... prepared for dynamic VRAM loading` log line). What remains is the encoder itself: the Qwen3.5-9B encoder costs roughly 0.8 s per prompt on a 16 GB card, the same as the stock workflow, so a 60-prompt grid spends about a minute encoding before the first image. SDXL's CLIPs take about 0.15 s per prompt, Krea 2 and Qwen-Image about 0.3 s. The pre-encode summary line reports the measured ms/prompt.
* Text-encoder batching (`te_batch_size`, default 1 = off) puts several prompts through one encoder forward, which amortises the encoder's fixed per-forward cost. Prompts that can't be batched (weighted, `AND`/`CAT`/`AVG()`/`BREAK`) and encoders other than MOSS-Image / Krea 2 / Qwen-Image keep the per-prompt path. The summary line shows how many prompts were batched.
* Text-encoder batching (`te_batch_size`, default 1 = off) puts several prompts through one encoder forward, which amortises the encoder's fixed per-forward cost; details in [`te_batch_size`](#performance-settings).
### Model-Specific Prompts
@@ -801,25 +833,6 @@ Both live in the **Run Settings** section of the Config Builder and default to t
* A cell is identified by a **config fingerprint** (model, LoRA stack, VAE, text encoders, sampler, scheduler, CFG, denoise, seed, raw prompts, full resolution, plus advanced-sampling/SLG/deep-shrink settings; *not* steps). Cells whose seed is re-randomised on every run (`random_before`, extra random seeds) cannot be matched to a preview and simply render again.
* ETA: preview and full jobs cost roughly 10x apart and the rolling average lags when slow models are grouped together; treat the ETA as approximate under Minimize.
### Workflow Sweep (sweep any ComfyUI workflow)
**Add Workflow Sweep** (next to Add Config Array) turns a config array into a sweep over *your own ComfyUI workflow*. Instead of USCG re-implementing a model family, the workflow you exported runs as-is through ComfyUI's own executor, one prompt per job, and every image lands in a normal USCG session (manifest, dashboard, favorites, filters, export).
1. In ComfyUI export the workflow with **Workflow > Export (API)**. Load that file (or paste it) in the Workflow Sweep section. Editor-format JSON (with `nodes` and `links`) is rejected with a hint.
2. **Image to collect**: the IMAGE socket that feeds the first Save Image is picked automatically; change it in the dropdown. Only the nodes upstream of that socket run (the workflow's own Save Image / Preview nodes are skipped unless you tick *Also run the workflow's own outputs*).
3. **Sweep axes**: click a *Suggested* chip or `＋ sweep` next to any input in the node list. Each axis is one input (or several that must share a value, e.g. a seed feeding two nodes) with a list of values; the job count is the product of all axes. Special axes: **Positive / Negative prompt** (values come from the config's Prompts section, with the usual groups and 1-to-1 pairing rule), **LoRA** (bind LoraLoader nodes as slots; `name:0.8`, weight arrays `name:[0.5,1.0]`, folders and `None` work as in standard configs), **Resolution** (`WxH` into two width/height inputs). A linked input can be replaced by a literal with `＋ override link`.
4. **Dry run** shows the first concrete prompts (template plus your substitutions plus the collector node), the validation result of job 1 and the exact job count, and lets you download a job as API JSON to compare by hand. **Start sweep** starts the run from the Builder; queueing the **Generator** node starts the same sweep and returns the dashboard immediately. **Pause / Resume / Cancel** control it, and a status line follows it.
How it behaves:
* Jobs are queued two at a time through ComfyUI's normal queue (so you can keep using ComfyUI, and nothing blocks the executor). Axes are ordered so loader values stay adjacent (checkpoint outermost, sampler settings innermost), which lets ComfyUI's cache keep models loaded between jobs. `--cache-none` or very tight RAM reloads models every job.
* Each job has a fingerprint of the exact substituted workflow. Starting the same sweep again (also after a ComfyUI restart) skips jobs already in the session; *Overwrite existing* re-runs them and replaces the old image.
* A failed job is recorded and the sweep continues; three failures in a row pause it. Pressing Cancel in ComfyUI during a job, or clearing the queue, pauses the sweep instead of silently finishing; Resume re-runs the affected jobs.
* Manifest items get the normal fields (model, sampler, scheduler, steps, cfg, seed, denoise, vae, size, prompts) read from the workflow, plus `wf_params` (your bound values), `wf_fingerprint`, `wf_sweep_id`. Dashboard REVISE is hidden for these items.
* Each sweep writes `benchmarks/<session>/workflow_sweeps/<sweep_id>.json` (templates, bindings, per-job status) so any executed prompt can be reconstructed.
* v1 limits: one image output per job (no video), one sweep at a time, a workflow sweep cannot share a Generator run with standard configs (use two Builders), no upscale/distribution/trigger-word features, and Revise does not apply.
---
### Image Metrics (Optional)
Run Settings has a **Compute Image Metrics** checkbox (default OFF). When it is on, every saved still image (base generations and upscales; videos are skipped) gets a `metrics` object in `manifest.json`, computed from the decoded image at save time (about 1-5 ms per image on the GPU, nothing when the box is unchecked):
@@ -843,6 +856,25 @@ Notes:
* **Older sessions:** the cog panel's **IMAGE METRICS** section has **Compute metrics for this session**, a background job (CPU only, no VRAM, about 50 ms per image) that measures every image without metrics, skips videos, shows progress and can be cancelled by clicking the button again. It rewrites only the `metrics` field under the manifest lock, so favorites and notes you change meanwhile survive. Do not run it on a session that is still generating.
* The setting is session-level (like GPU Cooldown), so it is NOT a per-config field: it lives in `_session_settings.metrics`, not in `_EXPECTED_CONFIG_KEYS` or `expand_configs`.
### Workflow Sweep (sweep any ComfyUI workflow)
**Add Workflow Sweep** (next to Add Config Array) turns a config array into a sweep over *your own ComfyUI workflow*. Instead of USCG re-implementing a model family, the workflow you exported runs as-is through ComfyUI's own executor, one prompt per job, and every image lands in a normal USCG session (manifest, dashboard, favorites, filters, export).
1. In ComfyUI export the workflow with **Workflow > Export (API)**. Load that file (or paste it) in the Workflow Sweep section. Editor-format JSON (with `nodes` and `links`) is rejected with a hint.
2. **Image to collect**: the IMAGE socket that feeds the first Save Image is picked automatically; change it in the dropdown. Only the nodes upstream of that socket run (the workflow's own Save Image / Preview nodes are skipped unless you tick *Also run the workflow's own outputs*).
3. **Sweep axes**: click a *Suggested* chip or `＋ sweep` next to any input in the node list. Each axis is one input (or several that must share a value, e.g. a seed feeding two nodes) with a list of values; the job count is the product of all axes. Special axes: **Positive / Negative prompt** (values come from the config's Prompts section, with the usual groups and 1-to-1 pairing rule), **LoRA** (bind LoraLoader nodes as slots; `name:0.8`, weight arrays `name:[0.5,1.0]`, folders and `None` work as in standard configs), **Resolution** (`WxH` into two width/height inputs). A linked input can be replaced by a literal with `＋ override link`.
4. **Dry run** shows the first concrete prompts (template plus your substitutions plus the collector node), the validation result of job 1 and the exact job count, and lets you download a job as API JSON to compare by hand. **Start sweep** starts the run from the Builder; queueing the **Generator** node starts the same sweep and returns the dashboard immediately. **Pause / Resume / Cancel** control it, and a status line follows it.
How it behaves:
* Jobs are queued two at a time through ComfyUI's normal queue (so you can keep using ComfyUI, and nothing blocks the executor). Axes are ordered so loader values stay adjacent (checkpoint outermost, sampler settings innermost), which lets ComfyUI's cache keep models loaded between jobs. `--cache-none` or very tight RAM reloads models every job.
* Each job has a fingerprint of the exact substituted workflow. Starting the same sweep again (also after a ComfyUI restart) skips jobs already in the session; *Overwrite existing* re-runs them and replaces the old image.
* A failed job is recorded and the sweep continues; three failures in a row pause it. Pressing Cancel in ComfyUI during a job, or clearing the queue, pauses the sweep instead of silently finishing; Resume re-runs the affected jobs.
* Manifest items get the normal fields (model, sampler, scheduler, steps, cfg, seed, denoise, vae, size, prompts) read from the workflow, plus `wf_params` (your bound values), `wf_fingerprint`, `wf_sweep_id`. Dashboard REVISE is hidden for these items.
* Each sweep writes `benchmarks/<session>/workflow_sweeps/<sweep_id>.json` (templates, bindings, per-job status) so any executed prompt can be reconstructed.
* v1 limits: one image output per job (no video), one sweep at a time, a workflow sweep cannot share a Generator run with standard configs (use two Builders), no upscale/distribution/trigger-word features, and Revise does not apply.
---
## 🖥️ Dashboard Interface
### Header Bar
@@ -913,6 +945,19 @@ Clicking **REVISE** on a card opens the studio view:
4. **Bottom:** "Related Variants" reel showing other images with the same seed.
5. **GENERATE NEW:** Queues the specific config you just edited.
### Pairwise Review (blind A/B compare + scoreboard)
The star is an absolute signal that says little about which *setting* wins. Compare mode records fast two-up votes and rolls them up per setting.
* **Start:** the two-rectangles button in the dashboard header, the `C` key, or the cog panel section **PAIRWISE REVIEW**. Votes go through the current filter and never include rejected items.
* **Pairing modes:** *Matched* (same prompt and seed, different settings), *Same settings, different seed* (noise-floor check), *One axis at a time* (exactly one of model, LoRA, sampler, scheduler, CFG, steps, denoise, resolution, prompt, upscale differs; optional axis focus) and *Random* (same media type). Pairs prefer items with few votes and never mix images with videos. When every unvoted pair is used up the queue repeats voted pairs and shows a badge.
* **Blind by default:** only "A" and "B" are shown, placement is randomised per pair. After a vote a short reveal (off / 1 s / 2 s) outlines the winner and shows only the settings that differ. Any key or tap during the reveal advances without voting. Tap or click an image to vote for it. Panes stack on portrait phones.
* **Ratings:** votes feed a Bradley-Terry model (ghost-player prior, 1500 = average). Sort the grid by **Rating**, or enable the **Rating** field in Label Mode to see `★rating · votes` on voted cards.
* **Scoreboard:** per axis, win % (ties count half) with 95% Wilson intervals and the mean rating of the items with that value. Only votes where exactly one axis differs are attributed (a hidden difference such as the VAE blocks attribution; the footer lists the most common extra differences). Rows with fewer than 3 pairs read "not enough data". Seed is a noise-floor block: if seeds alone "win" as much as settings do, the other numbers are noise. Copy JSON / Copy CSV export the board.
* **Storage:** votes live in the session `manifest.json` as a top-level `comparisons` list `[{a, b, winner, ts, mode, blind?}]` (`a`/`b` are item ids, `a` is the left/top item, `winner` is `a`, `b` or `tie`). Old manifests without the field work unchanged. Votes for items that were deleted stay on disk and are ignored. Every manifest writer (generation saves, the remote VAE worker, the legacy `save_manifest` route) keeps the on-disk `comparisons` untouched. A directory re-scan (`scan_directory`) regenerates item ids, so votes of a scanned session are not carried over.
* **Routes:** `POST /config_tester/save_comparisons` `{session_name, append: [...], remove: ["ts:a:b", ...]}` and `POST /config_tester/clear_comparisons` `{session_name}`. They only touch `comparisons`.
* **Caveat:** win % across mixed opponents is not a controlled effect size; pair choice is biased by the filter and the low-vote preference. Use it to rank candidates, then confirm with a matched one-axis run.
### Standalone Dashboard (phone / second monitor)
The dashboard is also served as a normal web page, with the same live updates as the node:
@@ -1313,6 +1358,8 @@ The Config Builder generates JSON in the exact format required by the Ultimate S
For advanced users, the Config Builder exposes several backend endpoints:
Dashboard, review, metrics and sweep routes are listed at the end of this section; the full route table (method, body, purpose) is in [ProjectStructure.md](ProjectStructure.md).
### `/configbuilder/lookup_triggers` (POST)
Bulk lookup trigger words for multiple LoRAs.
@@ -1410,6 +1457,8 @@ Scan an external directory for image sessions and load them into the dashboard.
### Distribution API (registered via `distribution_routes.py`)
Provided by the ComfyUI-USCG-Distributed companion plugin.
These endpoints are used internally by the distribution system:
| Method | Route | Purpose |
@@ -1449,6 +1498,25 @@ Response: `{"prompt_id", "number", "session_name", "dashboard_path", "dashboard_
Bot pattern: `POST /run`, post `dashboard_url` to the channel, poll `run_status` every 10-30 s (or pass `client_id` and listen on `/ws?clientId=...` for `ultimate_grid.progress`). `/run` is unauthenticated, the same exposure as ComfyUI's `/prompt`.
### Dashboard, Review, Metrics & Sweep Routes
| Method | Route | Purpose |
|---|---|---|
| POST | `/configbuilder/preview` | Body `{state}` (the Builder state); returns `{configs_json}` from the same function the node uses (what the JSON Preview panel shows) |
| GET | `/config_tester/dashboard` | The Dashboard as a standalone page, optional `?session=` |
| POST | `/config_tester/save_comparisons` | Append / remove Pairwise Review votes `{session_name, append, remove}` |
| POST | `/config_tester/clear_comparisons` | Delete every vote of a session `{session_name}` |
| POST | `/config_tester/compute_metrics` | Start the background metrics job for a session `{session_name, overwrite?}`; returns `{job_id, total_images}` |
| GET | `/config_tester/metrics_status` | Progress of a metrics job, `?job_id=` |
| POST | `/config_tester/cancel_metrics` | Cancel a metrics job `{job_id}` |
| POST | `/config_tester/sweep/parse` | Workflow Sweep: describe an API-format workflow `{api_json}` |
| POST | `/config_tester/sweep/dry_run` | Workflow Sweep: first concrete prompts and validation of job 1 `{state or configs_json, limit?}`; queues nothing |
| POST | `/config_tester/sweep/start` | Workflow Sweep: start a sweep `{state or configs_json, node_id?, client_id?}` (409 when one is already active) |
| GET | `/config_tester/sweep/status` | Workflow Sweep: state, counts and errors, `?sweep_id=` or `?session_name=` |
| POST | `/config_tester/sweep/pause` | Workflow Sweep: pause `{sweep_id}` |
| POST | `/config_tester/sweep/resume` | Workflow Sweep: resume `{sweep_id}` |
| POST | `/config_tester/sweep/cancel` | Workflow Sweep: cancel `{sweep_id}` |
---
## File Locations
@@ -1486,19 +1554,6 @@ ComfyUI/output/benchmarks/model-data/
---
## Pairwise Review (blind A/B compare + scoreboard)
The star is an absolute signal that says little about which *setting* wins. Compare mode records fast two-up votes and rolls them up per setting.
* **Start:** the two-rectangles button in the dashboard header, the `C` key, or the cog panel section **PAIRWISE REVIEW**. Votes go through the current filter and never include rejected items.
* **Pairing modes:** *Matched* (same prompt and seed, different settings), *Same settings, different seed* (noise-floor check), *One axis at a time* (exactly one of model, LoRA, sampler, scheduler, CFG, steps, denoise, resolution, prompt, upscale differs; optional axis focus) and *Random* (same media type). Pairs prefer items with few votes and never mix images with videos. When every unvoted pair is used up the queue repeats voted pairs and shows a badge.
* **Blind by default:** only "A" and "B" are shown, placement is randomised per pair. After a vote a short reveal (off / 1 s / 2 s) outlines the winner and shows only the settings that differ. Any key or tap during the reveal advances without voting. Tap or click an image to vote for it. Panes stack on portrait phones.
* **Ratings:** votes feed a Bradley-Terry model (ghost-player prior, 1500 = average). Sort the grid by **Rating**, or enable the **Rating** field in Label Mode to see `★rating · votes` on voted cards.
* **Scoreboard:** per axis, win % (ties count half) with 95% Wilson intervals and the mean rating of the items with that value. Only votes where exactly one axis differs are attributed (a hidden difference such as the VAE blocks attribution; the footer lists the most common extra differences). Rows with fewer than 3 pairs read "not enough data". Seed is a noise-floor block: if seeds alone "win" as much as settings do, the other numbers are noise. Copy JSON / Copy CSV export the board.
* **Storage:** votes live in the session `manifest.json` as a top-level `comparisons` list `[{a, b, winner, ts, mode, blind?}]` (`a`/`b` are item ids, `a` is the left/top item, `winner` is `a`, `b` or `tie`). Old manifests without the field work unchanged. Votes for items that were deleted stay on disk and are ignored. Every manifest writer (generation saves, the remote VAE worker, the legacy `save_manifest` route) keeps the on-disk `comparisons` untouched. A directory re-scan (`scan_directory`) regenerates item ids, so votes of a scanned session are not carried over.
* **Routes:** `POST /config_tester/save_comparisons` `{session_name, append: [...], remove: ["ts:a:b", ...]}` and `POST /config_tester/clear_comparisons` `{session_name}`. They only touch `comparisons`.
* **Caveat:** win % across mixed opponents is not a controlled effect size; pair choice is biased by the filter and the low-vote preference. Use it to rank candidates, then confirm with a matched one-axis run.
## Keyboard Shortcuts
### Dashboard
@@ -1511,7 +1566,7 @@ The star is an absolute signal that says little about which *setting* wins. Comp
| `0` | Reset zoom to 1:1 |
| `F` | Auto-fit first row to viewport width |
| `Escape` | Close active modal/popup (priority: Revision > Scoreboard > Analytics > Settings > Filters) |
| `C` | Open Compare mode (blind pairwise review, see below) |
| `C` | Open Compare mode (blind pairwise review, see [Pairwise Review](#pairwise-review-blind-ab-compare--scoreboard)) |
### Dashboard: Compare mode (blind pairwise review)
| Shortcut | Action |
@@ -1609,28 +1664,20 @@ This is a **hybrid application** with three layers:
### Development / Tests
```bash
# Python unit tests (ComfyUI + torch are stubbed): needs pytest, numpy, pillow
pip install -r tests/requirements-ci.txt
python -m pytest tests/ -q -ra --ignore=tests/e2e # 137 passed, 15 skipped without torch
# With real torch (e.g. ComfyUI's python), the 15 `requires_torch` tests run too.
# USCG_REQUIRE_TORCH=1 makes them fail instead of skip if torch is missing:
USCG_REQUIRE_TORCH=1 python -m pytest tests/ -q -ra --ignore=tests/e2e # 152 passed
# Dashboard SPA browser smoke test (Playwright, Chromium, no ComfyUI needed)
pip install -r tests/e2e/requirements.txt
python -m playwright install chromium
python -m pytest tests/e2e -q -ra
python -m pytest tests/ -q -ra --ignore=tests/e2e # Python tests, ComfyUI + torch stubbed
node --test tests/js/ # Dashboard JS unit tests
python -m pytest tests/e2e -q -ra --browser chromium # Dashboard SPA smoke test (needs tests/e2e/requirements.txt)
```
GitHub Actions (`.github/workflows/tests.yml`) runs four jobs on every push and PR: `pytest-stubs`, `pytest-torch` (CPU torch), `js-syntax` (`node --check` on `web/` and `resources/`, the assembled Dashboard page, and `node --test tests/js/` when that folder exists) and `e2e-dashboard`.
Full instructions, the CI jobs and the pre-PR checklist: [CONTRIBUTING.md](CONTRIBUTING.md#running-tests).
Tests live in `tests/` and must not contain network or process-spawning code (no `socket`, `http.server`, `subprocess`, `urllib`): `tests/` ships in the registry zip and is yara-scanned.
### Key Gotchas for AI Development
1. **Config Builder output is ALWAYS wrapped** — `config_builder_node.py` outputs `{"configs": [...], "_distribution": {...}}`. The `sampler_node.py` MUST unwrap the `"configs"` key before passing to `expand_configs()`. If you add new top-level keys to the config output, update the extraction logic in `sampler_node.py` lines 269-286.
2. **Two independent config-to-JSON implementations** — `convertStateToConfigs()` in `conf-builder-utilities.js` (JS preview) and `generate_config()` in `config_builder_node.py` (Python output) both transform `node.state` into config JSON. **Both must stay in sync** when adding new config fields.
2. **One config-to-JSON implementation**: `UltimateConfigBuilder.state_to_configs_json` in `config_builder_node.py` is the single source of truth. The Builder preview POSTs the state to `/configbuilder/preview`, which runs the same function; there is no JS converter any more.
3. **Dashboard runs in an iframe** — The dashboard HTML is set via `iframe.srcdoc`. All JS/CSS is inlined (no external requests). Communication between the parent ComfyUI page and the iframe uses `postMessage()`. The `dashboard.js` file bridges server events to iframe messages.
@@ -1652,7 +1699,9 @@ Tests live in `tests/` and must not contain network or process-spawning code (no
12. **ComfyUI Registry security** — No `import requests`, no `subprocess`/`eval`/`exec`, no custom file-serving endpoints. Use `urllib.request` for HTTP and ComfyUI's `/view` endpoint for serving files. See `ProjectStructure.md` for full rules.
13. **Job plan and config fingerprint** — `run_generation_loop` walks a flat `job_plan` (`config_utils.build_job_plan`, one entry per job, binding `conf, conf_idx, job_idx, job, w, h, batch_idx`) instead of nested resolution x config loops; `order_job_plan` reorders it for `job_order: minimize_reloads` (stable sort on first-appearance ranks). `check_if_job_completed(..., preview=False)` keeps its positional signature (the distributed companion calls it positionally); the new `preview` flag is keyword-only and previews/full items never match each other. `config_fingerprint()` is the identity of a grid cell (stored as `config_fp` on every new item; preview items add `preview`, `preview_of`, `full_*`; upscaled items add `base_fp`). Bump `FP_VERSION` (and keep the old reader) if you ever change its payload; a golden-value test pins it. The main-loop body still sits under an `if True:` only to keep its indentation (diff size) after flattening. Preview/promotion/job-order features are ignored by the distributed path.
13. **Job plan and config fingerprint** — `run_generation_loop` walks a flat `job_plan` (`config_utils.build_job_plan`, one entry per job, binding `conf, conf_idx, job_idx, job, w, h, batch_idx`) instead of nested resolution x config loops; `order_job_plan` reorders it for `job_order: minimize_reloads` (stable sort on first-appearance ranks). `check_if_job_completed(..., preview=False)` keeps its positional signature (the distributed companion calls it positionally); the new `preview` flag is keyword-only and previews/full items never match each other. `config_fingerprint()` is the identity of a grid cell (stored as `config_fp` on every new item; preview items add `preview`, `preview_of`, `full_*`; upscaled items add `base_fp`). Bump `FP_VERSION` (and keep the old reader) if you ever change its payload; a golden-value test pins it. Preview/promotion/job-order features are ignored by the distributed path.
14. **More gotchas live in ProjectStructure.md**: the full list (36 gotchas: manifest writers, pinned_clip, sweeps, js_files order, ...) is in ProjectStructure.md, Critical Development Gotchas.
### File Dependencies (Import Graph)
@@ -1669,16 +1718,19 @@ sampler_node.py
→ conditioning_cache.py (ConditioningCache)
→ remote_vae.py (RemoteVAEDecodeWorker)
→ html_generator.py (get_html_template)
→ distribution_manager.py (DistributionManager)
→ distribution_worker.py (WorkerThread)
→ distribution.py (companion facade; the distribution code lives in ComfyUI-USCG-Distributed)
→ image_metrics.py (optional per-image metrics)
config_builder_node.py
→ lora_utils.py (trigger word lookup)
__init__.py
→ all node classes
→ distribution_routes.py (distribution API endpoints)
→ html_generator.py (on-demand HTML generation)
→ run_api.py (POST /config_tester/run helpers)
→ metrics_runner.py (background metrics job)
→ workflow_sweep.py, workflow_sweep_runner.py, workflow_sweep_collect.py (Workflow Sweep)
→ upscale_runner.py (dashboard upscale job)
→ metadata_packer.py (export favorites)
→ directory_scanner.py (scan external directories)
→ manifest_utils.py (save/load manifests)
@@ -1689,7 +1741,10 @@ __init__.py
| Event Name | Sender | Data | Purpose |
|---|---|---|---|
| `ultimate_grid.update` | `generation_orchestrator.py` | `{session_name, node, manifest, meta, new_items}` | New images generated |
| `ultimate_grid.progress` | `generation_orchestrator.py` | `{session_name, progress_pct, eta_str, ...}` | ETA/progress updates |
| `ultimate_grid.update_data` | `generation_orchestrator.py`, `upscale_runner.py` | per-item replacement payload | An existing item changed (for example a preview replaced by its full image) |
| `ultimate_grid.progress` | `generation_orchestrator.py` | `{session_name, progress_pct, eta_str, ...}` | ETA/progress updates. Optional `phase`: `upscaling` (deferred upscales) or `sweep` (Workflow Sweep, which adds `sweep_id, state, errors`) |
| `ultimate_grid.next_run_seeds` | `generation_orchestrator.py` | `{node, seeds}` | Seeds the next run will use |
| `ultimate_grid.metrics_progress` | `metrics_runner.py` | `{session_name, job_id, status, completed, total}` | Progress of a "compute metrics for this session" job |
| `ultimate_grid.distribution_status` | `generation_orchestrator.py` | `{session_name, workers, stats}` | Distribution status |
### Testing Changes
@@ -1700,6 +1755,18 @@ After modifying backend Python files, restart ComfyUI. After modifying `web/` JS
## 📝 Changelog
### Update 11/4/26: Standalone dashboard, pairwise review, image metrics, preview pass, workflow sweeps
* 📱 **Standalone Dashboard + Run API:** open the dashboard as a normal web page at `/config_tester/dashboard?session=<name>` (phone, second monitor) with live updates over its own websocket. `POST /config_tester/run` and `GET /config_tester/run_status` start and track headless runs (Discord bot, curl) without any workflow JSON.
* ⚖️ **Pairwise Review:** blind A/B compare mode (`C` key) with matched, same-settings, one-axis and random pairing, Bradley-Terry ratings (Sort by Rating) and a per-setting scoreboard with 95% intervals. Votes are stored in the session manifest under `comparisons`.
* 📏 **Image Metrics (optional):** a Run Settings checkbox stores sharpness, entropy, colorfulness, contrast, luma and saturation for every saved image. Sort, filter and label by them in the Dashboard, or compute them for older sessions from the cog panel.
* 🧭 **Job Order & Preview Pass:** `Minimize model/LoRA reloads` reorders the whole job list so each model and LoRA stack loads as few times as possible. Preview Pass renders the whole grid cheaply first; reject the losers and only the survivors render at full quality (PREVIEW badge, Pass filter, "Full pass for survivors" button).
* 🔀 **Workflow Sweep:** sweep any ComfyUI workflow exported as API JSON. Bind any input as an axis (prompts, LoRAs, resolution, seeds, any widget), dry run it, then pause, resume or cancel; every image lands in a normal USCG session.
* ⚡ **Faster prompt encoding:** the text encoder now loads once per pre-encode pass instead of once per prompt, and the new `te_batch_size` setting encodes several prompts per pass for MOSS-Image, Krea 2 and Qwen-Image (off by default, self-checked, OOM-safe).
* 🧩 **Skip Layer Guidance:** Builder option that wraps ComfyUI's `SkipLayerGuidanceDiTSimple` for DiT models (MOSS-Image defaults: block 22, 0-50%).
* 🟦 **MOSS-Image / Flux.2 / Krea 2 fixes:** the empty latent follows the model's latent format (16 px per cell for the Flux.2 family), so these models render at the size you asked for and at stock-workflow speed. The CLIP type is picked for the detected family (`moss_image`).
* 📂 **Examples:** [`examples/`](examples/README.md) has one ready workflow per model family (SDXL/Illustrious, Anima, Z-Image Turbo, Krea 2, Flux.2 Klein 9B, MOSS-Image).
* 🧪 **Tests & CI:** GitHub Actions runs the Python tests (stubbed and with real CPU torch), JS syntax checks, JS unit tests and a Playwright smoke test of the Dashboard on every push and PR.
### Update 11/3/26 — Additional Random Seeds finished, LTX prompt arrays
* 🌱 **Additional Random Seeds:** Run Settings control (0-100) now has full defaults/migration and tests; extra seeds are deterministic per base seed. Thanks to DRuggeri (PR #21).
* 🎬 **LTX Builder prompts:** Builder prompt arrays (`[["p1","p2"]]`) now expand into separate LTX runs instead of being passed as a list.
Roadmap.md
+24 -1
View File
@@ -129,7 +129,7 @@ A button in the builder UI to pack short sha256 into config with an explanation
* **Problem:** Can only load full sessions, not merge snippets.
#### **23. Pseudo-JSON Nodes (Recursion)** (low priority) - STARTED: v1 is the **Workflow Sweep** (Builder: Add Workflow Sweep): any API-format workflow runs as-is with chosen inputs swept (see README, 'Workflow Sweep'). Still open: inserting nodes into the USCG pipeline itself, video outputs, mixed workflow + standard runs.
#### **23. Pseudo-JSON Nodes (Recursion)** (low priority) - STARTED: v1 is the **Workflow Sweep** (Builder: Add Workflow Sweep): any API-format workflow runs as-is with chosen inputs swept (see README, 'Workflow Sweep'). Still open: inserting nodes into the USCG pipeline itself, video outputs, mixed workflow + standard runs. (v1 shipped 11/4/26)
* **Problem:** Advanced. Running a raw JSON workflow as a sub-node. Inserting any node into any part of the genereation would make this tool have an immense customizability increase.
More info needed on how this could work, would like to see a visual interface for it in the builder UI eventaully. (big job, very low priority)
@@ -150,19 +150,42 @@ Compute objective quality metrics during generation and store in manifest:
- **Sobel edge quality** (detail preservation)
Add as sortable/filterable fields in Dashboard. Enables data-driven parameter selection instead of purely visual comparison. Prior art: SamplerSchedulerMetricsTester (proves the concept, only 9 stars but validates demand).
DONE (11/4/26): built-in metrics (sharpness, entropy, colorfulness, contrast, luma, saturation), Dashboard sort/filter/labels, backfill job for old sessions. Still open: Sobel edge quality, scorer packs.
#### **26. Analytics Panel in Dashboard** (transforms tool from "look at pictures" to "data-driven decisions")
* **Milestone 1:** Aggregate stats bar — avg generation time by sampler/scheduler, favorites rate by model/LoRA
* **Milestone 2:** Canvas-based charts — bar charts, scatter plots (steps vs duration, CFG vs quality metrics)
* **Milestone 3:** Export — downloadable reports with charts + top images
Pure client-side using canvas API from existing `activeData`. Prior art: W&B dashboards.
STARTED (11/4/26): the Pairwise Review scoreboard (per-axis win % + ratings) covers part of Milestone 1.
#### **27. Static Dashboard** (eliminate HTML regeneration requirement)
* **Milestone 1:** REST manifest fetch — Dashboard loads manifest via `fetch('/config_tester/get_manifest?session=name')` instead of inline `__JSON_DATA__`
* **Milestone 2:** Remove html_generator dependency — serve Dashboard as static HTML from `WEB_DIRECTORY`
* **Milestone 3:** Hot-reload in dev — JS changes visible on browser refresh without re-running sampler
Biggest architectural leverage: eliminates the #1 developer friction point.
STARTED (11/4/26): `/config_tester/dashboard` serves the SPA as a standalone page with its own websocket; the manifest is still inlined by html_generator (Milestones 1-3 open).
#### **28. Onboarding Wizard & Tutorial**
* **Milestone 1:** First-run wizard in Builder UI — walks through: create session → pick model → add LoRA → run → view dashboard (5 interactive steps)
* **Milestone 2:** Video tutorial (3-minute YouTube walkthrough)
README is 2000+ words; most users won't read it. Efficiency Nodes grew via community tutorials.
---
## Open follow-ups (11/4/26)
#### **29. Text-Encoder Batching: live check, then flip the default**
Verify `te_batch_size` on the 5060 Ti with the real MOSS-Image, Krea 2 and Qwen-Image encoders, then decide whether the default of 1 (off) should change.
#### **30. Distributed companion parity**
Preview pass, `job_order` and metrics are ignored in distributed mode today. Teach the ComfyUI-USCG-Distributed companion about them.
#### **31. Scorer-pack adapters**
Verify the node ids of the optional aesthetic / reward scorer packs against the real packs, then enable the shipped adapter entries.
#### **32. Workflow Sweep: video outputs and mixed runs**
Allow video outputs from a sweep and sweep + standard configs in one Generator run (both are v1 limits).