nodes/JasonHoku-ComfyUI-Ultimate-Auto-Sampler-Config-Grid-Testing-Suite
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update README.md and ProjectStructure.md for v3.0

Browse Source 
- Add distributed processing features (multi-worker, master text
  encoding, model sync, per-worker logging)
- Add comprehensive AI development notes with architecture diagrams,
  file dependency graphs, gotchas, and testing instructions
- Update file structure to include all 21 Python modules, distribution
  files, and conf-builder-distribution.js
- Add distribution API routes reference
- Add distribution module descriptions and communication patterns
- Update data flow to include distribution pipeline
- Add 8 new critical development gotchas (13-20)
- Add new changelog entry for March 2026
- Bump version to 3.0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
JasonHoku
2026-03-06 01:08:34 -08:00
co-authored by Claude Opus 4.6
parent 25d7c9e7b2
commit 989da199a3
2 changed files with 479 additions and 36 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ProjectStructure.md
+248 -22
View File
@@ -36,9 +36,9 @@ ComfyUI-Ultimate-Sampler-Grid/
├── config_builder_node.py # "Config Builder" Node
├── json_text_node.py # "Smart JSON" Node
│
├── generation_orchestrator.py # The "Conductor" (Loop Logic)
├── generation_orchestrator.py # The "Conductor" (Loop Logic, ~1700 lines)
├── image_generation.py # KSampler & VAE Wrappers
├── model_loader.py # Checkpoint/LoRA Loading Logic
├── model_loader.py # Checkpoint/LoRA/VAE/GGUF Loading Logic
├── model_cache.py # 3-Tier Model Caching System
├── remote_vae.py # Remote VAE Offloading
│
@@ -52,15 +52,19 @@ ComfyUI-Ultimate-Sampler-Grid/
├── manifest_utils.py # Manifest Read/Write/Merge Helpers
├── html_generator.py # Reads /resources/ to build the Dashboard HTML
│
├── distribution_manager.py # Distributed Job Queue Coordinator (Master)
├── distribution_worker.py # Worker Thread for Remote Processing
├── distribution_routes.py # Distribution REST API Endpoints
│
├── web/ # [ComfyUI Integration Layer]
│ ├── dashboard.js # Registers Dashboard Node & Message Handling
│ ├── config_builder.js # Registers Builder Node & Custom UI Widget
│ ├── smart_json_text.js # Registers JSON Node & Syntax Highlighting
│ └── conf_builder/ # [Config Builder UI Modules]
│ ├── conf-builder-main.js # Main entry, widget registration
│ ├── conf-builder-ui-components.js # UI rendering (dropdowns, sliders, cards)
│ ├── conf-builder-config-management.js # Save/Load/Duplicate config logic
│ └── conf-builder-utilities.js # State-to-JSON conversion, iteration counting
│ ├── conf-builder-utilities.js # State-to-JSON conversion, iteration counting
│ └── conf-builder-distribution.js # Distribution settings UI (workers, toggles)
│
└── resources/ # [The Dashboard SPA Core]
├── template.html # HTML Skeleton for the generated report
@@ -143,6 +147,17 @@ All routes registered in `__init__.py` via `PromptServer.instance.routes`.
| POST | `/config_tester/export_favorites` | Export favorited images `{session_name, pack_metadata, organize_by_prompt, organize_by_lora}` |
| POST | `/config_tester/scan_directory` | Scan external dir, create symlinks, use built-in `/view` `{directory_path, session_name}` |
**Distribution APIs** (`/distribution/`, registered in `distribution_routes.py`):
| Method | Route | Purpose |
|---|---|---|
| GET | `/distribution/claim_job` | Worker claims next pending job (200 + job JSON, 204 = no jobs, 503 = inactive) |
| POST | `/distribution/submit_result` | Worker uploads completed image (multipart: {metadata, image}) |
| GET | `/distribution/status` | Get distribution status, worker stats, job counts |
| POST | `/distribution/register_worker` | Worker registration with master |
| GET | `/distribution/download_model` | Download model file for model sync `{category, filename}` |
| POST | `/distribution/heartbeat` | Worker keepalive signal |
---
### **1. The Backend: Core Logic & Orchestration**
@@ -157,9 +172,9 @@ All routes registered in `__init__.py` via `PromptServer.instance.routes`.
* **Optional Inputs:** `optional_model` (MODEL), `optional_clip` (CLIP), `optional_vae` (VAE), `optional_positive` (CONDITIONING), `optional_negative` (CONDITIONING), `optional_latent` (LATENT)
* **`generation_orchestrator.py`** (The Conductor — largest file, ~987 lines)
* **Role:** Manages the generation loop.
* **Key Functions:** `run_generation_loop()`, `check_if_job_completed()`, `setup_session_directories()`, `load_model_by_type()`, `get_model_cache_key()`, `calculate_clip_hash()`
* **`generation_orchestrator.py`** (The Conductor — largest file, ~1700 lines)
* **Role:** Manages the generation loop, distributed processing, and master text encoding.
* **Key Functions:** `run_generation_loop()`, `check_if_job_completed()`, `setup_session_directories()`, `load_model_by_type()`, `get_model_cache_key()`, `calculate_clip_hash()`, `_conditioning_to_serializable()`, `_serializable_to_conditioning()`, `_preencode_all_conditionings()`, `_run_distributed_generation()`, `_send_distribution_status()`
* **Key Logic:**
* **Smart Skip:** `check_if_job_completed()` matches against manifest items by seed/resolution/sampler/scheduler/steps/cfg/denoise. Skips model/lora/prompt matching when optional inputs connected.
* **Lookahead:** Triggers `model_cache.preload_lora_model()` / `preload_base_model()` for the *next* job while the current one runs.
@@ -168,6 +183,9 @@ All routes registered in `__init__.py` via `PromptServer.instance.routes`.
* **Node Change Detection:** Detects when connected upstream nodes have changed for smarter job resuming.
* **Pre-encoding:** Single model → batch-encode all unique prompts upfront. Multi-model → re-encode per model switch.
* **VAE Switching:** Flushes pending batch before switching VAE. Tracks `default_model_vae` for reverting to "Default".
* **Distribution:** `_run_distributed_generation()` manages the distributed pipeline — populates job queue, optionally pre-encodes conditionings, notifies workers, and processes master's share of jobs.
* **Master Text Encoding:** `_preencode_all_conditionings()` groups configs by (model, LoRA), loads each combination, encodes all unique prompts, and serializes via `_conditioning_to_serializable()`. Handles multi-entry conditionings (AND combinator).
* **Per-Worker Logging:** Final summary prints each worker's completed/failed job counts from `manager.get_status()["workers"]`.
@@ -181,6 +199,7 @@ All routes registered in `__init__.py` via `PromptServer.instance.routes`.
* **Role:** `UltimateConfigBuilder` node. All state stored in single `lora_config` widget as JSON string.
* **Key Functions:** `generate_config()`, `process_lora_array()`, `get_available_sessions()`, `expand_lora_folders()`, `lookup_lora_triggers()`
* **Key Logic:** Python reads ONLY `lora_config` widget — all other INPUT_TYPES widgets (samplers, schedulers, steps, cfg) are vestigial and ignored. Also defines API routes for trigger/metadata lookup and model lists.
* **Distribution Output:** When distribution is enabled, wraps configs in `{"configs": [...], "_distribution": {...}}` with keys: `enabled`, `worker_urls`, `claim_timeout` (default 600), `use_master_encoding`, `sync_models_to_workers`. When disabled, still wraps in `{"configs": [...]}` — `sampler_node.py` must unwrap in both cases.
* **`html_generator.py`** (The Builder)
@@ -247,6 +266,39 @@ All routes registered in `__init__.py` via `PromptServer.instance.routes`.
* **Key Logic:** Packs ComfyUI-compatible metadata into PNG for CivitAI uploads. Maintains SHA256 hash cache at `benchmarks/model_hashes.json`.
---
### **2b. The Backend: Distribution System**
*Located in the root directory. Enables multi-machine parallel processing.*
* **`distribution_manager.py`** (The Job Coordinator)
* **Role:** Thread-safe distributed job queue running on the master. Manages job lifecycle: PENDING → CLAIMED → COMPLETED/FAILED.
* **Classes:** `JobState` (constants), `DistributedJob` (individual job), `DistributionManager` (coordinator)
* **Key Methods:** `populate_jobs()`, `claim_job(worker_id)`, `complete_job(job_id)`, `get_stats()`, `set_encoded_conditionings()`, `get_encoded_conditionings_for_job()`
* **Key Logic:**
* Thread-safe with `RLock` for concurrent worker access
* Job timeout handling (default 600s) — re-queues timed-out jobs as PENDING
* `MAX_FAIL_COUNT = 3` — jobs that fail 3 times are abandoned
* `_job_to_dict()` attaches pre-encoded conditionings to job claim response if available
* Tracks per-worker stats (completed, failed, last_seen)
* **`distribution_worker.py`** (The Remote Worker)
* **Role:** Background thread running on remote ComfyUI instances. Polls master for jobs, processes locally, uploads results.
* **Class:** `WorkerThread(threading.Thread)` — daemon thread
* **Key Methods:** `run()` (main loop), `_process_job()`, `_download_model_from_master()`, `_prefetch_models()`, `stop()`
* **Key Logic:**
* Polls `/distribution/claim_job` every 2 seconds
* Caches model/CLIP/VAE between jobs (reuses when same model)
* Checks for `encoded_positive`/`encoded_negative` in job data — uses master conditionings if available, falls back to local CLIP encoding otherwise
* Model sync: downloads models from master via `/distribution/download_model` if enabled
* Exits after 30 consecutive empty polls or 503 response (master finished)
* **`distribution_routes.py`** (The API Layer)
* **Role:** Registers `/distribution/*` API endpoints on the PromptServer.
* **Global State:** `_distribution_manager` (active manager, master mode), `_worker_thread` (active worker, worker mode) — both thread-safe via getters/setters
* **Key Endpoints:** `claim_job` (GET), `submit_result` (POST multipart), `status` (GET), `register_worker` (POST), `download_model` (GET), `heartbeat` (POST)
---
### **Key Data Shapes**
@@ -274,10 +326,46 @@ All routes registered in `__init__.py` via `PromptServer.instance.routes`.
"model_prompt_prefix": "",
"model_prompt_suffix": "",
"attention_modes": ["default"],
"resolutions": [],
"seed_behavior": "fixed",
"text_encoders": [],
"clip_type": "stable_diffusion",
"gguf_options": {}
"gguf_options": {},
"model_sampling_override": "None",
"model_sampling_shift": 3.0,
"use_advanced_sampling": false,
"advanced_guider": "Basic",
"use_flux_guidance": false,
"flux_guidance_value": 3.5
}
```
**Distribution Config** (embedded in `config_builder_node.py` output as `_distribution` key):
```json
{
"enabled": true,
"worker_urls": ["http://192.168.1.100:8188", "http://192.168.1.101:8188"],
"claim_timeout": 600,
"use_master_encoding": false,
"sync_models_to_workers": false
}
```
**Top-level state fields** (in `conf-builder-main.js` default state, outside `config_arrays`):
```json
{
"session_name": "my_session",
"config_name": "my_config",
"auto_save": false,
"include_none": true,
"label_mode": "full",
"global_positive_groups": [],
"global_negative": "",
"distribution_enabled": false,
"worker_urls": [],
"claim_timeout": 600,
"use_master_encoding": false,
"sync_models_to_workers": false
}
```
@@ -327,6 +415,25 @@ iframe postMessage({type: 'toggle_fullscreen', node_id})
→ dashboard.js listener → toggle .dashboard-fullscreen class
```
**Distribution Status Updates (Server → Dashboard iframe):**
```
generation_orchestrator → PromptServer.send_sync("ultimate_grid.distribution_status")
→ dashboard.js api.addEventListener → postMessage({type: 'distribution_status'})
→ iframe displays worker activity
```
**Distribution Job Flow (Master ↔ Worker):**
```
Master:
1. generation_orchestrator → DistributionManager.populate_jobs()
2. [Optional] _preencode_all_conditionings() → manager.set_encoded_conditionings()
3. POST /distribution/register_worker on each worker
4. Workers poll GET /distribution/claim_job
5. manager._job_to_dict() attaches encoded_positive/encoded_negative if available
6. Worker processes job → POST /distribution/submit_result (multipart: metadata + image)
7. Master saves image + updates manifest
```
---
### **3. The Frontend: ComfyUI Integration**
@@ -334,9 +441,14 @@ iframe postMessage({type: 'toggle_fullscreen', node_id})
*Located in `/web/`.*
* **`dashboard.js`**
* **Role:** Registers the Dashboard node (`UltimateGridDashboard`). Listens for `ultimate_grid.update` and `ultimate_grid.progress` server events.
* **Key Logic:** Forwards events to matching iframe via `postMessage()`. Auto-loads session when update received for unloaded session. Handles fullscreen toggle via `.dashboard-fullscreen` CSS class.
* **Role:** Registers the Dashboard node (`UltimateGridDashboard`). Listens for `ultimate_grid.update`, `ultimate_grid.progress`, and `ultimate_grid.distribution_status` server events.
* **Key Logic:** Forwards events to matching iframe via `postMessage()`. Auto-loads session when update received for unloaded session. Handles fullscreen toggle via `.dashboard-fullscreen` CSS class. **Auto-loads on node creation** with 500ms delay via `forceLoadSession()` so the dashboard shows content immediately (landing page or session grid) instead of a blank black box.
* **Widgets:** "RELOAD / SHOW SESSION" button → `forceLoadSession()`, "DELETE SESSION" button → POST `/config_tester/delete_session`
* **Event Listeners (Singletons):** Each uses `window.__ultimateGrid*ListenerInstalled` guard to prevent duplicate registration:
* `ultimate_grid.update` — Live data updates to iframe
* `ultimate_grid.progress` — ETA/progress bar updates
* `ultimate_grid.distribution_status` — Distribution worker status
* `window "message"` — Fullscreen toggle from iframe
* **`config_builder.js`** (entry point, loads `conf_builder/` modules)
@@ -359,9 +471,14 @@ iframe postMessage({type: 'toggle_fullscreen', node_id})
* **`conf_builder/conf-builder-ui-components.js`** (The Widget Library)
* **Key Exports:** `createSearchableSelect()`, `createSlider()`, `createInputGroup()`, `getStyles()`
* **Key Exports:** `createSearchableSelect()`, `createSlider()`, `createInputGroup()`, `createTopBar()`, `createSidebar()`, `createSectionHeader()`, `getStyles()`
* **`conf_builder/conf-builder-distribution.js`** (Distribution Settings UI)
* **Role:** Renders the distribution settings section in the Config Builder.
* **Key Exports:** `renderDistributionSection()`, `renderDistributionSettingsSection()`
* **Features:** Worker URL management (add/remove), connection testing via `/distribution/test_worker`, status indicators (green/yellow/red dots), claim timeout slider (30-900s, default 600), toggles for "Use Master Text Encoding" and "Sync Models to Workers".
---
@@ -430,7 +547,7 @@ iframe postMessage({type: 'toggle_fullscreen', node_id})
|---|---|
| `sampler_node.py` | `run_tests()`, `find_existing_match()`, `get_latent_channels()`, `IS_CHANGED()` |
| `config_builder_node.py` | `generate_config()`, `process_lora_array()`, `get_available_sessions()`, `expand_lora_folders()` |
| `generation_orchestrator.py` | `run_generation_loop()`, `check_if_job_completed()`, `setup_session_directories()`, `load_model_by_type()` |
| `generation_orchestrator.py` | `run_generation_loop()`, `check_if_job_completed()`, `setup_session_directories()`, `load_model_by_type()`, `_preencode_all_conditionings()`, `_conditioning_to_serializable()`, `_serializable_to_conditioning()`, `_run_distributed_generation()` |
| `batch_encoding.py` | `encode_prompt_with_combinators()`, `batch_encode_prompts()` |
| `config_utils.py` | `expand_configs()`, `parse_prompt_input_nested()`, `prepare_input_jobs()`, `parse_lora_definition()` |
| `model_loader.py` | `load_checkpoint()`, `load_loras()`, `load_vae_by_name()`, `load_diffusion_model_and_clip()` |
@@ -439,6 +556,9 @@ iframe postMessage({type: 'toggle_fullscreen', node_id})
| `manifest_utils.py` | `load_existing_manifest()`, `save_manifest()`, `merge_manifest_user_changes()` |
| `directory_scanner.py` | `scan_directory_for_images()`, `parse_a1111_parameters()` |
| `metadata_packer.py` | `pack_metadata_into_image()`, `extract_metadata_from_image()`, `calculate_file_hash()` |
| `distribution_manager.py` | `DistributionManager.populate_jobs()`, `.claim_job()`, `.complete_job()`, `.get_stats()`, `.set_encoded_conditionings()`, `.get_encoded_conditionings_for_job()` |
| `distribution_worker.py` | `WorkerThread.run()`, `._process_job()`, `._download_model_from_master()`, `._prefetch_models()`, `.stop()` |
| `distribution_routes.py` | `get_distribution_manager()`, `set_distribution_manager()`, endpoint handlers for `/distribution/*` |
**JavaScript (by file):**
@@ -447,7 +567,8 @@ iframe postMessage({type: 'toggle_fullscreen', node_id})
| `conf-builder-utilities.js` | `convertStateToConfigs()`, `getIterationCount()`, `parseLoraString()`, `buildLoraString()`, `refreshAllConfigBuilders()` |
| `conf-builder-config-management.js` | `renderUI()`, `updatePreview()`, `createConfigArrayElement()`, `createModelElement()` |
| `conf-builder-ui-components.js` | `createSearchableSelect()`, `createSlider()`, `getStyles()` |
| `conf-builder-main.js` | `ensureModulesLoaded()`, `node.saveState()`, `node.renderUI()`, `node.loadSession()` |
| `conf-builder-main.js` | `ensureModulesLoaded()`, `node.saveState()`, `node.renderUI()`, `node.loadSession()`, `node.triggerAutoSave()` |
| `conf-builder-distribution.js` | `renderDistributionSection()`, `renderDistributionSettingsSection()` |
| `logic_ui.js` | `toggleCogMenu()`, `toggleFiltersPopup()`, `initFilters()` |
| `logic_utils.js` | `loadSession()`, `exportFavorites()`, `scanDirectory()`, `triggerGen()`, `toggleFullscreen()` |
| `logic_virtual.js` | `renderDOM()`, `updateVisibleItems()`, `calculateVisibleRange()`, `autoFitZoom()`, `goToImage()` |
@@ -458,19 +579,23 @@ iframe postMessage({type: 'toggle_fullscreen', node_id})
### **Summary of Data Flow**
1. **User Config:** You set parameters in the **Sampler Node** (Python).
2. **Expansion:** `generation_orchestrator.py` expands this into a list of jobs.
3. **Generation:** The backend loops through jobs, using **ModelCache** and **ConditioningCache** for speed.
4. **Storage:** Images are saved to disk; metadata is appended to `manifest.json`.
5. **Compilation:** `html_generator.py` reads the `manifest.json` and all `.js` files in `/resources/`, baking them into a single `dashboard_html` string.
6. **Display:** The **Dashboard Node** (JS) renders this HTML in an iframe.
7. **Boot:** `logic_init.js` wakes up, loads the data into `logic_state.js`, and `logic_virtual.js` starts rendering cards.
8. **Interaction:** You click "Favorite" in the UI -> `logic_events.js` sends an API request to `__init__.py` -> Python updates `manifest.json` on disk.
1. **User Config:** You set parameters in the **Config Builder** (visual UI) or **Sampler Node** (JSON text).
2. **Config Output:** `config_builder_node.py` wraps configs in `{"configs": [...], "_distribution": {...}}`. The `sampler_node.py` unwraps the `"configs"` key and extracts distribution settings.
3. **Expansion:** `generation_orchestrator.py` expands configs into a full list of jobs (Cartesian product).
4. **Distribution (optional):** If enabled, `_run_distributed_generation()` populates the job queue, optionally pre-encodes conditionings, and notifies workers. Workers poll for jobs, process locally, and upload results.
5. **Generation:** The backend loops through its share of jobs, using **ModelCache** and **ConditioningCache** for speed.
6. **Storage:** Images are saved to disk; metadata is appended to `manifest.json`.
7. **Compilation:** `html_generator.py` reads the `manifest.json` and all `.js` files in `/resources/`, baking them into a single `dashboard_html` string.
8. **Display:** The **Dashboard Node** (JS) renders this HTML in an iframe. Auto-loads on node creation.
9. **Boot:** `logic_init.js` calls `init()` which calls `showSessionLandingIfEmpty()`. If no data, the session landing page is shown. If data exists, `logic_virtual.js` renders the grid.
10. **Interaction:** You click "Favorite" in the UI -> `logic_events.js` sends an API request to `__init__.py` -> Python updates `manifest.json` on disk.
---
### **Critical Development Gotchas**
> ⚠️ **GOLDEN RULE: DO NOT REMOVE ANY CODE. DO NOT REMOVE ANY COMMENTS. ONLY CHANGE WHAT IS NECESSARY.**
1. **Dual Data Path** — Frontend `convertStateToConfigs()` (in `conf-builder-utilities.js`) and Python `generate_config()` (in `config_builder_node.py`) are **independent implementations** that both transform `node.state` into config JSON. Both must stay in sync when adding new config fields or the UI preview will mismatch the actual output.
2. **Widget Bridge** — The `lora_config` widget (STRING type) holds the **entire** node state as serialized JSON. Python `generate_config()` ignores all other widget params and reads only `lora_config`. The `samplers`, `schedulers`, `steps`, `cfg` widgets in `INPUT_TYPES` are vestigial (kept for ComfyUI schema compliance only).
@@ -493,4 +618,105 @@ iframe postMessage({type: 'toggle_fullscreen', node_id})
11. **Field Name Inconsistency: `favorite` vs `favorited`** — `manifest_utils.py` `merge_manifest_user_changes()` uses the key `"favorite"`. But `__init__.py` routes like `save_manifest` and `export_favorites` check `"favorited"`. When touching favorite/unfavorite logic, check which key name the specific code path uses.
12. **Prompt Expansion is Recursive** — `parse_prompt_input_nested()` in `config_utils.py` supports arbitrarily deep nested arrays. Flat list = OR (options). List containing sub-lists = AND (Cartesian product). This is used by both the node text inputs and per-config prompt groups from the builder UI.
12. **Prompt Expansion is Recursive** — `parse_prompt_input_nested()` in `config_utils.py` supports arbitrarily deep nested arrays. Flat list = OR (options). List containing sub-lists = AND (Cartesian product). This is used by both the node text inputs and per-config prompt groups from the builder UI.
13. **Config Builder Output is ALWAYS Wrapped** — `config_builder_node.py` always outputs `{"configs": [...]}` (with optional `"_distribution"` key). The `sampler_node.py` MUST unwrap the `"configs"` key before passing to `expand_configs()`. Both the `_distribution` present AND absent cases must be handled (see lines 269-286 of `sampler_node.py`).
14. **Dashboard Auto-Load on Creation** — `web/dashboard.js` calls `forceLoadSession()` with 500ms delay when a dashboard node is created. The `get_session_html` endpoint in `__init__.py` returns a full HTML template with empty manifest for non-existent sessions (NOT a 404). This ensures the landing page shows immediately.
15. **`logic_pipeline.js` Legacy Init Guard** — Lines 489-505 have a `DOMContentLoaded` handler that calls `loadSession()` ONLY when `fullManifest.items.length > 0`. Without this guard, `loadSession()` triggers an alert popup for non-existent sessions. The proper initialization path is `logic_init.js` → `init()` → `showSessionLandingIfEmpty()`.
16. **Distribution Default Values Must Stay in Sync** — `claim_timeout` default (600) appears in: `distribution_manager.py` (constructor), `config_builder_node.py` (fallback), `conf-builder-distribution.js` (slider default), `conf-builder-main.js` (default state). All four must match. Same for `use_master_encoding` default (false).
17. **Conditioning Serialization Uses np.copy()** — `_serializable_to_conditioning()` in `generation_orchestrator.py` must call `np.frombuffer(...).copy()` before `torch.from_numpy()` because `np.frombuffer()` returns read-only arrays that are incompatible with GPU tensor operations.
18. **Multi-Entry Conditioning (AND combinator)** — The AND combinator creates multiple `[tensor, {pooled_output}]` entries in a conditioning list. Serialization in `_conditioning_to_serializable()` must iterate ALL entries, not just `[0]`. Deserialization in `_serializable_to_conditioning()` must reconstruct the full list.
19. **State Migration for New Fields** — When adding new fields to the Config Builder default state in `conf-builder-main.js` (around line 89), also add a migration check (around line 363) to backfill the default for existing saved workflows. Without migration, old workflows will have `undefined` for the new field.
20. **ComfyUI Registry Security** — No `import requests`, no `subprocess`/`os.system`/`eval()`/`exec()`, no runtime `pip install`, no custom file-serving endpoints, no code obfuscation. Use `urllib.request` for HTTP. Use ComfyUI's `/view` endpoint for serving files. Sanitize all user-supplied paths. See top of this file for full rules.
---
### **Notes for AI Development**
This section provides critical context for AI assistants working on this codebase.
**Architecture Mental Model:**
```
┌─────────────────────────────────────────────────────────────┐
│ ComfyUI Browser Tab │
│ ┌──────────────────┐ ┌──────────────────────────────────┐│
│ │ Config Builder │ │ Dashboard Node ││
│ │ (conf-builder-*) │ │ (dashboard.js) ││
│ │ Direct DOM widget │ │ ┌──────────────────────────────┐││
│ │ │ │ │ iframe (srcdoc) │││
│ │ │ │ │ Dashboard SPA │││
│ │ │ │ │ (resources/logic_*.js) │││
│ │ │ │ │ postMessage() ↕ │││
│ │ │ │ └──────────────────────────────┘││
│ └──────────────────┘ └──────────────────────────────────┘│
│ ↕ widget value ↕ server events │
├─────────────────────────────────────────────────────────────┤
│ ComfyUI Server (Python) │
│ ┌──────────────┐ ┌────────────────┐ ┌────────────────┐ │
│ │config_builder│ │sampler_node.py │ │__init__.py │ │
│ │_node.py │→ │ │ │(API endpoints) │ │
│ └──────────────┘ │ ↓ │ └────────────────┘ │
│ │generation_ │ │
│ │orchestrator.py │ │
│ │ ↓ │ │
│ ┌─────────┐ │model_loader + │ ┌─────────────────┐ │
│ │ModelCache│←────│image_generation│ │DistributionMgr │ │
│ └─────────┘ │ ↓ │ │(master jobs) │ │
│ │html_generator │ └─────────────────┘ │
│ └────────────────┘ ↕ HTTP │
│ ┌─────────────────┐ │
│ │WorkerThread │ │
│ │(remote workers) │ │
│ └─────────────────┘ │
└─────────────────────────────────────────────────────────────┘
```
**File Dependency Graph (Python):**
```
sampler_node.py
→ generation_orchestrator.py (main entry)
→ config_utils.py (expand_configs, prepare_input_jobs)
→ model_loader.py (load_checkpoint, load_loras)
→ image_generation.py (generate_image, flush_batch)
→ batch_encoding.py (encode_prompt_with_combinators)
→ trigger_words.py (build_prompt_with_triggers)
→ manifest_utils.py (save_manifest, load_existing_manifest)
→ model_cache.py (ModelCache)
→ conditioning_cache.py (ConditioningCache)
→ remote_vae.py (RemoteVAEDecodeWorker)
→ html_generator.py (get_html_template)
→ distribution_manager.py (DistributionManager)
→ distribution_worker.py (WorkerThread)
config_builder_node.py
→ lora_utils.py (trigger word lookup, CivitAI API)
__init__.py
→ all node classes (sampler, dashboard, config_builder, json_text)
→ distribution_routes.py (distribution API endpoints)
→ html_generator.py (on-demand HTML generation)
→ metadata_packer.py (export favorites)
→ directory_scanner.py (scan external directories)
→ manifest_utils.py (save/load manifests)
```
**Server Event Types:**
| Event Name | Sender | Data Fields | Purpose |
|---|---|---|---|
| `ultimate_grid.update` | `generation_orchestrator.py` | `session_name, node, manifest, meta, new_items` | New images generated |
| `ultimate_grid.progress` | `generation_orchestrator.py` | `session_name, progress_pct, eta_str, current_job, total_jobs, avg_duration, total_generated, total_elapsed, finish_time` | ETA/progress bar |
| `ultimate_grid.distribution_status` | `generation_orchestrator.py` | `session_name, workers, stats` | Distribution worker status |
**Testing Changes:**
- **Python backend files:** Restart ComfyUI
- **`web/` JS files:** Refresh browser (Ctrl+F5)
- **`resources/` JS/CSS/HTML files:** Dashboard HTML must be regenerated — re-run the sampler node or call `/config_tester/get_session_html`
- **Config Builder state fields:** Also update migration checks in `conf-builder-main.js`
README.md
+231 -14
View File
@@ -87,6 +87,12 @@ Stop guessing which Sampler, Scheduler, Prompt, Denoise, Model, Lora or CFG valu
- [API Integration (Config Builder)](#api-integration-config-builder)
- [File Locations](#file-locations)
- [Keyboard Shortcuts](#keyboard-shortcuts)
- [Distributed Processing](#-distributed-processing)
- [Setting Up Distribution](#setting-up-distribution)
- [Master Text Encoding](#master-text-encoding)
- [Model Sync](#model-sync)
- [Distribution Troubleshooting](#distribution-troubleshooting)
- [Notes for AI Development](#-notes-for-ai-development)
- [Changelog](#-changelog)
- [License](#-license)
- [Credits](#-credits)
@@ -128,6 +134,17 @@ Stop guessing which Sampler, Scheduler, Prompt, Denoise, Model, Lora or CFG valu
* **Recursive Cartesian Prompt Builder:** Visual chip-based prompt editor with `{option1|option2|option3}` syntax that expands into all combinations. Nest multiple groups for exponential prompt variation.
* **Model & LoRA On/Off Switches:** Toggle individual models and LoRAs on/off without removing them from the config. Bypassed entries are omitted from both the preview JSON and the generated config output.
### 🌐 Distributed Processing (Multi-Machine)
* **Multi-Worker Distribution:** Split large grid runs across multiple ComfyUI instances. A master node coordinates job distribution while workers process images in parallel and upload results back.
* **One-Click Setup:** Enable distribution in the Config Builder's Distribution Settings section. Add worker URLs (e.g., `http://192.168.1.100:8188`) and toggle "Enable Distribution."
* **Worker Connection Testing:** Test connectivity to each worker with a single click — status dots show green (reachable), yellow (testing), or red (error).
* **Master Text Encoding:** Optional "Use Master Text Encoding" toggle — the master pre-encodes ALL unique prompts (including LoRA-specific combinations) and sends the encoded conditionings to workers. Workers skip CLIP encoding entirely, saving significant time on prompt-heavy grids.
* **Model Sync:** Optional "Sync Models to Workers" toggle — automatically transfers checkpoint and LoRA files from the master to workers that don't have them.
* **Smart Job Queue:** Thread-safe job distribution with automatic re-queuing on worker timeout (configurable, default 600s) and max 3 retries before abandoning a job.
* **Per-Worker Logging:** Final generation summary shows each worker's completed/failed job counts for full visibility into the distributed run.
* **Live Distribution Status:** Dashboard receives real-time distribution status updates showing worker activity and progress.
* **Backward Compatible:** Workers gracefully fall back to local CLIP encoding if master-encoded conditionings aren't available.
### 🎨 Interactive Dashboard (The "IDE")
* **Infinite Canvas with Pan/Zoom:** Google Maps-style navigation with mouse drag, mousewheel zoom, and keyboard shortcuts.
* **Virtual Scrolling:** Ultra-optimized rendering handles thousands of images smoothly by only loading visible items — scroll through 5000+ images without lag.
@@ -184,18 +201,52 @@ The Config Builder node is included automatically — no separate installation i
**File Structure:**
```
ComfyUI/custom_nodes/ComfyUI-Ultimate-Auto-Sampler/
├── config_builder_node.py # Config Builder backend
├── js/conf_builder/
│ ├── conf-builder-main.js # Entry point & node registration
│ ├── conf-builder-utilities.js # Data fetching & parsing
│ ├── conf-builder-ui-components.js # Reusable UI elements
│ └── conf-builder-config-management.js # Rendering & state logic
├── trigger_words.py # LoRA trigger word handling
├── batch_encoding.py # CLIP batch encoding with caching
├── manifest_utils.py # Manifest file management
├── model_loader.py # Model/LoRA loading and patching
├── image_generation.py # Image generation and sampling
└── generation_orchestrator.py # Main orchestration layer
├── __init__.py # Server routes, API endpoints, node mappings
├── sampler_node.py # Main "Sampler Grid" node
├── dashboard_node.py # "Dashboard Viewer" node
├── config_builder_node.py # "Config Builder" node backend
├── json_text_node.py # "Smart JSON" node
│
├── generation_orchestrator.py # Main orchestration layer & generation loop
├── image_generation.py # KSampler & VAE wrappers
├── model_loader.py # Checkpoint/LoRA/VAE loading and patching
├── model_cache.py # 3-tier model caching system
├── remote_vae.py # Remote VAE offloading
│
├── batch_encoding.py # CLIP batch encoding with combinator support
├── conditioning_cache.py # Disk-based conditioning cache
├── lora_utils.py # LoRA searching & validation
├── trigger_words.py # CivitAI trigger word fetching & prompt assembly
├── config_utils.py # Config parsing, expansion & Cartesian products
├── directory_scanner.py # External directory session scanner
├── metadata_packer.py # PNG/WebP metadata embedding for exports
├── manifest_utils.py # Manifest read/write/merge helpers
├── html_generator.py # Reads /resources/ to build dashboard HTML
│
├── distribution_manager.py # Distributed job queue coordinator (master)
├── distribution_worker.py # Worker thread for remote processing
├── distribution_routes.py # Distribution API endpoints
│
├── web/ # [ComfyUI Integration Layer]
│ ├── dashboard.js # Dashboard node registration & messaging
│ ├── smart_json_text.js # JSON node & syntax highlighting
│ └── conf_builder/ # [Config Builder UI Modules]
│ ├── conf-builder-main.js # Entry point, widget registration
│ ├── conf-builder-ui-components.js # Reusable UI (dropdowns, sliders, cards)
│ ├── conf-builder-config-management.js # Save/Load/state rendering logic
│ ├── conf-builder-utilities.js # Data fetching, caching, calculations
│ └── conf-builder-distribution.js # Distribution settings UI
│
└── resources/ # [Dashboard SPA Core]
├── template.html # HTML skeleton for dashboard
├── report.css # Styling (infinite canvas, cards, modals)
├── logic_state.js # Global state variables
├── logic_utils.js # API & helper functions
├── logic_ui.js # UI renderer (DOM creation, modals)
├── logic_virtual.js # Virtual scroll engine & viewport
├── logic_pipeline.js # Sorting & filtering pipeline
├── logic_events.js # Event listeners (messages, keyboard)
└── logic_init.js # Initialization & startup
```
---
@@ -1169,6 +1220,19 @@ Scan an external directory for image sessions and load them into the dashboard.
}
```
### Distribution API (registered via `distribution_routes.py`)
These endpoints are used internally by the distribution system:
| Method | Route | Purpose |
|--------|-------|---------|
| GET | `/distribution/claim_job` | Worker claims next pending job (200 + job JSON, 204 = no jobs, 503 = inactive) |
| POST | `/distribution/submit_result` | Worker uploads completed image (multipart: metadata + image) |
| GET | `/distribution/status` | Get distribution status and worker stats |
| POST | `/distribution/register_worker` | Worker registration with master |
| GET | `/distribution/download_model` | Download model file for model sync |
| POST | `/distribution/heartbeat` | Worker keepalive signal |
---
## File Locations
@@ -1229,8 +1293,161 @@ ComfyUI/output/benchmarks/model-data/
---
## 🌐 Distributed Processing
Distribute large grid runs across multiple ComfyUI instances for parallel image generation.
### Setting Up Distribution
1. **Ensure Workers Are Running:** Each worker must be a separate ComfyUI instance with this node suite installed and running.
2. **Open Config Builder:** In the Config Builder node, scroll to the **Distribution Settings** section.
3. **Add Worker URLs:** Enter the full URL of each worker (e.g., `http://192.168.1.100:8188`). Click **+ Add Worker** for additional workers.
4. **Test Connections:** Click the test button next to each worker URL. A green dot means the worker is reachable.
5. **Enable Distribution:** Toggle the distribution switch ON.
6. **Run Generation:** Queue the workflow. The master node will coordinate job distribution automatically.
**How It Works:**
```
Master (your machine):
1. Expands configs into individual jobs
2. [Optional] Pre-encodes all prompts (Master Text Encoding)
3. Notifies workers to start polling
4. Processes its own jobs while workers claim theirs
5. Workers upload completed images back to master
6. Master saves everything to the session manifest
```
### Master Text Encoding
When enabled, the master pre-encodes ALL unique positive/negative prompts for every (model, LoRA) combination before distributing jobs. Workers receive the encoded conditionings in their job claim response and skip CLIP encoding entirely.
**Benefits:**
- Workers don't need to load text encoders at all
- Eliminates duplicate CLIP encoding across workers
- Significant time savings for prompt-heavy grids
**Enable:** Toggle "Use Master Text Encoding" in Distribution Settings.
**Technical Notes:**
- Encoded conditionings are serialized as base64-encoded NumPy arrays (~640KB per job)
- Multi-entry conditionings (AND combinator) are fully supported
- Workers fall back to local encoding if pre-encoded data is unavailable (backward compatible)
### Model Sync
When enabled, the master automatically transfers checkpoint and LoRA files to workers that don't have them.
**Enable:** Toggle "Sync Models to Workers" in Distribution Settings.
### Distribution Troubleshooting
* **Workers not connecting:** Verify the worker URL is correct (include port). Check firewalls. Both master and worker must have this node suite installed.
* **Jobs timing out:** Increase the "Claim Timeout" slider (default 600s). First-time model loads on workers can take several minutes.
* **Duplicate images:** If a worker times out but actually completes the job, duplicates may appear. Increase claim timeout to prevent this.
* **Workers not using pre-encoded prompts:** Ensure "Use Master Text Encoding" is enabled. Check worker console for `"Using master pre-encoded conditionings"` log message.
* **Model sync failing:** Ensure both master and worker have sufficient disk space. Check worker console for download errors.
---
## 🤖 Notes for AI Development
This section provides critical context for AI assistants (Claude, GPT, etc.) working on this codebase.
### Critical Constraints
**DO NOT REMOVE ANY CODE. DO NOT REMOVE ANY COMMENTS. ONLY CHANGE WHAT IS NECESSARY.** This is the #1 rule for all code changes. The codebase has many interconnected parts and removing code often breaks things in unexpected ways.
### Architecture Mental Model
This is a **hybrid application** with three layers:
1. **Python Backend** — Runs inside ComfyUI's server process. Handles generation, caching, file I/O, and API endpoints.
2. **JS Frontend Bridge** (`web/` directory) — Runs in the ComfyUI browser tab. Registers custom nodes, manages widget state, and forwards server events.
3. **Dashboard SPA** (`resources/` directory) — Runs **inside an iframe** in the dashboard node. A standalone app with its own state management, virtual scrolling, and event handling.
### Key Gotchas for AI Development
1. **Config Builder output is ALWAYS wrapped** — `config_builder_node.py` outputs `{"configs": [...], "_distribution": {...}}`. The `sampler_node.py` MUST unwrap the `"configs"` key before passing to `expand_configs()`. If you add new top-level keys to the config output, update the extraction logic in `sampler_node.py` lines 269-286.
2. **Two independent config-to-JSON implementations** — `convertStateToConfigs()` in `conf-builder-utilities.js` (JS preview) and `generate_config()` in `config_builder_node.py` (Python output) both transform `node.state` into config JSON. **Both must stay in sync** when adding new config fields.
3. **Dashboard runs in an iframe** — The dashboard HTML is set via `iframe.srcdoc`. All JS/CSS is inlined (no external requests). Communication between the parent ComfyUI page and the iframe uses `postMessage()`. The `dashboard.js` file bridges server events to iframe messages.
4. **Dashboard auto-loads on node creation** — `web/dashboard.js` calls `forceLoadSession()` with a 500ms delay when a dashboard node is created. The `get_session_html` API endpoint returns a full HTML template with empty manifest for non-existent sessions (NOT a 404).
5. **`logic_pipeline.js` has a legacy init block** — Lines 489-505 have a `DOMContentLoaded` handler that calls `loadSession()` only when `fullManifest.items.length > 0`. The proper initialization happens in `logic_init.js` which calls `init()`. Don't add a second unconditional `loadSession()` call — it causes alert popups for empty sessions.
6. **Manifest merge on save** — `save_manifest()` always reloads from disk first to preserve user favorites/rejected/notes. This is intentional for concurrent access (generation + user interaction). Don't bypass this.
7. **`onNodeCreated` must be synchronous** — In `conf-builder-main.js`, async work runs inside fire-and-forget `(async () => { ... })()`. Making `onNodeCreated` async breaks ComfyUI widget registration.
8. **Optional inputs disable caches** — When `optional_model`/`optional_clip`/`optional_positive`/`optional_negative` are connected to the sampler node, conditioning cache is disabled and `IS_CHANGED()` returns `NaN` (forces re-execution every time).
9. **Distribution conditionings are serialized with np.copy()** — `np.frombuffer()` returns read-only arrays. Must call `.copy()` before `torch.from_numpy()` or GPU tensor operations will fail.
10. **Default state migration** — When adding new fields to the Config Builder state (in `conf-builder-main.js`), also add a migration check in the `onNodeCreated` function (around line 363) to backfill the default value for existing saved workflows.
11. **claim_timeout default is 600** — This value appears in `distribution_manager.py`, `config_builder_node.py`, `conf-builder-distribution.js`, and `conf-builder-main.js`. Keep all four in sync.
12. **ComfyUI Registry security** — No `import requests`, no `subprocess`/`eval`/`exec`, no custom file-serving endpoints. Use `urllib.request` for HTTP and ComfyUI's `/view` endpoint for serving files. See `ProjectStructure.md` for full rules.
### File Dependencies (Import Graph)
```
sampler_node.py
→ generation_orchestrator.py (main entry point)
→ config_utils.py (expand_configs, prepare_input_jobs)
→ model_loader.py (load_checkpoint, load_loras)
→ image_generation.py (generate_image, flush_batch)
→ batch_encoding.py (encode_prompt_with_combinators)
→ trigger_words.py (build_prompt_with_triggers)
→ manifest_utils.py (save_manifest, load_existing_manifest)
→ model_cache.py (ModelCache)
→ conditioning_cache.py (ConditioningCache)
→ remote_vae.py (RemoteVAEDecodeWorker)
→ html_generator.py (get_html_template)
→ distribution_manager.py (DistributionManager)
→ distribution_worker.py (WorkerThread)
config_builder_node.py
→ lora_utils.py (trigger word lookup)
__init__.py
→ all node classes
→ distribution_routes.py (distribution API endpoints)
→ html_generator.py (on-demand HTML generation)
→ metadata_packer.py (export favorites)
→ directory_scanner.py (scan external directories)
→ manifest_utils.py (save/load manifests)
```
### Server Event Types
| Event Name | Sender | Data | Purpose |
|---|---|---|---|
| `ultimate_grid.update` | `generation_orchestrator.py` | `{session_name, node, manifest, meta, new_items}` | New images generated |
| `ultimate_grid.progress` | `generation_orchestrator.py` | `{session_name, progress_pct, eta_str, ...}` | ETA/progress updates |
| `ultimate_grid.distribution_status` | `generation_orchestrator.py` | `{session_name, workers, stats}` | Distribution status |
### Testing Changes
After modifying backend Python files, restart ComfyUI. After modifying `web/` JS files, refresh the browser (Ctrl+F5). After modifying `resources/` JS/CSS/HTML files, the dashboard HTML must be regenerated — either re-run the sampler node or call the `/config_tester/get_session_html` endpoint.
---
## 📝 Changelog
### Update 3/6/26 — Distributed Processing & Dashboard Improvements
* 🌐 **Multi-Worker Distribution:** Split grid runs across multiple ComfyUI instances with automatic job coordination, claiming, timeout handling, and result upload.
* 🧠 **Master Text Encoding:** Master pre-encodes all prompts and sends conditionings to workers, eliminating duplicate CLIP encoding.
* 📦 **Model Sync:** Optionally transfer checkpoint and LoRA files from master to workers automatically.
* 📊 **Per-Worker Logging:** Final summary shows each worker's completed/failed job counts.
* ⚙️ **Distribution Settings UI:** Config Builder section with worker URL management, connection testing, and toggles for text encoding and model sync.
* 🏠 **Session Landing Page:** Dashboard now auto-loads and shows a session picker on startup instead of a blank black box.
* 🔄 **Dashboard Auto-Load:** Dashboard iframe automatically loads session HTML on node creation with 500ms delay.
* 🛠️ **Non-Existent Session Handling:** `get_session_html` endpoint returns a full template with empty manifest instead of 404.
* ⏱️ **Claim Timeout Update:** Default claim timeout increased from 300s to 600s across all files.
* 🤖 **AI Development Notes:** Added comprehensive development notes to README and ProjectStructure for AI-assisted development.
### Update 2/16/26 — Config Builder & Dashboard Overhaul
* 🎛️ **VAE Selection & Iteration:** Test multiple VAEs per config with searchable dropdown and folder expansion. VAEs are a full Cartesian dimension.
* 🧠 **Attention Mode Testing:** Grid-test different attention implementations (`default`, `sageattn`, `xformers`) as a combinatorial dimension.
@@ -1309,5 +1526,5 @@ Created for the ComfyUI community. Special thanks to all contributors and tester
**Support development:** [Buy me a coffee on Ko-fi](https://ko-fi.com/jasonhoku) ☕
**Version:** 2.0
**Last Updated:** February 2026
**Version:** 3.0
**Last Updated:** March 2026