diff --git a/AI-CONTEXT.md b/AI-CONTEXT.md index 98d4f30..695aafe 100644 --- a/AI-CONTEXT.md +++ b/AI-CONTEXT.md @@ -40,11 +40,11 @@ Read `ProjectStructure.md` in this directory for the full reference (file struct | `metadata_packer.py` | 561 | PNG metadata embedding for CivitAI. `pack_metadata_into_image()` accepts `workflow_data` kwarg | | `run_api.py` | 144 | Pure helpers behind `POST /config_tester/run` and `GET /config_tester/run_status` | | `image_metrics.py`, `metrics_runner.py` | 668, 167 | Optional per-image metrics at save time; background job that backfills them for old sessions | -| `workflow_sweep.py`, `workflow_sweep_runner.py`, `workflow_sweep_collect.py` | 969, 912, 140 | Workflow Sweep: pure expansion core, in-process queue feeder (never blocks the executor), `USCGSweepCollect` output node | +| `workflow_sweep.py`, `workflow_sweep_runner.py`, `workflow_sweep_collect.py` | 1106, 1027, 281 | Workflow Sweep: pure expansion core, in-process queue feeder (never blocks the executor), `USCGSweepCollect` / `USCGSweepCollectVideo` output nodes | | `web/conf_builder/conf-builder-main.js` | 779 | Node registration. Default state and its migration blocks define ALL fields including upscaling/cooldown | | `web/conf_builder/conf-builder-utilities.js` | 866 | Model-list cache (`LS_CACHE_VERSION`), `getIterationCount()`, LoRA string helpers. No config converter: the preview calls `/configbuilder/preview` | | `web/conf_builder/conf-builder-config-management.js` | 7736 | All config builder UI rendering. Upscaling section, preview fetch | -| `web/conf_builder/conf-builder-workflow-sweep.js` | 744 | Workflow Sweep UI (load API workflow, axes, dry run, start/pause/cancel) | +| `web/conf_builder/conf-builder-workflow-sweep.js` | 806 | Workflow Sweep UI (load API workflow, axes, dry run, start/pause/cancel) | | `resources/logic_utils.js` | 1073 | Dashboard helpers: `exportFavorites()`, `deleteNonFavorites()`, `loadSession()`, `scanDirectory()` | | `resources/logic_ui.js` | 2907 | Dashboard DOM: cards, modals, `buildComfyNodesWorkflow()`, analytics | | `resources/logic_virtual.js` | 1092 | Virtual scroller: renders only visible cards from thousands | diff --git a/ProjectStructure.md b/ProjectStructure.md index 0ff8101..dff66bf 100644 --- a/ProjectStructure.md +++ b/ProjectStructure.md @@ -59,9 +59,9 @@ ComfyUI-Ultimate-Sampler-Grid/ ├── upscale_runner.py # Dashboard async upscaling background thread (642 lines) ├── civitai.py # Facade for the ComfyUI-USCG-CivitAI companion, no network code here (40 lines) ├── distribution.py # Facade for the ComfyUI-USCG-Distributed companion, no network code here (107 lines) -├── workflow_sweep.py # Workflow Sweep core: parse API workflow, bindings -> jobs, prompt substitution, fingerprints (pure, no torch/server) (969 lines) -├── workflow_sweep_runner.py # Workflow Sweep runner: ComfyBridge (in-process post_prompt mirror), windowed feeder thread, pause/resume/cancel, route helpers (912 lines) -├── workflow_sweep_collect.py # USCGSweepCollect output node: saves each job's image + manifest item into the USCG session (140 lines) +├── workflow_sweep.py # Workflow Sweep core: parse API workflow, bindings -> jobs, prompt substitution, fingerprints (pure, no torch/server) (1106 lines) +├── workflow_sweep_runner.py # Workflow Sweep runner: ComfyBridge (in-process post_prompt mirror), windowed feeder thread, pause/resume/cancel, route helpers (1027 lines) +├── workflow_sweep_collect.py # USCGSweepCollect / USCGSweepCollectVideo output nodes: save each job's image (or mp4 + poster) + manifest item into the USCG session (281 lines) ├── html_generator.py # Reads /resources/ to build the Dashboard HTML (66 lines) ├── run_api.py # Pure helpers for POST /config_tester/run + run_status, no server imports (144 lines) ├── conftest.py # Registry import stub: lets pytest import this package without ComfyUI (213 lines) @@ -77,7 +77,7 @@ ComfyUI-Ultimate-Sampler-Grid/ │ ├── conf-builder-config-management.js # Save/Load/Duplicate config logic, all Builder rendering (7736 lines) │ ├── conf-builder-utilities.js # Model-list cache, iteration counting, LoRA string helpers (866 lines) │ ├── conf-builder-distribution.js # Distribution settings UI (workers, toggles) (620 lines) -│ ├── conf-builder-workflow-sweep.js # Workflow Sweep UI (load API workflow, axes, dry run, start/pause/cancel) (744 lines) +│ ├── conf-builder-workflow-sweep.js # Workflow Sweep UI (load API workflow, axes, dry run, start/pause/cancel, image/video output picker) (806 lines) │ └── conf-builder-readme.md # Notes on the Builder module split (134 lines) │ ├── resources/ # [The Dashboard SPA Core] (listed in html_generator.js_files order) @@ -195,6 +195,7 @@ Defined at the end of `__init__.py`: | `UltimateConfigBuilder` | `UltimateConfigBuilder` | Visual config builder — outputs `configs_json` string | | `SmartJSONText` | `SmartJSONTextNode` | JSON text widget with syntax highlighting | | `USCGSweepCollect` | `USCGSweepCollect` | Workflow Sweep output node (`workflow_sweep_collect.py`): saves each job's image + manifest item into the USCG session | +| `USCGSweepCollectVideo` | `USCGSweepCollectVideo` | Workflow Sweep video output node (same file): takes a VIDEO, saves one mp4 via `VideoInput.save_to` + a middle-frame poster, writes an LTX-shaped manifest item. VHS frames are converted by a core `CreateVideo` node the sweep splices into the job prompt (`workflow_sweep.build_job_prompt`) | `WEB_DIRECTORY = "./web"` — tells ComfyUI to serve `web/` files at `/extensions/`. @@ -254,7 +255,7 @@ All routes registered via `PromptServer.instance.routes`. One row per route this | POST | `/config_tester/compute_metrics` | `__init__.py` | `{session_name, overwrite?}` | Start the background metrics job for a session; returns `{job_id, total_images}` | | GET | `/config_tester/metrics_status` | `__init__.py` | `?job_id=` | Metrics job progress | | POST | `/config_tester/cancel_metrics` | `__init__.py` | `{job_id}` | Cancel a metrics job; returns `{cancelled}` | -| POST | `/config_tester/sweep/parse` | `__init__.py` | `{api_json}` | Workflow Sweep: describe an API-format workflow (nodes, outputs, image sources, suggested bindings) | +| POST | `/config_tester/sweep/parse` | `__init__.py` | `{api_json}` | Workflow Sweep: describe an API-format workflow (nodes, outputs, `image_sources`, `video_sources`, `default_output`, suggested bindings) | | POST | `/config_tester/sweep/dry_run` | `__init__.py` | `{state or configs_json, limit?}` | Workflow Sweep: first N concrete prompts + validation of job 1; queues nothing | | GET | `/config_tester/sweep/job_prompt` | `__init__.py` | `?session_name=&sweep_id=&job_index=` | Workflow Sweep: `{prompt, params, fingerprint, output, ...}`, the job's API prompt exactly as it ran (template + substitutions) minus the USCG collector node; ids only, no paths (400 bad id/index, 404 unknown) | | POST | `/config_tester/sweep/start` | `__init__.py` | `{state or configs_json, node_id?, client_id?}` | Workflow Sweep: start a sweep `{state or configs_json, node_id?}` (400 invalid; 200 `attached:true` when the IDENTICAL sweep is already active; 409 `{error, sweep_id, session_name}` for a different one) | @@ -876,7 +877,6 @@ __init__.py 29. **Image Metrics Run Context** - `image_metrics.begin_run(spec)` (wrapped around `run_generation_loop` in `sampler_node.py`) returns a token and `end_run(token)` closes only that context, so a second run or the dashboard upscale thread can never clobber it. Generation-side hooks read the active context (`decode_latent_with_vae(..., metrics_sink=)`, `flush_batch_with_vae`, orchestrator upscale inserts via `image_metrics.finalize`); the dashboard upscale thread (`upscale_runner.py`) must NOT read generation-run state, so it passes an explicit spec derived from the source item (`spec_from_item`). Every place that copies a source item into an upscaled entry excludes `"metrics"` (never inherit); the new entry is re-measured. `metrics` is session-level (`_session_settings.metrics`), not a per-config key. 30. **Workflow Sweep never blocks the executor.** `USCGSweepCollect` and the Generator in workflow mode only start/record work; the runner feeds sweep prompts through `PromptServer.instance.prompt_queue` from its own thread (an in-process mirror of `server.py post_prompt`, loop-bound calls via `ComfyBridge.run_on_loop`, never from the loop thread, no lock held across it). A node that waited on a prompt it queued would deadlock ComfyUI's single executor thread. The expansion/substitution core (`workflow_sweep.py`) is pure; only `ComfyBridge` touches ComfyUI, so tests use a fake bridge. - 31. **pytest package-name trap**: pytest runs with `--import-mode=importlib` and a root `pytest.ini`. A parent folder named `integration`, `metrics` etc. (no hyphen) is imported as a package and the stub conftest breaks. Worktree folder names need a hyphen; never add `tests/__init__.py`. 32. **Manifest writers preserve disk state**: every writer (generation saves, the remote VAE worker, the `save_manifest` route) keeps the on-disk `comparisons` (`manifest_utils.preserve_disk_comparisons`) and the per-item `metrics` written by `metrics_runner` (an item's on-disk `metrics` survives when the in-memory copy has none, in `manifest_utils.merge_manifest_user_changes`). Only `update_manifest_comparisons` writes votes; only `metrics_runner` rewrites `metrics` in place. @@ -889,6 +889,8 @@ __init__.py 36. **`html_generator.js_files` order is load-bearing** (one shared script scope): `logic_ratings.js` (pure) before `logic_ui.js`, `logic_compare.js` after `logic_pipeline.js`, `logic_standalone.js` LAST (top-level-page overrides). CI assembles the page and syntax-checks it. +37. **Workflow Sweep video outputs.** `workflow_output` carries optional `media: "video"` and (VHS frames) `frames_from: `; both are emitted only for video, and `fingerprint()` / `template_hash()` add `m`/`f` to the hashed payload only for video, so image fingerprints stay byte-identical (pinned by golden values in `tests/test_workflow_sweep_video.py`; changing the image payload would orphan every old session's resume). VHS frames are converted IN THE GRAPH by a spliced core `CreateVideo` (`build_job_prompt`), never in the collector, so the executor handles cache/hidden inputs. `USCGSweepCollectVideo` saves with `VideoInput.save_to` (SaveVideo itself needs executor-populated hidden inputs). `manifest_utils.append_items(remove_ids=...)` deletes a removed video's `.preview.png` along with the mp4. No new top-level Builder field, so the rule-5 checklist does not apply. + --- - **Item id ranges:** generated ids are `int(time.time()*100000)+rand` (below 2^52 until the year ~3400); scanned ids are `manifest_utils.stable_item_id("scan:")` in `[2^52, 2^53)` (JS-safe integers). A re-scan of an existing session reuses old ids (legacy random ids included) via `merge_rescan`/`write_rescan_manifest` and carries favorited/rejected/note/notes/metrics/comparisons. diff --git a/README.md b/README.md index b1dc463..4c48a03 100644 --- a/README.md +++ b/README.md @@ -883,7 +883,8 @@ How it behaves: * Each sweep writes `benchmarks//workflow_sweeps/.json` (templates, bindings, per-job status) so any executed prompt can be reconstructed. * **Re-running the same sweep attaches to it.** Queueing the Generator again (or pressing Start) while the identical sweep is active does not fail and does not queue anything twice: it attaches to the running sweep (console: "Same workflow sweep already running ... attaching") and a paused sweep stays paused (Resume is the Builder's button). "Identical" means the same session name, the same ordered job fingerprints, the same *Start at job* and the same *Overwrite existing*. Anything else gets a clear refusal ("A workflow sweep is already active (session ..., sweep ..., n/m done, state). This run was NOT started. Pause or cancel it ..."; ComfyUI shows it in its error dialog, the route answers 409 with `session_name`). *Add random seeds* are deterministic (derived from the base seed, 0 like the Generator), so the same sweep always produces the same jobs: re-queueing attaches, and a resume after a finished or interrupted run skips the jobs already in the session. * **Loading a sweep session in the Builder** (Load Session) restores the Workflow Sweep arrays (workflow, output, bindings, custom prompts) from the `configs_json` that sweep sessions now store in their manifest meta. Sessions made before this change have no saved configs: the Builder shows a notice ("Load the workflow JSON again into a Workflow Sweep array; finished jobs are skipped by fingerprint") and leaves the arrays alone. -* v1 limits: one image output per job, one sweep at a time, a workflow sweep cannot share a Generator run with standard configs (use two Builders), no upscale/distribution/trigger-word features, and Revise does not apply. **Video outputs are not collected yet** (planned: core `SaveVideo.video` / `CreateVideo` VIDEO sockets and the frames feeding `VHS_VideoCombine.images`, through a `USCGSweepCollectVideo` node). +* * **Video outputs.** *Output to collect* lists *Images* and *Video* sources. Video sources are: the feeder of a `SaveVideo` (a VIDEO socket, e.g. `CreateVideo`), the frames feeding a `VHS_VideoCombine` (marked "frames"; the sweep splices a core `CreateVideo` node in front of the collector with the VHS node's `frame_rate` and `audio`, so a swept `frame_rate` works), and any node that returns a VIDEO. A SaveImage/PreviewImage feeder is still the default when the workflow has one; a video-only workflow (no SaveImage) now defaults to its video output. Each job saves ONE mp4 (H.264, through ComfyUI's own `VideoInput.save_to`, the workflow embedded as metadata) plus a middle-frame poster `vid_.preview.png` into `benchmarks//images`, with the same manifest fields as LTX videos (`media_type: "video"`, `video_path`, `preview_path`, `poster`, `frames`, `frame_rate`, `duration_seconds`). Cards play the video; **JOB** stays available and the Compare view plays both videos. Resume and *Overwrite existing* work as for images (the old mp4 and its poster are deleted); a video and an image collection of the same socket never skip each other. Limits: VHS `format`, `loop_count` and `pingpong` are ignored (always mp4/h264); a VHS node fed with latents (`vae` input) or a `meta_batch` is not offered as a source; one video per job; the workflow's own SaveVideo/VHS node only runs with *Also run the workflow's own outputs*; collecting a file-backed VIDEO (e.g. `LoadVideo`) decodes all its frames into RAM once to make the poster. +* v1 limits: one image or video output per job, one sweep at a time, a workflow sweep cannot share a Generator run with standard configs (use two Builders), no upscale/distribution/trigger-word features, and Revise does not apply. **Dashboard for sweeps.** Items carry `wf_params` (the bound values, numbers or strings). The **Quick Filter** gets one extra type per *varied* key (`WF cfg`, `WF sampler_name`, ...; picking a value filters and the facet panel lists them), the **Logic Filter** gets a *Workflow params* group of the numeric keys (`wf:KSampler.cfg >= 5`), and the **Labels** panel has a *Workflow params* checkbox that tags each card with its varied values (`cfg:4`). Standard sessions show none of this. The **JOB** button opens a read-only *Workflow job* panel (source, output, sweep id, job number, parameter table) with **Download job prompt (API JSON)** and **Copy job prompt**: `GET /config_tester/sweep/job_prompt?session_name=&sweep_id=&job_index=` rebuilds the prompt exactly as it ran (template + substitutions) minus the USCG collector node, so queueing it in ComfyUI reproduces the image. If the browser blocks the download inside the dashboard iframe, use Copy. diff --git a/Roadmap.md b/Roadmap.md index fa87e3c..303359c 100644 --- a/Roadmap.md +++ b/Roadmap.md @@ -129,7 +129,7 @@ A button in the builder UI to pack short sha256 into config with an explanation * **Problem:** Can only load full sessions, not merge snippets. -#### **23. Pseudo-JSON Nodes (Recursion)** (low priority) - STARTED: v1 is the **Workflow Sweep** (Builder: Add Workflow Sweep): any API-format workflow runs as-is with chosen inputs swept (see README, 'Workflow Sweep'). Still open: inserting nodes into the USCG pipeline itself, video outputs, mixed workflow + standard runs. (v1 shipped 11/4/26) +#### **23. Pseudo-JSON Nodes (Recursion)** (low priority) - STARTED: v1 is the **Workflow Sweep** (Builder: Add Workflow Sweep): any API-format workflow runs as-is with chosen inputs swept (see README, 'Workflow Sweep'). Still open: inserting nodes into the USCG pipeline itself, mixed workflow + standard runs (video outputs shipped 11/4/26). (v1 shipped 11/4/26) * **Problem:** Advanced. Running a raw JSON workflow as a sub-node. Inserting any node into any part of the genereation would make this tool have an immense customizability increase. More info needed on how this could work, would like to see a visual interface for it in the builder UI eventaully. (big job, very low priority) @@ -189,3 +189,4 @@ Verify the node ids of the optional aesthetic / reward scorer packs against the #### **32. Workflow Sweep: video outputs and mixed runs** Allow video outputs from a sweep and sweep + standard configs in one Generator run (both are v1 limits). +DONE (11/4/26): video outputs (SaveVideo / CreateVideo VIDEO sockets, VHS_VideoCombine frames via a spliced CreateVideo, `USCGSweepCollectVideo`). Still open: mixed sweep + standard runs in one Generator run; SaveWEBM / SaveAnimatedWEBP / SaveAnimatedPNG as frame sources (same CreateVideo splice); several outputs per job.