Commit Graph
565 Commits
Author SHA1 Message Date
Jason HokuandClaude Fable 5.1 daf3cbce49 Merge branch 'fix/review-d' (ultrareview slices D + C fixes)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 17:22:29 -08:00
Jason HokuandClaude Fable 5.1 f88129cb37 fix: ultrareview slice C (workflow sweep) findings
- append_items never replaces an unreadable manifest with a fresh one (raises, file untouched) and
  writes through the retrying atomic helper; the sweep spec write retries PermissionError too
- breaker only pauses while jobs remain; a paused sweep with nothing in flight or pending completes
  instead of blocking new sweeps with SweepBusy
- sampler IS_CHANGED returns NaN when configs_json is linked (absent from kwargs), so a re-queued
  sweep is never served from ComfyUI's cache
- Start At Job # = N skips N-1 jobs like the orchestrator
- dry_run / api_start expand the sweep in an executor, off the event loop
- Builder sweep section re-reads configArray.workflow on every render (load/remove replaced it)
- favorites export and GENERATE NEW skip workflow-sweep items (no template)
- Pairwise Review excludes all wf_* per-job keys
- test: same-sweep attach waits for the feeder's first enqueue (pre-existing race)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 16:44:19 -08:00
Jason HokuandClaude Fable 5.1 e35030f0a4 fix: ultrareview slice D (waves 2+3) findings
- lazy shell: SAVE never persists __uscg_lazy__ (saveState guard + save_manifest pop); the manifest
  fetch route heals files that already carry it
- dashboard node: a Builder session switch does a full reload (session_name from the payload, not
  the stale widget) instead of re-fetching the old session
- dashboard node: window message handlers only accept a dashboard iframe as source
- scanned ids are namespaced by session (same relpath in two sessions no longer collides on merge);
  rescan merge donates each old id once and only via unambiguous basenames
- deferred upscale resume is keyed on base_fp == config_fp (fixed-seed grids upscale every cell)
- legacy distributed companion ignores the preview pass: master keeps the upscale phase
- connected latent: recompute w/h before gen_w/gen_h so 16x models record real dimensions
- Start on an identical paused sweep resumes it
- outputFromKey keeps subgraph node ids ("57:35") whole

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 15:25:46 -08:00
Jason Hoku 756bae4b9b Merge branch 'feat/lazy-manifest' into int/wave3 2026-11-04 03:31:09 -08:00
Jason HokuandClaude Fable 5.1 f9724aeaa8 test(distribution): source guard accepts the plan-based populate call
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:29:07 -08:00
Jason HokuandClaude Fable 5.1 d5e6e0662a perf(dashboard): defer the session-list fetch until the first manifest fetch is done
/config_tester/list_sessions scans every session folder on the server's event loop; fired in
parallel at boot it held the (executor-compressed) manifest response back by ~200 ms.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:29:06 -08:00
Jason Hoku 87096b0e4d Merge branch 'feat/dist-parity' into int/wave3 2026-11-04 03:28:34 -08:00
Jason HokuandClaude Fable 5.1 a6eaa1b469 docs: distributed parity (job order, preview pass, metrics)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:27:42 -08:00
Jason HokuandClaude Fable 5.1 e65583fe04 docs: lazy manifest route, shell page and the roadmap status of item 27
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:25:59 -08:00
Jason HokuandClaude Fable 5.1 5cf0782ce0 feat(dashboard): lazy manifest loading, the page is a shell and the SPA fetches /config_tester/manifest
html_generator.get_html_template now emits a shell (empty fullManifest flagged
__uscg_lazy__, inline_manifest=True restores the old behaviour). New route
GET /config_tester/manifest?session=<name>: compact JSON, weak ETag, 304 on
If-None-Match, gzip, torn-read retry (logic in run_api.load_manifest_for_fetch).
/config_tester/dashboard gets gzip + content ETag. The SPA boots with one fetch
(no more double load), resync is conditional and non-destructive, and
web/dashboard.js asks the iframe to re-fetch (reload_manifest) instead of
parsing data out of the Generator's executed message. Also fixes the dead
session-name regex in dashboard_node.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:25:59 -08:00
Jason HokuandClaude Fable 5.1 1637087e57 feat(dist): preview pass in distributed mode (master renders and stamps previews like the normal loop)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:10:46 -08:00
Jason HokuandClaude Fable 5.1 c6e1b94234 fix(compare): sweep per-job bookkeeping (wf_fingerprint, wf_sweep_id, wf_job_index, poster) no longer counts as a setting
Matched and one-axis pairing on a sweep session found zero pairs because every item differed in these per-image fields (same class as the wave-2 config_fp bug).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:10:44 -08:00
Jason HokuandClaude Fable 5.1 34b963793e feat(dist): image metrics for worker-rendered images
Workers send builtin metrics measured on the decoded tensor; the master scores adapters
(and any missing builtins, from the saved file) in one pass after collection.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:10:16 -08:00
Jason Hoku c231303179 Merge branch 'feat/sweep-video' into int/wave3 2026-11-04 03:10:14 -08:00
Jason HokuandClaude Fable 5.1 6564b65372 docs(sweep): video outputs (README, ProjectStructure, AI-CONTEXT, Roadmap 32)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:09:35 -08:00
Jason HokuandClaude Fable 5.1 db90661d0d fix(sweep): overwrite deletes a video's poster too; related-variants reel shows the poster for videos
Found in the live check on a local ComfyUI: overwriting left 4 orphaned .preview.png files, and the
modal's related-variants reel rendered broken <img> tags for mp4 items.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:09:35 -08:00
Jason HokuandClaude Fable 5.1 f2c3e1a620 feat(dist): job order and fingerprints in distributed mode, master decides the plan
The master computes the ordered plan, skips and config fingerprints and hands them to the
companion (populate_from_plan, falls back to populate_jobs on an old companion). Items are
tagged dist_worker, which the pairwise review treats as bookkeeping, not a setting.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:08:03 -08:00
Jason Hoku dd36d19b5c Merge branch 'feat/orch-fixes' into int/wave3 2026-11-04 03:07:53 -08:00
Jason HokuandClaude Fable 5.1 e1e268213f test(sweep): video sources, fingerprints (golden image hashes), collector node, output keys
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:04:55 -08:00
Jason HokuandClaude Fable 5.1 0501686340 fix(latent-input): connected latent size = cells x the model's latent ratio, not x 8
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:03:01 -08:00
Jason HokuandClaude Fable 5.1 3a27bba9d1 fix(distribution): master post-phase upscales this run's images per model group
The distributed post-phase called run_deferred_upscales with the wrong arguments and
raised on every run. Build the queue from this run's manifest items matched to their job
configs, and upscale one model/LoRA/VAE group at a time with a fresh conditioning cache.
HiRes prompt is now encoded when not cached.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:02:58 -08:00
Jason HokuandClaude Fable 5.1 7fd1a64741 feat(sweep): Builder output picker lists video sources; dashboard keeps JOB for sweep videos
No new top-level Builder field: workflow_output gains optional media/frames_from keys, emitted only
for video, so the rule-5 checklist does not apply and image configs JSON is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:02:38 -08:00
Jason HokuandClaude Fable 5.1 884c1d7ef5 fix(dist): distributed runs keep rejected-preview tombstones, job order and run settings in manifest meta
The distributed path overwrote meta with a short dict in four places, erasing
rejected_preview_fps and meta.session_settings.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:02:31 -08:00
Jason HokuandClaude Fable 5.1 dc7f8c7b8c docs(metrics): ImageReward stays disabled, with the transformers-5 blockers found by the wave-3 feasibility check
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:01:56 -08:00
Jason HokuandClaude Fable 5.1 930874f068 feat(sweep): USCGSweepCollectVideo collector node (mp4 via VideoInput.save_to, poster, LTX-shaped item)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:01:52 -08:00
Jason HokuandClaude Fable 5.1 9aad5ad6da feat(sweep): runner passes template media to build_job_prompt, spec records it
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:01:11 -08:00
Jason HokuandClaude Fable 5.1 dd0c12ec4d feat(sweep): video output sources, media-aware fingerprint and collector prompt
find_video_sources (SaveVideo / VHS frames / VIDEO-returning nodes), default_output rule
(SaveImage feeder still wins), normalize_output returns media + frames_from, video-only
fingerprint/template_hash payload additions (image hashes byte-identical), CreateVideo
splice in build_job_prompt. No new Builder field (workflow_output gains optional media keys).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 02:58:49 -08:00
Jason HokuandClaude Fable 5.1 6b601632f6 fix(upscale): deferred upscales (run_upscales_at_end) no longer skip every image with 'no filename in manifest'
Generated manifest items carry only a /view URL in 'file'; the deferred phase read 'filename', which only
upscaled and scanned items have, so every deferred job was skipped. New pure helper manifest_utils.item_filename
(filename key, else the URL's filename= parameter) is used there; the dashboard upscale path already parsed the URL.
Found by the wave-2 live base-dims check on the 3070.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:29:46 -08:00
Jason HokuandClaude Fable 5.1 197b3f1a5d ci: refresh the expected test counts after wave 2 (comment only)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:22:47 -08:00
Jason Hoku 9296006317 Merge branch 'feat/pairwise-polish' into int/wave2 2026-11-04 01:21:37 -08:00
Jason HokuandClaude Fable 5.1 097c3648f7 feat(compare): phone compare with swipe-to-vote and reliable muted video autoplay
Swipe left = A, right = B via a node-testable cmpSwipeSide(); pane taps go
through cmpPaneTap so a swipe never also casts a tap vote. Videos get
muted/playsinline set before the source loads and play() only after insertion,
with a tap-to-play fallback. Docs: README + ProjectStructure.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:20:18 -08:00
Jason HokuandClaude Fable 5.1 aa7950d958 docs: no outbound HTTP rule replaces the urllib advice; scan_directory route documented as it is; companion facades in the file tree; superseded_previews JS test moved under tests/js so CI runs it
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:17:42 -08:00
Jason Hoku e88d769815 Merge branch 'feat/drag-drop' into int/wave2 2026-11-04 01:10:41 -08:00
Jason HokuandClaude Fable 5.1 4174791916 feat(standalone): share-link button and session picker button for phones
Header buttons (standalone only): share via navigator.share, else clipboard on
secure contexts, else execCommand with the result checked, else a prompt.
The picker button opens the settings panel at the session list; phone CSS gives
the rows touch-sized targets. A background websocket resync no longer closes
the settings panel the user has open.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:09:45 -08:00
Jason Hoku 2a19e1ca99 Merge branch 'feat/docs-pass' into int/wave2 2026-11-04 01:07:49 -08:00
Jason HokuandClaude Fable 5.1 40ef4fb4e8 feat(dashboard): stacked phone lightbox with a Compare action
Preview on top, form below, six-button action bar (Prev, Favorite, Reject,
Compare, Revise, Next) as icon+label spans. Desktop and the fine-pointer node
iframe are untouched. Favorite state is synced via _syncModalFavBtn (also on
open). e2e: new tests/e2e/test_phone_dashboard.py + phone_dashboard fixture.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:05:42 -08:00
Jason HokuandClaude Fable 5.1 e7eacd4bd5 test(dashboard): e2e for modal drag carriers, send button bridge and standalone download; docs (#22)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:05:41 -08:00
Jason HokuandClaude Fable 5.1 0c813d58f6 docs: bring README, ProjectStructure, Roadmap, AI-CONTEXT and CONTRIBUTING in line with main (11/4/26)
README.md
- Moved: "Pairwise Review (blind A/B compare + scoreboard)" from a stranded "##" between File
  Locations and Keyboard Shortcuts to a "###" inside Dashboard Interface (after Revision Modal,
  before Standalone Dashboard); anchor unchanged.
- Moved: "Image Metrics (Optional)" from after the Config Builder's closing rule to right after
  "Job Order & Preview Pass", before "Workflow Sweep"; the rule now follows Workflow Sweep.
- TOC rebuilt to mirror document order (Quick Start, Optional Companion Plugins, Config Builder
  as its own branch, Pairwise/Standalone/Image Metrics/Job Order/Skip Layer Guidance/MOSS/Workflow
  Sweep, API routes, Development / Tests); fixes the broken "json-bars" anchor (three hyphens).
- Key Features: one bullet each for Workflow Sweep, Job Order, Preview Pass, Skip Layer Guidance,
  16 px latent models, Examples folder, Pairwise Review, Standalone Dashboard + Run API.
- Run Settings: pointer line to Job Order & Preview Pass, Image Metrics and te_batch_size.
- API Integration: pointer sentence after the intro, "Provided by the ComfyUI-USCG-Distributed
  companion plugin" under Distribution API, new "Dashboard, Review, Metrics & Sweep Routes" table.
- Notes for AI Development: gotcha 2 rewritten (one config-to-JSON implementation), gotcha 13 loses
  the "if True" sentence, new gotcha 14 pointing at ProjectStructure.md (36 gotchas), File
  Dependencies and Server Event Types updated, Development / Tests shortened to 3 commands + link.
- Compare mode shortcut row: "see below" is now a link to the moved Pairwise section.
- Changelog: new "Update 11/4/26" entry above 11/3/26 (older entries untouched).
- Removed duplicate sentences (TE batching): in the MOSS-Image bullet "Prompts that can't be
  batched (weighted, AND/CAT/AVG()/BREAK) and encoders other than MOSS-Image / Krea 2 /
  Qwen-Image keep the per-prompt path." (already in the te_batch_size bullets) and "The summary
  line shows how many prompts were batched." (moved into the te_batch_size bullet as
  "The pre-encode summary line shows how many prompts were batched."). Dropped from
  Development / Tests: the stale counts (137 passed / 15 skipped / 152 passed), the torch and
  playwright install lines and the CI job paragraph (all kept in CONTRIBUTING.md).

ProjectStructure.md
- File map regenerated with measured line counts; adds florence2_hires, ltx_video_generation,
  civitai/distribution facades, conftest, pytest.ini, sampler_help_note.js, conf-builder-workflow-sweep.js,
  conf-builder-readme.md, tests/, tests/e2e, tests/js, examples, .github/workflows; removes
  network_utils, distribution_manager, distribution_worker, distribution_routes; resources/ in
  js_files order. Section 2b rewritten as the companion facade.
- Node Class Mappings: USCGSweepCollect, no line number.
- API Routes: one row per registered route with File and Body / query columns (Line column
  dropped); Distribution table moved under "Companion plugin routes".
- Server Event Types: update_data, next_run_seeds, metrics_progress, phase payloads.
- Gotchas: #1 now "Single Data Path" with the add-a-field checklist, #2/#26/Testing Changes/Key
  Functions/utilities.js rows stop naming convertStateToConfigs, the unnumbered Workflow Sweep
  quote became #30 (text verbatim), new #31-#36.

Roadmap.md: #23, #25, #26, #27 status notes appended, new "Open follow-ups (11/4/26)" #29-#32.
AI-CONTEXT.md: real line counts and new file rows, "One Source of Truth for Config JSON" replaces
the two-codepaths section, network_utils/convertStateToConfigs/js load order/gotcha count fixed.
CONTRIBUTING.md: measured counts, CI job table, the CI JS syntax loops and assembled-page check,
node --test, e2e flags, checklist updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:05:30 -08:00
Jason HokuandClaude Fable 5.1 ecf28d45aa test(docs): consistency checks between the docs and the code, refresh stale CI count comments
tests/test_docs_consistency.py (stdlib only, reads files, no process or network modules):
- README anchor links resolve (GitHub slug rule pinned), code fences balanced
- every README "## " section and the key features are linked from the TOC
- newest changelog entry is the 11/4/26 one
- every route registered with routes.get/post is in ProjectStructure.md (method + path)
- the file map lists every root .py, web/ file, resources/ file, tests/e2e, tests/js,
  examples, tests.yml and names no .py that does not exist
- every ultimate_grid.* event sent by the code is in Server Event Types (+ sweep phase payload)
- html_generator.js_files ends with logic_standalone.js and is listed in the docs
- no stale convertStateToConfigs / network_utils.py mentions
- CONTRIBUTING.md names every CI job id and the CI flags

tests.yml: comment-only edit, the "# expect:" counts now match the measured
499 passed / 16 skipped (stub leg).

Note: the docs checks are expected to be red at this commit; the next commit
(docs) makes them green.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:05:17 -08:00
Jason Hoku cff2144ae5 Merge branch 'feat/metrics-adapter' into int/wave2 2026-11-04 01:05:06 -08:00
Jason HokuandClaude Fable 5.1 da307385c2 docs(metrics): Optional scorer packs section (aesthetic install recipe, measured cost, ImageReward status)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:04:13 -08:00
Jason HokuandClaude Fable 5.1 3d6b14d82a feat(scan): stable item ids so a re-scan keeps votes, favorites, notes and metrics
Scanned items get ids from blake2b(scan:<relpath>) in [2^52, 2^53) (JS-safe,
disjoint from generated ids), collisions resolved deterministically by salt.
A re-scan of an existing session reuses old ids (legacy random ids included)
and carries favorited/rejected/note/notes/metrics, top-level comparisons and
meta.rejected_preview_fps, written under _manifest_lock. Fixes the route
dropping the dashboard's 'note' field.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:01:56 -08:00
Jason Hoku 637fac3cf8 Merge branch 'feat/builder-polish' into int/wave2 2026-11-04 01:00:33 -08:00
Jason HokuandClaude Fable 5.1 2dbd42d35b feat(upscale): exact base_width/base_height on every upscaled item; fix matched-mode ctx keys and no-detection pairing
- Record the size of the generated image an upscale started from at all
  10 sites (deferred, inline Florence2 + sentinel, inline SeedVR2, inline
  hires/model, and the dashboard upscale_runner paths via base_dims_of).
- CMP_EXCLUDED_KEYS now ignores per-image fields (config_fp, base_fp, metrics,
  preview_of, florence2_*), so matched/one-axis pairing works on current manifests.
- Compare pool skips Florence2 no-detection sentinels (they alias the base file).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 00:59:54 -08:00
Jason HokuandClaude Fable 5.1 8ce4398c4d feat(builder-polish): run-API full pass, vae drift fix, honest minimize_reloads wording
- Record session_settings + has_optional_inputs in manifest meta (run_meta_extras) so the
  Dashboard "Full pass for survivors" can queue POST /config_tester/run headless; the iframe
  falls through to it when the parent has no Builder / no queuePrompt.
- Exclude the deliberately omitted "vae" key from _EXPECTED_CONFIG_KEYS; test_examples asserts zero drift.
- minimize_reloads: log VAE switches + a "changed nothing" note, accurate tooltip/README wording.
- examples/README: Klein base-variant note; fix stale LTX stub comment.
- Tests: run_meta_extras, job order, state_to_configs_json, e2e full pass (new file).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 00:59:45 -08:00
Jason Hoku f746ed4a61 Merge branch 'feat/sweep-polish' into int/wave2 2026-11-04 00:59:05 -08:00
Jason HokuandClaude Fable 5.1 e88b18a549 fix(dashboard): drag the Revise image onto Load Image / canvas, plus a Send to Load Image button (#22)
Rich dragstart (asset-info JSON, uri-list, plain, DownloadURL; reference not copy), a modal button that
fills the selected/only/new Load Image via the parent bridge, standalone Download image fallback.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 00:58:31 -08:00
Jason HokuandClaude Fable 5.1 b45fde0410 fix(dashboard): remove leftover merge marker that dropped the first pairwise CSS rule
CSS error recovery merged the stray ======= line with the following selector,
so .cmp-setting-row was lost in every browser. A lint test guards resources/.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 00:58:30 -08:00
Jason HokuandClaude Fable 5.1 9b8eed833b fix(sweep): deterministic extra seeds
add_random_seeds_to_gens now uses config_utils.derive_extra_seeds (base seed 0, like the Generator),
so a re-queue attaches to the running sweep and a resume skips finished jobs.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 00:56:59 -08:00
Jason HokuandClaude Fable 5.1 bd18b5ae32 docs: sweep attach rule, wf_params dashboard, job prompt route, Builder session load
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 00:56:25 -08:00