Commit Graph
581 Commits
Author SHA1 Message Date
Jason Hoku e755f6d60e Merge branch 'feat/dist-open' into int/wave4 2026-11-04 18:50:05 -08:00
Jason Hoku 4b1180dc93 Merge branch 'feat/sweep-mixed' into int/wave4 2026-11-04 18:49:26 -08:00
Jason HokuandClaude Fable 5.1 b109ded9e3 feat(distributed): Start At Job, claim-time reject re-check, LTX on the master, live ETA, no lost worker results (Roadmap 30/73/13)
- plan: start_at_job skips the first N-1 plan entries (counted as skipped), LTX entries are never
  skipped as "existing" and are split out of the worker queue (legacy companion: clear error)
- claim filter: _make_reject_recheck re-reads the manifest (throttled, mtime/size keyed, under the
  manifest lock) so a preview rejected mid-run is dropped when a worker claims its full render
- LTX phase: video jobs render on the master (_render_ltx_job, shared with the local loop) while
  workers render the image jobs; Start At Job also skips LTX in the local loop
- progress: ultimate_grid.progress events during distributed runs (phase "distributed", machines
  count), ETA from the manager's rolling per-claimer durations via on_status; format_eta_str
- model sync (73-A): the master's claim loop now waits for workers inside the loop and renders
  reclaimed jobs itself after wait_timeout (max(600, claim_timeout)), so a long worker download
  no longer ends with the worker's image deleted as a duplicate
- tests: 35 python, 5 node; docs (ProjectStructure events + wait_timeout, README parity/model
  sync/troubleshooting, Roadmap 30/73/13)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 18:48:52 -08:00
Jason HokuandClaude Fable 5.1 725cb79fd4 feat(sweep): mixed standard + Workflow Sweep runs; SaveWEBM/AnimatedWEBP/AnimatedPNG frame sources (Roadmap 32)
- one Generator run may hold standard configs AND workflow-sweep entries: the sweep is validated
  first (job 1 + busy check, nothing registered), the standard loop runs, then start_sweep launches
  into the same session with one global job numbering (Start At Job counts standard jobs first).
  An interrupt in the loop never starts the sweep; meta.configs_json keeps the full Builder list
  (_standard_run_meta/_completion_payload helpers; next_phase on the completion event; the
  dashboard keeps the ETA bar up with "Workflow sweep next…" and clears the 30 s hide timer on
  any later progress event)
- frame sources: FRAME_SINKS table (VHS_VideoCombine, SaveWEBM, SaveAnimatedWEBP, SaveAnimatedPNG)
  feeding the CreateVideo splice; fps read per sink, clamped to CreateVideo's 1-120 with a plan
  warning; find_output_nodes fallback names; Builder label via framesFps
- tests: 33 python (mixed routing, Start At Job translation, interrupt, pre-flight busy/invalid,
  post-loop busy, three sink fixtures, clamping, VHS golden unchanged), 2 node, 2 Playwright

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 18:48:44 -08:00
Jason Hoku 02de2bf03b Merge branch 'feat/lora-strength-rows' into int/wave4 2026-11-04 18:39:41 -08:00
Jason Hoku 00617e60aa Merge branch 'feat/bundle-routes' into int/wave4 2026-11-04 18:39:41 -08:00
Jason Hoku 20089b58b3 Merge branch 'feat/roadmap-verify' into int/wave4 2026-11-04 18:39:41 -08:00
Jason HokuandClaude Fable 5.1 eb745988b0 test: verify the roadmap's "needs testing" items; fix what they found
- Builder VAE beats the sampler node's remote VAE (normalize_config_vae; the GGUF pre-check
  now treats "None"/"" like Default instead of failing late)
- batch encoding interrupt: verified (no further prompt encoded after the interrupt)
- pre-encode only configs with work left: configs_needing_work() extracted and tested;
  optional-input tracking stays open (fingerprints know nothing about upstream nodes)
- omit logic: "(masterpiece)"-style terms never matched (\b next to punctuation) and a blank
  term omitted everything; fixed with lookarounds + blank skip; the Builder's omit-vs-lookup
  warning fires only when LoRA Triggerwords Mode is None (was a false positive in every other
  mode) and a console warning (omit_without_lookup_warning) prints at run start
- drag reorder: pure reorderTarget/moveItem extracted and tested; doubled handle glyph on
  text-encoder rows and non-handle drops firing a move fixed; browser pass still open
- Builder slowness: section-preset GETs cached per section (one uncached fetch per row per
  render before); three remaining hotspots recorded in Roadmap.md

Roadmap.md: items 77, 81, 104, 9, 10 marked DONE; 83 and 91 annotated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 18:39:33 -08:00
Jason HokuandClaude Fable 5.1 65e8df81c4 feat(builder): LoRA strength rows (Roadmap 12)
A "+" beside the model-strength slider adds another strength to compare (one slider when
Lock is on, model + CLIP when off; up to 8 per LoRA, duplicates greyed out). The state keeps
only the extra rows (lora_extra_strengths[name] = [{model, clip}]), row 0 stays the verbatim
base string so existing jobs keep their fingerprints. state_to_configs_json emits concrete
"name:m:c" strings on the lora axis (stacks = every combination, first LoRA slowest); the old
"Compare Strengths" comma box is removed and converted on first edit. The Iterations count
now mirrors process_lora_array (fixes the stack overcount). Dashboard: a LoRA whose name is on
every card but with several strengths keeps its label; filter buttons keep the strengths.

Tests: 33 python (shared fixture, byte-identical output without the key, expand_configs end to
end, xfail for the pre-existing folder-drop bug), 8 node, 10 Playwright (the Builder modules
load from the fake origin with the real state_to_configs_json answering the preview).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 18:39:25 -08:00
Jason HokuandClaude Fable 5.1 48ab82dacc feat(dashboard): JS/CSS bundle routes with immutable caching (Roadmap 27 M2)
The dashboard shell no longer inlines ~450 KB of scripts and CSS: two fixed routes,
GET /config_tester/dashboard_bundle_js and _css (?v=<content hash>, weak ETag, 304, gzip,
Cache-Control immutable when the hash matches, no-cache otherwise), serve the concatenated
resources (pinned order, standalone last; rebuilt when a resource file changes, so F5 hot
reload still works). The shell drops from ~527 KB to ~68 KB (12 KB gzipped) and repeat loads
cost ~0 KB; the node iframe (srcdoc inherits the parent's base URL) uses the same routes, so
ComfyUI's executed/history payloads shrink too. inline_assets=True (implied by inline_manifest,
or USCG_INLINE_DASHBOARD_ASSETS=1) keeps a fully self-contained page. The v query is only
compared, never used for lookup; no path parameters.

Tests: 16 unit (order, page independence, inline == bundled bytes, hash rebuild, headers,
v-is-never-a-path, node --check of globals + bundle), e2e (top-level + srcdoc iframe + inline +
500 path), CI assembled-page step covers globals + bundle as one script.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 18:39:18 -08:00
Jason Hoku 69b93afa5e Merge branch 'feat/issue11-couple' into int/issues 2026-11-04 17:50:15 -08:00
Jason HokuandClaude Fable 5.1 0e563a19cb feat(prompts): COUPLE regional prompts via native masked conditioning (#11)
`base COUPLE left | right` (2-4 equal columns; COUPLE(rows) for bands; region ` :w` weight)
splits a prompt into an unmasked base cond plus one masked cond per region, blended by
ComfyUI's own mask/mask_strength path (latent-couple style, one extra UNet pass per region,
works for every model family). COUPLE is parsed first; each part still takes AND/CAT/AVG/BREAK.

- regional_prompt.py: parser, join helpers (byte-identical output without COUPLE, cache keys
  depend on it), size-independent region tags, latent-size mask builder with tile/crop frames,
  apply_regional_masks at sample time
- batch_encoding: COUPLE prompts take the combinator path (never the batched TE path)
- conditioning_cache: conditioning_v2 keeps every entry and extra key (the old format silently
  dropped AND prompts); legacy entries are a one-time miss
- trigger words, model suffix and HiRes append_end land in the base, not the last region
- tiled HiRes and Florence2 crops pass their frame; invalid syntax stops the run early;
  LTX video warns; Builder tooltip + README section + ProjectStructure/AI-CONTEXT
- 107 new tests (parser, masks, encoding, cache round trip, source guards)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 17:50:15 -08:00
Jason Hoku 02dfde3355 Merge branch 'feat/issue6-sweep-example' into int/issues 2026-11-04 17:35:03 -08:00
Jason Hoku 858b9c823e Merge branch 'feat/issue16-attention' into int/issues 2026-11-04 17:35:02 -08:00
Jason HokuandClaude Fable 5.1 6498fdb19a docs(examples): Workflow Sweep example on an SDXL API graph (#6)
The answer to "non standard workflow" grids: the sweep runs the user's exact exported graph per
job (reference-image conditioning, ControlNet, custom nodes...), changing only the bound inputs.
uscg-workflow-sweep-sdxl.json sweeps cfg x sampler x prompt (8 jobs) on an embedded SDXL graph;
README section; test_examples covers it (8 jobs, 8 distinct fingerprints).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 17:32:49 -08:00
Jason HokuandClaude Fable 5.1 bc0cca324c fix(attention): attention modes never worked - wrap the override the way ComfyUI calls it (#16)
ComfyUI's wrap_attn() calls transformer_options["optimized_attention_override"] as
override(original_func, q, k, v, heads, ...). We installed the registered attention
function itself, so q was the original function: "'function' object has no attribute
'shape'" for every installed non-default mode (sage, flash, xformers, pytorch, ...).
A mode that was not installed fell back to default silently, which is why "sage3 worked".

- make_attention_override() drops the leading function and forwards the rest
- generate_image_attn_safe(): one clear log line naming the mode, one retry with the
  default attention; the item records attention_fallback
- README lists the real mode names (sageattn was never one)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 17:29:46 -08:00
Jason HokuandClaude Fable 5.1 daf3cbce49 Merge branch 'fix/review-d' (ultrareview slices D + C fixes)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 17:22:29 -08:00
Jason HokuandClaude Fable 5.1 f88129cb37 fix: ultrareview slice C (workflow sweep) findings
- append_items never replaces an unreadable manifest with a fresh one (raises, file untouched) and
  writes through the retrying atomic helper; the sweep spec write retries PermissionError too
- breaker only pauses while jobs remain; a paused sweep with nothing in flight or pending completes
  instead of blocking new sweeps with SweepBusy
- sampler IS_CHANGED returns NaN when configs_json is linked (absent from kwargs), so a re-queued
  sweep is never served from ComfyUI's cache
- Start At Job # = N skips N-1 jobs like the orchestrator
- dry_run / api_start expand the sweep in an executor, off the event loop
- Builder sweep section re-reads configArray.workflow on every render (load/remove replaced it)
- favorites export and GENERATE NEW skip workflow-sweep items (no template)
- Pairwise Review excludes all wf_* per-job keys
- test: same-sweep attach waits for the feeder's first enqueue (pre-existing race)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 16:44:19 -08:00
Jason HokuandClaude Fable 5.1 e35030f0a4 fix: ultrareview slice D (waves 2+3) findings
- lazy shell: SAVE never persists __uscg_lazy__ (saveState guard + save_manifest pop); the manifest
  fetch route heals files that already carry it
- dashboard node: a Builder session switch does a full reload (session_name from the payload, not
  the stale widget) instead of re-fetching the old session
- dashboard node: window message handlers only accept a dashboard iframe as source
- scanned ids are namespaced by session (same relpath in two sessions no longer collides on merge);
  rescan merge donates each old id once and only via unambiguous basenames
- deferred upscale resume is keyed on base_fp == config_fp (fixed-seed grids upscale every cell)
- legacy distributed companion ignores the preview pass: master keeps the upscale phase
- connected latent: recompute w/h before gen_w/gen_h so 16x models record real dimensions
- Start on an identical paused sweep resumes it
- outputFromKey keeps subgraph node ids ("57:35") whole

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 15:25:46 -08:00
Jason Hoku 756bae4b9b Merge branch 'feat/lazy-manifest' into int/wave3 2026-11-04 03:31:09 -08:00
Jason HokuandClaude Fable 5.1 f9724aeaa8 test(distribution): source guard accepts the plan-based populate call
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:29:07 -08:00
Jason HokuandClaude Fable 5.1 d5e6e0662a perf(dashboard): defer the session-list fetch until the first manifest fetch is done
/config_tester/list_sessions scans every session folder on the server's event loop; fired in
parallel at boot it held the (executor-compressed) manifest response back by ~200 ms.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:29:06 -08:00
Jason Hoku 87096b0e4d Merge branch 'feat/dist-parity' into int/wave3 2026-11-04 03:28:34 -08:00
Jason HokuandClaude Fable 5.1 a6eaa1b469 docs: distributed parity (job order, preview pass, metrics)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:27:42 -08:00
Jason HokuandClaude Fable 5.1 e65583fe04 docs: lazy manifest route, shell page and the roadmap status of item 27
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:25:59 -08:00
Jason HokuandClaude Fable 5.1 5cf0782ce0 feat(dashboard): lazy manifest loading, the page is a shell and the SPA fetches /config_tester/manifest
html_generator.get_html_template now emits a shell (empty fullManifest flagged
__uscg_lazy__, inline_manifest=True restores the old behaviour). New route
GET /config_tester/manifest?session=<name>: compact JSON, weak ETag, 304 on
If-None-Match, gzip, torn-read retry (logic in run_api.load_manifest_for_fetch).
/config_tester/dashboard gets gzip + content ETag. The SPA boots with one fetch
(no more double load), resync is conditional and non-destructive, and
web/dashboard.js asks the iframe to re-fetch (reload_manifest) instead of
parsing data out of the Generator's executed message. Also fixes the dead
session-name regex in dashboard_node.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:25:59 -08:00
Jason HokuandClaude Fable 5.1 1637087e57 feat(dist): preview pass in distributed mode (master renders and stamps previews like the normal loop)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:10:46 -08:00
Jason HokuandClaude Fable 5.1 c6e1b94234 fix(compare): sweep per-job bookkeeping (wf_fingerprint, wf_sweep_id, wf_job_index, poster) no longer counts as a setting
Matched and one-axis pairing on a sweep session found zero pairs because every item differed in these per-image fields (same class as the wave-2 config_fp bug).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:10:44 -08:00
Jason HokuandClaude Fable 5.1 34b963793e feat(dist): image metrics for worker-rendered images
Workers send builtin metrics measured on the decoded tensor; the master scores adapters
(and any missing builtins, from the saved file) in one pass after collection.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:10:16 -08:00
Jason Hoku c231303179 Merge branch 'feat/sweep-video' into int/wave3 2026-11-04 03:10:14 -08:00
Jason HokuandClaude Fable 5.1 6564b65372 docs(sweep): video outputs (README, ProjectStructure, AI-CONTEXT, Roadmap 32)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:09:35 -08:00
Jason HokuandClaude Fable 5.1 db90661d0d fix(sweep): overwrite deletes a video's poster too; related-variants reel shows the poster for videos
Found in the live check on a local ComfyUI: overwriting left 4 orphaned .preview.png files, and the
modal's related-variants reel rendered broken <img> tags for mp4 items.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:09:35 -08:00
Jason HokuandClaude Fable 5.1 f2c3e1a620 feat(dist): job order and fingerprints in distributed mode, master decides the plan
The master computes the ordered plan, skips and config fingerprints and hands them to the
companion (populate_from_plan, falls back to populate_jobs on an old companion). Items are
tagged dist_worker, which the pairwise review treats as bookkeeping, not a setting.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:08:03 -08:00
Jason Hoku dd36d19b5c Merge branch 'feat/orch-fixes' into int/wave3 2026-11-04 03:07:53 -08:00
Jason HokuandClaude Fable 5.1 e1e268213f test(sweep): video sources, fingerprints (golden image hashes), collector node, output keys
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:04:55 -08:00
Jason HokuandClaude Fable 5.1 0501686340 fix(latent-input): connected latent size = cells x the model's latent ratio, not x 8
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:03:01 -08:00
Jason HokuandClaude Fable 5.1 3a27bba9d1 fix(distribution): master post-phase upscales this run's images per model group
The distributed post-phase called run_deferred_upscales with the wrong arguments and
raised on every run. Build the queue from this run's manifest items matched to their job
configs, and upscale one model/LoRA/VAE group at a time with a fresh conditioning cache.
HiRes prompt is now encoded when not cached.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:02:58 -08:00
Jason HokuandClaude Fable 5.1 7fd1a64741 feat(sweep): Builder output picker lists video sources; dashboard keeps JOB for sweep videos
No new top-level Builder field: workflow_output gains optional media/frames_from keys, emitted only
for video, so the rule-5 checklist does not apply and image configs JSON is unchanged.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:02:38 -08:00
Jason HokuandClaude Fable 5.1 884c1d7ef5 fix(dist): distributed runs keep rejected-preview tombstones, job order and run settings in manifest meta
The distributed path overwrote meta with a short dict in four places, erasing
rejected_preview_fps and meta.session_settings.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:02:31 -08:00
Jason HokuandClaude Fable 5.1 dc7f8c7b8c docs(metrics): ImageReward stays disabled, with the transformers-5 blockers found by the wave-3 feasibility check
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:01:56 -08:00
Jason HokuandClaude Fable 5.1 930874f068 feat(sweep): USCGSweepCollectVideo collector node (mp4 via VideoInput.save_to, poster, LTX-shaped item)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:01:52 -08:00
Jason HokuandClaude Fable 5.1 9aad5ad6da feat(sweep): runner passes template media to build_job_prompt, spec records it
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:01:11 -08:00
Jason HokuandClaude Fable 5.1 dd0c12ec4d feat(sweep): video output sources, media-aware fingerprint and collector prompt
find_video_sources (SaveVideo / VHS frames / VIDEO-returning nodes), default_output rule
(SaveImage feeder still wins), normalize_output returns media + frames_from, video-only
fingerprint/template_hash payload additions (image hashes byte-identical), CreateVideo
splice in build_job_prompt. No new Builder field (workflow_output gains optional media keys).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 02:58:49 -08:00
Jason HokuandClaude Fable 5.1 6b601632f6 fix(upscale): deferred upscales (run_upscales_at_end) no longer skip every image with 'no filename in manifest'
Generated manifest items carry only a /view URL in 'file'; the deferred phase read 'filename', which only
upscaled and scanned items have, so every deferred job was skipped. New pure helper manifest_utils.item_filename
(filename key, else the URL's filename= parameter) is used there; the dashboard upscale path already parsed the URL.
Found by the wave-2 live base-dims check on the 3070.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:29:46 -08:00
Jason HokuandClaude Fable 5.1 197b3f1a5d ci: refresh the expected test counts after wave 2 (comment only)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:22:47 -08:00
Jason Hoku 9296006317 Merge branch 'feat/pairwise-polish' into int/wave2 2026-11-04 01:21:37 -08:00
Jason HokuandClaude Fable 5.1 097c3648f7 feat(compare): phone compare with swipe-to-vote and reliable muted video autoplay
Swipe left = A, right = B via a node-testable cmpSwipeSide(); pane taps go
through cmpPaneTap so a swipe never also casts a tap vote. Videos get
muted/playsinline set before the source loads and play() only after insertion,
with a tap-to-play fallback. Docs: README + ProjectStructure.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:20:18 -08:00
Jason HokuandClaude Fable 5.1 aa7950d958 docs: no outbound HTTP rule replaces the urllib advice; scan_directory route documented as it is; companion facades in the file tree; superseded_previews JS test moved under tests/js so CI runs it
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:17:42 -08:00
Jason Hoku e88d769815 Merge branch 'feat/drag-drop' into int/wave2 2026-11-04 01:10:41 -08:00
Jason HokuandClaude Fable 5.1 4174791916 feat(standalone): share-link button and session picker button for phones
Header buttons (standalone only): share via navigator.share, else clipboard on
secure contexts, else execCommand with the result checked, else a prompt.
The picker button opens the settings panel at the session list; phone CSS gives
the rows touch-sized targets. A background websocket resync no longer closes
the settings panel the user has open.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:09:45 -08:00