Commit Graph
48 Commits
Author SHA1 Message Date
Jason Hoku 3d097fc962 Merge branch 'feat/upscale-phase2-pack-nodes' into int/wave4 2026-11-04 19:16:09 -08:00
Jason HokuandClaude Fable 5.1 e37e8e861c feat: Dashboard Upscale phase 2 (inline editing, cancel) + embed a ComfyUI nodes workflow in every image (Roadmap 93, 71)
Item 71: comfy_nodes_workflow.py is an exact Python port of the Revise view's "copy as nodes" graph
builder (pinned to a golden file generated from the JS, now in resources/logic_comfynodes.js) plus
lossless PNG tEXt / WebP EXIF writers in the format ComfyUI's frontend reads on drop. The Builder's new
Run Setting "Embed ComfyUI Nodes Workflow" (session_settings.pack_nodes_metadata) stamps every saved
image at every save site (generation, remote VAE, inline/deferred upscales, dashboard upscales, worker
images via a master pass after collection); export_favorites falls back to the Python builder.

Item 93: the upscale modal gets step/pipeline reordering, inline validation (mirrored server-side,
400 on bad configs), a preset bar (Update / Save as / Revert / Delete, read-modify-write), "Run once
(unsaved)", an output summary, separate Cancel and Close, a running lock and reattach after closing.
Cancel now lands within one sampler step through a per-job unet function wrapper on a cloned model
(never the process-wide interrupt flag), keeps completed outputs, and sends the final progress event.
Also fixed: the dashboard received the session's OLDEST items as "new" after an upscale (slice at the
wrong end), the poll targeted the wrong button, jobs were never pruned, two jobs could rewrite one
session's manifest (409 guard).

Known quirk recorded, not changed: a final pipeline step with several ratio/denoise values saves only
the last combination; the summary and a warning now say so.

Tests: 101 python, 59 node, 9 Playwright.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 19:01:49 -08:00
Jason Hoku 851ec552d7 Merge branch 'feat/analytics-sobel' into int/wave4 2026-11-04 18:52:33 -08:00
Jason Hoku e755f6d60e Merge branch 'feat/dist-open' into int/wave4 2026-11-04 18:50:05 -08:00
Jason HokuandClaude Fable 5.1 a1f3d3f646 feat(dashboard): Stats panel per setting (Roadmap 26 M1) + Sobel edge metric (Roadmap 25)
- resources/logic_analytics.js: computeAnalytics(items) in ONE pass over the items: per-axis rows
  (model, lora, sampler, scheduler, steps, cfg, denoise, resolution, attention, upscale, pass,
  prompt, and every wf_params key) with count, favorites %, rejects %, mean duration, mean of
  every builtin metric and the pairwise rating; sortAnalyticsRows, analyticsToMarkdown
- resources/logic_analytics_panel.js: docked non-modal panel (header button + cog entry), scope
  "current filter (favs/rejects counted)" or "whole session", sortable columns, dim rows under
  n=3, Copy Markdown, re-renders (throttled) on every data change, Escape closes; builds DOM
  with textContent only
- logic_pipeline.js: pipelineFilterCtx/pipelineMatches extracted (behaviour identical)
- sobel: mean Sobel gradient magnitude of luma (kernels /4; a full step edge = 1), numpy and
  torch parity; Dashboard sort/filter/labels, Builder metric list; backfill gains fill_missing
  (adds keys missing from older items, never changes stored values); the distributed metrics
  pass fills metrics a worker on an older USCG omitted
- tests: 15 python, 12 node, 5 Playwright (incl. an XSS-shaped manifest value rendered as text)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 18:49:25 -08:00
Jason HokuandClaude Fable 5.1 b109ded9e3 feat(distributed): Start At Job, claim-time reject re-check, LTX on the master, live ETA, no lost worker results (Roadmap 30/73/13)
- plan: start_at_job skips the first N-1 plan entries (counted as skipped), LTX entries are never
  skipped as "existing" and are split out of the worker queue (legacy companion: clear error)
- claim filter: _make_reject_recheck re-reads the manifest (throttled, mtime/size keyed, under the
  manifest lock) so a preview rejected mid-run is dropped when a worker claims its full render
- LTX phase: video jobs render on the master (_render_ltx_job, shared with the local loop) while
  workers render the image jobs; Start At Job also skips LTX in the local loop
- progress: ultimate_grid.progress events during distributed runs (phase "distributed", machines
  count), ETA from the manager's rolling per-claimer durations via on_status; format_eta_str
- model sync (73-A): the master's claim loop now waits for workers inside the loop and renders
  reclaimed jobs itself after wait_timeout (max(600, claim_timeout)), so a long worker download
  no longer ends with the worker's image deleted as a duplicate
- tests: 35 python, 5 node; docs (ProjectStructure events + wait_timeout, README parity/model
  sync/troubleshooting, Roadmap 30/73/13)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 18:48:52 -08:00
Jason HokuandClaude Fable 5.1 48ab82dacc feat(dashboard): JS/CSS bundle routes with immutable caching (Roadmap 27 M2)
The dashboard shell no longer inlines ~450 KB of scripts and CSS: two fixed routes,
GET /config_tester/dashboard_bundle_js and _css (?v=<content hash>, weak ETag, 304, gzip,
Cache-Control immutable when the hash matches, no-cache otherwise), serve the concatenated
resources (pinned order, standalone last; rebuilt when a resource file changes, so F5 hot
reload still works). The shell drops from ~527 KB to ~68 KB (12 KB gzipped) and repeat loads
cost ~0 KB; the node iframe (srcdoc inherits the parent's base URL) uses the same routes, so
ComfyUI's executed/history payloads shrink too. inline_assets=True (implied by inline_manifest,
or USCG_INLINE_DASHBOARD_ASSETS=1) keeps a fully self-contained page. The v query is only
compared, never used for lookup; no path parameters.

Tests: 16 unit (order, page independence, inline == bundled bytes, hash rebuild, headers,
v-is-never-a-path, node --check of globals + bundle), e2e (top-level + srcdoc iframe + inline +
500 path), CI assembled-page step covers globals + bundle as one script.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 18:39:18 -08:00
Jason HokuandClaude Fable 5.1 0e563a19cb feat(prompts): COUPLE regional prompts via native masked conditioning (#11)
`base COUPLE left | right` (2-4 equal columns; COUPLE(rows) for bands; region ` :w` weight)
splits a prompt into an unmasked base cond plus one masked cond per region, blended by
ComfyUI's own mask/mask_strength path (latent-couple style, one extra UNet pass per region,
works for every model family). COUPLE is parsed first; each part still takes AND/CAT/AVG/BREAK.

- regional_prompt.py: parser, join helpers (byte-identical output without COUPLE, cache keys
  depend on it), size-independent region tags, latent-size mask builder with tile/crop frames,
  apply_regional_masks at sample time
- batch_encoding: COUPLE prompts take the combinator path (never the batched TE path)
- conditioning_cache: conditioning_v2 keeps every entry and extra key (the old format silently
  dropped AND prompts); legacy entries are a one-time miss
- trigger words, model suffix and HiRes append_end land in the base, not the last region
- tiled HiRes and Florence2 crops pass their frame; invalid syntax stops the run early;
  LTX video warns; Builder tooltip + README section + ProjectStructure/AI-CONTEXT
- 107 new tests (parser, masks, encoding, cache round trip, source guards)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 17:50:15 -08:00
Jason Hoku 756bae4b9b Merge branch 'feat/lazy-manifest' into int/wave3 2026-11-04 03:31:09 -08:00
Jason Hoku 87096b0e4d Merge branch 'feat/dist-parity' into int/wave3 2026-11-04 03:28:34 -08:00
Jason HokuandClaude Fable 5.1 a6eaa1b469 docs: distributed parity (job order, preview pass, metrics)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:27:42 -08:00
Jason HokuandClaude Fable 5.1 e65583fe04 docs: lazy manifest route, shell page and the roadmap status of item 27
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:25:59 -08:00
Jason Hoku c231303179 Merge branch 'feat/sweep-video' into int/wave3 2026-11-04 03:10:14 -08:00
Jason HokuandClaude Fable 5.1 6564b65372 docs(sweep): video outputs (README, ProjectStructure, AI-CONTEXT, Roadmap 32)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:09:35 -08:00
Jason HokuandClaude Fable 5.1 0501686340 fix(latent-input): connected latent size = cells x the model's latent ratio, not x 8
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 03:03:01 -08:00
Jason Hoku 9296006317 Merge branch 'feat/pairwise-polish' into int/wave2 2026-11-04 01:21:37 -08:00
Jason HokuandClaude Fable 5.1 097c3648f7 feat(compare): phone compare with swipe-to-vote and reliable muted video autoplay
Swipe left = A, right = B via a node-testable cmpSwipeSide(); pane taps go
through cmpPaneTap so a swipe never also casts a tap vote. Videos get
muted/playsinline set before the source loads and play() only after insertion,
with a tap-to-play fallback. Docs: README + ProjectStructure.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:20:18 -08:00
Jason HokuandClaude Fable 5.1 aa7950d958 docs: no outbound HTTP rule replaces the urllib advice; scan_directory route documented as it is; companion facades in the file tree; superseded_previews JS test moved under tests/js so CI runs it
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:17:42 -08:00
Jason Hoku e88d769815 Merge branch 'feat/drag-drop' into int/wave2 2026-11-04 01:10:41 -08:00
Jason Hoku 2a19e1ca99 Merge branch 'feat/docs-pass' into int/wave2 2026-11-04 01:07:49 -08:00
Jason HokuandClaude Fable 5.1 e7eacd4bd5 test(dashboard): e2e for modal drag carriers, send button bridge and standalone download; docs (#22)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:05:41 -08:00
Jason HokuandClaude Fable 5.1 0c813d58f6 docs: bring README, ProjectStructure, Roadmap, AI-CONTEXT and CONTRIBUTING in line with main (11/4/26)
README.md
- Moved: "Pairwise Review (blind A/B compare + scoreboard)" from a stranded "##" between File
  Locations and Keyboard Shortcuts to a "###" inside Dashboard Interface (after Revision Modal,
  before Standalone Dashboard); anchor unchanged.
- Moved: "Image Metrics (Optional)" from after the Config Builder's closing rule to right after
  "Job Order & Preview Pass", before "Workflow Sweep"; the rule now follows Workflow Sweep.
- TOC rebuilt to mirror document order (Quick Start, Optional Companion Plugins, Config Builder
  as its own branch, Pairwise/Standalone/Image Metrics/Job Order/Skip Layer Guidance/MOSS/Workflow
  Sweep, API routes, Development / Tests); fixes the broken "json-bars" anchor (three hyphens).
- Key Features: one bullet each for Workflow Sweep, Job Order, Preview Pass, Skip Layer Guidance,
  16 px latent models, Examples folder, Pairwise Review, Standalone Dashboard + Run API.
- Run Settings: pointer line to Job Order & Preview Pass, Image Metrics and te_batch_size.
- API Integration: pointer sentence after the intro, "Provided by the ComfyUI-USCG-Distributed
  companion plugin" under Distribution API, new "Dashboard, Review, Metrics & Sweep Routes" table.
- Notes for AI Development: gotcha 2 rewritten (one config-to-JSON implementation), gotcha 13 loses
  the "if True" sentence, new gotcha 14 pointing at ProjectStructure.md (36 gotchas), File
  Dependencies and Server Event Types updated, Development / Tests shortened to 3 commands + link.
- Compare mode shortcut row: "see below" is now a link to the moved Pairwise section.
- Changelog: new "Update 11/4/26" entry above 11/3/26 (older entries untouched).
- Removed duplicate sentences (TE batching): in the MOSS-Image bullet "Prompts that can't be
  batched (weighted, AND/CAT/AVG()/BREAK) and encoders other than MOSS-Image / Krea 2 /
  Qwen-Image keep the per-prompt path." (already in the te_batch_size bullets) and "The summary
  line shows how many prompts were batched." (moved into the te_batch_size bullet as
  "The pre-encode summary line shows how many prompts were batched."). Dropped from
  Development / Tests: the stale counts (137 passed / 15 skipped / 152 passed), the torch and
  playwright install lines and the CI job paragraph (all kept in CONTRIBUTING.md).

ProjectStructure.md
- File map regenerated with measured line counts; adds florence2_hires, ltx_video_generation,
  civitai/distribution facades, conftest, pytest.ini, sampler_help_note.js, conf-builder-workflow-sweep.js,
  conf-builder-readme.md, tests/, tests/e2e, tests/js, examples, .github/workflows; removes
  network_utils, distribution_manager, distribution_worker, distribution_routes; resources/ in
  js_files order. Section 2b rewritten as the companion facade.
- Node Class Mappings: USCGSweepCollect, no line number.
- API Routes: one row per registered route with File and Body / query columns (Line column
  dropped); Distribution table moved under "Companion plugin routes".
- Server Event Types: update_data, next_run_seeds, metrics_progress, phase payloads.
- Gotchas: #1 now "Single Data Path" with the add-a-field checklist, #2/#26/Testing Changes/Key
  Functions/utilities.js rows stop naming convertStateToConfigs, the unnumbered Workflow Sweep
  quote became #30 (text verbatim), new #31-#36.

Roadmap.md: #23, #25, #26, #27 status notes appended, new "Open follow-ups (11/4/26)" #29-#32.
AI-CONTEXT.md: real line counts and new file rows, "One Source of Truth for Config JSON" replaces
the two-codepaths section, network_utils/convertStateToConfigs/js load order/gotcha count fixed.
CONTRIBUTING.md: measured counts, CI job table, the CI JS syntax loops and assembled-page check,
node --test, e2e flags, checklist updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:05:30 -08:00
Jason Hoku cff2144ae5 Merge branch 'feat/metrics-adapter' into int/wave2 2026-11-04 01:05:06 -08:00
Jason HokuandClaude Fable 5.1 da307385c2 docs(metrics): Optional scorer packs section (aesthetic install recipe, measured cost, ImageReward status)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 01:04:13 -08:00
Jason HokuandClaude Fable 5.1 bd18b5ae32 docs: sweep attach rule, wf_params dashboard, job prompt route, Builder session load
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 00:56:25 -08:00
Jason Hoku d1714a2b8b Merge branch 'feat/workflow-sweep' into integration
# Conflicts:
#	README.md
#	__init__.py
#	sampler_node.py
2026-11-04 00:21:39 -08:00
Jason HokuandClaude Fable 5.1 e25c08a039 feat(workflow-sweep): Builder workflow-sweep section and dashboard labels
New ES module conf-builder-workflow-sweep.js (load API workflow, output picker,
suggested bindings, node table, axis editors, dry run modal, start/pause/resume/cancel
with status polling), wired additively into the config-array list, default state and
migrations. Dashboard ETA bar shows 'Sweep job' with an error count and hides REVISE
for workflow items. README, ProjectStructure and Roadmap updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 00:19:10 -08:00
Jason Hoku add7fbb2cb Merge branch 'feat/metrics' into integration
# Conflicts:
#	resources/logic_pipeline.js
#	resources/logic_state.js
#	resources/logic_ui.js
#	web/conf_builder/conf-builder-config-management.js
#	web/conf_builder/conf-builder-main.js
2026-11-04 00:16:58 -08:00
Jason Hoku 3c3fd7a49d Merge branch 'feat/preview-pass-ordering' into integration
# Conflicts:
#	tests/test_state_to_configs_json.py
#	web/conf_builder/conf-builder-main.js
2026-11-04 00:15:02 -08:00
Jason HokuandClaude Fable 5.1 a6146c0cc9 feat(metrics): compute metrics for an existing session (background job)
metrics_runner follows the upscale_runner job pattern: CPU-only PIL->numpy,
manifest writes under the lock (reload, set by id, atomic replace), progress
event + polling endpoints, cancel. Dashboard cog-panel button.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 00:13:05 -08:00
Jason Hoku 58bd73fb64 Merge branch 'feat/pairwise-review' into integration
# Conflicts:
#	ProjectStructure.md
#	resources/report.css
2026-11-04 00:10:09 -08:00
Jason Hoku ecf01d62b8 Merge branch 'feat/batched-te' into integration
# Conflicts:
#	web/conf_builder/conf-builder-main.js
2026-11-04 00:09:39 -08:00
Jason HokuandClaude Fable 5.1 acee4fd545 feat(metrics): Builder Run Settings control, session-settings emission and metrics_info route
Session-level setting (not a per-config key). state_to_configs_json emits
_session_settings.metrics only when enabled and non-empty; GET
/configbuilder/metrics_info reports built-ins and scorer-pack install status.
Faster numpy fallback; README and ProjectStructure updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 00:09:19 -08:00
Jason HokuandClaude Fable 5.1 6aabea97de docs: pairwise review
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 00:06:55 -08:00
Jason Hoku fd7f39cc33 Merge branch 'feat/dashboard-url-api' into integration 2026-11-04 00:06:41 -08:00
Jason HokuandClaude Fable 5.1 ffb352fb92 feat(builder): job order and preview pass run settings
State keys job_order, preview_pass{enabled,steps,scale,cfg_override}, skip_rejected_previews,
only_favorited_previews: new-state defaults + migration, Run Settings controls, emission with
coercion in state_to_configs_json, tests, README/ProjectStructure.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 00:06:08 -08:00
Jason HokuandClaude Fable 5.1 98fcf0861c feat(encode): te_batch_size Run Setting wires text-encoder batching into pre-encode
batch_encode_with_cache/batch_encode_prompts take te_batch_size (default 1 =
off, DEFAULT_TE_BATCH_SIZE) and a hidden te_batch_verify; batched prompts are
cached, leftovers take the unchanged per-prompt loop, and the summary line
reports batched prompts/forwards. Builder state default + migration, Run
Settings control, state_to_configs_json emission, orchestrator read, README.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 00:05:57 -08:00
Jason HokuandClaude Fable 5.1 0d1c44d1b4 feat(examples): ready-to-load workflows per model family (closes #6)
examples/ ships one workflow per family (SDXL/Illustrious, Anima, Z-Image
Turbo, Krea 2 Turbo, Flux.2 Klein 9B, MOSS-Image), each a Builder -> Sampler
Grid -> Dashboard graph with the Builder state pre-filled (model, text
encoder, CLIP type, VAE, sampling axes, prompt). examples/README.md lists
the files and folders each needs and what the grid tests.
tests/test_examples.py validates graph wiring, Builder state, emitted
configs and the image counts promised in the README.

Closes #6

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 00:02:21 -08:00
Jason HokuandClaude Fable 5.1 dd0e190257 feat(dashboard): phone layout, lightbox action bar, docs for standalone page and run API
Touch-sized card/lightbox/header buttons under a coarse-pointer media query,
a sticky Prev/Favorite/Reject/Revise/Next bar in the lightbox, and a 2-column
session landing on the standalone page at phone width (all scoped so the node
iframe is unchanged). README documents the standalone dashboard and the run
API with curl examples; ProjectStructure lists the routes and new files.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-11-04 00:00:07 -08:00
Jason Hoku dbc10089a9 docs: update /distribution/* references to /uscg-distribution/*
Phase 2 renamed the routes to /uscg-distribution/* during the extraction
of the Distributed companion plugin. ProjectStructure.md (8 occurrences)
and README.md (6 occurrences) still referenced the old paths.

Caught alongside the worker URL bug during Phase C smoke test.
2026-05-19 15:55:22 -07:00
Jason HokuandClaude Opus 4.6 be233a9369 docs: update AI-CONTEXT, ProjectStructure, and Roadmap for this session
- AI-CONTEXT: added network_utils.py, upscale_runner.py, updated
  session settings format (pipelines, deferred upscale, start_at_job)
- ProjectStructure: added new files (network_utils, upscale_runner,
  logic_upscale_modal), preset files, updated line counts
- Pipeline editor refactor plan saved for next session

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-18 00:17:59 -07:00
JasonHokuandClaude Opus 4.6 fc6137462d docs: restore project documentation to tracking
Re-add AI-CONTEXT.md, ProjectStructure.md, README.md, and Roadmap.md
to git tracking. Only docs/plans/ remains gitignored as local-only.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 15:34:28 -07:00
JasonHokuandClaude Opus 4.6 6d1fcdb87f chore: gitignore local-only docs and remove from tracking
ProjectStructure.md, README.md, Roadmap.md, AI-CONTEXT.md, and docs/
are local development references — not needed in the repo.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-09 15:32:37 -07:00
JasonHokuandClaude Opus 4.6 989da199a3 Update README.md and ProjectStructure.md for v3.0
- Add distributed processing features (multi-worker, master text
  encoding, model sync, per-worker logging)
- Add comprehensive AI development notes with architecture diagrams,
  file dependency graphs, gotchas, and testing instructions
- Update file structure to include all 21 Python modules, distribution
  files, and conf-builder-distribution.js
- Add distribution API routes reference
- Add distribution module descriptions and communication patterns
- Update data flow to include distribution pipeline
- Add 8 new critical development gotchas (13-20)
- Add new changelog entry for March 2026
- Bump version to 3.0

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 01:08:34 -08:00
JasonHoku 824157d4cf use comfy built-in endpoitns to comply with comfy best practices 2026-02-16 17:36:15 -08:00
JasonHoku 67a8cf65dd Integrate New Security Rules From Comfy Registry Scans. 2026-02-16 16:03:05 -08:00
JasonHoku 048ee3ca1b Update Readme & Project Structure 2026-02-16 14:56:26 -08:00
JasonHoku 30894c4c82 Massive Overhaul, Config Builder Updates, Fixes, Smarter Optional Caching, UI Improvements, Filter Improvements & More! 2026-02-10 16:30:10 -08:00