diff --git a/docs/maintainers/README.md b/docs/maintainers/README.md index eb10885..c4d43b0 100644 --- a/docs/maintainers/README.md +++ b/docs/maintainers/README.md @@ -18,6 +18,7 @@ and should be kept in sync with these prose docs. | [`published-setup-external-ui-contract.md`](published-setup-external-ui-contract.md) | Product/design contract for how ComfyUI publish nodes expose controls and result fields to an external frontend. | | [`published-setup-full-circle-plan.md`](published-setup-full-circle-plan.md) | Remaining implementation plan for the full authoring → publish → exported setup/API → external runner loop. | | [`external-frontend-job-submitter-brief.md`](external-frontend-job-submitter-brief.md) | Handoff brief for an AI or developer session building the real external frontend job submitter. | +| [`external-frontend-kit.md`](external-frontend-kit.md) | Complete file/dependency/fixture checklist for another AI session building the production external frontend. | | [`node-versioning.md`](node-versioning.md) | Hard-won rules for changing `INPUT_TYPES` / `RETURN_TYPES` / class names without breaking saved user workflows. The five rules + the suffix-version pattern (`MyNode` → `MyNodeV2`) + the alias-then-deprecate migration path. | | [`curated-sidebar.md`](curated-sidebar.md) | The Kforge Labs sidebar starter-preset workflow. How to customize the bundled `web/starter_preset.json` shipped to fresh users (Snapshot → Load → starter). Storage keys, library directory resolution, reset procedure. | | [`workflows-sidebar.md`](workflows-sidebar.md) | The "Workflows" section of the same sidebar. Save canvas / save selection / directories / archive on collision / load with named tab. Storage shape, action semantics, reset procedure. | diff --git a/docs/maintainers/external-frontend-job-submitter-brief.md b/docs/maintainers/external-frontend-job-submitter-brief.md index 67dc3a3..ad4753a 100644 --- a/docs/maintainers/external-frontend-job-submitter-brief.md +++ b/docs/maintainers/external-frontend-job-submitter-brief.md @@ -8,6 +8,11 @@ Primary reference: - [`published-setup-external-ui-contract.md`](published-setup-external-ui-contract.md) - [`published-setup-full-circle-plan.md`](published-setup-full-circle-plan.md) - [`published-setups.md`](published-setups.md) +- [`external-frontend-kit.md`](external-frontend-kit.md) + +For a complete package of files, dependencies, fixtures, and implementation +handoff notes to give another AI session, start with +[`external-frontend-kit.md`](external-frontend-kit.md). ## Product Goal diff --git a/docs/maintainers/external-frontend-kit.md b/docs/maintainers/external-frontend-kit.md new file mode 100644 index 0000000..34f1bdb --- /dev/null +++ b/docs/maintainers/external-frontend-kit.md @@ -0,0 +1,267 @@ +# External Frontend Kit For Published Setups + +This kit is the handoff package for an AI or developer session building a +production external frontend that runs Koolook-published ComfyUI setups. + +It assumes the frontend is not embedded in ComfyUI. The frontend talks to the +running Koolook/ComfyUI server, renders controls from the published setup +contract, submits declared values, polls run status, and shows the resolved +result path. + +## Current State + +The current merged implementation has the core full-circle pieces needed by an +external frontend: + +- Sidebar publishing captures ComfyUI's API-format prompt and stores it as + `apiPrompt`. +- Published setup records preserve the visual graph, executable `apiPrompt`, + inferred `setupSurface.app`, metadata, validation state, and sidebar source + breadcrumbs. +- `source.inventoryPath` records the sidebar folder hierarchy so external apps + can group setups the same way the ComfyUI sidebar was organized. +- The runner clones the stored `apiPrompt`, injects only declared fields, + prunes switch-selected writer branches when an execution map is available, + submits to the same local ComfyUI server, and reports queued/running/ + succeeded/failed/lost states. +- The simulator under `web/setup_runner_simulator.html` is a working technical + reference for reading the API contract and submitting runs. + +The remaining product work belongs mostly to the external app: interface +design, auth/session model if needed, job history, multiuser behavior, and +optional file previews/open actions. + +## Files To Give The Other Session + +Give the external frontend session these docs first: + +- [`published-setup-external-ui-contract.md`](published-setup-external-ui-contract.md) + explains the product contract, authoring nodes, app surface, result path, and + source breadcrumb behavior. +- [`external-frontend-job-submitter-brief.md`](external-frontend-job-submitter-brief.md) + is the shorter implementation brief. +- [`published-setups.md`](published-setups.md) documents storage, validation, + routes, request/response shapes, and runner status behavior. +- [`published-setup-full-circle-plan.md`](published-setup-full-circle-plan.md) + records the intended full loop and remaining non-core gaps. +- This file, [`external-frontend-kit.md`](external-frontend-kit.md), is the + packaging checklist. + +Give these code files as reference implementation, not as production UI: + +- [`../../web/setup_runner_simulator.html`](../../web/setup_runner_simulator.html) +- [`../../web/setup_runner_simulator.js`](../../web/setup_runner_simulator.js) +- [`../../tests/sidebar/test_setup_runner_simulator.py`](../../tests/sidebar/test_setup_runner_simulator.py) + +Give these code files if the session needs to understand the server boundary: + +- [`../../koolook_routes.py`](../../koolook_routes.py) +- [`../../koolook_setups.py`](../../koolook_setups.py) +- [`../../koolook_setup_runner.py`](../../koolook_setup_runner.py) +- [`../../web/sidebar/published_setups.js`](../../web/sidebar/published_setups.js) + +For the best start, also provide one real published setup detail JSON captured +from your live ComfyUI session after publishing from the sidebar: + +```text +GET /koolook/api/setups/{id} +``` + +That real payload should include `apiPrompt`, `setupSurface.app`, and +`source.inventoryPath`. It is the most useful fixture for a separate frontend +AI session because it proves the exact setup shape coming out of your current +workflow. + +## Runtime Dependencies + +The external frontend needs: + +- A running ComfyUI instance with the Koolook extension installed. +- The custom nodes required by the published setup installed in that same + ComfyUI instance. +- Access to the Koolook API base URL, normally the local ComfyUI server such + as `http://127.0.0.1:8188`. +- Published setup records in the Koolook registry, usually created by + right-clicking a saved sidebar workflow and choosing **Publish setup...**. +- A browser/runtime strategy for same-origin requests, CORS, or a small proxy + if the production frontend is served from a different origin. + +The external frontend does not need to construct ComfyUI API prompts. The +Koolook server owns prompt cloning, value injection, custom-node execution, +branch pruning, and history/queue polling. + +## API Surface + +List setups: + +```text +GET /koolook/api/setups +``` + +Load one setup: + +```text +GET /koolook/api/setups/{id} +``` + +Run one setup: + +```text +POST /koolook/api/setups/{id}/run +Content-Type: application/json + +{ "inputs": { ...declared fields only... } } +``` + +Poll a run: + +```text +GET /koolook/api/runs/{runId} +``` + +Terminal statuses are: + +```text +succeeded +failed +lost +``` + +Non-terminal statuses are: + +```text +queued +running +``` + +Treat `lost` as terminal. It means the prompt is missing from both ComfyUI +history and queue, so the frontend should stop polling and show an actionable +failure state. + +## What The Frontend Should Render + +Use only these setup detail fields to build the user-facing form: + +- `metadata` +- `validation` +- `source.inventoryPath` +- `source.name` +- `setupSurface.app.switch` +- `setupSurface.app.inputs` +- `setupSurface.app.outputs` +- `setupSurface.app.results` + +Do not render from arbitrary ComfyUI node classes. Do not render from +`apiPrompt`. Do not parse `visualGraph` to decide what the external user should +see. + +The usual first-version app form is: + +```text +Setup selector/catalog grouped by source.inventoryPath + +Input type: EXR / QT / Img +Source path field: + EXR -> sequence_folder + QT -> qt_file + Img -> single_file + +Output folder +Output name +Version + +Run + +Status and resolved result path +``` + +Path convention: + +- Keys ending in `_folder` are directory paths. +- Keys ending in `_file` are full file paths, including directory and filename. + +Preserve switch option values exactly. For example, if `QT` has `"value": 1`, +submit `1`; do not renumber visible options. + +## Minimal Adapter The Frontend Should Build + +The production frontend can be organized around a small adapter layer: + +```text +loadSetups() -> setup summaries +loadSetup(id) -> setup detail +buildCatalogTree(setups, source.inventoryPath) -> grouped setup list +buildForm(setup.setupSurface.app) -> fields, defaults, visibility +buildRunInputs(formState, setup.setupSurface.app) -> declared inputs only +runSetup(id, inputs) -> { runId, promptId, status } +pollRun(runId) -> run status and outputs +extractResultPath(run.outputs) -> string or null +``` + +The simulator already contains working examples of most of this logic in +`web/setup_runner_simulator.js`. + +## Example Run Payload + +For a QuickTime input mode, the frontend normally submits: + +```json +{ + "inputs": { + "switch": 1, + "qt_file": "/shots/shot010/source/source.mov", + "folder": "/shots/shot010/output", + "name": "shot010_rmgb", + "version": "1" + } +} +``` + +Only submit fields declared by the setup. If the frontend submits undeclared +keys, the API returns `400` with an `errors[]` payload. + +## Result Handling + +The first production behavior can be simple: + +1. Show queued/running/succeeded/failed/lost status. +2. On terminal success, scan `run.outputs` for result items. +3. Display the returned `value` string as the resolved result path. + +When present, `Koolook_PublishResult` is the preferred result source because +it represents the graph-selected resolved path. If it is not present, the +runner may still return selected writer/history output items for the selected +branch. + +Previewing images, movies, or folders can be added later. The first frontend +does not need publish-time file-kind metadata as long as it displays the path +string returned by the run. + +## Fixture Checklist + +Before handing work to another AI session, capture these fixtures from the +current live server when possible: + +- `GET /koolook/api/setups` response. +- `GET /koolook/api/setups/{id}` for one real published setup. +- One successful `POST /koolook/api/setups/{id}/run` response. +- One terminal `GET /koolook/api/runs/{runId}` success response that includes + the resolved output path. +- One error response, such as an undeclared input key or missing setup id. + +These fixtures let the external session build and test against the real +Koolook contract without needing to guess from prose. + +## Gaps To Confirm Before Production + +The core API contract is ready to build against, but a production app still +needs decisions that live outside this repository: + +- How the app is served relative to ComfyUI: same origin, CORS, or proxy. +- Whether users browse a flat setup list or a tree grouped by + `source.inventoryPath`. +- Whether the app is single-user/local only or needs auth and multiuser job + ownership. +- Where run history lives if users need previous jobs after a page reload. +- Whether result paths should gain preview/open actions beyond showing the + returned string. diff --git a/docs/maintainers/published-setup-external-ui-contract.md b/docs/maintainers/published-setup-external-ui-contract.md index d996be4..208c57c 100644 --- a/docs/maintainers/published-setup-external-ui-contract.md +++ b/docs/maintainers/published-setup-external-ui-contract.md @@ -65,7 +65,7 @@ Published setups carry two related but separate artifacts: - Preserves custom nodes, subgraphs, and widget serialization exactly as ComfyUI expects. -The current PR already allows publish requests to supply `apiPrompt`; when it +The implementation now allows publish requests to supply `apiPrompt`; when it is supplied, the registry stores it instead of re-converting the visual graph. The fallback converter only exists for older/simple flows and should not be the main product path. @@ -300,33 +300,36 @@ Koolook_PublishRouter outputN -> selected writer branch optional calculated result path -> Koolook_PublishResult(result) ``` -## Current Implementation In This PR +## Current Implementation Implemented: - Catalog and detail routes for published setup records. - Run routes that clone the stored `apiPrompt`, inject declared inputs, queue ComfyUI `/prompt`, and poll history/queue status. +- Sidebar publish captures ComfyUI's API-format prompt automatically for normal + publish-node setup publishing. - External runner simulator as a separate app under `web/`; it renders the `setupSurface.app` mode switch, active source path field, output controls, generated JSON payload, run status, ComfyUI prompt id, and returned result path. -- Optional `apiPrompt` in publish requests; supplied API prompts are preserved - as the execution source of truth. +- The simulator can also load local published setup records for contract + review; running still uses the live Koolook registry API. +- `apiPrompt` in publish requests is preserved as the execution source of + truth. - Fallback visual-to-API conversion repairs some legacy/editor artifacts such as labels, reroutes, and subgraph wrapper ids. - `setupSurface.app` support in the server schema and runner summaries. +- Switch-selected writer execution maps let the runner prune unselected writer + branches and report the selected result/writer output. - Publish contract nodes: - `Koolook_PublishInput` - `Koolook_PublishOutput` - `Koolook_PublishRouter` - `Koolook_PublishResult` -Known gaps: +Remaining gaps: -- The sidebar publish action does not yet automatically capture ComfyUI's API - export prompt and pass it as `apiPrompt`. The server/client boundary can - accept it, but the right-click publish UI integration is not complete. - The publish modal still exposes advanced contract JSON. For publish-node setups, this should become secondary because the graph nodes should define the app surface. diff --git a/docs/maintainers/published-setup-full-circle-plan.md b/docs/maintainers/published-setup-full-circle-plan.md index db9f6b2..3ade32b 100644 --- a/docs/maintainers/published-setup-full-circle-plan.md +++ b/docs/maintainers/published-setup-full-circle-plan.md @@ -8,6 +8,29 @@ Related design docs: - [`published-setup-external-ui-contract.md`](published-setup-external-ui-contract.md) - [`published-setups.md`](published-setups.md) - [`external-frontend-job-submitter-brief.md`](external-frontend-job-submitter-brief.md) +- [`external-frontend-kit.md`](external-frontend-kit.md) + +## Current Status After PRs 231, 233, And 235 + +The core loop is now implemented far enough for an external frontend session to +build against the published setup API: + +- The sidebar publish action captures ComfyUI's API-format prompt and stores it + as `apiPrompt`. +- Published setup records include the visual graph, executable `apiPrompt`, + inferred `setupSurface.app`, validation state, and sidebar source metadata. +- Published setup source metadata includes `source.inventoryPath` and + `source.name` so external frontends can recreate the sidebar hierarchy. +- The simulator can load setup records from the live API and can render local + setup files for contract review. +- The runner clones the stored API prompt, injects declared inputs, prunes + switch-selected writer branches when an execution map is present, and returns + run status plus selected output/result summaries. +- Missing prompts are reported as terminal `lost` state instead of polling + forever. + +The main remaining work is the production external frontend itself, plus any +polish around the ComfyUI publish modal preview experience. ## Target Loop @@ -196,42 +219,47 @@ normalize to this same `setupSurface.app` shape. The external frontend should not need to know whether the surface came from publish nodes, a config UI script, or another authoring helper. -## Completed In PR 231 +## Implemented In PRs 231, 233, And 235 - Published setup catalog and detail routes. - Published setup run and run-status routes. -- Optional `apiPrompt` storage on publish requests. +- Required sidebar API prompt capture for normal publish-node setup publishing. +- `apiPrompt` storage on publish requests. - Server-side validation for declared app inputs, app output controls, and app switch values. - `Koolook_PublishInput`, `Koolook_PublishOutput`, and `Koolook_PublishResult` contract nodes. +- `Koolook_PublishRouter` execution maps for switch-selected writer branches. - `Koolook_PublishResult` emits UI text history so the runner can return a selected result path when a setup uses an explicit reporting node. - External runner simulator under `web/setup_runner_simulator.html`. - Simulator form rendering from `setupSurface.app`. +- Simulator local setup-file loading for contract review. +- Run response validation in the simulator/client harness. +- Terminal `lost` status when a run is missing from both ComfyUI history and + queue. +- Selected writer/result output summaries for switch-routed setups. -## Gaps To Close +## Remaining Gaps To Close -### 1. Automatic API Prompt Capture +### 1. Publish Modal Simplification -The sidebar **Publish setup...** action must automatically capture the ComfyUI -API prompt and pass it as `apiPrompt`. +The publish modal can still expose advanced contract JSON. For publish-node +setups, that should become secondary. Acceptance criteria: -- The author does not manually export or paste API JSON for normal setup - publishing. -- The stored `apiPrompt` is the exact executable prompt shape ComfyUI would - submit/export for the graph. -- Custom nodes, subgraphs, and ComfyUI widget serialization survive unchanged. -- If API prompt capture fails, publish fails with a clear diagnostic instead - of silently falling back to a broken visual conversion for custom-node - workflows. +- Normal publish-node setups require metadata and confirmation, not hand-written + input/output contract JSON. +- Advanced JSON remains available for diagnostics or unusual workflows. +- The modal previews the inferred app surface so authors can verify what the + external frontend will show. -### 2. Exported File Shape +### 2. Exported/Fixture File Shape -The exporter must produce a published setup record or registry wrapper that -contains both `apiPrompt` and `setupSurface.app`. +When a setup is handed to a separate frontend session as a file fixture, it +must use the same published setup record shape returned by the setup detail +API. A direct exported-file workflow should continue to use this same shape. Acceptance criteria: @@ -244,45 +272,7 @@ Acceptance criteria: - The file includes validation diagnostics when a setup is draft/invalid. - Exported setup records keep stable ids and metadata. -### 3. Publish Modal Simplification - -The publish modal still exposes advanced contract JSON as primary UI. For -publish-node setups, this should become secondary. - -Acceptance criteria: - -- Normal publish-node setups require metadata and confirmation, not hand-written - input/output contract JSON. -- Advanced JSON remains available for diagnostics or unusual workflows. -- The modal previews the inferred app surface so authors can verify what the - external frontend will show. - -### 4. Simulator File Loading - -The simulator currently loads setups through the Koolook API and demo data. If -the product requires direct exported-file review, add a file loader that accepts -the canonical published setup shape. - -Acceptance criteria: - -- The simulator can load one setup object or `{ "setups": [...] }`. -- The same renderer is used for API-loaded and file-loaded setups. -- The raw JSON payload remains secondary/debug-only. - -### 5. Run Result Robustness - -The first result contract is a selected writer/history item, with an optional -path string from `Koolook_PublishResult` when authors need custom reporting. - -Acceptance criteria: - -- Successful runs return `outputs[]` with a `result` summary and at least one - item whose `value` is the resolved path. -- The result path reflects the graph-selected branch and submitted output - controls. -- Failed ComfyUI runs return actionable error payloads. - -### 6. External Production Frontend +### 3. External Production Frontend The simulator is a maintainer harness. A production external app still needs its own design, auth/session model, job list, run history, and multiuser @@ -301,8 +291,10 @@ Acceptance criteria: ## Suggested Implementation Order -1. Implement and test automatic API prompt capture in sidebar publish. -2. Make published/exported setup files use the canonical setup record shape. -3. Simplify publish modal UX around inferred publish-node surfaces. -4. Add optional direct-file loading to the simulator if needed. -5. Hand the external frontend brief to the separate app session. +1. Capture one or more real published setup detail payloads from the live API. +2. Hand [`external-frontend-kit.md`](external-frontend-kit.md) and the captured + fixtures to the separate frontend session. +3. Build the production frontend against the API/detail/run contract. +4. Simplify the ComfyUI publish modal UX around inferred publish-node surfaces. +5. Add direct exported-file polish only if the production workflow needs more + than the current API/detail fixture path.