nodes/malkuthro-ComfyUI-Koolook
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

docs(setups): add external frontend kit (#236)

Browse Source 
External frontend kit handoff for published-setup consumers; refreshes full-circle/status docs and neutralizes stale current-PR apiPrompt wording per review #4485848881.
This commit is contained in:
malkuthro
2026-06-12 16:43:51 +02:00
committed by GitHub
parent c6a155a033
commit 91cee1daae
5 changed files with 337 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/maintainers/README.md
+1
View File
@@ -18,6 +18,7 @@ and should be kept in sync with these prose docs.
| [`published-setup-external-ui-contract.md`](published-setup-external-ui-contract.md) | Product/design contract for how ComfyUI publish nodes expose controls and result fields to an external frontend. |
| [`published-setup-full-circle-plan.md`](published-setup-full-circle-plan.md) | Remaining implementation plan for the full authoring → publish → exported setup/API → external runner loop. |
| [`external-frontend-job-submitter-brief.md`](external-frontend-job-submitter-brief.md) | Handoff brief for an AI or developer session building the real external frontend job submitter. |
| [`external-frontend-kit.md`](external-frontend-kit.md) | Complete file/dependency/fixture checklist for another AI session building the production external frontend. |
| [`node-versioning.md`](node-versioning.md) | Hard-won rules for changing `INPUT_TYPES` / `RETURN_TYPES` / class names without breaking saved user workflows. The five rules + the suffix-version pattern (`MyNode` → `MyNodeV2`) + the alias-then-deprecate migration path. |
| [`curated-sidebar.md`](curated-sidebar.md) | The Kforge Labs sidebar starter-preset workflow. How to customize the bundled `web/starter_preset.json` shipped to fresh users (Snapshot → Load → starter). Storage keys, library directory resolution, reset procedure. |
| [`workflows-sidebar.md`](workflows-sidebar.md) | The "Workflows" section of the same sidebar. Save canvas / save selection / directories / archive on collision / load with named tab. Storage shape, action semantics, reset procedure. |
docs/maintainers/external-frontend-job-submitter-brief.md
+5
View File
@@ -8,6 +8,11 @@ Primary reference:
- [`published-setup-external-ui-contract.md`](published-setup-external-ui-contract.md)
- [`published-setup-full-circle-plan.md`](published-setup-full-circle-plan.md)
- [`published-setups.md`](published-setups.md)
- [`external-frontend-kit.md`](external-frontend-kit.md)
For a complete package of files, dependencies, fixtures, and implementation
handoff notes to give another AI session, start with
[`external-frontend-kit.md`](external-frontend-kit.md).
## Product Goal
docs/maintainers/external-frontend-kit.md
+267
View File
@@ -0,0 +1,267 @@
# External Frontend Kit For Published Setups
This kit is the handoff package for an AI or developer session building a
production external frontend that runs Koolook-published ComfyUI setups.
It assumes the frontend is not embedded in ComfyUI. The frontend talks to the
running Koolook/ComfyUI server, renders controls from the published setup
contract, submits declared values, polls run status, and shows the resolved
result path.
## Current State
The current merged implementation has the core full-circle pieces needed by an
external frontend:
- Sidebar publishing captures ComfyUI's API-format prompt and stores it as
`apiPrompt`.
- Published setup records preserve the visual graph, executable `apiPrompt`,
inferred `setupSurface.app`, metadata, validation state, and sidebar source
breadcrumbs.
- `source.inventoryPath` records the sidebar folder hierarchy so external apps
can group setups the same way the ComfyUI sidebar was organized.
- The runner clones the stored `apiPrompt`, injects only declared fields,
prunes switch-selected writer branches when an execution map is available,
submits to the same local ComfyUI server, and reports queued/running/
succeeded/failed/lost states.
- The simulator under `web/setup_runner_simulator.html` is a working technical
reference for reading the API contract and submitting runs.
The remaining product work belongs mostly to the external app: interface
design, auth/session model if needed, job history, multiuser behavior, and
optional file previews/open actions.
## Files To Give The Other Session
Give the external frontend session these docs first:
- [`published-setup-external-ui-contract.md`](published-setup-external-ui-contract.md)
explains the product contract, authoring nodes, app surface, result path, and
source breadcrumb behavior.
- [`external-frontend-job-submitter-brief.md`](external-frontend-job-submitter-brief.md)
is the shorter implementation brief.
- [`published-setups.md`](published-setups.md) documents storage, validation,
routes, request/response shapes, and runner status behavior.
- [`published-setup-full-circle-plan.md`](published-setup-full-circle-plan.md)
records the intended full loop and remaining non-core gaps.
- This file, [`external-frontend-kit.md`](external-frontend-kit.md), is the
packaging checklist.
Give these code files as reference implementation, not as production UI:
- [`../../web/setup_runner_simulator.html`](../../web/setup_runner_simulator.html)
- [`../../web/setup_runner_simulator.js`](../../web/setup_runner_simulator.js)
- [`../../tests/sidebar/test_setup_runner_simulator.py`](../../tests/sidebar/test_setup_runner_simulator.py)
Give these code files if the session needs to understand the server boundary:
- [`../../koolook_routes.py`](../../koolook_routes.py)
- [`../../koolook_setups.py`](../../koolook_setups.py)
- [`../../koolook_setup_runner.py`](../../koolook_setup_runner.py)
- [`../../web/sidebar/published_setups.js`](../../web/sidebar/published_setups.js)
For the best start, also provide one real published setup detail JSON captured
from your live ComfyUI session after publishing from the sidebar:
```text
GET /koolook/api/setups/{id}
```
That real payload should include `apiPrompt`, `setupSurface.app`, and
`source.inventoryPath`. It is the most useful fixture for a separate frontend
AI session because it proves the exact setup shape coming out of your current
workflow.
## Runtime Dependencies
The external frontend needs:
- A running ComfyUI instance with the Koolook extension installed.
- The custom nodes required by the published setup installed in that same
ComfyUI instance.
- Access to the Koolook API base URL, normally the local ComfyUI server such
as `http://127.0.0.1:8188`.
- Published setup records in the Koolook registry, usually created by
right-clicking a saved sidebar workflow and choosing **Publish setup...**.
- A browser/runtime strategy for same-origin requests, CORS, or a small proxy
if the production frontend is served from a different origin.
The external frontend does not need to construct ComfyUI API prompts. The
Koolook server owns prompt cloning, value injection, custom-node execution,
branch pruning, and history/queue polling.
## API Surface
List setups:
```text
GET /koolook/api/setups
```
Load one setup:
```text
GET /koolook/api/setups/{id}
```
Run one setup:
```text
POST /koolook/api/setups/{id}/run
Content-Type: application/json
{ "inputs": { ...declared fields only... } }
```
Poll a run:
```text
GET /koolook/api/runs/{runId}
```
Terminal statuses are:
```text
succeeded
failed
lost
```
Non-terminal statuses are:
```text
queued
running
```
Treat `lost` as terminal. It means the prompt is missing from both ComfyUI
history and queue, so the frontend should stop polling and show an actionable
failure state.
## What The Frontend Should Render
Use only these setup detail fields to build the user-facing form:
- `metadata`
- `validation`
- `source.inventoryPath`
- `source.name`
- `setupSurface.app.switch`
- `setupSurface.app.inputs`
- `setupSurface.app.outputs`
- `setupSurface.app.results`
Do not render from arbitrary ComfyUI node classes. Do not render from
`apiPrompt`. Do not parse `visualGraph` to decide what the external user should
see.
The usual first-version app form is:
```text
Setup selector/catalog grouped by source.inventoryPath
Input type: EXR / QT / Img
Source path field:
EXR -> sequence_folder
QT -> qt_file
Img -> single_file
Output folder
Output name
Version
Run
Status and resolved result path
```
Path convention:
- Keys ending in `_folder` are directory paths.
- Keys ending in `_file` are full file paths, including directory and filename.
Preserve switch option values exactly. For example, if `QT` has `"value": 1`,
submit `1`; do not renumber visible options.
## Minimal Adapter The Frontend Should Build
The production frontend can be organized around a small adapter layer:
```text
loadSetups() -> setup summaries
loadSetup(id) -> setup detail
buildCatalogTree(setups, source.inventoryPath) -> grouped setup list
buildForm(setup.setupSurface.app) -> fields, defaults, visibility
buildRunInputs(formState, setup.setupSurface.app) -> declared inputs only
runSetup(id, inputs) -> { runId, promptId, status }
pollRun(runId) -> run status and outputs
extractResultPath(run.outputs) -> string or null
```
The simulator already contains working examples of most of this logic in
`web/setup_runner_simulator.js`.
## Example Run Payload
For a QuickTime input mode, the frontend normally submits:
```json
{
"inputs": {
"switch": 1,
"qt_file": "/shots/shot010/source/source.mov",
"folder": "/shots/shot010/output",
"name": "shot010_rmgb",
"version": "1"
}
}
```
Only submit fields declared by the setup. If the frontend submits undeclared
keys, the API returns `400` with an `errors[]` payload.
## Result Handling
The first production behavior can be simple:
1. Show queued/running/succeeded/failed/lost status.
2. On terminal success, scan `run.outputs` for result items.
3. Display the returned `value` string as the resolved result path.
When present, `Koolook_PublishResult` is the preferred result source because
it represents the graph-selected resolved path. If it is not present, the
runner may still return selected writer/history output items for the selected
branch.
Previewing images, movies, or folders can be added later. The first frontend
does not need publish-time file-kind metadata as long as it displays the path
string returned by the run.
## Fixture Checklist
Before handing work to another AI session, capture these fixtures from the
current live server when possible:
- `GET /koolook/api/setups` response.
- `GET /koolook/api/setups/{id}` for one real published setup.
- One successful `POST /koolook/api/setups/{id}/run` response.
- One terminal `GET /koolook/api/runs/{runId}` success response that includes
the resolved output path.
- One error response, such as an undeclared input key or missing setup id.
These fixtures let the external session build and test against the real
Koolook contract without needing to guess from prose.
## Gaps To Confirm Before Production
The core API contract is ready to build against, but a production app still
needs decisions that live outside this repository:
- How the app is served relative to ComfyUI: same origin, CORS, or proxy.
- Whether users browse a flat setup list or a tree grouped by
`source.inventoryPath`.
- Whether the app is single-user/local only or needs auth and multiuser job
ownership.
- Where run history lives if users need previous jobs after a page reload.
- Whether result paths should gain preview/open actions beyond showing the
returned string.
docs/maintainers/published-setup-external-ui-contract.md
+11 -8
View File
@@ -65,7 +65,7 @@ Published setups carry two related but separate artifacts:
- Preserves custom nodes, subgraphs, and widget serialization exactly as
ComfyUI expects.
The current PR already allows publish requests to supply `apiPrompt`; when it
The implementation now allows publish requests to supply `apiPrompt`; when it
is supplied, the registry stores it instead of re-converting the visual graph.
The fallback converter only exists for older/simple flows and should not be the
main product path.
@@ -300,33 +300,36 @@ Koolook_PublishRouter outputN -> selected writer branch
optional calculated result path -> Koolook_PublishResult(result)
```
## Current Implementation In This PR
## Current Implementation
Implemented:
- Catalog and detail routes for published setup records.
- Run routes that clone the stored `apiPrompt`, inject declared inputs, queue
ComfyUI `/prompt`, and poll history/queue status.
- Sidebar publish captures ComfyUI's API-format prompt automatically for normal
publish-node setup publishing.
- External runner simulator as a separate app under `web/`; it renders the
`setupSurface.app` mode switch, active source path field, output controls,
generated JSON payload, run status, ComfyUI prompt id, and returned result
path.
- Optional `apiPrompt` in publish requests; supplied API prompts are preserved
as the execution source of truth.
- The simulator can also load local published setup records for contract
review; running still uses the live Koolook registry API.
- `apiPrompt` in publish requests is preserved as the execution source of
truth.
- Fallback visual-to-API conversion repairs some legacy/editor artifacts such
as labels, reroutes, and subgraph wrapper ids.
- `setupSurface.app` support in the server schema and runner summaries.
- Switch-selected writer execution maps let the runner prune unselected writer
branches and report the selected result/writer output.
- Publish contract nodes:
- `Koolook_PublishInput`
- `Koolook_PublishOutput`
- `Koolook_PublishRouter`
- `Koolook_PublishResult`
Known gaps:
Remaining gaps:
- The sidebar publish action does not yet automatically capture ComfyUI's API
export prompt and pass it as `apiPrompt`. The server/client boundary can
accept it, but the right-click publish UI integration is not complete.
- The publish modal still exposes advanced contract JSON. For publish-node
setups, this should become secondary because the graph nodes should define
the app surface.
docs/maintainers/published-setup-full-circle-plan.md
+53 -61
View File
@@ -8,6 +8,29 @@ Related design docs:
- [`published-setup-external-ui-contract.md`](published-setup-external-ui-contract.md)
- [`published-setups.md`](published-setups.md)
- [`external-frontend-job-submitter-brief.md`](external-frontend-job-submitter-brief.md)
- [`external-frontend-kit.md`](external-frontend-kit.md)
## Current Status After PRs 231, 233, And 235
The core loop is now implemented far enough for an external frontend session to
build against the published setup API:
- The sidebar publish action captures ComfyUI's API-format prompt and stores it
as `apiPrompt`.
- Published setup records include the visual graph, executable `apiPrompt`,
inferred `setupSurface.app`, validation state, and sidebar source metadata.
- Published setup source metadata includes `source.inventoryPath` and
`source.name` so external frontends can recreate the sidebar hierarchy.
- The simulator can load setup records from the live API and can render local
setup files for contract review.
- The runner clones the stored API prompt, injects declared inputs, prunes
switch-selected writer branches when an execution map is present, and returns
run status plus selected output/result summaries.
- Missing prompts are reported as terminal `lost` state instead of polling
forever.
The main remaining work is the production external frontend itself, plus any
polish around the ComfyUI publish modal preview experience.
## Target Loop
@@ -196,42 +219,47 @@ normalize to this same `setupSurface.app` shape. The external frontend should
not need to know whether the surface came from publish nodes, a config UI
script, or another authoring helper.
## Completed In PR 231
## Implemented In PRs 231, 233, And 235
- Published setup catalog and detail routes.
- Published setup run and run-status routes.
- Optional `apiPrompt` storage on publish requests.
- Required sidebar API prompt capture for normal publish-node setup publishing.
- `apiPrompt` storage on publish requests.
- Server-side validation for declared app inputs, app output controls, and app
switch values.
- `Koolook_PublishInput`, `Koolook_PublishOutput`, and
`Koolook_PublishResult` contract nodes.
- `Koolook_PublishRouter` execution maps for switch-selected writer branches.
- `Koolook_PublishResult` emits UI text history so the runner can return a
selected result path when a setup uses an explicit reporting node.
- External runner simulator under `web/setup_runner_simulator.html`.
- Simulator form rendering from `setupSurface.app`.
- Simulator local setup-file loading for contract review.
- Run response validation in the simulator/client harness.
- Terminal `lost` status when a run is missing from both ComfyUI history and
queue.
- Selected writer/result output summaries for switch-routed setups.
## Gaps To Close
## Remaining Gaps To Close
### 1. Automatic API Prompt Capture
### 1. Publish Modal Simplification
The sidebar **Publish setup...** action must automatically capture the ComfyUI
API prompt and pass it as `apiPrompt`.
The publish modal can still expose advanced contract JSON. For publish-node
setups, that should become secondary.
Acceptance criteria:
- The author does not manually export or paste API JSON for normal setup
publishing.
- The stored `apiPrompt` is the exact executable prompt shape ComfyUI would
submit/export for the graph.
- Custom nodes, subgraphs, and ComfyUI widget serialization survive unchanged.
- If API prompt capture fails, publish fails with a clear diagnostic instead
of silently falling back to a broken visual conversion for custom-node
workflows.
- Normal publish-node setups require metadata and confirmation, not hand-written
input/output contract JSON.
- Advanced JSON remains available for diagnostics or unusual workflows.
- The modal previews the inferred app surface so authors can verify what the
external frontend will show.
### 2. Exported File Shape
### 2. Exported/Fixture File Shape
The exporter must produce a published setup record or registry wrapper that
contains both `apiPrompt` and `setupSurface.app`.
When a setup is handed to a separate frontend session as a file fixture, it
must use the same published setup record shape returned by the setup detail
API. A direct exported-file workflow should continue to use this same shape.
Acceptance criteria:
@@ -244,45 +272,7 @@ Acceptance criteria:
- The file includes validation diagnostics when a setup is draft/invalid.
- Exported setup records keep stable ids and metadata.
### 3. Publish Modal Simplification
The publish modal still exposes advanced contract JSON as primary UI. For
publish-node setups, this should become secondary.
Acceptance criteria:
- Normal publish-node setups require metadata and confirmation, not hand-written
input/output contract JSON.
- Advanced JSON remains available for diagnostics or unusual workflows.
- The modal previews the inferred app surface so authors can verify what the
external frontend will show.
### 4. Simulator File Loading
The simulator currently loads setups through the Koolook API and demo data. If
the product requires direct exported-file review, add a file loader that accepts
the canonical published setup shape.
Acceptance criteria:
- The simulator can load one setup object or `{ "setups": [...] }`.
- The same renderer is used for API-loaded and file-loaded setups.
- The raw JSON payload remains secondary/debug-only.
### 5. Run Result Robustness
The first result contract is a selected writer/history item, with an optional
path string from `Koolook_PublishResult` when authors need custom reporting.
Acceptance criteria:
- Successful runs return `outputs[]` with a `result` summary and at least one
item whose `value` is the resolved path.
- The result path reflects the graph-selected branch and submitted output
controls.
- Failed ComfyUI runs return actionable error payloads.
### 6. External Production Frontend
### 3. External Production Frontend
The simulator is a maintainer harness. A production external app still needs
its own design, auth/session model, job list, run history, and multiuser
@@ -301,8 +291,10 @@ Acceptance criteria:
## Suggested Implementation Order
1. Implement and test automatic API prompt capture in sidebar publish.
2. Make published/exported setup files use the canonical setup record shape.
3. Simplify publish modal UX around inferred publish-node surfaces.
4. Add optional direct-file loading to the simulator if needed.
5. Hand the external frontend brief to the separate app session.
1. Capture one or more real published setup detail payloads from the live API.
2. Hand [`external-frontend-kit.md`](external-frontend-kit.md) and the captured
fixtures to the separate frontend session.
3. Build the production frontend against the API/detail/run contract.
4. Simplify the ComfyUI publish modal UX around inferred publish-node surfaces.
5. Add direct exported-file polish only if the production workflow needs more
than the current API/detail fixture path.