diff --git a/README.md b/README.md index c4dcd118..bd0c74d5 100644 --- a/README.md +++ b/README.md @@ -53,6 +53,8 @@ FunPack uses your existing ComfyUI/PyTorch install. Start with [How to start generating with FunPack](docs/how-to-start.md). The app also has a guided tour: Help -> Start tour. +Writing a module or a UI feature? Start with [Developing FunPack](docs/README.md). + Version history is available in [CHANGELOG.md](CHANGELOG.md). ## Classic FunPack (4.x) diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 00000000..101b6ebb --- /dev/null +++ b/docs/README.md @@ -0,0 +1,15 @@ +# FunPack documentation + +For people who use FunPack, start with [How to start generating with FunPack](how-to-start.md). + +For people who write code for it, read in this order: + +1. [How FunPack is built](architecture.md): the idea, the layers, the failure model. Read this first. +2. [Writing a module](writing-a-module.md): a Python module, step by step, with a tested example. +3. [Module reference](module-reference.md): constants, setting keys, capabilities and helpers, in tables. +4. [Writing a UI feature](writing-a-ui-feature.md): a screen feature in the app, with a tested example. +5. [UI design guidelines](ui-design-guidelines.md): how the app looks and behaves, and the words it uses. +6. [UI kit reference](ui-kit-reference.md): every kit element, and how to add one. +7. [Testing and verification](testing.md): running the tests and what counts as proof. + +If a page and the code disagree, the code is right. Please fix the page. diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 00000000..4789b89c --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,124 @@ +# How FunPack is built + +Read this first. The other guides assume it. + +## The idea + +FunPack is a browser app (`/funpack/`) on top of ComfyUI. Everything it does is a **module that announces itself**. Core keeps no list of features, so adding a feature is adding a folder, and removing one is deleting it. + +The same idea runs on both sides: + +| Side | Things that announce themselves | Where they announce | +|---|---|---| +| Backend (Python) | modules | constants in `modules///__init__.py` | +| App (JavaScript) | features | `export default { id, mount, setup }` listed in `app/ui/modules.js` | +| App kit | UI elements | `define(group, variant, factory)` in `app/ui/composer/elements/` | + +Anything that is announced can be absent. A module that fails to import, fails validation, or breaks during a run is left out, the Log says why, and everything else carries on. The UI follows the same rule: a feature that cannot load is not drawn at all (no grey button, no warning chip). + +## The layers + +``` +browser app/ui/ shell + kit + features (plain ES modules, no build step) + | +aiohttp core/routes.py routes mounted on ComfyUI's own server under /funpack/ + | +core/ Python finds modules, validates them, builds the ComfyUI prompt + | +modules/ Python features, grouped by domain + | +ComfyUI nodes + sampler executes the prompt +``` + +- **core/** knows how to serve the app, load modules and build a ComfyUI request. It never names a feature. +- **modules/** is every feature: loaders, sampling techniques, model support, output, learning from ratings. +- **app/ui/shell/** is the app's own plumbing: the page frame, project and pipeline state, the feature loader, the API client. +- **app/ui/composer/** is the UI kit (called the Composer): tokens, internals, elements. It is the only code that styles anything. +- **app/ui/features/** is every screen feature: timeline, media bin, preview, settings sections, Generate. + +## How a setting travels + +1. A module declares `SETTINGS` (see [Writing a module](writing-a-module.md)). +2. `GET /funpack/api/modules` sends the browser the validated declarations, ordered by stage and relations, with the model's traits applied. +3. **Settings > Engine** (and the Sampler window) draw one row per setting. Nothing is hand-built per module. +4. An edit is saved with the project. The values travel as one JSON object, `{module_id: {key: value}}`, in the `settings` input of the node a module named as its `settings_sink` (today `FunPack Modifier Settings`). +5. At run time `FunPack Modifier Settings` checks every value against its declaration and `FunPack Load Modifiers` calls each enabled module's `install(patcher, values, key)` in order. + +The screen is the truth: Generate reads the live UI state, and a module never reads the UI. + +## The pipeline is data + +The pipeline is a list of slots (`core/graph.py`): each names a node and what feeds it. Nothing is a fixed chain, so a slot can be pointed at another node or removed, and the server refuses what would not build and says why. + +- A slot's `roles` put one of its inputs on the app's surface (the prompt box, width and height, steps, seed). A role's `at` names the place (`generation.prompt`, `project.video`, ...). +- A slot's `group` is how the pipeline reads to a person (the cards in Models & Pipeline). +- Model modules offer ready-made pipelines through `pipeline_presets`. + +## Failure model + +The rules below are enforced by code, not by habit. Each one names where. + +| What goes wrong | What happens | Where | +|---|---|---| +| A module raises on import | Absent. One Warning in the Log. | `core/registry.py` | +| A declaration is invalid | Not loaded. The Log names the setting and the rule. | `core/schema.py` | +| Two modules share an id or a node id | The second is left out and said. | `registry.add`, `core/nodes.py` | +| Saved settings name a module or key that no longer exists | Skipped with one Log line. Old projects keep working. | `schema.check_values`, modifier nodes | +| A setting value is wrong | The run is refused, naming it (unless Full control is on). | `schema.refuse_or_warn` | +| `install` raises | That module is off for the run and quarantined. | `core/control.py` | +| A hook raises on step 3 of 30 | The hook is dropped for the rest of the run, the run continues. | `core/patching.py` | +| A module was quarantined | It stays off until its code changes (and ComfyUI restarts) or the person turns it on in Settings > Modules. | `core/control.py` | +| A relation cannot be satisfied (cycle) | The modules on the cycle are left out, and said. | `core/relations.py` | +| A model lacks a trait a module requires | The module is not offered for that model. | `core/traits.py` | + +Out-of-memory errors and interrupts are not a module's fault and never quarantine it. + +## Independence + +- A module reaches core only through `modules/_core.py` (`from ..._core import log, registry`). A direct `import core` works in tests and breaks inside ComfyUI, so a test forbids it. +- A module never imports another module when the file loads. It asks by capability: `registry.current().ask("name", ...)`. If the provider is missing or raises, the answer is `None` and the asking module copes. A test forbids the import. +- A feature may not break the baseline, and the baseline does not depend on a feature on top of it. +- The same holds in the browser: a feature never imports another feature. It talks through `app` (see [Writing a UI feature](writing-a-ui-feature.md)). + +## Who may touch what + +There are three trust levels in the app code. Know which one you are writing. + +| You are writing | May style or build DOM? | Checked by | +|---|---|---| +| A **backend module's settings** (Python only) | No JavaScript at all. The kit draws the rows. | `core/schema.py`, `core/tests/test_shipped_hints.py` | +| An optional **module `ui.js`** (`modules///ui.js`) | No. `setup({ values, on, shell })` and nothing else. | `tools/kit_lint` scans `modules/` | +| A **UI feature** in `app/ui/features/` | It builds with kit elements. A few features touch `document` or add `fp-` classes in `frame.css`; new work should not. | review; the kit rules are enforced on `composer/` and `shell/` | +| The **kit** itself | Yes. It is the only place that styles. | `tools/kit_lint` `KIT_RULES`, tests in `composer/tests/` | + +Be exact about what the lint is. `tools/kit_lint` is a discipline check, not a security boundary: a module `ui.js` runs in the page, so `document` is reachable. First-party modules are reviewed. A third-party module would need real isolation (a worker or another realm) before it could be trusted. + +## Where things live + +| Path | What | +|---|---| +| `__init__.py` | ComfyUI entry point. Mounts the routes and hands ComfyUI the nodes modules announced. | +| `core/` | Registry, schema, traits, relations, chain, control, patching, graph, routes, run, log. | +| `modules/_core.py` | The one place a module reaches core from. | +| `modules///` | One module. `__init__.py` is the announcement. | +| `modules//*.py` | Helpers shared inside a group. Announce nothing. May not import a module. | +| `app/ui/boot.js` | Starts the frame, the core services, then loads features. Names no feature. | +| `app/ui/modules.js` | The one list of UI features. | +| `app/ui/shell/` | Frame, mounts, project and pipeline state, API client, settings window. | +| `app/ui/composer/` | The kit: `tokens/`, `internals/`, `elements/`, `panel.js`, `tests/`. | +| `app/ui/features//` | One feature. | +| `app/ui/catalogue/` | A page that shows every kit element and token, in both themes. | +| `app/assets/fonts/` | Bundled fonts (Inter, Inter Tight, IBM Plex Mono, all OFL). | +| `tools/devserver.py` | Serves the app and the real routes without ComfyUI. | +| `tools/kit_lint/` | The rule scanner. | +| `docs/` | These documents. | + +Python tests sit beside what they test (`core/tests/`, `modules///tests/`). JavaScript tests sit beside the code (`*.test.js`) or in `composer/tests/` and `shell/tests/`. + +## Running it locally + +- **With ComfyUI**: clone into `ComfyUI/custom_nodes`, `pip install -r requirements.txt`, start ComfyUI, open `/funpack/`. +- **Without ComfyUI**: `python tools/devserver.py 8199` serves the app on the real routes. There is no queue behind it, so Generate says so. Set `COMFYUI_ROOT` to a ComfyUI checkout, or most modules will be absent (they import `comfy`). +- **The catalogue**: `/funpack/app/ui/catalogue/index.html` (tokens), `elements.html` (every element, both themes), `hide.html` (what happens when a module is broken), `log.html`. + +Tests are covered in [Testing and verification](testing.md). diff --git a/docs/module-reference.md b/docs/module-reference.md new file mode 100644 index 00000000..a23628bf --- /dev/null +++ b/docs/module-reference.md @@ -0,0 +1,187 @@ +# Module reference + +Facts for module authors, in tables. For the how and why, read [Writing a module](writing-a-module.md). Everything here is read from `core/contract.py`, `core/schema.py`, `core/registry.py` and the shipped modules. If this page and the code ever disagree, the code wins: please fix the page. + +## Announcement constants + +Read by `core/registry.py` from the module's `__init__.py` (the `FIELDS` table). A constant the registry does not know is ignored. + +| Constant | Python type | Default | Rule | +|---|---|---|---| +| `ID` | `str` | none | Required, non-empty. Unique across modules (a second claimant is left out and said). Keep it free of `.`: a fault is traced back to a module from its hook key `funpack.`. | +| `TITLE` | `str` | none | Required, non-empty. | +| `MOUNT` | `str` | `""` | Required and non-empty when the module has `SETTINGS` or a `ui.js`. | +| `SETTINGS` | `dict` | `{}` | `{key: setting}`. Must be a dict. | +| `STAGE` | `str` | `"sampling"` | One of `load`, `conditioning`, `latent`, `guidance`, `sampling`, `post`. | +| `CATEGORY` | `str` | `""` | `""` or one of `continuity`, `guidance`, `conditioning`, `sampling`, `post`, `system`. Empty shows under "Other". | +| `STATUS` | `str` | `"experimental"` | `proven` or `experimental`. | +| `REQUIRES` | `list[str]` | `[]` | Model traits that must all be present. | +| `USES` | `list[str]` | `[]` | Capabilities this module asks other modules for. | +| `USES_WHEN` | `dict` | `{}` | Same shape as `when`: own settings under which it asks. Keys must be its own settings. | +| `ROLES` | `list[str]` | `[]` | Pipeline role `at` names it acts through. Inert in a pipeline where none is filled. | +| `AFTER`, `BEFORE` | `list[str]` | `[]` | Module ids. Order within a stage. | +| `HOOKS` | `list[str]` | `[]` | Hook points a sampler must offer for a `sampler_modifier`. | +| `NODES` | `list[type]` | `[]` | Classes with `define_schema` and `execute` (V3 `io.ComfyNode`). | +| `TRAITS` | callable | none | `traits(model) -> iterable[str]`. | +| `PROVIDES` | `dict[str, callable]` | `{}` | Capability name to function. Names must be non-empty strings. | + +Not read from constants: `ui`. Core sets it to `/funpack/modules///ui.js` when that file exists. + +`CONTRACT_VERSION` in `core/contract.py` is `1`. It is sent to the browser as `contract` in `/funpack/api/modules`. Core does not read a version from modules, so there is nothing for a module to declare. + +## Setting keys + +| Key | Applies to | Notes | +|---|---|---| +| `type` | all | `bool`, `int`, `float`, `enum`, `text`, `multiline`, `path`, `color`. | +| `default` | all | Required. `bool` is a bool; `int` is an int (not a float); `float`/`int` are finite and inside `min`..`max`; `enum` is one of the options; `color` is `#rgb`, `#rrggbb` or `#rrggbbaa`; the rest are strings. | +| `min`, `max` | `int`, `float` | Finite numbers, `min <= max`. A value outside is refused at run time. | +| `step` | `int`, `float` | Positive. | +| `options` | `enum` | Non-empty list of `{"value": scalar, "label": str}`. No repeated values. | +| `label` | all | String. Unique within the module (test). | +| `hint` | all | String. Required by test, one plain sentence. | +| `unit` | numbers | String. Shown after the hint. | +| `placeholder` | text types | String. Used by the panel renderer, not by the Engine page. | +| `ui` | all | A renderer allowed for the type (next table). | +| `when` | all | `{sibling_key: value or [values]}`. Row shown only while every condition holds. Sibling must exist; an enum value must be one of its options; no self-reference. | + +The panel renderer also reads `precision`, `presets` (for `macroSlider`) and an option's `icon` and `hint`. The schema does not check them. + +### Renderers + +Allowed `ui` per type (the first is the default for `composer/panel.js`): + +| Type | Allowed `ui` | +|---|---| +| `bool` | `checkboxRow`, `toggle` | +| `int`, `float` | `number`, `slider`, `stepper`, `macroSlider` | +| `enum` | `select`, `segmented`, `radioGroup`, `filterList`, `wheel` | +| `text` | `input`, `search` | +| `multiline` | `textarea`, `autoTextarea` | +| `path` | `filterList` | +| `color` | `swatch` | + +What **Settings > Engine** and the **Sampler** window draw (`features/settings/field.js`), whatever `ui` says: + +| Type | Drawn as | +|---|---| +| `bool` | toggle with label and hint | +| `int`, `float` | slider if `ui` is absent or `slider` and both `min` and `max` are set, else a number box | +| `enum` | select | +| `color` | swatch | +| `multiline` | text area (3 rows) | +| `text`, `path` | text box (commits on blur or Enter) | + +## Stages and order + +`STAGE` order is the list above. Within a stage modules run in id order unless `AFTER`/`BEFORE` say otherwise. Shipped modules today: `load` (loaders, models, pipeline, taste), `conditioning`, `latent`, `guidance` (taste-steered techniques), `sampling`, `post`. + +## Categories + +Used only to group settings in Settings > Engine and to pick what the Sampler window shows (`sampling`). Order on the page: Continuity, Guidance, Conditioning, Sampling, Post, System, Other. + +## Pipeline roles + +A slot in a pipeline lists `roles`: `{"at": ..., "input": ..., "label": ..., "drives": ...}`. `at` is where the input appears in the app. Names in use: + +| `at` | Shown as | +|---|---| +| `generation.prompt` | the prompt | +| `project.negative` | the project's negative prompt | +| `project.video` | width, height, length, frame rate (`drives`: `frames` or `fps`) | +| `generation.sampling` | steps, sampler, scheduler (Sampler window) | +| `generation.seed` | the seed | +| `generation.model` | the model choice | +| `assets.source_image` | the start picture (what `continuity` needs) | + +A role naming a place the app does not offer is simply not shown. + +## Traits + +Read off any loaded model by core (`traits.universal`): + +| Trait | True when | +|---|---| +| `temporal_latent`, `spatial_latent`, `waveform_latent` | the latent has 3, 2 or 1 axes | +| `temporal_compression` | the latent format compresses time | +| `predict_` | the model type, lowercased (`predict_eps`, `predict_flow`, ...) | + +Announced by model modules: `audio_stream` (H3, LTX), `dit_block_hooks` (H3: honours block replacement and routes attention through the override), `adaln_modalities` (H3 with the AdaLN projection), `ltx_av` (LTX audio+video). Add new ones in the model's own module. Modality is deliberately not a trait. + +## Capabilities + +A capability is a name in `PROVIDES`. Core defines none: the contract is between the provider and the askers. All of these are in use today. `None` as an answer from a model-level provider means "not my model"; a raise means "it was mine and it broke". + +### Plugging into a run + +| Name | Provided by | Signature | Called by | +|---|---|---|---| +| `modifier` | most sampling and guidance modules | `install(patcher, values, key) -> str or None` | `FunPack Load Modifiers` | +| `sampler_modifier` | none yet | `make(values, accepts) -> object with active(ctx), process(ctx, latent)` or `None` | `FunPack Sampler` via `core/chain.py` | +| `settings_sink` | `sampling_modifiers` | `() -> {"node": node id, "input": input name}` | the manifest; the app writes settings there | +| `cache_key` | `phrase_probe` | `() -> value` that changes with its switch | `FunPack Load Modifiers` cache key | +| `guards_off` | `full_control` | `(settings) -> bool` | modifier nodes and the sampler | +| `routes` | several | `routes(table, base, web)` | `core/routes.py`, base `/funpack/api/m/` | + +### Pipelines + +| Name | Provided by | Signature | +|---|---|---| +| `default_pipeline` | `pipeline` | `() -> list of slots` | +| `pipeline_presets` | each model module | `() -> [{"id", "title", "slots"}]` | +| `optional_nodes` | H3, LTX | `() -> {node class: {"pass": {output index: input name}}}` slots that may be skipped when their module is absent | +| `upgrade_slots` | `output_save` | `(slots, outputs) -> slots` migrates old saved pipelines | + +### Knowing a model + +| Name | Signature | Notes | +|---|---|---| +| `detect` | `(keys) -> bool` | keys are tensor names from a safetensors header. Never load to answer. | +| `probe_traits` | `(keys) -> list[str]` | traits knowable from the header alone | +| `empty_latent` | `(model, width, height, length, batch_size=1) -> {"samples": ...}` or `None` | claim by identity | +| `decode` | `(latent, model=None, vae=None, audio_vae=None, tile_size=0) -> result` or `None` | claim by identity | +| `video_stream` | `(x, named) -> (video, rebuild)` or `None` | the picture inside a packed latent; `core/streams.video_of` asks | +| `text_rows` | `(model_kwargs, length) -> bool mask` or `None` | which conditioning rows are the prompt's words | +| `text_tokenizer` | `(clip) -> tokenizer` or `None` | a tokenizer that returns offsets | +| `prompt_rows` | `(meta, cond_len, prompt_tokens) -> (start, end)` or `None` | | +| `video_time_rows` | `(samples, t0, t1) -> (rows, start, end)` or `None` | window in seconds | +| `target_rows` | `(mod_segments, seq_len, device, stream) -> bool mask` or `None` | `stream` is `"video"` or `"audio"` | +| `query_rotation` | `(rope_freqs, seq_len) -> rotate(x, rows, inverse=False)` or `None` | | +| `anchor_pin` | `(kwargs, transform) -> kwargs` or `None` | | +| `rescale_conditioning` | `(conditioning, height, width) -> (conditioning, count)` or `None` | | +| `latent_upscaler` | `(state_dict) -> object with funpack_latent_upscale(x, scale)` or `None` | | + +### Learning + +| Name | Provided by | Signature | +|---|---|---| +| `taste_store` | `taste` | `(patcher) -> handle` with `collect`, `rows`, `capture`, `direction`, `directions`, `describe`; `None` without a key | +| `taste_kind` | `taste` | `(key, kind) -> Kind` for one kind of one key | +| `taste_latest_key` | `taste` | `() -> the key the most recent capture went to, or None` | +| `on_rating` | `taste` calls every module that provides it | `(prompt_id, rating, axis) -> None` | + +## Helpers a module may import from `_core` + +`from ..._core import ...` offers exactly these: `addons`, `chain`, `config`, `contract`, `control`, `dit_hooks`, `input_steer`, `log`, `media`, `patching`, `projects`, `rated_dial`, `registry`, `relations`, `run`, `schema`, `shortcuts`, `streams`, `temp_files`, `traits`, `vae_tiles`. Adding one means editing `modules/_core.py` in both of its branches. + +Most used: + +| Helper | For | +|---|---| +| `log` | `info`, `alert`, `warning`, `critical`, `once`, `broke`, `failed` | +| `registry` | `registry.current().ask(...)`, `.providers(...)`, `.specs` | +| `patching` | `tag`, `strip`, `guard`, `Dropped` | +| `dit_hooks` | `add_block_hook`, `add_attention_override`, `current_step`, `late_half`, `last_step`, `probing`, `weak_branch`, `parse_blocks`, `block_count`, `publish` | +| `streams` | `video_of`, `av_video_stream`, `model_args`, `with_options` | +| `traits` | `has_block`, `traits_of` | +| `control` | `Unavailable` | + +## Served paths + +| Path | What | +|---|---| +| `/funpack/` | the app (`index.html`) | +| `/funpack/api/modules` | the manifest (`?traits=a,b` narrows to a model) | +| `/funpack/api/m//...` | a module's own routes | +| `/funpack/modules///ui.js` | a module's `ui.js`. Only `.js` is served from `modules/`. | +| `/funpack/app/...` | the app. Allowed types: `.js .css .html .woff2 .svg`. | diff --git a/docs/testing.md b/docs/testing.md new file mode 100644 index 00000000..512420ce --- /dev/null +++ b/docs/testing.md @@ -0,0 +1,94 @@ +# Testing and verification + +Every rule in these guides is held by a test or says who checks it. This page shows how to run the tests, how to write a good one, and what counts as proof. + +## Python tests (core and modules) + +``` +pip install -r requirements.txt pytest +COMFYUI_ROOT=/path/to/ComfyUI python -m pytest +``` + +- `pyproject.toml` sets `--import-mode=importlib` and `testpaths = ["core/tests", "modules"]`. The root `__init__.py` is ComfyUI's entry point and cannot be imported on its own, which is why the mode matters. +- Tests sit beside what they test: `core/tests/` and `modules///tests/`. +- `COMFYUI_ROOT` points at a ComfyUI checkout (it must contain `comfy/supported_models.py`). Without it, the default is `~/Documents/ComfyUI`. With none found, tests that need ComfyUI skip, and a module that imports `comfy` at load is simply absent. +- ComfyUI needs `torch`. A CPU build is enough for the tests. If torch complains about CUDA, force the CPU device for the test run rather than editing a test. +- Tests that need `aiohttp` skip themselves with `importorskip` when it is missing. + +### Fixtures (`conftest.py` at the repo root) + +| Fixture | Gives you | +|---|---| +| `comfyui` | The ComfyUI path on `sys.path`, or skips the test. | +| `tiny_h3` | A real, very small H3 model with a patcher. Use it to prove a modifier or hook on a real model. | +| `tiny_ltx` | A small real LTX model, built once per run. Clone its patcher; never leave anything installed on the model itself. | +| autouse | A private addons file, a private quarantine file, and block-influence switched off, all in `tmp_path`, so a developer's own settings never change a result. | + +`packed_av` and `unpacked` build and undo the flat audio and video packing H3 uses. + +### What the suite already enforces + +These run with the rest and fail the build if you break them. + +| Test | Rule | +|---|---| +| `core/tests/test_module_imports.py` | A module reaches core only through `_core` and never imports another module. | +| `core/tests/test_registry.py`, `test_retired_modules.py` | Announcement, ids, absence on failure. | +| `core/tests/test_shipped_hints.py` | Every shipped setting has a plain one-sentence hint and a unique label. | +| `core/tests/test_loads_as_comfyui_does.py` | The pack loads the way ComfyUI loads it. | +| `core/tests/test_log_call_sites.py` | Log calls use the right level and shape. | +| `core/tests/test_control.py` | Quarantine and `Unavailable` behave as described in [Architecture](architecture.md). | + +### Environment-dependent failures + +A few tests depend on the machine: a real GPU, a git checkout on a named branch, or a subprocess probe. On a stripped CPU box or a detached checkout they can fail without your change being the cause. Before you blame your change, run the same test on a clean checkout of the base branch. Name any such failure in your pull request instead of hiding it. + +## JavaScript tests (the app) + +``` +cd app +npm install # dev only: jsdom and Playwright. Users never run npm. +npm test # node --test --test-timeout=30000 +``` + +- Plain ES modules, no build step, no bundler. +- Tests sit beside the code (`*.test.js`) or in `composer/tests/` and `shell/tests/`. +- `composer/tests/_dom.js` gives a jsdom `window`. It has a URL, because without one `localStorage` throws and anything that remembers a preference silently takes its fallback. +- jsdom is honest about structure and events and dishonest about layout. It has no box model. Anything that depends on measured geometry (placement, scrolling, wrapping) belongs in a real browser: check it in the catalogue, or in a Playwright pass. + +### Testing a UI feature + +The pattern is in [Writing a UI feature](writing-a-ui-feature.md): build a host, a fake `app`, call `setup`, assert on the handle's node. Also test the hide paths: a missing need, an unoffered mount and a throwing `setup` must each leave nothing drawn. + +### Testing a kit element + +See [Adding an element](ui-kit-reference.md#adding-an-element). The registry-wide rule tests apply to your element the moment it has a demo. + +### `tools/kit_lint` + +`scan.js` applies `MODULE_RULES` to module `ui.js` files and `KIT_RULES` to the kit. `rule_no_module_styling.test.js` runs it and also runs the files in `composer/tests/lint_fixtures`, so a rule that stops catching its own bad example fails. It is a discipline check, not a security boundary. + +## Looking at it + +``` +python tools/devserver.py 8199 +``` + +Open `http://localhost:8199/funpack/`. The real routes are served and nothing is queued, so Generate says there is no queue. Set `COMFYUI_ROOT`, or most modules will be absent. The catalogue pages are in [UI kit reference](ui-kit-reference.md#the-catalogue). Check any visual change in dark and light and at a narrow width. + +## How to write a test that proves something + +- **Use fixtures, not real projects.** Build the smallest input that shows the behaviour. A test must not depend on someone's saved project, models or addons file. +- **Prove the mechanism end to end.** A unit test of a helper does not show the feature works. For a modifier: install it on `tiny_h3` or `tiny_ltx`, run a step, and compare the output with the module off. For a route: call it through the real router. For a UI feature: mount it with the real kit. +- **Attack your own change.** Ask what a hostile or careless caller would do: a missing key, a value outside `min`..`max`, a hook that raises on step 3, a name containing markup, a module that is absent. Write the test for the one most likely to happen. +- **Test the failure path as hard as the happy path.** The promise of this project is that a broken piece is absent and the rest carries on. A test for a new module should show that. +- **Fail first.** Make sure the test fails without your change. A test that passes either way proves nothing. +- **Say what was and was not checked.** A pull request states what was run and what could not be (for example, no GPU, so no real render). Do not describe a probabilistic effect as certain. +- **Real renders are a separate step.** Judging quality needs a real model on a real GPU and a person looking at the result. The unit tests prove the wiring; they do not prove the picture is better. + +## Before you push + +- [ ] `python -m pytest` and `npm test` pass, or the failures are shown to be unrelated. +- [ ] New code has a test that fails without it. +- [ ] No test depends on your machine's paths, projects or models. +- [ ] No long dashes in new text. No `node_modules` or scratch files committed. diff --git a/docs/ui-design-guidelines.md b/docs/ui-design-guidelines.md new file mode 100644 index 00000000..f1c44cb1 --- /dev/null +++ b/docs/ui-design-guidelines.md @@ -0,0 +1,150 @@ +# UI design guidelines + +How FunPack looks and behaves, and why. Read this before you add a screen, a setting or a kit element. For the element list see [UI kit reference](ui-kit-reference.md). For where a feature plugs in see [Writing a UI feature](writing-a-ui-feature.md). + +## Principles + +1. **Hide, do not warn.** If a feature cannot work (its module is missing, the model lacks a trait, it failed to load), it is not drawn. No grey button, no "unavailable" chip. A person never sees a control that cannot be used. The one place that explains absence is the Log. +2. **The screen is the truth.** Generate reads the live UI state. A module never reads the UI, and the UI never keeps a second copy of what the project already stores. +3. **Every control that changes the output is visible.** Nothing that alters a result hides behind a mode, a tab you did not open or a default you cannot see. Advanced does not mean secret. +4. **Never disrupt input.** A value the person is typing is not replaced under their hands. A panel redraw keeps focus and the caret. A toast never steals focus. +5. **Content is data.** Every string is shown as text. Markup in a name or a prompt is displayed, never run. The kit has no way to render HTML on purpose. +6. **Say only what is true.** A label describes what the control does today. Do not call something "always" if it is a chance, and do not call a trial feature proven. +7. **Calm by default.** One primary action per area. Quiet fills, no outlines for the sake of outlines, no row of loud buttons. + +## Where styling lives + +Only the kit styles anything: `app/ui/composer/` (tokens, internals, elements). A feature builds with kit elements and names no colour, size or class. + +- A rule test fails the build if kit code parses content as markup or compiles code (`tools/kit_lint` `KIT_RULES`). +- The same scanner is run over module `ui.js` files with stricter rules (`MODULE_RULES`): no inline style, no `classList`, no `document`, no raw hex colour, no `px` size. +- A few older features touch `document` or add `fp-` classes in `shell/frame.css`. Do not copy them. New work uses the kit, and if the kit lacks the element, add the element (see [Adding an element](ui-kit-reference.md#adding-an-element)). + +## Tokens + +Never write a literal colour, size, radius, duration or z-index. Use a token. Tokens live in `composer/tokens/tokens.css`; `polish.css` is loaded last and retunes some of them (fonts, weights, control heights, card radius), so read the final values in the catalogue (`catalogue/index.html`), not only in `tokens.css`. + +| Family | Tokens | Use | +|---|---|---| +| Type size | `--fs-1` .. `--fs-8`, display `--fs-d1` .. `--fs-d3` | Body text is `--fs-4`. Use the `text` and `header` elements instead of setting sizes. | +| Spacing | `--sp-0` .. `--sp-11` (2px base) | Gaps and padding. Pick the step, do not compute pixels. | +| Radius | `--r-1` .. `--r-5`, `--r-pill`, `--card-r` | Small controls low, cards high, pills for chips. | +| Control height | `--ctl-h-sm`, `--ctl-h-md`, `--ctl-h-lg` | Every button, input and select in a row shares one of these. | +| Motion | `--dur-1` .. `--dur-3`, `--ease-standard` | All become zero under reduced motion. | +| Density | `--cell-thumb`, `--cell-card`, `--cell-panel`, `--cell-icon`, `--gutter`, `--control-col`, `--menubar-h` | Media grid cells and layout columns. | +| Surface | `--elev-0` .. `--elev-3` | Page, panel, raised row, floating. Higher is closer to the person. | +| Lines and states | `--line`, `--hover`, `--active`, `--focus-ring` | Hairlines, hover and pressed fills, focus outline. | +| Text | `--text`, `--muted`, `--faint`, `--on-accent` | Main, secondary, tertiary, text on an accent fill. | +| Meaning | `--accent*`, `--teal`, `--violet`, `--good`, `--warn`, `--danger`, `--pink` | See the next section. | +| Depth | `--shadow-1` .. `--shadow-3`, `--grad-*`, `--void` | Derived. Do not redefine them. | + +Fonts are bundled (Inter, Inter Tight, IBM Plex Mono) and declared in `composer/tokens/fonts.css`. Use `--font-ui`, `--font-display` and `--font-mono`. Do not link a font from the web: the app must render offline. + +### Semantic colour + +Colour carries one meaning each. Do not reuse a colour for decoration. + +| Colour | Means | +|---|---| +| Accent (amber) | The one primary action, the selection, the focus of attention. | +| Teal | Rendering and audio. | +| Violet | Learned or taste-driven things. | +| Good | Done, saved, on target. | +| Warn | Needs a look, but nothing is broken. | +| Danger | Failed or destructive. | +| Muted, faint | Help, units, secondary detail. | + +Never carry meaning by colour alone. A chip says its state in words, an error has text, and a toggle has a visible on position. + +### Themes + +`shell/theme.js` sets `data-theme` (`dark` or `light`) on ``. The choice behind it is `data-theme-pref` (`auto`, `dark`, `light`) and is stored in `localStorage` under `funpack_theme`. `auto` follows the system and is resolved before first paint so there is no wrong-theme flash. + +- Token values are scoped to any `[data-theme]` element, so a single pane can show the other theme (the catalogue does this). +- Test every new element in both themes in the catalogue. +- A custom property used but never defined resolves to nothing without any error, so check that every `var(--x)` you write exists. + +## Layout and density + +- Build layout with `region.stack`, `splitPane`, `panel`, `toolbar`, `group` and `workspace`, not with ad hoc CSS. +- Align on the spacing scale. Related things sit close (`--sp-1`, `--sp-2`), separate groups sit apart (`--sp-4` and up). +- Media grids use `gallery` and its density control (`internals/density.js`, 0 to 4 columns, remembered per gallery id in `localStorage` as `cx.density.`, always wrapped in try/catch because storage can be blocked). +- Text never forces the layout wider. Long names truncate with the full name in the title. Numbers that change use tabular figures. +- Support narrow windows: a panel scrolls inside itself, the page does not scroll sideways. + +## Which element for what + +| You need | Use | Not | +|---|---|---| +| One action | `button` (`md`; one `primary` per area) | A link styled as a button | +| An action with only an icon | `iconButton` (the `label` is required and becomes the tooltip and the accessible name) | An icon with no name | +| An on/off setting | `toggle`, or `checkboxRow` in a list of options | A select with Yes and No | +| A number | `number`; add `slider` when the range is small and exploring helps | A text box for a number | +| One of 2 to 4 short options | `segmented` | A select | +| One of many options | `select`; `filterList` when the list is long | A very long segmented row | +| Short text | `input` | `textarea` | +| A prompt or notes | `textarea` | A one-line input | +| A colour | `color.swatch` | A hex text box | +| A message that stays | `banner` | A toast | +| A message that fades | `toast` | A banner that never leaves | +| A small state word | `chip` | Coloured plain text | +| Ask before a risky step | `modal.dialogue` with `tone: "danger"` | `window.confirm` | +| A list of things | `list.rows` or `gallery` | Hand-made divs | +| Group of related settings | `panel`, `collapsible` or `settingsRow` | A bare run of inputs | +| Wait with no known length | `progress.indeterminate`, `spinner`, `skeleton` | Frozen controls with no sign | + +Never call `alert`, `confirm` or `prompt`. They block the page, cannot be themed and cannot be tested. + +## Interaction + +- **Click targets** are at least the control height for their size. Dense toolbars use `iconButton.sm`; `micro` is for a row that already has its own hit area. +- **Commit rules.** Text and number inputs commit on blur or Enter (`onCommit`). Sliders report live movement through `onChange` and the settled value through `onCommit`. Do not start a run or write the project on every keystroke. +- **Focus.** The focus ring (`--focus-ring`) is always visible for keyboard use and is never removed. Overlays trap focus (`internals/focus.js` `trap`) and hand it back on close. Lists and menus use `roving` (arrow keys move, Tab leaves). +- **Escape.** Overlays register with the dismiss stack (`internals/dismiss.js`). Escape closes only the top entry, a click outside dismisses the top entry only if it was opened with `closeOnOutside`. `closeOnEsc` works the same way. Do not add your own key handler for this. +- **Busy.** A button doing work shows `busy` and ignores further clicks. It does not change its label to "Please wait". +- **Undo beats confirm** when the step can be undone. Confirm only for what cannot. +- **Drag.** Dragging uses `internals/drag.js` and `dragReorder.js`. Every drag has a keyboard or button way to do the same thing. + +## Overlays and layers + +`z-index` is written in one place only: `internals/zlayer.js`. Elements ask for a layer; they never pick a number. + +Order, low to high: `local`, `dropline`, `sticky`, `menubar`, `backdrop`, `wizard`, `floatingWindow`, `modal`, `popover`, `toast`, `autocomplete`, `tour`. Each rung owns a block of 100 slots, so peers stack inside a rung and never cross into the next. + +- A modal is above every floating window, because nothing behind it may be used. +- Autocomplete is above a modal on purpose: it opens from a field that is often inside one. +- Overlays are drawn into the portal root (`internals/portal.js`). The portal root must not create a stacking context, or the ladder stops working. +- An anchored popover positions itself with `internals/anchor.js` and flips to stay on screen. + +## Accessibility + +- Every control has an accessible name. Icon-only controls get it from `label`. +- State is exposed (`pressed`, `disabled`, `busy`) and not only drawn. +- Contrast: text on a surface uses `--text` or `--muted`. `--faint` is for non-essential detail only. Check both themes. +- Reduced motion: `--dur-*` become zero and `internals/motion.js` checks `prefers-reduced-motion`. A new animation must go through them. Do not use a raw `transition: 200ms`. +- Everything reachable by mouse is reachable by keyboard. + +## Words in the UI + +Dex's rule is plain words. Apply it to every label, hint, error and toast. + +- Short and plain. A label is a noun or a short verb phrase ("Steps", "Save project"). +- A hint is one sentence that says what the setting does for the result, not how it works inside ("Keeps the first picture's look through the clip"). +- Sentence case. No ending full stop on a label. A hint ends with one. +- A plain hyphen. No em dash or en dash anywhere in the repo, docs and code strings included. Existing code comments still contain some; do not add more. +- An error says what happened and what to do next ("The seed must be a whole number"). It never shows a stack trace or an internal name. +- Do not promise what is only likely. Not "always", not "guaranteed". Say "usually" or describe the setting. +- Do not describe a feature by its mechanism first. Lead with what the person gets. +- Use the same word for the same thing everywhere. If the app says "Project", do not say "Session" in a hint. +- No jargon a person would need a manual for. If a technical word is unavoidable, the hint explains it in a few plain words. + +## Checklist before you open a pull request + +- [ ] Only kit elements; no literal colour, size or z-index; no `document` in new feature code. +- [ ] Every string is passed as text. A name containing `` shows as ``. +- [ ] Looks right in dark and light, at a narrow width, with reduced motion on. +- [ ] Usable by keyboard alone; Escape closes the top overlay only. +- [ ] If the feature cannot load, nothing is drawn and the Log says why. +- [ ] Text follows the word rules above, and has no long dashes. +- [ ] Nothing a person is typing is overwritten by a redraw. +- [ ] `npm test` passes (see [Testing](testing.md)); a new element has a demo in the catalogue. diff --git a/docs/ui-kit-reference.md b/docs/ui-kit-reference.md new file mode 100644 index 00000000..d4b6a191 --- /dev/null +++ b/docs/ui-kit-reference.md @@ -0,0 +1,201 @@ +# UI kit reference + +The kit is called the Composer: `app/ui/composer/`. It is the only code that styles or builds raw DOM. For when to use which element, read [UI design guidelines](ui-design-guidelines.md). This page is the reference. + +## How elements work + +```js +import { composer } from "../../composer/composer.js"; + +const save = composer.button.md({ label: "Save", tone: "primary", onClick: save }); +host.appendChild(save.node); +save.setBusy(true); +// later +save.destroy(); +``` + +- An element is `composer..(props)`. It returns a **handle**: `{ node, destroy, ...methods }`. +- `node` is the DOM node to place. `destroy()` removes listeners and the node. A feature that creates elements after `setup` returns must destroy them itself. +- Methods differ by element. The common ones are `setValue`, `get`, `setText`, `setItems`, `setDisabled`, `setBusy`, `setPressed`, `open` and `close`. Read the factory in `composer/elements/.js` for the exact set. +- **Content is text.** Every string prop goes through `asText` (`internals/text.js`) and is shown as text, never parsed. A prop that would take HTML does not exist. +- **Containers take handles**, not nodes or markup: `panel.default({ children: [handleA, handleB] })`. +- **Unknown elements throw `UnknownElement`.** `composer.button.huge(...)` fails with the list of known variants. A feature that asks for an element that does not exist therefore hides itself instead of showing something approximate. +- The kit's own CSS classes start with `cx-`. App frame classes start with `fp-` (`shell/frame.css`). Never use either from a feature. + +### Where each group lives + +| File | Groups | +|---|---| +| `text.js` | `brand`, `code`, `header`, `hint`, `label`, `text` | +| `button.js` | `button`, `buttonGroup`, `iconButton` | +| `input.js` | `filterList`, `input`, `number`, `search`, `stepper`, `textarea` | +| `choice.js` | `checkbox`, `checkboxRow`, `checklist`, `color`, `radioGroup`, `segmented`, `select`, `toggle` | +| `slider.js` | `range`, `slider` | +| `layout.js` | `actionBar`, `collapsible`, `field`, `frame`, `group`, `list`, `panel`, `region`, `settingsRow`, `sidebar`, `splitPane`, `tabs`, `toolbar`, `workspace` | +| `status.js` | `banner`, `chip`, `dot`, `dropzone`, `emptyState`, `inlineError`, `progress`, `skeleton`, `spinner`, `toast` | +| `modal.js` | `modal` | +| `popover.js` | `autocomplete`, `menu`, `popover`, `splitButton`, `tooltip` | +| `floating.js` | `floating`, `overlay`, `slideOver` | +| `gallery.js` | `gallery`, `ruler`, `viewer` | +| `track.js`, `nle.js` | `track`, `timeline` | +| `wheel.js` | `wheel` | + +## Elements + +Variants are listed after the group. Props shown are the ones every use needs; the catalogue (`catalogue/elements.html`) and `catalogue/demos.js` show each element with a working set. + +### Text + +| Element | Variants | Main props | +|---|---|---| +| `header` | `display`, `xl`, `lg`, `md`, `sm` | `text` | +| `text` | `lg`, `md`, `sm`, `xs` | `text`. Has `setText`. | +| `hint` | `default` | `text` | +| `label` | `field`, `section` | `text`, `for` | +| `code` | `inline`, `block` | `text`, `label` (block) | +| `brand` | `default` | `name` | + +### Actions + +| Element | Variants | Main props | +|---|---|---| +| `button` | `xl`, `lg`, `md`, `sm`, `menu` | `label`, `onClick`, `tone` (`neutral`, `primary`, `danger`, `ghost`, `render`), `icon`, `disabled`, `busy`, `type`, `title`, `pressed`. Methods `setBusy`, `setDisabled`, `setPressed`, `setLabel`. | +| `iconButton` | `md`, `sm`, `micro` | `icon`, `label` (required), `onClick`, `tone`, `disabled`, `pressed` | +| `buttonGroup` | `md` | `label`, `value`, `items` | +| `splitButton` | `md` | `label`, `items`, `onClick` | + +### Inputs and choices + +| Element | Variants | Main props | +|---|---|---| +| `input` | `md`, `sm` | `value`, `placeholder`, `label`, `onCommit` | +| `search` | `md` | `placeholder`, `label` | +| `textarea` | `md` | `value`, `rows`, `label` | +| `number` | `md` | `value`, `min`, `max`, `step`, `precision`, `unit`, `snap`, `onChange`, `disabled`, `id`, `label` | +| `stepper` | `md` | `value`, `min`, `max`, `step`, `label` | +| `slider` | `md`, `sm`, `readout`, `macro` | `value`, `min`, `max`, `step`, `label`, `onChange`, `onCommit` | +| `range` | `md` | `from`, `to`, `min`, `max`, `step`, `label` | +| `toggle` | `default` | `label`, `hint`, `checked`, `onChange` | +| `checkbox` | `default` | `checked`, `label` | +| `checkboxRow` | `default` | `label`, `hint`, `checked` | +| `checklist` | `default` | `label`, `values`, `items` | +| `radioGroup` | `default` | `label`, `value`, `options` | +| `segmented` | `md`, `sm` | `label`, `value`, `options`, `onChange` | +| `select` | `md`, `sm` | `label`, `value`, `options`, `onChange` | +| `filterList` | `md` | `value`, `items` | +| `autocomplete` | `default` | anchored to an input; sits above modals | +| `color` | `swatch` | `label`, `value` | +| `wheel` | `picker`, `half` | `edge`, `items` | +| `dropzone` | `default` | `label`, `hint` | + +`options` is a list of `{ value, label }`. Changes are reported as the value, not as an event. + +### Layout + +| Element | Variants | Notes | +|---|---|---| +| `frame` | `app` | The page frame. Owned by the shell. | +| `workspace` | `docked` | Docked areas around a centre. | +| `splitPane` | `h`, `v` | Two children with a draggable divider. | +| `region` | `stack` | `children`, `gap`, `label`, `fill`. `.set(children)` replaces them. | +| `panel` | `default`, `zone` | A titled card. | +| `group` | `default` | A quiet sub-block. | +| `field` | `default`, `row` | A label, a control and a hint. | +| `settingsRow` | `default` | The row the Settings window uses. | +| `collapsible` | `default` | | +| `tabs` | `underline`, `dock` | `value`, `tabs`. | +| `sidebar` | `rail` | `value`, `items`. | +| `toolbar`, `actionBar` | `default`, `sticky` | Toolbars wrap; an action bar sticks to the bottom of its scroll area. | +| `list` | `rows` | Grouped rows. | + +### Status and feedback + +| Element | Variants | Main props | +|---|---|---| +| `banner` | `info`, `warn`, `danger` | `text` | +| `chip` | `neutral`, `good`, `warn`, `danger`, `info` | `label` | +| `dot` | `default` | `tone`, `label` | +| `inlineError` | `default` | `text` | +| `toast` | `info`, `good`, `warn`, `danger` | `text`, `action`, `duration` (default 3200 ms) | +| `progress` | `bar`, `indeterminate` | `value`, `max`, `label` | +| `spinner` | `md`, `sm` | `label` | +| `skeleton` | `line`, `block`, `grid` | `count`, `label` | +| `emptyState` | `default` | `icon`, `title`, `hint` | + +### Overlays + +| Element | Variants | Main props | +|---|---|---| +| `modal` | `generic`, `stacked`, `dialogue`, `prompt`, `choice` | `generic`: `title`, `subtitle`, `size` (`sm`, `md`, `lg`, `xl`), `body`, `actions`, `onClose`, `closeOnOutside`, `closeOnEsc`. `dialogue`: `title`, `message`, `tone`, `confirmLabel`, `cancelLabel`, `onResult`, and a `.result` promise. | +| `popover` | `anchored` | Positioned by `internals/anchor.js`, flips at an edge. | +| `menu` | `dropdown`, `context` | A context menu is anchored to a point, not an element. | +| `tooltip` | `default` | | +| `floating` | `window` | A draggable window (the Settings and Sampler windows). | +| `slideOver` | `md` | A panel that slides in from the side. | +| `overlay` | `blocking` | `message`. Blocks the page while something runs. | + +All overlays use the portal, the z-ladder and the dismiss stack. + +### Media and time + +| Element | Variants | Main props | +|---|---|---| +| `gallery` | `adaptive`, `icons`, `list`, `rows`, `cards`, `strip` | `id`, `cols`, `items`. Density through `internals/density.js`. | +| `viewer` | `media` | `src`, `empty` | +| `ruler` | `default` | `label`, `total`, `marks` | +| `track` | `default` | `label`, `pxPerSecond`, `playhead`, `selection`, `items` | +| `timeline` | `stage` | `pxPerSecond`, `lanes`. Methods `setLanes`, `setPlayhead`, `setZoom`. | + +## Internals + +Used by elements. A feature does not import them. They exist in `composer/internals/`: + +| File | Does | +|---|---| +| `register.js` | `define`, `UnknownElement`, lookups. | +| `el.js`, `text.js` | Safe DOM building, `asText`, `setText`. | +| `ids.js` | Unique ids for label and control pairs. | +| `zlayer.js` | The z-ladder. The only place a `z-index` is written. | +| `portal.js` | Where overlays are drawn. | +| `dismiss.js` | One Escape and outside-click stack. | +| `focus.js` | `trap` and `roving`. | +| `anchor.js` | Pure placement (measure, flip, clamp) plus `anchorTo`. | +| `drag.js`, `dragReorder.js`, `snap.js`, `swapping.js` | Dragging, reordering and snapping. | +| `density.js` | Grid columns 0 to 4, saved per id. | +| `motion.js` | Reduced motion check and shared timings. | + +## Adding an element + +1. **Write it** in the right file under `composer/elements/` (or a new file). Call `define(group, variant, factory)` once per variant. The factory takes props and returns a handle. Use only `el`, `asText` and the internals. Never write `innerHTML`. +2. **Register the file.** A new file is imported in `elements/index.js` and its stylesheet is added to `composer.css` before `polish.css`. +3. **Style it** with tokens only, using `cx-` class names. Cover both themes and reduced motion. +4. **Add a demo** to `catalogue/demos.js` under `"."`. A registered element with no demo fails the suite. A demo is a props object, or a function that receives `composer` when the props need other handles. +5. **Teach the content test** any new string prop name. `composer/tests/rule_content_is_data.test.js` attacks every string under the names in `CONTENT_KEYS` with a markup payload and proves it comes back as text. If your prop name is not in the set (for example `caption` or `empty`), add it, or the test fails with a message naming the demo. +6. **Test behaviour** in `composer/tests/elements_behaviour.test.js` or a new `*.test.js` beside it: what the element does when used, not only that it renders. +7. **Look at it** in `catalogue/elements.html` in dark and light. + +## Rule tests that run over the whole kit + +| Test | Proves | +|---|---| +| `rule_content_is_data.test.js` | Every string prop of every demo is shown as text. | +| `rule_unknown_element_hides.test.js` | Asking for a missing element throws and hides the feature. | +| `rule_no_module_styling.test.js` | `tools/kit_lint` finds nothing in `modules/`, and its own fixtures (`tests/lint_fixtures`) still trip every rule. | +| `register.test.js` | Duplicate definitions are refused. | +| `overlays.test.js`, `dismiss.test.js`, `zlayer.test.js`, `portal.test.js` | Layering, Escape and portal behaviour. | +| `panel_schema.test.js` | `panel.js` draws every setting type. | + +## `panel.js` and the catalogue + +`composer/panel.js` renders a module's `SETTINGS` as a panel with the allowed `ui` renderers. The live app does not use it for Settings (it uses `features/settings/field.js`); the catalogue and the tests do. So a `ui` value changes what the catalogue shows, not what Settings > Engine draws. See [Module reference](module-reference.md#renderers). + +## The catalogue + +Open `/funpack/app/ui/catalogue/` with the devserver running (`python tools/devserver.py 8199`). + +| Page | Shows | +|---|---| +| `index.html` | Tokens: type, colour, spacing, radius, motion. | +| `elements.html` | Every registered element in both themes. | +| `hide.html` | What happens when a module is broken or missing. | +| `log.html` | The Log view. | diff --git a/docs/writing-a-module.md b/docs/writing-a-module.md new file mode 100644 index 00000000..8e47b8b5 --- /dev/null +++ b/docs/writing-a-module.md @@ -0,0 +1,353 @@ +# Writing a module + +A module is one folder of Python that adds one feature. This guide builds a small one, then explains each part. Every field and capability is listed in the [Module reference](module-reference.md). Read [How FunPack is built](architecture.md) first if you have not. + +## Where a module lives + +``` +modules/// + __init__.py the announcement (required) + nodes.py ComfyUI nodes, if the module has any + tests/test_*.py tests (required for anything you want merged) +``` + +- Exactly two levels: a group, then a module. Groups today: `conditioning`, `latent`, `loaders`, `media`, `models`, `output`, `sampling`, `system`. A new group is just a new folder. +- A folder that starts with `.` or `_`, or has no `__init__.py`, is not scanned. +- A folder whose `__init__.py` has no `ID` is a helper package, not an error. +- A plain `.py` file at group level (`modules/loaders/common.py`) is a shared helper. It announces nothing and may not import a module. +- Core finds modules by scanning. There is no list to edit. + +## A small modifier, end to end + +This module scales the model's prediction down a little on every step. It is a teaching example (not shipped). + +`modules/sampling/calm/__init__.py` + +```python +"""Calm: scales the model's prediction down a little on every step.""" + +from comfy.patcher_extension import WrappersMP + +from ..._core import log + +ID = "calm" +TITLE = "Calm" +MOUNT = "generation.sampling" +STAGE = "guidance" +CATEGORY = "guidance" +STATUS = "experimental" + +SETTINGS = { + "enabled": { + "type": "bool", "default": False, + "label": "Calm the picture", + "hint": "Scales the model's prediction down a little on every step.", + }, + "strength": { + "type": "float", "default": 0.1, "min": 0.0, "max": 0.5, "step": 0.01, + "label": "How much", "ui": "slider", + "hint": "0 = no change. Higher scales the prediction down further.", + "when": {"enabled": True}, + }, +} + + +def install(patcher, values, key): + strength = float(values["strength"]) + if not values["enabled"] or strength <= 0.0: + return None # off: say nothing, install nothing + + def per_call(executor, *args, **kwargs): + video, audio = executor(*args, **kwargs) + log.once(f"{ID}:ran", log.INFO, "FunPack Calm", f"scaled the picture by {1.0 - strength:g}") + return video * (1.0 - strength), audio # the picture only; sound is never touched + + patcher.add_wrapper_with_key(WrappersMP.DIFFUSION_MODEL, key, per_call) + return f"strength {strength:g}" # shown in the run's status + + +PROVIDES = {"modifier": install} +``` + +`modules/sampling/calm/tests/test_calm.py` + +```python +import pytest +import torch + + +@pytest.fixture(autouse=True) +def _needs_comfy(comfyui): + """Imports comfy.""" + + +def _load(tiny, **values): + from modules.sampling.modifiers.nodes import FunPackLoadModifiers + return FunPackLoadModifiers.execute(tiny.patcher, {"calm": values}).result + + +def test_it_is_off_until_switched_on(tiny_h3): + _patched, status = _load(tiny_h3) + assert "calm" not in status.splitlines()[0] + + +def test_it_scales_the_picture_and_never_the_sound(tiny_h3): + base_video, base_audio = tiny_h3.run() + patched, status = _load(tiny_h3, enabled=True, strength=0.5) + assert "calm: strength 0.5" in status + video, audio = tiny_h3.run(patched) + assert torch.allclose(video, base_video * 0.5, atol=1e-5) + assert torch.equal(audio, base_audio) + + +def test_it_leaves_the_original_model_alone(tiny_h3): + _load(tiny_h3, enabled=True, strength=0.5) + base_video, _ = tiny_h3.run() + assert torch.equal(base_video, tiny_h3.run(tiny_h3.patcher)[0]) +``` + +Run it with `pytest modules/sampling/calm` (see [Testing](testing.md) for the ComfyUI setup). That is the whole module. It needed no UI code, no registration and no list entry: the settings appear in **Settings > Engine > Guidance**, the node `FunPack Load Modifiers` installs it, and the Log names it when it runs. + +## The announcement + +Constants at the top of `__init__.py`, uppercase because they are declarations, not behaviour. Core reads them without running anything. + +| Constant | Needed | Meaning | +|---|---|---| +| `ID` | yes | Unique, lowercase, no `.`. Saved projects refer to it, so never rename one lightly. Write it as `ID = "name"` on one line (tests read it that way). | +| `TITLE` | yes | The name people see. | +| `MOUNT` | when it has `SETTINGS` | A non-empty string. Use `"generation.sampling"`. See the note below. | +| `STAGE` | no, default `sampling` | Coarse run order: `load`, `conditioning`, `latent`, `guidance`, `sampling`, `post`. | +| `CATEGORY` | no | Where its settings are grouped: `continuity`, `guidance`, `conditioning`, `sampling`, `post`, `system`. | +| `STATUS` | no, default `experimental` | `proven` or `experimental`. Be honest, see below. | +| `SETTINGS` | no | What people can set. | +| `PROVIDES` | no | Things this module offers other modules, and how it plugs into a run. | +| `NODES` | no | ComfyUI node classes. | +| `REQUIRES` | no | Model traits it needs. | +| `USES`, `USES_WHEN` | no | Capabilities it asks other modules for. | +| `ROLES` | no | Pipeline inputs (`at` names) it acts through. | +| `AFTER`, `BEFORE` | no | Order among modules of the same stage. | +| `HOOKS` | no | Hook points a sampler must offer. | +| `TRAITS` | no | A model module's way of teaching the system what a model is. | + +A declaration that is wrong does not load, and the Log says exactly what is wrong. There is no best-effort mode. + +**About `MOUNT`.** Core requires it when a module has settings, so give it. In the app as shipped, though, module settings are drawn by Settings > Engine and the Sampler window, which group by `CATEGORY`, not `MOUNT`. `MOUNT` only matters to the optional `ui.js` path described at the end. + +## Settings + +A setting is a **preference**, a plain value. Data (an image, a mask, a clip) is never a setting: it travels as a ComfyUI socket. + +```python +"name": { + "type": "float", # bool, int, float, enum, text, multiline, path, color + "default": 0.5, # required; must fit the type and the range + "min": 0.0, "max": 1.0, "step": 0.05, + "label": "Short name", + "hint": "What it does, in one plain sentence.", + "unit": "x", + "ui": "slider", + "when": {"enabled": True}, +} +``` + +What is checked when the module loads (`core/schema.py`): + +- `type` is one of the eight. `default` is present and fits (an `int` default is an `int`, a `bool` is a `bool`, an enum default is one of its options, a colour is `#rgb`, `#rrggbb` or `#rrggbbaa`). +- Numbers are finite and the default sits inside `min` and `max`. `min` is not above `max`. `step` is positive. +- An enum has `options`: a non-empty list of `{"value": scalar, "label": text}` with no repeated values. +- `ui`, if given, is a renderer allowed for that type. See [what the app really draws](#what-the-app-draws) below. +- `label`, `hint`, `unit` and `placeholder` are strings. +- `when` names other settings of the same module, and (for an enum) values it can actually hold. A condition that could never be true is refused instead of silently hiding the row. + +What is checked on every run: each saved value is checked against the declaration. A wrong value (out of range, wrong type, NaN) refuses the run and names the setting. A key the module no longer declares is skipped with one Log line, so old projects keep working. + +What the tests add (`core/tests/test_shipped_hints.py`), so treat them as rules: + +- Every setting has a non-empty `hint`. +- No two settings of one module share a `label`. +- No hint tells people to look in "the inspector" (there is no such control for settings). + +### Conventions that make a module feel like the others + +- A feature gets a bool `enabled`, default `False`, and every other row has `"when": {"enabled": True}`. A module that `USES` a capability only counts as working when its `enabled` is true, which is what keeps the provider's settings (the taste key) hidden until a learning feature is on. +- A strength of `0` means off, or "learn only" if the module learns. Say which in the hint. +- Chance values: `1` is always, `0` is never. Do not call a probabilistic feature "always". +- Write labels and hints in plain words, one sentence, what it does for the clip and not how it works inside. "Makes the motion smoother" beats a description of the mechanism. +- Every control that changes the output must be a setting, so it is visible and switchable in the app. Do not hide a knob in code or only on a node. +- Never make a user's value inert. If a value cannot act, the module says so in the Log or refuses the run. + +### What the app draws + +Two code paths can draw a setting, and they differ: + +- **Settings > Engine and the Sampler window** (`app/ui/features/settings/field.js`). This is what people use. It draws by `type`: `bool` as a toggle; `int` and `float` as a slider when `ui` is absent or `"slider"` and both `min` and `max` are set, otherwise as a number box; `enum` as a select; `color` as a swatch; `multiline` as a text area; `text` and `path` as a text box. A `ui` of `segmented`, `wheel`, `radioGroup`, `filterList` and so on is accepted by the schema but this page still draws the default for the type. `placeholder` is not used here. +- **`composer/panel.js`** honours every `ui` hint in `UI_HINTS`. It is used by the catalogue and the tests, and by the optional `ui.js` route. + +So: choose `ui` only between `slider` and `number` for numbers, and design for the select and toggle everywhere else. + +### Which settings are shown + +The app hides a module's settings when nothing would read them (`usefulness` in `pipeline_state.js`): + +- its `NODES` are not in the current pipeline; +- its `ROLES` (pipeline input `at` names) are not filled by any slot; +- it provides a capability that other modules `USES`, and none of those users is on (the taste key is hidden until a learning feature is on); +- the model lacks a trait in `REQUIRES`. + +Hide, do not warn: a feature that cannot work here is removed from the screen, not explained. The one exception is a switched-on setting that cannot act, which should warn beside Generate. + +## Modifiers: changing how a model runs + +A modifier is the usual module. It provides `"modifier"`, a function `install(patcher, values, key)`. + +- `patcher` is a **clone** of the model's patcher, wrapped so every hook you add is guarded (see below). The original model is never touched. +- `values` is a dict with every declared setting, defaults filled in and checked. Read `values["x"]` directly. +- `key` is `"funpack."`. Use it as the hook key, or `f"{key}.suffix"` for a second hook. +- Return `None` if the module decided it is off. Return a short string if it installed: it appears in the run's status as `: `. Say what it did, with the numbers ("strength 0.3, blocks 29-46"). +- Raising makes the module off for the run, logs it and quarantines it (see failure model in the architecture page). + +### Hooks + +Install through ComfyUI's own mechanisms on the patcher you were handed: + +| You want | Use | +|---|---| +| To wrap one model call | `patcher.add_wrapper_with_key(WrappersMP.APPLY_MODEL, key, fn)` or `DIFFUSION_MODEL` | +| To wrap a whole sample or its setup | `WrappersMP.OUTER_SAMPLE`, `PREPARE_SAMPLING`, `SAMPLER_SAMPLE` | +| To change guidance | `patcher.set_model_sampler_pre_cfg_function(fn)` or `..._post_cfg_function` | +| To replace a transformer block or attention | `core/dit_hooks.py`: `add_block_hook(patcher, key, index, hook)`, `add_attention_override(patcher, key, override)` | + +Rules that exist because they were learned the hard way: + +- **Tag what you install.** `add_wrapper_with_key` with your `key` is removable by key. Anonymous hooks (the `set_model_*` lists, `add_object_patch`) must be tagged: `patching.tag(fn, key)`. Before every install, `FunPack Load Modifiers` strips everything tagged `funpack.*`, so a model run through twice is not patched twice. An untagged hook cannot be removed and doubles every time. +- **Chain, do not assign.** Block replacement and attention override are single slots. `dit_hooks` chains them so your hook can call `extra["original_block"](args)` and never knows other modules exist. +- **Gate on step position.** `dit_hooks.current_step(transformer_options)` gives `(index, total)`. Never gate on a sigma threshold: schedules differ and a gate can be open on none of the steps. `late_half`, `last_step` and `weak_branch` are ready-made. +- **Ignore probe calls.** `dit_hooks.probing(to)` is true for throwaway model calls (a candidate scored and discarded). Learn and capture nothing from one. +- **Expect more than one call per step.** Second-order samplers call the model twice. If you must act once per step, track it and say so once when you cannot (see `velocity_bias`). +- **The picture and the sound.** `streams.video_of(x, named)` returns the picture and a function to put a changed picture back, whatever the model packs around it. A picture modifier never changes the sound. +- **Reset your state at the start of every run.** ComfyUI caches a node whose inputs have not changed, and between most generations only the seed changes, so `install` does not run again and a closure variable lives on into the next clip. Clear per-run state in an `OUTER_SAMPLE` wrapper (see `momentum`'s `forget`), and clear anything the size of a latent when the run ends. If your install depends on something outside the node's inputs (a switch in Settings), also provide `"cache_key"`: a function returning a value that changes with it, so ComfyUI re-runs the node. +- **Read call arguments by name.** `streams.model_args(args, kwargs)` returns `c_crossattn`, `transformer_options` and so on whether ComfyUI passed them positionally or by name. + +### Guards + +You do not opt in to being safe. The patcher you receive guards each hook: if it raises, it is dropped for the rest of the run, the run goes on without it, and the Log carries the traceback once. What the executor itself raises (an interrupt, running out of memory) passes through untouched and is not blamed on you. + +One mode removes the guards: the **Full control** module, for people who want the raw behaviour. Do not write code that only works with guards off. + +If a module cannot run on this machine (an older ComfyUI, a missing package), raise `control.Unavailable`. It is said in the Log but the module is not quarantined. + +## Sampler modifiers + +For effects that need the step index, the schedule, or to run a step twice, a module can provide `"sampler_modifier"` instead: a function `make(values, accepts)` returning an object with `active(ctx)` and `process(ctx, latent)`, plus `HOOKS = [...]` naming hook points the sampler must offer (today `"latent"`). `core/chain.py` runs the members in order after each step; a member that raises is dropped for the run. No shipped module uses this yet; prefer `"modifier"` unless you need it. + +## Nodes + +A module may contribute ComfyUI nodes with `NODES = [MyNode]`. + +- Write them against `comfy_api.latest.io` (the V3 `io.ComfyNode`): `define_schema()` and `execute()`. +- `node_id` must start with `FunPack`. Node ids are global in ComfyUI and a clash silently overwrites, so a node without the prefix, or reusing an id, is left out and said. +- Category `FunPack/`. Give every input a `tooltip` that says what it does. +- Return a status string as the last output when it helps ("applied", "off") and use `log` for the reasons. +- A node whose result depends on something outside its inputs (a switch in Settings) should answer `fingerprint_inputs` so ComfyUI re-runs it instead of handing back a cached result. +- A **node widget** describes a file or how it is read (a loader's dtype). A **setting** is a feature knob (a strength, a window). Do not put feature knobs on loader nodes. + +A module with only nodes needs no `MOUNT` and no `SETTINGS`. + +## Model modules + +Support for a model is a module in `modules/models/`. Core names no model. + +- `detect(keys) -> bool` and `probe_traits(keys) -> list` read a safetensors header (tensor names) so the app can recognise a file before loading it. +- `TRAITS = traits` where `traits(model)` returns extra trait names for a loaded model. +- `empty_latent(model, width, height, length, batch_size)` and `decode(latent, model=..., vae=..., audio_vae=..., tile_size=...)` return `None` for models that are not yours. Claim by identity, never by shape. If you claim the model and fail, raise: falling back would feed a wrong shape to another model's code. +- `pipeline_presets()` returns ready-made pipelines (lists of slots). `optional_nodes()` says which slots can be passed over when their module is missing. +- Other capabilities teach shared code about your model: `video_stream`, `text_rows`, `prompt_rows`, `video_time_rows`, `query_rotation`, `target_rows`, `anchor_pin`, `latent_upscaler`, `rescale_conditioning`. +- A model you have not run for real stays labelled **UNTESTED** in its presets, its module docstring and the Log (see `modules/models/ltx`). + +### Traits + +Compatibility is by trait, never by model name. A model says what it **is**, a module says what it **needs** (`REQUIRES`). Core reads universal traits off the loaded model: `temporal_latent`, `spatial_latent`, `waveform_latent`, `temporal_compression`, `predict_`. Model modules add their own (`audio_stream`, `dit_block_hooks`, `adaln_modalities`, `ltx_av`). A module that declares no traits works everywhere, and declaring one is how it leaves models it cannot work on. To test a structure, use `traits.has_block(model, "pkg.module.ClassName")` with a dotted name: bare names like `Attention` collide across models. + +## Order + +`STAGE` is the coarse order. `AFTER` and `BEFORE` list module ids for finer order inside a stage. There are no priority numbers, on purpose: a number is a global namespace. A relation naming a module that is absent is ignored. A relation that tries to cross stages is dropped and said. A cycle drops the modules on it, and says which. + +## Talking to other modules + +Never import another module at the top of a file (a test fails the build). Instead: + +- **Offer**: `PROVIDES = {"name": callable}`. Anyone can ask for it. +- **Ask**: `registry.current().ask("name", *args)` returns the first provider's non-`None` answer, in id order, or `None`. A provider that raises is skipped and said once. `registry.current().providers("name")` lists every provider. +- **Declare**: `USES = ["name"]` tells the app you depend on a capability, so a provider's settings (the taste key) are hidden while no user is on. `USES_WHEN = {"mode": "learned"}` says you only ask while your own settings hold those values. +- If you need another module's code, import it inside the function that needs it, guarded. + +Core never defines a capability name; the contract is between the module that provides it and the ones that ask. The names in use are in the [Module reference](module-reference.md#capabilities). + +### Routes + +A module that needs endpoints provides `"routes": routes(table, base, web)`. `base` is `/funpack/api/m/`, so two modules never collide and removing a module removes its routes. Return JSON; on a bad request return `{"why": "..."}` with status 400, and say what to fix. + +### Learning from ratings + +Modules that learn from a person's ratings use the taste key (`modules/system/taste`). The pattern, as `velocity_bias` does it: + +1. `USES = ["taste_store"]`. +2. In `install`, `taste = registry.current().ask("taste_store", patcher)`. If it is `None`, say "set a Taste key first" once and return `None`. +3. `taste.collect(patcher, key, KIND, ...)` captures this run's rows, `taste.rows(KIND)` reads the rated rows, and a rating arrives through the taste module. +4. A module that must hear every rating itself provides `"on_rating": fn(prompt_id, rating, axis)`. + +Ratings are odds, not bans. Say in the hint how many liked (and disliked) clips it needs before it acts, and say so in the Log when it does not have enough. + +## Saying what happened + +`log` (`from ..._core import log`) has four levels. The difference between the middle two matters. + +| Level | Use it when | +|---|---| +| `log.info` | It worked, and this is what it actually did. "applied: the average of 3 liked clips" is how people learn that a setting did nothing. | +| `log.alert` | Something is unset or unconnected and the run continues without it. | +| `log.warning` | Something will not do what its name says. Never swallow a failure and carry on silently. | +| `log.critical` | The run cannot proceed. | + +`log.once(key, level, source, message)` says it the first time per run. Use it for anything inside a per-step hook. Thirty copies of one line say nothing the first did not. `log.broke(what, exc, doing)` and `log.failed(what, exc)` are for exceptions. Use `FunPack ` as the source. + +Research or debug data does not go in the console: put a toggle, a Download button and a Clear button in Settings (see Refinement & Taste). + +## Honest labels + +- `STATUS = "proven"` means it has been run on a real model and judged good. Everything else is `experimental`. The [CHANGELOG](../CHANGELOG.md) lists the two apart, and the list must stay true. +- Do not describe a feature as a guarantee. A feature that depends on chance or on learning says so. +- Keep overhead small. A per-step feature that adds a third to the step time is already costly; one that doubles it is not acceptable. Say the cost in the hint if it is noticeable. +- Fewest words in docstrings, hints and tooltips. No backstory and no future plans in the repo. +- Nothing NSFW in code, tests, comments, commits or docs. Build test data from constructed fixtures. + +## Optional `ui.js` + +A module may have `ui.js` next to its `__init__.py`. Core serves it at `/funpack/modules/<group>/<name>/ui.js` (only `.js` files are served from `modules/`, so a module cannot ship CSS) and puts the path in its manifest. The contract in `app/ui/shell/panels.js`: + +```js +export function setup({ values, on, shell }) { + // values.get() / values.set(key, value) this module's settings only + // on((key, value) => ...) hears this module's changes only + // shell.theme, shell.density the small list of app services + return () => { /* teardown */ }; +} +``` + +A module's `ui.js` gets **no element factory and no DOM**. It cannot style, build or inject anything: `tools/kit_lint` fails the build on inline styles, class changes, `document`, `window`, `globalThis`, `innerHTML`, hardcoded colours or pixel sizes, stylesheets and `eval`. The settings panel is drawn for it from the declaration. + +Be aware that `ui.js` is only loaded by `mountAll` in `shell/panels.js`, which the catalogue and the tests call. The running app (`boot.js`) does not call it, and no shipped module has a `ui.js`. Anything a person needs to see in the app is a setting, or a [UI feature](writing-a-ui-feature.md). + +## Before you open a pull request + +- [ ] Folder in `modules/<group>/<name>/`, unique `ID`, honest `STATUS`. +- [ ] Reaches core only with `from ..._core import ...`; imports no other module at load. +- [ ] Every setting has a default, a plain `label` (unique in the module) and a one-sentence `hint`. +- [ ] Every control that changes the output is a setting. +- [ ] `install` returns `None` when off, a status string when on, and a hook key that starts with `funpack.`. +- [ ] Anonymous hooks are tagged. Nothing leaks from one run to the next. +- [ ] Tests on a real tiny model (`tiny_h3`, `tiny_ltx`) or a real object, not a stand-in: off by default, the effect, nothing leaks, bad input. +- [ ] The Log says what happened, once, at the right level. +- [ ] `pytest` and `cd app && npm test` pass. +- [ ] CHANGELOG has the feature under Proven or Experimental. diff --git a/docs/writing-a-ui-feature.md b/docs/writing-a-ui-feature.md new file mode 100644 index 00000000..e1fab1e0 --- /dev/null +++ b/docs/writing-a-ui-feature.md @@ -0,0 +1,238 @@ +# Writing a UI feature + +A UI feature is one folder of JavaScript that adds one thing to the screen: a button, a panel, a timeline lane, a Settings section, a menu entry. Features are plain ES modules. There is no build step, no framework and no bundler, so what you write is what the browser runs. + +Before you start, decide whether you need one. A **module setting** (see [Writing a module](writing-a-module.md)) is drawn for you in Settings > Engine and needs no JavaScript. Write a feature only when the screen needs something a setting cannot say. Read [UI design guidelines](ui-design-guidelines.md) too: they apply to every feature. + +## A feature in 30 lines + +`app/ui/features/scene-count/index.js` + +```js +// The number of scenes in the project, beside the dock buttons. +import { composer as c } from "../../composer/composer.js"; + +export default { + id: "scene-count", + mount: "timeline.status", + needs: ["project"], + setup({ host, app }) { + const label = c.text.xs({ text: "" }); + const draw = () => { + const n = app.project.scenes.length; + label.setText(`${n} ${n === 1 ? "scene" : "scenes"}`); + }; + draw(); + host.append(label.node); + const off = app.on(draw); + return () => { off(); label.destroy(); }; + }, +}; +``` + +`app/ui/features/scene-count/hub.test.js` + +```js +import test from "node:test"; +import assert from "node:assert/strict"; +import { setupDom, teardownDom } from "../../composer/tests/_dom.js"; +import { offer, _reset } from "../../shell/mounts.js"; +import { loadFeatures } from "../../shell/features.js"; + +let hub; +test.before(async () => { setupDom(); hub = (await import("./index.js")).default; }); +test.after(() => teardownDom()); + +test("shows the number of scenes, follows edits, and leaves nothing behind", async () => { + _reset(); + const host = document.createElement("div"); + offer("timeline.status", host); + const heard = new Set(); + const app = { project: { scenes: [{ id: "a" }] }, on: (fn) => { heard.add(fn); return () => heard.delete(fn); } }; + const { mounted, hidden } = await loadFeatures(["x"], { load: async () => ({ default: hub }), app, has: (n) => n in app }); + assert.deepEqual(hidden, []); + assert.match(host.textContent, /1 scene$/); + + app.project.scenes.push({ id: "b" }); + heard.forEach((fn) => fn("change")); + assert.match(host.textContent, /2 scenes$/); + + mounted[0].teardown(); + assert.equal(host.textContent, ""); + assert.equal(heard.size, 0); +}); + +test("without a project service the feature is hidden, not broken", async () => { + _reset(); + offer("timeline.status", document.createElement("div")); + const { mounted, hidden } = await loadFeatures(["x"], { load: async () => ({ default: hub }), app: {}, has: (n) => n in {} }); + assert.equal(mounted.length, 0); + assert.match(hidden[0].why, /needs project/); +}); +``` + +Then add one line to `app/ui/modules.js`: + +```js +"./features/scene-count/index.js", +``` + +Run it with `cd app && npm test`. That is the whole job. + +## The hub + +A feature file's default export is one hub, or an array of hubs. + +| Field | Needed | Meaning | +|---|---|---| +| `id` | yes | Unique name, shown in the console when the feature is hidden. | +| `mount` | yes | The region to put it in. See the table below. | +| `setup({ host, app })` | yes | Builds the feature. `host` is the region's element (or, for `settings`, an object with `add`). May return a teardown function. | +| `needs` | no | Names that must exist on `app` (`["project", "api"]`). | +| `modules` | no | Backend module ids that must be installed (`["render", "continuity"]`). A button whose backend did not load would only fail when pressed. | + +The order in `modules.js` is the order features are set up, so a feature that reads something another registers (`app.sceneSections`, `app.mediaMenu`) should come after it. + +### When a feature is hidden + +Nothing of it reaches the screen. The reason is written once to the browser console (`[FunPack] <id> hidden: <why>`) and returned by the loader. A feature is hidden when: + +- its file does not load (syntax error, bad import); +- `mount` names a region nobody offers (the message lists what is offered); +- anything in `needs` is missing from `app`; +- anything in `modules` is not installed on the server (if the server cannot be asked, nothing is hidden for this reason); +- `setup` throws. Whatever it appended to `host` and whatever it subscribed to through `app.on` is removed again. + +The rest of the app loads normally in every case. This is the "hide, don't warn" rule at work: do not draw a disabled button with an apology. + +### What you must clean up yourself + +The loader only undoes `host` children and `app.on` subscriptions, and only when `setup` throws. Everything else is yours, in the function `setup` returns: + +- `document` and `window` listeners, timers, `ResizeObserver`s; +- entries you pushed onto `app` lists (see below), keys you bound, actions you offered; +- classes you put on `<html>`; +- overlays, popovers and modals you opened (`handle.destroy()` or `.close()`). + +Do the steps that can throw first and the registrations last, so a `setup` that fails halfway leaves nothing registered. + +## Regions (`mount`) + +The shell offers these. A region is announced by the code that creates it, so the list is whatever `shell/frame.js` and `shell/settings.js` offer. + +| `mount` | Where | +|---|---| +| `menubar.menus` | the menu bar, after the brand. File, Edit, View style menus go here. | +| `menubar.right` | the far end of the menu bar | +| `assets` zone: `assets.actions`, `assets.status`, `assets.projects`, `assets.media` | the Assets panel: header actions, footer, the project list, the media bin | +| `preview`, `preview.actions`, `preview.status` | the monitor panel: its body, header actions, footer | +| `inspector`, `inspector.actions`, `inspector.status` | the Properties panel | +| `timeline` | the timeline stage | +| `timeline.toolbar`, `timeline.toolbar.end` | the row over the stage: what acts on the cut, and zoom/hints | +| `timeline.actions`, `timeline.status` | the Timeline panel's header and footer | +| `settings` | the Settings window. Not a DOM region: `host.add(section)` | + +A body region (`preview`, `inspector`, `timeline`, the two `assets.*`) is a stack that takes children. Other regions are toolbars. + +### A Settings section + +```js +export default { + id: "my-settings", + mount: "settings", + needs: ["api"], + setup({ host, app }) { + host.add({ + id: "mine", group: "System", title: "My thing", subtitle: "One sentence.", + icon: "◎", tone: "neutral", keywords: "search words", + mount({ setFooter, close }) { return { node: someHandle.node, destroy() {} }; }, + }); + }, +}; +``` + +`group` is one of `""`, `"Generation"`, `"Learning"`, `"System"`, in that order. The window searches `title` and `keywords`. `mount` returns a handle (`node`, `destroy`); `setFooter({ actions, note })` puts buttons below the scrolling body. + +A small preference belongs on the **Editor** page instead: push a function that returns a kit handle onto `app.editorSettings` (see `features/autosave`). + +## What `app` offers + +`app` is built in `app/ui/boot.js`. Only a name listed in `needs` is guaranteed; check the table for what each is. Where a method is not listed here, read its file header: each one starts with a comment that says what it is for. + +| Name | What | File | +|---|---|---| +| `project` | The open project: `scenes`, `selectedId`, `edit(fn)`, `undo()`, `redo()`, `pref(key, fallback)`, `setPref(key, value)`, `flush()`, `unsaved`. Saved to the server. | `shell/project.js` | +| `selection` | Which scenes are picked: `ids`, `focus`, `pick`, `on(fn)` | `shell/selection.js` | +| `playhead` | `at` (seconds), `set(sec)`, `on(fn)`, `playing` | `shell/playhead.js` | +| `pipeline` | The live pipeline and module values: `slots()`, `activeModules()`, `currentValues()`, `setModuleValue(id, key, value)`, `usefulness()`, `subscribe(fn)`, `ensureLoaded()`, `allOff()` | `shell/pipeline_state.js` | +| `generate`, `runner` | Starting runs. `runner.units(ids, times)` is set by the Generate feature. | `shell/generate.js` | +| `api` | Every server call, one method each. Use it; do not call `fetch`. | `shell/api.js` | +| `keys` | `bind("mod+z", fn)` returns an unbind. Return `false` to let the key through. Typing in a field is never intercepted. | `shell/keys.js` | +| `dock` | Which of Assets, Preview, Properties show | `shell/dock.js` | +| `theme` | `window.ComposerTheme` (`apply`, `get`, `resolved`) | `shell/theme.js` | +| `maintenance` | Update, switch branch, restart. Ends in a page reload. | `shell/maintenance.js` | +| `on(fn)`, `say(name)` | The app's event line. `fn` hears `"open"` (another project was opened), `"change"` (an edit), `"select"`, and named events other features `say` (`"zoom.in"`, `"play.toggle"`, `"timeline.lanes"`, `"media.peek"`). It returns an unsubscribe. | `boot.js` | +| `hasModule(id)` | Whether the server has that backend module. `true` for everything if the server is unreachable. | `boot.js` | +| `openSettings(id)` | Opens Settings at a section | `shell/settings.js` | +| `title(mount, text)` | Sets a zone's title (`mount` is the zone's name: `assets`, `preview`, `inspector`, `timeline`) | `shell/frame.js` | +| `resetLayout()` | Restores the default layout | `boot.js` | + +A feature never imports another feature. It reaches the others through `app`, and the rest of the app is reached through a service. (Two existing imports break this: `sampler` uses `settings/field.js` and `settings/models.js`, and `settings/modules.js` uses `addons`. Do not add more: move the shared helper into `shell/` instead.) + +### Lists other features read + +A feature that adds to the app pushes onto one of these lists and removes its entry in teardown. The reader wraps every entry in a `try`, so one broken entry does not stop the rest. + +| List | An entry is | Read by | +|---|---|---| +| `app.editorSettings` | `() => handle` for the Editor settings page | `settings/editor.js` | +| `app.timelineLanes` | `{ owns(id), lanes(project), select/move/trim(id, ...) }`; `app.say("timeline.lanes")` after a change | `timeline` | +| `app.mediaMenu` | `(item, raw) => [{ id, label, run }]` entries for a media tile's menu | `media` | +| `app.mediaPeek` | an object: `media` sets `item` and says `"media.peek"`, `preview/peek.js` opens it | `preview` | +| `app.sceneSections` | `{ rows(scene, project), key(scene, project) }` rows under a scene's own in Properties | `inspector` | +| `app.clipTags` | `(scene) => tags` shown on a clip | `timeline` | +| `app.actions` | `{ id, label, icon, run, when?, optIn? }`. Add with `offer(app, action)` from `shell/actions.js`; it returns the remover. The wheel and menus list them. | `wheel` | +| `app.inputHooks` | `async ({ project, scene, slots }) => ({ inputs, notes })`: put values into the run's inputs; `notes` are shown | `generate` | +| `app.promptPrefix` | `(scene, project) => [text, ...]` lines put before the scene's prompt at run time (never written into the prompt box) | `generate`, `composer` | +| `app.beforeRun` | `async () => {}` awaited before a run is built, so an edit still on its way lands first | `generate` | + +To add a new list, add it in `boot.js` and name it in this table. Keep the entry shape small. + +## Pairing with a backend module + +A feature that exists only because a backend module exists names it: `modules: ["continuity"]`. If the module is not installed or failed to load, the feature is not drawn. Pair the two ids in a test the way `core/tests/test_ui_module_ids.py` does: it fails the build when a feature names a module id that does not exist. + +When the backend module offers routes, call them through `app.api` (add the method in `shell/api.js`, and test it in `shell/tests/api.test.js`). The path is `/funpack/api/m/<module id>/...`. + +## Building the screen + +- **Use kit elements.** `composer.button.md({ label, onClick })`. Every element returns a handle `{ node, destroy, ...setters }`. Put `handle.node` in the page and call `handle.destroy()` when done. Containers take **handles**, not nodes. +- **Content is text.** Pass strings. The kit sets them as text, never as markup, and throws on anything else. +- **Do not style.** No `style`, no new classes, no colours or pixel sizes. If the kit lacks what you need, add an element to the kit (see [UI kit reference](ui-kit-reference.md#adding-an-element)) and use that. +- **A redraw replaces content, not controls.** `region.stack` has `set(children)`; it keeps the caret in a field being typed in. Do not redraw while a field has focus (the Engine page skips redraws while typing). +- **The screen is the truth.** Generate reads live UI state and never waits for saves. Autosave must not disturb typing. Do not cache generated content across requests. +- **Per-browser preferences** (a collapsed section, the dock) go in `localStorage`, wrapped in `try`/`catch`, and the feature must work when storage throws. Anything that should follow a project goes in `app.project.setPref`. + +### Elements that are not in the kit + +If you ask for one that does not exist (`composer.hologram`), it throws `UnknownElement` naming what does exist. In `setup` that makes the feature hidden, which is the intended result: better nothing than something that looks like it works. + +## Tests + +Put `hub.test.js` (or `<name>.test.js`) beside the code. `npm test` finds every `*.test.js` under `app/`. + +- Use jsdom through `composer/tests/_dom.js` (`setupDom`, `teardownDom`, `fire`, `key`). jsdom has no layout, so anything that depends on measured size belongs in the browser, not here. +- Load the feature through `loadFeatures` with a fake `app` that has only what `needs` names. That tests the real hiding rules too. +- Test teardown: nothing left in `host`, no listener left in `app`. +- Use real objects (a real `project` store, a real `composer`) over hand-made stand-ins where you can. + +## Before you open a pull request + +- [ ] One folder under `app/ui/features/`, one line in `app/ui/modules.js`. +- [ ] `needs` and `modules` name everything it depends on. +- [ ] Built from kit elements only. No style, no raw HTML, no colour, no pixel size. +- [ ] Teardown removes every listener, timer, list entry and class. +- [ ] A control that changes the output is visible and switchable in the app in the same change. +- [ ] Works in light and dark, and with keyboard only. +- [ ] A test for the hub: mounts, follows changes, leaves nothing behind, hides when a need is missing. +- [ ] `cd app && npm test` passes.