malkuthroandClaude Opus 4.7
9f063345f7
* feat(investigations): scaffold #177 LTX23 audio-file lip-sync investigation Reorganize docs/investigations/ into numbered topic folders, each self-contained with narrative + upstream pin + patches + per-run snapshots. Add a generic sync helper that mirrors the dev-sync pattern for any investigation's third-party node modifications. Structure (new): docs/investigations/ 00_LTX23-base-1-step/README.md (rename of ltx-director-4k-transitions.md) 01_LTX23-audio-file-lipsync/ README.md -- problem, mechanism, patches, reversion, promotion upstream.json -- repo + version + pinned commit + sync target env + files patches/ -- source of truth for modified upstream files ltx_director.py -- adds `relay_overrides` multiline JSON input prompt_relay.py -- per-segment sigma per Prompt-Relay paper formula runs/ LOOP.md -- iteration protocol log.md -- rolling table (empty header for now) Sync helper: scripts/sync_investigation_patches.py <NN>_<topic> [--dry-run] - Reads upstream.json for target env var + file list - Backs up the target's current state once per day (<file>.bak.<YYYYMMDD>) - Worktree-aware: falls back to MAIN's .env via git rev-parse --git-common-dir - Prints a two-line report matching scripts/sync_to_dev.py shape - Per-investigation trigger phrase via `trigger:` key (sync-audio for #177) Env: .env.example -- documents new KOLOOK_WHATDREAMSCOST_PATH .env -- local path set Glossary updates (docs/reference/glossary.md): + Investigation folder + JSON file folder + Loop + Run + Investigation patches + sync_investigation_patches.py Closes the workflow gap from #177: MAIN is now the source of truth for investigation patches; the live install is a sync destination only. No manual file copying between iterations — `sync-audio` does it. * chore(ruff): exclude docs/investigations from lint The investigation patches under docs/investigations/*/patches/ are vendored upstream source (WhatDreamsCost-ComfyUI v1.3.2) carried so sync_investigation_patches.py can mirror them into the live install and so iterations can be diff'd against the pinned upstream commit. Linting them would force divergence from upstream and defeat that purpose. Mirrors the existing forks/ exclude precedent. Fixes the CI Lint & Smoke failure on PR #178. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: per-task automation modules + WhatDreamsCost Koolook fork Pivot docs/automations/ from "one folder per model" to "one folder per automation module" so the same diffusion model can host several independent tasks side by side. Replace the parallel docs/investigations/ tree with a fork of WhatDreamsCost-ComfyUI under forks/. Automation modules (new layout) - docs/automations/LTX-2.3/base-1step/ the existing single-stage automation, moved from docs/automations/LTX-2.3/ with its README, findings, and handoff-checklist rescoped to the task. The 4k-transitions backstory that produced the module moves here as backstory/4k-transitions.md. The earlier docs/investigations/experiments/ card PNGs/HTML land in example-cards/. - docs/automations/LTX-2.3/audio-lipsync/ the audio-file lip-sync module previously living under docs/investigations/01_…/. README, handoff, findings stub, backstory/, and runs/{LOOP,log}.md, all referencing the new fork. - docs/automations/README.md + CHEATSHEET.md updated to describe the module-per-task pivot and the new tree shape. - docs/automations/CONVENTIONS.md unchanged (cross-cutting loop contract still holds). WhatDreamsCost Koolook fork (new) - forks/whatdreamscost_koolook/versions/v1_3_2/ UPSTREAM_PIN.yaml pinned to upstream e81223a (v1.3.2), GPL-3.0 verified 2026-05-28. ltx_director.py MODIFIED — adds the relay_overrides multiline JSON widget on LTXDirector. Moved from docs/investigations/01_…/patches/. prompt_relay.py MODIFIED — per-segment σ per the Prompt-Relay paper formula σ = (L − w_eff) / (2 · √ln(1/ε)). Moved from docs/investigations/01_…/patches/. patches.py UNMODIFIED — vendored verbatim from upstream because ltx_director.py imports detect_model_type and apply_patches from it (same precedent as radiance_koolook's color_helpers.py). - Registered as LTXDirector__koolook_v1_3_2 ("LTX Director (Koolook v1.3.2)") so it coexists with an installed copy of upstream WhatDreamsCost-ComfyUI. Wired into the root __init__.py NODE_CLASS_MAPPINGS merge. - forks/forks_manifest.yaml + forks/THIRD_PARTY.md updated with the new entry (attribution, change log, license verified date, exposed node IDs). Removed - docs/investigations/ entire tree retired; its contents either moved to automation modules or to forks/. - scripts/sync_investigation_patches.py no longer needed — fork code lives in the Koolook tree and is synced via the standard dev-sync flow. - KOLOOK_WHATDREAMSCOST_PATH in .env.example same reason. - docs/investigations in pyproject.toml's ruff extend-exclude added in96bf8adto silence lint on vendored patches; no longer needed now that vendored code is under forks/ (already excluded). Docs - docs/reference/glossary.md Investigation folder, Investigation patches, JSON file folder, and the sync_investigation_patches.py entries replaced by Automation module, Backstory, Working folder, and Card. Loop and Run rewritten to point at the new structure. - CHANGELOG.md [Unreleased] describes the restructure + the new fork. Verification - ruff check . → all checks passed - tools/preflight_release.py --skip manager-meta → 42 files parsed; 11 unique node IDs collected including LTXDirector; VAE dispatch branches all OK; fixture workflows still reference all registered Koolook IDs. - pytest → 173 passed, 0 failed. Workflow JSON heads-up The active audio-lipsync workflow LTX-23-audio_tests_v01.json continues to reference the upstream LTXDirector ID. To pick up the new fork's behaviour, swap that node to "LTX Director (Koolook v1.3.2)" (registered ID LTXDirector__koolook_v1_3_2) — sockets and wiring are preserved. The relay_overrides widget appears at the bottom of the new node. Documented in the audio-lipsync README's "Workflow JSON — Koolook node ID" section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(scripts): add dev-sync-audio — scoped sync for audio-lipsync iteration `dev-sync-audio` is a scoped wrapper around `dev-sync` for the [`docs/automations/LTX-2.3/audio-lipsync/`](docs/automations/LTX-2.3/audio-lipsync/) iteration loop. It copies *only* the paths that module touches — the `forks/whatdreamscost_koolook/` subtree and the root `__init__.py` — instead of `rmtree`-ing the whole dest `forks/` and re-copying every Koolook fork on every render. Same `KOLOOK_COMFYUI_DEV_PATH` target, same ComfyUI-Manager restart, same two-line chat-report shape. Changes - `scripts/sync_to_dev_audio.py` (new). Thin wrapper that reuses `sync_to_dev`'s env loader, target validator, restart helper, and chat-report builder, supplying its own `AUDIO_PATHS` tuple. Defaults the `--scope` tag to `"audio-lipsync fork edit"` so the build identifier names this module even when invoked without an explicit scope. - `scripts/sync_to_dev.py` — `sync()` gains an optional `paths` kwarg (defaults to the existing `RUNTIME_PATHS`). Backwards-compatible with every existing call site. - `docs/automations/LTX-2.3/audio-lipsync/README.md` and `runs/LOOP.md` updated to point at `dev-sync-audio` instead of plain `dev-sync` for the iteration step. - Project `CLAUDE.md` — new "Per-module scoped variants" subsection under the existing `dev-sync` section documents the `dev-sync-<module>` pattern, the trigger-name → script-name mapping, and reiterates that the user-initiated-only rule applies verbatim to every `dev-sync-*` variant. Verification - ruff check . → all passed - python scripts/sync_to_dev_audio.py --dry-run -v → would copy: __init__.py would copy: forks/whatdreamscost_koolooke7f3ab4- beautiful-mahavira-ae6da6 would sync 2 entries -> <KOLOOK_COMFYUI_DEV_PATH> (dev-sync-audio) (Confirms scope is correct and the existing dev-sync chat-report shape is preserved.) User-initiated only — never automatic. Trigger phrase: `dev-sync-audio` in chat (also: "copy audio fork", "sync audio fork"). Same rule as `dev-sync` — see project CLAUDE.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(audio-lipsync): only snapshot runs when explicitly kept Every render adds a row to runs/log.md (always). The heavier run-NNN_<label>/ snapshot folders (workflow JSON copy + relay_overrides + patch state + notes) are now created only when the maintainer says "keep", "save run", "save this", or "snapshot" in chat. Without that trigger, the render is just a log row. Why: an active knob sweep can do 10+ renders/hour. Saving a folder per render bloats runs/ fast and most of those folders are noise the moment the next render lands. Cheap log row everywhere, expensive snapshot only for runs worth pinning (knob landed, surprising failure, something heading for findings.md). Changes - docs/automations/LTX-2.3/audio-lipsync/runs/LOOP.md Step 5 split into "always log" + "snapshot only on request". Trigger-detection table updated. Run-labels section reframed (NNN matches log.md row number). Added a one-paragraph "why this retention policy" rationale. - docs/automations/LTX-2.3/audio-lipsync/README.md Folder-map flags run-NNN_<label>/ as conditional. Iteration loop step 5 names the keep / save-run / save-this triggers explicitly. Not a code change. No script edits — the retention rule lives in the agent protocol, not in dev-sync-audio (which stays scoped to code sync, not run management). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(review): harden relay_overrides parser + polish (PR #178 review) Addresses verified findings from the /review-pr pass on PR #178. H1 — relay_overrides values are now type-checked - Allowlist of recognised knobs (`video_strength`, `video_window_scale`, `audio_strength`, `audio_window_scale`, `audio_epsilon`) with explicit float-coercion. Numeric strings ("10") coerce silently; non-numeric values (e.g. "ten") now raise a ValueError naming the offending knob and showing a minimal correct example — instead of a TypeError deep inside `build_segments` arithmetic, or a SUCCESS render at upstream defaults that polluted the A/B iteration log. H2 — malformed JSON no longer silently falls back to defaults - `json.JSONDecodeError`, non-dict JSON (arrays, strings, bare numbers, null), and non-numeric values for recognised knobs all now raise `ValueError` with operator-actionable messages. The only silent fallback is the explicit empty/whitespace case — "use upstream defaults" is the operator's intent there. - Typo'd keys (e.g. `viedo_strength`) log a warning naming the unknown keys + the recognised set, then drop the typo. Previously they were silently retained in the cleaned dict but never queried by `build_segments`, indistinguishable from a default render. Architectural side-effect — shrinks the diff vs vendored upstream - `_parse_relay_overrides` extracted into a stdlib-only sibling module `_relay_overrides.py`. `ltx_director.py` shrinks closer to upstream baseline (one import + one alias instead of a 30-line function body) and is easier to diff against `e81223a`. The parser is also now unit-testable without ComfyUI's `comfy_api` runtime. L5 — tests added (31 cases, all passing) - New `tests/forks/__init__.py` + `tests/forks/test_whatdreamscost_v1_3_2_relay_overrides.py`. - Covers every confirmed failure mode (H1 + H2), the happy path, the comment-key convention, the unknown-key warning, and an allowlist- consistency assertion that fails loudly if `build_segments` ever grows a knob without the parser being updated. - Test module uses `importlib.util.spec_from_file_location` so it can load the parser without walking through the package hierarchy (which would trigger the `comfy_api` import inside `ltx_director.py`). L3 — `_build_line` promoted to public `build_line` - `scripts/sync_to_dev.py` renames the private helper. The underscore alias is kept for one release cycle as a deprecation slot. - `scripts/sync_to_dev_audio.py` switches to `_dev.build_line()`. No behaviour change; chat-report shape is bit-identical to before. L2 — `source_ref` clarified in pin metadata - `forks/whatdreamscost_koolook/versions/v1_3_2/UPSTREAM_PIN.yaml` and `forks/forks_manifest.yaml` both now spell out that `e81223a` is the post-v1.3.2 README-only commit on upstream's `main`, not the v1.3.2 release tag itself. Code is byte-identical to the tagged release; the pin differs only in trailing docs. L1, L4 are explicitly left alone (L1 is dead upstream cruft, kept for minimum-diff; L4 is helper duplication tracked for a separate follow-up issue when a third fork lands). Verification - ruff check . → clean - pytest → 204 passed (173 prior + 31 new parser tests) - tools/preflight_release.py --skip manager-meta → all PASS (static AST extraction still picks up LTXDirector from the fork) - scripts/sync_to_dev_audio.py --dry-run → two-line chat report still emits via the renamed `build_line` helper Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(audio-lipsync): config-driven loop-audio script + module cheat sheet End-to-end automation for one render cycle of the audio-lipsync iteration loop. The chat phrase `loop-audio` invokes a single script that does the whole chain — no manual artifact-shuffling. Scripts (cross-cutting infrastructure at the repo root, like make_card and sync_to_dev; they get lifted into the module folder in the `docs/automations/` → `automations/` move queued for the follow-up PR) - scripts/loop_audio.py Reads its settings from loop_audio.config.json (alongside), auto-discovers the most recent workflow JSON in the ComfyUI user workflows dir (derived from KOLOOK_COMFYUI_DEV_PATH or overridden via KOLOOK_COMFYUI_WORKFLOWS_DIR), extracts the five tracked Text Multilines + LTX Director state + sampler chain, captures fork-state (HEAD SHA + last dev-sync-audio sync info + fork-dir git status), writes the run-NNN_<auto-label>/ snapshot, appends a row to runs/log.md, and chains the card renderer in. Flags mirror sync_to_dev: --dry-run, --label, --no-log, --no-card, --workflow, --config. - scripts/loop_audio.config.json Single source of truth for the customisable settings (job_name, module_path, workflow_pattern, comfyui_workflows_subpath, skip_filename_substring, tracked_multilines, fork_to_track, render_card). Underscore-prefixed keys are inline documentation and ignored by the loader. When the convention shifts — new multiline title, new fork path, different filename pattern — edit the JSON, not the Python. - scripts/make_card_audio.py Audio-lipsync-specific card renderer. Five sections (KNOB STATE, FORK STATE, SAMPLER, BASE notes, OUTCOME) plus an INERT warning band when the workflow's Director is upstream `LTXDirector` with a non-empty relay_overrides. Same palette + font fallback chain as scripts/make_card.py so the base-1step and audio-lipsync card families read as a set. PIL/Pillow added to the `test` optional deps so CI doesn't hit ImportError if any future test touches this script. Docs - docs/automations/LTX-2.3/audio-lipsync/CHEATSHEET.md Module-scoped operating instructions per the cheat-sheet pattern: what the module is for, the chat triggers, the required workflow shape (5 multilines + the Koolook Director variant), the render cycle steps, what lands in each run-NNN_ folder, findings- promotion rule, where every customisable setting lives, and a card-section legend. - docs/reference/glossary.md `Card` rewritten to describe the per-module renderer fan-out (`make_card.py` for base-1step, `make_card_audio.py` for audio-lipsync) — cards no longer live at a single stable `_AI/card.png` path. New `Loop config (per-module)` entry documenting the `<script>.config.json` convention + the required-keys list. - docs/automations/LTX-2.3/audio-lipsync/runs/log.md Header rewritten with column set matching the actual log-row output (Director / relay_overrides / Custom audio / Phase 1 / Phase 2 / Scores / Notes). First real run captured - docs/automations/LTX-2.3/audio-lipsync/runs/run-001_bear_3x-fr_w.audio_upstream_audio-off_vstr10.0/ Baseline reference render — `Bear_3x-FR_w.Audio`, upstream `LTXDirector` still wired into the workflow, use_custom_audio off. Motion 4/5 · Sync 5/5 · Sharp 4/5 per maintainer feedback. The relay_overrides `{"video_strength": 10.0}` is set but inert (Director not yet swapped to the Koolook variant); the card surfaces this with the amber INERT band at the bottom. Captures: workflow.json (185KB), relay_overrides.txt (with INERT note), patch_state.txt (3ffb7b7/15d5688/ clean), notes.md (feedback + scores + mechanical interp), card.png (61KB). Verification - ruff check . → clean - pytest -q → 204 passed, 0 failed - tools/preflight_release.py --skip manager-meta → all PASS - scripts/loop_audio.py --dry-run → identifies the latest workflow, emits the two-line chat-report header, doesn't touch disk - scripts/loop_audio.py (real) → wrote the run-001 folder + card + log row in a single invocation Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(audio-lipsync card): match base-1step visual style 1:1 The first cut of the audio-lipsync card used solid colored header bands and block section panels — visually heavier than base-1step's renderer. Refactored to copy make_card.py's primitives directly: - small uppercase accent-colored section labels (no header bars) - subtle BG_SECTION panel outlines instead of full-width fills - left-accented note boxes (4px coloured bar + dark BG + label inside) for Base · notes, Feedback (video), and the conditional Director note warning - same font fallback chain + sizes as make_card.py (F_SECTION 17pt, F_MONO 19pt, F_NOTE italic 18pt) - oversized canvas + post-render crop matches make_card.py's pattern so cards size to actual content - palette imported in lockstep (BG_OUTER/CARD/SECTION, BORDER, the three accent triples plus the new ACCENT_FORK violet) Two tuning fixes spotted during re-render: - _wrap_text gained a keep_blank_lines flag so prose blocks can collapse the empty separator lines maintainer canvases often have - max_lines bumped on the prose blocks: Base · notes 18 (was 6, truncated the OVERLAY - INFO body), Feedback 6 (was 5), Director note 8 (was 4, truncated mid-sentence) - make_card_audio.py's standalone CLI now loads .env via loop_audio.load_dotenv before rebuilding state, so re-renders show the same FORK STATE lines (MAIN sha + sync sha + scope) that the live loop_audio run captures Visual reference: docs/automations/LTX-2.3/base-1step/example-cards/ run-001-card.png is the layout target. The audio-lipsync card now reads as a sibling of that family — KNOB STATE in amber, FORK STATE in violet, SAMPLER + Base · notes in sky, FEEDBACK + OUTCOME in green. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(fork): ship LTX Director timeline-editor JS to the Koolook variant The Koolook variant of LTXDirector was loading as a bare schema — no image strip, no audio strip, no canvas timeline editor — because the upstream timeline-editor JS bundle never came across with the fork. The widget upstream paints onto the node lives entirely in `js/ltx_director.js`; the Python class only declares the inputs. Compounding it, the JS's binding check is hard-coded ``if (nodeData.name === "LTXDirector")``, so even if the bundle had been loaded, it would have skipped our suffixed registry ID (`LTXDirector__koolook_v1_3_2`). Fix: vendor the JS into the fork and rebind it. - web/whatdreamscost_koolook_v1_3_2/ltx_director.js Verbatim copy of upstream's custom_nodes/WhatDreamsCost-ComfyUI/js/ltx_director.js at the pinned commit (e81223a). Two string changes vs upstream: * `name: "LTXDirector"` -> `"LTXDirector_Koolook_v1_3_2"` so the registered ComfyUI extension doesn't collide with the upstream extension (both load in parallel). * `if (nodeData.name === "LTXDirector")` -> `if (nodeData.name === "LTXDirector__koolook_v1_3_2")` so the canvas widget mounts on the Koolook variant only; the upstream extension keeps owning the upstream node. A short comment block at the top documents both deltas. - scripts/sync_to_dev_audio.py AUDIO_PATHS gains `web/whatdreamscost_koolook_v1_3_2` so dev-sync-audio carries the JS into the live install alongside the fork's Python files + root __init__.py. After dev-sync-audio + ComfyUI restart, the Koolook variant mounts the same timeline editor as upstream: image/audio strips, segment dragging, the visual preview row, the [Audio] tag handling, the img_compression/divisible_by widgets. The `relay_overrides` field declared in the Python schema renders at the bottom of the node alongside everything else. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
170 lines
8.3 KiB
Markdown
170 lines
8.3 KiB
Markdown
# Glossary
|
|
|
|
Short, plain-English definitions for this repo workflow.
|
|
|
|
## Root `__init__.py`
|
|
- The ComfyUI entry file for this plugin.
|
|
- Its job is to import node mappings and expose `NODE_CLASS_MAPPINGS` / `NODE_DISPLAY_NAME_MAPPINGS`.
|
|
- Keep it clean and focused on loading nodes.
|
|
|
|
## Main Repo
|
|
- This repository: `ComfyUI-Koolook`.
|
|
- Stores Koolook-owned code, docs, and lightweight integration wrappers.
|
|
- Should not store full external third-party repositories.
|
|
|
|
## External Forks
|
|
- Raw upstream/fork reference repos kept outside MAIN.
|
|
- Default location is a sibling folder: `../ComfyUI-Forks`.
|
|
- Each external repo should be pinned to a version/tag + exact commit.
|
|
|
|
## Wrapper
|
|
- A small adapter module in `forks/` that loads locally tracked modified nodes and namespaces node IDs.
|
|
- Purpose: avoid collisions and keep external logic separate from root loader.
|
|
- For Radiance, the package entrypoint is `forks/radiance_koolook/__init__.py`.
|
|
|
|
## Version Folder
|
|
- A folder under `forks/radiance_koolook/versions/` that stores one maintained codebase snapshot (example: `v1_0_1`).
|
|
- Keep upstream parity metadata in `UPSTREAM_PIN.yaml` inside each version folder.
|
|
|
|
## Namespace Suffix
|
|
- A suffix added to internal node IDs (example: `__koolook_v1_0_1`).
|
|
- Prevents collisions between versions (v1, v2, upstream, etc.).
|
|
- Protects old workflows from breaking.
|
|
|
|
## `forks/forks_manifest.yaml`
|
|
- Machine-readable tracker for external dependencies.
|
|
- Contains source repo, source ref, pinned commit, and external relative path.
|
|
- This is the source of truth for reproducible setup on new machines.
|
|
|
|
## `forks/THIRD_PARTY.md`
|
|
- Human-readable attribution and notes.
|
|
- Explains what was modified and why.
|
|
|
|
## Portable Paths
|
|
- Prefer relative paths so setup works on Windows/macOS/Linux.
|
|
- Keep external references under `../ComfyUI-Forks` for portable machine setup.
|
|
|
|
## Pinned Commit
|
|
- The exact git commit hash you lock to for reproducibility.
|
|
- Use this when moving to another computer so behavior matches exactly.
|
|
|
|
## Sibling Project
|
|
- A related repository or folder that lives **outside MAIN** and is consulted (read-only) but never imported by MAIN at runtime.
|
|
- Examples: external forks root, ComfyUI knowledge database, personal knowledge database.
|
|
- Always referenced by env var (e.g. `KOLOOK_COMFYUI_KB_DIR`) — never by absolute path in committed files.
|
|
- Real paths live in `.env` (gitignored). The committed `.env.example` declares the variable names.
|
|
- Treated as runtime-optional: if the env var is unset and there is no portable default, the sibling is considered unavailable.
|
|
|
|
## `.env` / `.env.example`
|
|
- `.env.example` (committed) — public template listing the env var names with safe placeholder values.
|
|
- `.env` (gitignored) — each user's local file with real machine-specific paths.
|
|
- Pattern matches the existing `KOLOOK_FORKS_DIR` convention.
|
|
|
|
## EasyUse GET/SET Virtual Tunnel
|
|
- A pair of frontend-only EasyUse utility nodes: `easy setNode` stores a
|
|
named value source and `easy getNode` exposes that value elsewhere in the
|
|
graph by key.
|
|
- Render-time ComfyUI execution resolves the tunnel, but a browser-side
|
|
preview button that simply reads the GET node widget will see only the key
|
|
name (for example `OUT-folder`) rather than the actual upstream value.
|
|
- Preview code must follow `easy getNode` -> matching `easy setNode` -> set
|
|
input link -> source widget value to match render-time behavior.
|
|
- Debug by reading the installed EasyUse frontend source
|
|
(`comfyui-easy-use/web_version/v1/js/getset.js`) and simulating the relevant
|
|
LiteGraph shape locally when the live `app.graph` object is not exposed.
|
|
|
|
## Automation module
|
|
- A self-contained AI-managed ComfyUI iteration task. Lives under
|
|
`docs/automations/<grouping>/<task>/`, with the grouping typically being
|
|
the diffusion model (`LTX-2.3/`) and the task being a specific generation
|
|
goal (`base-1step/`, `audio-lipsync/`).
|
|
- Each module owns: a `README.md` (loop entry + workflow contract), a
|
|
`handoff-checklist.md` (5-minute bootstrap), a `findings.md` (locked-in
|
|
conclusions for that task), optionally a `backstory/` folder (the
|
|
narrative that produced the module), optionally a `runs/` folder for
|
|
in-repo run snapshots when the iteration touches code (not just widget
|
|
values).
|
|
- Modules are independent: own workflow JSON, own working folder, own
|
|
findings. The model grouping is only structural — it does not carry
|
|
shared docs unless duplication forces a future `_shared/` extraction.
|
|
- When a module needs to modify an upstream node's code, the modification
|
|
lives under `forks/<package>_koolook/` per the fork pattern (see
|
|
*Wrapper* and *Version Folder* above), and the module's README
|
|
cross-references the fork.
|
|
|
|
## Loop
|
|
- The save → render → feedback → snapshot cycle inside an automation
|
|
module.
|
|
- Step 1: maintainer edits a knob in ComfyUI (or in a `forks/` source
|
|
file) and **saves** the workflow.
|
|
- Step 2: maintainer queues the render.
|
|
- Step 3: maintainer reports the result in chat.
|
|
- Step 4: agent snapshots current state into the module's
|
|
`runs/run-NNN_<label>/` (when the module ships in-repo runs) or just
|
|
appends to the working-folder `_AI/iterations.md` (when card-and-log
|
|
is all that's needed).
|
|
- Module-specific protocol lives at `<module>/runs/LOOP.md` for the
|
|
fork-touching modules; lighter modules document the loop inline in
|
|
their `README.md`.
|
|
|
|
## Run
|
|
- One iteration of the loop.
|
|
- For modules that ship in-repo run snapshots, a `run-NNN_<short-knob-summary>_<short-result-tag>`
|
|
folder under `<module>/runs/` containing at minimum a `workflow.json`
|
|
copy and a `notes.md` (maintainer feedback + agent interpretation).
|
|
- For all modules, also a row in the per-project working-folder
|
|
`_AI/iterations.md` log (auto-rendered by `/make-card`).
|
|
|
|
## Backstory
|
|
- The running narrative that produced an automation module — the
|
|
problem, the mechanism, the hypotheses considered, the rationale for
|
|
why the module exists. Lives at `<module>/backstory/<topic>.md`.
|
|
- Distinct from `findings.md` (locked-in conclusions) and
|
|
`runs/log.md` (rolling table of renders). The backstory is a
|
|
reference for *why* and *how it got here*; findings are *what's
|
|
true now*.
|
|
|
|
## Working folder
|
|
- The outside-the-repo per-project folder pointed at by
|
|
`KOLOOK_AUTOMATIONS_WORK_DIR` in `.env`. Holds workflow JSON,
|
|
rendered video, and the agent-managed `_AI/` subfolder with
|
|
`card.png` + append-only `iterations.md`.
|
|
- One working folder per project. Multiple automation modules can
|
|
point at the same working folder if they share workflow files;
|
|
more commonly, each module's iteration uses its own working folder.
|
|
|
|
## Card
|
|
- A per-render tracking visual (PNG) rendered automatically at the end
|
|
of a loop iteration. Travels alongside the rendered video into the
|
|
NLE for side-by-side comparison.
|
|
- Each automation module ships its own card renderer in
|
|
[scripts/](../../scripts/) to highlight what matters for *that*
|
|
module's iteration:
|
|
- `make_card.py` → base-1step card (Phase 1 / Phase 2 / Base · model /
|
|
Base · locked / Base · scene / Outcome). Writes to
|
|
`<working folder>/_AI/card.png`.
|
|
- `make_card_audio.py` → audio-lipsync card (KNOB STATE / FORK STATE /
|
|
SAMPLER / BASE notes / OUTCOME, plus an INERT warning when the
|
|
Director is upstream). Writes inside the module's
|
|
`runs/run-NNN_<label>/card.png`.
|
|
- Palette + font fallback chain is shared (see
|
|
`scripts/make_card.py` and `scripts/make_card_audio.py`) so the two
|
|
card families read as a set.
|
|
|
|
## Loop config (per-module)
|
|
- A JSON file co-located with the module's loop script that holds the
|
|
customisable settings — workflow filename pattern, ComfyUI workflows
|
|
subpath, tracked multiline titles, fork dir to pin, whether to
|
|
render a card. Underscore-prefixed keys are ignored by the loader
|
|
(carry inline documentation).
|
|
- Convention: `<script-name>.config.json` next to the script. The
|
|
audio-lipsync loop config lives at
|
|
[scripts/loop_audio.config.json](../../scripts/loop_audio.config.json).
|
|
- Required keys (validated at script start): `job_name`, `module_path`,
|
|
`comfyui_workflows_subpath`, `workflow_pattern`,
|
|
`skip_filename_substring`, `tracked_multilines`, `fork_to_track`,
|
|
`render_card`.
|
|
- Per-module operating instructions live in the module's
|
|
`CHEATSHEET.md` (e.g.
|
|
[docs/automations/LTX-2.3/audio-lipsync/CHEATSHEET.md](../automations/LTX-2.3/audio-lipsync/CHEATSHEET.md)).
|