Files
malkuthro-ComfyUI-Koolook/scripts/watch_cards.py
T
malkuthroandClaude Opus 4.7 092c204060 feat(automations): LTX Director iteration-loop framework + 4K investigation docs (#159)
* docs(investigations): kickoff for LTX Director 4K transition breakdown

Captures the maintainer's problem statement (HD multi-keyframe works,
4K single-keyframe works, 4K multi-keyframe degenerates into dissolves
between keyframes) and an initial conclusion about the WhatDreamsCost
LTXDirector node's implementation.

Traces the lineage to the Prompt Relay paper (Chen, Huang, Liu —
arXiv:2604.10030) via Kijai's ComfyUI port, separates the three
mechanisms the node bundles (cross-attention penalty, guide-image
keyframes, audio/image preprocessing), and flags the candidates for
the 4K dissolve: model-side limit at LTX sampling time vs. node-side
length-independent σ in the penalty matrix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(investigations): round 2 — 2K vs 4K + two-stage upscale findings

Adds the maintainer's round-2 observations:
 - 2K renders behave much better than 4K (segments morph instead of
   dissolving) — model is closer to its training distribution.
 - The two-stage workflow's upscale stage (stage 2) is overwriting
   stage 1's audio/animation: lip sync degrades and segment routing
   weakens.

Captures the hypothesis space:
 - Why 2K > 4K: model prior (out of distribution at 4K) is dominant;
   token-grid scaling with a length-independent sigma is secondary.
 - Why stage 2 overwrites stage 1: KSampler denoise re-applies
   conditioning from scratch; if LTX Director isn't wired into
   stage 2 the Prompt Relay + audio_attn2 patches go missing;
   latent shape change between stages forces a penalty rebuild.

Includes a 3-option pros/cons table (4K direct vs 2K direct vs
2-stage) and 6 investigation directions ordered cheapest-first,
starting with auditing the stage-2 wiring.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(investigations): round 3 — both stages run LTX Director; new prime suspect

Maintainer's screenshot confirms both stages route through LTX Director,
so round 2's missing-patch hypothesis is ruled out. Audited the stock
LTX nodes used in stage 2 against ComfyUI source:

 - LTXVLatentUpsampler explicitly pop()s the noise_mask and runs a
   learned (EXPERIMENTAL) neural upscaler — not interpolation.
 - LTXVConcatAVLatent fills a missing noise_mask with ones_like
   (denoise fully) for the side that's missing.
 - LTXVCropGuides removes keyframe slots and clears keyframe_idxs.

Traced the audio noise_mask end-to-end through stage 2 and found:
audio waveform stays preserved (noise_mask=0), video keyframes stay
pinned, but every non-keyframe video frame goes into stage 2's
sampler at noise_mask=1 — i.e. fully regenerated at denoise=0.80.
That's where stage 1's audio-aligned mouth shapes live, and that's
what's getting thrown away.

Prime suspect: denoise=0.80 is too high for "preserve stage 1"
semantics — most LTX two-stage reference workflows use 0.30–0.40.
Secondary suspect: the LTXVLatentUpsampler [BETA] is a learned model
that can hallucinate spatially-dense detail (faces, mouths) before
stage 2 even starts denoising.

Includes a 4-step cheapest-first test plan starting with dropping
stage 2 denoise to 0.30.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(investigations): round 4 — scheduler-shape finding + tracking plan

Captures two outputs from the planning session so they're preserved
for the next pickup:

1. Promotes the maintainer's empirical finding to a locked-in
   baseline: `linear_quadratic` scheduler at 8 steps produces the
   right sigma curve shape (flat plateau near sigma=1.0 for ~5
   steps, then steep drop in the last 3) for both phases. Any
   other scheduler degrades fast motion and keyframe transitions.

2. Records the agreed tracking-system plan (BASELINE.md +
   log.md + per-run screenshot folder, with three 1-5 score axes
   matching the live trade-off: motion / audio sync / Phase-2
   sharpness). Not yet scaffolded — explicit "next step" callout
   for whoever picks this up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(investigations): /make-card skill + PNG card renderer toolkit

Per-iteration tracking card for the LTX Director 4K investigation.
Solves the documentation-drift problem we hit on v01->v02 (multiline
summary diverged from actual node values) by treating the saved
workflow JSON as the source of truth.

Adds:
 - scripts/make_card.py — reads a ComfyUI workflow JSON, paints a
   self-contained 540px PNG card via Pillow. Extracts Phase 1 run
   params (Format/Mult/Scheduler/Steps/Denoise), locked base
   (Sampler/CFG/Seed/epsilon/Audio src), scene (duration/segments
   with prompt/audio/keyframe checkboard), free-text BASE notes
   from `OVERLAY - INFO` Text Multiline, FEEDBACK from `OVERLAY -
   FEEDBACK` Text Multiline, and a POST-RENDER block that
   auto-computes render duration from JSON->newest-video mtime
   delta in the same folder. Layout mirrors the project's
   snapshot-dialogs design language (panelled sections, sky/amber/
   green accents matching docs/designs/snapshot-dialogs.html).
 - scripts/watch_cards.py — background poller. Watches a working
   folder; auto-renders <stem>_card.png whenever a workflow JSON
   appears or is updated. Skips up-to-date PNGs.
 - .claude/skills/make-card/SKILL.md — /make-card slash command;
   globs the newest JSON in the working folder, calls make_card.py,
   shows the PNG inline.
 - .claude/skills/make-card/work-folder.txt — one-line config of
   the working folder path. Edit to switch projects.
 - docs/investigations/experiments/{baseline,combined-card,run-001-card}.html
   — design prototypes that fed into the PNG layout; kept as
   history.
 - docs/investigations/experiments/run-001-card.png and
   run-v04-card.png — sample card outputs from v03 and v04 of the
   workflow JSON, included for reference.

Issue #151 tracks the eventual native ComfyUI custom node so this
flow happens inside the graph; for now the external script + watcher
is the loop.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(launch): use absolute Windows python path for design-server

The default `python3` runtimeExecutable fails on this Windows host
(triggers the Microsoft Store install stub). Hard-coding
C:\Python313\python.exe lets the preview MCP and the design-server
target work on the maintainer's machine. Worktree-scoped change —
main keeps the cross-platform `python3` default.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(automations): docs/loop scaffold + env-var working folder + _AI subdir

Promotes the LTX Director card-rendering loop from one-off scripts to a
documented, project-portable automation framework.

Adds `docs/automations/` as the home for AI-assisted iteration loops:
 - README.md — the pattern; how to start a loop for a new model
 - CONVENTIONS.md — cross-cutting: working folder, Text Multiline node
   contract (Working_Folder_PATH / OVERLAY - INFO / OVERLAY - FEEDBACK),
   card format, iterations log, skill, watcher, run-id rules, tool
   locations
 - LTX-2.3/README.md — entry point for the LTX 2.3 loop; per-iteration
   steps; full extraction table; active investigation status
 - LTX-2.3/handoff-checklist.md — 5-minute onboarding for a new machine
   or new person
 - LTX-2.3/findings.md — promoted findings (linear_quadratic scheduler
   locked-in; resolution behaviour; audio mode; open two-stage suspects)

Replaces the committed `.claude/skills/make-card/work-folder.txt` with
an env-var-driven path. Follows the project's existing `KOLOOK_*`
convention — `.env.example` documents the new
`KOLOOK_AUTOMATIONS_WORK_DIR`. Real path lives in `.env` (gitignored).
No more leaking client-specific absolute paths into committed files.

`scripts/make_card.py`:
 - Loads `.env` automatically; worktree-aware (walks the .git common-dir
   marker to find the main repo when run from a worktree).
 - Output folder priority: Working_Folder_PATH node in JSON → env var →
   JSON's parent dir.
 - Run with no args: auto-discovers newest *.json in the working folder.
 - Writes `_AI/card.png` + `_AI/iterations.md` inside the working folder,
   not the root. Lets the maintainer wipe AI artifacts or user content
   (JSONs/MP4s) independently.

`/make-card` skill simplified to a single command — `python
scripts/make_card.py` — the script handles env loading and discovery
itself.

baseline.html: stripped the project-specific label.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(automations): sync all path references to _AI/ + add layout diagram

Audit pass after introducing _AI/ subfolder. Updated every doc and
script that still referenced root-level card.png / iterations.md:

 - docs/automations/CONVENTIONS.md (3 spots)
 - docs/automations/LTX-2.3/README.md (per-iteration steps + composite)
 - docs/automations/LTX-2.3/handoff-checklist.md (4 spots, including
   the off-boarding section which conflated repo commits with working-
   folder artifacts)
 - docs/automations/LTX-2.3/findings.md (promotion criterion)
 - scripts/watch_cards.py (target PNG path now matches make_card.py's
   _AI/card.png — was writing to a stale <stem>_card.png location)

Adds a clear "Where things live" section to docs/automations/README.md
showing both layouts side-by-side:
 - the repo tree (where the tooling, skill, env-example, and per-model
   loop docs sit)
 - the per-project working-folder tree (user content at root, _AI/
   subfolder for managed artifacts)

So anyone picking this up can answer "what's in the repo vs. what's in
my project folder" without reading prose.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(automations): add CHEATSHEET.md — one-page dense reference

Compact reminder card for returning users. Structured by application
context: ComfyUI · Scripts · Folder structure · Conventions · Quick-fail
diagnostics. Skippable for first-timers; targets the maintainer who
already knows the loop and just needs the right node names, score
syntax, and command list at a glance.

Linked from docs/automations/README.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(automations): add Bootstrap section to CHEATSHEET — bare-bones setup

What's needed to get a fresh machine running the loop, separated from
the daily-loop reference. Three checklists:
 - Repo side: Python + Pillow + .env config
 - ComfyUI side: was-node-suite + the three required Text Multiline
   nodes, with exact titles
 - First card: /make-card → expect _AI/card.png

Deliberately skips model-specific setup (checkpoints, LoRAs, per-model
custom nodes) — those live in each model's handoff-checklist.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(automations): skip *loop* files in auto-discovery + watcher

The maintainer's workflow includes a "loop" stage in ComfyUI that
composites the card next to the video and writes the result back into
the working folder as `<base>_loop_<seq>.{mp4,json}`. Without a skip
rule, auto-discovery treats this composited-card output as the newest
input — the script then re-renders the card from yesterday's metadata.

Now: any JSON or video file with `loop` in its stem (case-insensitive)
is ignored by:
 - scripts/make_card.py — auto-discover newest JSON, newest video
 - scripts/watch_cards.py — polling glob

Explicit invocation still works: `python scripts/make_card.py
<wf_loop.json>` will render a card for that file specifically.

Documents the rule in docs/automations/CONVENTIONS.md (new §7) and adds
a one-line bullet to CHEATSHEET conventions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style(automations): satisfy Ruff lint — split if-on-one-line, rename l, drop os

CI's Lint & Smoke step flagged 17 violations across the new scripts.
Cleaned up:

 - scripts/make_card.py: 11 × E701 (multi-statement lines)
   split every `if cond: stmt` onto two lines.
 - scripts/make_card.py: 5 × E741 (ambiguous variable `l`)
   renamed to `link` in the two link-dict comprehensions and their
   downstream lookups.
 - scripts/watch_cards.py: 1 × F401 (unused `os` import).

No functional change — `make_card.py` re-renders the working folder's
card identically post-cleanup. Verified `ruff check` returns
"All checks passed!" on both files.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(automations): render time from ComfyUI log, not disk timestamps

Disk mtimes don't reliably bracket the model run — the maintainer's
workflow saves the workflow JSON manually after rendering, assembles
QuickTime from EXRs separately, and ComfyUI's metadata bundle writes
JSON + MP4 at the same instant. None of those give us actual render
duration.

Render time now sourced from ComfyUI's own log
(`<comfyui-root>/user/comfyui.log`):

 - Find the latest `saving images: 100%` line → pair it with the next
   `Prompt executed in X` → that's the most recent EXR-producing render
   wall-time.
 - Supports both `Prompt executed in 18.21 seconds` and
   `Prompt executed in HH:MM:SS` log formats.
 - Skips trivial follow-up prompts (e.g. audio-VAE load between render
   and /make-card invocation).

Log path resolution:
 1. `KOLOOK_COMFYUI_LOG` env var (explicit override).
 2. Inferred from existing `KOLOOK_COMFYUI_DEV_PATH` →
    `<comfyui-root>/user/comfyui.log`.

Fallbacks remain in priority order: EXR sequence span (oldest to newest
in the latest version subfolder) → MP4 mtime delta → "pending".

Docs:
 - .env.example: documents the optional KOLOOK_COMFYUI_LOG.
 - CONVENTIONS.md: new §4 "Render duration" describing the resolution
   chain; section numbers renumbered.
 - CHEATSHEET.md: updated the "Render time —" diagnostic.

Lint: ruff clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 22:44:52 +02:00

96 lines
3.5 KiB
Python

"""Watch the make-card working folder; auto-render a card whenever a JSON
is added or saved.
Usage:
python scripts/watch_cards.py # uses .claude/skills/make-card/work-folder.txt
python scripts/watch_cards.py <folder> # override
python scripts/watch_cards.py --interval 1 # poll every 1 s (default 2)
Run it once in a terminal at the start of a session and forget about it.
Each time ComfyUI (or you) drops a new workflow JSON into the folder, a
matching `<stem>_card.png` is rendered next to it within a couple of
seconds. Skips JSONs whose card is already up to date.
Stop with Ctrl+C.
"""
from __future__ import annotations
import sys
import time
import subprocess
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
MAKE_CARD = ROOT / "scripts" / "make_card.py"
DEFAULT_CFG = ROOT / ".claude" / "skills" / "make-card" / "work-folder.txt"
PYTHON_EXE = sys.executable or "python"
def resolve_folder(argv: list[str]) -> Path:
args = [a for a in argv if not a.startswith("-")]
if args:
return Path(args[0]).expanduser().resolve()
if DEFAULT_CFG.exists():
return Path(DEFAULT_CFG.read_text(encoding="utf-8").strip()).resolve()
raise SystemExit(f"No folder argument and no config at {DEFAULT_CFG}")
def needs_render(json_path: Path) -> Path | None:
"""Return the target PNG path if it's missing or older than the JSON.
Target lives in <working folder>/_AI/card.png — matches make_card.py."""
ai_dir = json_path.parent / "_AI"
png = ai_dir / "card.png"
if not png.exists() or png.stat().st_mtime < json_path.stat().st_mtime:
return png
return None
def render(json_path: Path, png_path: Path) -> bool:
print(f"[{time.strftime('%H:%M:%S')}] rendering {json_path.name} ...", flush=True)
# Let make_card.py compute its own output path (writes to _AI/card.png).
# Passing only the JSON keeps the watcher and the script in lockstep.
r = subprocess.run(
[PYTHON_EXE, str(MAKE_CARD), str(json_path)],
capture_output=True, text=True,
)
if r.returncode == 0:
print(f" -> {png_path.relative_to(png_path.parent.parent)}", flush=True)
return True
print(f" !! make_card.py failed (rc={r.returncode})\n{r.stderr}", flush=True)
return False
def main():
interval = 2.0
if "--interval" in sys.argv:
i = sys.argv.index("--interval")
interval = float(sys.argv[i + 1])
folder = resolve_folder(sys.argv[1:])
if not folder.is_dir():
raise SystemExit(f"Folder does not exist: {folder}")
print(f"watching {folder} (every {interval}s, Ctrl+C to stop)", flush=True)
seen_mtimes: dict[Path, float] = {}
# Skip post-loop outputs — the user's composited card+video saves a
# JSON with "loop" in its name; treating it as input would feed the
# script its own output.
SKIP_MARKER = "loop"
try:
while True:
for json_path in folder.glob("*.json"):
if SKIP_MARKER in json_path.stem.lower():
continue
mtime = json_path.stat().st_mtime
if seen_mtimes.get(json_path) == mtime:
continue
target = needs_render(json_path)
if target is not None:
render(json_path, target)
seen_mtimes[json_path] = mtime
time.sleep(interval)
except KeyboardInterrupt:
print("\nstopped.", flush=True)
if __name__ == "__main__":
main()