Files
malkuthroandClaude Opus 4.7 092c204060 feat(automations): LTX Director iteration-loop framework + 4K investigation docs (#159)
* docs(investigations): kickoff for LTX Director 4K transition breakdown

Captures the maintainer's problem statement (HD multi-keyframe works,
4K single-keyframe works, 4K multi-keyframe degenerates into dissolves
between keyframes) and an initial conclusion about the WhatDreamsCost
LTXDirector node's implementation.

Traces the lineage to the Prompt Relay paper (Chen, Huang, Liu —
arXiv:2604.10030) via Kijai's ComfyUI port, separates the three
mechanisms the node bundles (cross-attention penalty, guide-image
keyframes, audio/image preprocessing), and flags the candidates for
the 4K dissolve: model-side limit at LTX sampling time vs. node-side
length-independent σ in the penalty matrix.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(investigations): round 2 — 2K vs 4K + two-stage upscale findings

Adds the maintainer's round-2 observations:
 - 2K renders behave much better than 4K (segments morph instead of
   dissolving) — model is closer to its training distribution.
 - The two-stage workflow's upscale stage (stage 2) is overwriting
   stage 1's audio/animation: lip sync degrades and segment routing
   weakens.

Captures the hypothesis space:
 - Why 2K > 4K: model prior (out of distribution at 4K) is dominant;
   token-grid scaling with a length-independent sigma is secondary.
 - Why stage 2 overwrites stage 1: KSampler denoise re-applies
   conditioning from scratch; if LTX Director isn't wired into
   stage 2 the Prompt Relay + audio_attn2 patches go missing;
   latent shape change between stages forces a penalty rebuild.

Includes a 3-option pros/cons table (4K direct vs 2K direct vs
2-stage) and 6 investigation directions ordered cheapest-first,
starting with auditing the stage-2 wiring.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(investigations): round 3 — both stages run LTX Director; new prime suspect

Maintainer's screenshot confirms both stages route through LTX Director,
so round 2's missing-patch hypothesis is ruled out. Audited the stock
LTX nodes used in stage 2 against ComfyUI source:

 - LTXVLatentUpsampler explicitly pop()s the noise_mask and runs a
   learned (EXPERIMENTAL) neural upscaler — not interpolation.
 - LTXVConcatAVLatent fills a missing noise_mask with ones_like
   (denoise fully) for the side that's missing.
 - LTXVCropGuides removes keyframe slots and clears keyframe_idxs.

Traced the audio noise_mask end-to-end through stage 2 and found:
audio waveform stays preserved (noise_mask=0), video keyframes stay
pinned, but every non-keyframe video frame goes into stage 2's
sampler at noise_mask=1 — i.e. fully regenerated at denoise=0.80.
That's where stage 1's audio-aligned mouth shapes live, and that's
what's getting thrown away.

Prime suspect: denoise=0.80 is too high for "preserve stage 1"
semantics — most LTX two-stage reference workflows use 0.30–0.40.
Secondary suspect: the LTXVLatentUpsampler [BETA] is a learned model
that can hallucinate spatially-dense detail (faces, mouths) before
stage 2 even starts denoising.

Includes a 4-step cheapest-first test plan starting with dropping
stage 2 denoise to 0.30.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(investigations): round 4 — scheduler-shape finding + tracking plan

Captures two outputs from the planning session so they're preserved
for the next pickup:

1. Promotes the maintainer's empirical finding to a locked-in
   baseline: `linear_quadratic` scheduler at 8 steps produces the
   right sigma curve shape (flat plateau near sigma=1.0 for ~5
   steps, then steep drop in the last 3) for both phases. Any
   other scheduler degrades fast motion and keyframe transitions.

2. Records the agreed tracking-system plan (BASELINE.md +
   log.md + per-run screenshot folder, with three 1-5 score axes
   matching the live trade-off: motion / audio sync / Phase-2
   sharpness). Not yet scaffolded — explicit "next step" callout
   for whoever picks this up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(investigations): /make-card skill + PNG card renderer toolkit

Per-iteration tracking card for the LTX Director 4K investigation.
Solves the documentation-drift problem we hit on v01->v02 (multiline
summary diverged from actual node values) by treating the saved
workflow JSON as the source of truth.

Adds:
 - scripts/make_card.py — reads a ComfyUI workflow JSON, paints a
   self-contained 540px PNG card via Pillow. Extracts Phase 1 run
   params (Format/Mult/Scheduler/Steps/Denoise), locked base
   (Sampler/CFG/Seed/epsilon/Audio src), scene (duration/segments
   with prompt/audio/keyframe checkboard), free-text BASE notes
   from `OVERLAY - INFO` Text Multiline, FEEDBACK from `OVERLAY -
   FEEDBACK` Text Multiline, and a POST-RENDER block that
   auto-computes render duration from JSON->newest-video mtime
   delta in the same folder. Layout mirrors the project's
   snapshot-dialogs design language (panelled sections, sky/amber/
   green accents matching docs/designs/snapshot-dialogs.html).
 - scripts/watch_cards.py — background poller. Watches a working
   folder; auto-renders <stem>_card.png whenever a workflow JSON
   appears or is updated. Skips up-to-date PNGs.
 - .claude/skills/make-card/SKILL.md — /make-card slash command;
   globs the newest JSON in the working folder, calls make_card.py,
   shows the PNG inline.
 - .claude/skills/make-card/work-folder.txt — one-line config of
   the working folder path. Edit to switch projects.
 - docs/investigations/experiments/{baseline,combined-card,run-001-card}.html
   — design prototypes that fed into the PNG layout; kept as
   history.
 - docs/investigations/experiments/run-001-card.png and
   run-v04-card.png — sample card outputs from v03 and v04 of the
   workflow JSON, included for reference.

Issue #151 tracks the eventual native ComfyUI custom node so this
flow happens inside the graph; for now the external script + watcher
is the loop.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(launch): use absolute Windows python path for design-server

The default `python3` runtimeExecutable fails on this Windows host
(triggers the Microsoft Store install stub). Hard-coding
C:\Python313\python.exe lets the preview MCP and the design-server
target work on the maintainer's machine. Worktree-scoped change —
main keeps the cross-platform `python3` default.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(automations): docs/loop scaffold + env-var working folder + _AI subdir

Promotes the LTX Director card-rendering loop from one-off scripts to a
documented, project-portable automation framework.

Adds `docs/automations/` as the home for AI-assisted iteration loops:
 - README.md — the pattern; how to start a loop for a new model
 - CONVENTIONS.md — cross-cutting: working folder, Text Multiline node
   contract (Working_Folder_PATH / OVERLAY - INFO / OVERLAY - FEEDBACK),
   card format, iterations log, skill, watcher, run-id rules, tool
   locations
 - LTX-2.3/README.md — entry point for the LTX 2.3 loop; per-iteration
   steps; full extraction table; active investigation status
 - LTX-2.3/handoff-checklist.md — 5-minute onboarding for a new machine
   or new person
 - LTX-2.3/findings.md — promoted findings (linear_quadratic scheduler
   locked-in; resolution behaviour; audio mode; open two-stage suspects)

Replaces the committed `.claude/skills/make-card/work-folder.txt` with
an env-var-driven path. Follows the project's existing `KOLOOK_*`
convention — `.env.example` documents the new
`KOLOOK_AUTOMATIONS_WORK_DIR`. Real path lives in `.env` (gitignored).
No more leaking client-specific absolute paths into committed files.

`scripts/make_card.py`:
 - Loads `.env` automatically; worktree-aware (walks the .git common-dir
   marker to find the main repo when run from a worktree).
 - Output folder priority: Working_Folder_PATH node in JSON → env var →
   JSON's parent dir.
 - Run with no args: auto-discovers newest *.json in the working folder.
 - Writes `_AI/card.png` + `_AI/iterations.md` inside the working folder,
   not the root. Lets the maintainer wipe AI artifacts or user content
   (JSONs/MP4s) independently.

`/make-card` skill simplified to a single command — `python
scripts/make_card.py` — the script handles env loading and discovery
itself.

baseline.html: stripped the project-specific label.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(automations): sync all path references to _AI/ + add layout diagram

Audit pass after introducing _AI/ subfolder. Updated every doc and
script that still referenced root-level card.png / iterations.md:

 - docs/automations/CONVENTIONS.md (3 spots)
 - docs/automations/LTX-2.3/README.md (per-iteration steps + composite)
 - docs/automations/LTX-2.3/handoff-checklist.md (4 spots, including
   the off-boarding section which conflated repo commits with working-
   folder artifacts)
 - docs/automations/LTX-2.3/findings.md (promotion criterion)
 - scripts/watch_cards.py (target PNG path now matches make_card.py's
   _AI/card.png — was writing to a stale <stem>_card.png location)

Adds a clear "Where things live" section to docs/automations/README.md
showing both layouts side-by-side:
 - the repo tree (where the tooling, skill, env-example, and per-model
   loop docs sit)
 - the per-project working-folder tree (user content at root, _AI/
   subfolder for managed artifacts)

So anyone picking this up can answer "what's in the repo vs. what's in
my project folder" without reading prose.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(automations): add CHEATSHEET.md — one-page dense reference

Compact reminder card for returning users. Structured by application
context: ComfyUI · Scripts · Folder structure · Conventions · Quick-fail
diagnostics. Skippable for first-timers; targets the maintainer who
already knows the loop and just needs the right node names, score
syntax, and command list at a glance.

Linked from docs/automations/README.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(automations): add Bootstrap section to CHEATSHEET — bare-bones setup

What's needed to get a fresh machine running the loop, separated from
the daily-loop reference. Three checklists:
 - Repo side: Python + Pillow + .env config
 - ComfyUI side: was-node-suite + the three required Text Multiline
   nodes, with exact titles
 - First card: /make-card → expect _AI/card.png

Deliberately skips model-specific setup (checkpoints, LoRAs, per-model
custom nodes) — those live in each model's handoff-checklist.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(automations): skip *loop* files in auto-discovery + watcher

The maintainer's workflow includes a "loop" stage in ComfyUI that
composites the card next to the video and writes the result back into
the working folder as `<base>_loop_<seq>.{mp4,json}`. Without a skip
rule, auto-discovery treats this composited-card output as the newest
input — the script then re-renders the card from yesterday's metadata.

Now: any JSON or video file with `loop` in its stem (case-insensitive)
is ignored by:
 - scripts/make_card.py — auto-discover newest JSON, newest video
 - scripts/watch_cards.py — polling glob

Explicit invocation still works: `python scripts/make_card.py
<wf_loop.json>` will render a card for that file specifically.

Documents the rule in docs/automations/CONVENTIONS.md (new §7) and adds
a one-line bullet to CHEATSHEET conventions.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* style(automations): satisfy Ruff lint — split if-on-one-line, rename l, drop os

CI's Lint & Smoke step flagged 17 violations across the new scripts.
Cleaned up:

 - scripts/make_card.py: 11 × E701 (multi-statement lines)
   split every `if cond: stmt` onto two lines.
 - scripts/make_card.py: 5 × E741 (ambiguous variable `l`)
   renamed to `link` in the two link-dict comprehensions and their
   downstream lookups.
 - scripts/watch_cards.py: 1 × F401 (unused `os` import).

No functional change — `make_card.py` re-renders the working folder's
card identically post-cleanup. Verified `ruff check` returns
"All checks passed!" on both files.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(automations): render time from ComfyUI log, not disk timestamps

Disk mtimes don't reliably bracket the model run — the maintainer's
workflow saves the workflow JSON manually after rendering, assembles
QuickTime from EXRs separately, and ComfyUI's metadata bundle writes
JSON + MP4 at the same instant. None of those give us actual render
duration.

Render time now sourced from ComfyUI's own log
(`<comfyui-root>/user/comfyui.log`):

 - Find the latest `saving images: 100%` line → pair it with the next
   `Prompt executed in X` → that's the most recent EXR-producing render
   wall-time.
 - Supports both `Prompt executed in 18.21 seconds` and
   `Prompt executed in HH:MM:SS` log formats.
 - Skips trivial follow-up prompts (e.g. audio-VAE load between render
   and /make-card invocation).

Log path resolution:
 1. `KOLOOK_COMFYUI_LOG` env var (explicit override).
 2. Inferred from existing `KOLOOK_COMFYUI_DEV_PATH` →
    `<comfyui-root>/user/comfyui.log`.

Fallbacks remain in priority order: EXR sequence span (oldest to newest
in the latest version subfolder) → MP4 mtime delta → "pending".

Docs:
 - .env.example: documents the optional KOLOOK_COMFYUI_LOG.
 - CONVENTIONS.md: new §4 "Render duration" describing the resolution
   chain; section numbers renumbered.
 - CHEATSHEET.md: updated the "Render time —" diagnostic.

Lint: ruff clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-17 22:44:52 +02:00

802 lines
32 KiB
Python
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""Render an experiment-tracking card PNG from a ComfyUI workflow JSON.
Usage:
python scripts/make_card.py <workflow.json> [output.png]
Reads the LTX Director run/base context from the workflow JSON, paints a
self-contained card matching the look of
docs/investigations/experiments/combined-card.html, and writes it to PNG.
PNG can then be loaded into a ComfyUI workflow via a LoadImage node and
composited alongside the rendered video, or dropped straight into an NLE.
Tested against LTX_Director_4k_v0[1-3].json.
"""
from __future__ import annotations
import json
import os
import re
import sys
import time
from pathlib import Path
from PIL import Image, ImageDraw, ImageFont
def _load_dotenv(env_path: Path) -> None:
"""Minimal `.env` loader. Mirrors scripts/sync_to_dev.py — no python-dotenv dep."""
if not env_path.exists():
return
for line in env_path.read_text(encoding="utf-8").splitlines():
line = line.strip()
if not line or line.startswith("#") or "=" not in line:
continue
key, value = line.split("=", 1)
os.environ.setdefault(key.strip(), value.strip().strip('"').strip("'"))
def _find_dotenv() -> Path | None:
"""Find .env in the current worktree first, then fall back to the main repo
root (resolved via git's common-dir) when running from a worktree.
Returns the first existing .env found, or None."""
repo_root = Path(__file__).resolve().parent.parent
direct = repo_root / ".env"
if direct.exists():
return direct
# Try the main repo root via git's common-dir.
git_marker = repo_root / ".git"
common_dir = None
if git_marker.is_file():
# Worktree marker file contains: "gitdir: <path-to>/<main>/.git/worktrees/<name>"
try:
content = git_marker.read_text(encoding="utf-8").strip()
if content.startswith("gitdir:"):
gitdir = Path(content.split(":", 1)[1].strip())
# walk up from .git/worktrees/<name>/ to find <main repo>/
if "worktrees" in gitdir.parts:
idx = gitdir.parts.index("worktrees")
common_dir = Path(*gitdir.parts[:idx]) # ends at <main>/.git
main_root = common_dir.parent
candidate = main_root / ".env"
if candidate.exists():
return candidate
except Exception:
pass
return None
# Resolve repo root and load .env once on import — same pattern as sync_to_dev.py.
_REPO_ROOT = Path(__file__).resolve().parent.parent
_env_file = _find_dotenv()
if _env_file is not None:
_load_dotenv(_env_file)
# ---------- palette + geometry (kept in sync with combined-card.html) ----------
W = 540 # slim vertical card — sits beside a video
PAD_X = 28
PAD_TOP = 28
PAD_BOTTOM = 28
# Palette mirrored from docs/designs/snapshot-dialogs.html
BG_OUTER = (14, 14, 14) # --bg
BG_CARD = (21, 21, 21) # --panel
BG_SECTION = (26, 26, 31) # --panel-soft (each section gets this)
BORDER = (48, 47, 47) # --border
BORDER_STR = (79, 79, 84) # --border-strong
TEXT = (249, 250, 251) # --text
MUTED = (143, 149, 156) # --muted
DIM = (200, 204, 209) # close to --text but slightly down
HEADER_GREY = (201, 204, 209) # snapshot dialog H2 colour
ACCENT_RUN = (255, 184, 77) # --amber (PHASE 1 / changeable)
ACCENT_BASE = (109, 180, 255) # --sky (frozen base)
ACCENT_OUT = (123, 207, 128) # --green (post-render)
NOTE_BG = (12, 12, 12)
RADIUS = 14
SECTION_RADIUS = 8
# ---------- font loading ----------
WIN_FONTS = Path(r"C:/Windows/Fonts")
def load_font(filenames: list[str], size: int) -> ImageFont.FreeTypeFont:
for fn in filenames:
for p in (WIN_FONTS / fn, Path("/Library/Fonts") / fn, Path("/usr/share/fonts") / fn):
if p.exists():
return ImageFont.truetype(str(p), size)
return ImageFont.load_default()
F_TITLE = load_font(["segoeuib.ttf", "Arial Bold.ttf"], 28)
F_SUB = load_font(["segoeui.ttf", "Arial.ttf"], 17)
F_H2 = load_font(["segoeuib.ttf", "Arial Bold.ttf"], 16)
F_TAG = load_font(["segoeuib.ttf", "Arial Bold.ttf"], 12) # FEEDBACK / OUTCOME sub-labels
F_SECTION = load_font(["segoeuib.ttf", "Arial Bold.ttf"], 17) # main section titles (PHASE 1 etc.)
F_MONO = load_font(["consola.ttf", "Menlo.ttc", "DejaVuSansMono.ttf"], 19)
F_NOTE = load_font(["segoeuii.ttf", "Arial Italic.ttf"], 18)
# ---------- workflow extraction ----------
VIDEO_EXTS = (".mp4", ".mov", ".mkv", ".webm", ".avi")
LOG_NAME = "iterations.md"
AI_SUBDIR = "_AI" # AI-managed artifacts live here, separate from user content
CARD_NAME = "card.png"
# Files containing this marker (case-insensitive) in their basename are
# post-loop outputs (the card already composited with the video) — skip
# them in auto-discovery so the script never picks its own output as input.
SKIP_MARKER = "loop"
def _is_loop_output(p: Path) -> bool:
return SKIP_MARKER in p.stem.lower()
LOG_HEADER = (
"# LTX Director — iterations log\n\n"
"Append-only. Newest at bottom. Generated by `scripts/make_card.py`.\n\n"
"| # | When | Run | Format | Denoise | JSON | Video | Δ from base | Feedback |\n"
"| - | ---- | --- | ------ | ------- | ---- | ----- | ----------- | -------- |\n"
)
def log_iteration(folder: Path, json_path: Path, data: dict) -> None:
"""Append one row to <folder>/_AI/iterations.md. Dedupe by (json filename + json mtime).
Markdown table — newest at bottom. Creates the file + the _AI subdir on first call."""
ai_dir = folder / AI_SUBDIR
ai_dir.mkdir(exist_ok=True)
log_path = ai_dir / LOG_NAME
json_mtime = json_path.stat().st_mtime
json_mtime_iso = time.strftime("%Y-%m-%d %H:%M", time.localtime(json_mtime))
fingerprint = f"{json_path.name}@{int(json_mtime)}"
existing = ""
last_fp = ""
next_idx = 1
if log_path.exists():
existing = log_path.read_text(encoding="utf-8")
rows = [ln for ln in existing.splitlines()
if ln.startswith("| ") and not ln.startswith("| -")
and not ln.startswith("| #")]
next_idx = len(rows) + 1
if rows:
# last row's hidden HTML-comment fingerprint, if present
tail_comment_re = re.compile(r"<!--fp:([^>]+)-->")
for ln in reversed(rows):
m = tail_comment_re.search(ln)
if m:
last_fp = m.group(1)
break
if last_fp == fingerprint:
# Same JSON, same mtime — already logged. Skip.
return
def trunc(s, n):
s = (s or "").replace("\n", " ").strip()
return (s[: n - 1] + "…") if len(s) > n else s
row = (
f"| {next_idx} "
f"| {json_mtime_iso} "
f"| {data.get('run_label', '?')} "
f"| {data.get('format', '?')} "
f"| {data.get('denoise', '?')} "
f"| `{json_path.name}` "
f"| `{data.get('render_output', '—')}` "
f"| {trunc(data.get('note', ''), 40)} "
f"| {trunc(data.get('feedback', ''), 40)} "
f"|<!--fp:{fingerprint}-->\n"
)
if not existing:
log_path.write_text(LOG_HEADER + row, encoding="utf-8")
else:
with log_path.open("a", encoding="utf-8") as f:
f.write(row)
def load_workflow(path: Path) -> dict:
"""Load a ComfyUI workflow JSON. Accepts two formats:
1. raw editor JSON (top-level 'nodes' + 'links' keys), or
2. metadata bundle ({"CreationTime": ..., "prompt": ..., "workflow": {...}})
— the format ComfyUI writes alongside saved outputs.
"""
raw = json.loads(path.read_text(encoding="utf-8"))
if "nodes" in raw:
return raw
if "workflow" in raw and isinstance(raw["workflow"], dict) and "nodes" in raw["workflow"]:
return raw["workflow"]
raise SystemExit(f"Unrecognised JSON format in {path} — no 'nodes' key found.")
def find_newest_video(folder: Path):
"""Return (Path, mtime) for the newest video file in folder, or (None, None).
Skips files marked as loop outputs (see SKIP_MARKER)."""
candidates = [p for p in folder.iterdir()
if p.is_file()
and p.suffix.lower() in VIDEO_EXTS
and not _is_loop_output(p)]
if not candidates:
return None, None
best = max(candidates, key=lambda p: p.stat().st_mtime)
return best, best.stat().st_mtime
def find_comfyui_log_path():
"""Resolve the ComfyUI log path. Priority:
1. `KOLOOK_COMFYUI_LOG` env var (explicit override).
2. Inferred from `KOLOOK_COMFYUI_DEV_PATH` — the dev path points at
`<comfyui-root>/custom_nodes/<koolook>`, so the log sits at
`<comfyui-root>/user/comfyui.log`.
Returns Path or None."""
override = os.environ.get("KOLOOK_COMFYUI_LOG", "").strip()
if override:
p = Path(override)
return p if p.exists() else None
dev = os.environ.get("KOLOOK_COMFYUI_DEV_PATH", "").strip()
if not dev:
return None
candidate = Path(dev).parent.parent / "user" / "comfyui.log"
return candidate if candidate.exists() else None
_LOG_DUR_PATTERNS = [
# "Prompt executed in 18.21 seconds"
(re.compile(r"Prompt executed in ([\d.]+) seconds"),
lambda m: float(m.group(1))),
# "Prompt executed in 00:10:33"
(re.compile(r"Prompt executed in (\d+):(\d+):(\d+)"),
lambda m: int(m.group(1)) * 3600 + int(m.group(2)) * 60 + int(m.group(3))),
]
def render_time_from_log(log_path: Path):
"""Return the duration of the most recent render that actually wrote
frames. Locates the latest `saving images: 100%` line in the Comfy log
and pairs it with the next `Prompt executed in X` line — that's the
completed EXR-producing render. Skips trivial follow-up prompts that
happen between the render finishing and the user invoking /make-card.
Returns duration in seconds, or None."""
if not log_path or not log_path.exists():
return None
text = log_path.read_text(encoding="utf-8", errors="replace")
saves = list(re.finditer(r"saving images: 100%", text))
if not saves:
return None
after = saves[-1].end()
candidates = []
for pat, to_seconds in _LOG_DUR_PATTERNS:
m = pat.search(text, after)
if m:
candidates.append((m.start(), to_seconds(m)))
if not candidates:
return None
# Take the first 'Prompt executed in X' line after the save (smallest start).
candidates.sort(key=lambda x: x[0])
return candidates[0][1]
def exr_sequence_duration(folder: Path):
"""Return (duration_seconds, version_dir_name) for the most recent EXR
sequence in the working folder. Treats render time as the wall-clock
span between the first and last frame in the version subdirectory of
the newest EXR — accurate even when the JSON is saved after rendering.
Returns (None, None) if no EXR sequence is found."""
all_exrs = list(folder.rglob("*.exr"))
if not all_exrs:
return None, None
newest = max(all_exrs, key=lambda p: p.stat().st_mtime)
seq_dir = newest.parent
seq_mtimes = sorted(p.stat().st_mtime for p in seq_dir.glob("*.exr"))
if len(seq_mtimes) < 2:
return None, seq_dir.name
return seq_mtimes[-1] - seq_mtimes[0], seq_dir.name
def fmt_duration(seconds: float) -> str:
s = int(round(seconds))
if s < 60:
return f"{s}s"
m, s = divmod(s, 60)
if m < 60:
return f"{m}m {s:02d}s"
h, m = divmod(m, 60)
return f"{h}h {m:02d}m"
def find(wf, t):
return [n for n in wf["nodes"] if n.get("type") == t]
def widgets(n): return n.get("widgets_values", []) or []
def resolve_input(wf, node, name):
"""Return the upstream node feeding a named input, or None."""
links = {link[0]: link for link in wf["links"]}
by_id = {n["id"]: n for n in wf["nodes"]}
for inp in node.get("inputs", []) or []:
if inp.get("name") == name and inp.get("link") is not None:
link = links.get(inp["link"])
if link:
return by_id.get(link[1])
return None
def follow_setnode(wf, getnode):
"""GetNode -> the value-producing source via SetNode of the same variable."""
var = widgets(getnode)[0] if widgets(getnode) else None
if not var:
return None
for n in wf["nodes"]:
if n.get("type") == "SetNode" and widgets(n)[:1] == [var]:
# SetNode's incoming link is the actual value source
links = {link[0]: link for link in wf["links"]}
by_id = {n["id"]: n for n in wf["nodes"]}
for inp in n.get("inputs", []) or []:
if inp.get("link") is not None:
link = links.get(inp["link"])
if link:
src = by_id.get(link[1])
# If it's a switch, resolve the selected value
if src and src.get("type") == "easy anythingIndexSwitch":
idx_node = resolve_input(wf, src, "index")
idx = widgets(idx_node)[0] if idx_node and widgets(idx_node) else 0
val_node = resolve_input(wf, src, f"value{idx}")
return val_node
return src
return None
def res_label(w, h):
return {(3744,2112):"4K", (3840,2160):"4K", (2560,1440):"2K",
(1920,1080):"HD", (1280,720):"HD"}.get((w,h), f"{w}x{h}")
def extract(wf):
"""Pull every field we need to draw the card."""
d = find(wf, "LTXDirector")[0]
dw = widgets(d)
# LTXDirector widget order (verified in v01-v03):
# 0 global_prompt, 1 dur_frames, 2 dur_sec, 3 timeline_data, 4 local_prompts,
# 5 segment_lengths, 6 epsilon, 7 guide_strength, 8 use_custom_audio,
# 9 frame_rate, 10 display_mode, 11 custom_width, 12 custom_height,
# 13 resize_method, 14 divisible_by, 15 img_compression
epsilon = dw[6]
seg_lengths_str = dw[5]
fps = dw[9]
use_custom_aud = dw[8]
dur_frames = dw[1]
dur_seconds = dw[2]
# custom_width / height may be widget value OR fed by GetNode chain
cw = resolve_input(wf, d, "custom_width")
ch = resolve_input(wf, d, "custom_height")
if cw and cw.get("type") == "GetNode":
cw = follow_setnode(wf, cw)
if ch and ch.get("type") == "GetNode":
ch = follow_setnode(wf, ch)
width = widgets(cw)[0] if cw and widgets(cw) else dw[11]
height = widgets(ch)[0] if ch and widgets(ch) else dw[12]
# parse timeline_data segments
try:
tl = json.loads(dw[3]) if dw[3] else {}
segments = tl.get("segments", [])
except Exception:
segments = []
seg_lengths = [int(x.strip()) for x in seg_lengths_str.split(",") if x.strip()]
# phase 1 chain — first BasicScheduler with denoise == 1.0, else first one
schedulers = find(wf, "BasicScheduler")
p1 = next((s for s in schedulers if widgets(s)[2] == 1), schedulers[0] if schedulers else None)
sched_name, steps, denoise = (widgets(p1) if p1 else ("?", "?", "?"))
# Director Guide scale_by per phase — pick the one fed by LTXDirector (phase 1)
guides = find(wf, "LTXDirectorGuide")
p1_guide = None
for g in guides:
latent_src = resolve_input(wf, g, "latent")
if latent_src and latent_src.get("type") == "LTXDirector":
p1_guide = g
break
if p1_guide is None and guides:
p1_guide = guides[0]
mult = widgets(p1_guide)[0] if p1_guide else "?"
# Pull three named Text Multiline nodes from the workflow:
# - "OVERLAY - INFO" (or any title containing INFO) -> base notes (Δ from baseline)
# - "OVERLAY - FEEDBACK" / "FEEDBACK" / "OBSERVATIONS" -> video feedback + scores
# - "Working_Folder_PATH" / "OUT_working_folder" -> authoritative output folder
note, feedback, work_folder = "", "", ""
for n in wf["nodes"]:
if n.get("type") != "Text Multiline":
continue
title = (n.get("title") or "").upper()
ws = widgets(n)
if not (ws and isinstance(ws[0], str)):
continue
text = ws[0]
if not note and ("INFO" in title or ("OVERLAY" in title and "FEEDBACK" not in title)):
for marker in ("BASE (notes):", "BASE:"):
if marker in text:
note = text.split(marker, 1)[1].strip()
break
if not feedback and ("FEEDBACK" in title or "OBSERVATION" in title or "VIDEO" in title):
feedback = text.strip()
if not work_folder and ("WORKING_FOLDER" in title or "WORK_FOLDER" in title
or "OUT_WORKING" in title or "WORKING-FOLDER" in title):
work_folder = text.strip().strip("\"' ").rstrip("\\/")
# Parse scores out of the feedback text. Pattern: "motion: 4/5" / "sync 3" / "sharpness: 5/5".
scores = {"motion": None, "sync": None, "sharp": None}
feedback_lines = []
score_re = re.compile(
r"^\s*(motion|sync|sharp(?:ness)?|sharpness)\s*[:=]?\s*(\d+)\s*(?:/\s*\d+)?\s*$",
re.IGNORECASE,
)
for line in feedback.splitlines():
m = score_re.match(line)
if m:
key = m.group(1).lower()
if key.startswith("sharp"):
key = "sharp"
scores[key] = int(m.group(2))
elif line.strip():
feedback_lines.append(line.rstrip())
feedback = "\n".join(feedback_lines).strip()
# model stack
ckpt = (widgets(find(wf, "CheckpointLoaderSimple")[0]) if find(wf, "CheckpointLoaderSimple") else [""])[0]
lora_nodes = find(wf, "LoraLoaderModelOnly")
lora = ""
if lora_nodes:
lw = widgets(lora_nodes[0])
lora_name = lw[0].replace("\\", "/").split("/")[-1].replace(".safetensors", "")
lora_strength = lw[1] if len(lw) > 1 else ""
lora = f"{lora_name} @ {lora_strength}"
clip_nodes = find(wf, "DualCLIPLoader")
clip = ""
if clip_nodes:
cw_ = widgets(clip_nodes[0])
clip = cw_[0].replace("\\", "/").split("/")[-1].replace(".safetensors", "")
vaes = find(wf, "VAELoaderKJ")
video_vae = audio_vae = ""
for v in vaes:
n = widgets(v)[0].replace(".safetensors", "")
if "video" in n:
video_vae = n
elif "audio" in n:
audio_vae = n
# seed
rn = find(wf, "RandomNoise")
seed = widgets(rn[0])[0] if rn else "?"
return {
"format": res_label(width, height),
"resolution": f"{width} × {height}",
"mult": mult,
"scheduler": sched_name,
"steps": steps,
"denoise": denoise,
"note": note,
"feedback": feedback,
"scores": scores,
"work_folder_from_wf": work_folder,
"epsilon": epsilon,
"seed": seed,
"fps": fps,
"dur_frames": dur_frames,
"dur_seconds": dur_seconds,
"segments": segments,
"seg_lengths": seg_lengths,
"use_custom_audio": use_custom_aud,
"ckpt": Path(ckpt).name.replace(".safetensors", ""),
"lora": lora,
"clip": Path(clip).name,
"video_vae": video_vae,
"audio_vae": audio_vae,
}
# ---------- drawing helpers ----------
def draw_kv_row(draw, x, y, key, val, key_w):
draw.text((x, y), key, font=F_MONO, fill=MUTED)
draw.text((x + key_w, y), str(val), font=F_MONO, fill=TEXT)
return y + 26
def draw_h2(draw, x, y, label, color):
draw.text((x, y), label.upper(), font=F_H2, fill=color)
return y + 30
def wrap_text(text, max_chars):
"""Naive word-wrap to fit max_chars per line."""
out = []
for raw_line in text.split("\n"):
if not raw_line.strip():
out.append("")
continue
words, cur = raw_line.split(), ""
for w in words:
if len(cur) + len(w) + 1 <= max_chars:
cur = (cur + " " + w).strip()
else:
if cur:
out.append(cur)
cur = w
if cur:
out.append(cur)
return out
def draw_text_box(draw, x, y, width, label, content, accent, max_lines=3):
"""Draw a left-accented note box with a header label + wrapped content lines."""
char_per_line = 42
lines = wrap_text(content, char_per_line)[:max_lines]
box_h = 26 + 22 * max(1, len(lines)) + 14
draw.rounded_rectangle(
[x - 12, y, x + width, y + box_h],
radius=4, fill=NOTE_BG,
)
draw.rectangle([x - 12, y, x - 8, y + box_h], fill=accent)
draw.text((x + 4, y + 8), label, font=F_H2, fill=accent)
line_y = y + 34
for line in lines:
draw.text((x + 4, line_y), line, font=F_NOTE, fill=DIM)
line_y += 22
return y + box_h + 4
def draw_section(draw, x, y, width, accent, label, body_lines_height,
bg=BG_SECTION, border=BORDER):
"""Draw a section panel header + return (content_x, content_y, content_w, end_y).
Body content is drawn by the caller starting at (content_x, content_y)."""
HEADER_H = 36
section_h = HEADER_H + body_lines_height + 14
draw.rounded_rectangle(
[x, y, x + width, y + section_h],
radius=SECTION_RADIUS, fill=bg, outline=border, width=1,
)
# header label — bigger, no side bar, accent colour
draw.text((x + 14, y + 10), label.upper(), font=F_SECTION, fill=accent)
return x + 14, y + HEADER_H, width - 28, y + section_h + 10
def section_body_rows(num_rows, extra=0):
"""Pixel height needed for N kv rows plus optional extra padding."""
return num_rows * 26 + extra
def render(data: dict, out_path: Path) -> None:
# Oversized canvas; crop after laying out.
canvas_h = 2000
img = Image.new("RGB", (W, canvas_h), BG_OUTER)
draw = ImageDraw.Draw(img)
inset = 18
# outer card
draw.rounded_rectangle(
[inset, inset, W - inset, canvas_h - inset],
radius=RADIUS, fill=BG_CARD, outline=BORDER, width=1,
)
x = inset + PAD_X
y = inset + PAD_TOP
inner_w = W - 2 * inset - 2 * PAD_X
# ---------- HEADER ----------
import datetime
today = datetime.date.today().isoformat()
ver = data.get("run_label", "?")
stage_label = "single-stage" if data.get("denoise", 1) == 1 else "two-stage"
res_label = data.get("format", "")
title_line = f"Run {ver or '?'} — {stage_label} {res_label}".strip()
sub_line = f"{today} · {data.get('json_name', '')}"
draw.text((x, y), title_line, font=F_TITLE, fill=TEXT)
y += 38
draw.text((x, y), sub_line, font=F_SUB, fill=MUTED)
y += 30
# hairline divider under header
draw.line([(x, y), (x + inner_w, y)], fill=BORDER, width=1)
y += 18
key_w = 122
# ---------- PHASE 1 ----------
body_h = section_body_rows(5)
cx, cy, cw, end_y = draw_section(draw, x, y, inner_w, ACCENT_RUN, "Phase 1", body_h)
cy = draw_kv_row(draw, cx, cy, "Format", f"{data['format']} · {data['resolution']}", key_w)
cy = draw_kv_row(draw, cx, cy, "Mult", data["mult"], key_w)
cy = draw_kv_row(draw, cx, cy, "Scheduler", data["scheduler"], key_w)
cy = draw_kv_row(draw, cx, cy, "Steps", data["steps"], key_w)
cy = draw_kv_row(draw, cx, cy, "Denoise", data["denoise"], key_w)
y = end_y
# ---------- BASE NOTES ----------
note_text = data["note"] or "(no changes)"
note_lines = wrap_text(note_text, 42)[:4]
body_h = max(28, 24 * len(note_lines)) + 4
cx, cy, cw, end_y = draw_section(draw, x, y, inner_w, ACCENT_RUN,
"Base · notes (Δ this run)", body_h)
for line in note_lines:
draw.text((cx, cy), line, font=F_NOTE, fill=DIM)
cy += 24
y = end_y
# ---------- BASE · LOCKED ----------
body_h = section_body_rows(5)
cx, cy, cw, end_y = draw_section(draw, x, y, inner_w, ACCENT_BASE, "Base · locked", body_h)
cy = draw_kv_row(draw, cx, cy, "Sampler", "euler", key_w)
cy = draw_kv_row(draw, cx, cy, "CFG", "1.0", key_w)
cy = draw_kv_row(draw, cx, cy, "Seed", f"{data['seed']} (fixed)", key_w)
cy = draw_kv_row(draw, cx, cy, "epsilon", data["epsilon"], key_w)
cy = draw_kv_row(draw, cx, cy, "Audio src", "model-gen"
if not data["use_custom_audio"] else "custom", key_w)
y = end_y
# ---------- BASE · SCENE ----------
seg_rows = min(len(data["segments"]), 6)
body_h = section_body_rows(2 + seg_rows, extra=4)
cx, cy, cw, end_y = draw_section(draw, x, y, inner_w, ACCENT_BASE, "Base · scene", body_h)
cy = draw_kv_row(draw, cx, cy, "Duration",
f"{data['dur_frames']} f · {data['dur_seconds']} s @ {data['fps']} fps", key_w)
cy = draw_kv_row(draw, cx, cy, "Segments",
f"{len(data['segments'])} × {data['seg_lengths'][0] if data['seg_lengths'] else '?'} f", key_w)
for i, seg in enumerate(data["segments"][:seg_rows]):
start_s = seg.get("start", 0) / data["fps"]
end_s = (seg.get("start", 0) + seg.get("length", 0)) / data["fps"]
has_prompt = "[x]" if seg.get("prompt") else "[ ]"
has_audio = "[x]" if "Audio:" in (seg.get("prompt") or "") else "[ ]"
has_kf = "[x]" if seg.get("imageFile") else "[ ]"
all_set = (has_prompt == has_audio == has_kf == "[x]")
fill_col = ACCENT_OUT if all_set else TEXT
draw.text((cx, cy), f"{i+1})", font=F_MONO, fill=ACCENT_BASE)
draw.text((cx + 32, cy), f"{start_s:.0f}-{end_s:.0f}s", font=F_MONO, fill=DIM)
draw.text((cx + 110, cy), f"P{has_prompt} A{has_audio} K{has_kf}",
font=F_MONO, fill=fill_col)
cy += 26
y = end_y
# ---------- POST-RENDER ----------
feedback_text = data["feedback"] or "(add observations)"
feedback_lines = wrap_text(feedback_text, 42)[:5]
body_h = (
section_body_rows(2) # render time + output
+ 8 # spacer
+ 20 # FEEDBACK sublabel
+ max(24, 22 * len(feedback_lines))
+ 14 # spacer before outcome
+ 20 # OUTCOME sublabel
+ 26 # outcome scores row
)
cx, cy, cw, end_y = draw_section(draw, x, y, inner_w, ACCENT_OUT, "Post-render", body_h)
cy = draw_kv_row(draw, cx, cy, "Render time", data.get("render_duration", "?"), key_w)
cy = draw_kv_row(draw, cx, cy, "Output", data.get("render_output", "?")[:36], key_w)
cy += 8
# FEEDBACK sublabel + body
draw.text((cx, cy), "FEEDBACK", font=F_TAG, fill=ACCENT_OUT)
cy += 20
for line in feedback_lines:
draw.text((cx, cy), line, font=F_NOTE, fill=DIM)
cy += 22
cy += 6
# OUTCOME sublabel + scores (auto-filled from FEEDBACK parsing)
draw.text((cx, cy), "OUTCOME", font=F_TAG, fill=ACCENT_OUT)
cy += 20
s = data.get("scores", {}) or {}
def _s(v): return f"{v}/5" if v is not None else "?/5"
draw.text((cx, cy),
f"Motion {_s(s.get('motion'))} Sync {_s(s.get('sync'))} Sharp {_s(s.get('sharp'))}",
font=F_MONO, fill=TEXT)
y = end_y
# ---------- final crop ----------
final_h = y + PAD_BOTTOM + inset
final = Image.new("RGB", (W, final_h), BG_OUTER)
draw2 = ImageDraw.Draw(final)
draw2.rounded_rectangle(
[inset, inset, W - inset, final_h - inset],
radius=RADIUS, fill=BG_CARD, outline=BORDER, width=1,
)
content = img.crop((inset + 1, inset + 1, W - inset - 1, final_h - inset - 1))
final.paste(content, (inset + 1, inset + 1))
final.save(out_path)
print(f"wrote {out_path} ({W}×{final_h})")
# ---------- CLI ----------
def _autodiscover_json() -> Path:
"""Find the newest *.json in $KOLOOK_AUTOMATIONS_WORK_DIR. Fails clearly."""
folder = os.environ.get("KOLOOK_AUTOMATIONS_WORK_DIR", "").strip()
if not folder:
raise SystemExit(
"KOLOOK_AUTOMATIONS_WORK_DIR not set. Add it to `.env` (see `.env.example`) "
"or pass the JSON path explicitly: `make_card.py <workflow.json>`."
)
p = Path(folder)
if not p.is_dir():
raise SystemExit(f"KOLOOK_AUTOMATIONS_WORK_DIR does not exist on disk: {p}")
candidates = sorted(
(j for j in p.glob("*.json") if not _is_loop_output(j)),
key=lambda x: x.stat().st_mtime,
reverse=True,
)
if not candidates:
raise SystemExit(f"No *.json files found in {p}. Export a workflow from ComfyUI first.")
return candidates[0]
def main():
if len(sys.argv) >= 2:
wf_path = Path(sys.argv[1])
else:
wf_path = _autodiscover_json()
print(f"auto-discovered: {wf_path}")
wf = load_workflow(wf_path)
data = extract(wf)
data["json_name"] = wf_path.name
data["json_stem"] = wf_path.stem
# Output folder priority (first existing wins):
# 1. Working_Folder_PATH Text Multiline inside the workflow JSON.
# 2. KOLOOK_AUTOMATIONS_WORK_DIR env var (from .env).
# 3. The JSON's own parent directory.
candidates = []
wf_folder = data.get("work_folder_from_wf") or ""
if wf_folder:
candidates.append(("Working_Folder_PATH node", Path(wf_folder)))
env_folder = os.environ.get("KOLOOK_AUTOMATIONS_WORK_DIR", "").strip()
if env_folder:
candidates.append(("KOLOOK_AUTOMATIONS_WORK_DIR", Path(env_folder)))
candidates.append(("JSON parent directory", wf_path.parent))
folder = None
for label, p in candidates:
try:
p = p.resolve()
except Exception:
continue
if p.is_dir():
folder = p
break
if folder is None:
# Last-resort fallback — use the unresolved JSON parent.
folder = wf_path.parent
# Output path: stable filename in the _AI subfolder of the working folder.
# Keeps managed artifacts (card.png + iterations.md) separated from user
# content (JSONs, MP4s) so the maintainer can wipe one without the other.
if len(sys.argv) > 2:
out_path = Path(sys.argv[2])
else:
ai_dir = folder / AI_SUBDIR
ai_dir.mkdir(exist_ok=True)
out_path = ai_dir / CARD_NAME
video, vmtime = find_newest_video(folder)
# Render duration priority:
# 1. ComfyUI's own `Prompt executed in X` log line — authoritative
# wall-time for the model run. Skips trivial follow-up prompts.
# 2. EXR sequence first-to-last-frame span.
# 3. JSON-vs-MP4 mtime delta (last resort).
# Disk mtimes alone are unreliable: ComfyUI writes JSON + MP4 together
# at end-of-render, and the maintainer often assembles MP4s separately.
json_mtime = wf_path.stat().st_mtime
log_path = find_comfyui_log_path()
log_duration = render_time_from_log(log_path) if log_path else None
if log_duration:
data["render_duration"] = fmt_duration(log_duration) + " (Comfy log)"
data["render_log_path"] = str(log_path)
else:
exr_span, exr_version = exr_sequence_duration(folder)
if exr_span is not None and exr_span > 0:
suffix = f" ({exr_version})" if exr_version else ""
data["render_duration"] = fmt_duration(exr_span) + suffix
elif video is not None:
# Fall back to MP4 mtime delta only if no EXRs are present.
same_stem = video.stem.startswith(wf_path.stem) or wf_path.stem.startswith(video.stem)
delta = vmtime - json_mtime
if same_stem:
data["render_duration"] = "—"
elif delta >= 0:
data["render_duration"] = fmt_duration(delta)
else:
data["render_duration"] = "pending"
else:
data["render_duration"] = "pending"
data["render_output"] = video.name if video is not None else "(no video yet)"
# run_label: prefer v01/v04 style, else fall back to ComfyUI sequence number
stem = wf_path.stem
m = re.search(r"(?:^|[_\-])v(\d{1,4})(?:[_\-]|$)", stem)
if m:
data["run_label"] = f"v{m.group(1)}"
else:
m = re.search(r"_(\d{4,6})(?:_|$)", stem)
data["run_label"] = f"#{int(m.group(1))}" if m else "?"
render(data, out_path)
log_iteration(folder, wf_path, data)
if __name__ == "__main__":
main()