bda4df576abeb51d2cf5f49e09c3bf3e9ee545bc
14
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
6f158c9020 |
fix(nodes): Easy_LoadVideo empty-branch degrade (bundles #243 publish-tab cleanup) (#244)
* docs(setups): add setup runner quickstart runbook Standalone reference for opening the published-setup mockup frontend (setup runner simulator): the `<comfy-address>/koolook/setup_runner_simulator.html` URL pattern, the live-registry vs setup-file (inspect-only) load paths, and demo mode. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(sidebar): close throwaway workflow tabs left by publish capture Publishing a setup captures ComfyUI's API-format prompt by loading the graph onto the canvas (graphToPrompt) and reloading the prior canvas. In current Comfy each loadGraphData spawns an "Unsaved Workflow" tab, so every publish left two stray tabs behind. captureWorkflowApiPrompt now snapshots the open workflows before capture and, afterward, re-activates the user's original tab and closes whatever the capture opened, via app.extensionManager.workflow. All of it is guarded and best-effort: the workflow-store API is Comfy-version specific, so if it is absent capture still succeeds and behavior falls back to today (no crash, no broken publish). Covered by a source-pattern test. Needs live verification against the 1.45 frontend (dev-sync) to confirm the closeWorkflow API shape. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(nodes): Easy_LoadVideo degrades to empty on an unselected folder branch When an upstream mode switch routes a path into Easy_LoadVideo, both the EXR sequence loader and the video loader execute eagerly. On the EXR branch the video loader was handed a folder (no filename) and raised ValueError, crashing the whole prompt -- even though that branch is discarded downstream. Lenient sequence loaders just ignore a wrong-type path; the video loader was the only one that hard-failed. _compose_input_video_path now returns EMPTY_BRANCH_SENTINEL instead of raising when input_path resolves to an existing directory. load_video turns that into a correctly-shaped empty result (IMAGE zeros, frame_count 0, empty AUDIO, empty video_info) matching VHS_LoadVideoPath returns, so the dead branch no longer crashes the prompt. VALIDATE_INPUTS and IS_CHANGED handle the sentinel too. A path that does not exist is still a genuine mistake and keeps raising. torch is imported lazily in the empty-result builder so the path logic stays importable/testable without torch. Covered by unit tests for the sentinel (absolute + relative dirs), the still-raises case, and a load_video integration test that asserts the empty result and that the loader never runs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(setups): expose prompt as an always-on field in the publish frontend The publish surface modelled `prompt` as a 4th source mode (mutually exclusive with EXR/QT/Img) and hid it (visible:false), so a published setup couldn't offer "pick a source AND describe the shot". This makes prompt a standalone, always-on input. - published_surface.js: prompt becomes visible + standalone + multiline, rendered independently of the source switch; the author's prompt-widget text is surfaced as a `placeholder` hint (submitted default is empty so an untouched hint is never sent), with a `help` line. The switch excludes standalone fields from its source options. - setup_runner_simulator.js: renders always-on standalone inputs alongside the selected source (textarea + placeholder + help), includes them in the run payload, and updates the demo setup. - setup_runner_simulator.html: .field-help styling. - contract doc: documents the always-on prompt behaviour. - launch.json: portability fix (hardcoded C:\Python313 -> python). Contract test updated + a new test asserting prompt is always-on, carries the placeholder, and is excluded from the switch source options (runs on CI where Node is available). Behaviour verified in the runner simulator (demo mode) in a browser: textarea + hint, EXR/QT/Img-only switch, prompt stays visible across source changes, payload includes the typed prompt. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(setups): mirror always-on prompt field in the Python publish inference The publish surface is inferred server-side in Python (koolook_setups.py _infer_app_surface, called by publishSetup), which overrides the client. The earlier change only updated the JS (published_surface.js), so re-publishing still produced the old surface with a hidden prompt. Mirror the JS here: prompt is a standalone, always-on input (visible + multiline) with the author's prompt-widget text as a placeholder hint and an empty submitted default; it is excluded from the source switch options. Keeps koolook_setups.py and published_surface.js in sync. Registry test updated to the new shape. Server + publish-contract suites green (115); full non-sidebar suite green (530). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(setups): keep the runner simulator polling through long renders The poller used a fixed 30s total budget, so a multi-minute render (e.g. 428s) timed out mid-run with "Timed out waiting for run" even though ComfyUI was still rendering and the run later landed as succeeded. timeoutMs is now a "no-progress" window: while ComfyUI keeps reporting the run as queued/running, the deadline is pushed forward, so a long-but-active render never spuriously times out; only a stalled/vanished run gives up. Default raised 30s -> 120s. The run status line shows elapsed seconds while active so a long wait visibly ticks instead of looking frozen. Happy path (queued -> running -> succeeded) unchanged; verified in the runner simulator (demo mode) in a browser. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(setups): serve simulator assets with no-store so dev-syncs take effect The runner-simulator HTML/JS route returned no cache headers, so browsers cached the assets and kept running stale JS after a dev-sync -- e.g. the old 30s poller persisted client-side even though the server was serving the long-render fix, which looked like "the fix didn't land". Serve these dev-synced assets with Cache-Control: no-store so a normal reload always picks up the current code. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(setups): add live run/poll smoke check for the published-setup API tools/check_setup_run.py queues a published setup through the same routes the runner simulator uses and polls the run endpoint with the same deadline-extension logic, printing the status timeline. Lets us self-verify end-to-end run tracking against a live ComfyUI (e.g. "does a long render reach succeeded?") without driving the browser UI. Used it to confirm the run/poll server flow is correct (terminal status detected in ~2s; run-000001 reported succeeded), isolating the simulator's earlier "Timed out" to stale browser cache rather than a server bug. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(nodes): Koolook_NextVersion auto-detects the next output version Adds filesystem-aware version auto-detection so a re-run lands on a fresh version instead of failing with "Output file already exists and overwrite is disabled" (the real EasyAIPipeline failure seen on the RMGB setup). - koolook_versioning.next_version_token(directory, name, ...): scans for existing <name>_vNNN files AND sequence-folders (handles name_v002.png, name_v002.0001.exr, name_v003/), returns highest+1 zero-padded; missing/ empty dir -> start (v001); exact base-name match so other shots never bump this one; empty name matches bare vNNN; configurable prefix/padding. - Koolook_NextVersion node (Koolook/Publish): folder + name -> next version token. Wire once into both the EXR and video version inputs so branches stay in sync. IS_CHANGED returns NaN to re-scan every run. Tested: helper edge cases + node registration/run. Preflight clean (20 node IDs). Full non-sidebar suite green (539). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(nodes): 'auto' version keyword for EasyAIPipeline + Easy_VideoCombine No new node or wiring needed: type 'auto' (or 'next') in the version field and the node scans its own output folder for existing <name>_vNNN entries and writes the next free version -- fixing "Output file already exists and overwrite is disabled" without manual version bumps. - koolook_versioning.is_auto_version(): detects the auto/next keyword. - EasyAIPipeline (build_pipeline_outputs): on auto, scans the version-folder parent (base, or base/shot/method when subfolders are on) for vNNN dirs and picks the next; version computed after path segments so the scan dir is known. - Easy_VideoCombine: on auto, resolves the token after the output dir + name root are composed, scanning that dir for <name>_vNNN outputs. - Non-auto/typed/wired tokens and disable_versioning are unchanged. Tests: is_auto_version keywords + EasyAIPipeline auto (v003 after v001/v002, v001 first run). Full non-sidebar suite green (542), preflight clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(nodes): drop redundant Koolook_NextVersion node The 'auto' version keyword on EasyAIPipeline + Easy_VideoCombine covers auto-detection without any new node or wiring, so the standalone Koolook_NextVersion node is redundant. Remove it (and its node test) to keep the node list clean; the shared next_version_token helper stays (the 'auto' keyword uses it). Node-ID count back to 19; full non-sidebar suite green (541), preflight clean. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(tools): satisfy security gate for setup smoke check * fix(pr244): address verified review findings * fix(video): fail if relative auto-version scan root is unavailable --------- Co-authored-by: Cristian Predut <cristian.predut@x.baconcph.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: AI Machine <ai.machine@x.baconcph.com> |
||
|
malkuthro
|
8c8546c17c |
Add queue-driven loop status controller (#205)
* Add queue-driven loop status node * Document ComfyUI server API testing * Harden loop queue controller * Guard loop controller HTTP requests |
||
|
malkuthroandClaude Opus 4.8
|
447465ecd8 |
refactor(nodes): remove superseded LTX Director extra nodes + harden registry (#198) (#199)
* refactor(nodes): remove superseded LTX Director extra nodes + harden registry (#198) Remove the two experimental nodes layered on the WhatDreamsCost fork — KoolookAudioTranscriptTimeline (k_audio_timeline.py) and KoolookTimelineEditor (k_timeline_editor.py) — plus the transcribe_audio_timeline.py CLI and its test. The audio-transcript to Director-timing job now lives inside the fork via the Director's audio_transcript_json input, so the standalone nodes are redundant. Also drops the orphaned [audio]/faster-whisper extra and updates packaging, dev-sync, web, and docs references. Harden the node registry against the #183 failure: __init__.py now imports each node group independently and installs the /koolook/presets/* routes regardless of node-import results, so one missing or broken module degrades to 'that node absent' instead of zeroing every node and 404'ing the sidebar disk routes. A new test keeps every root k_*.py listed in dev-sync's RUNTIME_PATHS. Closes #198. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(nodes): ship loader gates via dev-sync-audio; generalize RUNTIME_PATHS guard (#198) Address the K-Flame review on #199: MEDIUM — the #198 refactor made koolook_versioning.py a single registration gate, but dev-sync-audio shipped the new __init__.py without it (and the k_*.py-glob guard test couldn't see it). Add koolook_versioning.py AND koolook_install_guard.py (whose absolute-import fallback raises uncaught) to AUDIO_PATHS so a scoped sync can never leave the loader pointing at a module the target lacks. Generalize the guard test: parse __init__.py's actual load-time import targets (relative imports + _merge_node_group calls) and assert each is in RUNTIME_PATHS, instead of globbing k_*.py which missed the non-k_ gates. Add an AUDIO_PATHS gate-coverage test. LOW — CHANGELOG now records the [audio]/faster-whisper + transcribe_audio_timeline.py removal as an external-contract break. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
malkuthro
|
3a0f835814 | fix(audio-loop): read active director card values (#195) | ||
|
malkuthro
|
63f5c1db7b |
fix(audio-loop): capture card metadata and Director pins (#193)
* fix(audio-loop): capture card metadata and Director pins * fix(audio-loop): address card metadata review * fix(audio-loop): remove duplicate card import |
||
|
malkuthro
|
201cdf3001 |
Track audio loop card setup metadata (#191)
* feat(audio): surface director flavor and prompt mode * Track audio loop card setup metadata * Fix audio card review issues * Harden audio card metadata capture |
||
|
malkuthro
|
1d9d40ab70 |
Add audio transcript timeline node (#189)
* Add audio transcript timeline node * Hook audio transcripts into Koolook Director * Add modular timeline audio timing workflow * fix(frontend): guard Comfy workflow draft cache * fix(audio): compose transcript timeline prompts * fix(audio): address transcript review findings * fix(ci): keep transcript tests lightweight * fix(timeline): restore image previews from saved paths * fix(timeline): scrub restored preview blobs |
||
|
malkuthro
|
a5b6494848 | fix(audio): remove stale director web sync folder (#188) | ||
|
malkuthro
|
031af80f43 |
feat(forks): upgrade Koolook Director to stable v1.3.9 (#187)
* feat(forks): upgrade Koolook Director to stable v1.3.9 * docs(forks): clarify Director upgrade compatibility |
||
|
malkuthroandClaude Opus 4.7
|
26124e08d5 |
refactor(loop-audio): card reads only multilines + Koolook Director (#177) (#185)
* refactor(loop-audio): card reads only multilines + Koolook Director (#177) The audio-lipsync card is now bounded to two source families: the five tracked Text Multiline nodes and the LTX Director (Koolook v1.3.2) node's own widgets + input wiring. SAMPLER block (BasicScheduler / KSamplerSelect / RandomNoise / CFGGuider scrapes), FORK STATE block (_dev_build.json + git status), and INERT warning are gone — they were either widget scrapes outside the two source families or duplicates of the Director's own state. New card layout (top to bottom): HEADER run NNN — {name} + date + workflow filename KNOB STATE relay_overrides (the per-render knob) BASE · NOTES overlay - info verbatim, blank lines preserved BASE · LOCKED epsilon · Audio src · Working folder (path-wrapped) BASE · SCENE Segments (N) parent, indented `1) 0 to 5 seconds` child list, flat Prompt/Audio/Keyframe coverage POST-RENDER overlay - feedback + parsed motion/sync/sharp Audio src now derived structurally from a 4-state machine (audio_vae link × use_custom_audio × audioSegments count) that mirrors ltx_director.py:610-622 — never from substring matches on prompt text. Per-segment Audio coverage checks audioSegments[] overlap, not the maintainer's [Audio]: prompt formatting. Also fixed: upstream-LTXDirector widget index bug (saved workflow keeps legacy order even after Koolook fork reordered the schema; DIRECTOR_WIDX matches the serialised order); loop_audio.py's .env discovery now falls back to the main repo when run from a worktree (same pattern as make_card.py); notes.md drops the widget-scraped Mechanical interp section; log.md columns realigned (Audio src + Segments replace Phase 1 / Phase 2 / Custom audio). Three new docs alongside the module: reading-graph.html visual flow (dark theme, color-coded bands) reading-graph.schema.yaml machine-readable mirror of the same graph CHEATSHEET.md refreshed against the new design All 231 tests pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(lint): split semicolon statements + drop unused symbols in make_card_audio Ruff (project's Lint & Smoke gate) flagged 10 issues in the refactored card renderer: - F401 unused imports — `typing.Optional` at the top of the file and `loop_audio.REPO_ROOT` in the standalone CLI's import block. - F841 unused locals — `dur_f` and `dur_s` were left over from the dropped "Duration" row; segment time ranges convey the same info now. Kept in the state dict for notes.md; not read in the renderer. - E702 six semicolon-joined statements in the HEADER and POST-RENDER sections (`draw.text(...); y += N`). Split into separate lines. Rendering is unchanged — verified by re-running the standalone CLI against run-002. All 249 tests still pass; ruff now clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(loop-audio): resolve PR #185 multi-agent review findings Anchored by the verified multi-agent review on PR #185 — every fix traces to a confirmed finding (false-positive rate was 0/11). HIGH: - derive_audio_state now distinguishes "(no director)" from "off (no VAE)". Pre-fix, is_input_wired(None, ...) returned None, the truthiness check collapsed director-missing into VAE-unwired, and the card / log silently agreed on the wrong story. Added an early return when director_node is None. Also documented that vae_wired can be None (input socket absent in older schemas) and that we intentionally collapse it with False at the unwired branch — after the missing-director case has been split off. - extract_multilines now iterates needles longest-first. Pre-fix the per-node `break` was order-dependent on config insertion: a future short substring (e.g. "relay") in the same tracked_multilines list would have silently shadowed "relay_overrides". Sorting by length descending removes that footgun without breaking the current 5-needle layout. - parse_timeline now coerces per-segment numeric fields (start / length / trimStart) to int via int(float(...)) with try/except → 0 fallback. The upstream Director itself does this defensively at ltx_director.py:232-234; without coercion a string-valued field silently breaks video_segment_has_audio's `<` comparisons (lexical) and crashes the renderer at `start / fps`. Also drops non-dict entries from segments / audioSegments arrays so a malformed JSON can't slip past json.loads + isinstance checks. - Duplicated wrap_path + video_segment_has_audio bodies in make_card_audio.py are gone. Renderer now imports them at module level from loop_audio so the two scripts agree byte-for-byte on path-wrapping rules and audio-overlap geometry. MEDIUM: - DIRECTOR_WIDX now carries a MIRROR: comment pointing at the YAML schema's `widget_order:` block (the cross-link the review asked for as the minimum drift-prevention fix). - duration_frames / duration_seconds removed from the state dict and from render_audio_card's docstring. The renderer dropped the Duration row earlier in the PR; notes.md reads them straight from director_node via director_widget, so keeping them in state was documenting a contract the renderer didn't honour. YAML schema now splits them out under a notes_only section to match. - Log-row score formatting now uses an explicit `is not None` check (`_score()` helper) instead of `or '?'` — 0 was a legitimate score the maintainer could type that pre-fix collapsed to '?' because 0 is falsy. Matches the card's existing `_s()` helper for parity. - Card already surfaces the missing-Director state distinctly thanks to derive_audio_state returning "(no director)" — the BASE · LOCKED Audio src row shows that label loudly when applicable. No additional row needed. LOW: - Renderer falls back to "1) frames 0-120 (no fps)" when frame_rate is None instead of computing "0 to 0 seconds" silently. Test coverage: - New tests/scripts/test_loop_audio.py — 41 tests covering all five audio_src states, multiline ordering edge cases, parse_timeline coercion + malformed inputs, video_segment_has_audio overlap boundaries, parse_feedback (with explicit zero-score regression test), is_input_wired None vs False semantics, wrap_path separator-only wrapping, pick_existing_path filesystem fallback, autogen_label slug derivation. Docs: - reading-graph.html lists five audio states (was four). - reading-graph.schema.yaml audio_state_machine adds "(no director)" state + director_present input; state_dict moves duration_frames / duration_seconds into a notes_only section; invariants block grows with the missing-director split, type-coercion guarantee, longest- needle-wins guarantee, and zero-score preservation. - CHEATSHEET.md Audio src table grows from 4 to 5 rows. Validation: - 290 tests pass (was 249 — 41 new + 0 regressions). - ruff clean on scripts/loop_audio.py, scripts/make_card_audio.py, tests/scripts/test_loop_audio.py. - Standalone CLI re-renders run-002 card identically (same workflow, same pixels). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
malkuthroandClaude Opus 4.7
|
9f063345f7 |
feat(investigations): LTX23 audio-file lip-sync scaffold (#177) (#178)
* feat(investigations): scaffold #177 LTX23 audio-file lip-sync investigation Reorganize docs/investigations/ into numbered topic folders, each self-contained with narrative + upstream pin + patches + per-run snapshots. Add a generic sync helper that mirrors the dev-sync pattern for any investigation's third-party node modifications. Structure (new): docs/investigations/ 00_LTX23-base-1-step/README.md (rename of ltx-director-4k-transitions.md) 01_LTX23-audio-file-lipsync/ README.md -- problem, mechanism, patches, reversion, promotion upstream.json -- repo + version + pinned commit + sync target env + files patches/ -- source of truth for modified upstream files ltx_director.py -- adds `relay_overrides` multiline JSON input prompt_relay.py -- per-segment sigma per Prompt-Relay paper formula runs/ LOOP.md -- iteration protocol log.md -- rolling table (empty header for now) Sync helper: scripts/sync_investigation_patches.py <NN>_<topic> [--dry-run] - Reads upstream.json for target env var + file list - Backs up the target's current state once per day (<file>.bak.<YYYYMMDD>) - Worktree-aware: falls back to MAIN's .env via git rev-parse --git-common-dir - Prints a two-line report matching scripts/sync_to_dev.py shape - Per-investigation trigger phrase via `trigger:` key (sync-audio for #177) Env: .env.example -- documents new KOLOOK_WHATDREAMSCOST_PATH .env -- local path set Glossary updates (docs/reference/glossary.md): + Investigation folder + JSON file folder + Loop + Run + Investigation patches + sync_investigation_patches.py Closes the workflow gap from #177: MAIN is now the source of truth for investigation patches; the live install is a sync destination only. No manual file copying between iterations — `sync-audio` does it. * chore(ruff): exclude docs/investigations from lint The investigation patches under docs/investigations/*/patches/ are vendored upstream source (WhatDreamsCost-ComfyUI v1.3.2) carried so sync_investigation_patches.py can mirror them into the live install and so iterations can be diff'd against the pinned upstream commit. Linting them would force divergence from upstream and defeat that purpose. Mirrors the existing forks/ exclude precedent. Fixes the CI Lint & Smoke failure on PR #178. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: per-task automation modules + WhatDreamsCost Koolook fork Pivot docs/automations/ from "one folder per model" to "one folder per automation module" so the same diffusion model can host several independent tasks side by side. Replace the parallel docs/investigations/ tree with a fork of WhatDreamsCost-ComfyUI under forks/. Automation modules (new layout) - docs/automations/LTX-2.3/base-1step/ the existing single-stage automation, moved from docs/automations/LTX-2.3/ with its README, findings, and handoff-checklist rescoped to the task. The 4k-transitions backstory that produced the module moves here as backstory/4k-transitions.md. The earlier docs/investigations/experiments/ card PNGs/HTML land in example-cards/. - docs/automations/LTX-2.3/audio-lipsync/ the audio-file lip-sync module previously living under docs/investigations/01_…/. README, handoff, findings stub, backstory/, and runs/{LOOP,log}.md, all referencing the new fork. - docs/automations/README.md + CHEATSHEET.md updated to describe the module-per-task pivot and the new tree shape. - docs/automations/CONVENTIONS.md unchanged (cross-cutting loop contract still holds). WhatDreamsCost Koolook fork (new) - forks/whatdreamscost_koolook/versions/v1_3_2/ UPSTREAM_PIN.yaml pinned to upstream e81223a (v1.3.2), GPL-3.0 verified 2026-05-28. ltx_director.py MODIFIED — adds the relay_overrides multiline JSON widget on LTXDirector. Moved from docs/investigations/01_…/patches/. prompt_relay.py MODIFIED — per-segment σ per the Prompt-Relay paper formula σ = (L − w_eff) / (2 · √ln(1/ε)). Moved from docs/investigations/01_…/patches/. patches.py UNMODIFIED — vendored verbatim from upstream because ltx_director.py imports detect_model_type and apply_patches from it (same precedent as radiance_koolook's color_helpers.py). - Registered as LTXDirector__koolook_v1_3_2 ("LTX Director (Koolook v1.3.2)") so it coexists with an installed copy of upstream WhatDreamsCost-ComfyUI. Wired into the root __init__.py NODE_CLASS_MAPPINGS merge. - forks/forks_manifest.yaml + forks/THIRD_PARTY.md updated with the new entry (attribution, change log, license verified date, exposed node IDs). Removed - docs/investigations/ entire tree retired; its contents either moved to automation modules or to forks/. - scripts/sync_investigation_patches.py no longer needed — fork code lives in the Koolook tree and is synced via the standard dev-sync flow. - KOLOOK_WHATDREAMSCOST_PATH in .env.example same reason. - docs/investigations in pyproject.toml's ruff extend-exclude added in |
||
|
malkuthro
|
8447c05795 | docs: add automation loop visual overview (#163) | ||
|
malkuthroandClaude Opus 4.7
|
9372032b46 |
feat(automations): workflow validator + LTX 2.3 stage-2-only docs (#170)
* feat(automations): workflow JSON validator + LTX 2.3 stage-2-only findings Adds scripts/validate_workflow.py — a stdlib-only CLI that checks a ComfyUI workflow JSON for internal consistency: link-table declarations vs node-IO references, slot index bounds and types, endpoint cross- checks, last_node_id/last_link_id bounds, and duplicate IDs. Catches the class of bugs that come from hand-editing workflow JSONs (e.g. when an agent re-wires a graph to build a slim variant from a Director workflow). Comfy's loader silently drops or mis-wires bad parts, so the cost of an undetected drift is "graph looks fine but behaves wrong" — exactly the failure mode this script blocks. Exit codes: 0 clean, 1 problems on stderr, 2 file missing or bad JSON. Docs: - docs/automations/CONVENTIONS.md §11 — when and how to run the validator, full list of checks performed. - docs/automations/LTX-2.3/findings.md — two new sections covering the spatial upscaler's architectural facts (ltx-2.3-spatial-upscaler-x2 is hardcoded x2, the brief enters only via Director > guide_data > DirectorGuide, attention scales (W * H * T)^2) plus three open hypotheses for a stage-2-only refine path. - CHANGELOG.md [Unreleased] entry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(scripts): make validate_workflow.py robust against malformed JSON Addresses PR #170 review (Codex GPT-5.3) — the validator crashed with AttributeError on malformed-but-valid JSON, defeating its goal as a safe preflight tool. Two repros from the review: {"nodes":[1],"links":[],"last_node_id":0,"last_link_id":0} -> AttributeError: 'int' object has no attribute 'get' [1,2] -> AttributeError: 'list' object has no attribute 'get' Both confirmed locally. Now every shape assumption is guarded: - Top-level must be a JSON object; non-object short-circuits with one clear problem line. - 'nodes' and 'links' must be lists; non-list reported and treated as empty so subsequent checks still run. - Each node entry must be a dict; bad entries reported and dropped. - Each input/output entry must be a dict (extracted into a _sanitize_slots helper, cached per node so duplicates aren't reported when the same node is touched by multiple links). - Each link entry must be a 6-element list; malformed link entries reported and skipped without crashing later phases. - node_by_id only keys on int node IDs (non-int IDs can't match a link's int src/dst, so we don't mask bad data by accepting them). Adds tests/scripts/test_validate_workflow.py — 24 tests covering both review repros, every new guard, and a sampling of the existing consistency checks (link-to-missing-node, type mismatch, endpoint desync, duplicate IDs, last_node_id bound, CLI exit codes). Full suite: 168 passed in 1.59s. CHANGELOG entry updated to reflect the defensive behavior + test coverage. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
malkuthroandClaude Opus 4.7
|
092c204060 |
feat(automations): LTX Director iteration-loop framework + 4K investigation docs (#159)
* docs(investigations): kickoff for LTX Director 4K transition breakdown Captures the maintainer's problem statement (HD multi-keyframe works, 4K single-keyframe works, 4K multi-keyframe degenerates into dissolves between keyframes) and an initial conclusion about the WhatDreamsCost LTXDirector node's implementation. Traces the lineage to the Prompt Relay paper (Chen, Huang, Liu — arXiv:2604.10030) via Kijai's ComfyUI port, separates the three mechanisms the node bundles (cross-attention penalty, guide-image keyframes, audio/image preprocessing), and flags the candidates for the 4K dissolve: model-side limit at LTX sampling time vs. node-side length-independent σ in the penalty matrix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(investigations): round 2 — 2K vs 4K + two-stage upscale findings Adds the maintainer's round-2 observations: - 2K renders behave much better than 4K (segments morph instead of dissolving) — model is closer to its training distribution. - The two-stage workflow's upscale stage (stage 2) is overwriting stage 1's audio/animation: lip sync degrades and segment routing weakens. Captures the hypothesis space: - Why 2K > 4K: model prior (out of distribution at 4K) is dominant; token-grid scaling with a length-independent sigma is secondary. - Why stage 2 overwrites stage 1: KSampler denoise re-applies conditioning from scratch; if LTX Director isn't wired into stage 2 the Prompt Relay + audio_attn2 patches go missing; latent shape change between stages forces a penalty rebuild. Includes a 3-option pros/cons table (4K direct vs 2K direct vs 2-stage) and 6 investigation directions ordered cheapest-first, starting with auditing the stage-2 wiring. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(investigations): round 3 — both stages run LTX Director; new prime suspect Maintainer's screenshot confirms both stages route through LTX Director, so round 2's missing-patch hypothesis is ruled out. Audited the stock LTX nodes used in stage 2 against ComfyUI source: - LTXVLatentUpsampler explicitly pop()s the noise_mask and runs a learned (EXPERIMENTAL) neural upscaler — not interpolation. - LTXVConcatAVLatent fills a missing noise_mask with ones_like (denoise fully) for the side that's missing. - LTXVCropGuides removes keyframe slots and clears keyframe_idxs. Traced the audio noise_mask end-to-end through stage 2 and found: audio waveform stays preserved (noise_mask=0), video keyframes stay pinned, but every non-keyframe video frame goes into stage 2's sampler at noise_mask=1 — i.e. fully regenerated at denoise=0.80. That's where stage 1's audio-aligned mouth shapes live, and that's what's getting thrown away. Prime suspect: denoise=0.80 is too high for "preserve stage 1" semantics — most LTX two-stage reference workflows use 0.30–0.40. Secondary suspect: the LTXVLatentUpsampler [BETA] is a learned model that can hallucinate spatially-dense detail (faces, mouths) before stage 2 even starts denoising. Includes a 4-step cheapest-first test plan starting with dropping stage 2 denoise to 0.30. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(investigations): round 4 — scheduler-shape finding + tracking plan Captures two outputs from the planning session so they're preserved for the next pickup: 1. Promotes the maintainer's empirical finding to a locked-in baseline: `linear_quadratic` scheduler at 8 steps produces the right sigma curve shape (flat plateau near sigma=1.0 for ~5 steps, then steep drop in the last 3) for both phases. Any other scheduler degrades fast motion and keyframe transitions. 2. Records the agreed tracking-system plan (BASELINE.md + log.md + per-run screenshot folder, with three 1-5 score axes matching the live trade-off: motion / audio sync / Phase-2 sharpness). Not yet scaffolded — explicit "next step" callout for whoever picks this up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(investigations): /make-card skill + PNG card renderer toolkit Per-iteration tracking card for the LTX Director 4K investigation. Solves the documentation-drift problem we hit on v01->v02 (multiline summary diverged from actual node values) by treating the saved workflow JSON as the source of truth. Adds: - scripts/make_card.py — reads a ComfyUI workflow JSON, paints a self-contained 540px PNG card via Pillow. Extracts Phase 1 run params (Format/Mult/Scheduler/Steps/Denoise), locked base (Sampler/CFG/Seed/epsilon/Audio src), scene (duration/segments with prompt/audio/keyframe checkboard), free-text BASE notes from `OVERLAY - INFO` Text Multiline, FEEDBACK from `OVERLAY - FEEDBACK` Text Multiline, and a POST-RENDER block that auto-computes render duration from JSON->newest-video mtime delta in the same folder. Layout mirrors the project's snapshot-dialogs design language (panelled sections, sky/amber/ green accents matching docs/designs/snapshot-dialogs.html). - scripts/watch_cards.py — background poller. Watches a working folder; auto-renders <stem>_card.png whenever a workflow JSON appears or is updated. Skips up-to-date PNGs. - .claude/skills/make-card/SKILL.md — /make-card slash command; globs the newest JSON in the working folder, calls make_card.py, shows the PNG inline. - .claude/skills/make-card/work-folder.txt — one-line config of the working folder path. Edit to switch projects. - docs/investigations/experiments/{baseline,combined-card,run-001-card}.html — design prototypes that fed into the PNG layout; kept as history. - docs/investigations/experiments/run-001-card.png and run-v04-card.png — sample card outputs from v03 and v04 of the workflow JSON, included for reference. Issue #151 tracks the eventual native ComfyUI custom node so this flow happens inside the graph; for now the external script + watcher is the loop. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(launch): use absolute Windows python path for design-server The default `python3` runtimeExecutable fails on this Windows host (triggers the Microsoft Store install stub). Hard-coding C:\Python313\python.exe lets the preview MCP and the design-server target work on the maintainer's machine. Worktree-scoped change — main keeps the cross-platform `python3` default. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(automations): docs/loop scaffold + env-var working folder + _AI subdir Promotes the LTX Director card-rendering loop from one-off scripts to a documented, project-portable automation framework. Adds `docs/automations/` as the home for AI-assisted iteration loops: - README.md — the pattern; how to start a loop for a new model - CONVENTIONS.md — cross-cutting: working folder, Text Multiline node contract (Working_Folder_PATH / OVERLAY - INFO / OVERLAY - FEEDBACK), card format, iterations log, skill, watcher, run-id rules, tool locations - LTX-2.3/README.md — entry point for the LTX 2.3 loop; per-iteration steps; full extraction table; active investigation status - LTX-2.3/handoff-checklist.md — 5-minute onboarding for a new machine or new person - LTX-2.3/findings.md — promoted findings (linear_quadratic scheduler locked-in; resolution behaviour; audio mode; open two-stage suspects) Replaces the committed `.claude/skills/make-card/work-folder.txt` with an env-var-driven path. Follows the project's existing `KOLOOK_*` convention — `.env.example` documents the new `KOLOOK_AUTOMATIONS_WORK_DIR`. Real path lives in `.env` (gitignored). No more leaking client-specific absolute paths into committed files. `scripts/make_card.py`: - Loads `.env` automatically; worktree-aware (walks the .git common-dir marker to find the main repo when run from a worktree). - Output folder priority: Working_Folder_PATH node in JSON → env var → JSON's parent dir. - Run with no args: auto-discovers newest *.json in the working folder. - Writes `_AI/card.png` + `_AI/iterations.md` inside the working folder, not the root. Lets the maintainer wipe AI artifacts or user content (JSONs/MP4s) independently. `/make-card` skill simplified to a single command — `python scripts/make_card.py` — the script handles env loading and discovery itself. baseline.html: stripped the project-specific label. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(automations): sync all path references to _AI/ + add layout diagram Audit pass after introducing _AI/ subfolder. Updated every doc and script that still referenced root-level card.png / iterations.md: - docs/automations/CONVENTIONS.md (3 spots) - docs/automations/LTX-2.3/README.md (per-iteration steps + composite) - docs/automations/LTX-2.3/handoff-checklist.md (4 spots, including the off-boarding section which conflated repo commits with working- folder artifacts) - docs/automations/LTX-2.3/findings.md (promotion criterion) - scripts/watch_cards.py (target PNG path now matches make_card.py's _AI/card.png — was writing to a stale <stem>_card.png location) Adds a clear "Where things live" section to docs/automations/README.md showing both layouts side-by-side: - the repo tree (where the tooling, skill, env-example, and per-model loop docs sit) - the per-project working-folder tree (user content at root, _AI/ subfolder for managed artifacts) So anyone picking this up can answer "what's in the repo vs. what's in my project folder" without reading prose. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(automations): add CHEATSHEET.md — one-page dense reference Compact reminder card for returning users. Structured by application context: ComfyUI · Scripts · Folder structure · Conventions · Quick-fail diagnostics. Skippable for first-timers; targets the maintainer who already knows the loop and just needs the right node names, score syntax, and command list at a glance. Linked from docs/automations/README.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(automations): add Bootstrap section to CHEATSHEET — bare-bones setup What's needed to get a fresh machine running the loop, separated from the daily-loop reference. Three checklists: - Repo side: Python + Pillow + .env config - ComfyUI side: was-node-suite + the three required Text Multiline nodes, with exact titles - First card: /make-card → expect _AI/card.png Deliberately skips model-specific setup (checkpoints, LoRAs, per-model custom nodes) — those live in each model's handoff-checklist.md. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(automations): skip *loop* files in auto-discovery + watcher The maintainer's workflow includes a "loop" stage in ComfyUI that composites the card next to the video and writes the result back into the working folder as `<base>_loop_<seq>.{mp4,json}`. Without a skip rule, auto-discovery treats this composited-card output as the newest input — the script then re-renders the card from yesterday's metadata. Now: any JSON or video file with `loop` in its stem (case-insensitive) is ignored by: - scripts/make_card.py — auto-discover newest JSON, newest video - scripts/watch_cards.py — polling glob Explicit invocation still works: `python scripts/make_card.py <wf_loop.json>` will render a card for that file specifically. Documents the rule in docs/automations/CONVENTIONS.md (new §7) and adds a one-line bullet to CHEATSHEET conventions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * style(automations): satisfy Ruff lint — split if-on-one-line, rename l, drop os CI's Lint & Smoke step flagged 17 violations across the new scripts. Cleaned up: - scripts/make_card.py: 11 × E701 (multi-statement lines) split every `if cond: stmt` onto two lines. - scripts/make_card.py: 5 × E741 (ambiguous variable `l`) renamed to `link` in the two link-dict comprehensions and their downstream lookups. - scripts/watch_cards.py: 1 × F401 (unused `os` import). No functional change — `make_card.py` re-renders the working folder's card identically post-cleanup. Verified `ruff check` returns "All checks passed!" on both files. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(automations): render time from ComfyUI log, not disk timestamps Disk mtimes don't reliably bracket the model run — the maintainer's workflow saves the workflow JSON manually after rendering, assembles QuickTime from EXRs separately, and ComfyUI's metadata bundle writes JSON + MP4 at the same instant. None of those give us actual render duration. Render time now sourced from ComfyUI's own log (`<comfyui-root>/user/comfyui.log`): - Find the latest `saving images: 100%` line → pair it with the next `Prompt executed in X` → that's the most recent EXR-producing render wall-time. - Supports both `Prompt executed in 18.21 seconds` and `Prompt executed in HH:MM:SS` log formats. - Skips trivial follow-up prompts (e.g. audio-VAE load between render and /make-card invocation). Log path resolution: 1. `KOLOOK_COMFYUI_LOG` env var (explicit override). 2. Inferred from existing `KOLOOK_COMFYUI_DEV_PATH` → `<comfyui-root>/user/comfyui.log`. Fallbacks remain in priority order: EXR sequence span (oldest to newest in the latest version subfolder) → MP4 mtime delta → "pending". Docs: - .env.example: documents the optional KOLOOK_COMFYUI_LOG. - CONVENTIONS.md: new §4 "Render duration" describing the resolution chain; section numbers renumbered. - CHEATSHEET.md: updated the "Render time —" diagnostic. Lint: ruff clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |