main
7
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
ba37ec3d41 |
feat: LTX Director 2.0.2 fork, keyframe grid-snap, and audio→video bind schedule (#259)
* feat(forks): LTX Director v2_0_2 fork + keyframe latent-grid snap (#258) Bump the WhatDreamsCost Koolook fork to upstream 2.0.2 (pinned fe09f73) in a new versions/v2_0_2 namespace, and add `snap_keyframes_to_grid` (default on) to fix hard-pinned image keyframes "jumping" at latent-bucket edges. Why: a Director image keyframe is injected as a single latent frame at the segment's left edge, quantized to the model temporal stride s (L(f)=0 if f==0 else 1+(f-1)//s). Pins near a bucket boundary round ambiguously and two pins in one bucket collide. Snapping each pin to its bucket center removes the ambiguity without changing which latent frame it lands on; colliding pins are bumped to the next free bucket with a warning. - forks/whatdreamscost_koolook/versions/v2_0_2/: vendored 2.0.2 sources with the v1.3.9 Koolook deltas carried forward (relay_overrides, audio_transcript_json, per-segment sigma — build_segments is byte-identical 1.3.9->2.0.2 so the sigma patch drops in verbatim), plus new pure keyframe_grid.py snapper wired into the insert_frames build. - Canonical LTXDirector__koolook rolls to 2.0.2 (entrypoint loads v2_0_2 last); LTXDirector__koolook_v1_3_2 alias preserved; v1.3.9 kept on disk. - LICENSE re-verified GPL-3.0 (2026-06-22). Manifest + THIRD_PARTY + CHANGELOG + audio-lipsync findings/CHEATSHEET updated. - 37 new unit tests for the snapper; 107 fork tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(forks): append Koolook LTX Director widgets last for clean load Move the v2_0_2 Koolook widgets (relay_overrides, audio_transcript_json, snap_keyframes_to_grid) to the end of the schema, after override_audio. They were inserted after img_compression, which shifted the last upstream widget when an upstream-2.0.x-saved workflow is loaded onto LTXDirector__koolook. With them last, an upstream graph's positional widgets_values map onto the upstream widgets untouched and only the Koolook widgets append with defaults (snap_keyframes_to_grid=on). Execution is unaffected (kwargs by name). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(forks): make v2_0_2 a clean upstream-2.0.2 replica + grid-snap only Per maintainer direction: the v2_0_2 Koolook LTX Director should faithfully replicate upstream 2.0.2 (so all latest features work) rather than carry the older customizations, which are unused. - Drop relay_overrides, audio_transcript_json, and the per-segment sigma patch. prompt_relay.py and patches.py are now vendored VERBATIM from upstream 2.0.2; _relay_overrides.py removed. - ltx_director.py is upstream 2.0.2 verbatim except two additive changes: the namespaced node ID LTXDirector__koolook, and the snap_keyframes_to_grid latent-bucket keyframe snap (issue #258), appended last in the schema. - v2_0_2 registers only LTXDirector__koolook; the legacy LTXDirector__koolook_v1_3_2 ID stays backed by the v1_3_9 package. v1.3.9 (with the older customizations) remains on disk for rollback. - Manifest / THIRD_PARTY / UPSTREAM_PIN / CHANGELOG updated to the clean scope. grid-snap unit tests unchanged (107 fork tests pass). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(forks): vendor upstream 2.0.2 timeline editor JS for LTXDirector__koolook The v2_0_2 Python was upstream-exact but the node still rendered with the old 1.3.9-era timeline editor (fewer toolbar actions, no IC-LoRA track, leaked *_second widgets, frame_rate NaN). Replace web/whatdreamscost_koolook/ ltx_director.js with upstream 2.0.2's js/ltx_director.js vendored verbatim, retargeted with only two edits: unique extension name 'LTXDirector_Koolook' and node guard nodeData.name === 'LTXDirector__koolook'. UX now matches upstream 2.0.2 exactly; only the Python differs (grid-snap). The old koolook base64-scrub deltas are dropped as obsolete — upstream 2.0.2 persists timeline media via imageFile/audioFile + /api/view URLs (saved timelines are already lean), and the localStorage quota guard stays in the global web/koolook_draft_guard.js (upstream does not override localStorage). Rewrote tests/sidebar/test_ltx_director_draft_guard.py to the new contract (no embedded guard, retargeted to the koolook node, pinned-upstream provenance). 110 fork+contract tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(scripts): resolve .env from main repo when dev-sync runs in a worktree Unrelated to the LTX work: plain `dev-sync` failed with "KOLOOK_COMFYUI_DEV_PATH not set" from any worktree because scripts/sync_to_dev.py only loaded `.env` from the worktree root, where the committed `.env` does not exist (it lives in the main checkout). sync_to_dev_audio.py already had the worktree->main fallback, which is why dev-sync-audio worked. Move that fallback into a shared sync_to_dev.find_dotenv() and use it in both scripts so they can't drift. Adds regression tests for the worktree fallback, the worktree-own-.env preference, and the not-a-worktree case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: gitignore root-level generated workflow JSONs Node-swapped/converted workflow copies (e.g. *.koolook.json) are dropped at the repo root during dev iteration; they're scratch deliverables, not package content. Ignore root-level *.json only (leading `/`), with a `!config.json` exception, so the 30+ tracked JSON in subdirs (test fixtures, web/, .claude/, docs/runs/*/workflow.json) stay tracked. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: LTX A/V Bind Schedule node — gate audio→video binding by sigma New Koolook node LTXAVBindSchedule decouples big motion from lip-sync in LTX 2.3. The AV transformer (comfy/ldm/lightricks/av_model.py, BasicAVTransformerBlock) adds audio into video via a gated `audio_to_video_attn` per block, run only when transformer_options["a2v_cross_attn"] is set. This node patches each block's audio_to_video_attn.forward to scale its output by a per-step gain, and a unet-function-wrapper computes that gain from the current sigma: early/high-sigma steps run near `early_gain` (coarse motion settles audio-blind), ramping (smoothstep) to full at low sigma (lips bind onto settled motion). One pass, no re-noise — the fix for keyframe transitions jumping on audio peaks. early_gain=1.0 reproduces stock behavior; a graph without the node is unchanged. Registered as a root node + added to RUNTIME_PATHS so dev-sync ships it. Pure ramp (a2v_gain) unit-tested (9 cases); model wiring validated by live render. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: LTX A/V Bind Schedule — add transition-window mode (lips synced + smooth) The global sigma ramp decoupled motion from lips but desynced the mouth (lip timing is coarse motion too, so holding audio out globally moved the mouth audio-blind). Add a WINDOWED mode: audio binds at full strength everywhere (lips sync across the whole clip) EXCEPT a +/-transition_window band around each keyframe transition, where the sigma ramp applies — so only those moments are motion-protected early and rejoined late. Implemented as a per-latent-frame gain on a2v_out: read grid_sizes from transformer_options to map frame-major video tokens -> latent frames, build a gain vector (1.0 outside bands, smoothstep-tapered to sigma_gain at each transition center), expand per-frame -> per-token, scale. New optional inputs: transition_frames, total_frames, transition_window. Empty transition_frames = prior global-ramp behavior. windowed_frame_gains unit-tested (7 cases). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: LTXAVBindSchedule — one-shot diagnostic log (shapes/grid/mode) Logs out.shape, grid_sizes, transition_fracs, and whether the per-frame mask fired vs fell back to scalar — so a single render reveals whether the a2v patch hooks the model and whether the windowed token->frame mapping aligns, instead of guessing blind. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(audio-lipsync): record validated a2v bind-schedule knee (0.35/0.25/0.55) Promote the finding that LTX 2.3's audio->video coupling must be sigma-ramped (not full-on) to keep motion smooth AND lips synced, with the validated knee early_gain=0.35 / bind_start=0.25 / bind_end=0.55 on the 8-step base pass, plus the trade-axis table and the per-frame-mode caveat (grid_sizes None + appended guide frames -> scalar fallback). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(forks): keyframe ease — strength-ramped neighbor pins to de-robotic transitions Adds `keyframe_ease` / `ease_falloff` to the Koolook LTX Director (default off). Each hard image pin keeps its exact center pose, but emits same-pose neighbor pins one latent bucket apart at decreasing strength (center*falloff**k), so the model glides into and out of the locked pose instead of snapping to a dead stop — softening the "stop/dissolve/stop" robotic feel without softening the pose. Implemented as a pure list expansion (`expand_keyframe_ease` in keyframe_grid.py, 6 unit tests) applied to guide_data after grid-snap, so the upstream guide node is untouched (no fork). Center stays exact; neighbors that fall off-clip or onto another center are skipped. Costs extra guide frames (more compute), hence opt-in. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(audio-lipsync): downgrade a2v knee to 'promising' — note lows-placement confound The 0.35/0.25/0.55 win was measured on a clip whose transitions sit on audio lows, so 'no jumps' may be the placement, not the node. Mark the controlled on-peak test (early_gain 1.0 vs 0.35 on the same on-peak layout) as still pending. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: mark vendored WhatDreamsCost fork files linguist-vendored The koolook Director ships upstream 2.0.2 code (the 8.5k-line timeline-editor JS + verbatim prompt_relay/patches) so the pack runs and distributes, but it's upstream, not authored here. Mark those linguist-vendored (drop from language stats) and the JS -diff (collapse the large blob in PR diffs). Our edited/new files stay attributed to us. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(ltx-av): drop unused `import math` (ruff F401) a2v_gain uses an explicit smoothstep polynomial, not math.*, so the import was dead. Unblocks the Lint & Smoke CI gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci(codeql): advanced setup that excludes vendored upstream code CodeQL default setup cannot read a config file, so it scanned the verbatim-vendored WhatDreamsCost editor (web/whatdreamscost_koolook/ ltx_director.js) and raised 4 high-sev DOM-XSS alerts in upstream code we do not own. Switch to advanced setup with a committed config whose paths-ignore mirrors the .gitattributes linguist-vendored block, so the scanner treats vendored upstream as out-of-scope while still fully scanning our own nodes and sidebar JS. Languages match the prior default-setup matrix (actions, javascript-typescript, python). NOTE: requires disabling "default setup" once in repo Settings before this workflow can run (GitHub rejects advanced runs while default setup is enabled). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(ci): scope vendored fork exclusions to v2_0_2 * fix(forks): preserve final LTX keyframe snap bucket --------- Co-authored-by: AI Machine <ai.machine@x.baconcph.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
malkuthroandClaude Fable 5
|
c6a155a033 |
fix(web): generation-agnostic Comfy draft-quota guard (#237)
* fix(web): generation-agnostic Comfy draft-quota guard "Failed to save workflow draft" toasts returned on ComfyUI frontend 1.44+ because draft storage moved to new localStorage key names (per-workspace V1 keys, then a V2 index+payload scheme) that the old exact-name guard embedded in ltx_director.js never matched - and the 1.44 V2 layer latches a session-wide storage-unavailable flag after one failed save+evict cycle, re-toasting on every edit until reload. Move the guard to its own global extension, web/koolook_draft_guard.js: - match draft keys by prefix (Comfy.Workflow.Draft) across all known generations, with unknown future keys as last-resort evictions - rescue quota failures inside localStorage.setItem (evict the oldest draft across generations, retry) before Comfy's own catch sees the error, so the save succeeds and the latch never trips - boot prune: drop the suffixed-V1 dead weight the frontend's V1->V2 migration never deletes, scope corrupt-key cleanup to the offending key, drop oversized entries, enforce a total draft budget - ship the guard with dev-sync-audio so a scoped sync cannot strand a dev install guard-less Behavior is covered by tests/js/test_draft_guard.mjs (12 scenarios, wired into CI next to the existing node smoke test); static integration contracts live in tests/sidebar/test_draft_guard.py. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(web): address deep-review findings on the draft guard - latch the "still full" hint toast once per session so a genuinely full origin cannot re-toast on repeated failing autosave attempts; regression assertion added to the exhaustion scenario - document the bounded-eviction termination contract and the post-rethrow storage-unavailable latch at the wrapper, plus the budget's V1-blob-only (1.5M) -> all-generations (2M) scope widening (also now stated in the CHANGELOG entry) - version-robust wording for the suffixed-V1 cleanup rationale: the shipped 1.44 migration leaves the keys behind (verified against the installed 1.44.19 dist - no removeItem exists there), later frontends clean up on successful migration, and an interrupted migration can leak the keys on any version - comment the intentional best-effort swallow in the toast helper Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
malkuthroandClaude Opus 4.8
|
447465ecd8 |
refactor(nodes): remove superseded LTX Director extra nodes + harden registry (#198) (#199)
* refactor(nodes): remove superseded LTX Director extra nodes + harden registry (#198) Remove the two experimental nodes layered on the WhatDreamsCost fork — KoolookAudioTranscriptTimeline (k_audio_timeline.py) and KoolookTimelineEditor (k_timeline_editor.py) — plus the transcribe_audio_timeline.py CLI and its test. The audio-transcript to Director-timing job now lives inside the fork via the Director's audio_transcript_json input, so the standalone nodes are redundant. Also drops the orphaned [audio]/faster-whisper extra and updates packaging, dev-sync, web, and docs references. Harden the node registry against the #183 failure: __init__.py now imports each node group independently and installs the /koolook/presets/* routes regardless of node-import results, so one missing or broken module degrades to 'that node absent' instead of zeroing every node and 404'ing the sidebar disk routes. A new test keeps every root k_*.py listed in dev-sync's RUNTIME_PATHS. Closes #198. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(nodes): ship loader gates via dev-sync-audio; generalize RUNTIME_PATHS guard (#198) Address the K-Flame review on #199: MEDIUM — the #198 refactor made koolook_versioning.py a single registration gate, but dev-sync-audio shipped the new __init__.py without it (and the k_*.py-glob guard test couldn't see it). Add koolook_versioning.py AND koolook_install_guard.py (whose absolute-import fallback raises uncaught) to AUDIO_PATHS so a scoped sync can never leave the loader pointing at a module the target lacks. Generalize the guard test: parse __init__.py's actual load-time import targets (relative imports + _merge_node_group calls) and assert each is in RUNTIME_PATHS, instead of globbing k_*.py which missed the non-k_ gates. Add an AUDIO_PATHS gate-coverage test. LOW — CHANGELOG now records the [audio]/faster-whisper + transcribe_audio_timeline.py removal as an external-contract break. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
malkuthro
|
1d9d40ab70 |
Add audio transcript timeline node (#189)
* Add audio transcript timeline node * Hook audio transcripts into Koolook Director * Add modular timeline audio timing workflow * fix(frontend): guard Comfy workflow draft cache * fix(audio): compose transcript timeline prompts * fix(audio): address transcript review findings * fix(ci): keep transcript tests lightweight * fix(timeline): restore image previews from saved paths * fix(timeline): scrub restored preview blobs |
||
|
malkuthro
|
a5b6494848 | fix(audio): remove stale director web sync folder (#188) | ||
|
malkuthro
|
031af80f43 |
feat(forks): upgrade Koolook Director to stable v1.3.9 (#187)
* feat(forks): upgrade Koolook Director to stable v1.3.9 * docs(forks): clarify Director upgrade compatibility |
||
|
malkuthroandClaude Opus 4.7
|
9f063345f7 |
feat(investigations): LTX23 audio-file lip-sync scaffold (#177) (#178)
* feat(investigations): scaffold #177 LTX23 audio-file lip-sync investigation Reorganize docs/investigations/ into numbered topic folders, each self-contained with narrative + upstream pin + patches + per-run snapshots. Add a generic sync helper that mirrors the dev-sync pattern for any investigation's third-party node modifications. Structure (new): docs/investigations/ 00_LTX23-base-1-step/README.md (rename of ltx-director-4k-transitions.md) 01_LTX23-audio-file-lipsync/ README.md -- problem, mechanism, patches, reversion, promotion upstream.json -- repo + version + pinned commit + sync target env + files patches/ -- source of truth for modified upstream files ltx_director.py -- adds `relay_overrides` multiline JSON input prompt_relay.py -- per-segment sigma per Prompt-Relay paper formula runs/ LOOP.md -- iteration protocol log.md -- rolling table (empty header for now) Sync helper: scripts/sync_investigation_patches.py <NN>_<topic> [--dry-run] - Reads upstream.json for target env var + file list - Backs up the target's current state once per day (<file>.bak.<YYYYMMDD>) - Worktree-aware: falls back to MAIN's .env via git rev-parse --git-common-dir - Prints a two-line report matching scripts/sync_to_dev.py shape - Per-investigation trigger phrase via `trigger:` key (sync-audio for #177) Env: .env.example -- documents new KOLOOK_WHATDREAMSCOST_PATH .env -- local path set Glossary updates (docs/reference/glossary.md): + Investigation folder + JSON file folder + Loop + Run + Investigation patches + sync_investigation_patches.py Closes the workflow gap from #177: MAIN is now the source of truth for investigation patches; the live install is a sync destination only. No manual file copying between iterations — `sync-audio` does it. * chore(ruff): exclude docs/investigations from lint The investigation patches under docs/investigations/*/patches/ are vendored upstream source (WhatDreamsCost-ComfyUI v1.3.2) carried so sync_investigation_patches.py can mirror them into the live install and so iterations can be diff'd against the pinned upstream commit. Linting them would force divergence from upstream and defeat that purpose. Mirrors the existing forks/ exclude precedent. Fixes the CI Lint & Smoke failure on PR #178. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor: per-task automation modules + WhatDreamsCost Koolook fork Pivot docs/automations/ from "one folder per model" to "one folder per automation module" so the same diffusion model can host several independent tasks side by side. Replace the parallel docs/investigations/ tree with a fork of WhatDreamsCost-ComfyUI under forks/. Automation modules (new layout) - docs/automations/LTX-2.3/base-1step/ the existing single-stage automation, moved from docs/automations/LTX-2.3/ with its README, findings, and handoff-checklist rescoped to the task. The 4k-transitions backstory that produced the module moves here as backstory/4k-transitions.md. The earlier docs/investigations/experiments/ card PNGs/HTML land in example-cards/. - docs/automations/LTX-2.3/audio-lipsync/ the audio-file lip-sync module previously living under docs/investigations/01_…/. README, handoff, findings stub, backstory/, and runs/{LOOP,log}.md, all referencing the new fork. - docs/automations/README.md + CHEATSHEET.md updated to describe the module-per-task pivot and the new tree shape. - docs/automations/CONVENTIONS.md unchanged (cross-cutting loop contract still holds). WhatDreamsCost Koolook fork (new) - forks/whatdreamscost_koolook/versions/v1_3_2/ UPSTREAM_PIN.yaml pinned to upstream e81223a (v1.3.2), GPL-3.0 verified 2026-05-28. ltx_director.py MODIFIED — adds the relay_overrides multiline JSON widget on LTXDirector. Moved from docs/investigations/01_…/patches/. prompt_relay.py MODIFIED — per-segment σ per the Prompt-Relay paper formula σ = (L − w_eff) / (2 · √ln(1/ε)). Moved from docs/investigations/01_…/patches/. patches.py UNMODIFIED — vendored verbatim from upstream because ltx_director.py imports detect_model_type and apply_patches from it (same precedent as radiance_koolook's color_helpers.py). - Registered as LTXDirector__koolook_v1_3_2 ("LTX Director (Koolook v1.3.2)") so it coexists with an installed copy of upstream WhatDreamsCost-ComfyUI. Wired into the root __init__.py NODE_CLASS_MAPPINGS merge. - forks/forks_manifest.yaml + forks/THIRD_PARTY.md updated with the new entry (attribution, change log, license verified date, exposed node IDs). Removed - docs/investigations/ entire tree retired; its contents either moved to automation modules or to forks/. - scripts/sync_investigation_patches.py no longer needed — fork code lives in the Koolook tree and is synced via the standard dev-sync flow. - KOLOOK_WHATDREAMSCOST_PATH in .env.example same reason. - docs/investigations in pyproject.toml's ruff extend-exclude added in |