2babbafb6ba15abf2739ca463f87a88c042aa0c8
2
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 malkuthro
|
2babbafb6b |
chore(security): pin Actions and harden test lock (#273)
* chore(security): pin Actions and update aiohttp * fix(security): generate universal test lock * fix(security): harden relock safeguards |
||
|
|
ba37ec3d41 |
feat: LTX Director 2.0.2 fork, keyframe grid-snap, and audio→video bind schedule (#259)
* feat(forks): LTX Director v2_0_2 fork + keyframe latent-grid snap (#258) Bump the WhatDreamsCost Koolook fork to upstream 2.0.2 (pinned fe09f73) in a new versions/v2_0_2 namespace, and add `snap_keyframes_to_grid` (default on) to fix hard-pinned image keyframes "jumping" at latent-bucket edges. Why: a Director image keyframe is injected as a single latent frame at the segment's left edge, quantized to the model temporal stride s (L(f)=0 if f==0 else 1+(f-1)//s). Pins near a bucket boundary round ambiguously and two pins in one bucket collide. Snapping each pin to its bucket center removes the ambiguity without changing which latent frame it lands on; colliding pins are bumped to the next free bucket with a warning. - forks/whatdreamscost_koolook/versions/v2_0_2/: vendored 2.0.2 sources with the v1.3.9 Koolook deltas carried forward (relay_overrides, audio_transcript_json, per-segment sigma — build_segments is byte-identical 1.3.9->2.0.2 so the sigma patch drops in verbatim), plus new pure keyframe_grid.py snapper wired into the insert_frames build. - Canonical LTXDirector__koolook rolls to 2.0.2 (entrypoint loads v2_0_2 last); LTXDirector__koolook_v1_3_2 alias preserved; v1.3.9 kept on disk. - LICENSE re-verified GPL-3.0 (2026-06-22). Manifest + THIRD_PARTY + CHANGELOG + audio-lipsync findings/CHEATSHEET updated. - 37 new unit tests for the snapper; 107 fork tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(forks): append Koolook LTX Director widgets last for clean load Move the v2_0_2 Koolook widgets (relay_overrides, audio_transcript_json, snap_keyframes_to_grid) to the end of the schema, after override_audio. They were inserted after img_compression, which shifted the last upstream widget when an upstream-2.0.x-saved workflow is loaded onto LTXDirector__koolook. With them last, an upstream graph's positional widgets_values map onto the upstream widgets untouched and only the Koolook widgets append with defaults (snap_keyframes_to_grid=on). Execution is unaffected (kwargs by name). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(forks): make v2_0_2 a clean upstream-2.0.2 replica + grid-snap only Per maintainer direction: the v2_0_2 Koolook LTX Director should faithfully replicate upstream 2.0.2 (so all latest features work) rather than carry the older customizations, which are unused. - Drop relay_overrides, audio_transcript_json, and the per-segment sigma patch. prompt_relay.py and patches.py are now vendored VERBATIM from upstream 2.0.2; _relay_overrides.py removed. - ltx_director.py is upstream 2.0.2 verbatim except two additive changes: the namespaced node ID LTXDirector__koolook, and the snap_keyframes_to_grid latent-bucket keyframe snap (issue #258), appended last in the schema. - v2_0_2 registers only LTXDirector__koolook; the legacy LTXDirector__koolook_v1_3_2 ID stays backed by the v1_3_9 package. v1.3.9 (with the older customizations) remains on disk for rollback. - Manifest / THIRD_PARTY / UPSTREAM_PIN / CHANGELOG updated to the clean scope. grid-snap unit tests unchanged (107 fork tests pass). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(forks): vendor upstream 2.0.2 timeline editor JS for LTXDirector__koolook The v2_0_2 Python was upstream-exact but the node still rendered with the old 1.3.9-era timeline editor (fewer toolbar actions, no IC-LoRA track, leaked *_second widgets, frame_rate NaN). Replace web/whatdreamscost_koolook/ ltx_director.js with upstream 2.0.2's js/ltx_director.js vendored verbatim, retargeted with only two edits: unique extension name 'LTXDirector_Koolook' and node guard nodeData.name === 'LTXDirector__koolook'. UX now matches upstream 2.0.2 exactly; only the Python differs (grid-snap). The old koolook base64-scrub deltas are dropped as obsolete — upstream 2.0.2 persists timeline media via imageFile/audioFile + /api/view URLs (saved timelines are already lean), and the localStorage quota guard stays in the global web/koolook_draft_guard.js (upstream does not override localStorage). Rewrote tests/sidebar/test_ltx_director_draft_guard.py to the new contract (no embedded guard, retargeted to the koolook node, pinned-upstream provenance). 110 fork+contract tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(scripts): resolve .env from main repo when dev-sync runs in a worktree Unrelated to the LTX work: plain `dev-sync` failed with "KOLOOK_COMFYUI_DEV_PATH not set" from any worktree because scripts/sync_to_dev.py only loaded `.env` from the worktree root, where the committed `.env` does not exist (it lives in the main checkout). sync_to_dev_audio.py already had the worktree->main fallback, which is why dev-sync-audio worked. Move that fallback into a shared sync_to_dev.find_dotenv() and use it in both scripts so they can't drift. Adds regression tests for the worktree fallback, the worktree-own-.env preference, and the not-a-worktree case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: gitignore root-level generated workflow JSONs Node-swapped/converted workflow copies (e.g. *.koolook.json) are dropped at the repo root during dev iteration; they're scratch deliverables, not package content. Ignore root-level *.json only (leading `/`), with a `!config.json` exception, so the 30+ tracked JSON in subdirs (test fixtures, web/, .claude/, docs/runs/*/workflow.json) stay tracked. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: LTX A/V Bind Schedule node — gate audio→video binding by sigma New Koolook node LTXAVBindSchedule decouples big motion from lip-sync in LTX 2.3. The AV transformer (comfy/ldm/lightricks/av_model.py, BasicAVTransformerBlock) adds audio into video via a gated `audio_to_video_attn` per block, run only when transformer_options["a2v_cross_attn"] is set. This node patches each block's audio_to_video_attn.forward to scale its output by a per-step gain, and a unet-function-wrapper computes that gain from the current sigma: early/high-sigma steps run near `early_gain` (coarse motion settles audio-blind), ramping (smoothstep) to full at low sigma (lips bind onto settled motion). One pass, no re-noise — the fix for keyframe transitions jumping on audio peaks. early_gain=1.0 reproduces stock behavior; a graph without the node is unchanged. Registered as a root node + added to RUNTIME_PATHS so dev-sync ships it. Pure ramp (a2v_gain) unit-tested (9 cases); model wiring validated by live render. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: LTX A/V Bind Schedule — add transition-window mode (lips synced + smooth) The global sigma ramp decoupled motion from lips but desynced the mouth (lip timing is coarse motion too, so holding audio out globally moved the mouth audio-blind). Add a WINDOWED mode: audio binds at full strength everywhere (lips sync across the whole clip) EXCEPT a +/-transition_window band around each keyframe transition, where the sigma ramp applies — so only those moments are motion-protected early and rejoined late. Implemented as a per-latent-frame gain on a2v_out: read grid_sizes from transformer_options to map frame-major video tokens -> latent frames, build a gain vector (1.0 outside bands, smoothstep-tapered to sigma_gain at each transition center), expand per-frame -> per-token, scale. New optional inputs: transition_frames, total_frames, transition_window. Empty transition_frames = prior global-ramp behavior. windowed_frame_gains unit-tested (7 cases). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: LTXAVBindSchedule — one-shot diagnostic log (shapes/grid/mode) Logs out.shape, grid_sizes, transition_fracs, and whether the per-frame mask fired vs fell back to scalar — so a single render reveals whether the a2v patch hooks the model and whether the windowed token->frame mapping aligns, instead of guessing blind. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(audio-lipsync): record validated a2v bind-schedule knee (0.35/0.25/0.55) Promote the finding that LTX 2.3's audio->video coupling must be sigma-ramped (not full-on) to keep motion smooth AND lips synced, with the validated knee early_gain=0.35 / bind_start=0.25 / bind_end=0.55 on the 8-step base pass, plus the trade-axis table and the per-frame-mode caveat (grid_sizes None + appended guide frames -> scalar fallback). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(forks): keyframe ease — strength-ramped neighbor pins to de-robotic transitions Adds `keyframe_ease` / `ease_falloff` to the Koolook LTX Director (default off). Each hard image pin keeps its exact center pose, but emits same-pose neighbor pins one latent bucket apart at decreasing strength (center*falloff**k), so the model glides into and out of the locked pose instead of snapping to a dead stop — softening the "stop/dissolve/stop" robotic feel without softening the pose. Implemented as a pure list expansion (`expand_keyframe_ease` in keyframe_grid.py, 6 unit tests) applied to guide_data after grid-snap, so the upstream guide node is untouched (no fork). Center stays exact; neighbors that fall off-clip or onto another center are skipped. Costs extra guide frames (more compute), hence opt-in. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(audio-lipsync): downgrade a2v knee to 'promising' — note lows-placement confound The 0.35/0.25/0.55 win was measured on a clip whose transitions sit on audio lows, so 'no jumps' may be the placement, not the node. Mark the controlled on-peak test (early_gain 1.0 vs 0.35 on the same on-peak layout) as still pending. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: mark vendored WhatDreamsCost fork files linguist-vendored The koolook Director ships upstream 2.0.2 code (the 8.5k-line timeline-editor JS + verbatim prompt_relay/patches) so the pack runs and distributes, but it's upstream, not authored here. Mark those linguist-vendored (drop from language stats) and the JS -diff (collapse the large blob in PR diffs). Our edited/new files stay attributed to us. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(ltx-av): drop unused `import math` (ruff F401) a2v_gain uses an explicit smoothstep polynomial, not math.*, so the import was dead. Unblocks the Lint & Smoke CI gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci(codeql): advanced setup that excludes vendored upstream code CodeQL default setup cannot read a config file, so it scanned the verbatim-vendored WhatDreamsCost editor (web/whatdreamscost_koolook/ ltx_director.js) and raised 4 high-sev DOM-XSS alerts in upstream code we do not own. Switch to advanced setup with a committed config whose paths-ignore mirrors the .gitattributes linguist-vendored block, so the scanner treats vendored upstream as out-of-scope while still fully scanning our own nodes and sidebar JS. Languages match the prior default-setup matrix (actions, javascript-typescript, python). NOTE: requires disabling "default setup" once in repo Settings before this workflow can run (GitHub rejects advanced runs while default setup is enabled). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(ci): scope vendored fork exclusions to v2_0_2 * fix(forks): preserve final LTX keyframe snap bucket --------- Co-authored-by: AI Machine <ai.machine@x.baconcph.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |