* feat(forks): LTX Director v2_0_2 fork + keyframe latent-grid snap (#258) Bump the WhatDreamsCost Koolook fork to upstream 2.0.2 (pinned fe09f73) in a new versions/v2_0_2 namespace, and add `snap_keyframes_to_grid` (default on) to fix hard-pinned image keyframes "jumping" at latent-bucket edges. Why: a Director image keyframe is injected as a single latent frame at the segment's left edge, quantized to the model temporal stride s (L(f)=0 if f==0 else 1+(f-1)//s). Pins near a bucket boundary round ambiguously and two pins in one bucket collide. Snapping each pin to its bucket center removes the ambiguity without changing which latent frame it lands on; colliding pins are bumped to the next free bucket with a warning. - forks/whatdreamscost_koolook/versions/v2_0_2/: vendored 2.0.2 sources with the v1.3.9 Koolook deltas carried forward (relay_overrides, audio_transcript_json, per-segment sigma — build_segments is byte-identical 1.3.9->2.0.2 so the sigma patch drops in verbatim), plus new pure keyframe_grid.py snapper wired into the insert_frames build. - Canonical LTXDirector__koolook rolls to 2.0.2 (entrypoint loads v2_0_2 last); LTXDirector__koolook_v1_3_2 alias preserved; v1.3.9 kept on disk. - LICENSE re-verified GPL-3.0 (2026-06-22). Manifest + THIRD_PARTY + CHANGELOG + audio-lipsync findings/CHEATSHEET updated. - 37 new unit tests for the snapper; 107 fork tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(forks): append Koolook LTX Director widgets last for clean load Move the v2_0_2 Koolook widgets (relay_overrides, audio_transcript_json, snap_keyframes_to_grid) to the end of the schema, after override_audio. They were inserted after img_compression, which shifted the last upstream widget when an upstream-2.0.x-saved workflow is loaded onto LTXDirector__koolook. With them last, an upstream graph's positional widgets_values map onto the upstream widgets untouched and only the Koolook widgets append with defaults (snap_keyframes_to_grid=on). Execution is unaffected (kwargs by name). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(forks): make v2_0_2 a clean upstream-2.0.2 replica + grid-snap only Per maintainer direction: the v2_0_2 Koolook LTX Director should faithfully replicate upstream 2.0.2 (so all latest features work) rather than carry the older customizations, which are unused. - Drop relay_overrides, audio_transcript_json, and the per-segment sigma patch. prompt_relay.py and patches.py are now vendored VERBATIM from upstream 2.0.2; _relay_overrides.py removed. - ltx_director.py is upstream 2.0.2 verbatim except two additive changes: the namespaced node ID LTXDirector__koolook, and the snap_keyframes_to_grid latent-bucket keyframe snap (issue #258), appended last in the schema. - v2_0_2 registers only LTXDirector__koolook; the legacy LTXDirector__koolook_v1_3_2 ID stays backed by the v1_3_9 package. v1.3.9 (with the older customizations) remains on disk for rollback. - Manifest / THIRD_PARTY / UPSTREAM_PIN / CHANGELOG updated to the clean scope. grid-snap unit tests unchanged (107 fork tests pass). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(forks): vendor upstream 2.0.2 timeline editor JS for LTXDirector__koolook The v2_0_2 Python was upstream-exact but the node still rendered with the old 1.3.9-era timeline editor (fewer toolbar actions, no IC-LoRA track, leaked *_second widgets, frame_rate NaN). Replace web/whatdreamscost_koolook/ ltx_director.js with upstream 2.0.2's js/ltx_director.js vendored verbatim, retargeted with only two edits: unique extension name 'LTXDirector_Koolook' and node guard nodeData.name === 'LTXDirector__koolook'. UX now matches upstream 2.0.2 exactly; only the Python differs (grid-snap). The old koolook base64-scrub deltas are dropped as obsolete — upstream 2.0.2 persists timeline media via imageFile/audioFile + /api/view URLs (saved timelines are already lean), and the localStorage quota guard stays in the global web/koolook_draft_guard.js (upstream does not override localStorage). Rewrote tests/sidebar/test_ltx_director_draft_guard.py to the new contract (no embedded guard, retargeted to the koolook node, pinned-upstream provenance). 110 fork+contract tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(scripts): resolve .env from main repo when dev-sync runs in a worktree Unrelated to the LTX work: plain `dev-sync` failed with "KOLOOK_COMFYUI_DEV_PATH not set" from any worktree because scripts/sync_to_dev.py only loaded `.env` from the worktree root, where the committed `.env` does not exist (it lives in the main checkout). sync_to_dev_audio.py already had the worktree->main fallback, which is why dev-sync-audio worked. Move that fallback into a shared sync_to_dev.find_dotenv() and use it in both scripts so they can't drift. Adds regression tests for the worktree fallback, the worktree-own-.env preference, and the not-a-worktree case. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: gitignore root-level generated workflow JSONs Node-swapped/converted workflow copies (e.g. *.koolook.json) are dropped at the repo root during dev iteration; they're scratch deliverables, not package content. Ignore root-level *.json only (leading `/`), with a `!config.json` exception, so the 30+ tracked JSON in subdirs (test fixtures, web/, .claude/, docs/runs/*/workflow.json) stay tracked. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: LTX A/V Bind Schedule node — gate audio→video binding by sigma New Koolook node LTXAVBindSchedule decouples big motion from lip-sync in LTX 2.3. The AV transformer (comfy/ldm/lightricks/av_model.py, BasicAVTransformerBlock) adds audio into video via a gated `audio_to_video_attn` per block, run only when transformer_options["a2v_cross_attn"] is set. This node patches each block's audio_to_video_attn.forward to scale its output by a per-step gain, and a unet-function-wrapper computes that gain from the current sigma: early/high-sigma steps run near `early_gain` (coarse motion settles audio-blind), ramping (smoothstep) to full at low sigma (lips bind onto settled motion). One pass, no re-noise — the fix for keyframe transitions jumping on audio peaks. early_gain=1.0 reproduces stock behavior; a graph without the node is unchanged. Registered as a root node + added to RUNTIME_PATHS so dev-sync ships it. Pure ramp (a2v_gain) unit-tested (9 cases); model wiring validated by live render. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: LTX A/V Bind Schedule — add transition-window mode (lips synced + smooth) The global sigma ramp decoupled motion from lips but desynced the mouth (lip timing is coarse motion too, so holding audio out globally moved the mouth audio-blind). Add a WINDOWED mode: audio binds at full strength everywhere (lips sync across the whole clip) EXCEPT a +/-transition_window band around each keyframe transition, where the sigma ramp applies — so only those moments are motion-protected early and rejoined late. Implemented as a per-latent-frame gain on a2v_out: read grid_sizes from transformer_options to map frame-major video tokens -> latent frames, build a gain vector (1.0 outside bands, smoothstep-tapered to sigma_gain at each transition center), expand per-frame -> per-token, scale. New optional inputs: transition_frames, total_frames, transition_window. Empty transition_frames = prior global-ramp behavior. windowed_frame_gains unit-tested (7 cases). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: LTXAVBindSchedule — one-shot diagnostic log (shapes/grid/mode) Logs out.shape, grid_sizes, transition_fracs, and whether the per-frame mask fired vs fell back to scalar — so a single render reveals whether the a2v patch hooks the model and whether the windowed token->frame mapping aligns, instead of guessing blind. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(audio-lipsync): record validated a2v bind-schedule knee (0.35/0.25/0.55) Promote the finding that LTX 2.3's audio->video coupling must be sigma-ramped (not full-on) to keep motion smooth AND lips synced, with the validated knee early_gain=0.35 / bind_start=0.25 / bind_end=0.55 on the 8-step base pass, plus the trade-axis table and the per-frame-mode caveat (grid_sizes None + appended guide frames -> scalar fallback). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(forks): keyframe ease — strength-ramped neighbor pins to de-robotic transitions Adds `keyframe_ease` / `ease_falloff` to the Koolook LTX Director (default off). Each hard image pin keeps its exact center pose, but emits same-pose neighbor pins one latent bucket apart at decreasing strength (center*falloff**k), so the model glides into and out of the locked pose instead of snapping to a dead stop — softening the "stop/dissolve/stop" robotic feel without softening the pose. Implemented as a pure list expansion (`expand_keyframe_ease` in keyframe_grid.py, 6 unit tests) applied to guide_data after grid-snap, so the upstream guide node is untouched (no fork). Center stays exact; neighbors that fall off-clip or onto another center are skipped. Costs extra guide frames (more compute), hence opt-in. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(audio-lipsync): downgrade a2v knee to 'promising' — note lows-placement confound The 0.35/0.25/0.55 win was measured on a clip whose transitions sit on audio lows, so 'no jumps' may be the placement, not the node. Mark the controlled on-peak test (early_gain 1.0 vs 0.35 on the same on-peak layout) as still pending. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: mark vendored WhatDreamsCost fork files linguist-vendored The koolook Director ships upstream 2.0.2 code (the 8.5k-line timeline-editor JS + verbatim prompt_relay/patches) so the pack runs and distributes, but it's upstream, not authored here. Mark those linguist-vendored (drop from language stats) and the JS -diff (collapse the large blob in PR diffs). Our edited/new files stay attributed to us. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(ltx-av): drop unused `import math` (ruff F401) a2v_gain uses an explicit smoothstep polynomial, not math.*, so the import was dead. Unblocks the Lint & Smoke CI gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci(codeql): advanced setup that excludes vendored upstream code CodeQL default setup cannot read a config file, so it scanned the verbatim-vendored WhatDreamsCost editor (web/whatdreamscost_koolook/ ltx_director.js) and raised 4 high-sev DOM-XSS alerts in upstream code we do not own. Switch to advanced setup with a committed config whose paths-ignore mirrors the .gitattributes linguist-vendored block, so the scanner treats vendored upstream as out-of-scope while still fully scanning our own nodes and sidebar JS. Languages match the prior default-setup matrix (actions, javascript-typescript, python). NOTE: requires disabling "default setup" once in repo Settings before this workflow can run (GitHub rejects advanced runs while default setup is enabled). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(ci): scope vendored fork exclusions to v2_0_2 * fix(forks): preserve final LTX keyframe snap bucket --------- Co-authored-by: AI Machine <ai.machine@x.baconcph.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
130 lines
5.5 KiB
Python
130 lines
5.5 KiB
Python
"""Guards on which modules ``dev-sync`` ships, so the loader can never point
|
|
at a file the install lacks.
|
|
|
|
The #183 incident (root cause behind #198): a module that ``__init__.py``
|
|
imported but ``scripts/sync_to_dev.py``'s ``RUNTIME_PATHS`` never copied raised
|
|
``ImportError`` at startup. After the #198 refactor the blast radius of such a
|
|
gap depends on *where* the import sits — a per-group node import degrades to
|
|
"that node absent", but the context probe (``koolook_versioning``) and the
|
|
install guard gate *all* registration. So the real invariant is: every module
|
|
``__init__.py`` imports at load must be shipped by ``RUNTIME_PATHS``. This test
|
|
derives those imports from ``__init__.py`` itself (rather than a ``k_*.py``
|
|
glob, which misses the non-``k_`` gates) so it can't drift.
|
|
"""
|
|
from __future__ import annotations
|
|
|
|
import ast
|
|
import importlib.util
|
|
from pathlib import Path
|
|
|
|
REPO_ROOT = Path(__file__).resolve().parents[2]
|
|
_SCRIPT = REPO_ROOT / "scripts" / "sync_to_dev.py"
|
|
_spec = importlib.util.spec_from_file_location("koolook_sync_to_dev_under_test", _SCRIPT)
|
|
assert _spec is not None and _spec.loader is not None
|
|
sync_to_dev = importlib.util.module_from_spec(_spec)
|
|
_spec.loader.exec_module(sync_to_dev)
|
|
|
|
|
|
def _init_load_time_module_deps() -> set[str]:
|
|
"""Top-level package modules/packages ``__init__.py`` imports at load.
|
|
|
|
Covers relative ``from .X import ...`` / ``from . import X`` statements and
|
|
the ``_merge_node_group(label, ".X")`` node-group imports. Returns the
|
|
first path segment of each (e.g. ``forks.radiance_koolook`` -> ``forks``),
|
|
the granularity ``RUNTIME_PATHS`` ships at. Absolute imports (the install
|
|
guard's out-of-package fallback) are level 0 and intentionally ignored.
|
|
"""
|
|
tree = ast.parse((REPO_ROOT / "__init__.py").read_text(encoding="utf-8"))
|
|
deps: set[str] = set()
|
|
for node in ast.walk(tree):
|
|
if isinstance(node, ast.ImportFrom) and (node.level or 0) >= 1:
|
|
if node.module: # from .pkg import ...
|
|
deps.add(node.module.split(".")[0])
|
|
else: # from . import a, b
|
|
deps.update(alias.name.split(".")[0] for alias in node.names)
|
|
elif (
|
|
isinstance(node, ast.Call)
|
|
and getattr(node.func, "id", None) == "_merge_node_group"
|
|
):
|
|
for arg in node.args:
|
|
if (
|
|
isinstance(arg, ast.Constant)
|
|
and isinstance(arg.value, str)
|
|
and arg.value.startswith(".")
|
|
):
|
|
deps.add(arg.value.lstrip(".").split(".")[0])
|
|
return deps
|
|
|
|
|
|
def _route_load_time_module_deps() -> set[str]:
|
|
tree = ast.parse((REPO_ROOT / "koolook_routes.py").read_text(encoding="utf-8"))
|
|
deps: set[str] = set()
|
|
for node in ast.walk(tree):
|
|
if isinstance(node, ast.ImportFrom):
|
|
if (node.level or 0) >= 1 and node.module:
|
|
deps.add(node.module.split(".")[0])
|
|
elif node.module and node.module.startswith("koolook_"):
|
|
deps.add(node.module.split(".")[0])
|
|
return deps
|
|
|
|
|
|
def test_init_load_deps_are_shipped_by_runtime_paths():
|
|
runtime_paths = set(sync_to_dev.RUNTIME_PATHS)
|
|
deps = _init_load_time_module_deps() | _route_load_time_module_deps()
|
|
# Sanity: the parse actually found the imports (guards against a refactor
|
|
# that renames _merge_node_group and silently makes this test vacuous).
|
|
assert "koolook_versioning" in deps and "k_ai_pipeline" in deps, (
|
|
f"__init__.py import parse looks broken — only found: {sorted(deps)}"
|
|
)
|
|
# A single-file module ships as "<dep>.py"; a package (e.g. forks) ships as
|
|
# its top-level dir entry.
|
|
missing = sorted(
|
|
dep
|
|
for dep in deps
|
|
if f"{dep}.py" not in runtime_paths and dep not in runtime_paths
|
|
)
|
|
assert not missing, (
|
|
"__init__.py imports these at load but dev-sync RUNTIME_PATHS does not "
|
|
f"ship them: {missing}. Add them to RUNTIME_PATHS in "
|
|
"scripts/sync_to_dev.py so a plain dev-sync can't 404 the loader."
|
|
)
|
|
|
|
|
|
def test_dev_sync_ships_manager_visible_package_metadata():
|
|
"""The live dev install should report the same pack version as the source.
|
|
|
|
ComfyUI Manager reads package metadata from ``pyproject.toml``. If dev-sync
|
|
skips it, the code can be current while the UI still reports an older
|
|
installed version.
|
|
"""
|
|
assert "pyproject.toml" in sync_to_dev.RUNTIME_PATHS
|
|
|
|
|
|
# --- find_dotenv: worktree -> main-repo .env fallback (dev-sync from a worktree) ---
|
|
|
|
def test_find_dotenv_prefers_worktree_env(tmp_path, monkeypatch):
|
|
wt = tmp_path / "wt"
|
|
wt.mkdir()
|
|
(wt / ".env").write_text("X=1\n", encoding="utf-8")
|
|
monkeypatch.setattr(sync_to_dev, "REPO_ROOT", wt)
|
|
assert sync_to_dev.find_dotenv() == wt / ".env"
|
|
|
|
|
|
def test_find_dotenv_falls_back_to_main_repo_from_worktree(tmp_path, monkeypatch):
|
|
main = tmp_path / "ComfyUI-Koolook"
|
|
(main / ".git" / "worktrees" / "wt").mkdir(parents=True)
|
|
(main / ".env").write_text("KOLOOK_COMFYUI_DEV_PATH=/somewhere\n", encoding="utf-8")
|
|
wt = main / ".claude" / "worktrees" / "wt"
|
|
wt.mkdir(parents=True)
|
|
gitdir = main / ".git" / "worktrees" / "wt"
|
|
(wt / ".git").write_text(f"gitdir: {gitdir.as_posix()}\n", encoding="utf-8")
|
|
monkeypatch.setattr(sync_to_dev, "REPO_ROOT", wt)
|
|
assert sync_to_dev.find_dotenv() == main / ".env"
|
|
|
|
|
|
def test_find_dotenv_none_when_no_env_and_not_a_worktree(tmp_path, monkeypatch):
|
|
plain = tmp_path / "plain"
|
|
plain.mkdir()
|
|
monkeypatch.setattr(sync_to_dev, "REPO_ROOT", plain)
|
|
assert sync_to_dev.find_dotenv() is None
|