nodes/exportAnything-ComfyUI-DiffusionGemmaPromptBuilder
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Add synchronized music-video production planning

Browse Source 
Add measured-audio, timed-lyrics, H3 lane planning, identity relay, upload-song routing, model-specific camera contracts, LTX long-horizon planning, workflow migrations, the validated V6 example, regression coverage, and an Unreleased changelog.
This commit is contained in:
danrh
2026-08-23 08:03:39 -05:00
parent 07f3dc254f
commit 8f23c95546
49 changed files with 44289 additions and 188 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.comfyignore
+3
View File
@@ -8,9 +8,12 @@ __pycache__/
.github/
docs/
tests/
benchmarks/
tools/
GITHUB_RELEASE_CHECKLIST.md
temp/
tmp/
runtime_artifacts/
# Keep GitHub demo media out of the Comfy Registry install archive.
*.mp4
.gitignore
+1
View File
@@ -9,5 +9,6 @@ venv/
output/
temp/
tmp/
runtime_artifacts/
.DS_Store
Thumbs.db
CHANGELOG.md
+55
View File
@@ -0,0 +1,55 @@
# Changelog
All notable changes to DiffusionGemma Director are recorded here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); version numbers continue to follow the package metadata in `pyproject.toml`.
## [Unreleased]
### Added
- Added 17 exported audio, timing, and production-planning nodes, bringing the registered ComfyUI surface to 33 nodes.
- Added decoded-song audition and locking with measured tempo, onset, tonal, clipping, silence, artifact, vocal-activity, recovery-window, excerpt, and whole-song diagnostics. Candidate and waveform-hash locks now fail closed instead of silently selecting different music.
- Added a custom-song path with a lazy upload node and independent **Generate with ACE-Step / Upload song** router. Uploaded songs reuse the established selector, excerpt, hash-lock, audio-guide, planner, and pristine final-mux path without waking dormant ACE candidate lanes.
- Added a single **Natural / audio-led sync**, **Dance / music sync**, or **Lyrics + lip sync** performance control shared by Director guidance, lyric selection, timed analysis, and H3 planning.
- Added lazy timed-lyrics analysis using local vocal separation and Whisper alignment. Only verified, soundtrack-hash-locked vocal events guide articulation; instrumental, missing, stale, weak, or failed evidence safely falls back to Natural/audio-led behavior.
- Added experimental ACE **Compose new / Cover reference** conditioning. Cover mode remains lazy and fails closed until compatible reference-audio conditioning is connected.
- Added a deterministic Project Master contract, audio-aware H3 generation-lane planner, separate H3 video-seed fanout, optional previous-lane relay reference, exact-frame assembler, and plan-only multi-format delivery manifest.
- Added dual-reference identity preparation and relay support. Two clean references can remain the identity authorities while a downscaled previous-lane tail is used only as optional continuity evidence.
- Added Experimental Long Horizon planning for LTX with four contiguous duration-scaled pacing phases, compact output budgets, advisory diagnostics, and an explicitly unexecuted offline 5-, 20-, and 60-second benchmark protocol and manifest.
- Added hash-guarded workflow migrations for the synchronized music-video graph, MiniMax-H3 expansion, Turbo/fast test path, dual-identity relay, timed lyrics V5, uploaded-song V6, and an isolated identity-relay smoke graph.
- Added the validated MiniMax-H3 synchronized music-video V6 graph as a checked-in example with a repository-local integrity and contract test, so the working production artifact is preserved independently of mutable Desktop and ComfyUI copies.
- Added regression coverage for audio QC and routing, timed lyrics, production planning, exact assembly, H3 dialogue and shot counts, reference-contract repair, identity relay, uploaded songs, camera capability, and long-horizon planning.
### Changed
- Separated MiniMax-H3 native `[Shot N]` blocks from duration-bounded generation lanes. A single H3 invocation may now retain several native shots, and up to four planned lanes can cover a 60-second project when the configured 15-second lane ceiling and capacity constraints permit it.
- Reworked H3 lane boundaries to prefer native cuts with measured recovery, then native or mixed evidence, with deterministic balanced seams as a safe fallback. Non-native seams carry the active source shot and rebase timestamped cues rather than changing the master timeline.
- Made Project Master duration, aspect, soundtrack hash, excerpt, lane ceiling, and deliverables authoritative while leaving native shot choreography to the Director and H3 target profile.
- Made MiniMax-H3 camera policy model-specific. H3 may preserve requested orbiting, swirling, sweeping, whip movement, pronounced parallax, and coherent compound paths; unspecified cameras inherit an explicit creativity-scaled direction instead of being rejected.
- Added **Stable** and **Advanced** camera capability to the LTX target contract. LTX keeps its conservative motion safeguards where needed without imposing those restrictions on H3.
- Updated SplatStage planning so requested genre, subgenre, era, instrumentation, groove, and production style override visual nightclub or dance cues. Scene evidence may shape visuals and lyrics but no longer forces unrelated music into EDM.
- Extended SplatStage duration and aspect handling, measured-audio Director context, source-passthrough semantics, production concepts, and reproducible song-seed fanout.
- Revised Director caching and workflow fixtures so unchanged verified branches can reuse their packet while explicit refresh remains a one-run diagnostic choice.
- Codified the tested MiniMax-H3 fast path around the pruned INT8 model, official four-step Turbo LoRA, separate video/audio sigma shifts, and Euler/simple sampling. EasyCache and TorchCompile remain disconnected bypasses in that workflow because the measured configuration reported no effective acceleration.
### Fixed
- Repaired MiniMax-H3 Ref2VA packets deterministically when model output contains malformed subject definitions, reference usage or retention rows, misplaced or short detailed descriptions, invalid cut timestamps, or omitted per-shot camera direction.
- Removed the one-shot Project Master mismatch that incorrectly compared H3 native shot blocks with the number of generation invocations.
- Prevented generic camera-safety text from making H3 output creatively anemic while retaining fail-closed validation for genuinely malformed camera contracts.
- Prevented Dance mode from forcing singing or mouth motion, and prevented raw written lyrics from acting as an unverified timing schedule. Lyrics mode now supplies articulation cues only during measured vocal intervals.
- Preserved soundtrack timing across lane-local lyric windows and exact-frame assembly, including explicit non-vocal handling for instrumental gaps.
- Reduced long-video identity drift by keeping clean identity references authoritative across lanes and making relay continuity optional, bounded, and independently lazy.
- Made missing ideal H3 seam evidence advisory rather than fatal while continuing to block invalid clocks, malformed shot structures, stale audio locks, impossible lane capacity, and bad identity manifests.
- Hardened workflow migrations with immutable-source hashes, additive policy markers, exact wiring checks, and regression tests for the validated Desktop and ComfyUI copies when those local artifacts are available.
### Repository hygiene
- Excluded generated runtime videos and identity/reference frames from Git and ComfyUI package archives. These files remain local and are not required by the source or tests.
- Kept migration tools and offline benchmark manifests in Git while excluding them from the install archive, and updated package metadata and the release checklist for the 33-node surface.
- Made the legacy external V5 Desktop/ComfyUI byte-equality audit explicitly opt-in; the default release suite now relies on the checked-in V6 workflow's hermetic hash and contract validation.
## [0.2.0] - 2026-08-07
### Added
- Introduced the model-specific DiffusionGemma Director core, grounding guard, fail-closed generation gates, LTX and MiniMax-H3 target profiles, and packaged example workflows.
GITHUB_RELEASE_CHECKLIST.md
+13 -9
View File
@@ -4,13 +4,15 @@ Use this checklist from the standalone `ComfyUI-DiffusionGemmaPromptBuilder` rep
## Product Boundary
- Keep the exported surface at sixteen node ids:
- Keep the exported surface at thirty-three node ids:
- eleven stable Director nodes: Model Loader, Context Hub, H3 Reference Context, Grounding Guard Settings, CoT Generator, JSON Splitter, Generation Gate, Branch Generation Gate, and the LTX-2.5, MiniMax-H3, and Ideogram 4 target profiles
- one deprecated universal Target Profile retained for saved-workflow compatibility
- two optional reference-preparation helpers: Reference Prep and H3 Reference Pair Prep
- two experimental SplatStage planners
- eleven audio/music-video controls: ten decoded-audio, source-routing, ACE, performance, and production-concept nodes plus Timed Lyrics Analyzer
- six production-planning controls: Project Master, Audio-Aware H3 Multi-Shot Planner, H3 Relay Reference Gate, Multi-Format Delivery Planner, H3 Shot Seed Fanout, and H3 Shot Assembler
- Keep the two SplatStage planners visibly experimental. Their mapping ids stay compatible, their display names end in `(Experimental)`, and their category is `prompt/diffusiongemma/experimental-motion-planning`.
- Confirm `NODE_CLASS_MAPPINGS` and `NODE_DISPLAY_NAME_MAPPINGS` contain the same sixteen ids.
- Confirm `NODE_CLASS_MAPPINGS` and `NODE_DISPLAY_NAME_MAPPINGS` contain the same thirty-three ids.
- Treat `ltx_reframe.py` and the reframe workflows as unexported prototypes. Do not describe them as installed nodes unless the package entry point and frontend are deliberately restored together.
- Do not bundle model weights or generated tensor dumps. The NVFP4 bridge remains source code in `nodes.py`.
@@ -36,9 +38,10 @@ node --check web/js/model_loader.js
node --test tests/*.mjs
```
- The unit suite must validate the exact sixteen-node surface, the dedicated target-node boundaries, the Director/optional/experimental category split, both bundled SplatStage schemas, and retained reframe helpers.
- The unit suite must validate the exact thirty-three-node surface, the dedicated target-node boundaries, the Director/optional/experimental/audio/production-planning category split, both bundled SplatStage schemas, and retained reframe helpers.
- The readiness/schema smoke must finish with `Prompt Builder readiness/schema smoke passed`.
- The JavaScript suite must exercise the shipped `model_loader.js`. No test may import the retired `ltx_reframe_layout.js`.
- The default release suite validates checked-in artifacts only. Set `DG_VERIFY_DEPLOYED_V5_ARTIFACTS=1` solely when intentionally auditing the superseded, mutable V5 Desktop and ComfyUI copies; divergence there is deployment evidence to reconcile, not a hermetic release gate.
- Require all three CI jobs to pass on the release commit.
## Manual Runtime Gates
@@ -49,23 +52,24 @@ node --test tests/*.mjs
python proof_gates.py --model-path C:\ComfyUI\models\LLM\diffusiongemma-26B-A4B-it-NVFP4
```
2. Fresh-install the repository in a clean ComfyUI `0.30.1` custom-nodes folder and confirm all sixteen nodes appear: eleven stable Director nodes, one legacy compatibility target, two optional reference-preparation nodes, and two SplatStage planners only under the experimental motion-planning category.
2. Fresh-install the repository in a clean ComfyUI `0.30.1` custom-nodes folder and confirm all thirty-three nodes appear: eleven stable Director nodes, one legacy compatibility target, two optional reference-preparation nodes, two SplatStage planners only under the experimental motion-planning category, eleven audio/music-video controls, and six production-planning controls.
3. Copy `examples/assets/character_motion_transfer` to `ComfyUI/input/character_motion_transfer` and load `examples/07_ltx23_character_motion_transfer.json`.
4. Load the MiniMax H3 T2VA and Ref2VA examples and confirm the Generation Gate blocks invalid or empty prompt packets.
5. Treat longer experimental SplatStage and reframe workflows as separate opt-in validation, not as evidence for the stable Director compatibility promise.
5. Load `examples/15_minimax_h3_ref2va_music_video_v6.json`, replace both unbundled identity-image choices, and verify the default ACE source remains lazy with a blank upload. Exercise Upload song, Natural, Dance, Lyrics, and optional relay as separate branch tests; do not infer a successful media render from graph validation alone.
6. Treat longer experimental SplatStage and reframe workflows as separate opt-in validation, not as evidence for the stable Director compatibility promise.
## Files Tracked on GitHub
- Package entry point and source: `__init__.py`, `nodes.py`, `ltx_reframe.py`, `proof_gates.py`
- Package entry point and source: `__init__.py`, `nodes.py`, `audio_production_nodes.py`, `production_planning_nodes.py`, `timed_lyrics_nodes.py`, `ltx25_contract.py`, `ltx_reframe.py`, `proof_gates.py`
- Metadata and policy: `README.md`, `LICENSE`, `requirements.txt`, `pyproject.toml`, `.gitignore`, `.comfyignore`, this checklist
- Runtime contracts: `schemas/`, `web/`, `scripts/`
- Verification: `tests/`, `.github/workflows/ci.yml`
- Publishing: `.github/workflows/publish_action.yml`
- Documentation, current examples, and GitHub demo assets: `docs/`, `examples/`
- Documentation, current examples, workflow migrations, offline benchmarks, and GitHub demo assets: `docs/`, `examples/`, `tools/`, `benchmarks/`
## Registry Archive
- Keep `.github/`, `docs/`, `tests/`, caches, temporary files, and large demo video/GIF files out of the Registry archive through `.comfyignore`.
- Keep `.github/`, `docs/`, `tests/`, `tools/`, `benchmarks/`, runtime artifacts, caches, temporary files, and large demo video/GIF files out of the Registry archive through `.comfyignore`.
- Confirm both SplatStage schemas remain in the Registry archive because the experimental planners load them at runtime.
- Inspect the archive before publishing and confirm it contains no model weights, local paths, logs, caches, or generated outputs.
- Inspect the archive before publishing and confirm it contains no model weights, migration-only local paths, logs, caches, or generated outputs.
- Publish only after the GitHub release commit and version metadata agree.
README.md
+63 -6
View File
@@ -4,13 +4,13 @@
# ComfyUI DiffusionGemma Prompt Builder
DiffusionGemma Director provides a stable five-node prompt-authoring core for ComfyUI: choose one model-specific Target Profile node between Context Hub and the shared Director/validation path. Four stable Director companion nodes and two optional reference-preparation nodes extend that core. The package also exports two experimental SplatStage motion-planning nodes. DiffusionGemma writes and validates prompts or blueprints; it is not the downstream image or video sampler.
DiffusionGemma Director provides a stable five-node prompt-authoring core for ComfyUI: choose one model-specific Target Profile node between Context Hub and the shared Director/validation path. Four stable Director companion nodes, two optional reference-preparation nodes, eleven audio/music-video production controls, and six production-planning controls extend that core. The package also exports two experimental SplatStage motion-planning nodes. DiffusionGemma writes and validates prompts or blueprints; it is not the downstream image or video sampler.
The primary stable workflow is the working LTX 2.3 character motion-transfer graph in `examples/07_ltx23_character_motion_transfer.json`. Additional Director examples cover native MiniMax H3 T2VA and Ref2VA paths.
The primary stable workflow is the working LTX 2.3 character motion-transfer graph in `examples/07_ltx23_character_motion_transfer.json`. Additional Director examples cover native MiniMax H3 T2VA and Ref2VA paths. The accumulated synchronized music-video production graph is preserved in `examples/15_minimax_h3_ref2va_music_video_v6.json`; replace its two saved `LoadImage` choices with your own primary identity image and same-person contact sheet before running it.
## Exported Nodes
Sixteen nodes are registered in ComfyUI. Their stable, compatibility, optional, and experimental surfaces are intentionally separate.
Thirty-three nodes are registered in ComfyUI. Their stable, compatibility, optional, and experimental surfaces are intentionally separate.
### Stable Director Assistant
@@ -19,7 +19,7 @@ The five-node core path used by the primary LTX workflow is:
- `DiffusionGemmaModelLoader` - loads the local Transformers/NVFP4 DiffusionGemma runtime.
- `DiffusionGemmaContextHub` - combines user text, optional image/video inputs, and optional `visual_description` into one `DG_CONTEXT`.
- Choose exactly one model-specific target node:
- `DiffusionGemmaLTX25TargetProfile` - LTX generation mode, duration, style, joint audio/video guidance, and negative prompt controls.
- `DiffusionGemmaLTX25TargetProfile` - LTX generation mode, duration, Experimental Long Horizon planning, style, joint audio/video guidance, and negative prompt controls.
- `DiffusionGemmaMiniMaxH3TargetProfile` - H3 T2VA/Ref2VA mode, duration, audio, shot count, dialogue, and integrated exclusion controls.
- `DiffusionGemmaIdeogram4TargetProfile` - image aspect ratio, render style, exact visible text, JSON/prose representation, and negative prompt controls. It deliberately has no duration or audio widgets.
- `DiffusionGemmaCoTGenerator` - asks DiffusionGemma for the final prompt packet and returns final JSON plus debug outputs.
@@ -48,6 +48,46 @@ These larger planners are exported for opt-in SplatStage work, but they are not
Both are labeled `(Experimental)` and appear under `prompt/diffusiongemma/experimental-motion-planning` in ComfyUI. Their schemas ship with this repository, while executing a resulting show still requires the appropriate downstream SplatStage workflow and nodes.
`DiffusionGemmaSplatStagePlanner` accepts any positive finite planning duration and any positive `W:H` ratio or `WxH` resolution expression. These values guide the creative blueprint only; they do not override a downstream renderer's frame clock or raster. The synchronized LTX + ACE workflow supplies those settings independently. Classic SplatStage production graphs remain fixed to their own legacy clip and resolution contract.
### Decoded-Audio Production Controls
These controls appear under `prompt/diffusiongemma/audio-production` and support the synchronized ACE-Step + LTX workflow:
- `DiffusionGemmaMusicProductionConcept` is the single shared choice for both the SplatStage Planner and Song Blueprint Router. **Source passthrough means the authored ACE song blueprint passes through unchanged; it is not file-audio passthrough.** Source passthrough and Joint video-safe plan resolve to one ACE song; Audition and select activates a user-selected two to four candidates, with two as the bounded default.
- `DiffusionGemmaSongSeedFanout` derives four reproducible song-candidate seeds from a song seed that remains independent of the LTX sampling seed.
- `DiffusionGemmaUploadSong` provides a lazy-safe upload/selection widget for a custom soundtrack. A blank uploader cannot invalidate an ACE run, and a selected file is confined to ComfyUI's input directory before decoding.
- `DiffusionGemmaSongSourceRouter` is the independent soundtrack-origin toggle: **Generate with ACE-Step** preserves the existing candidate lanes, while **Upload song** keeps ACE audio generation dormant and submits the custom waveform to the same selector, excerpt, and hash-lock path. Uploaded BPM is optional (`0` uses signal-only analysis); uploaded lyrics are used only as downstream lyric evidence and may remain blank for Natural or Dance mode.
- `DiffusionGemmaAudioCandidateSelector` lazily renders only the requested candidate lanes, measures the decoded waveforms, selects a usable excerpt, and blocks LTX when no candidate passes. It can lock a candidate or exact waveform SHA-256 without silently substituting another song.
- `DiffusionGemmaLTXAudioGuide` separates the audio that conditions LTX from the untouched final soundtrack. `full_mix` is an identity path, `sync_safe` retains timing and center/vocal information while softening transient pressure, and `vocal_only` requires a connected vocal stem.
- `DiffusionGemmaLTXPerformancePrompt` sits after the validated Director gate and changes only prompt text. **Natural / audio-led sync** adds no lyric schedule and lets connected audio govern whether articulation naturally occurs. **Dance / music sync** deliberately suppresses singing, speaking, mouthing, and lip-sync while retaining beat-, phrase-, and dynamics-aware body motion. **Lyrics + lip sync** supplies a bounded authored-lyric window only as lexical/pronunciation candidates; it no longer orders a subject to sing every selected line. The node never changes the conditioning waveform or pristine final soundtrack.
- `DiffusionGemmaMusicVideoPerformanceMode` makes Natural, Dance, or Lyrics one shared choice before the video Director. It is the single authoritative mode supplied to target guidance, lyric-window selection, timed-lyrics analysis, and H3 lane planning.
- `DiffusionGemmaTimedLyricsAnalyzer` is lazy: Natural and Dance do not evaluate its separator or Whisper inputs. Lyrics mode validates a local Demucs vocal stem, transcribes only detected vocal intervals in chunks no longer than 15 seconds, aligns those phrases monotonically to authored lyric lines, and emits a song-hash-, lyric-hash-, and excerpt-locked timing report. Separator fallbacks, silence, weak alignment, stale evidence, or transcription/model errors produce a safe Natural/audio-led report instead of blocking video or inventing singing. Confirmed instrumental intervals are explicit non-vocal evidence.
- `DiffusionGemmaACEReferenceMode` and `DiffusionGemmaACECoverConditioning` expose a lazy Compose/Cover experiment. Cover fails closed until a valid reference-audio latent is connected and requires ACE semantic-code generation to be disabled.
The production concepts are deliberately different workflows, not post-generation repairs. The SplatStage Planner receives the Context Hub image pixels on a pixel-capable backend, so its song, lyrics, performer, environment, palette, and effects can be co-authored from the same image and creative brief used by the video Director. Source passthrough preserves one jointly authored `ace_tags`/lyrics/BPM/key/meter bundle. Joint video-safe plan asks DiffusionGemma to author lower transient, tonal, section, and lyric pressure before ACE generates anything. Audition and select keeps that coherent bundle intact, varies only the deterministic song seed, then judges the waveform that actually exists. The Song Blueprint Router never injects chord symbols, BPM, key, or theory prose into `ace_tags`, and it never compacts already-authored lyrics.
An explicitly requested soundtrack genre, subgenre, era, instrumentation, groove, or production style is authoritative over the image's visual setting. Neon, nightclub, dance, party, wardrobe, lighting, and other scene evidence may shape the visual lanes and lyrical imagery, but they do not turn another requested genre into EDM or a crossover. Words such as “juxtaposed” and “against” mean audiovisual contrast; the Planner creates a musical fusion only when the brief explicitly asks for one. Lyrics use ACE bracketed section labels such as `[Verse]` and `[Chorus]`.
Decoded-audio QC is a lightweight CPU signal heuristic, not transcription, source separation, note-level tuning proof, or a universal definition of good music. A half-time accent is accepted as an advisory when its canonical tempo is within 5% of the requested BPM. A canonically aligned double-time reading is advisory when canonical error is within 5%, excerpt onset pressure is at most 4.0/s, vocal activity is present, and at least two of four supporting signals pass: visual-recovery coverage, excerpt score, tonal-family consistency, or transition stability. It remains score-penalized and visibly advisory. Dense or weakly supported double-time, canonically misaligned or undetected tempo, dense selected excerpts, source-channel clipping, silence, broadband noise/artifact-like spectra, and strong tonal discontinuity remain blocking. `minimum_score` is only the weighted numeric floor and cannot bypass those hard QC checks. Automatic selection favors direct/aligned-half candidates, then recovery coverage for accepted double-time candidates, the actual LTX excerpt score, and the whole-song score. Listen to the selected song, then lock its candidate/hash and excerpt start for repeatable production. A failed lock blocks rather than falling back.
Uploaded songs use a separate source-locked posture because the user has already chosen the music. Invalid shape, excessive decoded size/duration, a track shorter than the requested excerpt, silence, clipping, and artifact-like broadband noise remain blocking. Tempo interpretation, vocal-presence proxy, onset pressure, tonal movement, and the generated-candidate production score remain measured and visible but advisory; they cannot replace or reject an otherwise technically usable uploaded song. ComfyUI still includes linked lazy ancestors in cache signatures, so changing a dormant upload selection may invalidate downstream cache entries even though the file is not decoded and ACE audio remains the only executed source.
In the synchronized workflow, the selected report is also supplied to Director so camera and body motion can follow the measured excerpt instead of requested metadata. The selected pristine excerpt remains the final muxed soundtrack. A separate timing-identical guide is Audio-VAE encoded with a zero noise mask and supplied to both LTX sampling stages; the original waveform is never replaced by the guide. `A2V influence` remains neutral at `1.0` by default and should be tested cautiously with the same locked song and video seed.
### Production Planning Controls
These controls appear under `prompt/diffusiongemma/production-planning` and provide the first project-orchestration layer for synchronized MiniMax-H3 production:
- `DiffusionGemmaProjectMasterContract` locks the approved song hash plus excerpt window, brief, duration, master aspect, H3 generation-lane ceiling, and requested deliverables into a deterministic project manifest. Its legacy `recommended_h3_shot_count` output means the number of duration-bounded H3 invocations, not the number of native camera cuts.
- `DiffusionGemmaAudioAwareMultiShotPlanner` divides a validated Ref2VA prompt into one to four H3 generation segments. One segment may preserve several consecutive native `[Shot N]` blocks. Boundary selection prefers native shot boundaries with measured low-density recovery, then native-only and mixed native/recovery sequences. If none can obey every hard lane-duration limit, the planner uses deterministic duration-balanced seams, records each seam's literal provenance, and carries the source shot already active at a non-native seam into the next lane so later native timestamps stay exact. Timestamped sub-cues are sliced with that carry: the latest elapsed cue becomes explicit opening-state history, in-lane cues are rebased, and future cues are deferred to the later lane instead of blocking or firing early. Each segment also carries a local clock, exact retained-frame allocation, continuity contract, and the shared Natural, Dance, or Lyrics performance mode. Lyrics mode accepts only a validated timed-lyrics report, clips verified events into each lane's local clock, marks fully instrumental lanes non-vocal, and otherwise falls back to Natural/audio-led behavior.
- `DiffusionGemmaH3RelayReferenceGate` optionally routes the retained tail of the previous H3 lane as a low-budget continuity reference for the next lane. The original identity references remain authoritative, relay is identity-first and off by default, and disabled downstream lanes do not request upstream media.
- `DiffusionGemmaH3ShotSeedFanout` derives deterministic per-generation-lane video seeds from the separate H3 root seed and project id.
- `DiffusionGemmaH3ShotAssembler` lazily requests only the planned H3 lanes, trims every decoded segment at its retained cut, verifies the master clock and soundtrack duration, concatenates exact frames, and passes the pristine audio object through unchanged.
- `DiffusionGemmaMultiFormatDeliveryPlanner` records requested reframes and cutdowns as plan-only deliverables; it never claims alternate media was rendered.
The Project Contract is authoritative for aspect, maximum H3 generation-segment length, and resulting lane count. Director and the MiniMax-H3 Target Profile remain authoritative for the unique native `[Shot N]` schedule inside those lanes. Missing ideal cut evidence is advisory rather than fatal; the plan exposes boundary-policy revision 2, selected strategy, evidence booleans, carry-in fragments, realized lane durations, and whether the non-binding preferred duration was actually realizable. Structural errors such as invalid clocks, duplicate/gapped/empty native shot blocks, out-of-range timestamps, stale audio locks, malformed identity manifests, impossible minimum/maximum limits, or more than four required lanes still fail closed. Missing, stale, weak, or malformed lyric timing also remains non-fatal: the planner records a warning and uses Natural/audio-led articulation. Written lyrics are never treated as a timing schedule. H3-generated audio is intentionally discarded; only the locked ACE excerpt is muxed into the assembled video.
The former universal `DiffusionGemmaTargetProfile` remains registered and marked deprecated so existing saved workflows continue to load with their exact widget order. New workflows should use one of the three model-specific targets. Other compatibility and utility classes remain in source where exported nodes reuse their behavior internally. The `ltx_reframe.py` module and LTX reframe workflows are retained as unexported prototypes; the retired `ltx_reframe_layout.js` frontend is not part of this release.
## Packaged Workflow
@@ -120,6 +160,12 @@ Use `DiffusionGemma LTX-2.5 Target Profile` and keep `generation_mode` on `Auto
Connect the same first/last images to the native LTX conditioning nodes. Director uses the Context Hub copies only for visual analysis and contract validation; it does not secretly wire the LTX sampler. For I2V and first+last-frame work, Director requires one continuous take, checks the generated opening against verified first-frame facts, and rejects cuts or shot/action overload that cannot fit the selected duration. The compiler asks every shot to open with one of LTX's six canonical types (`extreme wide shot`, `wide shot`, `medium shot`, `medium close-up`, `close-up`, or `extreme close-up`) before describing its camera path; a later destination scale does not replace that opening type. A connected conditioning frame supplies otherwise-unstated opening scale and viewpoint geometry instead of making Director guess and block, while frame-free T2V and every post-cut shot remain strict. Compatible simultaneous or sequential camera-motion phases may form one physically continuous path and evolve to a new endpoint scale without becoming a cut; unusually dense camera choreography is diagnostic advice, not a hard numeric gate. Generation Gate reports corrective sentences rather than requiring users to decode validation identifiers.
`Experimental Long Horizon` is an LTX Target Profile planning control for unusually long continuous generations. Its default is `Off`, preserving saved-workflow behavior. `Auto (>20 seconds)` activates only when the resolved planning duration is strictly greater than 20 seconds; exactly 20 seconds remains on the standard contract. `On` forces the policy at any duration.
When active, Director silently plans four duration-scaled phases—establish, commit, sustain/reveal, and settle/hold—then compiles them into one continuous paragraph of no more than 200 words and generally four to eight sentences. These phases are not chapters, shots, scene changes, emitted headings, or timecodes. I2V planning keeps only the minimal critical first-frame anchors needed for continuity, selects one physically continuous camera intention, preserves explicitly requested transformations, and defines a terminal composition. Missing phase, conservation, or terminal cues produce non-blocking diagnostics rather than a Generation Gate failure.
The toggle improves semantic pacing; it does not add geometric conditioning or guarantee one-minute object reconstruction. Pair it with a first/last frame or a duration-matched depth/control sequence when the final composition or camera geometry must be constrained. The offline [5/20/60-second benchmark protocol](benchmarks/ltx25_long_horizon_v1/README.md) defines the unexecuted bare/current/long-horizon/structural comparison and preserves both sampler seeds without claiming render results that have not been collected.
All five creativity modes apply to LTX I2V and first+last-frame video. A still anchor fixes its visible pose, framing, and camera geometry only at that anchored instant; it does not prove a static future path. An explicit user or verified-media camera instruction—including a locked/static hold—always wins. Faithful preserves requested camera speed, direction, and compound phases exactly, and chooses the least-invasive suitable path only when motion is unspecified. Editorial, cinematic, concept-art, and wild may author increasingly expressive, tempo-appropriate compound camera choreography when the future path is unspecified, while retaining the exact anchors, one continuous take, and no cuts. `creative_strength` scales that motion ambition without imposing a numeric phase cap.
The JSON Splitter's resolution controls are downstream sampler controls and do not invalidate the Director cache. Choose an explicit `resolution_aspect_ratio`, megapixel budget, and alignment multiple, then connect `resolution_width` and `resolution_height` to the native LTX generator. `Auto (target/source)` matches an attached LTX first frame to the nearest supported ratio and defaults frame-free LTX to `16:9`; it no longer inherits Ideogram's `1:1` default.
@@ -134,7 +180,7 @@ See [Grounding Guard](docs/GROUNDING_GUARD.md) for the mode contract, evidence s
`DiffusionGemma CoT Generator` has a `director_cache_mode` control. `reuse` (the default) lets ComfyUI keep an unchanged Director branch in its normal in-memory graph cache, so changing an LTX-only control does not rerun an independent MiniMax-H3 branch (and vice versa). It also builds a SHA-256 key from the model/checkpoint and processor identity, Director implementation and schemas, exact creative controls, Grounding Guard configuration, target profile, media metadata, and hashes of the ordered tensors actually sent to the processor. A verified ready packet is stored as small JSON/text in ComfyUI's private system-user cache. A disk hit is served before the full DiffusionGemma model is constructed or loaded and survives restarts, cache eviction, and equivalent Director nodes. The disk key does not include downstream H3 seed, steps, sampler, scheduler, LoRA, decode, preview, or save controls.
`refresh` forces a new Director run and atomically replaces an eligible disk entry; it does not randomize generation. Identical inputs with a fixed Grounding Guard/Director seed are intentionally reproducible, so change that seed (or its after-generate policy) when you want a new prompt variant. `creativity_mode` chooses the kind of compatible prompt detail and, for LTX video, camera choreography; `creative_strength` chooses its qualitative intensity and camera-path ambition. Neither is a sampling-temperature or RNG control. `off` disables disk lookup and writes and recomputes every queue. Because ComfyUI's in-memory cache cannot distinguish a ready packet from a failed or salvaged one, an unchanged blocked result in `reuse` remains blocked until an input changes or `refresh` is selected for one retry; the Branch Generation Gate status states this recovery step. Detailed grounding traces always bypass both reuse layers so their requested trace side effect is never silently skipped. Blocked, refused, salvaged, template-fallback, unverified-audit, and otherwise non-ready packets are not stored on disk. Cache files contain plaintext prompt/debug outputs; the default location is private, and `DG_DIRECTOR_CACHE_DIR` can select another local directory.
`refresh` forces a new Director run and atomically replaces an eligible disk entry; it does not randomize generation. It is a one-run diagnostic setting: queue once, then immediately return it to `reuse`, because leaving `refresh` selected reruns Director on every queue. Identical inputs with a fixed Grounding Guard/Director seed are intentionally reproducible, so change that seed (or its after-generate policy) when you want a new prompt variant. `creativity_mode` chooses the kind of compatible prompt detail and, for LTX video, camera choreography; `creative_strength` chooses its qualitative intensity and camera-path ambition. Neither is a sampling-temperature or RNG control. `off` disables disk lookup and writes and recomputes every queue. Because ComfyUI's in-memory cache cannot distinguish a ready packet from a failed or salvaged one, an unchanged blocked result in `reuse` remains blocked until an input changes or `refresh` is selected for one retry; the Branch Generation Gate status states this recovery step. Detailed grounding traces always bypass both reuse layers so their requested trace side effect is never silently skipped. Blocked, refused, salvaged, template-fallback, unverified-audit, and otherwise non-ready packets are not stored on disk. Cache files contain plaintext prompt/debug outputs; the default location is private, and `DG_DIRECTOR_CACHE_DIR` can select another local directory.
The JSON Splitter keeps its normal `ltx_prompt` and `negative_prompt` outputs fail-closed, but also exposes appended `candidate_ltx_prompt` and `candidate_negative_prompt` diagnostics. Those candidate sockets update even when validation blocks generation, which prevents a Preview Any node from appearing to show a stale prior prompt. They are inspection-only and must never be connected directly to a native generator; generation continues through the normal prompt outputs and a Generation Gate.
@@ -165,7 +211,9 @@ non_diegetic_music: ...
The first shot has no timestamp. Later cuts use increasing instants such as `[Shot 2] At 00:03.500, the camera cuts to...`, all inside the selected duration. Hard cuts are the default; cross-dissolves, fades, and wipes are used only when the brief requests them. Synchronized visible action, speech, and scene sounds stay in `integrated_multimodal_description`; ambient sound and Foley belong in `overall_soundscape`; audience-only score belongs in `non_diegetic_music`, where instrumentation, tempo/rhythm, and dynamic development are described concretely instead of as an abstract mood.
`shot_count` on the MiniMax-H3 Target Profile controls the number of generated shots for both T2VA and Ref2VA. `auto` honors an exact count written in the creative brief and otherwise lets Director choose the fewest useful shots. Select `1` through `12`, or select `custom` and type a value from `1` through `99` in `custom_shot_count`. The custom number is ignored unless the selector is set to `custom`. Any explicit value requires exactly that many consecutive `[Shot N]` sections and overrides conflicting count wording in the brief. A shot is a camera/cut segment, not necessarily a narrative beat, chapter, or scene change; multiple beats can occur inside one shot. Director validates the result and makes a bounded repair attempt; if the model still returns the wrong count, Generation Gate blocks it. Counts above 12 generally need more duration and Director output-token budget. This is a storyboard/compiler control, independent of the Grounding Guard's `evidence_token_budget`.
Every H3 shot also carries one explicit camera-behavior sentence. H3 does not inherit LTX's Stable / base-model Camera Capability: requested or reference-derived camera choreography is preserved, and non-faithful creativity modes may author expressive movement proportional to creative strength. Orbit/arc, roll/rotation, swirl, sweep or whip movement, controlled shake, pronounced parallax, and coherent compound paths are valid H3 tools. Moving-shot prose names the motion type, meaningful speed/amplitude, direction, subject relationship, and ending frame; compound phases must be physically compatible or explicitly ordered. The exact locked-off sentence remains only a deterministic last resort for a genuinely unspecified shot, so a valid complex path is never overwritten or paired with a contradictory static hold.
`shot_count` on the MiniMax-H3 Target Profile controls the number of native camera/cut blocks for both T2VA and Ref2VA. `auto` honors an exact count written in the creative brief and otherwise lets Director choose the fewest useful shots. Select `1` through `12`, or select `custom` and type a value from `1` through `99` in `custom_shot_count`. The custom number is ignored unless the selector is set to `custom`. Any explicit value requires exactly that many consecutive `[Shot N]` sections and overrides conflicting count wording in the brief. A shot is a camera/cut segment, not necessarily a narrative beat, chapter, scene change, or separate H3 generation pass; multiple native shots may execute inside one valid 4–15 second H3 clip. Do not connect the Project Master's generation-lane recommendation to this native shot-count control. Director validates the result and makes a bounded repair attempt; if the model still returns the wrong count, Generation Gate blocks it. Counts above 12 generally need more duration and Director output-token budget. This is a storyboard/compiler control, independent of the Grounding Guard's `evidence_token_budget`.
### MiniMax H3 dialogue controls
@@ -346,6 +394,13 @@ Place in:
ComfyUI/models/text_encoders
```
Optional MiniMax-H3 timed-lyrics analysis:
- [openai/whisper-large-v3-turbo](https://huggingface.co/openai/whisper-large-v3-turbo), placed as the complete local repository at `ComfyUI/models/whisper/whisper-large-v3-turbo`.
- Torchaudio's `HDEMUCS_HIGH_MUSDB_PLUS` checkpoint, installed through the torchaudio pipeline cache. On Windows this normally resolves below the user's `.cache/torch/hub/torchaudio/models` directory.
The V5 workflow keeps both branches lazy outside Lyrics mode. Its Whisper loader has `download_missing=False`; install the exact local snapshot before selecting Lyrics so a network change cannot silently alter a repeatable run.
Latent upscaler:
- [Lightricks/LTX-2.3 - ltx-2.3-spatial-upscaler-x2-1.1.safetensors](https://huggingface.co/Lightricks/LTX-2.3/blob/main/ltx-2.3-spatial-upscaler-x2-1.1.safetensors)
@@ -366,6 +421,8 @@ Install missing packs through ComfyUI Manager when possible:
- `comfyui_controlnet_aux`
- `comfyui_essentials`
- `comfyui-easy-use`
- `comfy-mtb` for the local Whisper loader
- `comfyui_fill-nodes` for Hybrid Demucs vocal separation
- `RES4LYF`
Core ComfyUI supplies several additional LTX nodes used by the workflow, including video loaders, LTX preprocessing/conditioning, samplers, latent/audio/video split and concat nodes, and latent upscaler loading.
__init__.py
+29 -1
View File
@@ -1,7 +1,35 @@
# Copyright (c) 2026 exportAnything. All rights reserved.
# SPDX-License-Identifier: MIT
from .nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
from .nodes import (
NODE_CLASS_MAPPINGS as _DIRECTOR_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as _DIRECTOR_NODE_DISPLAY_NAME_MAPPINGS,
)
from .audio_production_nodes import (
NODE_CLASS_MAPPINGS as _AUDIO_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as _AUDIO_NODE_DISPLAY_NAME_MAPPINGS,
)
from .production_planning_nodes import (
NODE_CLASS_MAPPINGS as _PRODUCTION_PLANNING_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as _PRODUCTION_PLANNING_NODE_DISPLAY_NAME_MAPPINGS,
)
from .timed_lyrics_nodes import (
NODE_CLASS_MAPPINGS as _TIMED_LYRICS_NODE_CLASS_MAPPINGS,
NODE_DISPLAY_NAME_MAPPINGS as _TIMED_LYRICS_NODE_DISPLAY_NAME_MAPPINGS,
)
NODE_CLASS_MAPPINGS = {
**_DIRECTOR_NODE_CLASS_MAPPINGS,
**_AUDIO_NODE_CLASS_MAPPINGS,
**_PRODUCTION_PLANNING_NODE_CLASS_MAPPINGS,
**_TIMED_LYRICS_NODE_CLASS_MAPPINGS,
}
NODE_DISPLAY_NAME_MAPPINGS = {
**_DIRECTOR_NODE_DISPLAY_NAME_MAPPINGS,
**_AUDIO_NODE_DISPLAY_NAME_MAPPINGS,
**_PRODUCTION_PLANNING_NODE_DISPLAY_NAME_MAPPINGS,
**_TIMED_LYRICS_NODE_DISPLAY_NAME_MAPPINGS,
}
WEB_DIRECTORY = "./web"
audio_production_nodes.py
+2900
View File
File diff suppressed because it is too large Load Diff
benchmarks/ltx25_long_horizon_v1/README.md
+40
View File
@@ -0,0 +1,40 @@
# LTX-2.5 Experimental Long Horizon benchmark
This directory defines an **unexecuted benchmark protocol** for the Director's
optional long-horizon policy. It is not a claim that the comparison has already
been rendered or scored. Before running it, map the four asset IDs listed in
the manifest to one first frame and three duration-matched depth guides. The
protocol then uses the same first frame at 5, 20, and 60 seconds so duration is
the principal changing condition.
The four variants are:
- `bare`: the one-line user brief with no Director expansion.
- `current_director`: the existing duration-aware LTX contract with long-horizon
planning off.
- `long_horizon`: Experimental Long Horizon forced on.
- `long_horizon_structural`: the same long-horizon prompt plus a duration-matched
depth guide. This fourth arm intentionally measures prompt plus structural
conditioning and is not part of the prompt-only A/B/C comparison.
Every arm keeps model, resolution, frame rate, sampler, scheduler, step counts,
CFG, negative prompt, and both sampler seeds fixed. The manifest uses 121, 481,
and 1,441 frames at 24 fps, corresponding to 5, 20, and 60 seconds under LTX's
`(frames - 1) / fps` convention.
The locked sampler settings mirror the audited 60-second Director run:
`euler_ancestral`, `bong_tangent`, 12 first-stage steps, and three refinement
evaluations. Results remain empty until all four variants have actually been
run for every listed seed pair and scored against the rubric.
For each seed pair, rate camera monotonicity, anchor persistence, duplication,
spatial drift, abrupt reframing, terminal-composition accuracy, prompt
adherence, and audio continuity. Rate at regular five-second checkpoints for
the 20- and 60-second cases. Treat a single successful seed as anecdotal; the
paired matrix is the acceptance evidence.
The manifest also records the prompt-contract rubric. Long-horizon prompts must
remain one paragraph of at most 200 words and generally four to eight
sentences. The four internal phases are planning concerns, not chapters, shots,
scene changes, emitted headings, or timecodes. Missing phase or terminal cues
are diagnostics and must not block generation.
benchmarks/ltx25_long_horizon_v1/manifest.json
+263
View File
@@ -0,0 +1,263 @@
{
"schema_version": "dg-ltx25-long-horizon-benchmark/1",
"name": "LTX-2.5 experimental long-horizon duration benchmark v1",
"description": "Controlled 5, 20, and 60 second I2V cases comparing a bare brief, the current Director contract, Experimental Long Horizon, and the same long-horizon prompt with structural conditioning.",
"execution_status": "protocol_only_unexecuted",
"asset_requirements": [
"mushroom_tableau_first_frame",
"mushroom_tableau_depth_5s",
"mushroom_tableau_depth_20s",
"mushroom_tableau_depth_60s"
],
"results": [],
"seed_pairs": [
{
"stage_1": 17,
"stage_2": 42
},
{
"stage_1": 101,
"stage_2": 420
},
{
"stage_1": 809,
"stage_2": 421
}
],
"variant_order": [
"bare",
"current_director",
"long_horizon",
"long_horizon_structural"
],
"cases": [
{
"id": "i2v_tableau_5s",
"mode": "image_to_video",
"duration_seconds": 5.0,
"fps": 24,
"frames": 121,
"brief": "Create a mind-bending shot grounded in the supplied first frame.",
"critical_anchors": [
"woman in a blue dress",
"chair",
"brick fireplace"
],
"terminal_composition": "A slightly wider elevated view holds on the unchanged anchor group.",
"expected_auto_long_horizon_active": false,
"fixed_generation_control": {
"model": "ltx-2.5-22b-distilled",
"text_encoder": "gemma4-12b-with-projection",
"precision": "int8-convrot",
"width": 1344,
"height": 768,
"sampler": "euler_ancestral",
"scheduler": "bong_tangent",
"stage_1_steps": 12,
"stage_2_steps": 3,
"video_cfg": 1.0,
"audio_cfg": 1.0,
"negative_prompt": "cuts, transitions, optical zoom, object duplication, subject replacement, subtitles, captions, logos, watermarks",
"prompt_enhancer": false,
"first_frame_asset_id": "mushroom_tableau_first_frame"
},
"variants": {
"bare": {
"prompt_producer": "bare_brief",
"long_horizon_mode": "off",
"prompt": "Create a mind-bending shot grounded in the supplied first frame.",
"structural_conditioning": "first_frame"
},
"current_director": {
"prompt_producer": "diffusiongemma_current_ltx_contract",
"long_horizon_mode": "off",
"prompt": "A wide shot begins from the exact supplied first-frame viewpoint, preserving the woman in a sleeveless blue dress beside the chair and brick fireplace in the surreal mushroom meadow. The camera slowly dollies backward and rises while keeping the woman centered. She brushes one mushroom stalk as the fire flickers and the grass moves in a light breeze. Natural parallax reveals more rooted mushrooms, accompanied by fireplace crackle, wind, and restrained lo-fi trip-hop. The camera finishes on a slightly wider elevated composition.",
"structural_conditioning": "first_frame"
},
"long_horizon": {
"prompt_producer": "diffusiongemma_experimental_long_horizon",
"long_horizon_mode": "on",
"prompt": "An eye-level wide shot begins from the supplied first frame, preserving the woman in the blue dress beside the chair and brick fireplace as the critical anchor group. Initially, for a brief opening moment, the camera holds while the flame flickers and grass moves in a light breeze. It then commits to one smooth backward-and-upward physical move with a fixed wide-angle lens, keeping the anchor group centered through natural parallax. Through most of the take, the surrounding rooted mushrooms are revealed by travel rather than growth or duplication, and steady wind and fireplace crackle continue. The woman, blue dress, chair, brickwork, mushroom colors, daylight, and relative positions remain stable, while only the surrounding field enters the widening frame. Wind, flame crackle, and distant birds persist as one seamless audio bed without any change of source or tempo. Only near the end does the movement decelerate into a wider elevated composition and hold on the unchanged tableau.",
"structural_conditioning": "first_frame"
},
"long_horizon_structural": {
"prompt_producer": "diffusiongemma_experimental_long_horizon",
"long_horizon_mode": "on",
"prompt": "An eye-level wide shot begins from the supplied first frame, preserving the woman in the blue dress beside the chair and brick fireplace as the critical anchor group. Initially, for a brief opening moment, the camera holds while the flame flickers and grass moves in a light breeze. It then commits to one smooth backward-and-upward physical move with a fixed wide-angle lens, keeping the anchor group centered through natural parallax. Through most of the take, the surrounding rooted mushrooms are revealed by travel rather than growth or duplication, and steady wind and fireplace crackle continue. The woman, blue dress, chair, brickwork, mushroom colors, daylight, and relative positions remain stable, while only the surrounding field enters the widening frame. Wind, flame crackle, and distant birds persist as one seamless audio bed without any change of source or tempo. Only near the end does the movement decelerate into a wider elevated composition and hold on the unchanged tableau.",
"structural_conditioning": "first_frame_plus_depth",
"structural_asset_id": "mushroom_tableau_depth_5s"
}
},
"rubric": {
"phase_names": [
"establish",
"commit",
"sustain_reveal",
"settle_hold"
],
"min_words": 150,
"max_words": 200,
"sentence_range": [
4,
8
],
"max_cuts": 0,
"require_one_paragraph": true,
"forbid_timecodes": true,
"diagnostics_block_generation": false
}
},
{
"id": "i2v_tableau_20s",
"mode": "image_to_video",
"duration_seconds": 20.0,
"fps": 24,
"frames": 481,
"brief": "Create a mind-bending shot grounded in the supplied first frame.",
"critical_anchors": [
"woman in a blue dress",
"chair",
"brick fireplace"
],
"terminal_composition": "A wide elevated view holds on the unchanged anchor group within the larger meadow.",
"expected_auto_long_horizon_active": false,
"fixed_generation_control": {
"model": "ltx-2.5-22b-distilled",
"text_encoder": "gemma4-12b-with-projection",
"precision": "int8-convrot",
"width": 1344,
"height": 768,
"sampler": "euler_ancestral",
"scheduler": "bong_tangent",
"stage_1_steps": 12,
"stage_2_steps": 3,
"video_cfg": 1.0,
"audio_cfg": 1.0,
"negative_prompt": "cuts, transitions, optical zoom, object duplication, subject replacement, subtitles, captions, logos, watermarks",
"prompt_enhancer": false,
"first_frame_asset_id": "mushroom_tableau_first_frame"
},
"variants": {
"bare": {
"prompt_producer": "bare_brief",
"long_horizon_mode": "off",
"prompt": "Create a mind-bending shot grounded in the supplied first frame.",
"structural_conditioning": "first_frame"
},
"current_director": {
"prompt_producer": "diffusiongemma_current_ltx_contract",
"long_horizon_mode": "off",
"prompt": "A wide shot begins from the exact supplied first-frame viewpoint, preserving the woman in a sleeveless blue dress beside the chair and brick fireplace in the surreal mushroom meadow. The camera holds briefly, then slowly dollies backward and rises while keeping the original tableau centered. She brushes one mushroom stalk as the fire flickers, embers lift, and the grass moves in a light breeze. The camera continues its retreat across the meadow, revealing more rooted mushrooms through natural parallax. Fireplace crackle, wind, distant birds, and restrained lo-fi trip-hop remain synchronized with the scene. The camera finishes on a wider elevated composition that frames the anchor group against the distant fields.",
"structural_conditioning": "first_frame"
},
"long_horizon": {
"prompt_producer": "diffusiongemma_experimental_long_horizon",
"long_horizon_mode": "on",
"prompt": "An eye-level wide shot begins exactly from the supplied first frame, preserving the woman in the blue dress, chair, and brick fireplace as the critical anchor group. During the opening moment, the camera holds while the fire flickers and a light breeze moves her hair, dress, and nearby grass. It then commits to one stabilized backward-and-upward physical path with a fixed wide-angle lens and a gentle downward tilt that keeps the group centered. Through most of the take, that motion continues at an even pace, revealing the already rooted mushroom meadow through natural parallax while subject identities, counts, colors, and relative positions remain stable. Fireplace crackle, soft wind, distant birds, and restrained lo-fi trip-hop form one continuous audio identity without an abrupt change in source, volume, or musical tempo. Only near the end does the camera decelerate into a wide elevated composition, settle, and hold on the unchanged tableau within the larger field.",
"structural_conditioning": "first_frame"
},
"long_horizon_structural": {
"prompt_producer": "diffusiongemma_experimental_long_horizon",
"long_horizon_mode": "on",
"prompt": "An eye-level wide shot begins exactly from the supplied first frame, preserving the woman in the blue dress, chair, and brick fireplace as the critical anchor group. During the opening moment, the camera holds while the fire flickers and a light breeze moves her hair, dress, and nearby grass. It then commits to one stabilized backward-and-upward physical path with a fixed wide-angle lens and a gentle downward tilt that keeps the group centered. Through most of the take, that motion continues at an even pace, revealing the already rooted mushroom meadow through natural parallax while subject identities, counts, colors, and relative positions remain stable. Fireplace crackle, soft wind, distant birds, and restrained lo-fi trip-hop form one continuous audio identity without an abrupt change in source, volume, or musical tempo. Only near the end does the camera decelerate into a wide elevated composition, settle, and hold on the unchanged tableau within the larger field.",
"structural_conditioning": "first_frame_plus_depth",
"structural_asset_id": "mushroom_tableau_depth_20s"
}
},
"rubric": {
"phase_names": [
"establish",
"commit",
"sustain_reveal",
"settle_hold"
],
"min_words": 150,
"max_words": 200,
"sentence_range": [
4,
8
],
"max_cuts": 0,
"require_one_paragraph": true,
"forbid_timecodes": true,
"diagnostics_block_generation": false
}
},
{
"id": "i2v_tableau_60s",
"mode": "image_to_video",
"duration_seconds": 60.0,
"fps": 24,
"frames": 1441,
"brief": "Create a mind-bending shot grounded in the supplied first frame.",
"critical_anchors": [
"woman in a blue dress",
"chair",
"brick fireplace"
],
"terminal_composition": "An extreme-wide elevated view holds with the original anchor group still readable at the visual center.",
"expected_auto_long_horizon_active": true,
"fixed_generation_control": {
"model": "ltx-2.5-22b-distilled",
"text_encoder": "gemma4-12b-with-projection",
"precision": "int8-convrot",
"width": 1344,
"height": 768,
"sampler": "euler_ancestral",
"scheduler": "bong_tangent",
"stage_1_steps": 12,
"stage_2_steps": 3,
"video_cfg": 1.0,
"audio_cfg": 1.0,
"negative_prompt": "cuts, transitions, optical zoom, object duplication, subject replacement, subtitles, captions, logos, watermarks",
"prompt_enhancer": false,
"first_frame_asset_id": "mushroom_tableau_first_frame"
},
"variants": {
"bare": {
"prompt_producer": "bare_brief",
"long_horizon_mode": "off",
"prompt": "Create a mind-bending shot grounded in the supplied first frame.",
"structural_conditioning": "first_frame"
},
"current_director": {
"prompt_producer": "diffusiongemma_current_ltx_contract",
"long_horizon_mode": "off",
"prompt": "A wide shot captures a surreal outdoor space where a woman with long dark hair, wearing a sleeveless blue dress, stands with her back to the camera facing a blue upholstered chair and a patterned rug. Surrounded by numerous oversized fantasy mushrooms with red, white, and blue spotted caps, the scene overlooks a vast green agricultural landscape under a clear sky. To the right, a brick fireplace contains a crackling orange fire. The lighting is bright and even, casting soft shadows on the grey-toned floor. The camera starts at a static eye-level wide-angle perspective, then begins a slow, cinematic dolly-out and slight upward tilt, moving away from the woman. As the woman reaches out to gently touch the stalk of a large white mushroom, the mushrooms begin to vibrate slightly with a magical shimmering light. The fire in the fireplace flickers intensely, sending embers dancing into the air. The camera continues its retreat, revealing more of the surreal field of mushrooms as they begin to sway gently in an invisible breeze. A lofi trip hop beat plays in the background, layered with the rhythmic crackle of the fire and the soft rustle of wind through the giant fungi. The shot ends on a wider composition, framing the entire surreal arrangement against the backdrop of the distant fields.",
"structural_conditioning": "first_frame"
},
"long_horizon": {
"prompt_producer": "diffusiongemma_experimental_long_horizon",
"long_horizon_mode": "on",
"prompt": "An eye-level wide shot begins exactly from the supplied first frame, preserving the woman in the blue dress, chair, and brick fireplace as the critical anchor group. During the opening five seconds, the camera holds while the fire flickers and a light breeze moves her hair, dress, and nearby grass. It then commits gradually to one stabilized backward-and-upward physical path with a fixed wide-angle lens, using a gentle downward tilt to keep the group centered. Through most of the take, the camera sustains that direction at an even speed, revealing the already rooted mushroom meadow through natural parallax rather than growth, appearance, or duplication. The woman, chair, fireplace, and nearby mushrooms retain their identities, proportions, colors, and relative positions while increasingly distant scenery supplies the visual development. Fireplace crackle, soft wind, distant birds, and restrained lo-fi trip-hop continue as one unbroken audio environment. Only during the final nine seconds does the movement decelerate into an extreme-wide elevated composition, ease to a stop, and hold with the original tableau still readable at the visual center.",
"structural_conditioning": "first_frame"
},
"long_horizon_structural": {
"prompt_producer": "diffusiongemma_experimental_long_horizon",
"long_horizon_mode": "on",
"prompt": "An eye-level wide shot begins exactly from the supplied first frame, preserving the woman in the blue dress, chair, and brick fireplace as the critical anchor group. During the opening five seconds, the camera holds while the fire flickers and a light breeze moves her hair, dress, and nearby grass. It then commits gradually to one stabilized backward-and-upward physical path with a fixed wide-angle lens, using a gentle downward tilt to keep the group centered. Through most of the take, the camera sustains that direction at an even speed, revealing the already rooted mushroom meadow through natural parallax rather than growth, appearance, or duplication. The woman, chair, fireplace, and nearby mushrooms retain their identities, proportions, colors, and relative positions while increasingly distant scenery supplies the visual development. Fireplace crackle, soft wind, distant birds, and restrained lo-fi trip-hop continue as one unbroken audio environment. Only during the final nine seconds does the movement decelerate into an extreme-wide elevated composition, ease to a stop, and hold with the original tableau still readable at the visual center.",
"structural_conditioning": "first_frame_plus_depth",
"structural_asset_id": "mushroom_tableau_depth_60s"
}
},
"rubric": {
"phase_names": [
"establish",
"commit",
"sustain_reveal",
"settle_hold"
],
"min_words": 150,
"max_words": 200,
"sentence_range": [
4,
8
],
"max_cuts": 0,
"require_one_paragraph": true,
"forbid_timecodes": true,
"diagnostics_block_generation": false
}
}
]
}
examples/13_ltx25_i2v_director.json
+1 -1
View File
File diff suppressed because one or more lines are too long
examples/14_ltx25_i2v_vs_minimax_h3_ref2va_comparison.json
+1 -1
View File
File diff suppressed because one or more lines are too long
examples/15_minimax_h3_ref2va_music_video_v6.json
+14021
View File
File diff suppressed because one or more lines are too long
examples/README.md
+16
View File
@@ -136,6 +136,22 @@ The H3 Reference Context sets `expected_subject_count=1`, so its Ref2VA packet m
The included raster settings are intentionally model-specific: LTX is 1.0 MP at 16:9 and H3 is 0.4 MP at 16:9. Match them manually if the hardware and comparison design require identical output dimensions. The conditioning is still not identical: LTX I2V anchors an opening frame, while H3 Ref2VA may recompose its full-sequence visual reference.
## MiniMax-H3 synchronized music-video production V6
The checked-in operational production workflow is:
```text
15_minimax_h3_ref2va_music_video_v6.json
```
It combines the accumulated MiniMax-H3 production upgrades in one graph: two same-person identity references, optional previous-lane continuity relay, one to four duration-bounded H3 generation lanes, multiple native `[Shot N]` blocks inside each lane, exact-frame assembly, measured soundtrack timing, Natural/Dance/Lyrics performance modes, local timed-lyrics analysis, and the pristine final soundtrack mux. The graph also includes the tested H3 Turbo/fast path and the H3-specific expressive camera contract.
`SONG SOURCE` selects either the existing lazy ACE-Step generation path or a custom file from `UPLOAD SONG`. Source passthrough under the separate production-concept control still means ACE blueprint passthrough; it is not the file-upload choice. Uploaded music reuses the same technical-integrity checks, excerpt selection, waveform hash, audio guide, lane planner, and final mux.
Replace both saved `LoadImage` choices with your own primary identity image and same-person contact sheet. Keep Picture 1 and Picture 2 as identity authorities for the same semantic subject. Continuity relay is off by default; enable Previous lane tail only when pose/spatial continuity is worth the additional identity-drift risk. The saved upload widget is blank and Generate with ACE-Step is selected, so loading the example does not require a custom song.
The repository copy is the validated V6 graph used by the Desktop and ComfyUI deployments at the time of this changelog. Its migration marker and canonical JSON SHA-256 are regression-tested without depending on those mutable external copies or platform-specific line endings.
## LTX 2.3 Reframe with selectable audio
The provisional, unexported repository prototype is:
ltx25_contract.py
+881 -31
View File
File diff suppressed because it is too large Load Diff
nodes.py
+1588 -109
View File
File diff suppressed because it is too large Load Diff
production_planning_nodes.py
+2910
View File
File diff suppressed because it is too large Load Diff
pyproject.toml
+1 -1
View File
@@ -1,7 +1,7 @@
[project]
name = "diffusiongemma-prompt-builder"
version = "0.2.0"
description = "Stable DiffusionGemma Director prompt authoring for LTX, Ideogram4, and MiniMax H3 in ComfyUI, plus experimental SplatStage planners."
description = "DiffusionGemma Director prompt authoring, measured-audio controls, and synchronized LTX/MiniMax-H3 production planning for ComfyUI."
license = { file = "LICENSE" }
requires-python = ">=3.11"
dependencies = [
scripts/smoke_prompt_builder.py
+17 -15
View File
@@ -212,7 +212,7 @@ def test_minimax_h3_contract_and_splitter_compatibility():
"continuity ledger",
"generic real animals",
"prior shot's resulting position and state",
"shot size, angle, or the word handheld alone is not a camera path",
"an incoming cut, shot size, angle, subject framing",
"event-specific sound immediately beside its visible cause",
"the final frame holds on",
):
@@ -559,13 +559,9 @@ def test_minimax_h3_ref2va_contract():
default_t2va_prompt,
{"minimax_h3_ref_sections_invalid"},
),
(
ref_prompt.replace("<Audio 1>", "the soundtrack"),
{"minimax_h3_ref_missing_asset_tag"},
),
(
ref_prompt.replace("<Video 1>", "<Video 2>"),
{"minimax_h3_ref_missing_asset_tag", "minimax_h3_ref_undefined_tag"},
{"minimax_h3_ref_undefined_tag"},
),
(
ref_prompt.replace(
@@ -575,23 +571,29 @@ def test_minimax_h3_ref2va_contract():
),
{"minimax_h3_ref_subject_definition_invalid"},
),
(
invalid_subject_usage_prompt,
{"minimax_h3_ref_subject_usage_invalid"},
),
(
ref_prompt.replace("fully_preserved", "preserved", 1),
{"minimax_h3_ref_retention_invalid"},
),
)
for invalid_prompt, expected_reasons in invalid_prompts:
invalid_packet = _packet("", {"ready_for_generation": True, "blocked_reasons": []}, segments=[])
invalid_packet["minimax_h3_prompt"] = invalid_prompt
invalid_result = _split(invalid_packet, context=context, target=target)
invalid_metadata = _metadata(invalid_result)
assert invalid_result[13] == "", invalid_metadata
assert invalid_result[13] == "", (expected_reasons, invalid_metadata)
assert expected_reasons.issubset(set(invalid_metadata["blocked_reasons"])), invalid_metadata
repairable_prompts = (
invalid_subject_usage_prompt,
ref_prompt.replace("fully_preserved", "preserved", 1),
ref_prompt.replace("<Audio 1>", "the soundtrack"),
)
for repairable_prompt in repairable_prompts:
repairable_packet = _packet("", {"ready_for_generation": True, "blocked_reasons": []}, segments=[])
repairable_packet["minimax_h3_prompt"] = repairable_prompt
repairable_result = _split(repairable_packet, context=context, target=target)
repairable_metadata = _metadata(repairable_result)
assert repairable_result[13], repairable_metadata
assert repairable_result[12] is True, repairable_metadata
assert repairable_metadata["blocked_reasons"] == [], repairable_metadata
def test_minimax_h3_ref2va_structure_repair_and_generation_gate():
manifest = (
tests/test_audio_production_nodes.py
+1729
View File
File diff suppressed because it is too large Load Diff
tests/test_grounding_guard_integration.py
+6 -2
View File
@@ -381,6 +381,7 @@ class GroundingGuardIntegrationTests(unittest.TestCase):
self.assertIn("did not pass validation", status)
self.assertIn("json_parse_invalid", status)
self.assertIn("minimax_h3_ref_missing_summary", status)
self.assertIn("produced no media output", status)
self.assertIn("select refresh once", status)
valid_prompt = "A validated prompt for an independent comparison branch."
@@ -2004,7 +2005,9 @@ class GroundingGuardIntegrationTests(unittest.TestCase):
)
compiler_packet = self.packet("")
compiler_packet["ltx_prompt"] = ""
compiler_packet["minimax_h3_prompt"] = "invalid timeline"
compiler_packet["minimax_h3_prompt"] = (
"[Shot 1] invalid timeline. [Shot 2] missing a required timestamp."
)
compiler_packet["metadata"]["target_profile"] = "minimax_h3"
evidence_report_id = self.nodes._grounding_report_id(ledger, registry)
stages: list[str] = []
@@ -2016,7 +2019,8 @@ class GroundingGuardIntegrationTests(unittest.TestCase):
if options.stage == "compiler":
result = self.backend_result(compiler_packet)
result.decoded_text = (
"integrated_multimodal_description: [Shot 1] invalid timeline\n\n"
"integrated_multimodal_description: [Shot 1] invalid timeline. "
"[Shot 2] missing a required timestamp.\n\n"
"overall_soundscape: N/A\n\n"
"non_diegetic_music: N/A\n\n"
f"GROUNDING_EVIDENCE_REPORT_ID: {evidence_report_id}\n"
tests/test_grounding_guard_workflows.py
+169
View File
@@ -17,6 +17,98 @@ STRICT_WORKFLOWS = {
}
def _expanded_subgraph_string_input_fragment(
workflow: dict,
*,
subgraph_node_id: int,
outer_input_name: str,
) -> dict:
"""Materialize the API prompt fragment for one promoted string input."""
outer_nodes = {node["id"]: node for node in workflow["nodes"]}
outer_links = {link[0]: link for link in workflow["links"]}
outer_node = outer_nodes[subgraph_node_id]
outer_input_index, outer_input = next(
(index, item)
for index, item in enumerate(outer_node["inputs"])
if item["name"] == outer_input_name
)
source_link = outer_links[outer_input["link"]]
if source_link[3:5] != [subgraph_node_id, outer_input_index]:
raise AssertionError("Outer link target does not match the promoted input slot")
source_ref = [str(source_link[1]), source_link[2]]
subgraph = next(
item
for item in workflow["definitions"]["subgraphs"]
if item["id"] == outer_node["type"]
)
subgraph_nodes = {node["id"]: node for node in subgraph["nodes"]}
subgraph_links = {link["id"]: link for link in subgraph["links"]}
subgraph_input_index = next(
index
for index, item in enumerate(subgraph["inputs"])
if item["name"] == outer_input_name
)
if outer_input_index != subgraph_input_index:
raise AssertionError(
f"Promoted input index mismatch: outer={outer_input_index}, "
f"subgraph={subgraph_input_index}"
)
inbound_links = [
link
for link in subgraph_links.values()
if link["origin_id"] == -10 and link["origin_slot"] == subgraph_input_index
]
if len(inbound_links) != 1:
raise AssertionError(
f"Expected one internal route for {outer_input_name!r}, got {inbound_links!r}"
)
inbound = inbound_links[0]
primitive = subgraph_nodes[inbound["target_id"]]
primitive_input = primitive["inputs"][inbound["target_slot"]]
if primitive["type"] != "PrimitiveStringMultiline" or primitive_input["name"] != "value":
raise AssertionError(
f"{outer_input_name!r} must enter a PrimitiveStringMultiline before encoding"
)
if primitive_input.get("link") != inbound["id"]:
raise AssertionError("Primitive input does not reference the promoted subgraph link")
encoder_route = None
for output_slot, output in enumerate(primitive["outputs"]):
for link_id in output.get("links") or []:
link = subgraph_links[link_id]
target = subgraph_nodes[link["target_id"]]
target_input = target["inputs"][link["target_slot"]]
if target["type"] == "CLIPTextEncode" and target_input["name"] == "text":
encoder_route = (output_slot, link, target, target_input)
break
if encoder_route is not None:
break
if encoder_route is None:
raise AssertionError(
f"{outer_input_name!r} is not routed from the primitive into CLIPTextEncode.text"
)
output_slot, encoder_link, encoder, encoder_input = encoder_route
if encoder_input.get("link") != encoder_link["id"]:
raise AssertionError("Encoder text input does not reference the primitive output link")
primitive_api_id = f"{subgraph_node_id}:{primitive['id']}"
encoder_api_id = f"{subgraph_node_id}:{encoder['id']}"
return {
primitive_api_id: {
"inputs": {"value": source_ref},
"class_type": "PrimitiveStringMultiline",
},
encoder_api_id: {
"inputs": {"text": [primitive_api_id, output_slot]},
"class_type": "CLIPTextEncode",
},
}
class StrictGroundingWorkflowTests(unittest.TestCase):
def _load(self, filename: str) -> dict:
return json.loads((EXAMPLES / filename).read_text(encoding="utf-8"))
@@ -258,6 +350,83 @@ class StrictGroundingWorkflowTests(unittest.TestCase):
all(links[link_id][3] != native_generator_id for link_id in candidate_link_ids)
)
def test_ltx_negative_prompt_survives_subgraph_api_expansion(self):
expected_fragment = {
"456:509": {
"inputs": {"value": ["187", 2]},
"class_type": "PrimitiveStringMultiline",
},
"456:427": {
"inputs": {"text": ["456:509", 0]},
"class_type": "CLIPTextEncode",
},
}
for filename in (
"13_ltx25_i2v_director.json",
"14_ltx25_i2v_vs_minimax_h3_ref2va_comparison.json",
):
with self.subTest(filename=filename):
workflow = self._load(filename)
target_node = next(
node
for node in workflow["nodes"]
if node["type"] == "DiffusionGemmaLTX25TargetProfile"
)
self.assertEqual(target_node["widgets_values"][-1], "Off")
self.assertEqual(len(target_node["widgets_values"]), 8)
outer_node = next(node for node in workflow["nodes"] if node["id"] == 456)
self.assertEqual(
outer_node["widgets_values"],
[
"",
5,
992,
544,
347267244880135,
24,
"",
"ltx-2.5-22b-distilled-transformer-comfy-int8-convrot.safetensors",
"ltx-2.5-video-vae-bf16.safetensors",
"ltx-2.5-audio-vae-bf16.safetensors",
"gemma4-12b-with-proj-ltx-2.5-comfy-int8-convrot.safetensors",
"ltx-2.5-latent-spatial-upscaler-x2-bf16-1.0.safetensors",
],
"The promoted negative-prompt widget needs an empty saved value so "
"the native model dropdown values do not shift left in ComfyUI.",
)
subgraph = next(
item
for item in workflow["definitions"]["subgraphs"]
if item["id"] == outer_node["type"]
)
outer_index = next(
index
for index, item in enumerate(outer_node["inputs"])
if item["name"] == "negative_prompt"
)
subgraph_index = next(
index
for index, item in enumerate(subgraph["inputs"])
if item["name"] == "negative_prompt"
)
self.assertEqual((outer_index, subgraph_index), (8, 8))
internal_links = {link["id"]: link for link in subgraph["links"]}
for input_index, subgraph_input in enumerate(subgraph["inputs"]):
for link_id in subgraph_input.get("linkIds") or []:
internal_link = internal_links[link_id]
self.assertEqual(
(internal_link["origin_id"], internal_link["origin_slot"]),
(-10, input_index),
)
self.assertEqual(
_expanded_subgraph_string_input_fragment(
workflow,
subgraph_node_id=456,
outer_input_name="negative_prompt",
),
expected_fragment,
)
def test_dual_ltx_h3_comparison_keeps_model_specific_contexts_and_gates(self):
workflow = self._load("14_ltx25_i2v_vs_minimax_h3_ref2va_comparison.json")
nodes = {node["id"]: node for node in workflow["nodes"]}
tests/test_ltx25_camera_capability.py
+410
View File
@@ -0,0 +1,410 @@
import importlib.util
import json
import sys
import unittest
import uuid
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
COMFY_ROOT = ROOT.parents[1]
def load_nodes_module():
if str(COMFY_ROOT) not in sys.path:
sys.path.insert(0, str(COMFY_ROOT))
package_name = f"diffusiongemma_camera_capability_{uuid.uuid4().hex}"
spec = importlib.util.spec_from_file_location(
package_name,
ROOT / "__init__.py",
submodule_search_locations=[str(ROOT)],
)
if spec is None or spec.loader is None:
raise RuntimeError("could not load DiffusionGemma Prompt Builder")
package = importlib.util.module_from_spec(spec)
sys.modules[package_name] = package
spec.loader.exec_module(package)
return sys.modules[f"{package_name}.nodes"]
class LTX25CameraCapabilityTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.nodes = load_nodes_module()
cls.contract = sys.modules[cls.nodes.__package__ + ".ltx25_contract"]
cls.mode_report = {
"resolved_mode": "text_to_video",
"first_frame_attached": False,
"last_frame_attached": False,
"warnings": [],
}
def validate(
self,
prompt: str,
capability: str = "stable",
duration: float = 12.0,
long_horizon_mode: str = "off",
) -> dict:
return self.contract.validate_ltx25_prompt(
prompt,
self.mode_report,
{},
duration,
long_horizon_mode,
capability,
)
def test_target_profile_missing_field_normalizes_to_stable(self) -> None:
normalized = self.nodes._target_profile_config_to_dict(
{
"target_profile": "ltx",
"ltx_generation_mode": "Image to video",
}
)
self.assertEqual(normalized["ltx_camera_capability"], "stable")
config = self.nodes._make_target_profile_config(target_profile="ltx")
self.assertEqual(config.ltx_camera_capability, "stable")
payload = json.loads(self.nodes._target_profile_node_result(config)[1])
self.assertEqual(payload["ltx_camera_capability"], "stable")
def test_ltx_target_profile_defaults_stable_and_requires_explicit_advanced_opt_in(self) -> None:
node = self.nodes.DiffusionGemmaLTX25TargetProfile()
stable, stable_json = node.build(
generation_mode="Image to video",
target_duration_seconds=25.0,
style_guidance="",
audio_mode="auto_scene_audio",
audio_guidance="",
negative_prompt_mode="auto",
negative_prompt_guidance="",
long_horizon_mode="On",
)
advanced, advanced_json = node.build(
generation_mode="Image to video",
target_duration_seconds=25.0,
style_guidance="",
audio_mode="auto_scene_audio",
audio_guidance="",
negative_prompt_mode="auto",
negative_prompt_guidance="",
long_horizon_mode="On",
camera_capability="Advanced / controlled camera",
)
self.assertEqual(stable.ltx_camera_capability, "stable")
self.assertEqual(json.loads(stable_json)["ltx_camera_capability"], "stable")
self.assertEqual(advanced.ltx_camera_capability, "advanced")
self.assertEqual(json.loads(advanced_json)["ltx_camera_capability"], "advanced")
def test_stable_compiler_removes_ambitious_unspecified_camera_policy(self) -> None:
stable = self.contract.ltx25_compiler_contract(
"image_to_video",
25.0,
long_horizon_mode="on",
camera_capability="stable",
).casefold()
advanced = self.contract.ltx25_compiler_contract(
"image_to_video",
25.0,
long_horizon_mode="on",
camera_capability="advanced",
).casefold()
self.assertIn("stable / base model", stable)
self.assertIn("one restrained single-axis move", stable)
self.assertIn("rather than sustaining camera travel", stable)
self.assertNotIn("fill time with sustained movement, parallax", stable)
self.assertIn("one dominant physically continuous camera intention", advanced)
self.assertIn("fill time with sustained movement, parallax", advanced)
def test_stable_creativity_keeps_art_direction_but_caps_camera_ambition(self) -> None:
for mode in ("editorial", "cinematic", "concept_art", "wild"):
instruction = self.nodes._ltx_creativity_instruction(
mode,
1.0,
"image_to_video",
"stable",
).casefold()
with self.subTest(mode=mode):
self.assertIn(mode.replace("_", "-"), instruction)
self.assertIn("stable / base model", instruction)
self.assertIn("one restrained single-axis move", instruction)
self.assertIn("camera swirl", instruction)
self.assertNotIn("sweeping arc or orbit", instruction)
advanced = self.nodes._ltx_creativity_instruction(
"wild",
1.0,
"image_to_video",
"advanced",
).casefold()
self.assertIn("sweeping arc or orbit", advanced)
self.assertIn("compound camera", advanced)
def test_stable_validation_rejects_each_high_risk_camera_family(self) -> None:
cases = {
"ltx_stable_camera_orbit_or_rotation": [
"The camera begins a slow dramatic circular orbit around the subject.",
"The camera slows its rotation.",
"The camera spins around her.",
"The camera swirls around her.",
],
"ltx_stable_camera_sweeping_or_whip_motion": [
"The camera makes a sweeping move around the performer.",
"The camera performs a whip-pan to the right.",
],
"ltx_stable_camera_dolly_zoom": [
"The camera performs a dolly zoom toward her face.",
],
"ltx_stable_camera_shake_or_handheld_pursuit": [
"The camera shakes violently beside the runner.",
"A handheld pursuit follows the runner down the alley.",
],
"ltx_stable_camera_compound_or_reversal": [
"The camera uses a compound multi-axis path around the performer.",
"The camera pushes in and then pulls back before the end.",
],
"ltx_stable_camera_pronounced_parallax": [
"The shot creates dizzying layered parallax around the subject.",
],
}
for expected_reason, clauses in cases.items():
for clause in clauses:
prompt = f"A medium shot at eye level frames a performer. {clause}"
with self.subTest(reason=expected_reason, clause=clause):
stable = self.validate(prompt, "stable")
advanced = self.validate(prompt, "advanced")
self.assertIn(expected_reason, stable["reasons"])
self.assertNotIn(expected_reason, advanced["reasons"])
def test_user_nightmare_prompt_is_hard_failure_only_in_stable_mode(self) -> None:
prompt = (
"A medium shot at eye level frames the performer. Through most of the shot, "
"the camera continues its sweeping arc, creating a dizzying parallax effect "
"with the bills that flutter occasionally as if caught in a faint draft. "
"Near the end, the camera slows its rotation and settles on her face."
)
stable = self.validate(prompt, "stable", 25.0, "on")
advanced = self.validate(prompt, "advanced", 25.0, "on")
self.assertFalse(stable["ready"])
self.assertIn("ltx_stable_camera_orbit_or_rotation", stable["reasons"])
self.assertIn("ltx_stable_camera_sweeping_or_whip_motion", stable["reasons"])
self.assertIn("ltx_stable_camera_pronounced_parallax", stable["reasons"])
self.assertIn("ltx_stable_camera_sustained_travel", stable["reasons"])
self.assertFalse(
any(reason.startswith("ltx_stable_camera_") for reason in advanced["reasons"])
)
def test_negated_risky_language_is_not_treated_as_positive_camera_direction(self) -> None:
prompt = (
"A medium shot at eye level frames the performer while the camera remains locked "
"and static. No sweeping arc, whip-pan, dolly zoom, camera shake, handheld pursuit, "
"circular orbit, rotation, spin, swirl, roll, compound multi-axis movement, direction "
"reversal, pronounced layered dizzying parallax, or sustained camera travel is used."
)
report = self.validate(prompt, "stable", 25.0, "on")
self.assertFalse(
any(reason.startswith("ltx_stable_camera_") for reason in report["reasons"]),
report,
)
def test_subject_swirl_is_not_misclassified_as_camera_swirl(self) -> None:
prompt = (
"A medium shot at eye level frames a dancer while the camera remains locked and "
"static. Her skirt swirls around her ankles as she turns once."
)
report = self.validate(prompt, "stable")
self.assertNotIn("ltx_stable_camera_orbit_or_rotation", report["reasons"])
def test_subject_and_environment_motion_do_not_bind_to_stable_camera(self) -> None:
safe_prompts = (
"A medium shot at eye level frames her; the camera stays static while coins circle around the performer.",
"A medium shot at eye level frames her; the camera is stabilized as ribbons orbit the dancer.",
"A medium shot at eye level frames her; the camera is locked while a sweeping curtain crosses the background.",
"A medium shot at eye level frames her; the camera is locked as the performer begins a sweeping arc with her arm.",
"A medium shot at eye level frames her; the shot stays steady while the dancer performs a sweeping arc.",
"A medium shot at eye level frames her; the camera is locked while the dancer performs an orbit around the stage.",
"A medium shot at eye level frames her; the camera is locked while the dancer moves throughout.",
"A static medium shot at eye level frames her while the performer moves continuously throughout.",
"A medium shot at eye level holds with framing fixed while bills travel throughout the background.",
"A medium shot at eye level holds; the camera is locked while fog moves for the duration.",
)
for prompt in safe_prompts:
with self.subTest(prompt=prompt):
report = self.validate(prompt, "stable", 25.0, "on")
self.assertFalse(
any(
reason.startswith("ltx_stable_camera_")
for reason in report["reasons"]
),
report,
)
def test_negating_a_safe_hold_does_not_negate_the_risky_move_that_follows(self) -> None:
prompts = (
"A medium shot at eye level frames her. Without a locked camera, the shot begins a slow circular orbit.",
"A medium shot at eye level frames her. No longer locked, the camera begins a slow circular orbit.",
"A medium shot at eye level frames her. The camera does not stop as it orbits her.",
"A medium shot at eye level frames her. The camera keeps moving as it orbits her.",
"A medium shot at eye level frames her. A circular tracking arc surrounds the subject.",
)
for prompt in prompts:
with self.subTest(prompt=prompt):
self.assertIn(
"ltx_stable_camera_orbit_or_rotation",
self.validate(prompt, "stable")["reasons"],
)
def test_short_range_camera_pronouns_preserve_risky_motion_ownership(self) -> None:
risky = (
"A medium shot at eye level frames her. The camera begins locked. Then it orbits.",
"A medium shot at eye level frames her. The shot starts static. It then spins.",
"A medium shot at eye level frames her. The camera starts stable, but it soon swirls.",
"A medium shot at eye level frames her. The camera begins in a stable view. Near the end, it slows its rotation.",
"A medium shot at eye level frames her. The camera begins with a stable view. Near the end, it slows its rotation.",
"A medium shot at eye level frames her while the camera begins locked. Then it orbits around her.",
"A medium shot at eye level frames her while the camera starts static. Near the end, it spins.",
"A medium shot frames her and the camera remains stable. Then it rotates around her.",
)
for prompt in risky:
with self.subTest(prompt=prompt):
self.assertIn(
"ltx_stable_camera_orbit_or_rotation",
self.validate(prompt, "stable")["reasons"],
)
subject_pronoun = (
"A medium shot at eye level frames a robot while the camera stays static. "
"The robot lifts one hand. It spins once."
)
self.assertNotIn(
"ltx_stable_camera_orbit_or_rotation",
self.validate(subject_pronoun, "stable")["reasons"],
)
ambiguous_subject_pronouns = (
"A medium shot at eye level frames a toy top while the camera remains locked. Then it spins.",
"A medium shot at eye level frames a coin while the shot remains static. It then rotates.",
"A medium shot at eye level frames a robot while the camera remains locked on it. Then it spins once.",
)
for prompt in ambiguous_subject_pronouns:
with self.subTest(prompt=prompt):
self.assertNotIn(
"ltx_stable_camera_orbit_or_rotation",
self.validate(prompt, "stable")["reasons"],
)
def test_negated_stop_or_static_state_does_not_hide_persistent_camera_rotation(self) -> None:
prompts = (
"A medium shot at eye level frames her. The camera never stops rotating around her.",
"A medium shot at eye level frames her. The camera does not stop rotating around her.",
"A medium shot at eye level frames her. The camera never stops its orbit around her.",
"A medium shot at eye level frames her. The camera does not cease to orbit her.",
"A medium shot at eye level frames her. The camera is never static and instead orbits her.",
"A medium shot at eye level frames her. The camera does not slow its sweeping arc around her.",
)
for prompt in prompts:
with self.subTest(prompt=prompt):
self.assertIn(
"ltx_stable_camera_orbit_or_rotation",
self.validate(prompt, "stable")["reasons"],
)
def test_simple_stable_camera_states_and_single_axis_moves_pass_risk_validation(self) -> None:
clauses = (
"the camera remains locked and static",
"the stabilized camera makes one short gentle push-in and settles",
"the stabilized camera makes one short gentle pull-back and settles",
"the stabilized camera makes one small pan left and settles",
"the stabilized camera makes one small tilt up and settles",
"the stabilized camera makes one short lateral track right and settles",
)
for clause in clauses:
prompt = f"A medium shot at eye level frames a performer while {clause}."
with self.subTest(clause=clause):
report = self.validate(prompt, "stable")
self.assertFalse(
any(
reason.startswith("ltx_stable_camera_")
for reason in report["reasons"]
),
report,
)
def test_long_horizon_stable_rejects_sustained_travel_and_advanced_allows_it(self) -> None:
prompt = (
"A medium shot at eye level frames the performer as the camera tracks right "
"throughout the shot and finally settles on the same subject."
)
stable = self.validate(prompt, "stable", 25.0, "on")
advanced = self.validate(prompt, "advanced", 25.0, "on")
self.assertIn("ltx_stable_camera_sustained_travel", stable["reasons"])
self.assertNotIn("ltx_stable_camera_sustained_travel", advanced["reasons"])
def test_low_level_legacy_defaults_remain_advanced_but_target_path_is_stable(self) -> None:
prompt = (
"A medium shot at eye level frames the performer while the camera orbits her."
)
legacy = self.contract.validate_ltx25_prompt(
prompt,
self.mode_report,
{},
12.0,
)
self.assertEqual(legacy["camera_capability"], "advanced")
self.assertNotIn("ltx_stable_camera_orbit_or_rotation", legacy["reasons"])
context = self.nodes.GemmaContext(user_prompt="A performer dances.")
target = self.nodes._target_profile_config_to_dict({"target_profile": "ltx"})
diagnostics = self.nodes._ltx25_contract_diagnostics(
prompt,
context,
target,
{},
)
self.assertEqual(diagnostics["camera_capability"], "stable")
self.assertIn("ltx_stable_camera_orbit_or_rotation", diagnostics["reasons"])
def test_stable_model_prompt_has_no_unconditional_advanced_camera_authorization(self) -> None:
metadata = {
"source": "image",
"reference_image_count": 1,
"ltx_first_frame_attached": True,
"ltx_last_frame_attached": False,
}
for long_horizon_mode in ("off", "on"):
stable = self.nodes._build_model_prompt(
"A performer dances among loose bills.",
self.nodes.DEFAULT_MASTER_PROMPT,
"ltx",
metadata,
creativity_mode="wild",
creative_strength=1.0,
ltx_generation_mode="image_to_video",
ltx_long_horizon_mode=long_horizon_mode,
ltx_camera_capability="stable",
).casefold()
with self.subTest(long_horizon_mode=long_horizon_mode):
self.assertIn("camera capability is stable / base model", stable)
self.assertIn("creativity-driven camera ambition", stable)
self.assertNotIn("zooming out while rotating 180 degrees", stable)
self.assertNotIn("may author mode-appropriate camera choreography", stable)
self.assertNotIn("may be assertive, fast, or compound", stable)
self.assertNotIn("fill the long middle with sustained travel, parallax", stable)
self.assertNotIn("one dominant continuous camera-and-subject path", stable)
advanced = self.nodes._build_model_prompt(
"A performer dances among loose bills.",
self.nodes.DEFAULT_MASTER_PROMPT,
"ltx",
metadata,
creativity_mode="wild",
creative_strength=1.0,
ltx_generation_mode="image_to_video",
ltx_long_horizon_mode="on",
ltx_camera_capability="advanced",
).casefold()
self.assertIn("zooming out while rotating 180 degrees", advanced)
self.assertIn("fill the long middle with sustained travel, parallax", advanced)
if __name__ == "__main__":
unittest.main()
tests/test_ltx25_camera_motion_contract.py
+53
View File
@@ -179,6 +179,59 @@ class Ltx25CameraMotionContractTests(unittest.TestCase):
)
self.assertNotIn("ltx_camera_state_missing", report["reasons"])
def test_nominal_tracking_arc_is_a_continuous_camera_path(self) -> None:
prompt = (
"A medium shot starts from the exact supplied first-frame viewpoint. "
"The camera begins a slow, handheld-style tracking arc around her, gaining kinetic "
"energy as she starts to dance. As the camera decelerates, it settles into a tight "
"close-up and holds."
)
report = self.contract.analyze_ltx25_prompt_complexity(
prompt,
"image_to_video",
25.0,
"auto",
)
self.assertIn(
"tracking arc",
{match["text"].casefold() for match in report["observed"]["camera_motion_matches"]},
)
self.assertNotIn("ltx_camera_state_missing", report["reasons"])
def test_replayed_25_second_candidate_passes_camera_state_validation(self) -> None:
prompt = (
"A medium shot captures a woman with curly blonde hair, wearing a green floral print "
"bikini, holding a green game controller aloft in a dimly lit, crowded party setting "
"with harsh flash lighting. Starting from this anchored frame, she lets out a joyful "
"shout as the muffled thumping of an alt-pop beat and ambient crowd chatter fills the "
"room. The camera begins a slow, handheld-style tracking arc around her, gaining "
"kinetic energy as she starts to dance rhythmically. Through most of the shot, she "
"spins slowly, the game controller held steady in her raised hand while the background "
"figures of partygoers blur in a bokeh of movement and light. The direct flash creates "
"glints on her skin and deep shadows behind her, emphasizing the late90s aesthetic. "
"As the camera decelerates, it settles into a tight close-up on her expressive face as "
"she laughs and looks directly into the lens. The clip ends holding on this vibrant, "
"high-energy expression as the music reaches a brief crescendo and the sound of a "
"cheering crowd fades into a rhythmic hum."
)
metadata = self._conditioned_metadata("image_to_video")
mode_report = self.contract.resolve_ltx25_generation_mode(
"image_to_video",
metadata,
1,
)
report = self.contract.validate_ltx25_prompt(
prompt,
mode_report,
metadata,
25.0,
"auto",
)
self.assertTrue(report["ready"], report["reasons"])
self.assertNotIn("ltx_camera_state_missing", report["reasons"])
def test_camera_pronoun_carries_across_a_compound_path(self) -> None:
prompt = (
"A medium shot at eye level follows the camera as it pans right, then tilts up, "
tests/test_ltx25_contracts.py
+96 -2
View File
@@ -1312,13 +1312,57 @@ class Ltx25ContractTests(unittest.TestCase):
"resolution_megapixels",
"resolution_multiple",
"resolution_aspect_ratio",
"resolution_aspect_ratio_override",
],
)
choices, config = optional["resolution_aspect_ratio"]
self.assertEqual(choices, self.nodes.RESOLUTION_SELECTOR_ASPECT_CHOICES)
self.assertEqual(config["default"], self.nodes.RESOLUTION_SELECTOR_AUTO)
override_type, override_config = optional["resolution_aspect_ratio_override"]
self.assertEqual(override_type, "STRING")
self.assertTrue(override_config["forceInput"])
def test_ltx_mode_is_appended_to_config_and_node_positional_signatures(self) -> None:
def test_splitter_project_aspect_override_is_authoritative(self) -> None:
context = self.nodes.GemmaContext(
user_prompt="Create a portrait music video.",
source="none",
media_metadata={"source": "none"},
)
target = self.nodes.TargetProfileConfig(target_profile="minimax_h3")
result = self.nodes.DiffusionGemmaJSONSplitter().split(
self.prompt_packet(),
context,
target,
resolution_megapixels=0.4,
resolution_multiple=32,
resolution_aspect_ratio="16:9 (Widescreen)",
resolution_aspect_ratio_override="9:16",
)
metadata = json.loads(result[4])
self.assertEqual(result[3], "9:16")
self.assertEqual(metadata["resolution_aspect_ratio_requested"], "9:16")
self.assertEqual(
metadata["resolution_aspect_ratio_source"], "project_master_contract"
)
self.assertLess(result[10], result[11])
def test_splitter_rejects_invalid_project_aspect_override(self) -> None:
context = self.nodes.GemmaContext(
user_prompt="Create a music video.",
source="none",
media_metadata={"source": "none"},
)
target = self.nodes.TargetProfileConfig(target_profile="minimax_h3")
with self.assertRaisesRegex(ValueError, "supported canonical W:H ratios"):
self.nodes.DiffusionGemmaJSONSplitter().split(
self.prompt_packet(),
context,
target,
resolution_aspect_ratio_override="5:7",
)
def test_ltx_controls_are_appended_to_config_and_node_positional_signatures(self) -> None:
legacy_values = (
"minimax_h3",
"explicit_sound_design",
@@ -1340,10 +1384,14 @@ class Ltx25ContractTests(unittest.TestCase):
direct = self.nodes.TargetProfileConfig(*legacy_values)
self.assertEqual(direct.minimax_h3_dialogue_guidance, "One short line.")
self.assertEqual(direct.ltx_generation_mode, "auto")
self.assertEqual(direct.ltx_long_horizon_mode, "off")
self.assertEqual(direct.ltx_camera_capability, "stable")
made = self.nodes._make_target_profile_config(*legacy_values)
self.assertEqual(made.minimax_h3_dialogue_guidance, "One short line.")
self.assertEqual(made.ltx_generation_mode, "auto")
self.assertEqual(made.ltx_long_horizon_mode, "off")
self.assertEqual(made.ltx_camera_capability, "stable")
optional = self.nodes.DiffusionGemmaTargetProfile.INPUT_TYPES()["optional"]
self.assertEqual(
@@ -1356,6 +1404,8 @@ class Ltx25ContractTests(unittest.TestCase):
"minimax_h3_dialogue_line_count",
"minimax_h3_dialogue_guidance",
"ltx_generation_mode",
"ltx_long_horizon_mode",
"ltx_camera_capability",
],
)
choices, config = optional["ltx_generation_mode"]
@@ -1364,6 +1414,15 @@ class Ltx25ContractTests(unittest.TestCase):
["Auto (recommended)", "Text to video", "Image to video", "First + last frame"],
)
self.assertEqual(config["default"], "Auto (recommended)")
long_choices, long_config = optional["ltx_long_horizon_mode"]
self.assertEqual(long_choices, ["Off", "Auto (>20 seconds)", "On"])
self.assertEqual(long_config["default"], "Off")
camera_choices, camera_config = optional["ltx_camera_capability"]
self.assertEqual(
camera_choices,
["Stable / base model", "Advanced / controlled camera"],
)
self.assertEqual(camera_config["default"], "Stable / base model")
required = (
"ltx",
@@ -1379,7 +1438,8 @@ class Ltx25ContractTests(unittest.TestCase):
"",
)
# Six legacy optional values end at dialogue guidance. The newly added
# LTX mode must retain its default instead of shifting an old widget.
# Appended LTX controls must retain their defaults instead of shifting
# old widgets.
legacy_node_config, legacy_json = self.nodes.DiffusionGemmaTargetProfile().build(
*required,
"t2va",
@@ -1390,7 +1450,11 @@ class Ltx25ContractTests(unittest.TestCase):
"",
)
self.assertEqual(legacy_node_config.ltx_generation_mode, "auto")
self.assertEqual(legacy_node_config.ltx_long_horizon_mode, "off")
self.assertEqual(legacy_node_config.ltx_camera_capability, "stable")
self.assertEqual(json.loads(legacy_json)["ltx_generation_mode"], "auto")
self.assertEqual(json.loads(legacy_json)["ltx_long_horizon_mode"], "off")
self.assertEqual(json.loads(legacy_json)["ltx_camera_capability"], "stable")
explicit_node_config, _json = self.nodes.DiffusionGemmaTargetProfile().build(
*required,
@@ -1403,6 +1467,36 @@ class Ltx25ContractTests(unittest.TestCase):
"First + last frame",
)
self.assertEqual(explicit_node_config.ltx_generation_mode, "first_last_frame")
self.assertEqual(explicit_node_config.ltx_long_horizon_mode, "off")
long_node_config, _json = self.nodes.DiffusionGemmaTargetProfile().build(
*required,
"t2va",
"auto",
12,
"off",
2,
"",
"Image to video",
"Auto (>20 seconds)",
)
self.assertEqual(long_node_config.ltx_generation_mode, "image_to_video")
self.assertEqual(long_node_config.ltx_long_horizon_mode, "auto")
advanced_node_config, advanced_json = self.nodes.DiffusionGemmaTargetProfile().build(
*required,
"t2va",
"auto",
12,
"off",
2,
"",
"Image to video",
"Off",
"Advanced / controlled camera",
)
self.assertEqual(advanced_node_config.ltx_camera_capability, "advanced")
self.assertEqual(json.loads(advanced_json)["ltx_camera_capability"], "advanced")
if __name__ == "__main__":
tests/test_ltx25_long_horizon.py
+396
View File
@@ -0,0 +1,396 @@
from __future__ import annotations
import importlib.util
import json
import re
import sys
import unittest
import uuid
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
COMFY_ROOT = ROOT.parents[1]
MANIFEST_PATH = ROOT / "benchmarks" / "ltx25_long_horizon_v1" / "manifest.json"
def load_nodes_module():
if str(COMFY_ROOT) not in sys.path:
sys.path.insert(0, str(COMFY_ROOT))
package_name = f"diffusiongemma_ltx25_long_horizon_{uuid.uuid4().hex}"
spec = importlib.util.spec_from_file_location(
package_name,
ROOT / "__init__.py",
submodule_search_locations=[str(ROOT)],
)
if spec is None or spec.loader is None:
raise RuntimeError("could not load DiffusionGemma Prompt Builder")
package = importlib.util.module_from_spec(spec)
sys.modules[package_name] = package
spec.loader.exec_module(package)
return sys.modules[f"{package_name}.nodes"]
class Ltx25LongHorizonTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.nodes = load_nodes_module()
cls.contract = sys.modules[f"{cls.nodes.__package__}.ltx25_contract"]
cls.manifest = json.loads(MANIFEST_PATH.read_text(encoding="utf-8"))
def test_target_profile_appends_backward_compatible_control(self) -> None:
inputs = self.nodes.DiffusionGemmaLTX25TargetProfile.INPUT_TYPES()
self.assertEqual(list(inputs["required"])[-1], "long_horizon_mode")
choices, options = inputs["required"]["long_horizon_mode"]
self.assertEqual(choices, ["Off", "Auto (>20 seconds)", "On"])
self.assertEqual(options["default"], "Off")
# A call saved before the new final widget existed keeps historical
# behavior instead of silently enabling experimental prompt planning.
legacy_config, legacy_json = self.nodes.DiffusionGemmaLTX25TargetProfile().build(
generation_mode="Image to video",
target_duration_seconds=60.0,
style_guidance="restrained surrealism",
audio_mode="explicit_sound_design",
audio_guidance="wind and fireplace crackle",
negative_prompt_mode="custom",
negative_prompt_guidance="no captions",
)
self.assertEqual(legacy_config.ltx_long_horizon_mode, "off")
self.assertEqual(json.loads(legacy_json)["ltx_long_horizon_mode"], "off")
auto_config, auto_json = self.nodes.DiffusionGemmaLTX25TargetProfile().build(
generation_mode="Image to video",
target_duration_seconds=60.0,
style_guidance="restrained surrealism",
audio_mode="explicit_sound_design",
audio_guidance="wind and fireplace crackle",
negative_prompt_mode="custom",
negative_prompt_guidance="no captions",
long_horizon_mode="Auto (>20 seconds)",
)
self.assertEqual(auto_config.ltx_long_horizon_mode, "auto")
self.assertEqual(json.loads(auto_json)["ltx_long_horizon_mode"], "auto")
def test_auto_activation_boundary_and_forced_modes(self) -> None:
cases = (
("off", 60.0, False),
("auto", 20.0, False),
("auto", 20.001, True),
("on", 5.0, True),
)
for mode, duration, expected_active in cases:
with self.subTest(mode=mode, duration=duration):
plan = self.contract.ltx25_long_horizon_plan(mode, duration)
self.assertEqual(plan["requested_mode"], mode)
self.assertEqual(plan["active"], expected_active)
self.assertEqual(plan["phase_count"], 4 if expected_active else 0)
self.assertEqual(len(plan["phases"]), 4 if expected_active else 0)
at_boundary = self.contract.ltx25_long_horizon_plan("auto", 20.0)
above_boundary = self.contract.ltx25_long_horizon_plan("auto", 20.001)
self.assertIn("at_or_below_20", at_boundary["activation_reason"])
self.assertIn("above_20", above_boundary["activation_reason"])
def test_active_plan_is_four_contiguous_phases_not_three_chapters(self) -> None:
plan = self.contract.ltx25_long_horizon_plan("on", 60.0)
phases = plan["phases"]
self.assertEqual(
[phase["name"] for phase in phases],
["establish", "commit", "sustain_reveal", "settle_hold"],
)
self.assertEqual(plan["phase_count"], 4)
self.assertNotEqual(plan["phase_count"], 3)
self.assertEqual(phases[0]["start_seconds"], 0.0)
self.assertEqual(phases[-1]["end_seconds"], 60.0)
for left, right in zip(phases, phases[1:]):
self.assertEqual(left["end_seconds"], right["start_seconds"])
for phase in phases:
self.assertLess(phase["start_seconds"], phase["end_seconds"])
self.assertIsInstance(phase["purpose"], str)
self.assertTrue(phase["purpose"].strip())
durations = {
phase["name"]: phase["end_seconds"] - phase["start_seconds"]
for phase in phases
}
self.assertEqual(max(durations, key=durations.get), "sustain_reveal")
def test_invalid_duration_uses_a_safe_five_second_plan(self) -> None:
plan = self.contract.ltx25_long_horizon_plan("on", 0)
self.assertTrue(plan["active"])
self.assertEqual(plan["phases"][0]["start_seconds"], 0.0)
self.assertEqual(plan["phases"][-1]["end_seconds"], 5.0)
def test_active_compiler_contract_requests_compact_continuous_output(self) -> None:
active = self.contract.ltx25_compiler_contract(
"image_to_video",
60.0,
long_horizon_mode="on",
).casefold()
inactive = self.contract.ltx25_compiler_contract(
"image_to_video",
60.0,
long_horizon_mode="off",
).casefold()
self.assertIn("silent", active)
self.assertIn("four-phase", active)
for label in ("establish", "commit", "sustain/reveal", "settle/hold"):
self.assertIn(label, active)
self.assertRegex(active, r"4\s*[-\u2013]\s*8(?:\s+\w+){0,2}\s+sentences")
self.assertRegex(
active,
r"(?:at most|no more than|maximum of|never more than)\s+200\s+words",
)
self.assertIn("minimal critical anchor set", active)
self.assertIn("one continuous take", active)
self.assertRegex(active, r"requested transformations?\b")
for prohibited_output in ("headings", "shot labels", "timecodes", "cuts"):
self.assertIn(prohibited_output, active)
self.assertNotRegex(active, r"\b(?:three|3)\s+chapters?\b")
self.assertIn("150-220 words", inactive)
self.assertNotIn("sustain/reveal", inactive)
def test_long_horizon_t2v_is_one_take_and_visual_only_is_silent(self) -> None:
budget = self.contract.ltx25_complexity_budget(
"text_to_video",
60.0,
"on",
)
self.assertEqual(budget["max_shots"], 1)
self.assertEqual(budget["max_cuts"], 0)
silent_contract = self.contract.ltx25_compiler_contract(
"text_to_video",
60.0,
long_horizon_mode="on",
audio_enabled=False,
).casefold()
self.assertIn("one continuous take with no cuts", silent_contract)
self.assertNotIn("a cut is allowed", silent_contract)
for positive_audio_instruction in (
"integrate sound",
"continuous audio identity",
"synchronized transient sounds",
"requested speech",
"audible facts",
):
self.assertNotIn(positive_audio_instruction, silent_contract)
model_prompt = self.nodes._build_model_prompt(
user_prompt="A camera slowly crosses a silent empty gallery.",
master_prompt="",
target_profile="ltx",
media_metadata={"source": "none"},
audio_mode="visual_only",
target_duration_seconds=60.0,
ltx_generation_mode="text_to_video",
ltx_long_horizon_mode="on",
).casefold()
self.assertIn(
"do not include sound, music, speech, narration, voiceover",
model_prompt,
)
for positive_audio_instruction in (
"integrate sound beside",
"one continuous audio identity",
"synchronized transient sounds",
"weave concrete sound into the timeline",
):
self.assertNotIn(positive_audio_instruction, model_prompt)
def test_long_horizon_diagnostics_are_advisory_only(self) -> None:
metadata = {
"source": "image",
"ltx25_context_schema": self.nodes.LTX25_CONTRACT_SCHEMA,
"ltx_first_frame_attached": True,
"ltx_last_frame_attached": False,
"ltx_frame_pair_attached": False,
}
mode_report = self.contract.resolve_ltx25_generation_mode(
"image_to_video", metadata, 1
)
sparse_prompt = (
"From the exact supplied first-frame viewpoint, a medium shot remains static "
"while the woman slowly turns her head and room tone continues."
)
report = self.contract.validate_ltx25_prompt(
sparse_prompt,
mode_report,
metadata,
60.0,
long_horizon_mode="on",
)
long_horizon_warnings = [
warning
for warning in report["warnings"]
if warning.startswith("ltx_long_horizon_")
]
self.assertTrue(report["ready"], report["reasons"])
self.assertTrue(long_horizon_warnings)
self.assertFalse(
[reason for reason in report["reasons"] if reason.startswith("ltx_long_horizon_")]
)
def test_long_horizon_mode_survives_salvage_and_template_fallback(self) -> None:
metadata = {
"source": "image",
"ltx25_context_schema": self.nodes.LTX25_CONTRACT_SCHEMA,
"ltx_first_frame_attached": True,
"ltx_last_frame_attached": False,
"ltx_frame_pair_attached": False,
}
context = self.nodes.GemmaContext(
user_prompt="Continue from the supplied first frame in one long take.",
source="image",
media_metadata=metadata,
)
target = self.nodes.TargetProfileConfig(
target_profile="ltx",
target_duration_seconds=60.0,
ltx_generation_mode="image_to_video",
ltx_long_horizon_mode="on",
)
cases = (
("", False),
(
"A wide shot at eye level begins from the supplied frame and slowly "
"moves backward while steady wind continues throughout.",
True,
),
)
for raw_output, expected_salvage in cases:
with self.subTest(expected_salvage=expected_salvage):
packet, parse_valid, warning = (
self.nodes.repair_or_salvage_prompt_packet(
raw_output,
context,
target,
)
)
self.assertFalse(parse_valid)
self.assertTrue(warning)
self.assertEqual(
bool(packet["metadata"].get("plain_text_salvage")),
expected_salvage,
)
controls = packet["metadata"]["controls"]
self.assertEqual(
controls["ltx_generation_mode_effective"],
"image_to_video",
)
self.assertEqual(controls["ltx_long_horizon_mode"], "on")
self.assertTrue(controls["ltx_long_horizon_active"])
self.assertEqual(
controls["ltx_long_horizon_plan"]["phase_count"],
4,
)
def test_duration_benchmark_has_fixed_controls_and_paired_seeds(self) -> None:
manifest = self.manifest
self.assertEqual(
manifest["schema_version"], "dg-ltx25-long-horizon-benchmark/1"
)
self.assertEqual(manifest["execution_status"], "protocol_only_unexecuted")
self.assertEqual(manifest["results"], [])
self.assertEqual(len(manifest["asset_requirements"]), 4)
self.assertEqual(
manifest["variant_order"],
["bare", "current_director", "long_horizon", "long_horizon_structural"],
)
self.assertEqual(
{(pair["stage_1"], pair["stage_2"]) for pair in manifest["seed_pairs"]},
{(17, 42), (101, 420), (809, 421)},
)
self.assertEqual(
[case["duration_seconds"] for case in manifest["cases"]],
[5.0, 20.0, 60.0],
)
self.assertEqual(
[case["expected_auto_long_horizon_active"] for case in manifest["cases"]],
[False, False, True],
)
for case in manifest["cases"]:
with self.subTest(case=case["id"]):
self.assertEqual(
case["frames"],
int(case["duration_seconds"] * case["fps"]) + 1,
)
self.assertEqual(case["frames"] % 8, 1)
self.assertEqual(list(case["variants"]), manifest["variant_order"])
self.assertEqual(case["mode"], "image_to_video")
self.assertEqual(
case["fixed_generation_control"]["sampler"],
"euler_ancestral",
)
self.assertEqual(
case["fixed_generation_control"]["scheduler"],
"bong_tangent",
)
self.assertEqual(
case["fixed_generation_control"]["stage_1_steps"],
12,
)
self.assertLessEqual(len(case["critical_anchors"]), 3)
self.assertEqual(
case["rubric"]["phase_names"],
["establish", "commit", "sustain_reveal", "settle_hold"],
)
self.assertFalse(case["rubric"]["diagnostics_block_generation"])
auto_plan = self.contract.ltx25_long_horizon_plan(
"auto", case["duration_seconds"]
)
self.assertEqual(
auto_plan["active"], case["expected_auto_long_horizon_active"]
)
long_prompt = case["variants"]["long_horizon"]["prompt"]
structural_prompt = case["variants"]["long_horizon_structural"]["prompt"]
self.assertEqual(long_prompt, structural_prompt)
self.assertEqual(
case["variants"]["long_horizon"]["structural_conditioning"],
"first_frame",
)
self.assertEqual(
case["variants"]["long_horizon_structural"][
"structural_conditioning"
],
"first_frame_plus_depth",
)
def test_checked_in_long_horizon_prompts_meet_output_shape_contract(self) -> None:
word_re = re.compile(r"\b[\w'-]+\b")
sentence_re = re.compile(r"[^.!?]+(?:[.!?]+|$)")
timecode_re = re.compile(r"\b\d{1,2}:\d{2}(?:\.\d+)?\b|\[\s*shot\b", re.I)
cut_re = re.compile(r"\b(?:hard\s+cut|jump\s+cut|cut|cuts|dissolve|wipe)\b", re.I)
for case in self.manifest["cases"]:
prompt = case["variants"]["long_horizon"]["prompt"]
with self.subTest(case=case["id"]):
word_count = len(word_re.findall(prompt))
sentence_count = sum(
1 for match in sentence_re.finditer(prompt) if match.group(0).strip()
)
self.assertGreaterEqual(word_count, case["rubric"]["min_words"])
self.assertLessEqual(word_count, case["rubric"]["max_words"])
self.assertGreaterEqual(sentence_count, case["rubric"]["sentence_range"][0])
self.assertLessEqual(sentence_count, case["rubric"]["sentence_range"][1])
self.assertNotIn("\n", prompt)
self.assertIsNone(timecode_re.search(prompt))
self.assertIsNone(cut_re.search(prompt))
diagnostics = self.contract.analyze_ltx25_long_horizon_prompt(
prompt,
"on",
case["duration_seconds"],
case["mode"],
)
self.assertEqual(diagnostics["warnings"], [])
if __name__ == "__main__":
unittest.main()
tests/test_measured_audio_director.py
+108
View File
@@ -0,0 +1,108 @@
from __future__ import annotations
import importlib.util
import json
import sys
import unittest
import uuid
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
COMFY_ROOT = ROOT.parents[1]
def load_nodes_module():
if str(COMFY_ROOT) not in sys.path:
sys.path.insert(0, str(COMFY_ROOT))
package_name = f"diffusiongemma_measured_audio_fixture_{uuid.uuid4().hex}"
spec = importlib.util.spec_from_file_location(
package_name,
ROOT / "__init__.py",
submodule_search_locations=[str(ROOT)],
)
if spec is None or spec.loader is None:
raise RuntimeError("could not load DiffusionGemma Prompt Builder")
package = importlib.util.module_from_spec(spec)
sys.modules[package_name] = package
spec.loader.exec_module(package)
return sys.modules[f"{package_name}.nodes"]
class MeasuredAudioDirectorTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.nodes = load_nodes_module()
def test_cot_exposes_optional_measured_audio_report_input(self) -> None:
optional = self.nodes.DiffusionGemmaCoTGenerator.INPUT_TYPES()["optional"]
self.assertIn("measured_audio_report_json", optional)
input_type, spec = optional["measured_audio_report_json"]
self.assertEqual(input_type, "STRING")
self.assertTrue(spec["forceInput"])
def test_context_enrichment_is_bounded_and_does_not_mutate_caller(self) -> None:
original = self.nodes.GemmaContext(
user_prompt="One continuous dance shot",
media_metadata={"source": "image"},
)
report = {
"schema": "diffusiongemma.music_audition_report",
"status": "ready",
"selected_audio_sha256": "abc123",
"selected_excerpt": {
"start_seconds": 44.0,
"duration_seconds": 20.0,
"onset_rate_hz": 4.2,
"tonal_family_stability": 0.86,
},
}
enriched = self.nodes._context_with_measured_audio_report(
original,
json.dumps(report),
)
self.assertNotIn("ltx_measured_audio_report", original.media_metadata)
self.assertEqual(
enriched.media_metadata["ltx_measured_audio_report"]["selected_audio_sha256"],
"abc123",
)
def test_invalid_measured_audio_report_fails_before_director(self) -> None:
for value in ("[]", "not json"):
with self.subTest(value=value):
with self.assertRaisesRegex(ValueError, "valid JSON object"):
self.nodes._normalize_measured_audio_report_json(value)
def test_ltx_compiler_treats_rendered_audio_as_authoritative(self) -> None:
metadata = {
"source": "none",
"ltx_measured_audio_report": {
"status": "ready",
"selected_excerpt": {
"start_seconds": 44.0,
"duration_seconds": 20.0,
"onset_rate_hz": 4.2,
"stable_windows": [[44.0, 50.0], [56.0, 64.0]],
},
},
}
prompt = self.nodes._build_model_prompt(
"A singer dances in one continuous shot",
self.nodes.DEFAULT_MASTER_PROMPT,
"ltx",
metadata,
audio_mode="explicit_sound_design",
audio_guidance="Perform to the connected selected song.",
target_duration_seconds=20.0,
ltx_generation_mode="text_to_video",
)
self.assertIn("exact rendered song excerpt", prompt)
self.assertIn("requested BPM, key, energy, or meter are not facts", prompt)
self.assertIn("stable or lower-density spans", prompt)
self.assertIn("Vocal-activity proxies do not by themselves authorize visible singing", prompt)
self.assertIn("performance-mode control decides whether the performer lip-syncs", prompt)
self.assertIn('"ltx_measured_audio_report"', prompt)
if __name__ == "__main__":
unittest.main()
tests/test_minimax_h3_camera_contract_repair.py
+245
View File
@@ -0,0 +1,245 @@
from __future__ import annotations
import importlib.util
import json
import sys
import unittest
import uuid
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
COMFY_ROOT = ROOT.parents[1]
def load_nodes_module():
if str(COMFY_ROOT) not in sys.path:
sys.path.insert(0, str(COMFY_ROOT))
package_name = f"diffusiongemma_h3_camera_repair_{uuid.uuid4().hex}"
spec = importlib.util.spec_from_file_location(
package_name,
ROOT / "__init__.py",
submodule_search_locations=[str(ROOT)],
)
if spec is None or spec.loader is None:
raise RuntimeError("could not load DiffusionGemma Prompt Builder")
package = importlib.util.module_from_spec(spec)
sys.modules[package_name] = package
spec.loader.exec_module(package)
return sys.modules[f"{package_name}.nodes"]
class MiniMaxH3CameraContractRepairTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.nodes = load_nodes_module()
@staticmethod
def multi_shot_t2va_prompt() -> str:
return (
"integrated_multimodal_description: [Shot 1] A runner waits at a rain-darkened starting line. "
"[Shot 2] At 00:06.000, the camera cuts to a low tracking view as the runner accelerates around the bend. "
"[Shot 3] At 00:13.000, the camera cuts to the runner stopping beneath the finish light. "
"The final frame holds on the runner standing still beneath the light.\n\n"
"overall_soundscape: Rain, measured footfalls, and distant crowd ambience surround the track.\n\n"
"non_diegetic_music: N/A"
)
@staticmethod
def fully_specified_t2va_prompt() -> str:
return (
"integrated_multimodal_description: [Shot 1] A locked-off medium shot holds on a runner at the starting line. "
"[Shot 2] At 00:06.000, the camera cuts to a low tracking view and tracks beside the accelerating runner. "
"[Shot 3] At 00:13.000, the camera cuts to a static close view as the runner stops beneath the finish light.\n\n"
"overall_soundscape: Rain and measured footfalls surround the track.\n\n"
"non_diegetic_music: N/A"
)
@classmethod
def ref2va_prompt(cls) -> str:
timeline = cls.multi_shot_t2va_prompt().split(
"integrated_multimodal_description: ", 1
)[1].split("\n\noverall_soundscape:", 1)[0]
return (
"subject_definitions:\n"
"<Subject 1>: A runner wearing a dark rain jacket and silver shoes.\n\n"
"summary:\n"
"[reference generation] A runner completes a measured lap in the rain.\n\n"
"retention_analysis:\n"
"<Subject 1>: fully_preserved - The face, dark jacket, silver shoes, and proportions remain unchanged.\n\n"
"detailed_description:\n"
"Naturalistic live-action sports photography uses cool rain-muted color and restrained film grain. "
f"{timeline}\n\n"
"overall_soundscape:\n"
"Rain, measured footfalls, and distant crowd ambience surround the track.\n\n"
"non_diegetic_music:\n"
"N/A"
)
def test_multi_shot_repair_adds_only_safe_locked_holds_and_is_idempotent(self) -> None:
prompt = self.multi_shot_t2va_prompt()
repaired, repaired_shots = (
self.nodes._repair_minimax_h3_unspecified_shot_cameras(prompt, "t2va")
)
self.assertEqual(repaired_shots, [1, 3])
clause = self.nodes._MINIMAX_H3_SAFE_CAMERA_HOLD
self.assertEqual(repaired.count(clause), 2)
self.assertEqual(repaired.replace(f" {clause}", ""), prompt)
self.assertIn(
"the camera cuts to a low tracking view as the runner accelerates",
repaired,
)
self.assertNotIn(
"minimax_h3_shot_camera_unspecified",
self.nodes._minimax_h3_prompt_validation_reasons(
repaired,
20.0,
"A runner completes a measured lap in the rain.",
),
)
second_pass, second_pass_shots = (
self.nodes._repair_minimax_h3_unspecified_shot_cameras(
repaired,
"t2va",
)
)
self.assertEqual(second_pass, repaired)
self.assertEqual(second_pass_shots, [])
def test_already_specified_camera_choreography_is_byte_stable(self) -> None:
prompt = self.fully_specified_t2va_prompt()
repaired, repaired_shots = (
self.nodes._repair_minimax_h3_unspecified_shot_cameras(prompt, "t2va")
)
self.assertEqual(repaired, prompt)
self.assertEqual(repaired_shots, [])
def test_expressive_h3_camera_choreography_is_recognized_and_byte_stable(self) -> None:
prompt = (
"integrated_multimodal_description: "
"[Shot 1] A dancer crosses a mirrored floor. The camera swirls clockwise around her and rises into a high view. "
"[Shot 2] At 00:05.000, the camera cuts to the dancer turning toward the lights. The camera rolls thirty degrees counterclockwise while tracking her. "
"[Shot 3] At 00:10.000, the camera cuts to a line of foreground prisms. The viewpoint rotates around the dancer as pronounced layered parallax reveals the stage. "
"[Shot 4] At 00:15.000, the camera cuts to the final beat. The camera whip-pans right, then cranes up along one continuous path to the ending wide frame.\n\n"
"overall_soundscape: Footfalls and fabric movement follow the performance.\n\n"
"non_diegetic_music: Percussion drives the four camera beats."
)
repaired, repaired_shots = (
self.nodes._repair_minimax_h3_unspecified_shot_cameras(prompt, "t2va")
)
self.assertEqual(repaired, prompt)
self.assertEqual(repaired_shots, [])
self.assertNotIn(self.nodes._MINIMAX_H3_SAFE_CAMERA_HOLD, repaired)
self.assertNotIn(
"minimax_h3_shot_camera_unspecified",
self.nodes._minimax_h3_prompt_validation_reasons(
repaired,
20.0,
"A dynamic four-shot dance with orbiting, rolling, whip movement, and pronounced parallax.",
minimax_h3_mode="t2va",
minimax_h3_shot_count="4",
),
)
def test_empty_shot_body_stays_fail_closed(self) -> None:
prompt = self.multi_shot_t2va_prompt().replace(
"[Shot 1] A runner waits at a rain-darkened starting line. ",
"[Shot 1] ",
)
repaired, repaired_shots = (
self.nodes._repair_minimax_h3_unspecified_shot_cameras(prompt, "t2va")
)
self.assertNotIn(1, repaired_shots)
self.assertIn(
"minimax_h3_empty_visual_timeline",
self.nodes._minimax_h3_prompt_validation_reasons(
repaired,
20.0,
"A runner completes a measured lap in the rain.",
),
)
def test_ref2va_repair_is_confined_to_detailed_description(self) -> None:
prompt = self.ref2va_prompt()
original_sections = self.nodes._minimax_h3_ref_sections(prompt)
self.assertIsNotNone(original_sections)
repaired, repaired_shots = (
self.nodes._repair_minimax_h3_unspecified_shot_cameras(
prompt,
"ref2va",
)
)
repaired_sections = self.nodes._minimax_h3_ref_sections(repaired)
self.assertIsNotNone(repaired_sections)
self.assertEqual(repaired_shots, [1, 3])
for field_name in self.nodes._MINIMAX_H3_REF_FIELDS:
if field_name != "detailed_description":
self.assertEqual(
repaired_sections[field_name],
original_sections[field_name],
)
self.assertTrue(
repaired_sections["detailed_description"].startswith(
"Naturalistic live-action sports photography"
)
)
def test_strict_packet_splitter_repairs_before_readiness_validation(self) -> None:
prompt = self.multi_shot_t2va_prompt()
packet = {
"ltx_prompt": "",
"ideogram_prompt": "",
"minimax_h3_prompt": prompt,
"negative_prompt": "",
"scene_segments": [],
"metadata": {},
}
context = self.nodes.GemmaContext(
user_prompt="A runner completes a measured lap in the rain.",
images=None,
source="text",
media_metadata={"source": "text", "duration_seconds": 20.0},
)
target = self.nodes.TargetProfileConfig(
target_profile="minimax_h3",
audio_mode="auto_scene_audio",
target_duration_seconds=20.0,
minimax_h3_mode="t2va",
minimax_h3_dialogue_mode="off",
)
result = self.nodes.DiffusionGemmaJSONSplitter().split(
json.dumps(packet),
context,
target,
)
metadata = json.loads(result[4])
repaired_prompt = result[13]
self.assertTrue(result[12], metadata)
self.assertEqual(
metadata["minimax_h3_camera_contract_repair"],
{
"strategy": "locked_off_stable_view",
"shots": [1, 3],
"repair_count": 2,
},
)
self.assertEqual(
repaired_prompt.count(self.nodes._MINIMAX_H3_SAFE_CAMERA_HOLD),
2,
)
self.assertNotIn(
"minimax_h3_shot_camera_unspecified",
metadata["blocked_reasons"],
)
if __name__ == "__main__":
unittest.main()
tests/test_minimax_h3_dialogue.py
+12
View File
@@ -151,6 +151,8 @@ class MiniMaxH3DialogueTests(unittest.TestCase):
"minimax_h3_dialogue_line_count",
"minimax_h3_dialogue_guidance",
"ltx_generation_mode",
"ltx_long_horizon_mode",
"ltx_camera_capability",
],
)
choices, mode_config = optional["minimax_h3_dialogue_mode"]
@@ -229,6 +231,16 @@ class MiniMaxH3DialogueTests(unittest.TestCase):
self.assertIn(guidance, refinement_prompt)
self.assertRegex(refinement_prompt, r"(?is)exactly\s+2.+?<d>")
def test_dialogue_off_does_not_override_the_separate_music_vocal_contract(self) -> None:
instruction = self.nodes._minimax_h3_dialogue_instruction(
"off",
minimax_h3_mode="ref2va",
)
self.assertIn("Spoken dialogue is disabled", instruction)
self.assertIn("governed separately by the music-performance contract", instruction)
self.assertNotIn("or singing", instruction)
self.assertNotIn("no singing", instruction.casefold())
def test_ref2va_required_dialogue_repeats_explicit_subject_speaker_binding(self) -> None:
model_prompt = self.nodes._build_model_prompt(
"Create a two-image reference sequence with two spoken lines.",
tests/test_minimax_h3_fast_test_workflow.py
+549
View File
@@ -0,0 +1,549 @@
from __future__ import annotations
import copy
import importlib.util
import json
import tempfile
import unittest
import urllib.error
import urllib.request
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
MIGRATION_PATH = REPO_ROOT / "tools" / "migrate_minimax_h3_fast_test.py"
SPEC = importlib.util.spec_from_file_location("minimax_h3_fast_test", MIGRATION_PATH)
if SPEC is None or SPEC.loader is None:
raise RuntimeError(f"Cannot load migration module: {MIGRATION_PATH}")
MIGRATION = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(MIGRATION)
LIVE_CAPTURE = Path(
r"C:\Users\danrh\Desktop\14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_LIVE_UNSAVED_CAPTURE_20260822_0057.json"
)
def _input(name: str, type_name: str) -> dict:
return {"name": name, "type": type_name, "link": None}
def _output(name: str, type_name: str) -> dict:
return {"name": name, "type": type_name, "links": []}
def _node(
node_id: int,
node_type: str,
*,
inputs: list[tuple[str, str]] = (),
outputs: list[tuple[str, str]] = (),
widgets: list | None = None,
pos: tuple[float, float] = (0.0, 0.0),
title: str | None = None,
) -> dict:
return {
"id": node_id,
"type": node_type,
"pos": list(pos),
"size": [300.0, 100.0],
"flags": {},
"order": node_id,
"mode": 0,
"inputs": [_input(name, type_name) for name, type_name in inputs],
"outputs": [_output(name, type_name) for name, type_name in outputs],
"title": title,
"properties": {"Node name for S&R": node_type},
"widgets_values": list(widgets or []),
}
def _main_connect(
workflow: dict,
link_id: int,
origin: dict,
output_name: str,
target: dict,
input_name: str,
link_type: str,
) -> None:
output_slot = next(
index for index, item in enumerate(origin["outputs"]) if item["name"] == output_name
)
input_slot = next(
index for index, item in enumerate(target["inputs"]) if item["name"] == input_name
)
workflow["links"].append(
[link_id, origin["id"], output_slot, target["id"], input_slot, link_type]
)
origin["outputs"][output_slot]["links"].append(link_id)
target["inputs"][input_slot]["link"] = link_id
def _sub_connect(
subgraph: dict,
link_id: int,
origin: dict,
output_name: str,
target: dict,
input_name: str,
link_type: str,
) -> None:
output_slot = next(
index for index, item in enumerate(origin["outputs"]) if item["name"] == output_name
)
input_slot = next(
index for index, item in enumerate(target["inputs"]) if item["name"] == input_name
)
subgraph["links"].append(
{
"id": link_id,
"origin_id": origin["id"],
"origin_slot": output_slot,
"target_id": target["id"],
"target_slot": input_slot,
"type": link_type,
}
)
origin["outputs"][output_slot]["links"].append(link_id)
target["inputs"][input_slot]["link"] = link_id
def _fixture() -> dict:
duration = _node(
178,
"PrimitiveFloat",
outputs=[("FLOAT", "FLOAT")],
widgets=[15.0],
title="Master duration",
)
splitter = _node(
187,
"DiffusionGemmaJSONSplitter",
widgets=["", 1.0, 32, "9:16 (Portrait Widescreen)"],
title="Resolution",
)
loader = _node(
661,
"UNETLoader",
outputs=[("MODEL", "MODEL")],
widgets=["minimax_h3_ref2va_int8_convrot.safetensors", "default"],
)
lora = _node(
713,
"LoraLoaderModelOnly",
inputs=[("model", "MODEL")],
outputs=[("MODEL", "MODEL")],
widgets=["minimax_h3_ref2v_turbo_4step_v0.1_bf16.safetensors", 1.0],
)
sage = _node(
715,
"PathchSageAttentionKJ",
inputs=[("model", "MODEL")],
outputs=[("MODEL", "MODEL")],
widgets=["auto", False],
)
cache = _node(
712,
"EasyCache",
inputs=[("model", "MODEL")],
outputs=[("MODEL", "MODEL")],
widgets=[0.2, 0.15, 0.95, True],
)
compile_node = _node(
714,
"TorchCompileModel",
inputs=[("model", "MODEL")],
outputs=[("MODEL", "MODEL")],
widgets=["inductor"],
)
outer = _node(
710,
"h3-fast-test-subgraph",
inputs=[("model", "MODEL")],
outputs=[("IMAGE", "IMAGE")],
)
project = _node(
676,
"DiffusionGemmaProjectMasterContract",
inputs=[("production_duration_seconds", "FLOAT")],
outputs=[("project_manifest_json", "STRING")],
widgets=[15.0, "9:16", 0.0, 15.0, "MiniMax H3 Ref2VA", "{}", 15.0],
)
target = _node(
673,
"DiffusionGemmaMiniMaxH3TargetProfile",
widgets=["ref2va", 0, "auto_scene_audio", "", "auto", 12, "off", 2, "", "auto", "", 0],
)
planner = _node(
677,
"DiffusionGemmaAudioAwareMultiShotPlanner",
inputs=[("project_manifest_json", "STRING")],
outputs=[("plan_json", "STRING")],
widgets=[0.0, 15.0, 12.0, 5.0, 15.0],
)
nodes = [
duration,
splitter,
loader,
lora,
sage,
cache,
compile_node,
outer,
project,
target,
planner,
]
workflow = {
"id": "fixture-workflow",
"revision": 12,
"last_node_id": 715,
"last_link_id": 1405,
"version": 0.4,
"nodes": nodes,
"links": [],
"groups": [],
"definitions": {"subgraphs": []},
"extra": {"fixture": {"native_shot_blocks": 4, "active_lanes": 1}},
}
_main_connect(workflow, 1398, loader, "MODEL", lora, "model", "MODEL")
_main_connect(workflow, 1404, lora, "MODEL", sage, "model", "MODEL")
_main_connect(workflow, 1405, sage, "MODEL", cache, "model", "MODEL")
_main_connect(workflow, 1402, cache, "MODEL", compile_node, "model", "MODEL")
_main_connect(workflow, 1403, compile_node, "MODEL", outer, "model", "MODEL")
_main_connect(
workflow, 1224, duration, "FLOAT", project, "production_duration_seconds", "FLOAT"
)
_main_connect(
workflow,
1244,
project,
"project_manifest_json",
planner,
"project_manifest_json",
"STRING",
)
sampler_select = _node(
657,
"KSamplerSelect",
outputs=[("SAMPLER", "SAMPLER")],
widgets=["res_multistep"],
)
scheduler = _node(
658,
"BasicScheduler",
inputs=[("model", "MODEL")],
outputs=[("SIGMAS", "SIGMAS")],
widgets=["simple", 8, 1.0],
)
h3_ids = [667, 684, 692, 700]
guider_ids = [660, 686, 694, 702]
sampler_ids = [659, 687, 695, 703]
sub_nodes = [sampler_select, scheduler]
h3_nodes = []
guiders = []
samplers = []
for lane, (h3_id, guider_id, sampler_id) in enumerate(
zip(h3_ids, guider_ids, sampler_ids)
):
y = float(lane * 970)
h3 = _node(
h3_id,
"MiniMaxH3ReferenceToVideo",
outputs=[("positive", "CONDITIONING"), ("LATENT", "LATENT")],
pos=(3748.0, y + 204.0),
)
guider = _node(
guider_id,
"BasicGuider",
inputs=[("model", "MODEL"), ("conditioning", "CONDITIONING")],
outputs=[("GUIDER", "GUIDER")],
pos=(4198.0, y + 354.0),
)
sampler = _node(
sampler_id,
"SamplerCustomAdvanced",
inputs=[
("noise", "NOISE"),
("guider", "GUIDER"),
("sampler", "SAMPLER"),
("sigmas", "SIGMAS"),
("latent_image", "LATENT"),
],
outputs=[("output", "LATENT")],
pos=(4618.0, y + 274.0),
)
h3_nodes.append(h3)
guiders.append(guider)
samplers.append(sampler)
sub_nodes.extend([h3, guider, sampler])
subgraph = {
"id": "h3-fast-test-subgraph",
"name": "New Subgraph",
"version": 1,
"revision": 12,
"state": {"lastNodeId": 715, "lastLinkId": 1405},
"inputNode": {"id": -10},
"outputNode": {"id": -20},
"inputs": [],
"outputs": [],
"nodes": sub_nodes,
"links": [],
"groups": [],
"widgets": [],
"config": {},
"extra": {},
}
link_id = 1259
for h3, guider, sampler in zip(h3_nodes, guiders, samplers):
_sub_connect(
subgraph, link_id, h3, "positive", guider, "conditioning", "CONDITIONING"
)
link_id += 1
_sub_connect(
subgraph, link_id, h3, "LATENT", sampler, "latent_image", "LATENT"
)
link_id += 1
workflow["definitions"]["subgraphs"].append(subgraph)
return workflow
def _node_by_id(workflow: dict, node_id: int) -> dict:
for node in workflow["nodes"]:
if int(node["id"]) == int(node_id):
return node
for subgraph in workflow["definitions"]["subgraphs"]:
for node in subgraph["nodes"]:
if int(node["id"]) == int(node_id):
return node
raise KeyError(node_id)
class FastTestMigrationUnitTests(unittest.TestCase):
def setUp(self) -> None:
self.source = _fixture()
self.source_before = copy.deepcopy(self.source)
self.migrated = MIGRATION.migrate_workflow(
self.source, source_file_sha256="a" * 64
)
def test_source_is_unchanged_and_migration_is_idempotent(self) -> None:
self.assertEqual(self.source, self.source_before)
MIGRATION.validate_workflow(
self.migrated, require_benchmark_defaults=True
)
self.assertEqual(
MIGRATION.migrate_workflow(self.migrated), self.migrated
)
def test_official_fast_asset_and_sampling_contract_is_active(self) -> None:
marker = self.migrated["extra"][MIGRATION.MIGRATION_SCHEMA]
loader = _node_by_id(self.migrated, marker["nodes"]["loader"])
lora = _node_by_id(self.migrated, marker["nodes"]["lora"])
sage = _node_by_id(self.migrated, marker["nodes"]["sage"])
sigma = _node_by_id(self.migrated, marker["nodes"]["sigma_shift"])
self.assertEqual(loader["widgets_values"][0], MIGRATION.FAST_MODEL)
self.assertEqual(lora["widgets_values"], [MIGRATION.FAST_LORA, 1.0])
self.assertEqual(sage["widgets_values"], ["auto", False])
self.assertEqual(
sigma["widgets_values"],
[MIGRATION.FAST_VIDEO_SHIFT, MIGRATION.FAST_AUDIO_SHIFT],
)
self.assertEqual(
marker["assets"]["turbo_lora"]["expected_sha256"],
MIGRATION.FAST_LORA_SHA256,
)
subgraph = self.migrated["definitions"]["subgraphs"][0]
sampler = next(
node for node in subgraph["nodes"] if node["type"] == "KSamplerSelect"
)
scheduler = next(
node for node in subgraph["nodes"] if node["type"] == "BasicScheduler"
)
self.assertEqual(sampler["widgets_values"], ["euler"])
self.assertEqual(scheduler["widgets_values"], ["simple", 4, 1.0])
def test_easycache_and_compile_are_disconnected_bypasses(self) -> None:
marker = self.migrated["extra"][MIGRATION.MIGRATION_SCHEMA]
disabled_ids = {
marker["nodes"]["disabled_easycache"],
marker["nodes"]["disabled_torch_compile"],
}
self.assertTrue(
all(_node_by_id(self.migrated, node_id)["mode"] == 4 for node_id in disabled_ids)
)
self.assertFalse(
any(
int(link[1]) in disabled_ids or int(link[3]) in disabled_ids
for link in self.migrated["links"]
)
)
def test_each_lane_cleans_conditioning_but_not_latent(self) -> None:
marker = self.migrated["extra"][MIGRATION.MIGRATION_SCHEMA]
self.assertEqual(len(marker["nodes"]["conditioning_cleanup"]), 4)
subgraph = self.migrated["definitions"]["subgraphs"][0]
links = {int(link["id"]): link for link in subgraph["links"]}
h3_nodes = sorted(
[node for node in subgraph["nodes"] if node["type"] == "MiniMaxH3ReferenceToVideo"],
key=lambda node: node["pos"][1],
)
guiders = sorted(
[node for node in subgraph["nodes"] if node["type"] == "BasicGuider"],
key=lambda node: node["pos"][1],
)
samplers = sorted(
[node for node in subgraph["nodes"] if node["type"] == "SamplerCustomAdvanced"],
key=lambda node: node["pos"][1],
)
for h3, cleanup_id, guider, sampler in zip(
h3_nodes,
marker["nodes"]["conditioning_cleanup"],
guiders,
samplers,
):
cleanup = _node_by_id(self.migrated, cleanup_id)
self.assertEqual(cleanup["type"], MIGRATION.CLEANUP_NODE_TYPE)
self.assertEqual(cleanup["inputs"][0]["name"], MIGRATION.CLEANUP_INPUT)
self.assertEqual(cleanup["outputs"][0]["name"], MIGRATION.CLEANUP_OUTPUT)
to_cleanup = links[int(cleanup["inputs"][0]["link"])]
to_guider = links[int(guider["inputs"][1]["link"])]
latent = links[int(sampler["inputs"][4]["link"])]
self.assertEqual(int(to_cleanup["origin_id"]), int(h3["id"]))
self.assertEqual(int(to_guider["origin_id"]), int(cleanup["id"]))
self.assertEqual(int(latent["origin_id"]), int(h3["id"]))
def test_fast_defaults_remain_configurable_after_creation(self) -> None:
self.assertEqual(_node_by_id(self.migrated, 178)["widgets_values"], [5.0])
self.assertEqual(_node_by_id(self.migrated, 187)["widgets_values"][1], 0.4)
edited = copy.deepcopy(self.migrated)
_node_by_id(edited, 178)["widgets_values"][0] = 12.0
_node_by_id(edited, 187)["widgets_values"][1] = 0.8
MIGRATION.validate_workflow(edited)
self.assertEqual(MIGRATION.migrate_workflow(edited), edited)
def test_project_native_shot_and_lane_contract_topology_is_preserved(self) -> None:
before_links = {
tuple(link)
for link in self.source["links"]
if int(link[1]) in {178, 676, 673, 677}
or int(link[3]) in {178, 676, 673, 677}
}
after_links = {
tuple(link)
for link in self.migrated["links"]
if int(link[1]) in {178, 676, 673, 677}
or int(link[3]) in {178, 676, 673, 677}
}
self.assertEqual(before_links, after_links)
self.assertEqual(
self.migrated["extra"]["fixture"],
{"native_shot_blocks": 4, "active_lanes": 1},
)
semantics = self.migrated["extra"][MIGRATION.MIGRATION_SCHEMA][
"generation_semantics"
]
self.assertEqual(semantics["lazy_generation_lane_capacity"], 4)
self.assertTrue(semantics["project_and_native_shot_contract_preserved"])
self.assertEqual(
semantics["cleanup_node_type"], MIGRATION.CLEANUP_NODE_TYPE
)
self.assertFalse(semantics["cleanup_backend_output_node"])
def test_changed_source_chain_fails_closed(self) -> None:
broken = _fixture()
outer = _node_by_id(broken, 710)
link_id = outer["inputs"][0]["link"]
link = next(link for link in broken["links"] if int(link[0]) == int(link_id))
link[1] = 715
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.migrate_workflow(broken, source_file_sha256="b" * 64)
def test_cleanup_or_latent_contract_break_is_rejected(self) -> None:
broken = copy.deepcopy(self.migrated)
marker = broken["extra"][MIGRATION.MIGRATION_SCHEMA]
cleanup = _node_by_id(broken, marker["nodes"]["conditioning_cleanup"][0])
cleanup["outputs"][0]["links"] = []
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.validate_workflow(broken)
def test_file_migration_creates_only_a_new_sibling_and_is_repeatable(self) -> None:
with tempfile.TemporaryDirectory() as temporary:
root = Path(temporary)
source = root / "source.json"
output = root / "source_FAST.json"
source_bytes = json.dumps(self.source, indent=2).encode("utf-8") + b"\n"
source.write_bytes(source_bytes)
source_hash = MIGRATION._file_hash(source_bytes)
migrated, observed_hash, written = MIGRATION.migrate_file(
source,
output_path=output,
write=True,
expected_source_sha256=source_hash,
)
self.assertEqual(observed_hash, source_hash)
self.assertEqual(written, output.resolve())
self.assertEqual(source.read_bytes(), source_bytes)
self.assertTrue(output.is_file())
self.assertEqual(json.loads(output.read_text(encoding="utf-8")), migrated)
_same, _hash, second_written = MIGRATION.migrate_file(
source,
output_path=output,
write=True,
expected_source_sha256=source_hash,
)
self.assertIsNone(second_written)
self.assertEqual(source.read_bytes(), source_bytes)
def test_output_may_never_equal_source(self) -> None:
with tempfile.TemporaryDirectory() as temporary:
source = Path(temporary) / "source.json"
source.write_text(json.dumps(self.source), encoding="utf-8")
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.migrate_file(
source,
output_path=source,
write=True,
expected_source_sha256=MIGRATION._file_hash(source.read_bytes()),
)
@unittest.skipUnless(LIVE_CAPTURE.is_file(), "exact live capture is absent")
class ExactLiveCaptureIntegrationTests(unittest.TestCase):
def test_exact_captured_unsaved_canvas_migrates_without_writing(self) -> None:
migrated, source_hash, written = MIGRATION.migrate_file(
LIVE_CAPTURE,
output_path=MIGRATION.DEFAULT_OUTPUT,
write=False,
expected_source_sha256=MIGRATION.SOURCE_CAPTURE_SHA256,
)
self.assertEqual(source_hash, MIGRATION.SOURCE_CAPTURE_SHA256)
self.assertIsNone(written)
MIGRATION.validate_workflow(migrated, require_benchmark_defaults=True)
marker = migrated["extra"][MIGRATION.MIGRATION_SCHEMA]
self.assertEqual(marker["source"]["file_sha256"], source_hash)
self.assertEqual(
marker["generation_semantics"]["lazy_generation_lane_capacity"], 4
)
class InstalledCleanupBackendContractTests(unittest.TestCase):
def test_live_backend_reports_cleanup_passthrough_is_not_an_output_node(self) -> None:
try:
with urllib.request.urlopen(
"http://127.0.0.1:8188/object_info/FL_UnloadAllModels",
timeout=2.0,
) as response:
payload = json.load(response)
except (OSError, urllib.error.URLError) as exc:
self.skipTest(f"local ComfyUI object_info is unavailable: {exc}")
contract = payload["FL_UnloadAllModels"]
self.assertFalse(contract["output_node"])
self.assertEqual(contract["input_order"]["required"], ["value"])
self.assertEqual(contract["output_name"], ["*"])
self.assertEqual(contract["output"], ["*"])
if __name__ == "__main__":
unittest.main()
tests/test_minimax_h3_identity_relay_workflow.py
+380
View File
@@ -0,0 +1,380 @@
from __future__ import annotations
import copy
import hashlib
import importlib.util
import json
import tempfile
import unittest
import urllib.error
import urllib.request
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
MIGRATION_PATH = REPO_ROOT / "tools" / "migrate_minimax_h3_identity_relay.py"
SPEC = importlib.util.spec_from_file_location("minimax_h3_identity_relay", MIGRATION_PATH)
if SPEC is None or SPEC.loader is None: # pragma: no cover
raise RuntimeError(f"Cannot load migration module: {MIGRATION_PATH}")
MIGRATION = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(MIGRATION)
CAPTURE = Path(
r"C:\Users\danrh\Desktop\14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_EXECUTED_423S_CAPTURE_b43059cc_20260822.json"
)
SIBLING = Path(
r"C:\Users\danrh\Desktop\14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_DUAL_IDENTITY_P3_RELAY_GUARDED_V4_20260822.json"
)
SUPERSEDED_SIBLINGS = (
Path(
r"C:\Users\danrh\Desktop\14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_DUAL_IDENTITY_P3_RELAY_GUARDED_20260822.json"
),
Path(
r"C:\Users\danrh\Desktop\14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_DUAL_IDENTITY_P3_RELAY_GUARDED_V2_20260822.json"
),
Path(
r"C:\Users\danrh\Desktop\14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_DUAL_IDENTITY_P3_RELAY_GUARDED_V3_20260822.json"
),
)
CAPTURE_SHA256 = "207ae087b7c8021ff396628480a4f382a21f993f1ddd27337bb051a0cbca3a17"
SIBLING_SHA256 = "fe4261c2dd4fb16df6acf05d8735beb274a8e2d80821936dcda0271d2978d6fa"
SUPERSEDED_SIBLING_SHA256 = (
"c03c26244239f878195fccd521a9017a32f105f7209af06e0189afa1879c1281",
"265f27188a689d517b6f55bda28b693b6881ce0068e712ea95fe5c4865a45f3f",
"a1d203aebe00235a8743f4811b629295b927b4c080e89ab5eb9f9a182bfd355c",
)
LEGACY_SAVED_COPIES = (
Path(
r"C:\Users\danrh\Desktop\14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_STABILIZED_H3.json"
),
Path(
r"C:\ComfyUI\app\user\default\workflows\14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_STABILIZED_H3.json"
),
)
LEGACY_SAVED_SHA256 = "4431ef6269782be1cafbef350800ab01b376b094db7c62d7b8fe335004aac7d8"
def _sha256(path: Path) -> str:
return hashlib.sha256(path.read_bytes()).hexdigest()
def _node(nodes: list[dict], node_id: int) -> dict:
matches = [item for item in nodes if int(item.get("id", -1)) == int(node_id)]
if len(matches) != 1:
raise AssertionError(f"Expected one node {node_id}; found {len(matches)}")
return matches[0]
def _one(nodes: list[dict], node_type: str) -> dict:
matches = [item for item in nodes if item.get("type") == node_type]
if len(matches) != 1:
raise AssertionError(f"Expected one {node_type}; found {len(matches)}")
return matches[0]
@unittest.skipUnless(CAPTURE.is_file(), "exact 423-second executed capture is absent")
class IdentityRelayMigrationTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.capture_bytes = CAPTURE.read_bytes()
cls.source = json.loads(cls.capture_bytes.decode("utf-8"))
cls.migrated = MIGRATION.migrate_workflow(
cls.source,
source_file_sha256=CAPTURE_SHA256,
source_history_prompt_id=MIGRATION.DEFAULT_HISTORY_PROMPT_ID,
)
def test_capture_is_the_exact_successful_history_graph(self) -> None:
self.assertEqual(_sha256(CAPTURE), CAPTURE_SHA256)
marker = self.source["extra"][MIGRATION.SOURCE_EXPANSION_SCHEMA]
self.assertEqual(marker["runtime_lane_node_ids"]["h3"], [667, 684, 692, 700])
self.assertEqual(
marker["runtime_lane_node_ids"]["tails"], [682, 690, 698]
)
def test_source_object_is_not_mutated(self) -> None:
before = copy.deepcopy(self.source)
MIGRATION.migrate_workflow(
self.source,
source_file_sha256=CAPTURE_SHA256,
source_history_prompt_id=MIGRATION.DEFAULT_HISTORY_PROMPT_ID,
)
self.assertEqual(self.source, before)
self.assertEqual(CAPTURE.read_bytes(), self.capture_bytes)
def test_dual_identity_and_shared_manifest_are_guarded(self) -> None:
nodes = self.migrated["nodes"]
planner = _one(nodes, MIGRATION.PLANNER_TYPE)
context = _one(nodes, MIGRATION.CONTEXT_TYPE)
pair = _one(nodes, MIGRATION.PAIR_PREP_TYPE)
batch = _one(nodes, "BatchImagesNode")
manifest = next(
item
for item in nodes
if item.get("type") == "PrimitiveStringMultiline"
and item.get("title", "").startswith("H3 REFERENCE MANIFEST")
)
self.assertEqual(planner["widgets_values"][-2:], [2, "Off"])
self.assertEqual(context["widgets_values"][-1], 1)
self.assertEqual(pair["widgets_values"][-2:], [1.0, 0.40])
self.assertEqual(manifest["widgets_values"], [MIGRATION.DUAL_IDENTITY_MANIFEST])
self.assertIn("do not transfer its layout, grid, seams", MIGRATION.DUAL_IDENTITY_MANIFEST)
self.assertIn("panels are not separate people", MIGRATION.DUAL_IDENTITY_MANIFEST)
context_origin, _ = MIGRATION._origin_main(
self.migrated, context, "reference_manifest"
)
planner_origin, _ = MIGRATION._origin_main(
self.migrated, planner, "reference_manifest"
)
image_origin, _ = MIGRATION._origin_main(
self.migrated, context, "reference_images"
)
self.assertEqual(context_origin["id"], manifest["id"])
self.assertEqual(planner_origin["id"], manifest["id"])
self.assertEqual(image_origin["id"], batch["id"])
def test_picture_2_uses_the_hash_guarded_generated_identity_sheet(self) -> None:
extension = self.migrated["extra"][MIGRATION.SOURCE_EXPANSION_SCHEMA][
"identity_relay_extension"
]
secondary = _node(
self.migrated["nodes"], extension["secondary_identity_loader"]
)
self.assertEqual(
secondary["widgets_values"][0],
MIGRATION.SECONDARY_IDENTITY_RELATIVE_PATH,
)
self.assertEqual(
Path(MIGRATION.SECONDARY_IDENTITY_RELATIVE_PATH).parent, Path(".")
)
self.assertNotIn("/", MIGRATION.SECONDARY_IDENTITY_RELATIVE_PATH)
self.assertNotIn("\\", MIGRATION.SECONDARY_IDENTITY_RELATIVE_PATH)
self.assertIn("generated multi-view identity sheet", secondary["title"])
self.assertEqual(
extension["secondary_identity_asset"]["sha256"],
MIGRATION.SECONDARY_IDENTITY_SHA256,
)
asset = MIGRATION.DEFAULT_COMFY_INPUT_ROOT / Path(
MIGRATION.SECONDARY_IDENTITY_RELATIVE_PATH
)
self.assertEqual(_sha256(asset), MIGRATION.SECONDARY_IDENTITY_SHA256)
nested_source = (
MIGRATION.DEFAULT_COMFY_INPUT_ROOT
/ "identity_refs"
/ MIGRATION.SECONDARY_IDENTITY_RELATIVE_PATH
)
self.assertTrue(nested_source.is_file())
self.assertEqual(_sha256(nested_source), MIGRATION.SECONDARY_IDENTITY_SHA256)
self.assertEqual(asset.read_bytes(), nested_source.read_bytes())
def test_explanatory_text_is_truthful_about_preserved_b430_settings(self) -> None:
start = _node(self.migrated["nodes"], 116)["widgets_values"][0]
outer = _node(self.migrated["nodes"], 710)
subgraph = next(
item
for item in self.migrated["definitions"]["subgraphs"]
if item["id"] == MIGRATION.FAST_SUBGRAPH_ID
)
sampler = _node(subgraph["nodes"], 657)
scheduler = _node(subgraph["nodes"], 658)
lora = _node(self.migrated["nodes"], 713)
self.assertNotIn("4-step", start)
self.assertNotIn("Euler/simple", start)
self.assertNotIn("Turbo 4-step", outer["title"])
self.assertIn("res_multistep", sampler["title"])
self.assertIn("simple / 8", scheduler["title"])
self.assertIn("0.60", lora["title"])
self.assertIn("res_multistep", subgraph["name"])
self.assertEqual(sampler["widgets_values"], ["res_multistep"])
self.assertEqual(scheduler["widgets_values"], ["simple", 8, 1])
self.assertEqual(lora["widgets_values"][1], 0.60)
extension = self.migrated["extra"][MIGRATION.SOURCE_EXPANSION_SCHEMA][
"identity_relay_extension"
]
self.assertEqual(
extension["observed_executed_fast_controls"],
{
"sampler_node": 657,
"sampler": "res_multistep",
"scheduler_node": 658,
"scheduler": "simple",
"steps": 8,
"denoise": 1.0,
"lora_node": 713,
"lora_name": lora["widgets_values"][0],
"lora_strength": 0.60,
},
)
def test_every_lane_has_two_clean_identity_inputs_and_relay_moves_to_p3(self) -> None:
extension = self.migrated["extra"][MIGRATION.SOURCE_EXPANSION_SCHEMA][
"identity_relay_extension"
]
subgraph = next(
item
for item in self.migrated["definitions"]["subgraphs"]
if item["id"] == MIGRATION.FAST_SUBGRAPH_ID
)
runtime = self.migrated["extra"][MIGRATION.SOURCE_EXPANSION_SCHEMA][
"runtime_lane_node_ids"
]
h3_nodes = [_node(subgraph["nodes"], item) for item in runtime["h3"]]
gates = [
_node(subgraph["nodes"], item) for item in extension["relay_gate_nodes"]
]
scalers = [
_node(subgraph["nodes"], item) for item in extension["relay_scale_nodes"]
]
p2_slot = next(
index
for index, item in enumerate(subgraph["inputs"])
if item["name"] == "identity_picture_2"
)
for lane, h3 in enumerate(h3_nodes, start=1):
p2_origin, p2_origin_slot, _ = MIGRATION._origin_subgraph(
subgraph, h3, "ref_images.ref_image_1"
)
self.assertEqual((p2_origin, p2_origin_slot), (-10, p2_slot))
if lane > 1:
p3_origin, _p3_slot, _ = MIGRATION._origin_subgraph(
subgraph, h3, "ref_images.ref_image_2"
)
self.assertEqual(p3_origin, gates[lane - 2]["id"])
for scaler in scalers:
self.assertEqual(scaler["type"], "ImageScaleToTotalPixels")
self.assertEqual(
scaler["widgets_values"],
[
"area",
MIGRATION.RELAY_REFERENCE_MEBIPIXELS,
MIGRATION.RELAY_REFERENCE_RESOLUTION_STEPS,
],
)
def test_423_second_fast_path_controls_are_identical(self) -> None:
source_fast = self.source["extra"]["diffusiongemma.minimax_h3_fast_test"]
migrated_fast = self.migrated["extra"]["diffusiongemma.minimax_h3_fast_test"]
self.assertEqual(migrated_fast, source_fast)
source_subgraph = next(
item
for item in self.source["definitions"]["subgraphs"]
if item["id"] == MIGRATION.FAST_SUBGRAPH_ID
)
migrated_subgraph = next(
item
for item in self.migrated["definitions"]["subgraphs"]
if item["id"] == MIGRATION.FAST_SUBGRAPH_ID
)
self.assertEqual(_node(self.migrated["nodes"], 713)["widgets_values"][1], 0.60)
self.assertEqual(
_node(migrated_subgraph["nodes"], 657)["widgets_values"],
["res_multistep"],
)
self.assertEqual(
_node(migrated_subgraph["nodes"], 658)["widgets_values"],
["simple", 8, 1.0],
)
for node_id in (
656,
657,
658,
659,
660,
663,
685,
686,
687,
688,
693,
694,
695,
696,
701,
702,
703,
704,
717,
718,
719,
720,
):
migrated_node = copy.deepcopy(_node(migrated_subgraph["nodes"], node_id))
source_node = copy.deepcopy(_node(source_subgraph["nodes"], node_id))
if node_id in {657, 658}:
migrated_node.pop("title", None)
source_node.pop("title", None)
self.assertEqual(migrated_node, source_node)
def test_validator_fails_closed_on_relay_or_fast_path_corruption(self) -> None:
broken_relay = copy.deepcopy(self.migrated)
extension = broken_relay["extra"][MIGRATION.SOURCE_EXPANSION_SCHEMA][
"identity_relay_extension"
]
subgraph = next(
item
for item in broken_relay["definitions"]["subgraphs"]
if item["id"] == MIGRATION.FAST_SUBGRAPH_ID
)
_node(subgraph["nodes"], extension["relay_scale_nodes"][0])["widgets_values"][
1
] = 1.0
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.validate_workflow(broken_relay, source=self.source)
broken_fast = copy.deepcopy(self.migrated)
_node(broken_fast["nodes"], 713)["widgets_values"][1] = 0.5
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.validate_workflow(broken_fast, source=self.source)
def test_exclusive_writer_never_overwrites_a_sibling(self) -> None:
with tempfile.TemporaryDirectory() as temporary:
output = Path(temporary) / "sibling.json"
first_hash = MIGRATION._exclusive_json_write(output, self.migrated)
before = output.read_bytes()
self.assertEqual(first_hash, hashlib.sha256(before).hexdigest())
with self.assertRaises(FileExistsError):
MIGRATION._exclusive_json_write(output, self.migrated)
self.assertEqual(output.read_bytes(), before)
@unittest.skipUnless(SIBLING.is_file(), "guarded dual-identity sibling is absent")
class WrittenSiblingTests(unittest.TestCase):
def test_written_sibling_hash_and_provenance(self) -> None:
self.assertEqual(_sha256(SIBLING), SIBLING_SHA256)
workflow = json.loads(SIBLING.read_text(encoding="utf-8"))
marker = workflow["extra"][MIGRATION.IDENTITY_RELAY_SCHEMA]
self.assertEqual(marker["source_workflow_file_sha256"], CAPTURE_SHA256)
self.assertEqual(
marker["source_history_prompt_id"], MIGRATION.DEFAULT_HISTORY_PROMPT_ID
)
MIGRATION.validate_workflow(workflow)
def test_original_saved_workflows_remain_byte_identical(self) -> None:
for path in LEGACY_SAVED_COPIES:
if path.is_file():
self.assertEqual(_sha256(path), LEGACY_SAVED_SHA256, str(path))
def test_superseded_sibling_was_not_overwritten(self) -> None:
for path, expected in zip(
SUPERSEDED_SIBLINGS, SUPERSEDED_SIBLING_SHA256
):
if path.is_file():
self.assertEqual(_sha256(path), expected, str(path))
def test_root_picture_2_is_enumerated_by_live_loadimage_inventory(self) -> None:
try:
with urllib.request.urlopen(
"http://127.0.0.1:8188/object_info/LoadImage", timeout=2.0
) as response:
contract = json.load(response)["LoadImage"]
except (OSError, KeyError, urllib.error.URLError) as exc:
self.skipTest(f"local ComfyUI LoadImage inventory unavailable: {exc}")
choices = contract["input"]["required"]["image"][0]
self.assertIn(MIGRATION.SECONDARY_IDENTITY_RELATIVE_PATH, choices)
self.assertNotIn(
f"identity_refs/{MIGRATION.SECONDARY_IDENTITY_RELATIVE_PATH}", choices
)
if __name__ == "__main__":
unittest.main()
tests/test_minimax_h3_identity_v4_smoke_artifact.py
+110
View File
@@ -0,0 +1,110 @@
from __future__ import annotations
import copy
import hashlib
import importlib.util
import json
import unittest
import uuid
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
TOOL_PATH = REPO_ROOT / "tools" / "create_minimax_h3_identity_v4_smoke.py"
SPEC = importlib.util.spec_from_file_location("minimax_h3_v4_smoke", TOOL_PATH)
if SPEC is None or SPEC.loader is None: # pragma: no cover
raise RuntimeError(f"Cannot load smoke tool: {TOOL_PATH}")
SMOKE = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(SMOKE)
EXPECTED_SMOKE_SHA256 = (
"19bd21f9e4b168aa3008fa6bf5ec1b8ce7b27e4e8c712f721333b2319f8fa746"
)
EXPECTED_SMOKE_UUID = "f71bf435-daf5-5b5f-9541-d4c7e8ce2228"
def _sha256(path: Path) -> str:
return hashlib.sha256(path.read_bytes()).hexdigest()
def _node(workflow: dict, node_id: int) -> dict:
matches = [item for item in workflow["nodes"] if int(item["id"]) == node_id]
if len(matches) != 1:
raise AssertionError(f"Expected one node {node_id}; found {len(matches)}")
return matches[0]
@unittest.skipUnless(
SMOKE.SOURCE_V4.is_file()
and SMOKE.DEFAULT_DESKTOP_OUTPUT.is_file()
and SMOKE.DEFAULT_USER_OUTPUT.is_file(),
"verified V4 smoke artifacts are absent",
)
class MiniMaxH3IdentityV4SmokeArtifactTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.source_bytes = SMOKE.SOURCE_V4.read_bytes()
cls.desktop_bytes = SMOKE.DEFAULT_DESKTOP_OUTPUT.read_bytes()
cls.user_bytes = SMOKE.DEFAULT_USER_OUTPUT.read_bytes()
cls.source = json.loads(cls.source_bytes.decode("utf-8"))
cls.smoke = json.loads(cls.desktop_bytes.decode("utf-8"))
def test_verified_v4_source_and_both_smoke_copies_have_exact_hashes(self) -> None:
self.assertEqual(_sha256(SMOKE.SOURCE_V4), SMOKE.SOURCE_V4_SHA256)
self.assertEqual(_sha256(SMOKE.DEFAULT_DESKTOP_OUTPUT), EXPECTED_SMOKE_SHA256)
self.assertEqual(_sha256(SMOKE.DEFAULT_USER_OUTPUT), EXPECTED_SMOKE_SHA256)
self.assertEqual(self.desktop_bytes, self.user_bytes)
def test_smoke_has_an_independent_workflow_uuid(self) -> None:
self.assertEqual(self.smoke["id"], EXPECTED_SMOKE_UUID)
self.assertNotEqual(self.smoke["id"], self.source["id"])
self.assertEqual(str(uuid.UUID(self.smoke["id"])), self.smoke["id"])
self.assertEqual(self.smoke["revision"], self.source["revision"] + 1)
def test_exact_four_smoke_widgets_and_relay_off(self) -> None:
self.assertEqual(_node(self.smoke, 178)["widgets_values"][0], 5.0)
target = _node(self.smoke, 673)["widgets_values"]
self.assertEqual(target[4], "1")
self.assertEqual(target[5], 15)
self.assertEqual(target[11], 0)
self.assertEqual(_node(self.smoke, 187)["widgets_values"][1], 0.2)
self.assertEqual(
_node(self.smoke, 651)["widgets_values"][0],
"minimax_h3/smoke_dual_identity_v4_5s",
)
self.assertEqual(_node(self.smoke, 677)["widgets_values"][-1], "Off")
def test_topology_and_all_non_smoke_node_fields_match_v4(self) -> None:
SMOKE.validate_smoke_workflow(self.smoke, source=self.source)
self.assertEqual(self.smoke["links"], self.source["links"])
self.assertEqual(self.smoke["definitions"], self.source["definitions"])
self.assertEqual(len(self.smoke["nodes"]), len(self.source["nodes"]))
def test_root_picture_2_and_provenance_remain_guarded(self) -> None:
self.assertEqual(
_node(self.smoke, 721)["widgets_values"][0],
SMOKE.BASE.SECONDARY_IDENTITY_RELATIVE_PATH,
)
self.assertEqual(
Path(SMOKE.BASE.SECONDARY_IDENTITY_RELATIVE_PATH).parent, Path(".")
)
asset = (
SMOKE.BASE.DEFAULT_COMFY_INPUT_ROOT
/ SMOKE.BASE.SECONDARY_IDENTITY_RELATIVE_PATH
)
self.assertEqual(_sha256(asset), SMOKE.BASE.SECONDARY_IDENTITY_SHA256)
marker = self.smoke["extra"][SMOKE.SMOKE_SCHEMA]
self.assertEqual(marker["source_workflow_file_sha256"], SMOKE.SOURCE_V4_SHA256)
self.assertEqual(marker["smoke_workflow_id"], EXPECTED_SMOKE_UUID)
self.assertTrue(marker["source_is_never_overwritten"])
self.assertTrue(marker["generation_was_not_queued_by_creator"])
def test_validator_fails_closed_on_a_fifth_execution_edit(self) -> None:
broken = copy.deepcopy(self.smoke)
_node(broken, 713)["widgets_values"][1] = 0.5
with self.assertRaises(SMOKE.BASE.WorkflowError):
SMOKE.validate_smoke_workflow(broken, source=self.source)
if __name__ == "__main__":
unittest.main()
tests/test_minimax_h3_ref2va_deterministic_contract_repair.py
+890
View File
@@ -0,0 +1,890 @@
from __future__ import annotations
import copy
import importlib.util
import json
import sys
import unittest
import uuid
from pathlib import Path
from unittest.mock import patch
ROOT = Path(__file__).resolve().parents[1]
COMFY_ROOT = ROOT.parents[1]
def load_nodes_module():
if str(COMFY_ROOT) not in sys.path:
sys.path.insert(0, str(COMFY_ROOT))
package_name = f"diffusiongemma_h3_deterministic_repair_{uuid.uuid4().hex}"
spec = importlib.util.spec_from_file_location(
package_name,
ROOT / "__init__.py",
submodule_search_locations=[str(ROOT)],
)
if spec is None or spec.loader is None:
raise RuntimeError("could not load DiffusionGemma Prompt Builder")
package = importlib.util.module_from_spec(spec)
sys.modules[package_name] = package
spec.loader.exec_module(package)
return sys.modules[f"{package_name}.nodes"]
class MiniMaxH3Ref2VADeterministicContractRepairTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.nodes = load_nodes_module()
@staticmethod
def user_prompt() -> str:
return (
"Create a rap music video; keep the black-and-white high-fashion "
"photography aesthetic."
)
@staticmethod
def manifest() -> str:
return (
"<Picture 1>: sole visual reference for visible subject, wardrobe, scene, palette, and composition; "
"preserve source-grounded identity and appearance across every shot.\n"
"<Audio 1>: locked selected soundtrack excerpt; copy its exact timing, vocal phrasing, beat, dynamics, "
"and dance synchronization without replacing or reinterpreting the composition."
)
@staticmethod
def valid_prompt() -> str:
return (
"subject_definitions:\n"
"<Subject 1>: A high-fashion female model with short dark hair, wearing a black sleeveless bodysuit with "
"thin straps and sheer tights, featuring a confident pose as seen in <Picture 1>.\n"
"<Audio 1>: A high-energy rap soundtrack with rhythmic vocal phrasing and heavy beats, referenced as "
"<Audio 1>.\n\n"
"summary:\n"
"[reference generation + audio reference] <Subject 1> performs a rhythmic rap performance in a high-contrast "
"black-and-white environment synchronized to the beats of <Audio 1>.\n\n"
"retention_analysis:\n"
"<Subject 1>: fully_preserved - The model's face, hair, bodysuit, and high-fashion aesthetic are maintained "
"across all shots.\n"
"<Audio 1>: fully_copy - The exact locked soundtrack timing and vocal dynamics dictate the performance "
"movements and camera cuts.\n\n"
"detailed_description:\n"
"The video is rendered in a high-contrast black-and-white fashion photography aesthetic, characterized by "
"deep shadows, blown highlights, and sharp film grain. The setting is a minimalist space with a tiled wall.\n\n"
"[Shot 1] <Subject 1> stands in a wide-stanced power pose derived from <Picture 1>, one hand raised to her "
"head and the other on her hip. The camera is a static medium shot, capturing her silhouette against the "
"white tiles as she begins to move rhythmically to the opening beat of <Audio 1>. Her shoulders settle on "
"the downbeat while the tiled geometry and hard side light remain fixed behind her.\n\n"
"[Shot 2] At 00:04.500, the camera cuts to a static close-up on <Subject 1>'s face. She leans toward the lens "
"and marks the rap cadence with a controlled head turn. The harsh lighting creates dramatic shadows across "
"her cheekbones and neck while the same hairstyle, wardrobe, and facial proportions remain clear.\n\n"
"[Shot 3] At 00:09.000, the camera cuts to a low-angle tracking shot. <Subject 1> performs dynamic dance "
"movements, shifting her weight and gesturing with her hands to the double-time pulse of <Audio 1>. The "
"camera tracks upward slightly, emphasizing her height and the geometry of the bodysuit without changing "
"the monochrome tiled setting or her screen direction.\n\n"
"[Shot 4] At 00:12.500, the camera cuts to a locked medium-wide shot. <Subject 1> resumes her original pose, "
"looking directly into the lens with an intense gaze as the music reaches its final accent. Her feet stop, "
"her hands settle, and the camera holds the resolved symmetrical composition through the final frame.\n\n"
"overall_soundscape:\n"
"Faint floor vibration, measured foot contacts, and clothing rustle stay synchronized with the visible "
"movement without adding speech.\n\n"
"non_diegetic_music:\n"
"The exact locked fast-paced rap track from <Audio 1>, with its driving bassline, crisp percussion, and "
"sharp vocal phrasing preserved unchanged."
)
def reasons(self, prompt: str) -> list[str]:
return self.nodes._minimax_h3_ref2va_validation_reasons(
prompt,
15.0,
self.user_prompt(),
"auto_scene_audio",
20_000,
self.manifest(),
"4",
"off",
2,
"",
0,
)
def exact_trio_candidate(self) -> str:
sections = self.nodes._minimax_h3_ref_sections(self.valid_prompt())
self.assertIsNotNone(sections)
sections = dict(sections)
sections["retention_analysis"] = sections["retention_analysis"].replace(
"<Audio 1>: fully_copy",
"<Audio 1>: fully_preserved",
1,
)
sections["detailed_description"] = sections["detailed_description"].replace(
"<Subject 1>",
"the performer",
).replace(
"[Shot 3] At 00:09.000,",
"[Shot 3]",
1,
)
return self.nodes._rebuild_minimax_h3_ref_prompt(sections)
def test_exact_three_reason_candidate_repairs_before_model_retry(self) -> None:
candidate = self.exact_trio_candidate()
self.assertEqual(
self.reasons(candidate),
[
"minimax_h3_ref_subject_usage_invalid",
"minimax_h3_ref_retention_invalid",
"minimax_h3_cut_timestamp_invalid",
],
)
repaired = self.nodes._repair_minimax_h3_ref2va_structure(
candidate,
self.user_prompt(),
self.manifest(),
15.0,
)
self.assertEqual(self.reasons(repaired), [])
self.assertIn("<Audio 1>: fully_copy - The exact locked soundtrack", repaired)
self.assertIn("[Shot 2] At 00:04.500,", repaired)
self.assertIn("[Shot 3] At 00:08.500,", repaired)
self.assertIn("[Shot 4] At 00:12.500,", repaired)
self.assertIn("<Subject 1> (derived from <Picture 1>)", repaired)
self.assertEqual(repaired.count("[Shot "), 4)
self.assertIn("performs dynamic dance movements", repaired)
self.assertEqual(
self.nodes._repair_minimax_h3_ref2va_structure(
repaired,
self.user_prompt(),
self.manifest(),
15.0,
),
repaired,
)
packet = {
"ltx_prompt": "",
"ideogram_prompt": "",
"minimax_h3_prompt": candidate,
"negative_prompt": "",
"scene_segments": [],
"metadata": {},
}
context = self.nodes.GemmaContext(
user_prompt=self.user_prompt(),
images=None,
source="minimax_h3_ref2va",
media_metadata={
"source": "minimax_h3_ref2va",
"duration_seconds": 15.0,
"reference_image_count": 1,
"minimax_h3_reference_image_batch_count": 1,
"minimax_h3_reference_manifest": self.manifest(),
"minimax_h3_expected_subject_count": 0,
},
)
target = self.nodes.TargetProfileConfig(
target_profile="minimax_h3",
audio_mode="auto_scene_audio",
target_duration_seconds=15.0,
minimax_h3_mode="ref2va",
minimax_h3_shot_count="4",
minimax_h3_dialogue_mode="off",
)
runtime = self.nodes.RuntimeConfig(
model_path="unused-local-checkpoint",
backend="transformers_inprocess",
dtype="auto",
quantization="none",
local_files_only=True,
unload_policy="unload_after_run",
max_memory_gb=20.0,
status={"ready": True, "supports_pixels": False},
)
with patch.object(
self.nodes,
"_run_backend_for_packet",
return_value=json.dumps(packet),
) as backend:
result = self.nodes._run_generation_packet_legacy(
runtime,
context,
target,
runtime_required=True,
max_output_chars=20_000,
manage_unload=False,
max_refinement_attempts_override=2,
)
self.assertEqual(backend.call_count, 1)
self.assertTrue(result[-1]["ready_for_generation"])
self.assertEqual(self.reasons(result[0]["minimax_h3_prompt"]), [])
def test_missing_detailed_heading_preserves_all_fifteen_authored_shots(self) -> None:
style_opening = (
"The target video uses an immutable black-and-white high-fashion photography aesthetic with hard contrast, "
"controlled film grain, and precise editorial composition."
)
shot_blocks: list[str] = []
for shot in range(1, 16):
if shot == 1:
opening = "[Shot 1]"
else:
milliseconds = (shot - 1) * 1500
minutes, remainder = divmod(milliseconds, 60_000)
seconds, millis = divmod(remainder, 1000)
opening = (
f"[Shot {shot}] At {minutes:02d}:{seconds:02d}.{millis:03d}, "
"the camera cuts to a"
)
ending = (
"The final frame holds on the resolved centered pose while the camera remains completely static."
if shot == 15
else "The camera remains completely static while the pose resolves before the next measured cut."
)
shot_blocks.append(
f"{opening} high-contrast fashion portrait of <Subject 1> derived from <Picture 1>, marking the exact "
f"rap cadence of <Audio 1> with a distinct controlled gesture for beat {shot}. {ending}"
)
misplaced_timeline = "\n\n".join((style_opening, *shot_blocks))
candidate = (
"subject_definitions:\n"
"<Picture 1>: The sole visual reference for the visible performer, wardrobe, monochrome set, lighting, and composition.\n"
"<Audio 1>: The exact locked rap soundtrack supplies beat timing, vocal phrasing, dynamics, and performance pacing.\n\n"
"summary:\n"
"[reference generation + audio reference] <Subject 1> performs a black-and-white fashion rap video derived "
"from <Picture 1> and synchronized to <Audio 1>.\n\n"
"retention_analysis:\n"
"<Picture 1>: fully_preserved - Preserve the performer, wardrobe, monochrome set, lighting, and composition.\n"
"<Audio 1>: fully_copy - Preserve the exact song, timing, vocal phrasing, dynamics, and cadence.\n"
f"{misplaced_timeline}\n\n"
"overall_soundscape:\n"
"Measured foot contacts and clothing movement remain subordinate to the exact locked song.\n\n"
"non_diegetic_music:\n"
"Use the exact supplied <Audio 1> composition without replacement or regeneration."
)
initial_reasons = self.nodes._minimax_h3_ref2va_validation_reasons(
candidate,
25.0,
self.user_prompt(),
"auto_scene_audio",
20_000,
self.manifest(),
"15",
"off",
2,
"",
0,
)
self.assertIn("minimax_h3_ref_missing_detailed_description", initial_reasons)
self.assertEqual(candidate.count("[Shot "), 15)
repaired = self.nodes._repair_minimax_h3_ref2va_structure(
candidate,
self.user_prompt(),
self.manifest(),
25.0,
)
repaired_sections = self.nodes._minimax_h3_ref_sections(repaired)
self.assertIsNotNone(repaired_sections)
self.assertEqual(repaired_sections["detailed_description"].count("[Shot "), 15)
self.assertIn("[Shot 15] At 00:21.000", repaired_sections["detailed_description"])
self.assertIn("<Subject 1>:", repaired_sections["subject_definitions"])
self.assertIn("Reference source: <Picture 1>.", repaired_sections["subject_definitions"])
self.assertIn("<Subject 1>: fully_preserved -", repaired_sections["retention_analysis"])
self.assertEqual(
self.nodes._minimax_h3_ref2va_validation_reasons(
repaired,
25.0,
self.user_prompt(),
"auto_scene_audio",
20_000,
self.manifest(),
"15",
"off",
2,
"",
0,
),
[],
)
self.assertEqual(
self.nodes._repair_minimax_h3_ref2va_structure(
repaired,
self.user_prompt(),
self.manifest(),
25.0,
),
repaired,
)
def test_canonical_picture_only_performer_definition_is_promoted_safely(self) -> None:
sections = self.nodes._minimax_h3_ref_sections(self.valid_prompt())
self.assertIsNotNone(sections)
sections = dict(sections)
sections["subject_definitions"] = sections["subject_definitions"].replace(
"<Subject 1>: A high-fashion female model",
"<Picture 1>: A high-fashion female model",
1,
)
sections["retention_analysis"] = sections["retention_analysis"].replace(
"<Subject 1>: fully_preserved",
"<Picture 1>: fully_preserved",
1,
)
candidate = self.nodes._rebuild_minimax_h3_ref_prompt(sections)
self.assertIn("minimax_h3_ref_subject_definition_invalid", self.reasons(candidate))
repaired = self.nodes._repair_minimax_h3_ref2va_structure(
candidate,
self.user_prompt(),
self.manifest(),
15.0,
)
self.assertEqual(self.reasons(repaired), [])
self.assertIn("<Subject 1>: A high-fashion female model", repaired)
self.assertIn("Reference source: <Picture 1>.", repaired)
self.assertIn("<Subject 1>: fully_preserved -", repaired)
def test_environment_picture_is_never_promoted_to_performer_subject(self) -> None:
plate_descriptions = (
"An empty monochrome dressing room with wardrobe racks, hard side lighting, and no visible inhabitants.",
"An empty tiled room with no person, human, performer, or model visible anywhere in the frame.",
"A fashion style board showing wardrobe palette, fabric swatches, lighting references, and set textures.",
)
for plate_description in plate_descriptions:
with self.subTest(plate_description=plate_description):
sections = self.nodes._minimax_h3_ref_sections(self.valid_prompt())
self.assertIsNotNone(sections)
sections = dict(sections)
sections["subject_definitions"] = (
f"<Picture 1>: {plate_description}\n"
"<Audio 1>: A high-energy rap soundtrack with rhythmic vocal phrasing and heavy beats."
)
sections["retention_analysis"] = (
"<Picture 1>: attribute_transfer - Preserve the authored plate attributes without inventing a performer.\n"
"<Audio 1>: fully_copy - Preserve the exact locked soundtrack."
)
candidate = self.nodes._rebuild_minimax_h3_ref_prompt(sections)
repaired = self.nodes._repair_minimax_h3_ref2va_structure(
candidate,
self.user_prompt(),
self.manifest(),
15.0,
)
repaired_sections = self.nodes._minimax_h3_ref_sections(repaired)
self.assertIsNotNone(repaired_sections)
self.assertNotIn("<Subject 1>:", repaired_sections["subject_definitions"])
self.assertIn("minimax_h3_ref_subject_definition_invalid", self.reasons(repaired))
def test_salvaged_picture_timeline_receives_promoted_subject_label(self) -> None:
sections = self.nodes._minimax_h3_ref_sections(self.valid_prompt())
self.assertIsNotNone(sections)
sections = dict(sections)
sections["subject_definitions"] = sections["subject_definitions"].replace(
"<Subject 1>: A high-fashion female model",
"<Picture 1>: A high-fashion female model",
1,
)
sections["retention_analysis"] = sections["retention_analysis"].replace(
"<Subject 1>: fully_preserved",
"<Picture 1>: fully_preserved",
1,
)
sections["detailed_description"] = sections["detailed_description"].replace(
"<Subject 1>",
"<Picture 1>",
)
candidate = (
f"subject_definitions:\n{sections['subject_definitions']}\n\n"
f"summary:\n{sections['summary']}\n\n"
f"retention_analysis:\n{sections['retention_analysis']}\n"
f"{sections['detailed_description']}\n\n"
f"overall_soundscape:\n{sections['overall_soundscape']}\n\n"
f"non_diegetic_music:\n{sections['non_diegetic_music']}"
)
repaired = self.nodes._repair_minimax_h3_ref2va_structure(
candidate,
self.user_prompt(),
self.manifest(),
15.0,
)
repaired_sections = self.nodes._minimax_h3_ref_sections(repaired)
self.assertIsNotNone(repaired_sections)
timeline = repaired_sections["detailed_description"]
self.assertIn("<Subject 1> (derived from <Picture 1>)", timeline)
self.assertNotIn("minimax_h3_ref_subject_usage_invalid", self.reasons(repaired))
def test_salvage_keeps_locked_audio_as_fully_copy(self) -> None:
sections = self.nodes._minimax_h3_ref_sections(self.valid_prompt())
self.assertIsNotNone(sections)
sections = dict(sections)
retention = sections["retention_analysis"].replace(
"<Audio 1>: fully_copy",
"<Audio 1>: fully_preserved",
1,
)
candidate = (
f"subject_definitions:\n{sections['subject_definitions']}\n\n"
f"summary:\n{sections['summary']}\n\n"
f"retention_analysis:\n{retention}\n"
f"{sections['detailed_description']}\n\n"
f"overall_soundscape:\n{sections['overall_soundscape']}\n\n"
f"non_diegetic_music:\n{sections['non_diegetic_music']}"
)
repaired = self.nodes._repair_minimax_h3_ref2va_structure(
candidate,
self.user_prompt(),
self.manifest(),
15.0,
)
repaired_sections = self.nodes._minimax_h3_ref_sections(repaired)
self.assertIsNotNone(repaired_sections)
self.assertIn("<Audio 1>: fully_copy -", repaired_sections["retention_analysis"])
self.assertNotIn("<Audio 1>: reference -", repaired_sections["retention_analysis"])
self.assertEqual(self.reasons(repaired), [])
def test_refinement_prompt_explicitly_rebuilds_the_four_failed_contract_parts(self) -> None:
prompt = self.nodes._build_minimax_h3_refinement_prompt(
self.user_prompt(),
"candidate",
25.0,
"auto_scene_audio",
[
"minimax_h3_ref_missing_detailed_description",
"minimax_h3_ref_subject_definition_invalid",
"minimax_h3_ref_style_opening_missing",
"minimax_h3_ref_detailed_description_short",
],
"ref2va",
self.manifest(),
strict_retry=True,
minimax_h3_shot_count="15",
minimax_h3_dialogue_mode="off",
)
self.assertIn("Rebuild detailed_description", prompt)
self.assertIn("exactly 15 consecutive [Shot N] blocks", prompt)
self.assertIn("at least 120 words", prompt)
self.assertIn("about 20 to 35 words", prompt)
self.assertIn("Rebuild subject_definitions", prompt)
self.assertIn("at least four descriptive words", prompt)
self.assertIn("replace every blank or invalid listed section", prompt)
def test_short_one_shot_detail_budget_scales_without_weakening_dense_work(self) -> None:
self.assertEqual(
self.nodes._minimax_h3_ref_detail_word_budget(5.0, 1),
(60, 60, 100),
)
self.assertEqual(
self.nodes._minimax_h3_ref_detail_word_budget(15.0, 1),
(90, 90, 130),
)
self.assertEqual(
self.nodes._minimax_h3_ref_detail_word_budget(25.0, 15),
(120, 320, 500),
)
for shot_count in (25, 99):
with self.subTest(shot_count=shot_count):
budget = self.nodes._minimax_h3_ref_detail_word_budget(
25.0,
shot_count,
)
self.assertEqual(budget, (120, 500, 500))
self.assertLessEqual(budget[1], budget[2])
refinement = self.nodes._build_minimax_h3_refinement_prompt(
self.user_prompt(),
"candidate with [Shot 1] and a static medium shot",
5.0,
"auto_scene_audio",
["minimax_h3_ref_detailed_description_short"],
"ref2va",
self.manifest(),
minimax_h3_shot_count="1",
minimax_h3_dialogue_mode="off",
)
self.assertIn("at least 60 words", refinement)
self.assertIn("target about 60 to 100 words", refinement)
self.assertNotIn("at least 120 words", refinement)
self.assertIn("do not pad it with extra cuts", refinement)
model_prompt = self.nodes._build_model_prompt(
self.user_prompt(),
self.nodes.DEFAULT_MASTER_PROMPT,
"minimax_h3",
{
"source": "minimax_h3_ref2va",
"minimax_h3_mode": "ref2va",
"minimax_h3_reference_manifest": self.manifest(),
},
target_duration_seconds=5.0,
minimax_h3_mode="ref2va",
minimax_h3_reference_manifest=self.manifest(),
minimax_h3_shot_count="1",
native_h3_output=True,
minimax_h3_dialogue_mode="off",
)
self.assertIn("about 60 to 100 words", model_prompt)
self.assertIn("never fewer than 60 words", model_prompt)
def test_initial_prompt_detail_budget_uses_resolved_duration_with_zero_override(self) -> None:
cases = (
(
self.user_prompt(),
{
"source": "minimax_h3_ref2va",
"minimax_h3_mode": "ref2va",
"minimax_h3_reference_manifest": self.manifest(),
"duration_seconds": 25.0,
},
"media duration",
),
(
f"{self.user_prompt()} Create a 25-second video.",
{
"source": "minimax_h3_ref2va",
"minimax_h3_mode": "ref2va",
"minimax_h3_reference_manifest": self.manifest(),
},
"user-brief duration",
),
)
for user_prompt, media_metadata, source in cases:
with self.subTest(source=source):
model_prompt = self.nodes._build_model_prompt(
user_prompt,
self.nodes.DEFAULT_MASTER_PROMPT,
"minimax_h3",
media_metadata,
target_duration_seconds=0.0,
minimax_h3_mode="ref2va",
minimax_h3_reference_manifest=self.manifest(),
minimax_h3_shot_count="1",
native_h3_output=True,
minimax_h3_dialogue_mode="off",
)
self.assertIn("planned for exactly 25.000 seconds", model_prompt)
self.assertIn("about 120 to 160 words", model_prompt)
self.assertIn("never fewer than 120 words", model_prompt)
self.assertNotIn("about 60 to 100 words", model_prompt)
def test_short_one_shot_prompt_no_longer_hits_dense_timeline_floor(self) -> None:
sections = self.nodes._minimax_h3_ref_sections(self.valid_prompt())
self.assertIsNotNone(sections)
sections = dict(sections)
sections["detailed_description"] = (
"The video remains a high-contrast black-and-white high-fashion photography study with hard side light, deep shadows, "
"blown highlights, and fine monochrome grain. "
"[Shot 1] <Subject 1> holds the source-derived stance from <Picture 1> in a static medium shot, then marks the exact "
"rap cadence of <Audio 1> with one controlled shoulder drop and a small head turn. Her short dark hair, feminine facial "
"structure, black bodysuit, screen position, and tiled background remain stable as the movement settles into a resolved "
"direct gaze on the final beat."
)
one_shot_prompt = self.nodes._rebuild_minimax_h3_ref_prompt(sections)
short_reasons = self.nodes._minimax_h3_ref2va_validation_reasons(
one_shot_prompt,
5.0,
self.user_prompt(),
"auto_scene_audio",
20_000,
self.manifest(),
"1",
"off",
2,
"",
0,
)
self.assertNotIn("minimax_h3_ref_detailed_description_short", short_reasons)
long_reasons = self.nodes._minimax_h3_ref2va_validation_reasons(
one_shot_prompt,
25.0,
self.user_prompt(),
"auto_scene_audio",
20_000,
self.manifest(),
"1",
"off",
2,
"",
0,
)
self.assertIn("minimax_h3_ref_detailed_description_short", long_reasons)
def test_best_parse_valid_rejected_refinement_replaces_stale_initial_candidate(self) -> None:
initial_prompt = "initial malformed candidate"
improved_prompt = "improved candidate with one remaining camera issue"
worse_prompt = "second candidate with two remaining issues"
def packet(prompt: str) -> str:
return json.dumps(
{
"ltx_prompt": "",
"ideogram_prompt": "",
"minimax_h3_prompt": prompt,
"negative_prompt": "",
"scene_segments": [],
"metadata": {},
}
)
initial_reasons = [
"minimax_h3_ref_missing_detailed_description",
"minimax_h3_ref_subject_definition_invalid",
"minimax_h3_ref_style_opening_missing",
"minimax_h3_ref_detailed_description_short",
]
def validation_reasons(candidate: str, *_args, **_kwargs) -> list[str]:
if candidate == improved_prompt:
return ["minimax_h3_shot_camera_unspecified"]
if candidate == worse_prompt:
return [
"minimax_h3_shot_camera_unspecified",
"minimax_h3_cut_transition_invalid",
]
return list(initial_reasons)
context = self.nodes.GemmaContext(
user_prompt=self.user_prompt(),
images=None,
source="minimax_h3_ref2va",
media_metadata={
"source": "minimax_h3_ref2va",
"duration_seconds": 25.0,
"reference_image_count": 1,
"minimax_h3_reference_image_batch_count": 1,
"minimax_h3_reference_manifest": self.manifest(),
"minimax_h3_expected_subject_count": 0,
},
)
target = self.nodes.TargetProfileConfig(
target_profile="minimax_h3",
audio_mode="auto_scene_audio",
target_duration_seconds=25.0,
minimax_h3_mode="ref2va",
minimax_h3_shot_count="15",
minimax_h3_dialogue_mode="off",
)
runtime = self.nodes.RuntimeConfig(
model_path="unused-local-checkpoint",
backend="transformers_inprocess",
dtype="auto",
quantization="none",
local_files_only=True,
unload_policy="unload_after_run",
max_memory_gb=20.0,
status={"ready": True, "supports_pixels": False},
)
with patch.object(
self.nodes,
"_run_backend_for_packet",
side_effect=[packet(initial_prompt), packet(improved_prompt), packet(worse_prompt)],
), patch.object(
self.nodes,
"_minimax_h3_prompt_validation_reasons",
side_effect=validation_reasons,
):
result = self.nodes._run_generation_packet_legacy(
runtime,
context,
target,
runtime_required=True,
max_output_chars=20_000,
manage_unload=False,
max_refinement_attempts_override=2,
)
self.assertEqual(result[0]["minimax_h3_prompt"], improved_prompt)
metadata = result[-1]
self.assertFalse(metadata["ready_for_generation"])
self.assertEqual(metadata["blocked_reasons"], ["minimax_h3_shot_camera_unspecified"])
refinement = metadata["minimax_h3_refinement"]
self.assertFalse(refinement["accepted"])
self.assertTrue(refinement["applied_best_rejected_candidate"])
self.assertEqual(refinement["applied_best_rejected_candidate_attempt"], 1)
self.assertEqual(
refinement["applied_best_rejected_candidate_reasons"],
["minimax_h3_shot_camera_unspecified"],
)
def test_valid_contract_is_byte_stable(self) -> None:
prompt = self.valid_prompt()
self.assertEqual(self.reasons(prompt), [])
self.assertEqual(
self.nodes._repair_minimax_h3_ref2va_structure(
prompt,
self.user_prompt(),
self.manifest(),
15.0,
),
prompt,
)
def test_missing_manifest_assets_are_restored_without_model_retry(self) -> None:
sections = self.nodes._minimax_h3_ref_sections(self.valid_prompt())
self.assertIsNotNone(sections)
sections = dict(sections)
original_subject = sections["subject_definitions"].split("\n<Audio 1>:", 1)[0]
sections["subject_definitions"] = original_subject.replace(
" as seen in <Picture 1>",
"",
1,
)
candidate = self.nodes._rebuild_minimax_h3_ref_prompt(sections)
self.assertIn("minimax_h3_ref_missing_asset_tag", self.reasons(candidate))
repaired = self.nodes._repair_minimax_h3_ref2va_structure(
candidate,
self.user_prompt(),
self.manifest(),
15.0,
)
repaired_sections = self.nodes._minimax_h3_ref_sections(repaired)
self.assertIsNotNone(repaired_sections)
self.assertTrue(
repaired_sections["subject_definitions"].startswith(
sections["subject_definitions"]
)
)
self.assertIn(
"<Picture 1>: sole visual reference for visible subject",
repaired_sections["subject_definitions"],
)
self.assertIn(
"<Audio 1>: locked selected soundtrack excerpt",
repaired_sections["subject_definitions"],
)
self.assertIn("<Picture 1>: attribute_transfer -", repaired)
self.assertIn("<Audio 1>: fully_copy -", repaired)
self.assertEqual(self.reasons(repaired), [])
self.assertEqual(
self.nodes._repair_minimax_h3_ref2va_structure(
repaired,
self.user_prompt(),
self.manifest(),
15.0,
),
repaired,
)
def test_manifest_completion_does_not_legalize_unknown_prompt_assets(self) -> None:
candidate = self.valid_prompt().replace(
"against the white tiles",
"beside the undeclared prop from <Picture 2> against the white tiles",
1,
)
repaired = self.nodes._repair_minimax_h3_ref2va_structure(
candidate,
self.user_prompt(),
self.manifest(),
15.0,
)
self.assertIn("minimax_h3_ref_undefined_tag", self.reasons(repaired))
def test_missing_locked_audio_retention_defaults_to_fully_copy(self) -> None:
sections = self.nodes._minimax_h3_ref_sections(self.valid_prompt())
self.assertIsNotNone(sections)
sections = dict(sections)
sections["retention_analysis"] = sections["retention_analysis"].split(
"\n<Audio 1>:",
1,
)[0]
candidate = self.nodes._rebuild_minimax_h3_ref_prompt(sections)
self.assertIn("minimax_h3_ref_retention_invalid", self.reasons(candidate))
repaired = self.nodes._repair_minimax_h3_ref2va_structure(
candidate,
self.user_prompt(),
self.manifest(),
15.0,
)
repaired_sections = self.nodes._minimax_h3_ref_sections(repaired)
self.assertIsNotNone(repaired_sections)
self.assertIn(
"<Audio 1>: fully_copy - A high-energy rap soundtrack",
repaired_sections["retention_analysis"],
)
self.assertNotIn("minimax_h3_ref_retention_invalid", self.reasons(repaired))
def test_nonmonotonic_cut_repairs_only_conflicting_timestamp(self) -> None:
candidate = self.valid_prompt().replace(
"[Shot 3] At 00:09.000,",
"[Shot 3] At 00:04.500,",
1,
)
self.assertIn("minimax_h3_cut_timestamp_invalid", self.reasons(candidate))
repaired = self.nodes._repair_minimax_h3_ref2va_structure(
candidate,
self.user_prompt(),
self.manifest(),
15.0,
)
self.assertIn("[Shot 2] At 00:04.500,", repaired)
self.assertIn("[Shot 3] At 00:08.500,", repaired)
self.assertIn("[Shot 4] At 00:12.500,", repaired)
self.assertEqual(repaired.count("performs dynamic dance movements"), 1)
self.assertEqual(self.reasons(repaired), [])
def test_out_of_range_authored_cut_remains_fail_closed(self) -> None:
candidate = self.valid_prompt().replace(
"[Shot 3] At 00:09.000,",
"[Shot 3] At 00:15.000,",
1,
)
self.assertIn("minimax_h3_cut_timestamp_out_of_range", self.reasons(candidate))
repaired = self.nodes._repair_minimax_h3_ref2va_structure(
candidate,
self.user_prompt(),
self.manifest(),
15.0,
)
self.assertEqual(repaired, candidate)
self.assertIn("minimax_h3_cut_timestamp_out_of_range", self.reasons(repaired))
def test_ambiguous_multi_subject_usage_remains_fail_closed(self) -> None:
sections = self.nodes._minimax_h3_ref_sections(self.valid_prompt())
self.assertIsNotNone(sections)
sections = copy.deepcopy(sections)
sections["subject_definitions"] += (
"\n<Subject 2>: A newly designed mirrored mannequin that remains separate from the referenced performer."
)
sections["retention_analysis"] += (
"\n<Subject 2>: fully_preserved - Its mirrored surface and mannequin proportions remain stable."
)
sections["detailed_description"] = sections["detailed_description"].replace(
"<Subject 1>",
"the performer",
)
candidate = self.nodes._rebuild_minimax_h3_ref_prompt(sections)
repaired = self.nodes._repair_minimax_h3_ref2va_structure(
candidate,
self.user_prompt(),
self.manifest(),
15.0,
)
repaired_sections = self.nodes._minimax_h3_ref_sections(repaired)
self.assertIsNotNone(repaired_sections)
self.assertNotIn("<Subject 1>", repaired_sections["detailed_description"])
self.assertNotIn("<Subject 2>", repaired_sections["detailed_description"])
self.assertIn("minimax_h3_ref_subject_usage_invalid", self.reasons(repaired))
if __name__ == "__main__":
unittest.main()
tests/test_minimax_h3_shot_count.py
+99
View File
@@ -89,6 +89,8 @@ class MiniMaxH3ShotCountTests(unittest.TestCase):
"minimax_h3_dialogue_line_count",
"minimax_h3_dialogue_guidance",
"ltx_generation_mode",
"ltx_long_horizon_mode",
"ltx_camera_capability",
],
)
choices, config = optional["minimax_h3_shot_count"]
@@ -289,6 +291,103 @@ class MiniMaxH3ShotCountTests(unittest.TestCase):
self.assertIn("Shot-count selector: 5; effective required count: 5; source: target_profile", refinement)
self.assertNotIn("Use exactly 0 consecutively numbered shots", refinement)
def test_initial_and_refinement_contracts_allow_expressive_h3_camera_paths(self) -> None:
brief = "Create four measured shots of a performer moving to the music."
exact_fallback = "The camera remains locked-off in a stable view."
for mode, candidate in (
("t2va", self.t2va_prompt()),
("ref2va", self.ref2va_prompt()),
):
with self.subTest(mode=mode, stage="initial"):
model_prompt = self.nodes._build_model_prompt(
brief,
self.nodes.DEFAULT_MASTER_PROMPT,
"minimax_h3",
{"source": "none"},
target_duration_seconds=20.0,
minimax_h3_mode=mode,
minimax_h3_shot_count="4",
)
self.assertIn(exact_fallback, model_prompt)
self.assertIn("immediately after its [Shot N] marker", model_prompt)
self.assertIn("An incoming cut, shot size, angle, subject framing", model_prompt)
self.assertIn("never omit it to save words in a dense multi-shot timeline", model_prompt)
self.assertIn("MiniMax H3 is not governed by the LTX Stable / base-model Camera Capability", model_prompt)
self.assertIn("orbit or arc, roll or rotation, spin or swirl", model_prompt)
self.assertIn("sweeping or whip movement", model_prompt)
self.assertIn("pronounced or layered parallax", model_prompt)
self.assertIn("coherent compound/multi-axis path", model_prompt)
self.assertIn("proportional to creative strength", model_prompt)
self.assertNotIn("Never invent an orbit, arc, roll, rotation, spin, swirl", model_prompt)
self.assertNotIn('"ltx_camera_capability"', model_prompt)
self.assertNotIn('"ltx_complexity_limits"', model_prompt)
native_model_prompt = self.nodes._build_model_prompt(
brief,
self.nodes.DEFAULT_MASTER_PROMPT,
"minimax_h3",
{"source": "none"},
target_duration_seconds=20.0,
minimax_h3_mode=mode,
minimax_h3_shot_count="4",
native_h3_output=True,
)
self.assertNotIn('"ltx_camera_capability"', native_model_prompt)
self.assertNotIn('"ltx_complexity_limits"', native_model_prompt)
self.assertIn("coherent compound/multi-axis path", native_model_prompt)
with self.subTest(mode=mode, stage="refinement"):
refinement = self.nodes._build_minimax_h3_refinement_prompt(
brief,
candidate,
20.0,
"auto_scene_audio",
["minimax_h3_shot_camera_unspecified"],
minimax_h3_mode=mode,
minimax_h3_shot_count="4",
)
self.assertIn(exact_fallback, refinement)
self.assertIn("Repair every action-only or framing-only shot", refinement)
self.assertIn("Orbiting, swirling, sweeping or whip movement", refinement)
self.assertIn("pronounced parallax", refinement)
self.assertIn("coherent compound paths are valid H3 choices", refinement)
self.assertNotIn("do not rewrite a safe hold as a more ambitious move", refinement)
def test_h3_camera_contract_examples_are_validator_recognizable(self) -> None:
camera_sentences = (
"The camera remains locked-off in a stable view.",
"The camera slowly pushes in, then settles.",
"The camera slowly pulls back, then settles.",
"The camera slowly pans left, then settles.",
"The camera slowly tilts down, then settles.",
"The camera slowly tracks right, then settles.",
"The camera orbits clockwise around the dancer, accelerating into a close profile view.",
"The camera swirls upward around the performer and resolves in an overhead view.",
"The camera rolls thirty degrees counterclockwise while tracking the singer.",
"The camera whip-pans right to catch the drummer on the downbeat.",
"The viewpoint rotates around the pair before settling behind them.",
"A lateral truck past foreground lights creates pronounced layered parallax.",
)
for sentence in camera_sentences:
with self.subTest(sentence=sentence):
self.assertTrue(self.nodes._minimax_h3_shot_has_camera_spec(sentence))
non_h3_prompt = self.nodes._build_model_prompt(
"Create a quiet portrait.",
self.nodes.DEFAULT_MASTER_PROMPT,
"ltx",
{"source": "none"},
)
self.assertNotIn("The camera remains locked-off in a stable view.", non_h3_prompt)
self.assertIn('"ltx_camera_capability"', non_h3_prompt)
def test_director_tooltips_explain_h3_camera_creativity_is_not_ltx_gated(self) -> None:
required = self.nodes.DiffusionGemmaCoTGenerator.INPUT_TYPES()["required"]
mode_tooltip = required["creativity_mode"][1]["tooltip"]
strength_tooltip = required["creative_strength"][1]["tooltip"]
self.assertIn("MiniMax H3 and LTX", mode_tooltip)
self.assertIn("expressive, physically coherent per-shot choreography", mode_tooltip)
self.assertIn("without inheriting LTX's Camera Capability", strength_tooltip)
def test_five_shot_fifteen_second_prompts_make_the_endpoint_exclusive(self) -> None:
expected_chain = "0 < t2 < t3 < t4 < t5 < 00:15.000"
brief = "Create a 15-second five-shot animated sequence."
tests/test_minimax_h3_timed_lyrics_v5_workflow.py
+345
View File
@@ -0,0 +1,345 @@
from __future__ import annotations
import copy
import hashlib
import importlib.util
import json
import os
import tempfile
import unittest
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
TOOL_PATH = REPO_ROOT / "tools" / "migrate_minimax_h3_timed_lyrics_v5.py"
SPEC = importlib.util.spec_from_file_location("minimax_h3_timed_lyrics_v5", TOOL_PATH)
if SPEC is None or SPEC.loader is None: # pragma: no cover
raise RuntimeError(f"Cannot load migration tool: {TOOL_PATH}")
MIGRATION = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(MIGRATION)
def _sha256(path: Path) -> str:
return hashlib.sha256(path.read_bytes()).hexdigest()
def _node(workflow: dict, node_id: int) -> dict:
matches = [item for item in workflow["nodes"] if int(item["id"]) == int(node_id)]
if len(matches) != 1:
raise AssertionError(f"Expected one node {node_id}; found {len(matches)}")
return matches[0]
def _origin(workflow: dict, target: dict, input_name: str) -> tuple[dict, str]:
return MIGRATION._origin(workflow, target, input_name)
@unittest.skipUnless(MIGRATION.SOURCE_V4.is_file(), "verified V4 workflow is absent")
class MiniMaxH3TimedLyricsV5WorkflowTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.source_bytes = MIGRATION.SOURCE_V4.read_bytes()
cls.source = json.loads(cls.source_bytes.decode("utf-8"))
cls.migrated = MIGRATION.migrate_workflow(
cls.source,
source_file_sha256=MIGRATION.SOURCE_V4_SHA256,
)
cls.marker = cls.migrated["extra"][MIGRATION.MIGRATION_SCHEMA]
def test_exact_v4_is_immutable_source_evidence(self) -> None:
self.assertEqual(_sha256(MIGRATION.SOURCE_V4), MIGRATION.SOURCE_V4_SHA256)
before = copy.deepcopy(self.source)
MIGRATION.migrate_workflow(
self.source,
source_file_sha256=MIGRATION.SOURCE_V4_SHA256,
)
self.assertEqual(self.source, before)
self.assertEqual(MIGRATION.SOURCE_V4.read_bytes(), self.source_bytes)
self.assertTrue(self.marker["source_is_never_overwritten"])
def test_revision_id_and_h3_subgraph_are_preserved(self) -> None:
self.assertEqual(self.migrated["id"], self.source["id"])
self.assertEqual(self.migrated["revision"], self.source["revision"] + 1)
self.assertEqual(self.migrated["definitions"], self.source["definitions"])
self.assertEqual(
self.marker["source_workflow_file_sha256"],
MIGRATION.SOURCE_V4_SHA256,
)
def test_one_authoritative_control_defaults_to_natural(self) -> None:
performance = _node(self.migrated, MIGRATION.PERFORMANCE_MODE_NODE_ID)
legacy = _node(self.migrated, MIGRATION.LEGACY_LYRIC_WINDOW_NODE_ID)
director = _node(self.migrated, MIGRATION.DIRECTOR_NODE_ID)
self.assertEqual(performance["widgets_values"][0], MIGRATION.NATURAL_MODE)
self.assertIn("one authoritative control", performance["title"])
self.assertEqual(legacy["widgets_values"][0], MIGRATION.NATURAL_MODE)
self.assertTrue(legacy["flags"]["collapsed"])
self.assertIn("mode comes only from H3 PERFORMANCE MODE", legacy["title"])
start_text = _node(self.migrated, MIGRATION.START_NODE_ID)["widgets_values"][0]
self.assertIn("Natural / audio-led sync", start_text)
self.assertIn("Only verified, hash-locked timing events", start_text)
self.assertIn("safely becomes Natural/audio-led behavior", start_text)
self.assertIn("expressive and physically coherent H3 camera work", start_text)
self.assertNotIn("camera-safe scene", start_text)
self.assertEqual(director["widgets_values"][-1], "reuse")
self.assertEqual(
self.marker["director_cache_mode"],
{
"node": MIGRATION.DIRECTOR_NODE_ID,
"default": "reuse",
"refresh_is_one_run_only": True,
},
)
self.assertEqual(
self.marker["audio_selector_policy"],
MIGRATION.AUDIO_SELECTOR_POLICY_MARKER,
)
self.assertEqual(
self.migrated["extra"]["diffusiongemma.music_video_production"]
["selector_policy"],
MIGRATION.AUDIO_SELECTOR_POLICY_MARKER,
)
self.assertEqual(
self.marker["generation_lane_boundary_policy"],
MIGRATION.BOUNDARY_POLICY_MARKER,
)
self.assertEqual(
self.migrated["extra"]["diffusiongemma.minimax_music_video_expansion"]
["generation_lane_boundary_policy"],
MIGRATION.BOUNDARY_POLICY_MARKER,
)
self.assertIn(MIGRATION.CURRENT_AUDIO_QC_COPY, start_text)
self.assertNotIn(MIGRATION.LEGACY_AUDIO_QC_COPY, start_text)
self.assertIn(MIGRATION.CURRENT_BOUNDARY_POLICY_COPY, start_text)
def test_lazy_separator_whisper_and_analyzer_are_exactly_wired(self) -> None:
nodes = self.marker["nodes"]
separator = _node(self.migrated, nodes["separator"])
whisper = _node(self.migrated, nodes["whisper_loader"])
analyzer = _node(self.migrated, nodes["analyzer"])
status_preview = _node(self.migrated, nodes["status_preview"])
alignment_preview = _node(self.migrated, nodes["alignment_preview"])
planner = _node(self.migrated, MIGRATION.PLANNER_NODE_ID)
self.assertEqual(separator["type"], MIGRATION.SEPARATOR_TYPE)
self.assertEqual(separator["widgets_values"], [10.0, 0.5, "half_sine"])
self.assertEqual(whisper["type"], MIGRATION.WHISPER_LOADER_TYPE)
self.assertEqual(whisper["widgets_values"], ["large-v3-turbo", False])
self.assertEqual(analyzer["type"], MIGRATION.ANALYZER_TYPE)
self.assertEqual(
analyzer["widgets_values"], [90.0, 0.0, 25.0, "en", 0.55]
)
self.assertEqual(
[item["name"] for item in analyzer["inputs"]],
[
"final_audio",
"performance_mode",
"lyrics",
"master_audio_sha256",
"song_duration_seconds",
"excerpt_start_seconds",
"excerpt_duration_seconds",
"vocal_stem",
"whisper_pipeline",
],
)
self.assertEqual(analyzer["inputs"][-2]["shape"], 7)
self.assertEqual(analyzer["inputs"][-1]["shape"], 7)
self.assertEqual(planner["inputs"][-1]["name"], "timed_lyrics_report_json")
self.assertEqual(planner["inputs"][-1]["shape"], 7)
self.assertEqual(
planner["title"],
"AUDIO-AWARE H3 MULTI-LANE PLAN — evidence-preferred / deterministic fallback",
)
def rectangle(node: dict) -> tuple[float, float, float, float]:
left, top = map(float, node["pos"])
width, height = map(float, node["size"])
return left, top, left + width, top + height
def overlaps(left: dict, right: dict) -> bool:
left_rect = rectangle(left)
right_rect = rectangle(right)
return not (
left_rect[2] <= right_rect[0]
or right_rect[2] <= left_rect[0]
or left_rect[3] <= right_rect[1]
or right_rect[3] <= left_rect[1]
)
added_nodes = [
separator,
whisper,
analyzer,
status_preview,
alignment_preview,
]
for left_index, left in enumerate(added_nodes):
for right in added_nodes[left_index + 1 :]:
self.assertFalse(
overlaps(left, right),
f"added node {left['id']} overlaps added node {right['id']}",
)
for existing in self.source["nodes"]:
self.assertFalse(
overlaps(left, existing),
f"added node {left['id']} overlaps preexisting node {existing['id']}",
)
planning_groups = [
group
for group in self.migrated.get("groups", [])
if str(group.get("title", "")).startswith(
"3. Verified H3 project + lazy timed lyrics"
)
]
self.assertEqual(len(planning_groups), 1)
group_left, group_top, group_width, group_height = map(
float, planning_groups[0]["bounding"]
)
group_right = group_left + group_width
group_bottom = group_top + group_height
for node in added_nodes:
left, top, right, bottom = rectangle(node)
self.assertGreaterEqual(left, group_left)
self.assertGreaterEqual(top, group_top)
self.assertLessEqual(right, group_right)
self.assertLessEqual(bottom, group_bottom)
expected = {
"final_audio": (MIGRATION.AUDIO_GUIDE_NODE_ID, "final_audio"),
"performance_mode": (
MIGRATION.PERFORMANCE_MODE_NODE_ID,
"performance_mode",
),
"lyrics": (MIGRATION.LYRICS_NODE_ID, "lyrics"),
"master_audio_sha256": (
MIGRATION.AUDIO_SELECTOR_NODE_ID,
"waveform_sha256",
),
"song_duration_seconds": (MIGRATION.SONG_DURATION_NODE_ID, "FLOAT"),
"excerpt_start_seconds": (
MIGRATION.AUDIO_SELECTOR_NODE_ID,
"suggested_start_seconds",
),
"excerpt_duration_seconds": (MIGRATION.VIDEO_DURATION_NODE_ID, "FLOAT"),
"vocal_stem": (separator["id"], "vocals"),
"whisper_pipeline": (whisper["id"], "pipeline"),
}
for input_name, (origin_id, output_name) in expected.items():
origin, output = _origin(self.migrated, analyzer, input_name)
self.assertEqual((origin["id"], output), (origin_id, output_name))
separator_origin, separator_output = _origin(
self.migrated, separator, "audio"
)
self.assertEqual(
(separator_origin["id"], separator_output),
(MIGRATION.AUDIO_GUIDE_NODE_ID, "final_audio"),
)
timing_origin, timing_output = _origin(
self.migrated, planner, "timed_lyrics_report_json"
)
self.assertEqual(
(timing_origin["id"], timing_output),
(analyzer["id"], "timed_lyrics_report_json"),
)
def test_marker_states_fail_safe_contract_without_claiming_forced_timing(self) -> None:
self.assertEqual(
self.marker["performance_modes"], list(MIGRATION.PERFORMANCE_MODES)
)
self.assertEqual(self.marker["default_performance_mode"], MIGRATION.NATURAL_MODE)
self.assertTrue(self.marker["analysis"]["lazy_for_non_lyrics_modes"])
self.assertEqual(
self.marker["timing_report"]["schema"],
"diffusiongemma.timed_lyrics_report",
)
self.assertTrue(self.marker["timing_report"]["lyrics_are_not_a_timing_schedule"])
fallback = self.marker["fallback_policy"]
self.assertEqual(fallback["mode"], MIGRATION.NATURAL_MODE)
self.assertFalse(fallback["generation_blocked_by_analysis_failure"])
self.assertEqual(fallback["instrumental_intervals"], "explicit non-vocal behavior")
def test_validator_fails_closed_on_timing_or_model_drift(self) -> None:
broken = copy.deepcopy(self.migrated)
whisper = _node(broken, self.marker["nodes"]["whisper_loader"])
whisper["widgets_values"][0] = "tiny"
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.validate_workflow(broken, source=self.source)
broken = copy.deepcopy(self.migrated)
planner = _node(broken, MIGRATION.PLANNER_NODE_ID)
planner["inputs"][-1]["shape"] = 0
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.validate_workflow(broken, source=self.source)
broken = copy.deepcopy(self.migrated)
director = _node(broken, MIGRATION.DIRECTOR_NODE_ID)
director["widgets_values"][-1] = "refresh"
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.validate_workflow(broken, source=self.source)
def test_writer_creates_new_artifact_and_refuses_different_overwrite(self) -> None:
_workflow, payload, _source_hash = MIGRATION.build_artifact()
with tempfile.TemporaryDirectory() as temporary:
target = Path(temporary) / MIGRATION.V5_FILENAME
self.assertEqual(MIGRATION._write_new_or_verify(target, payload), "created")
self.assertEqual(MIGRATION._write_new_or_verify(target, payload), "verified")
target.write_text("different", encoding="utf-8")
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION._write_new_or_verify(target, payload)
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION._write_new_or_verify(target, payload, replace_owned=True)
def test_installed_models_exist_at_the_exact_runtime_paths(self) -> None:
whisper = (
Path(r"C:\ComfyUI\app\models\whisper") / "whisper-large-v3-turbo"
)
demucs = Path(
r"C:\Users\danrh\.cache\torch\hub\torchaudio\models\hdemucs_high_trained.pt"
)
self.assertTrue((whisper / "model.safetensors").is_file())
self.assertEqual(
(whisper / "model.safetensors").stat().st_size,
MIGRATION.WHISPER_MODEL_BYTES,
)
self.assertEqual(
_sha256(whisper / "model.safetensors"),
MIGRATION.WHISPER_MODEL_SHA256,
)
self.assertTrue(demucs.is_file())
self.assertEqual(demucs.stat().st_size, MIGRATION.DEMUCS_CHECKPOINT_BYTES)
self.assertEqual(_sha256(demucs), MIGRATION.DEMUCS_CHECKPOINT_SHA256)
self.assertEqual(
self.marker["analysis"]["whisper_model_file"]["sha256"],
MIGRATION.WHISPER_MODEL_SHA256,
)
self.assertEqual(
self.marker["analysis"]["demucs_checkpoint"]["sha256"],
MIGRATION.DEMUCS_CHECKPOINT_SHA256,
)
_VERIFY_DEPLOYED_V5_ARTIFACTS = os.environ.get(
"DG_VERIFY_DEPLOYED_V5_ARTIFACTS", ""
).strip().lower() in {"1", "true", "yes", "on"}
@unittest.skipUnless(
_VERIFY_DEPLOYED_V5_ARTIFACTS
and MIGRATION.DEFAULT_DESKTOP_OUTPUT.is_file()
and MIGRATION.DEFAULT_USER_OUTPUT.is_file(),
"set DG_VERIFY_DEPLOYED_V5_ARTIFACTS=1 to audit mutable external V5 copies",
)
class MiniMaxH3TimedLyricsV5ArtifactTests(unittest.TestCase):
def test_desktop_and_comfyui_copies_are_exact_and_valid(self) -> None:
desktop = MIGRATION.DEFAULT_DESKTOP_OUTPUT.read_bytes()
user = MIGRATION.DEFAULT_USER_OUTPUT.read_bytes()
self.assertEqual(desktop, user)
workflow = json.loads(desktop.decode("utf-8"))
source = json.loads(MIGRATION.SOURCE_V4.read_text(encoding="utf-8"))
MIGRATION.validate_workflow(workflow, source=source)
if __name__ == "__main__":
unittest.main()
tests/test_minimax_h3_upload_song_v6_workflow.py
+426
View File
@@ -0,0 +1,426 @@
from __future__ import annotations
import copy
import hashlib
import importlib.util
import json
import tempfile
import unittest
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
TOOL_PATH = REPO_ROOT / "tools" / "migrate_minimax_h3_upload_song_v6.py"
SPEC = importlib.util.spec_from_file_location("minimax_h3_upload_song_v6", TOOL_PATH)
if SPEC is None or SPEC.loader is None: # pragma: no cover
raise RuntimeError(f"Cannot load migration tool: {TOOL_PATH}")
MIGRATION = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(MIGRATION)
def _sha256(path: Path) -> str:
return hashlib.sha256(path.read_bytes()).hexdigest()
def _canonical_sha256(workflow: dict) -> str:
payload = json.dumps(
workflow,
sort_keys=True,
separators=(",", ":"),
ensure_ascii=False,
).encode("utf-8")
return hashlib.sha256(payload).hexdigest()
def _node(workflow: dict, node_id: int) -> dict:
matches = [node for node in workflow["nodes"] if int(node["id"]) == int(node_id)]
if len(matches) != 1:
raise AssertionError(f"Expected one node {node_id}; found {len(matches)}")
return matches[0]
def _origin(workflow: dict, target: dict, input_name: str) -> tuple[dict, str]:
return MIGRATION._origin(workflow, target, input_name)
class MiniMaxH3UploadSongV6RepositoryArtifactTests(unittest.TestCase):
def test_checked_in_operational_workflow_is_exact_and_valid(self) -> None:
self.assertTrue(MIGRATION.REPOSITORY_EXAMPLE.is_file())
workflow = json.loads(
MIGRATION.REPOSITORY_EXAMPLE.read_text(encoding="utf-8")
)
self.assertEqual(
_canonical_sha256(workflow),
MIGRATION.REPOSITORY_EXAMPLE_CANONICAL_SHA256,
)
MIGRATION.validate_workflow(workflow)
marker = workflow["extra"][MIGRATION.MIGRATION_SCHEMA]
self.assertEqual(marker["version"], MIGRATION.MIGRATION_VERSION)
self.assertEqual(
marker["source_workflow_file_sha256"],
MIGRATION.SOURCE_V5_SHA256,
)
@unittest.skipUnless(MIGRATION.SOURCE_V5.is_file(), "attached timed-lyrics V5 is absent")
class MiniMaxH3UploadSongV6WorkflowTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.source_bytes = MIGRATION.SOURCE_V5.read_bytes()
cls.source = json.loads(cls.source_bytes.decode("utf-8"))
cls.migrated = MIGRATION.migrate_workflow(
cls.source, source_file_sha256=MIGRATION.SOURCE_V5_SHA256
)
cls.marker = cls.migrated["extra"][MIGRATION.MIGRATION_SCHEMA]
cls.loader = _node(cls.migrated, cls.marker["nodes"]["upload_loader"])
cls.router = _node(cls.migrated, cls.marker["nodes"]["source_router"])
def test_exact_attached_v5_is_immutable_source_evidence(self) -> None:
self.assertEqual(_sha256(MIGRATION.SOURCE_V5), MIGRATION.SOURCE_V5_SHA256)
before = copy.deepcopy(self.source)
MIGRATION.migrate_workflow(
self.source, source_file_sha256=MIGRATION.SOURCE_V5_SHA256
)
self.assertEqual(self.source, before)
self.assertEqual(MIGRATION.SOURCE_V5.read_bytes(), self.source_bytes)
self.assertTrue(self.marker["source_is_never_overwritten"])
self.assertEqual(
self.marker["source_workflow_file_sha256"],
MIGRATION.SOURCE_V5_SHA256,
)
def test_revision_identity_subgraphs_and_user_widgets_are_preserved(self) -> None:
self.assertEqual(self.migrated["id"], self.source["id"])
self.assertEqual(self.migrated["revision"], self.source["revision"] + 1)
self.assertEqual(self.migrated["definitions"], self.source["definitions"])
for node_id in (
MIGRATION.PRODUCTION_CONCEPT_NODE_ID,
MIGRATION.ACE_REFERENCE_MODE_NODE_ID,
MIGRATION.AUDIO_SELECTOR_NODE_ID,
MIGRATION.TIMED.PERFORMANCE_MODE_NODE_ID,
MIGRATION.PROJECT_CONTRACT_NODE_ID,
):
self.assertEqual(
_node(self.migrated, node_id).get("widgets_values"),
_node(self.source, node_id).get("widgets_values"),
)
self.assertEqual(
_node(self.migrated, MIGRATION.PRODUCTION_CONCEPT_NODE_ID)["widgets_values"],
["Source passthrough", 2],
)
self.assertEqual(
_node(self.migrated, MIGRATION.ACE_REFERENCE_MODE_NODE_ID)["widgets_values"],
["Compose new"],
)
self.assertEqual(
_node(self.migrated, MIGRATION.PROJECT_CONTRACT_NODE_ID)["widgets_values"][-1],
15,
)
def test_loader_and_router_have_explicit_safe_defaults(self) -> None:
self.assertEqual(self.loader["type"], MIGRATION.UPLOAD_LOADER_TYPE)
self.assertEqual(self.loader["widgets_values"], [""])
self.assertEqual(
[item["name"] for item in self.loader["inputs"]], ["audio_file"]
)
self.assertEqual(
[item["name"] for item in self.loader["outputs"]],
["audio", "duration_seconds", "waveform_sha256", "status", "ready"],
)
self.assertEqual(self.router["type"], MIGRATION.SOURCE_ROUTER_TYPE)
self.assertEqual(
self.router["widgets_values"], [MIGRATION.GENERATE_MODE, 0.0, ""]
)
self.assertIn("Upload song", self.router["title"])
self.assertEqual(
[item["name"] for item in self.router["outputs"]],
[
"candidate_1",
"candidate_2",
"candidate_3",
"candidate_4",
"effective_candidate_count",
"effective_expected_bpm",
"selected_lyrics",
"selected_duration_seconds",
"source_token",
"status",
"ready",
],
)
for item in self.router["inputs"][3:]:
self.assertEqual(item.get("shape"), 7, item["name"])
def test_generated_and_uploaded_sources_are_exactly_wired_to_router(self) -> None:
generated = {
"generated_candidate_count": (
MIGRATION.PRODUCTION_CONCEPT_NODE_ID,
"candidate_count",
),
"generated_expected_bpm": (MIGRATION.EXPECTED_BPM_NODE_ID, "FLOAT"),
"generated_lyrics": (MIGRATION.BLUEPRINT_ROUTER_NODE_ID, "lyrics"),
"generated_duration_seconds": (
MIGRATION.GENERATED_DURATION_NODE_ID,
"FLOAT",
),
"generated_candidate_1": (MIGRATION.ACE_CANDIDATES_NODE_ID, "AUDIO"),
"generated_candidate_2": (MIGRATION.ACE_CANDIDATES_NODE_ID, "AUDIO_1"),
"generated_candidate_3": (MIGRATION.ACE_CANDIDATES_NODE_ID, "AUDIO_2"),
"generated_candidate_4": (MIGRATION.ACE_CANDIDATES_NODE_ID, "AUDIO_3"),
}
for input_name, expected in generated.items():
origin, output = _origin(self.migrated, self.router, input_name)
self.assertEqual((origin["id"], output), expected)
uploaded = {
"uploaded_audio": "audio",
"uploaded_duration_seconds": "duration_seconds",
"uploaded_waveform_sha256": "waveform_sha256",
"uploaded_status": "status",
"uploaded_ready": "ready",
}
for input_name, output_name in uploaded.items():
origin, output = _origin(self.migrated, self.router, input_name)
self.assertEqual((origin["id"], output), (self.loader["id"], output_name))
def test_existing_selector_remains_single_qc_and_hash_authority(self) -> None:
selector = _node(self.migrated, MIGRATION.AUDIO_SELECTOR_NODE_ID)
expected = {
"candidate_1": "candidate_1",
"candidate_2": "candidate_2",
"candidate_3": "candidate_3",
"candidate_4": "candidate_4",
"candidate_count": "effective_candidate_count",
"expected_bpm": "effective_expected_bpm",
"source_policy": "source_token",
}
for input_name, output_name in expected.items():
origin, output = _origin(self.migrated, selector, input_name)
self.assertEqual((origin["id"], output), (self.router["id"], output_name))
# Existing output ordering is contract-sensitive and must not be rewritten.
self.assertEqual(
[item["name"] for item in selector["outputs"]],
[
"selected_audio",
"suggested_start_seconds",
"waveform_sha256",
"director_report_json",
"audit_report_json",
"status",
"ready",
],
)
project = _node(self.migrated, MIGRATION.PROJECT_CONTRACT_NODE_ID)
origin, output = _origin(self.migrated, project, "master_audio_sha256")
self.assertEqual(
(origin["id"], output),
(MIGRATION.AUDIO_SELECTOR_NODE_ID, "waveform_sha256"),
)
def test_selected_lyrics_and_exact_duration_replace_generated_metadata_only_at_seam(self) -> None:
for target_id in (
MIGRATION.LYRIC_WINDOW_NODE_ID,
MIGRATION.PROJECT_CONTRACT_NODE_ID,
MIGRATION.TIMED_ANALYZER_NODE_ID,
):
origin, output = _origin(
self.migrated, _node(self.migrated, target_id), "lyrics"
)
self.assertEqual(
(origin["id"], output), (self.router["id"], "selected_lyrics")
)
for target_id in (
MIGRATION.LYRIC_WINDOW_NODE_ID,
MIGRATION.TIMED_ANALYZER_NODE_ID,
):
origin, output = _origin(
self.migrated,
_node(self.migrated, target_id),
"song_duration_seconds",
)
self.assertEqual(
(origin["id"], output),
(self.router["id"], "selected_duration_seconds"),
)
planner = _node(self.migrated, MIGRATION.H3_PLANNER_NODE_ID)
origin, output = _origin(self.migrated, planner, "lyrics")
self.assertEqual(
(origin["id"], output),
(MIGRATION.LYRIC_WINDOW_NODE_ID, "selected_lyrics"),
)
def test_help_and_inherited_policy_markers_are_normalized_additively(self) -> None:
self.assertEqual(self.marker["policy"], MIGRATION.UPLOAD_SOURCE_POLICY_MARKER)
self.assertIn(
"not waveform passthrough",
self.marker["policy"]["source_passthrough_semantics"],
)
self.assertEqual(
self.marker["policy"]["upload_semantics"]["selector_policy"],
"source_locked_uploaded_song",
)
self.assertIn(
"tempo interpretation",
self.marker["policy"]["upload_semantics"]["advisory_qc"],
)
extra = self.migrated["extra"]
music = extra[MIGRATION.PRODUCTION.MIGRATION_SCHEMA]
self.assertEqual(music["version"], MIGRATION.PRODUCTION.MIGRATION_VERSION)
self.assertEqual(
music["selector_policy"], MIGRATION.PRODUCTION.SELECTOR_POLICY_MARKER
)
expansion = extra[MIGRATION.EXPANSION.MIGRATION_SCHEMA]
self.assertEqual(
expansion["generation_lane_boundary_policy"],
MIGRATION.EXPANSION.BOUNDARY_POLICY_MARKER,
)
timed = extra[MIGRATION.TIMED.MIGRATION_SCHEMA]
self.assertEqual(
timed["audio_selector_policy"],
MIGRATION.TIMED.AUDIO_SELECTOR_POLICY_MARKER,
)
self.assertEqual(
timed["generation_lane_boundary_policy"],
MIGRATION.TIMED.BOUNDARY_POLICY_MARKER,
)
start = _node(self.migrated, MIGRATION.START_NODE_ID)["widgets_values"][0]
self.assertIn(MIGRATION.UPLOAD_START_COPY, start)
self.assertIn(MIGRATION.TIMED.CURRENT_AUDIO_QC_COPY, start)
self.assertIn(MIGRATION.TIMED.CURRENT_BOUNDARY_POLICY_COPY, start)
self.assertIn("Source passthrough", start)
self.assertIn(MIGRATION.H3_DANCE_HELP, start)
self.assertNotIn(MIGRATION.LEGACY_DANCE_HELP, start)
guidance = _node(
self.migrated, MIGRATION.AUDIO_GUIDANCE_NODE_ID
)["widgets_values"][0]
self.assertEqual(guidance, MIGRATION.H3_AUDIO_GUIDANCE)
self.assertIn("expressive, physically coherent H3 camera choreography", guidance)
self.assertIn("orbiting", guidance)
self.assertIn("pronounced parallax", guidance)
self.assertNotIn("restrained camera accents", guidance)
self.assertEqual(
self.marker["h3_camera_policy"],
{
"revision": 1,
"ltx_camera_capability_applies": False,
"dynamic_paths_allowed": [
"orbit",
"swirl",
"sweep_or_whip",
"pronounced_parallax",
"coherent_compound_path",
],
"creative_energy_source": "director_creativity_mode_and_strength",
},
)
def test_added_nodes_do_not_overlap_existing_nodes_and_group_contains_them(self) -> None:
def rectangle(node: dict) -> tuple[float, float, float, float]:
x, y = map(float, node["pos"])
width, height = map(float, node["size"])
return x, y, x + width, y + height
def overlaps(left: dict, right: dict) -> bool:
a = rectangle(left)
b = rectangle(right)
return not (a[2] <= b[0] or b[2] <= a[0] or a[3] <= b[1] or b[3] <= a[1])
self.assertFalse(overlaps(self.loader, self.router))
for added in (self.loader, self.router):
for existing in self.source["nodes"]:
self.assertFalse(
overlaps(added, existing),
f"added node {added['id']} overlaps source node {existing['id']}",
)
groups = [group for group in self.migrated["groups"] if int(group["id"]) == 35]
self.assertEqual(len(groups), 1)
gx, gy, gw, gh = map(float, groups[0]["bounding"])
for added in (self.loader, self.router):
left, top, right, bottom = rectangle(added)
self.assertGreaterEqual(left, gx)
self.assertGreaterEqual(top, gy)
self.assertLessEqual(right, gx + gw)
self.assertLessEqual(bottom, gy + gh)
def test_migration_is_idempotent_without_duplicate_nodes_or_revision(self) -> None:
repeated = MIGRATION.migrate_workflow(
self.migrated, source_file_sha256="already-owned-v6"
)
self.assertEqual(repeated, self.migrated)
self.assertEqual(
len(
[
node
for node in repeated["nodes"]
if node["type"] == MIGRATION.UPLOAD_LOADER_TYPE
]
),
1,
)
self.assertEqual(
len(
[
node
for node in repeated["nodes"]
if node["type"] == MIGRATION.SOURCE_ROUTER_TYPE
]
),
1,
)
def test_validator_fails_closed_on_route_policy_or_user_control_drift(self) -> None:
broken = copy.deepcopy(self.migrated)
router = _node(broken, self.router["id"])
router["widgets_values"][0] = "Unsupported source"
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.validate_workflow(broken, source=self.source)
# A user's saved Upload choice remains a valid, idempotent V6 artifact.
upload_saved = copy.deepcopy(self.migrated)
router = _node(upload_saved, self.router["id"])
router["widgets_values"] = [MIGRATION.UPLOAD_MODE, 123.0, "Exact sung lyrics"]
MIGRATION.validate_workflow(upload_saved)
self.assertEqual(
MIGRATION.migrate_workflow(
upload_saved, source_file_sha256="already-owned-v6"
),
upload_saved,
)
broken = copy.deepcopy(self.migrated)
marker = broken["extra"][MIGRATION.MIGRATION_SCHEMA]
marker["policy"]["revision"] = 999
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.validate_workflow(broken, source=self.source)
broken = copy.deepcopy(self.migrated)
selector = _node(broken, MIGRATION.AUDIO_SELECTOR_NODE_ID)
selector["widgets_values"][4] = 0.1
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.validate_workflow(broken, source=self.source)
def test_writer_is_deterministic_and_replaces_only_owned_v6(self) -> None:
_workflow, payload, _source_hash = MIGRATION.build_artifact()
with tempfile.TemporaryDirectory() as temporary:
target = Path(temporary) / MIGRATION.V6_FILENAME
self.assertEqual(MIGRATION._write_new_or_verify(target, payload), "created")
self.assertEqual(MIGRATION._write_new_or_verify(target, payload), "verified")
owned = json.loads(payload.decode("utf-8"))
owned["extra"]["workflow_note"] += " changed"
target.write_text(json.dumps(owned), encoding="utf-8")
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION._write_new_or_verify(target, payload)
self.assertEqual(
MIGRATION._write_new_or_verify(target, payload, replace_owned=True),
"replaced",
)
target.write_text("unrecognized", encoding="utf-8")
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION._write_new_or_verify(target, payload, replace_owned=True)
def test_wrong_source_hash_is_rejected(self) -> None:
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.migrate_workflow(self.source, source_file_sha256="0" * 64)
if __name__ == "__main__": # pragma: no cover
unittest.main()
tests/test_minimax_music_video_expansion_workflow.py
+674
View File
@@ -0,0 +1,674 @@
from __future__ import annotations
import copy
import hashlib
import importlib.util
import json
import shutil
import tempfile
import unittest
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
MIGRATION_PATH = REPO_ROOT / "tools" / "migrate_minimax_music_video_expansion.py"
SPEC = importlib.util.spec_from_file_location("minimax_music_video_migration", MIGRATION_PATH)
if SPEC is None or SPEC.loader is None: # pragma: no cover
raise RuntimeError(f"Cannot load migration module: {MIGRATION_PATH}")
MIGRATION = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(MIGRATION)
WORKFLOW = Path(
r"C:\Users\danrh\Desktop\14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized.json"
)
BACKUP = WORKFLOW.with_suffix(WORKFLOW.suffix + MIGRATION.BACKUP_SUFFIX)
EXPECTED_ORIGINAL_SHA256 = (
"4cb0d937140822ee7dd82fcda41385d645a90de34101b88c55769d7ddb7009fe"
)
LIVE_STABILITY_CAPTURE = Path(
r"C:\Users\danrh\Desktop\14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_BASELINE_FAST_H3_LIVE_UNSAVED_PRE_REPAIR_20260822_0442.json"
)
LIVE_STABILITY_CAPTURE_SHA256 = (
"3d647ff4d1d147e64006902121b5f8ca0f03bcc2fc318a348cc3f9c56d2cc9a1"
)
def _fixture_path() -> Path:
if BACKUP.is_file():
return BACKUP
return WORKFLOW
def _node(workflow: dict, node_id: int) -> dict:
matches = [node for node in workflow["nodes"] if int(node["id"]) == node_id]
if len(matches) != 1:
raise AssertionError(f"Expected node {node_id}; found {len(matches)}")
return matches[0]
def _one(workflow: dict, node_type: str) -> dict:
matches = [node for node in workflow["nodes"] if node.get("type") == node_type]
if len(matches) != 1:
raise AssertionError(f"Expected one {node_type}; found {len(matches)}")
return matches[0]
def _output(node: dict, name: str) -> dict:
return next(item for item in node["outputs"] if item.get("name") == name)
def _input(node: dict, name: str) -> dict:
return next(item for item in node["inputs"] if item.get("name") == name)
def _origin(workflow: dict, node: dict, input_name: str) -> tuple[dict, str]:
link_id = _input(node, input_name).get("link")
if link_id is None:
raise AssertionError(f"{node['id']}.{input_name} is unconnected")
link = next(link for link in workflow["links"] if int(link[0]) == int(link_id))
origin = _node(workflow, int(link[1]))
return origin, origin["outputs"][int(link[2])]["name"]
def _as_v1(workflow: dict) -> dict:
legacy = copy.deepcopy(workflow)
marker = legacy["extra"][MIGRATION.MIGRATION_SCHEMA]
marker["version"] = MIGRATION.LEGACY_MIGRATION_VERSION
marker.pop("max_h3_generation_lanes", None)
marker.pop("max_h3_generation_lane_seconds", None)
marker.pop("count_semantics", None)
for key in (
"generation_lane_planner",
"generation_lane_seed_fanout",
"generation_lane_assembler",
):
marker["nodes"].pop(key, None)
return legacy
def _add_legacy_lane_count_reroute(
workflow: dict,
*,
connected: bool,
title: str | None = None,
mode: int = 4,
) -> tuple[int, int, int | None]:
graph = MIGRATION.MainGraph(workflow, MIGRATION.IdAllocator(workflow))
project = graph.one(MIGRATION.PROJECT_NODE_TYPE)
target = graph.node(673)
reroute_id = graph.ids.node()
reroute = {
"id": reroute_id,
"type": "Reroute",
"pos": [0.0, 0.0],
"size": [75.0, 26.0],
"flags": {},
"order": 0,
"mode": mode,
"inputs": [
{
"name": "",
"type": "*",
"widget": {"name": "value"},
"link": None,
}
],
"outputs": [{"name": "", "type": "INT", "links": []}],
"properties": {"horizontal": False, "showOutputText": False},
"widgets_values": None,
}
if title is not None:
reroute["title"] = title
graph.add(reroute)
input_link = graph.connect(
project,
"recommended_h3_shot_count",
reroute,
"",
"INT",
)
output_link = None
if connected:
output_link = graph.connect(
reroute,
"",
target,
"shot_count_override",
"INT",
)
workflow["last_node_id"] = graph.ids.last_node
workflow["last_link_id"] = graph.ids.last_link
return reroute_id, input_link, output_link
@unittest.skipUnless(WORKFLOW.is_file() or BACKUP.is_file(), "attached MiniMax workflow is absent")
class MiniMaxMusicVideoExpansionWorkflowTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.fixture_path = _fixture_path()
cls.source_bytes = cls.fixture_path.read_bytes()
cls.source = json.loads(cls.source_bytes.decode("utf-8"))
# Always run through the public entry point: a v1 expansion is upgraded
# in memory, while a v2 expansion remains byte-for-byte idempotent.
cls.migrated = MIGRATION.migrate_workflow(
cls.source,
source_file_sha256=hashlib.sha256(cls.source_bytes).hexdigest(),
)
def test_exact_attached_source_hash_is_known_before_first_write(self) -> None:
if MIGRATION.MIGRATION_SCHEMA not in self.source.get("extra", {}):
self.assertEqual(
hashlib.sha256(self.source_bytes).hexdigest(),
EXPECTED_ORIGINAL_SHA256,
)
def test_migration_validates_and_is_idempotent(self) -> None:
MIGRATION.validate_workflow(self.migrated)
self.assertEqual(
MIGRATION.migrate_workflow(self.migrated),
self.migrated,
)
def test_natural_mode_is_appended_default_without_reindexing_saved_modes(self) -> None:
self.assertEqual(
MIGRATION.PERFORMANCE_MODES,
(
"Dance / music sync",
"Lyrics + lip sync",
"Natural / audio-led sync",
),
)
self.assertEqual(
MIGRATION.DEFAULT_PERFORMANCE_MODE, "Natural / audio-led sync"
)
marker = self.migrated["extra"][MIGRATION.MIGRATION_SCHEMA][
"performance_mode_contract"
]
self.assertEqual(marker, MIGRATION.PERFORMANCE_MODE_MARKER)
note = _node(self.migrated, 116)["widgets_values"][0]
self.assertIn("defaults to **Natural / audio-led sync**", note)
self.assertIn("lyrics are not a timing schedule", note)
def test_boundary_policy_is_advisory_and_truthfully_documented(self) -> None:
marker = self.migrated["extra"][MIGRATION.MIGRATION_SCHEMA]
self.assertEqual(
marker["generation_lane_boundary_policy"],
MIGRATION.BOUNDARY_POLICY_MARKER,
)
note = _node(self.migrated, 116)["widgets_values"][0]
self.assertIn("deterministic balanced seams", note)
self.assertIn("Missing ideal cut evidence is advisory", note)
planner = _one(self.migrated, MIGRATION.PLANNER_NODE_TYPE)
self.assertEqual(
planner["title"],
"AUDIO-AWARE H3 MULTI-LANE PLAN — evidence-preferred / deterministic fallback",
)
self.assertIn(
"deterministic balanced fallback",
self.migrated["extra"]["workflow_note"],
)
def test_invalid_saved_mode_falls_back_to_natural_without_changing_valid_choices(self) -> None:
edited = copy.deepcopy(self.migrated)
control = _one(edited, MIGRATION.PERFORMANCE_MODE_NODE_TYPE)
control["widgets_values"] = ["unsupported legacy value"]
repaired = MIGRATION.migrate_workflow(edited)
repaired_control = _one(repaired, MIGRATION.PERFORMANCE_MODE_NODE_TYPE)
self.assertEqual(
repaired_control["widgets_values"], ["Natural / audio-led sync"]
)
self.assertEqual(_node(repaired, 621)["widgets_values"][0], "Lyrics + lip sync")
def test_user_controls_ace_subgraph_and_legacy_marker_are_preserved(self) -> None:
migrated = self.migrated
source_marker = self.source.get("extra", {}).get(MIGRATION.MIGRATION_SCHEMA)
if isinstance(source_marker, dict) and int(source_marker.get("version", 0)) >= 2:
self.skipTest("pre-v2 fixture is unavailable")
self.assertEqual(
migrated.get("definitions"),
self.source.get("definitions"),
)
self.assertEqual(
migrated["extra"]["diffusiongemma.music_video_production"],
self.source["extra"]["diffusiongemma.music_video_production"],
)
removed = {511, 549, 655, 669, 672}
changed_widget_contract = {116, 624, 665, 673}
migrated_ids = {int(node["id"]) for node in migrated["nodes"]}
for source_node in self.source["nodes"]:
node_id = int(source_node["id"])
if node_id in removed or node_id in changed_widget_contract:
continue
self.assertIn(node_id, migrated_ids)
self.assertEqual(
_node(migrated, node_id).get("widgets_values"),
source_node.get("widgets_values"),
f"widgets changed on original node {node_id}",
)
self.assertEqual(
_node(migrated, 177)["widgets_values"],
_node(self.source, 177)["widgets_values"],
)
self.assertEqual(_node(migrated, 178)["widgets_values"], [15])
self.assertEqual(_node(migrated, 621)["widgets_values"][0], "Lyrics + lip sync")
def test_song_context_and_h3_reference_context_are_separate(self) -> None:
migrated = self.migrated
song_context = _node(migrated, 183)
h3_context = _one(migrated, MIGRATION.H3_CONTEXT_NODE_TYPE)
self.assertEqual(song_context["type"], "DiffusionGemmaContextHub")
origin, output = _origin(migrated, _node(migrated, 578), "gemma_context")
self.assertEqual((origin["id"], output), (183, "gemma_context"))
for target_id in (186, 187):
origin, output = _origin(migrated, _node(migrated, target_id), "gemma_context")
self.assertEqual((origin["id"], output), (h3_context["id"], "gemma_context"))
manifest = h3_context["widgets_values"][0]
self.assertIn("<Picture 1>:", manifest)
self.assertIn("<Audio 1>:", manifest)
self.assertEqual(h3_context["widgets_values"][-1], 0)
def test_project_controls_aspect_and_generation_lanes_without_overriding_native_shots(self) -> None:
migrated = self.migrated
project = _one(migrated, MIGRATION.PROJECT_NODE_TYPE)
splitter = _node(migrated, 187)
target = _node(migrated, 673)
aspect_origin, aspect_output = _origin(
migrated, splitter, "resolution_aspect_ratio_override"
)
self.assertEqual((aspect_origin["id"], aspect_output), (project["id"], "master_aspect_ratio"))
self.assertIsNone(_input(target, "shot_count_override").get("link"))
self.assertEqual(
_input(target, "shot_count_override").get("widget"),
{"name": "shot_count_override"},
)
self.assertEqual(target["widgets_values"][-1], 0)
marker = migrated["extra"][MIGRATION.MIGRATION_SCHEMA]
self.assertEqual(marker["version"], 2)
self.assertEqual(marker["master_video_duration_seconds"], 15.0)
self.assertEqual(marker["master_aspect_ratio"], "9:16")
self.assertEqual(marker["max_h3_generation_lanes"], 4)
self.assertEqual(marker["max_h3_generation_lane_seconds"], 15.0)
self.assertFalse(
marker["count_semantics"][
"project_master_to_target_shot_override_connected"
]
)
self.assertIn("native_[Shot_N]_blocks", marker["count_semantics"]["target_profile_shot_count"])
def test_v1_upgrade_removes_only_the_conflated_link_and_preserves_user_state(self) -> None:
legacy = copy.deepcopy(self.migrated)
marker = legacy["extra"][MIGRATION.MIGRATION_SCHEMA]
marker["version"] = MIGRATION.LEGACY_MIGRATION_VERSION
marker.pop("max_h3_generation_lanes", None)
marker.pop("max_h3_generation_lane_seconds", None)
marker.pop("count_semantics", None)
for key in (
"generation_lane_planner",
"generation_lane_seed_fanout",
"generation_lane_assembler",
):
marker["nodes"].pop(key, None)
graph = MIGRATION.MainGraph(legacy, MIGRATION.IdAllocator(legacy))
project = graph.one(MIGRATION.PROJECT_NODE_TYPE)
target = graph.node(673)
target["widgets_values"][4:6] = ["custom", 7]
target["title"] = "USER CUSTOM H3 TARGET TITLE"
note = graph.node(116)
note["title"] = "USER CUSTOM START NOTE"
note["widgets_values"] = ["Keep my production instructions unchanged."]
unrelated = graph.node(177)
unrelated["title"] = "USER CUSTOM CREATIVE BRIEF"
unrelated["pos"] = [123.0, 456.0]
override_link = graph.connect(
project,
"recommended_h3_shot_count",
target,
"shot_count_override",
"INT",
)
legacy["last_link_id"] = graph.ids.last_link
preserved_links = [
copy.deepcopy(link)
for link in legacy["links"]
if int(link[0]) != int(override_link)
]
preserved_definitions = copy.deepcopy(legacy.get("definitions"))
preserved_node_ids = [int(node["id"]) for node in legacy["nodes"]]
upgraded = MIGRATION.migrate_workflow(legacy)
MIGRATION.validate_workflow(upgraded)
upgraded_target = _node(upgraded, 673)
self.assertIsNone(_input(upgraded_target, "shot_count_override").get("link"))
self.assertEqual(upgraded_target["widgets_values"][4:6], ["custom", 7])
self.assertEqual(upgraded_target["widgets_values"][-1], 0)
self.assertEqual(upgraded_target["title"], "USER CUSTOM H3 TARGET TITLE")
self.assertEqual(_node(upgraded, 116)["title"], "USER CUSTOM START NOTE")
self.assertEqual(
_node(upgraded, 116)["widgets_values"],
["Keep my production instructions unchanged."],
)
self.assertEqual(_node(upgraded, 177)["title"], "USER CUSTOM CREATIVE BRIEF")
self.assertEqual(_node(upgraded, 177)["pos"], [123.0, 456.0])
self.assertEqual(upgraded.get("definitions"), preserved_definitions)
self.assertEqual([int(node["id"]) for node in upgraded["nodes"]], preserved_node_ids)
self.assertEqual(upgraded["links"], preserved_links)
self.assertEqual(
upgraded["extra"][MIGRATION.MIGRATION_SCHEMA]["version"],
MIGRATION.MIGRATION_VERSION,
)
self.assertEqual(
upgraded["extra"][MIGRATION.MIGRATION_SCHEMA][
"generation_lane_boundary_policy"
],
MIGRATION.BOUNDARY_POLICY_MARKER,
)
def test_v1_upgrade_removes_exact_sole_legacy_reroute_chain(self) -> None:
legacy = _as_v1(self.migrated)
reroute_id, input_link, output_link = _add_legacy_lane_count_reroute(
legacy, connected=True
)
self.assertIsNotNone(output_link)
upgraded = MIGRATION.migrate_workflow(legacy)
MIGRATION.validate_workflow(upgraded)
self.assertNotIn(reroute_id, {int(node["id"]) for node in upgraded["nodes"]})
self.assertFalse(
{input_link, int(output_link)}
& {int(link[0]) for link in upgraded["links"]}
)
self.assertIsNone(_input(_node(upgraded, 673), "shot_count_override").get("link"))
def test_v1_upgrade_removes_exact_disconnected_legacy_reroute(self) -> None:
legacy = _as_v1(self.migrated)
reroute_id, input_link, output_link = _add_legacy_lane_count_reroute(
legacy, connected=False
)
self.assertIsNone(output_link)
upgraded = MIGRATION.migrate_workflow(legacy)
MIGRATION.validate_workflow(upgraded)
self.assertNotIn(reroute_id, {int(node["id"]) for node in upgraded["nodes"]})
self.assertNotIn(input_link, {int(link[0]) for link in upgraded["links"]})
self.assertIsNone(_input(_node(upgraded, 673), "shot_count_override").get("link"))
def test_v1_upgrade_refuses_shared_legacy_reroute(self) -> None:
legacy = _as_v1(self.migrated)
reroute_id, _input_link, _output_link = _add_legacy_lane_count_reroute(
legacy, connected=True
)
graph = MIGRATION.MainGraph(legacy, MIGRATION.IdAllocator(legacy))
reroute = graph.node(reroute_id)
sink_id = graph.ids.node()
sink = {
"id": sink_id,
"type": "Reroute",
"pos": [100.0, 0.0],
"size": [75.0, 26.0],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [{"name": "", "type": "*", "link": None}],
"outputs": [{"name": "", "type": "INT", "links": []}],
"properties": {},
"widgets_values": None,
}
graph.add(sink)
graph.connect(reroute, "", sink, "", "INT")
legacy["last_node_id"] = graph.ids.last_node
legacy["last_link_id"] = graph.ids.last_link
with self.assertRaisesRegex(MIGRATION.WorkflowError, "shared/user-authored"):
MIGRATION.migrate_workflow(legacy)
def test_v1_upgrade_refuses_customized_user_reroute(self) -> None:
legacy = _as_v1(self.migrated)
_add_legacy_lane_count_reroute(
legacy,
connected=True,
title="USER SHOT OVERRIDE ROUTE",
)
with self.assertRaisesRegex(MIGRATION.WorkflowError, "customized/user-authored"):
MIGRATION.migrate_workflow(legacy)
def test_v2_start_note_warns_that_refresh_is_one_run_only(self) -> None:
note = _node(self.migrated, 116)["widgets_values"][0]
self.assertIn("Leave it on **reuse**", note)
self.assertIn("**refresh** is a one-run diagnostic", note)
def test_new_node_socket_order_matches_the_landed_contracts(self) -> None:
project = _one(self.migrated, MIGRATION.PROJECT_NODE_TYPE)
planner = _one(self.migrated, MIGRATION.PLANNER_NODE_TYPE)
assembler = _one(self.migrated, MIGRATION.ASSEMBLER_NODE_TYPE)
self.assertEqual(
[item["name"] for item in project["inputs"]],
[
"creative_brief",
"production_duration_seconds",
"master_audio_sha256",
"lyrics",
"excerpt_start_seconds",
"excerpt_duration_seconds",
],
)
self.assertEqual(
[item["name"] for item in planner["inputs"]],
[
"measured_audio_report_json",
"base_h3_prompt",
"performance_mode",
"excerpt_start_seconds",
"excerpt_duration_seconds",
"max_shot_seconds",
"project_manifest_json",
"lyrics",
],
)
self.assertEqual(
[item["name"] for item in assembler["inputs"]],
[
"final_audio",
"plan_json",
"target_duration_seconds",
"shot_1_images",
"shot_2_images",
"shot_3_images",
"shot_4_images",
],
)
def test_performance_toggle_has_one_typed_mode_source_and_no_ltx_prompt_path(self) -> None:
migrated = self.migrated
guidance = _node(migrated, 624)["widgets_values"][0]
self.assertIn("expressive, physically coherent H3 camera choreography", guidance)
self.assertIn("orbiting", guidance)
self.assertIn("pronounced parallax", guidance)
self.assertNotIn("restrained camera accents", guidance)
performance = _node(migrated, 621)
performance_control = _one(migrated, MIGRATION.PERFORMANCE_MODE_NODE_TYPE)
planner = _one(migrated, MIGRATION.PLANNER_NODE_TYPE)
self.assertEqual(
[item["name"] for item in performance["outputs"]],
["ltx_prompt", "selected_lyrics", "status", "performance_report_json", "performance_mode"],
)
self.assertEqual(_output(performance, "ltx_prompt").get("links"), [])
mode_origin, mode_output = _origin(migrated, planner, "performance_mode")
lyric_origin, lyric_output = _origin(migrated, planner, "lyrics")
self.assertEqual(
(mode_origin["id"], mode_output),
(performance_control["id"], "performance_mode"),
)
self.assertEqual((lyric_origin["id"], lyric_output), (621, "selected_lyrics"))
lyric_mode_origin, lyric_mode_output = _origin(
migrated, performance, "performance_mode_override"
)
self.assertEqual(
(lyric_mode_origin["id"], lyric_mode_output),
(performance_control["id"], "performance_mode"),
)
guidance_origin, guidance_output = _origin(
migrated, _node(migrated, 673), "audio_guidance"
)
self.assertEqual(
(guidance_origin["id"], guidance_output),
(performance_control["id"], "target_audio_guidance"),
)
self.assertEqual(performance_control["widgets_values"], ["Lyrics + lip sync"])
self.assertEqual(_node(migrated, 673)["widgets_values"][6], "off")
maximum_origin, maximum_output = _origin(
migrated, planner, "max_shot_seconds"
)
project = _one(migrated, MIGRATION.PROJECT_NODE_TYPE)
self.assertEqual(
(maximum_origin["id"], maximum_output),
(project["id"], "max_h3_shot_seconds"),
)
def test_four_lazy_h3_lanes_use_audio_1_and_retained_cut_continuity(self) -> None:
migrated = self.migrated
marker = migrated["extra"][MIGRATION.MIGRATION_SCHEMA]
runtime = marker["runtime_lane_node_ids"]
self.assertEqual(len(runtime["h3"]), 4)
self.assertEqual(len(runtime["trims"]), 4)
self.assertEqual(len(runtime["tail_math"]), 3)
self.assertEqual(len(runtime["tails"]), 3)
h3_nodes = [_node(migrated, node_id) for node_id in runtime["h3"]]
for index, h3 in enumerate(h3_nodes, start=1):
names = [item["name"] for item in h3["inputs"]]
self.assertIn("ref_audios.ref_audio_0", names)
self.assertNotIn("ref_audios.ref_audio_1", names)
prompt_origin, prompt_output = _origin(migrated, h3, "prompt")
self.assertEqual(prompt_origin["type"], MIGRATION.PLANNER_NODE_TYPE)
self.assertEqual(prompt_output, f"shot_{index}_prompt")
for math_id in runtime["tail_math"]:
self.assertEqual(
_node(migrated, math_id)["widgets_values"],
["max(0, round((a + b) * 24) - round(a * 24) - 1)"],
)
def test_final_mux_uses_assembler_and_pristine_ace_audio(self) -> None:
migrated = self.migrated
assembler = _one(migrated, MIGRATION.ASSEMBLER_NODE_TYPE)
create_video = _node(migrated, 664)
audio_origin, audio_output = _origin(migrated, assembler, "final_audio")
self.assertEqual((audio_origin["type"], audio_output), ("DiffusionGemmaLTXAudioGuide", "final_audio"))
images_origin, images_output = _origin(migrated, create_video, "images")
mux_audio_origin, mux_audio_output = _origin(migrated, create_video, "audio")
self.assertEqual((images_origin["id"], images_output), (assembler["id"], "assembled_images"))
self.assertEqual((mux_audio_origin["id"], mux_audio_output), (assembler["id"], "final_audio"))
self.assertFalse(any(node["type"] == "VAEDecodeAudio" for node in migrated["nodes"]))
def test_link_and_security_corruption_fail_closed(self) -> None:
broken = copy.deepcopy(self.migrated)
assembler = _one(broken, MIGRATION.ASSEMBLER_NODE_TYPE)
_input(assembler, "final_audio")["link"] = None
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.validate_workflow(broken)
wrong_type = copy.deepcopy(self.migrated)
wrong_type["links"][0][5] = "DELIBERATELY_WRONG_TYPE"
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.validate_workflow(wrong_type)
secret = copy.deepcopy(self.source)
secret.setdefault("extra", {})["api_key"] = "do-not-write-this"
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.migrate_workflow(secret)
def test_atomic_write_makes_one_verified_recovery_backup(self) -> None:
source_marker = self.source.get("extra", {}).get(MIGRATION.MIGRATION_SCHEMA)
if isinstance(source_marker, dict) and int(source_marker.get("version", 0)) >= 2:
self.skipTest("pre-v2 fixture is unavailable")
with tempfile.TemporaryDirectory() as temporary:
target = Path(temporary) / WORKFLOW.name
shutil.copy2(self.fixture_path, target)
source_bytes = target.read_bytes()
migrated, source_hash, backup = MIGRATION.migrate_file(
target,
write=True,
expected_source_sha256=hashlib.sha256(source_bytes).hexdigest(),
)
self.assertIsNotNone(backup)
assert backup is not None
self.assertEqual(backup.read_bytes(), source_bytes)
self.assertEqual(source_hash, hashlib.sha256(source_bytes).hexdigest())
MIGRATION.validate_workflow(migrated)
migrated_bytes = target.read_bytes()
unchanged, _second_hash, second_backup = MIGRATION.migrate_file(
target,
write=True,
)
self.assertIsNone(second_backup)
self.assertEqual(target.read_bytes(), migrated_bytes)
self.assertEqual(backup.read_bytes(), source_bytes)
self.assertEqual(unchanged, migrated)
def test_mismatched_existing_backup_is_never_reused_or_overwritten(self) -> None:
source_marker = self.source.get("extra", {}).get(MIGRATION.MIGRATION_SCHEMA)
if isinstance(source_marker, dict) and int(source_marker.get("version", 0)) >= 2:
self.skipTest("pre-v2 fixture is unavailable")
with tempfile.TemporaryDirectory() as temporary:
target = Path(temporary) / WORKFLOW.name
shutil.copy2(self.fixture_path, target)
source_bytes = target.read_bytes()
backup = target.with_suffix(target.suffix + MIGRATION.BACKUP_SUFFIX)
backup.write_bytes(b"unrelated recovery copy")
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.migrate_file(
target,
write=True,
expected_source_sha256=hashlib.sha256(source_bytes).hexdigest(),
)
self.assertEqual(target.read_bytes(), source_bytes)
self.assertEqual(backup.read_bytes(), b"unrelated recovery copy")
def test_source_digest_guard_refuses_the_wrong_file(self) -> None:
with tempfile.TemporaryDirectory() as temporary:
target = Path(temporary) / WORKFLOW.name
shutil.copy2(self.fixture_path, target)
with self.assertRaises(MIGRATION.WorkflowError):
MIGRATION.migrate_file(
target,
write=True,
expected_source_sha256="0" * 64,
)
self.assertFalse(
target.with_suffix(target.suffix + MIGRATION.BACKUP_SUFFIX).exists()
)
@unittest.skipUnless(
LIVE_STABILITY_CAPTURE.is_file(), "exact FAST H3 live stability capture is absent"
)
class ExactFastH3LiveStabilityCaptureTests(unittest.TestCase):
def test_exact_capture_v1_to_v2_dry_run_removes_only_legacy_reroute_chain(self) -> None:
source_bytes = LIVE_STABILITY_CAPTURE.read_bytes()
self.assertEqual(
hashlib.sha256(source_bytes).hexdigest(),
LIVE_STABILITY_CAPTURE_SHA256,
)
source = json.loads(source_bytes.decode("utf-8"))
migrated = MIGRATION.migrate_workflow(
source,
source_file_sha256=LIVE_STABILITY_CAPTURE_SHA256,
)
MIGRATION.validate_workflow(migrated)
self.assertEqual(
migrated["extra"][MIGRATION.MIGRATION_SCHEMA]["version"],
MIGRATION.MIGRATION_VERSION,
)
self.assertNotIn(711, {int(node["id"]) for node in migrated["nodes"]})
self.assertFalse(
{1396, 1416} & {int(link[0]) for link in migrated["links"]}
)
target = _node(migrated, 673)
self.assertIsNone(_input(target, "shot_count_override").get("link"))
self.assertEqual(target["widgets_values"][4:6], ["custom", 15])
note = _node(migrated, 116)["widgets_values"][0]
self.assertIn("Leave it on **reuse**", note)
self.assertEqual(MIGRATION.migrate_workflow(migrated), migrated)
if __name__ == "__main__":
unittest.main()
tests/test_model_target_profile_nodes.py
+32 -2
View File
@@ -46,9 +46,16 @@ class ModelTargetProfileNodeTests(unittest.TestCase):
"audio_guidance",
"negative_prompt_mode",
"negative_prompt_guidance",
"long_horizon_mode",
],
)
self.assertNotIn("optional", ltx)
self.assertEqual(list(ltx["optional"]), ["camera_capability"])
camera_choices, camera_config = ltx["optional"]["camera_capability"]
self.assertEqual(
camera_choices,
["Stable / base model", "Advanced / controlled camera"],
)
self.assertEqual(camera_config["default"], "Stable / base model")
h3 = self.nodes.DiffusionGemmaMiniMaxH3TargetProfile.INPUT_TYPES()
self.assertEqual(
@@ -67,7 +74,12 @@ class ModelTargetProfileNodeTests(unittest.TestCase):
"negative_prompt_guidance",
],
)
self.assertNotIn("optional", h3)
self.assertEqual(list(h3["optional"]), ["shot_count_override"])
override_type, override_config = h3["optional"]["shot_count_override"]
self.assertEqual(override_type, "INT")
self.assertEqual(override_config["default"], 0)
self.assertIn("native [Shot N]", override_config["tooltip"])
self.assertIn("never connect a Project Master generation-lane count", override_config["tooltip"])
ideogram = self.nodes.DiffusionGemmaIdeogram4TargetProfile.INPUT_TYPES()
self.assertEqual(
@@ -114,6 +126,7 @@ class ModelTargetProfileNodeTests(unittest.TestCase):
self.assertEqual(json.loads(payload_json), json.loads(legacy_json))
self.assertEqual(config.target_profile, "ltx")
self.assertEqual(config.ltx_generation_mode, "image_to_video")
self.assertEqual(config.ltx_camera_capability, "stable")
def test_h3_node_preserves_audio_shot_dialogue_and_negative_controls(self) -> None:
config, payload_json = self.nodes.DiffusionGemmaMiniMaxH3TargetProfile().build(
@@ -137,6 +150,23 @@ class ModelTargetProfileNodeTests(unittest.TestCase):
self.assertEqual(config.minimax_h3_dialogue_line_count, 3)
self.assertEqual(payload["negative_prompt_guidance"], "no subtitles")
overridden, overridden_json = self.nodes.DiffusionGemmaMiniMaxH3TargetProfile().build(
generation_mode="ref2va",
target_duration_seconds=25.0,
audio_mode="auto_scene_audio",
audio_guidance="locked soundtrack",
shot_count="auto",
custom_shot_count=12,
dialogue_mode="off",
dialogue_line_count=2,
dialogue_guidance="",
negative_prompt_mode="auto",
negative_prompt_guidance="",
shot_count_override=2,
)
self.assertEqual(overridden.minimax_h3_shot_count, "2")
self.assertEqual(json.loads(overridden_json)["minimax_h3_shot_count"], "2")
with self.assertRaisesRegex(ValueError, "conflicts with audio_mode=visual_only"):
self.nodes.DiffusionGemmaMiniMaxH3TargetProfile().build(
generation_mode="t2va",
tests/test_music_video_production_workflow.py
+841
View File
@@ -0,0 +1,841 @@
from __future__ import annotations
import copy
import importlib.util
import json
import unittest
from pathlib import Path
REPO_ROOT = Path(__file__).resolve().parents[1]
MIGRATION_PATH = REPO_ROOT / "tools" / "migrate_music_video_production.py"
SPEC = importlib.util.spec_from_file_location("music_video_workflow_migration", MIGRATION_PATH)
if SPEC is None or SPEC.loader is None: # pragma: no cover - import machinery guard
raise RuntimeError(f"Cannot load migration module: {MIGRATION_PATH}")
MIGRATION = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(MIGRATION)
WORKFLOWS = (
Path(r"C:\ComfyUI\app\user\default\workflows\14_ltx25_i2v+AUDIO_IN_PROCESS_SYNC.json"),
Path(r"C:\Users\danrh\Desktop\14_ltx25_i2v+AUDIO_IN_PROCESS_SYNC.json"),
)
APP_WORKFLOW = WORKFLOWS[0]
ORGANIZED_WORKFLOW = Path(
r"C:\Users\danrh\Desktop\14_ltx25_i2v+AUDIO_IN_PROCESS_SYNC_organized.json"
)
def _node(workflow: dict, node_id: int) -> dict:
return next(node for node in workflow["nodes"] if int(node["id"]) == node_id)
def _first_widget(node: dict):
value = node.get("widgets_values")
return value[0] if isinstance(value, list) else value
def _all_nodes(workflow: dict):
yield ("main", None), workflow["nodes"]
for subgraph in workflow.get("definitions", {}).get("subgraphs", []):
yield ("subgraph", str(subgraph["id"])), subgraph.get("nodes", [])
def _candidate_encoders(workflow: dict) -> list[dict]:
return [
node
for _scope, nodes in _all_nodes(workflow)
for node in nodes
if node.get("type") == "TextEncodeAceStepAudio1.5"
]
def _topology_signature(workflow: dict) -> dict:
return {
"main_nodes": [
{
"id": node["id"],
"type": node["type"],
"inputs": [
(item.get("name"), item.get("type"), item.get("link"))
for item in node.get("inputs", [])
],
"outputs": [
(item.get("name"), item.get("type"), item.get("links"))
for item in node.get("outputs", [])
],
}
for node in workflow["nodes"]
],
"main_links": workflow["links"],
"subgraphs": [
{
"id": subgraph["id"],
"inputs": subgraph.get("inputs", []),
"outputs": subgraph.get("outputs", []),
"nodes": [
{
"id": node["id"],
"type": node["type"],
"inputs": [
(item.get("name"), item.get("type"), item.get("link"))
for item in node.get("inputs", [])
],
"outputs": [
(item.get("name"), item.get("type"), item.get("links"))
for item in node.get("outputs", [])
],
}
for node in subgraph.get("nodes", [])
],
"links": subgraph.get("links", []),
}
for subgraph in workflow.get("definitions", {}).get("subgraphs", [])
],
}
def _non_model_widget_signature(workflow: dict) -> dict:
signature = {}
for scope, nodes in _all_nodes(workflow):
for node in nodes:
if int(node["id"]) in {116, 510, 559}:
continue
if node.get("type") == MIGRATION.PERFORMANCE_NODE_TYPE:
continue
if str(node.get("title", "")) == MIGRATION.PERFORMANCE_PREVIEW_TITLE:
continue
signature[(scope, int(node["id"]))] = copy.deepcopy(
node.get("widgets_values")
)
return signature
@unittest.skipUnless(all(path.is_file() for path in WORKFLOWS), "local synchronized workflow copies are absent")
class MusicVideoProductionWorkflowTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.workflows = [json.loads(path.read_text(encoding="utf-8")) for path in WORKFLOWS]
def test_both_workflow_sources_migrate_and_pass_graph_and_production_invariants(self) -> None:
for path, workflow in zip(WORKFLOWS, self.workflows):
with self.subTest(path=str(path)):
migrated = MIGRATION.migrate_workflow(workflow)
MIGRATION.validate_workflow(migrated)
marker = migrated["extra"][MIGRATION.MIGRATION_SCHEMA]
self.assertEqual(marker["version"], MIGRATION.MIGRATION_VERSION)
self.assertEqual(marker["production_concept"], "Audition and select")
self.assertEqual(marker["audition_candidate_count"], 2)
self.assertEqual(marker["selector_policy"]["revision"], 4)
self.assertEqual(
marker["selector_policy"]["aligned_double_time"],
"mandatory_alignment_pressure_vocal_plus_two_of_four_quality_signals",
)
self.assertEqual(
marker["selector_policy"]["double_time"],
"hard_failure_outside_evidence_bundle",
)
self.assertEqual(
marker["selector_policy"]["supporting_signals_required"], 2
)
def test_performance_modes_append_natural_without_reindexing_legacy_choices(self) -> None:
self.assertEqual(
MIGRATION.PERFORMANCE_MODES,
(
"Dance / music sync",
"Lyrics + lip sync",
"Natural / audio-led sync",
),
)
self.assertEqual(
MIGRATION.DEFAULT_PERFORMANCE_MODE, "Natural / audio-led sync"
)
def test_user_values_remain_independent_between_the_two_copies(self) -> None:
app, desktop = self.workflows
self.assertEqual(_first_widget(_node(app, 178)), 20)
self.assertEqual(_first_widget(_node(desktop, 178)), 25)
self.assertEqual(_node(app, 569)["widgets_values"][0], 44)
self.assertEqual(_node(desktop, 569)["widgets_values"][0], 30)
self.assertEqual(_node(app, 418)["widgets_values"][0], 347267244880141)
self.assertEqual(_node(desktop, 418)["widgets_values"][0], 347267244880141)
self.assertEqual(
app["extra"][MIGRATION.MIGRATION_SCHEMA]["legacy_router_mode"],
"stabilized",
)
self.assertEqual(
desktop["extra"][MIGRATION.MIGRATION_SCHEMA]["legacy_router_mode"],
"default",
)
def test_migration_is_idempotent(self) -> None:
for path, workflow in zip(WORKFLOWS, self.workflows):
with self.subTest(path=str(path)):
migrated = MIGRATION.migrate_workflow(workflow)
self.assertEqual(MIGRATION.migrate_workflow(migrated), migrated)
def test_v7_to_current_upgrade_preserves_controls_and_both_ace_topologies(self) -> None:
outer_encoder_counts = []
for path, current in zip(WORKFLOWS, self.workflows):
with self.subTest(path=str(path)):
legacy = copy.deepcopy(current)
marker = legacy["extra"][MIGRATION.MIGRATION_SCHEMA]
marker["version"] = 7
canonical = marker["canonical_ace"]
canonical["second_language_model"] = "qwen_1.7b_ace15.safetensors"
canonical["ace_4b_upgrade"] = (
"not installed; requires qwen_4b_ace15.safetensors"
)
canonical.pop("second_language_model_sha256", None)
canonical.pop("second_language_model_parameter_class", None)
canonical.pop("second_language_model_role", None)
canonical.pop("second_language_model_url", None)
loader = next(
node for node in legacy["nodes"] if node["type"] == "DualCLIPLoader"
)
loader["widgets_values"][1] = "qwen_1.7b_ace15.safetensors"
loader["properties"]["models"][1].update(
{
"name": "qwen_1.7b_ace15.safetensors",
"url": (
"https://huggingface.co/Comfy-Org/ace_step_1.5_ComfyUI_files/"
"resolve/main/split_files/text_encoders/"
"qwen_1.7b_ace15.safetensors"
),
"directory": "text_encoders",
}
)
for encode in _candidate_encoders(legacy):
encode["title"] = str(encode["title"]).replace(
"ACE 4B LM", "ACE 1.7B LM"
)
subgraph_topology_before = _topology_signature(legacy)["subgraphs"]
widgets_before = _non_model_widget_signature(legacy)
migrated = MIGRATION.migrate_workflow(legacy)
MIGRATION.validate_workflow(migrated)
self.assertEqual(
_topology_signature(migrated)["subgraphs"],
subgraph_topology_before,
)
self.assertEqual(
_non_model_widget_signature(migrated),
widgets_before,
)
outer_encoder_counts.append(
sum(
node.get("type") == "TextEncodeAceStepAudio1.5"
for node in migrated["nodes"]
)
)
self.assertEqual(outer_encoder_counts, [0, 4])
def test_official_ace_4b_loader_metadata_and_labels_are_persisted(self) -> None:
for path, workflow in zip(WORKFLOWS, self.workflows):
with self.subTest(path=str(path)):
loader = next(
node for node in workflow["nodes"] if node["type"] == "DualCLIPLoader"
)
self.assertEqual(
loader["widgets_values"][1], MIGRATION.ACE_4B_LANGUAGE_MODEL
)
second_asset = loader["properties"]["models"][1]
self.assertEqual(second_asset["name"], MIGRATION.ACE_4B_LANGUAGE_MODEL)
self.assertEqual(second_asset["url"], MIGRATION.ACE_4B_LANGUAGE_MODEL_URL)
self.assertEqual(second_asset["directory"], "text_encoders")
encoders = _candidate_encoders(workflow)
self.assertEqual(len(encoders), 4)
self.assertTrue(
all("ACE 4B LM" in str(node.get("title", "")) for node in encoders)
)
canonical = workflow["extra"][MIGRATION.MIGRATION_SCHEMA]["canonical_ace"]
for key, expected in MIGRATION.ACE_4B_METADATA_MARKER.items():
self.assertEqual(canonical[key], expected)
self.assertNotIn("ace_4b_upgrade", canonical)
note_copy = " ".join(
str(value) for value in _node(workflow, 116)["widgets_values"]
)
self.assertIn("official ACE-specific **4B language model**", note_copy)
self.assertIn(MIGRATION.ACE_4B_LANGUAGE_MODEL, note_copy)
self.assertIn("juxtaposition is not musical fusion", note_copy)
@unittest.skipUnless(ORGANIZED_WORKFLOW.is_file(), "organized delivery is absent")
def test_attached_organized_delivery_is_already_persisted_at_current_revision(self) -> None:
organized = json.loads(ORGANIZED_WORKFLOW.read_text(encoding="utf-8"))
self.assertEqual(MIGRATION.migrate_workflow(organized), organized)
MIGRATION.validate_workflow(organized)
performance = next(
node
for node in organized["nodes"]
if node.get("type") == MIGRATION.PERFORMANCE_NODE_TYPE
)
self.assertEqual(
[item.get("name") for item in performance.get("inputs", [])],
[
"ltx_prompt",
"lyrics",
"song_duration_seconds",
"excerpt_start_seconds",
"excerpt_duration_seconds",
],
)
self.assertNotIn(
"performance_mode",
{item.get("name") for item in performance.get("inputs", [])},
)
def test_v9_to_v10_adds_only_the_stable_camera_widget(self) -> None:
for path, workflow in zip(WORKFLOWS, self.workflows):
with self.subTest(path=str(path)):
legacy = copy.deepcopy(MIGRATION.migrate_workflow(workflow))
marker = legacy["extra"][MIGRATION.MIGRATION_SCHEMA]
marker["version"] = 9
marker.pop("ltx_camera_capability", None)
target_before = copy.deepcopy(_node(legacy, 510))
if (
target_before.get("widgets_values")
and target_before["widgets_values"][-1]
in MIGRATION.CAMERA_CAPABILITIES
):
legacy_target = _node(legacy, 510)
legacy_target["widgets_values"] = legacy_target["widgets_values"][:-1]
target_before = copy.deepcopy(legacy_target)
topology_before = _topology_signature(legacy)
migrated = MIGRATION.migrate_workflow(legacy)
MIGRATION.validate_workflow(migrated)
target_after = _node(migrated, 510)
self.assertEqual(_topology_signature(migrated), topology_before)
self.assertEqual(target_after["pos"], target_before["pos"])
self.assertEqual(target_after["inputs"], target_before["inputs"])
self.assertEqual(
target_after["widgets_values"],
target_before["widgets_values"]
+ [MIGRATION.DEFAULT_CAMERA_CAPABILITY],
)
self.assertNotIn(
MIGRATION.CAMERA_CAPABILITY_INPUT,
{item.get("name") for item in target_after.get("inputs", [])},
)
self.assertEqual(
migrated["extra"][MIGRATION.MIGRATION_SCHEMA][
"ltx_camera_capability"
],
MIGRATION.CAMERA_CAPABILITY_MARKER,
)
self.assertEqual(MIGRATION.migrate_workflow(migrated), migrated)
def test_current_migration_preserves_advanced_camera_selection(self) -> None:
for path, workflow in zip(WORKFLOWS, self.workflows):
with self.subTest(path=str(path)):
edited = MIGRATION.migrate_workflow(workflow)
_node(edited, 510)["widgets_values"][-1] = (
"Advanced / controlled camera"
)
migrated = MIGRATION.migrate_workflow(edited)
self.assertEqual(
_node(migrated, 510)["widgets_values"][-1],
"Advanced / controlled camera",
)
def test_planner_uses_the_fixed_ace_song_duration_in_both_copies(self) -> None:
for path, workflow in zip(WORKFLOWS, self.workflows):
with self.subTest(path=str(path)):
planner = _node(workflow, 578)
duration_slot = next(
index
for index, item in enumerate(planner["inputs"])
if item["name"] == "duration_seconds"
)
link_id = planner["inputs"][duration_slot]["link"]
link = next(link for link in workflow["links"] if int(link[0]) == int(link_id))
self.assertEqual(int(link[1]), 562)
self.assertEqual(int(link[2]), 0)
self.assertEqual(int(link[3]), 578)
self.assertEqual(int(link[4]), duration_slot)
def test_ace_auraflow_shift_is_persisted_as_a_widget_list(self) -> None:
for path, workflow in zip(WORKFLOWS, self.workflows):
with self.subTest(path=str(path)):
self.assertEqual(_node(workflow, 563)["widgets_values"], [3.0])
def test_visible_qc_help_matches_the_selector_policy(self) -> None:
for path, workflow in zip(WORKFLOWS, self.workflows):
with self.subTest(path=str(path)):
note = _node(workflow, 116)
note_copy = " ".join(str(value) for value in note["widgets_values"])
self.assertIn(
"at least two of visual-recovery coverage",
note_copy,
)
audit = next(
node
for node in workflow["nodes"]
if str(node.get("title", "")).startswith("FULL MUSIC QC AUDIT")
)
self.assertIn("advisories", audit["title"])
def test_disabled_missing_node_experiment_is_absent(self) -> None:
for path, workflow in zip(WORKFLOWS, self.workflows):
with self.subTest(path=str(path)):
workflow = MIGRATION.migrate_workflow(workflow)
subgraph = next(
item
for item in workflow["definitions"]["subgraphs"]
if item["id"] == MIGRATION.LTX_SUBGRAPH_ID
)
self.assertNotIn("Film Grain", {node["type"] for node in subgraph["nodes"]})
self.assertNotIn(
"lut test (disabled)",
{str(group.get("title", "")).strip().casefold() for group in subgraph.get("groups", [])},
)
def test_a2v_is_neutral_by_default_and_final_audio_is_distinct(self) -> None:
for workflow in self.workflows:
outer = next(
node for node in workflow["nodes"] if node["type"] == MIGRATION.LTX_SUBGRAPH_ID
)
self.assertEqual(outer["widgets_values"][-1], 1.0)
self.assertNotEqual(
next(item for item in outer["inputs"] if item["name"] == "conditioning_audio")["link"],
next(item for item in outer["inputs"] if item["name"] == "final_soundtrack")["link"],
)
subgraph = next(
item
for item in workflow["definitions"]["subgraphs"]
if item["id"] == MIGRATION.LTX_SUBGRAPH_ID
)
self.assertEqual(subgraph["inputs"][15]["linkIds"], [985])
self.assertEqual(subgraph["inputs"][16]["linkIds"], [967])
self.assertEqual(len(subgraph["inputs"][17]["linkIds"]), 2)
def test_performance_toggle_is_post_gate_and_does_not_touch_audio_paths(self) -> None:
for path, workflow in zip(WORKFLOWS, self.workflows):
with self.subTest(path=str(path)):
workflow = MIGRATION.migrate_workflow(workflow)
controls = [
node
for node in workflow["nodes"]
if node.get("type") == MIGRATION.PERFORMANCE_NODE_TYPE
]
self.assertEqual(len(controls), 1)
control = controls[0]
self.assertIn(
control["widgets_values"][0], MIGRATION.PERFORMANCE_MODES
)
self.assertEqual(
control["widgets_values"][1:],
MIGRATION.PERFORMANCE_WIDGET_DEFAULTS[1:],
)
marker = workflow["extra"][MIGRATION.MIGRATION_SCHEMA][
"ltx_performance_mode"
]
self.assertEqual(marker["modes"], list(MIGRATION.PERFORMANCE_MODES))
self.assertEqual(marker["default"], MIGRATION.DEFAULT_PERFORMANCE_MODE)
self.assertIn("lyrics are not a timing schedule", marker["lyrics_policy"])
nodes = {int(node["id"]): node for node in workflow["nodes"]}
links = {int(link[0]): link for link in workflow["links"]}
def origin(input_name: str, node: dict = control):
item = next(
value for value in node["inputs"] if value["name"] == input_name
)
link = links[int(item["link"])]
return nodes[int(link[1])], int(link[2])
selector = next(
node
for node in workflow["nodes"]
if node.get("type") == "DiffusionGemmaAudioCandidateSelector"
)
self.assertEqual(origin("ltx_prompt"), (_node(workflow, 188), 0))
self.assertEqual(origin("lyrics"), (_node(workflow, 556), 4))
self.assertEqual(origin("song_duration_seconds"), (_node(workflow, 562), 0))
self.assertEqual(origin("excerpt_start_seconds"), (selector, 1))
self.assertEqual(origin("excerpt_duration_seconds"), (_node(workflow, 178), 0))
outer = next(
node
for node in workflow["nodes"]
if node.get("type") == MIGRATION.LTX_SUBGRAPH_ID
)
self.assertEqual(origin("value", outer), (control, 0))
self.assertFalse(
any(
int(link[1]) == 188 and int(link[3]) == int(outer["id"])
for link in workflow["links"]
)
)
guide = next(
node
for node in workflow["nodes"]
if node.get("type") == "DiffusionGemmaLTXAudioGuide"
)
self.assertEqual(origin("conditioning_audio", outer), (guide, 0))
self.assertEqual(origin("final_soundtrack", outer), (guide, 1))
previews = [
node
for node in workflow["nodes"]
if str(node.get("title", ""))
== MIGRATION.PERFORMANCE_PREVIEW_TITLE
]
self.assertEqual(len(previews), 1)
self.assertEqual(origin("source", previews[0]), (control, 1))
target = _node(workflow, 510)
self.assertEqual(
target["widgets_values"][4],
MIGRATION.NEUTRAL_PERFORMANCE_GUIDANCE,
)
def test_current_migration_preserves_a_user_selected_lip_sync_mode(self) -> None:
for path, workflow in zip(WORKFLOWS, self.workflows):
with self.subTest(path=str(path)):
edited = copy.deepcopy(workflow)
control = next(
node
for node in edited["nodes"]
if node.get("type") == MIGRATION.PERFORMANCE_NODE_TYPE
)
control["widgets_values"][0] = "Lyrics + lip sync"
migrated = MIGRATION.migrate_workflow(edited)
selected = next(
node
for node in migrated["nodes"]
if node.get("type") == MIGRATION.PERFORMANCE_NODE_TYPE
)
self.assertEqual(selected["widgets_values"][0], "Lyrics + lip sync")
self.assertEqual(
selected["widgets_values"][1:],
MIGRATION.PERFORMANCE_WIDGET_DEFAULTS[1:],
)
def test_v8_to_v9_adds_only_the_performance_seam_and_neutralizes_known_guidance(self) -> None:
for path, workflow in zip(WORKFLOWS, self.workflows):
with self.subTest(path=str(path)):
legacy = copy.deepcopy(MIGRATION.migrate_workflow(workflow))
control = next(
node
for node in legacy["nodes"]
if node.get("type") == MIGRATION.PERFORMANCE_NODE_TYPE
)
preview = next(
node
for node in legacy["nodes"]
if str(node.get("title", ""))
== MIGRATION.PERFORMANCE_PREVIEW_TITLE
)
removed_node_ids = {int(control["id"]), int(preview["id"])}
removed_link_ids = {
int(link[0])
for link in legacy["links"]
if int(link[1]) in removed_node_ids or int(link[3]) in removed_node_ids
}
legacy["nodes"] = [
node
for node in legacy["nodes"]
if int(node["id"]) not in removed_node_ids
]
legacy["links"] = [
link
for link in legacy["links"]
if int(link[0]) not in removed_link_ids
]
for node in legacy["nodes"]:
for item in node.get("inputs", []):
if item.get("link") in removed_link_ids:
item["link"] = None
for output in node.get("outputs", []):
if isinstance(output.get("links"), list):
output["links"] = [
link_id
for link_id in output["links"]
if int(link_id) not in removed_link_ids
]
gate = _node(legacy, 188)
outer = next(
node
for node in legacy["nodes"]
if node.get("type") == MIGRATION.LTX_SUBGRAPH_ID
)
direct_link_id = int(legacy["last_link_id"]) + 1
gate_slot = next(
index
for index, item in enumerate(gate["outputs"])
if item["name"] == "prompt"
)
outer_slot = next(
index
for index, item in enumerate(outer["inputs"])
if item["name"] == "value"
)
legacy["links"].append(
[
direct_link_id,
int(gate["id"]),
gate_slot,
int(outer["id"]),
outer_slot,
"STRING",
]
)
gate["outputs"][gate_slot].setdefault("links", []).append(
direct_link_id
)
outer["inputs"][outer_slot]["link"] = direct_link_id
legacy["last_link_id"] = direct_link_id
marker = legacy["extra"][MIGRATION.MIGRATION_SCHEMA]
marker["version"] = 8
marker.pop("ltx_performance_mode", None)
_node(legacy, 510)["widgets_values"][4] = (
MIGRATION.LEGACY_FORCED_LIP_SYNC_GUIDANCE
)
widgets_before = _non_model_widget_signature(legacy)
migrated = MIGRATION.migrate_workflow(legacy)
MIGRATION.validate_workflow(migrated)
self.assertEqual(
_non_model_widget_signature(migrated), widgets_before
)
self.assertEqual(
_node(migrated, 510)["widgets_values"][4],
MIGRATION.NEUTRAL_PERFORMANCE_GUIDANCE,
)
inserted = next(
node
for node in migrated["nodes"]
if node.get("type") == MIGRATION.PERFORMANCE_NODE_TYPE
)
self.assertEqual(
inserted["widgets_values"][0], "Natural / audio-led sync"
)
@unittest.skipUnless(APP_WORKFLOW.is_file(), "app workflow is absent")
class AppCameraCapabilityWorkflowTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.workflow = json.loads(APP_WORKFLOW.read_text(encoding="utf-8"))
def test_app_delivery_is_current_secure_and_idempotent(self) -> None:
migrated = MIGRATION.migrate_workflow(self.workflow)
MIGRATION.validate_workflow(migrated)
self.assertEqual(MIGRATION.migrate_workflow(migrated), migrated)
self.assertEqual(MIGRATION._sensitive_key_paths(migrated), [])
marker = migrated["extra"][MIGRATION.MIGRATION_SCHEMA]["ltx_performance_mode"]
self.assertEqual(marker["default"], "Natural / audio-led sync")
self.assertEqual(marker["modes"], list(MIGRATION.PERFORMANCE_MODES))
target = _node(migrated, 510)
self.assertEqual(len(target["widgets_values"]), 9)
self.assertEqual(
target["widgets_values"][-1], MIGRATION.DEFAULT_CAMERA_CAPABILITY
)
def test_v9_app_serialization_migrates_without_state_or_topology_changes(self) -> None:
legacy = copy.deepcopy(MIGRATION.migrate_workflow(self.workflow))
marker = legacy["extra"][MIGRATION.MIGRATION_SCHEMA]
marker["version"] = 9
marker.pop("ltx_camera_capability", None)
target = _node(legacy, 510)
if target["widgets_values"][-1] in MIGRATION.CAMERA_CAPABILITIES:
target["widgets_values"] = target["widgets_values"][:-1]
performance_before = copy.deepcopy(
next(
node
for node in legacy["nodes"]
if node.get("type") == MIGRATION.PERFORMANCE_NODE_TYPE
)
)
self.assertEqual(
[item.get("name") for item in performance_before.get("inputs", [])],
[
"ltx_prompt",
"lyrics",
"performance_mode",
"song_duration_seconds",
"excerpt_start_seconds",
"excerpt_duration_seconds",
],
)
topology_before = _topology_signature(legacy)
groups_before = copy.deepcopy(legacy.get("groups", []))
positions_before = {
(scope, int(node["id"])): copy.deepcopy(node.get("pos"))
for scope, nodes in _all_nodes(legacy)
for node in nodes
}
widgets_before = {
(scope, int(node["id"])): copy.deepcopy(node.get("widgets_values"))
for scope, nodes in _all_nodes(legacy)
for node in nodes
if int(node["id"]) != 510
}
target_values_before = copy.deepcopy(target["widgets_values"])
migrated = MIGRATION.migrate_workflow(legacy)
MIGRATION.validate_workflow(migrated)
self.assertEqual(_topology_signature(migrated), topology_before)
self.assertEqual(migrated.get("groups", []), groups_before)
self.assertEqual(
{
(scope, int(node["id"])): node.get("pos")
for scope, nodes in _all_nodes(migrated)
for node in nodes
},
positions_before,
)
self.assertEqual(
{
(scope, int(node["id"])): node.get("widgets_values")
for scope, nodes in _all_nodes(migrated)
for node in nodes
if int(node["id"]) != 510
},
widgets_before,
)
self.assertEqual(
next(
node
for node in migrated["nodes"]
if node.get("type") == MIGRATION.PERFORMANCE_NODE_TYPE
),
performance_before,
)
migrated_target = _node(migrated, 510)
self.assertEqual(
migrated_target["widgets_values"],
target_values_before + [MIGRATION.DEFAULT_CAMERA_CAPABILITY],
)
self.assertEqual(MIGRATION.migrate_workflow(migrated), migrated)
@unittest.skipUnless(ORGANIZED_WORKFLOW.is_file(), "organized delivery is absent")
class OrganizedCameraCapabilityWorkflowTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.workflow = json.loads(ORGANIZED_WORKFLOW.read_text(encoding="utf-8"))
cls.migrated = MIGRATION.migrate_workflow(cls.workflow)
def test_delivery_is_current_secure_and_idempotent(self) -> None:
MIGRATION.validate_workflow(self.migrated)
self.assertEqual(MIGRATION.migrate_workflow(self.migrated), self.migrated)
self.assertEqual(MIGRATION._sensitive_key_paths(self.migrated), [])
target = _node(self.migrated, 510)
self.assertEqual(
target["widgets_values"][-1], MIGRATION.DEFAULT_CAMERA_CAPABILITY
)
self.assertEqual(len(target["widgets_values"]), 9)
self.assertNotIn(
MIGRATION.CAMERA_CAPABILITY_INPUT,
{item.get("name") for item in target.get("inputs", [])},
)
marker = self.migrated["extra"][MIGRATION.MIGRATION_SCHEMA]
self.assertEqual(marker["version"], MIGRATION.MIGRATION_VERSION)
self.assertEqual(
marker["ltx_camera_capability"], MIGRATION.CAMERA_CAPABILITY_MARKER
)
def test_ui_resaved_performance_shape_is_preserved(self) -> None:
performance = next(
node
for node in self.migrated["nodes"]
if node.get("type") == MIGRATION.PERFORMANCE_NODE_TYPE
)
self.assertEqual(
[item.get("name") for item in performance.get("inputs", [])],
[
"ltx_prompt",
"lyrics",
"song_duration_seconds",
"excerpt_start_seconds",
"excerpt_duration_seconds",
],
)
self.assertIn(performance["widgets_values"][0], MIGRATION.PERFORMANCE_MODES)
self.assertEqual(
performance["widgets_values"][1:],
MIGRATION.PERFORMANCE_WIDGET_DEFAULTS[1:],
)
def test_v9_upgrade_changes_no_topology_or_protected_ltx_internals(self) -> None:
legacy = copy.deepcopy(self.migrated)
legacy_marker = legacy["extra"][MIGRATION.MIGRATION_SCHEMA]
legacy_marker["version"] = 9
legacy_marker.pop("ltx_camera_capability", None)
target = _node(legacy, 510)
target["widgets_values"] = target["widgets_values"][:-1]
topology_before = _topology_signature(legacy)
performance_before = copy.deepcopy(
next(
node
for node in legacy["nodes"]
if node.get("type") == MIGRATION.PERFORMANCE_NODE_TYPE
)
)
ltx_before = next(
item
for item in legacy["definitions"]["subgraphs"]
if item["id"] == MIGRATION.LTX_SUBGRAPH_ID
)
protected_nodes_before = copy.deepcopy(
[node for node in ltx_before["nodes"] if int(node["id"]) in {625, 626}]
)
protected_links_before = copy.deepcopy(
[
link
for link in ltx_before["links"]
if int(link["id"]) in {1138, 1139, 1140, 1141, 1142}
]
)
migrated = MIGRATION.migrate_workflow(legacy)
MIGRATION.validate_workflow(migrated)
self.assertEqual(_topology_signature(migrated), topology_before)
self.assertEqual(
next(
node
for node in migrated["nodes"]
if node.get("type") == MIGRATION.PERFORMANCE_NODE_TYPE
),
performance_before,
)
ltx_after = next(
item
for item in migrated["definitions"]["subgraphs"]
if item["id"] == MIGRATION.LTX_SUBGRAPH_ID
)
self.assertEqual(
[node for node in ltx_after["nodes"] if int(node["id"]) in {625, 626}],
protected_nodes_before,
)
self.assertEqual(
[
link
for link in ltx_after["links"]
if int(link["id"]) in {1138, 1139, 1140, 1141, 1142}
],
protected_links_before,
)
def test_advanced_camera_selection_survives_repeat_migration(self) -> None:
edited = copy.deepcopy(self.migrated)
_node(edited, 510)["widgets_values"][-1] = (
"Advanced / controlled camera"
)
migrated = MIGRATION.migrate_workflow(edited)
self.assertEqual(
_node(migrated, 510)["widgets_values"][-1],
"Advanced / controlled camera",
)
if __name__ == "__main__":
unittest.main()
tests/test_production_planning_nodes.py
+1875
View File
File diff suppressed because it is too large Load Diff
tests/test_splatstage_planner.py
+527 -8
View File
@@ -6,6 +6,9 @@ import sys
import unittest
import uuid
from pathlib import Path
from unittest import mock
import torch
ROOT = Path(__file__).resolve().parents[1]
@@ -75,11 +78,23 @@ class SplatStagePlannerMusicTests(unittest.TestCase):
self.assertIn("two separately generated performer clips", prompt)
self.assertIn("one performer object, one background object, and one FX object", prompt)
self.assertIn("never arrays or clip-specific lists", prompt)
self.assertIn("16 to 36 non-empty lines", prompt)
self.assertIn("target 10 to 14 sung lines", prompt)
self.assertIn("6 to 10 syllables per line", prompt)
self.assertIn("about two bars per sung line", prompt)
self.assertIn("Repeat the Chorus wording verbatim", prompt)
self.assertIn("[Instrumental] intro, interlude, or outro", prompt)
self.assertIn("dedicated metadata", prompt)
self.assertIn("ordinary caption language", prompt)
self.assertIn("Source passthrough mode is active", prompt)
self.assertIn("reach ACE-Step unchanged", prompt)
self.assertIn("When Context Hub image pixels are attached", prompt)
self.assertNotIn("Roman-numeral progression", prompt.split("Music production strategy:", 1)[0])
self.assertIn("SAM3 object-category bundle", prompt)
self.assertIn("wide-brimmed mariachi hat", prompt)
self.assertIn("electric guitar", prompt)
self.assertIn("positive_prompt must use affirmative visual language only", prompt)
self.assertIn("person, people, human, man, men, woman, women", prompt)
self.assertIn("silhouette, crowd, performer, presenter, dancer, or singer", prompt)
self.assertIn('"schema":"splatstage.show_blueprint"', prompt)
self.assertIn('"version":1', prompt)
self.assertIn(
@@ -87,13 +102,512 @@ class SplatStagePlannerMusicTests(unittest.TestCase):
prompt,
)
def test_planner_contract_supports_ten_or_twenty_seconds(self) -> None:
spec = self.nodes.DiffusionGemmaSplatStagePlanner.INPUT_TYPES()["required"][
"duration_seconds"
][1]
self.assertEqual(spec["min"], 10.0)
self.assertEqual(spec["max"], 20.0)
self.assertEqual(spec["default"], 20.0)
def test_planner_contract_preserves_input_order_and_allows_open_duration_and_aspect(self) -> None:
required = self.nodes.DiffusionGemmaSplatStagePlanner.INPUT_TYPES()["required"]
self.assertEqual(
list(required),
[
"model_config",
"gemma_context",
"root_seed",
"duration_seconds",
"aspect_ratio",
"temperature",
"max_new_tokens",
"production_mode",
],
)
duration_type, duration_spec = required["duration_seconds"]
self.assertEqual(duration_type, "FLOAT")
self.assertEqual(duration_spec["default"], 20.0)
self.assertGreater(duration_spec["min"], 0.0)
self.assertNotIn("max", duration_spec)
aspect_type, aspect_spec = required["aspect_ratio"]
self.assertEqual(aspect_type, "STRING")
self.assertEqual(aspect_spec["default"], "16:9")
production_choices, production_spec = required["production_mode"]
self.assertEqual(
production_choices,
["Source passthrough", "Joint video-safe plan", "Audition and select"],
)
self.assertEqual(production_spec["default"], "Source passthrough")
def test_production_modes_are_distinct_upstream_planning_contracts(self) -> None:
source = self.nodes._splatstage_blueprint_prompt(
"make a pop-music video",
42,
self.schema,
duration_seconds=20.0,
aspect_ratio="16:9",
production_mode="Source passthrough",
)
safe = self.nodes._splatstage_blueprint_prompt(
"make a pop-music video",
42,
self.schema,
duration_seconds=20.0,
aspect_ratio="16:9",
production_mode="Joint video-safe plan",
)
audition = self.nodes._splatstage_blueprint_prompt(
"make a pop-music video",
42,
self.schema,
duration_seconds=20.0,
aspect_ratio="16:9",
production_mode="Audition and select",
)
self.assertIn("reach ACE-Step unchanged", source)
self.assertIn("no double-time drum illusion", safe)
self.assertIn("stable tonal center", safe)
self.assertIn("several independent song seeds", audition)
self.assertIn("moderate and even transient density", audition)
self.assertIn("stable tonal family", audition)
for prompt in (source, safe, audition):
self.assertIn("Do not repeat a numeric BPM", prompt)
self.assertIn("6 to 10 syllables", prompt)
def test_explicit_music_genre_overrides_visual_setting_in_both_planners(self) -> None:
creative_brief = (
"Create a strict country soundtrack juxtaposed against a neon nightclub."
)
standard = self.nodes._splatstage_blueprint_prompt(
creative_brief,
42,
self.schema,
duration_seconds=20.0,
aspect_ratio="16:9",
visual_context="A singer dances beneath neon club lighting.",
)
efficient_schema = json.loads(
(
ROOT
/ "schemas"
/ "splatstage_efficient_show_blueprint.schema.json"
).read_text(encoding="utf-8")
)
efficient = self.nodes._splatstage_efficient_blueprint_prompt(
creative_brief,
42,
efficient_schema,
)
shared_rule = self.nodes._SPLATSTAGE_EXPLICIT_GENRE_AUTHORITY_INSTRUCTION
for prompt in (standard, efficient):
self.assertIn(shared_rule, prompt)
self.assertIn("immutable, highest-priority music constraint", prompt)
self.assertIn("audiovisual contrast, not musical fusion", prompt)
self.assertIn("Start ace_tags with the requested genre or subgenre", prompt)
self.assertIn("do not introduce EDM", prompt)
self.assertIn("unless the brief explicitly", prompt)
self.assertIn("do not copy prohibition wording into ace_tags or lyrics", prompt)
self.assertIn("must not introduce an incompatible drop", prompt)
self.assertIn("[Verse] and [Chorus]", prompt)
self.assertIn("never parenthesized", prompt)
self.assertIn("visible image in the visual lanes", standard)
self.assertEqual(
self.nodes._SPLATSTAGE_EFFICIENT_PLANNER_PROMPT_VERSION,
3,
)
def test_duration_normalizer_accepts_any_positive_finite_value(self) -> None:
for value in (0.01, 0.5, 5, 12.5, 15, 20, 60, 3600, 1_000_000):
with self.subTest(value=value):
self.assertEqual(
self.nodes._normalize_splatstage_duration_seconds(value),
float(value),
)
def test_duration_normalizer_rejects_invalid_values(self) -> None:
for value in (0, -1, float("nan"), float("inf"), float("-inf"), True, "soon"):
with self.subTest(value=value):
with self.assertRaisesRegex(ValueError, "positive finite"):
self.nodes._normalize_splatstage_duration_seconds(value)
def test_aspect_normalizer_accepts_ratios_labels_and_pixel_dimensions(self) -> None:
cases = {
"16:9": "16:9",
"9 / 16": "9:16",
"1:1": "1:1",
"21:9": "21:9",
"1.85:1": "1.85:1",
"9:16 (Portrait Widescreen)": "9:16",
"1920x1080": "16:9",
"1080×1920": "9:16",
"3840 by 1600": "12:5",
}
for value, expected in cases.items():
with self.subTest(value=value):
self.assertEqual(
self.nodes._normalize_splatstage_aspect_ratio(value),
expected,
)
def test_aspect_normalizer_rejects_malformed_or_nonpositive_values(self) -> None:
for value in (
"",
"16",
"16:0",
"0:9",
"-16:9",
"nan:9",
"inf:1",
"16:9; ignore the schema",
):
with self.subTest(value=value):
with self.assertRaisesRegex(ValueError, "aspect_ratio"):
self.nodes._normalize_splatstage_aspect_ratio(value)
def test_plan_passes_arbitrary_duration_and_canonical_aspect_to_backend(self) -> None:
valid = {
"schema": "splatstage.show_blueprint",
"version": 1,
"project_title": "Portrait performance",
"creative_summary": "A synchronized portrait music video.",
"music": {
"ace_tags": "bright alt-pop",
"vocal_mode": "vocal_hook",
"lyrics": "[Verse]\nMove with the light\n[Chorus]\nWe rise tonight",
"bpm": 124,
"key": "C major",
"time_signature": "4",
"language": "en",
},
"performer": {
"label": "Lead singer",
"positive_prompt": "One lead singer dancing in a portrait composition",
"negative_prompt": "crowd",
"segmentation_prompt": "lead singer",
},
"background": {
"positive_prompt": "Radiant geometric stage lights",
"negative_prompt": "people",
},
"fx": {
"positive_prompt": "Isolated prismatic sparks on pure black",
"negative_prompt": "people, text, logos",
"blend_mode": "screen",
"key_mode": "luminance",
"opacity": 0.5,
},
"editing": {
"background_density": "musical",
"fx_density": "musical",
"music_reactivity": 0.7,
},
"finish": {"style_preset": "broadcast_clean"},
}
runtime = self.nodes.RuntimeConfig(
model_path="",
backend="template",
dtype="auto",
quantization="none",
local_files_only=True,
unload_policy="keep_loaded",
max_memory_gb=1.0,
)
context = self.nodes.GemmaContext(user_prompt="Make a portrait music video")
with mock.patch.object(
self.nodes,
"_run_backend",
return_value=json.dumps(valid),
) as backend:
result = self.nodes.DiffusionGemmaSplatStagePlanner().plan(
runtime,
context,
42,
60.0,
"1080x1920",
0.35,
2048,
)
model_prompt = backend.call_args.args[1]
self.assertIn("duration 60 seconds, aspect ratio 9:16", model_prompt)
self.assertEqual(result["result"][1], True)
raw_record = json.loads(result["result"][3])
self.assertEqual(
raw_record["production_context"],
{
"duration_seconds": 60.0,
"aspect_ratio": "9:16",
"production_mode": "source_passthrough",
"pixels_sent_to_backend": False,
"visual_context_present": False,
},
)
def test_unparseable_initial_response_retries_the_full_schema_task(self) -> None:
valid = {
"schema": "splatstage.show_blueprint",
"version": 1,
"project_title": "Country contrast",
"creative_summary": "A country performance against neon club visuals.",
"music": {
"ace_tags": "traditional country, acoustic guitar, pedal steel, fiddle",
"vocal_mode": "vocal_hook",
"lyrics": "[Verse]\nDust turns gold tonight\n[Chorus]\nTake the long road home",
"bpm": 92,
"key": "G major",
"time_signature": "4",
"language": "en",
},
"performer": {
"label": "Country singer",
"positive_prompt": "One country singer performing beneath magenta light",
"negative_prompt": "crowd, duplicate person, text",
"segmentation_prompt": "country singer, acoustic guitar",
},
"background": {
"positive_prompt": "Empty neon nightclub interior with cobalt haze",
"negative_prompt": "people, performers, crowd, text",
},
"fx": {
"positive_prompt": "Isolated amber dust ribbons on pure black",
"negative_prompt": "people, faces, text, logos",
"blend_mode": "screen",
"key_mode": "luminance",
"opacity": 0.5,
},
"editing": {
"background_density": "musical",
"fx_density": "calm",
"music_reactivity": 0.5,
},
"finish": {"style_preset": "broadcast_clean"},
}
runtime = self.nodes.RuntimeConfig(
model_path="",
backend="template",
dtype="auto",
quantization="none",
local_files_only=True,
unload_policy="keep_loaded",
max_memory_gb=1.0,
)
context = self.nodes.GemmaContext(
user_prompt=(
"Create a strict country soundtrack juxtaposed against a neon nightclub."
)
)
with mock.patch.object(
self.nodes,
"_run_backend",
side_effect=["I cannot provide that blueprint.", json.dumps(valid)],
) as backend:
result = self.nodes.DiffusionGemmaSplatStagePlanner().plan(
runtime,
context,
42,
25.0,
"16:9",
0.35,
2048,
)
self.assertEqual(backend.call_count, 2)
retry_prompt = backend.call_args_list[1].args[1]
self.assertIn("previous answer had no parseable JSON object", retry_prompt)
self.assertIn("Creative brief:\n" + context.user_prompt, retry_prompt)
self.assertIn("JSON Schema:", retry_prompt)
self.assertIn(
self.nodes._SPLATSTAGE_EXPLICIT_GENRE_AUTHORITY_INSTRUCTION,
retry_prompt,
)
self.assertNotIn("Validation errors:", retry_prompt)
self.assertEqual(result["result"][1], True)
raw_record = json.loads(result["result"][3])
self.assertEqual(
raw_record["repair_strategy"],
"full_task_retry_after_unparseable_response",
)
def test_people_in_background_or_fx_positive_prompt_are_rejected_before_router(self) -> None:
valid = {
"schema": "splatstage.show_blueprint",
"version": 1,
"project_title": "Retro party",
"creative_summary": "A synchronized retro performance.",
"music": {
"ace_tags": "retro alt-pop",
"vocal_mode": "vocal_hook",
"lyrics": "[Verse]\nTurn up the light\n[Chorus]\nWe own the night",
"bpm": 120,
"key": "C major",
"time_signature": "4",
"language": "en",
},
"performer": {
"label": "Lead singer",
"positive_prompt": "One lead singer dancing beneath pink light",
"negative_prompt": "crowd",
"segmentation_prompt": "lead singer",
},
"background": {
"positive_prompt": "Dim party room with pink and blue ambient lighting",
"negative_prompt": "people, faces",
},
"fx": {
"positive_prompt": "Isolated prismatic sparks on pure black",
"negative_prompt": "people, text, logos",
"blend_mode": "screen",
"key_mode": "luminance",
"opacity": 0.5,
},
"editing": {
"background_density": "musical",
"fx_density": "musical",
"music_reactivity": 0.7,
},
"finish": {"style_preset": "broadcast_clean"},
}
leaked_background = json.loads(json.dumps(valid))
leaked_background["background"]["positive_prompt"] = (
"dimly lit indoor party room, pink and blue ambient lighting, "
"blurred silhouettes of people in distance, 90s decor, hazy atmosphere"
)
errors = self.nodes._splatstage_blueprint_errors(
leaked_background,
self.schema,
)
self.assertTrue(
any(
error.startswith("$.background.positive_prompt: people-free plate required")
and "'people'" in error
for error in errors
)
)
leaked_fx = json.loads(json.dumps(valid))
leaked_fx["fx"]["positive_prompt"] = "A singer-shaped light figure on pure black"
self.assertTrue(
any(
error.startswith("$.fx.positive_prompt: people-free plate required")
and "'singer'" in error
for error in self.nodes._splatstage_blueprint_errors(
leaked_fx,
self.schema,
)
)
)
def test_planner_repairs_people_leak_before_emitting_blueprint(self) -> None:
invalid = {
"schema": "splatstage.show_blueprint",
"version": 1,
"project_title": "Retro party",
"creative_summary": "A synchronized retro performance.",
"music": {
"ace_tags": "retro alt-pop",
"vocal_mode": "vocal_hook",
"lyrics": "[Verse]\nTurn up the light\n[Chorus]\nWe own the night",
"bpm": 120,
"key": "C major",
"time_signature": "4",
"language": "en",
},
"performer": {
"label": "Lead singer",
"positive_prompt": "One lead singer dancing beneath pink light",
"negative_prompt": "crowd",
"segmentation_prompt": "lead singer",
},
"background": {
"positive_prompt": "Blurred silhouettes of people in a hazy party room",
"negative_prompt": "faces",
},
"fx": {
"positive_prompt": "Isolated prismatic sparks on pure black",
"negative_prompt": "people, text, logos",
"blend_mode": "screen",
"key_mode": "luminance",
"opacity": 0.5,
},
"editing": {
"background_density": "musical",
"fx_density": "musical",
"music_reactivity": 0.7,
},
"finish": {"style_preset": "broadcast_clean"},
}
repaired = json.loads(json.dumps(invalid))
repaired["background"]["positive_prompt"] = (
"Dim party room with pink and blue ambient lighting, 90s decor, and haze"
)
repaired["background"]["negative_prompt"] = "people, faces"
runtime = self.nodes.RuntimeConfig(
model_path="",
backend="template",
dtype="auto",
quantization="none",
local_files_only=True,
unload_policy="keep_loaded",
max_memory_gb=1.0,
)
context = self.nodes.GemmaContext(user_prompt="Make a retro party music video")
with mock.patch.object(
self.nodes,
"_run_backend",
side_effect=[json.dumps(invalid), json.dumps(repaired)],
) as backend:
result = self.nodes.DiffusionGemmaSplatStagePlanner().plan(
runtime,
context,
42,
20.0,
"9:16",
0.35,
2048,
)
self.assertEqual(backend.call_count, 2)
repair_prompt = backend.call_args_list[1].args[1]
self.assertIn("$.background.positive_prompt: people-free plate required", repair_prompt)
emitted = json.loads(result["result"][0])
self.assertEqual(
emitted["background"]["positive_prompt"],
repaired["background"]["positive_prompt"],
)
self.assertEqual(
self.nodes._splatstage_blueprint_errors(emitted, self.schema),
[],
)
def test_planner_sends_context_hub_pixels_and_visual_context_to_backend(self) -> None:
runtime = self.nodes.RuntimeConfig(
model_path="unused",
backend="transformers_inprocess",
dtype="auto",
quantization="none",
local_files_only=True,
unload_policy="keep_loaded",
max_memory_gb=1.0,
status={"supports_pixels": True},
)
pixels = torch.zeros((1, 8, 8, 3), dtype=torch.float32)
context = self.nodes.GemmaContext(
user_prompt="Write a song and video for this image",
images=pixels,
source="image",
media_metadata={"visual_description": "A singer in a cobalt dress beneath amber lights."},
visual_description="A singer in a cobalt dress beneath amber lights.",
)
with mock.patch.object(self.nodes, "_run_backend", return_value="{}") as backend:
with self.assertRaisesRegex(RuntimeError, "could not produce a valid"):
self.nodes.DiffusionGemmaSplatStagePlanner().plan(
runtime,
context,
42,
90.0,
"9:16",
0.35,
2048,
)
first_call = backend.call_args_list[0]
self.assertIn("A singer in a cobalt dress beneath amber lights.", first_call.args[1])
media_context = first_call.args[2]
self.assertIsNotNone(media_context)
self.assertIs(media_context.images, pixels)
self.assertIs(backend.call_args_list[1].args[2], media_context)
def test_full_song_lyric_budget_accepts_48_lines_but_not_49(self) -> None:
def blueprint(line_count: int) -> dict:
@@ -358,6 +872,11 @@ class EfficientSplatStagePlannerTests(unittest.TestCase):
self.assertIn("Do not name TouchDesigner", prompt)
self.assertIn("locked TouchDesigner-inspired", prompt)
self.assertIn("affirmative visual language", prompt)
self.assertIn("10 to 14 sung lyric", prompt)
self.assertIn("6 to 10 syllables per line", prompt)
self.assertIn("repeat the Chorus verbatim", prompt)
self.assertIn("one stable", prompt)
self.assertIn("diatonic four-bar phrase loops", prompt)
def test_fx_creative_slot_rejects_software_or_interface_language(self) -> None:
value = self.valid_blueprint()
tests/test_timed_lyrics_nodes.py
+507
View File
@@ -0,0 +1,507 @@
from __future__ import annotations
import hashlib
import importlib.util
import json
import math
import sys
from pathlib import Path
from types import SimpleNamespace
import unittest
import uuid
from unittest import mock
import torch
ROOT = Path(__file__).resolve().parents[1]
COMFY_ROOT = ROOT.parents[1]
def load_timed_module():
module_name = f"diffusiongemma_timed_lyrics_fixture_{uuid.uuid4().hex}"
spec = importlib.util.spec_from_file_location(module_name, ROOT / "timed_lyrics_nodes.py")
if spec is None or spec.loader is None:
raise RuntimeError("could not load timed lyrics nodes")
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)
return module
def load_package():
if str(COMFY_ROOT) not in sys.path:
sys.path.insert(0, str(COMFY_ROOT))
package_name = f"diffusiongemma_timed_package_fixture_{uuid.uuid4().hex}"
spec = importlib.util.spec_from_file_location(
package_name,
ROOT / "__init__.py",
submodule_search_locations=[str(ROOT)],
)
if spec is None or spec.loader is None:
raise RuntimeError("could not load DiffusionGemma Prompt Builder")
package = importlib.util.module_from_spec(spec)
sys.modules[package_name] = package
spec.loader.exec_module(package)
return package
def audio_fixture(
*,
duration: float = 12.0,
sample_rate: int = 1_000,
vocal_sections: tuple[tuple[float, float], ...] = ((2.0, 5.0),),
):
samples = int(round(duration * sample_rate))
time = torch.arange(samples, dtype=torch.float32) / sample_rate
accompaniment = (
0.06 * torch.sin(2.0 * torch.pi * 73.0 * time)
+ 0.025 * torch.sin(2.0 * torch.pi * 117.0 * time)
)
vocals = torch.zeros_like(time)
for start, end in vocal_sections:
mask = (time >= start) & (time < end)
# A deterministic voice-like signal which is independent of the mix bed.
phrase = (
0.15 * torch.sin(2.0 * torch.pi * 181.0 * time)
+ 0.05 * torch.sin(2.0 * torch.pi * 263.0 * time)
)
vocals = torch.where(mask, phrase, vocals)
final = {
"waveform": (accompaniment + vocals).reshape(1, 1, -1).contiguous(),
"sample_rate": sample_rate,
}
stem = {
"waveform": vocals.reshape(1, 1, -1).contiguous(),
"sample_rate": sample_rate,
}
return final, stem
class FakeTranscriber:
def __init__(self, responses=None, exception: Exception | None = None):
self.responses = list(responses or [])
self.exception = exception
self.calls: list[dict[str, float | int | str]] = []
def __call__(
self,
audio,
language,
return_timestamps,
chunk_offset_seconds,
absolute_song_start_seconds,
):
self.calls.append(
{
"duration": audio["waveform"].shape[-1] / audio["sample_rate"],
"language": language,
"return_timestamps": int(bool(return_timestamps)),
"chunk_offset_seconds": chunk_offset_seconds,
"absolute_song_start_seconds": absolute_song_start_seconds,
}
)
if self.exception is not None:
raise self.exception
if self.responses:
return self.responses.pop(0)
return {
"text": "we light the night",
"chunks": [
{"text": "we light the night", "timestamp": [0.25, 1.75]},
],
}
class FakeModel:
def __init__(self):
self.config = SimpleNamespace(_name_or_path="fake/whisper-tiny")
self.moves: list[str] = []
def to(self, device):
self.moves.append(str(device))
return self
def fake_pipeline(transcriber: FakeTranscriber, *, with_model: bool = False):
result = {"model_id": "fake/whisper-tiny", "transcribe": transcriber}
if with_model:
result["model"] = FakeModel()
return result
class TimedLyricsAnalyzerTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.timed = load_timed_module()
def setUp(self) -> None:
self.timed._ANALYSIS_CACHE.clear()
def run_lyrics(
self,
final,
stem,
pipeline,
*,
lyrics="[Verse 1]\nWe light the night\nAnd hold the sky",
song_duration=None,
excerpt_start=0.0,
excerpt_duration=None,
confidence=0.55,
):
duration = final["waveform"].shape[-1] / final["sample_rate"]
return self.timed.DiffusionGemmaTimedLyricsAnalyzer().analyze(
final,
"Lyrics + lip sync",
lyrics,
self.timed._waveform_sha256(final),
duration if song_duration is None else song_duration,
excerpt_start,
duration if excerpt_duration is None else excerpt_duration,
"en",
confidence,
stem,
pipeline,
)
def test_input_schema_and_lazy_surface(self) -> None:
node = self.timed.DiffusionGemmaTimedLyricsAnalyzer
inputs = node.INPUT_TYPES()
required = inputs["required"]
optional = inputs["optional"]
self.assertEqual(required["performance_mode"][0], "STRING")
self.assertTrue(required["performance_mode"][1]["forceInput"])
self.assertTrue(required["lyrics"][1]["forceInput"])
self.assertTrue(required["lyrics"][1]["multiline"])
self.assertEqual(required["language"][1]["default"], "en")
self.assertAlmostEqual(required["minimum_alignment_confidence"][1]["default"], 0.55)
self.assertTrue(optional["vocal_stem"][1]["lazy"])
self.assertTrue(optional["whisper_pipeline"][1]["lazy"])
self.assertEqual(
node.check_lazy_status(
performance_mode="Lyrics + lip sync",
vocal_stem=None,
whisper_pipeline=None,
),
["vocal_stem", "whisper_pipeline"],
)
self.assertEqual(
node.check_lazy_status(
performance_mode="Lyrics + lip sync",
vocal_stem=object(),
whisper_pipeline=None,
),
["whisper_pipeline"],
)
self.assertEqual(
node.check_lazy_status(
performance_mode="Natural / audio-led sync",
vocal_stem=None,
whisper_pipeline=None,
),
[],
)
def test_natural_and_dance_never_evaluate_expensive_lazy_inputs(self) -> None:
final, _stem = audio_fixture()
class ExplodesOnAccess:
def __getattribute__(self, _name):
raise AssertionError("lazy input was evaluated")
node = self.timed.DiffusionGemmaTimedLyricsAnalyzer()
for mode in ("Natural / audio-led sync", "Dance / music sync"):
outputs = node.analyze(
final,
mode,
"We light the night",
"not-needed",
12.0,
0.0,
12.0,
"en",
0.55,
ExplodesOnAccess(),
ExplodesOnAccess(),
)
report = json.loads(outputs[0])
self.assertEqual(report["analysis_status"], "not_required")
self.assertFalse(report["timing_ready"])
self.assertTrue(outputs[2])
self.assertEqual(report["events"], [])
def test_exact_full_mix_separator_fallback_is_rejected_safely(self) -> None:
final, _stem = audio_fixture()
transcriber = FakeTranscriber()
outputs = self.run_lyrics(
final,
{"waveform": final["waveform"].clone(), "sample_rate": final["sample_rate"]},
fake_pipeline(transcriber),
)
report = json.loads(outputs[0])
self.assertEqual(report["analysis_status"], "natural_fallback")
self.assertFalse(report["timing_ready"])
self.assertTrue(outputs[2])
self.assertIn("vocal_stem_matches_full_mix_separator_fallback", report["warnings"])
self.assertEqual(transcriber.calls, [])
def test_near_identical_gain_shifted_full_mix_fallback_is_rejected(self) -> None:
final, _stem = audio_fixture()
generator = torch.Generator().manual_seed(91)
nearly_same = 1.013 * final["waveform"] + 1.0e-5 * torch.randn(
final["waveform"].shape, generator=generator
)
transcriber = FakeTranscriber()
outputs = self.run_lyrics(
final,
{"waveform": nearly_same, "sample_rate": final["sample_rate"]},
fake_pipeline(transcriber),
)
report = json.loads(outputs[0])
comparison = report["diagnostics"]["stem_full_mix_comparison"]
self.assertTrue(comparison["near_identical_full_mix"])
self.assertIn("vocal_stem_matches_full_mix_separator_fallback", report["warnings"])
self.assertEqual(transcriber.calls, [])
def test_valid_vocals_emit_relative_timing_events_and_intervals(self) -> None:
final, stem = audio_fixture()
transcriber = FakeTranscriber()
lyrics = "[Verse 1]\nWe light the night\nAnd hold the sky"
outputs = self.run_lyrics(final, stem, fake_pipeline(transcriber), lyrics=lyrics)
report = json.loads(outputs[0])
self.assertEqual(report["schema"], "diffusiongemma.timed_lyrics_report")
self.assertEqual(report["version"], 1)
self.assertEqual(report["analysis_status"], "timing_ready")
self.assertTrue(report["timing_ready"])
self.assertTrue(outputs[2])
self.assertEqual(report["lyrics_sha256"], hashlib.sha256(lyrics.encode("utf-8")).hexdigest())
self.assertEqual(report["master_audio_sha256"], self.timed._waveform_sha256(final))
self.assertEqual(report["model_id"], "fake/whisper-tiny")
self.assertEqual(report["excerpt"], {"start_seconds": 0.0, "duration_seconds": 12.0, "end_seconds": 12.0})
self.assertEqual(report["vocal_intervals"], [{"start_seconds": 2.0, "end_seconds": 5.0}])
self.assertEqual(
report["instrumental_intervals"],
[
{"start_seconds": 0.0, "end_seconds": 2.0},
{"start_seconds": 5.0, "end_seconds": 12.0},
],
)
event = report["events"][0]
self.assertAlmostEqual(event["start_seconds"], 2.25)
self.assertAlmostEqual(event["end_seconds"], 3.75)
self.assertEqual(event["text"], "we light the night")
self.assertEqual(event["transcript"], "we light the night")
self.assertEqual(event["authored_lines"], ["We light the night"])
self.assertEqual(event["confidence"], 1.0)
self.assertIn("Verified authored-lyric timing", outputs[3])
def test_direct_mtb_features_follow_the_loaded_model_dtype(self) -> None:
class Tokenizer:
@staticmethod
def convert_ids_to_tokens(_row):
return ["<|0.00|>", "hello", "<|1.00|>"]
class Processor:
tokenizer = Tokenizer()
@staticmethod
def __call__(_audio, sampling_rate, return_tensors):
self.assertEqual(sampling_rate, 16_000)
self.assertEqual(return_tensors, "pt")
return {"input_features": torch.ones((1, 80, 300), dtype=torch.float32)}
@staticmethod
def batch_decode(_predicted_ids, skip_special_tokens):
self.assertTrue(skip_special_tokens)
return ["hello"]
class HalfModel:
device = torch.device("cpu")
dtype = torch.float16
config = SimpleNamespace(max_length=32)
@staticmethod
def generate(input_features, **_kwargs):
self.assertEqual(input_features.dtype, torch.float16)
return torch.tensor([[1, 2, 3]], dtype=torch.long)
result = self.timed._direct_mtb_transcription(
{"processor": Processor(), "model": HalfModel()},
torch.zeros(16_000, dtype=torch.float32),
16_000,
"en",
)
self.assertEqual(result["text"], "hello")
self.assertEqual(result["tokens"], ["<|0.00|>", "hello", "<|1.00|>"])
def test_instrumental_excerpt_returns_natural_fallback_without_whisper(self) -> None:
final, stem = audio_fixture(vocal_sections=())
transcriber = FakeTranscriber()
outputs = self.run_lyrics(final, stem, fake_pipeline(transcriber))
report = json.loads(outputs[0])
self.assertEqual(report["analysis_status"], "natural_fallback")
self.assertFalse(report["timing_ready"])
self.assertIn("no_vocal_activity_detected", report["warnings"])
self.assertEqual(report["vocal_intervals"], [])
self.assertEqual(
report["instrumental_intervals"],
[{"start_seconds": 0.0, "end_seconds": 12.0}],
)
self.assertEqual(transcriber.calls, [])
def test_weak_authored_alignment_falls_back_instead_of_forcing_words(self) -> None:
final, stem = audio_fixture()
transcriber = FakeTranscriber(
[{"chunks": [{"text": "completely unrelated phrase", "timestamp": [0.0, 1.0]}]}]
)
outputs = self.run_lyrics(final, stem, fake_pipeline(transcriber), confidence=0.65)
report = json.loads(outputs[0])
self.assertEqual(report["analysis_status"], "natural_fallback")
self.assertFalse(report["timing_ready"])
self.assertEqual(report["events"], [])
self.assertIn("alignment_confidence_below_threshold", report["warnings"])
def test_adjacent_whisper_fragments_can_align_to_the_same_authored_line(self) -> None:
final, stem = audio_fixture()
transcriber = FakeTranscriber(
[
{
"chunks": [
{"text": "we light", "timestamp": [0.1, 0.8]},
{"text": "the night", "timestamp": [0.8, 1.5]},
]
}
]
)
outputs = self.run_lyrics(final, stem, fake_pipeline(transcriber))
report = json.loads(outputs[0])
self.assertTrue(report["timing_ready"])
self.assertEqual(len(report["events"]), 2)
self.assertEqual(
[event["authored_lines"] for event in report["events"]],
[["We light the night"], ["We light the night"]],
)
def test_chunk_timestamp_offset_after_fifteen_seconds_is_preserved(self) -> None:
final, stem = audio_fixture(duration=35.0, vocal_sections=((20.0, 23.0),))
transcriber = FakeTranscriber(
[{"chunks": [{"text": "late in the song", "timestamp": [0.4, 1.4]}]}]
)
outputs = self.run_lyrics(
final,
stem,
fake_pipeline(transcriber),
lyrics="[Verse]\nLate in the song",
)
report = json.loads(outputs[0])
self.assertTrue(report["timing_ready"])
self.assertAlmostEqual(report["events"][0]["start_seconds"], 20.4)
self.assertAlmostEqual(report["events"][0]["end_seconds"], 21.4)
self.assertAlmostEqual(transcriber.calls[0]["chunk_offset_seconds"], 20.0)
self.assertAlmostEqual(transcriber.calls[0]["absolute_song_start_seconds"], 20.0)
transcription = report["diagnostics"]["transcription"]
self.assertFalse(transcription["mtb_thirty_second_offset_logic_used"])
self.assertTrue(transcription["chunk_offsets_added_explicitly"])
self.assertLessEqual(transcriber.calls[0]["duration"], 15.0)
def test_excerpt_start_is_added_for_audio_crop_but_events_remain_excerpt_relative(self) -> None:
final, stem = audio_fixture(duration=40.0, vocal_sections=((21.0, 24.0),))
transcriber = FakeTranscriber(
[{"chunks": [{"text": "inside the window", "timestamp": [0.5, 1.5]}]}]
)
outputs = self.run_lyrics(
final,
stem,
fake_pipeline(transcriber),
lyrics="[Verse]\nInside the window",
excerpt_start=20.0,
excerpt_duration=10.0,
)
report = json.loads(outputs[0])
self.assertTrue(report["timing_ready"])
self.assertEqual(report["vocal_intervals"], [{"start_seconds": 1.0, "end_seconds": 4.0}])
self.assertAlmostEqual(report["events"][0]["start_seconds"], 1.5)
self.assertAlmostEqual(transcriber.calls[0]["absolute_song_start_seconds"], 21.0)
def test_transcription_exception_falls_back_offloads_model_and_clears_cuda(self) -> None:
final, stem = audio_fixture()
transcriber = FakeTranscriber(exception=RuntimeError("synthetic decoder failure"))
pipeline = fake_pipeline(transcriber, with_model=True)
with mock.patch.object(self.timed.torch.cuda, "empty_cache") as empty_cache:
outputs = self.run_lyrics(final, stem, pipeline)
report = json.loads(outputs[0])
self.assertEqual(report["analysis_status"], "natural_fallback")
self.assertFalse(report["timing_ready"])
self.assertTrue(outputs[2])
self.assertEqual(report["diagnostics"]["exception_type"], "RuntimeError")
self.assertEqual(pipeline["model"].moves[-1], "cpu")
empty_cache.assert_called_once()
def test_missing_stem_still_offloads_an_already_loaded_whisper_model(self) -> None:
final, _stem = audio_fixture()
pipeline = fake_pipeline(FakeTranscriber(), with_model=True)
outputs = self.timed.DiffusionGemmaTimedLyricsAnalyzer().analyze(
final,
"Lyrics + lip sync",
"We light the night",
self.timed._waveform_sha256(final),
12.0,
0.0,
12.0,
"en",
0.55,
None,
pipeline,
)
report = json.loads(outputs[0])
self.assertIn("vocal_stem_missing", report["warnings"])
self.assertEqual(pipeline["model"].moves[-1], "cpu")
def test_success_is_cached_by_audio_excerpt_lyrics_and_model(self) -> None:
final, stem = audio_fixture()
transcriber = FakeTranscriber()
pipeline = fake_pipeline(transcriber)
first = self.run_lyrics(final, stem, pipeline)
second = self.run_lyrics(final, stem, pipeline)
self.assertTrue(json.loads(first[0])["timing_ready"])
cached_report = json.loads(second[0])
self.assertTrue(cached_report["timing_ready"])
self.assertTrue(cached_report["diagnostics"]["cache_hit"])
self.assertEqual(len(transcriber.calls), 1)
self.assertIn("cache", second[1].casefold())
def test_nonfinite_or_misaligned_stem_is_never_a_blocker(self) -> None:
final, stem = audio_fixture()
bad = {"waveform": stem["waveform"].clone(), "sample_rate": stem["sample_rate"]}
bad["waveform"][..., 7] = math.nan
outputs = self.run_lyrics(final, bad, fake_pipeline(FakeTranscriber()))
report = json.loads(outputs[0])
self.assertEqual(report["analysis_status"], "natural_fallback")
self.assertTrue(outputs[2])
self.assertIn("vocal_stem_invalid_or_misaligned", report["warnings"])
shortened = {
"waveform": stem["waveform"][..., :-1000].clone(),
"sample_rate": stem["sample_rate"],
}
outputs = self.run_lyrics(final, shortened, fake_pipeline(FakeTranscriber()))
report = json.loads(outputs[0])
self.assertEqual(report["analysis_status"], "natural_fallback")
self.assertTrue(outputs[2])
self.assertIn("vocal_stem_invalid_or_misaligned", report["warnings"])
def test_package_registers_analyzer(self) -> None:
package = load_package()
self.assertIs(
package.NODE_CLASS_MAPPINGS["DiffusionGemmaTimedLyricsAnalyzer"],
sys.modules[f"{package.__name__}.timed_lyrics_nodes"].DiffusionGemmaTimedLyricsAnalyzer,
)
self.assertEqual(
package.NODE_DISPLAY_NAME_MAPPINGS["DiffusionGemmaTimedLyricsAnalyzer"],
"DiffusionGemma Timed Lyrics Analyzer",
)
if __name__ == "__main__":
unittest.main()
timed_lyrics_nodes.py
+1191
View File
File diff suppressed because it is too large Load Diff
tools/create_minimax_h3_identity_v4_smoke.py
+287
View File
@@ -0,0 +1,287 @@
#!/usr/bin/env python3
"""Create an isolated five-second smoke sibling from the verified V4 graph.
The source is never overwritten. The smoke graph changes exactly four
execution widgets, keeps relay Off, receives a fresh workflow UUID, and is
written byte-identically to Desktop and ComfyUI's user workflow directory.
"""
from __future__ import annotations
import argparse
import copy
import importlib.util
import json
import uuid
from pathlib import Path
from typing import Any
REPO_ROOT = Path(__file__).resolve().parents[1]
BASE_MIGRATION_PATH = REPO_ROOT / "tools" / "migrate_minimax_h3_identity_relay.py"
SPEC = importlib.util.spec_from_file_location(
"minimax_h3_identity_relay_base", BASE_MIGRATION_PATH
)
if SPEC is None or SPEC.loader is None: # pragma: no cover
raise RuntimeError(f"Cannot load base migration: {BASE_MIGRATION_PATH}")
BASE = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(BASE)
SMOKE_SCHEMA = "diffusiongemma.minimax_h3_identity_relay_smoke"
SMOKE_VERSION = 1
SOURCE_V4_SHA256 = (
"fe4261c2dd4fb16df6acf05d8735beb274a8e2d80821936dcda0271d2978d6fa"
)
SOURCE_V4 = Path(
r"C:\Users\danrh\Desktop\14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_DUAL_IDENTITY_P3_RELAY_GUARDED_V4_20260822.json"
)
DEFAULT_DESKTOP_OUTPUT = Path(
r"C:\Users\danrh\Desktop\14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_DUAL_IDENTITY_V4_SMOKE_5S_20260822.json"
)
DEFAULT_USER_OUTPUT = Path(
r"C:\ComfyUI\app\user\default\workflows\14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_DUAL_IDENTITY_V4_SMOKE_5S_20260822.json"
)
SMOKE_DURATION_SECONDS = 5.0
SMOKE_TARGET_WIDGET = "1"
SMOKE_RESOLUTION_SCALE = 0.2
SMOKE_OUTPUT_PREFIX = "minimax_h3/smoke_dual_identity_v4_5s"
ALLOWED_WIDGET_EDITS: dict[int, dict[int, Any]] = {
178: {0: SMOKE_DURATION_SECONDS},
673: {4: SMOKE_TARGET_WIDGET},
187: {1: SMOKE_RESOLUTION_SCALE},
651: {0: SMOKE_OUTPUT_PREFIX},
}
def _node(workflow: dict[str, Any], node_id: int) -> dict[str, Any]:
matches = [
item
for item in workflow.get("nodes", [])
if int(item.get("id", -1)) == int(node_id)
]
if len(matches) != 1:
raise BASE.WorkflowError(f"Expected one main node {node_id}; found {len(matches)}.")
return matches[0]
def _require_source(source: dict[str, Any], source_hash: str) -> None:
if source_hash.lower() != SOURCE_V4_SHA256:
raise BASE.WorkflowError(
f"Smoke source must be verified V4 SHA256 {SOURCE_V4_SHA256}; observed {source_hash}."
)
if SMOKE_SCHEMA in source.get("extra", {}):
raise BASE.WorkflowError("Smoke source is already a smoke artifact.")
BASE.validate_workflow(source)
secondary = _node(source, 721)
if (secondary.get("widgets_values") or [None])[0] != BASE.SECONDARY_IDENTITY_RELATIVE_PATH:
raise BASE.WorkflowError("Verified V4 does not use the root-level Picture 2 basename.")
if Path(BASE.SECONDARY_IDENTITY_RELATIVE_PATH).parent != Path("."):
raise BASE.WorkflowError("Picture 2 must remain at the ComfyUI input root.")
def create_smoke_workflow(
source: dict[str, Any],
*,
source_file_sha256: str,
source_path: Path | None = None,
) -> dict[str, Any]:
_require_source(source, source_file_sha256)
smoke = copy.deepcopy(source)
for node_id, edits in ALLOWED_WIDGET_EDITS.items():
node = _node(smoke, node_id)
widgets = list(node.get("widgets_values") or [])
for index, value in edits.items():
if index >= len(widgets):
raise BASE.WorkflowError(
f"Smoke node {node_id} has no widget index {index}."
)
widgets[index] = value
node["widgets_values"] = widgets
source_canonical = BASE._canonical_hash(source)
fresh_id = str(
uuid.uuid5(
uuid.NAMESPACE_URL,
f"{SMOKE_SCHEMA}/v{SMOKE_VERSION}/{source_canonical}",
)
)
if fresh_id == str(source.get("id", "")):
raise BASE.WorkflowError("Smoke workflow UUID did not diverge from V4.")
smoke["id"] = fresh_id
smoke["revision"] = int(source.get("revision", 0) or 0) + 1
marker = {
"schema": SMOKE_SCHEMA,
"version": SMOKE_VERSION,
"source_workflow_path": str(source_path.resolve()) if source_path else "",
"source_workflow_file_sha256": source_file_sha256.lower(),
"source_workflow_canonical_sha256": source_canonical,
"source_workflow_id": str(source.get("id", "")),
"smoke_workflow_id": fresh_id,
"duration_seconds": SMOKE_DURATION_SECONDS,
"target_profile_widget_4": SMOKE_TARGET_WIDGET,
"target_profile_widget_5_preserved": 15,
"target_profile_widget_11_preserved": 0,
"resolution_scale": SMOKE_RESOLUTION_SCALE,
"output_prefix": SMOKE_OUTPUT_PREFIX,
"continuity_relay_mode": "Off",
"exact_widget_edits": {
"178.widget_0": SMOKE_DURATION_SECONDS,
"673.widget_4": SMOKE_TARGET_WIDGET,
"187.widget_1": SMOKE_RESOLUTION_SCALE,
"651.widget_0": SMOKE_OUTPUT_PREFIX,
},
"source_is_never_overwritten": True,
"generation_was_not_queued_by_creator": True,
}
smoke.setdefault("extra", {})[SMOKE_SCHEMA] = marker
smoke["extra"]["workflow_note"] = (
"Isolated five-second V4 dual-identity smoke artifact. It preserves the verified V4 topology, "
"root-level Picture 2 asset, identity-first relay-Off policy, and b430 execution path while "
"changing only duration, target widget 4, resolution scale, and output prefix."
)
validate_smoke_workflow(smoke, source=source)
return smoke
def validate_smoke_workflow(
smoke: dict[str, Any],
*,
source: dict[str, Any],
) -> None:
BASE.validate_workflow(smoke)
marker = smoke.get("extra", {}).get(SMOKE_SCHEMA)
if not isinstance(marker, dict) or int(marker.get("version", 0)) != SMOKE_VERSION:
raise BASE.WorkflowError("Smoke provenance marker is missing or unsupported.")
if marker.get("source_workflow_file_sha256") != SOURCE_V4_SHA256:
raise BASE.WorkflowError("Smoke marker does not identify the verified V4 source hash.")
try:
uuid.UUID(str(smoke.get("id", "")))
except ValueError as exc:
raise BASE.WorkflowError("Smoke workflow id is not a UUID.") from exc
if str(smoke.get("id")) == str(source.get("id")):
raise BASE.WorkflowError("Smoke workflow must have an independent UUID.")
if int(smoke.get("revision", 0)) != int(source.get("revision", 0)) + 1:
raise BASE.WorkflowError("Smoke workflow revision is not exactly one after V4.")
if smoke.get("links") != source.get("links"):
raise BASE.WorkflowError("Smoke creation changed main-graph links.")
if smoke.get("definitions") != source.get("definitions"):
raise BASE.WorkflowError("Smoke creation changed subgraphs or runtime topology.")
for key in ("last_node_id", "last_link_id", "groups", "config", "version"):
if smoke.get(key) != source.get(key):
raise BASE.WorkflowError(f"Smoke creation changed protected workflow field {key!r}.")
source_nodes = {int(item["id"]): item for item in source["nodes"]}
smoke_nodes = {int(item["id"]): item for item in smoke["nodes"]}
if source_nodes.keys() != smoke_nodes.keys():
raise BASE.WorkflowError("Smoke creation added or removed nodes.")
for node_id, source_node in source_nodes.items():
comparable = copy.deepcopy(smoke_nodes[node_id])
if node_id in ALLOWED_WIDGET_EDITS:
widgets = list(comparable.get("widgets_values") or [])
for index in ALLOWED_WIDGET_EDITS[node_id]:
widgets[index] = source_node["widgets_values"][index]
comparable["widgets_values"] = widgets
if comparable != source_node:
raise BASE.WorkflowError(
f"Smoke node {node_id} changed outside the authorized widget edits."
)
if _node(smoke, 178)["widgets_values"][0] != SMOKE_DURATION_SECONDS:
raise BASE.WorkflowError("Smoke master duration is not 5.0 seconds.")
target_widgets = _node(smoke, 673)["widgets_values"]
if target_widgets[4] != SMOKE_TARGET_WIDGET:
raise BASE.WorkflowError("Smoke target widget 4 is not the requested string '1'.")
if target_widgets[5] != 15 or target_widgets[11] != 0:
raise BASE.WorkflowError("Smoke target widgets 5 or 11 changed unexpectedly.")
if _node(smoke, 187)["widgets_values"][1] != SMOKE_RESOLUTION_SCALE:
raise BASE.WorkflowError("Smoke resolution scale is not 0.2.")
if _node(smoke, 651)["widgets_values"][0] != SMOKE_OUTPUT_PREFIX:
raise BASE.WorkflowError("Smoke output prefix is incorrect.")
if _node(smoke, 677)["widgets_values"][-1] != "Off":
raise BASE.WorkflowError("Smoke relay must remain Off.")
if _node(smoke, 721)["widgets_values"][0] != BASE.SECONDARY_IDENTITY_RELATIVE_PATH:
raise BASE.WorkflowError("Smoke Picture 2 path is not the root-level guarded basename.")
source_extra = copy.deepcopy(source.get("extra", {}))
smoke_extra = copy.deepcopy(smoke.get("extra", {}))
smoke_extra.pop(SMOKE_SCHEMA, None)
source_extra.pop("workflow_note", None)
smoke_extra.pop("workflow_note", None)
if smoke_extra != source_extra:
raise BASE.WorkflowError("Smoke changed non-smoke provenance metadata.")
def _preflight_outputs(paths: tuple[Path, ...]) -> None:
resolved = [path.resolve() for path in paths]
if len(set(resolved)) != len(resolved):
raise BASE.WorkflowError("Smoke output paths must be distinct.")
existing = [str(path) for path in resolved if path.exists()]
if existing:
raise FileExistsError(
"Smoke output already exists; refusing overwrite: " + ", ".join(existing)
)
def write_smoke_copies(
smoke: dict[str, Any],
*,
desktop_output: Path,
user_output: Path,
) -> tuple[str, str]:
_preflight_outputs((desktop_output, user_output))
desktop_hash = BASE._exclusive_json_write(desktop_output, smoke)
user_hash = BASE._exclusive_json_write(user_output, smoke)
if desktop_hash != user_hash:
raise BASE.WorkflowError("Desktop and user smoke copies are not byte-identical.")
if desktop_output.resolve().read_bytes() != user_output.resolve().read_bytes():
raise BASE.WorkflowError("Desktop and user smoke bytes differ despite matching hashes.")
return desktop_hash, user_hash
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--source", type=Path, default=SOURCE_V4)
parser.add_argument("--desktop-output", type=Path, default=DEFAULT_DESKTOP_OUTPUT)
parser.add_argument("--user-output", type=Path, default=DEFAULT_USER_OUTPUT)
parser.add_argument("--check", action="store_true")
args = parser.parse_args(argv)
source, source_hash = BASE.load_workflow(args.source)
smoke = create_smoke_workflow(
source,
source_file_sha256=source_hash,
source_path=args.source,
)
result_hash = BASE._file_hash(BASE._json_bytes(smoke))
canonical_hash = BASE._canonical_hash(smoke)
if args.check:
print(
f"validated smoke in memory source_sha256={source_hash} "
f"output_sha256={result_hash} canonical_sha256={canonical_hash} "
f"workflow_id={smoke['id']}"
)
return 0
desktop_hash, user_hash = write_smoke_copies(
smoke,
desktop_output=args.desktop_output,
user_output=args.user_output,
)
print(
f"created desktop={args.desktop_output.resolve()} user={args.user_output.resolve()} "
f"sha256={desktop_hash} canonical_sha256={canonical_hash} "
f"workflow_id={smoke['id']}"
)
if desktop_hash != result_hash or user_hash != result_hash:
raise BASE.WorkflowError("Written smoke hash changed after validation.")
return 0
if __name__ == "__main__":
raise SystemExit(main())
tools/migrate_minimax_h3_fast_test.py
+1205
View File
File diff suppressed because it is too large Load Diff
tools/migrate_minimax_h3_identity_relay.py
+1288
View File
File diff suppressed because it is too large Load Diff
tools/migrate_minimax_h3_timed_lyrics_v5.py
+989
View File
@@ -0,0 +1,989 @@
#!/usr/bin/env python3
"""Create the guarded V5 MiniMax-H3 audio-led/timed-lyrics workflow sibling.
The verified V4 workflow is treated as immutable evidence. This migration
copies it in memory, adds a lazy vocal-separation/Whisper timing branch, wires
the resulting hash-locked report into the existing H3 lane planner, and writes
new V5 siblings only when their exact deterministic contents are known.
"""
from __future__ import annotations
import argparse
import copy
import hashlib
import json
import os
import tempfile
from pathlib import Path
from typing import Any, Iterable
MIGRATION_SCHEMA = "diffusiongemma.minimax_h3_timed_lyrics"
MIGRATION_VERSION = 1
TIMED_REPORT_SCHEMA = "diffusiongemma.timed_lyrics_report"
TIMED_REPORT_VERSION = 1
NATURAL_MODE = "Natural / audio-led sync"
DANCE_MODE = "Dance / music sync"
LYRICS_MODE = "Lyrics + lip sync"
PERFORMANCE_MODES = (DANCE_MODE, LYRICS_MODE, NATURAL_MODE)
AUDIO_SELECTOR_POLICY_MARKER = {
"revision": 4,
"aligned_half_time": "advisory_when_canonical_bpm_within_5_percent",
"aligned_double_time": "mandatory_alignment_pressure_vocal_plus_two_of_four_quality_signals",
"double_time": "hard_failure_outside_evidence_bundle",
"supporting_signals_required": 2,
"supporting_signals": [
"visual_recovery_coverage",
"excerpt_score",
"tonal_family_consistency",
"transition_stability",
],
"auto_selection_order": "tempo_class_then_recovery_then_excerpt_then_full_song_then_index",
}
BOUNDARY_POLICY_MARKER = {
"revision": 2,
"priority": [
"native_measured",
"native_only",
"mixed_native_or_measured",
"duration_balanced",
],
"missing_ideal_cut_evidence": "advisory_not_blocking",
"non_native_seam_policy": (
"carry_active_source_shot_and_preserve_later_timestamps"
),
"hard_failures": [
"invalid_project_or_excerpt_clock",
"malformed_or_nonconsecutive_native_shot_blocks",
"non_monotonic_or_out_of_range_native_timestamps",
"stale_or_mismatched_measured_audio_report",
"invalid_identity_reference_manifest",
"duration_outside_generation_lane_capacity",
"required_generation_lane_count_exceeds_four",
],
}
LEGACY_AUDIO_QC_COPY = (
"A canonically aligned double-time subdivision is accepted only when the selected excerpt proves low onset pressure, "
"at least 70% visual-recovery coverage, strong excerpt quality, tonal stability, and vocal activity; otherwise it remains blocking."
)
CURRENT_AUDIO_QC_COPY = (
"A canonically aligned double-time subdivision is advisory when canonical error is within 5%, excerpt onset pressure is at most "
"4.0/s, vocal activity is present, and at least two of visual-recovery coverage, excerpt score, tonal-family consistency, or "
"transition stability pass their strict thresholds; weakly supported, dense, or misaligned double-time remains blocking."
)
CURRENT_BOUNDARY_POLICY_COPY = (
"Generation-lane seams prefer native shot boundaries with measured recovery, "
"then native-only or mixed evidence. If those grids cannot satisfy the hard "
"duration limits, deterministic balanced seams are used. A non-native seam "
"carries the active source shot into the next lane so later timestamps remain "
"truthful. The plan records why each seam was selected separately from the "
"evidence present there. Missing ideal cut evidence never blocks generation; "
"invalid clocks, malformed shot blocks, timestamps, identity references, stale "
"audio locks, and impossible lane capacity remain hard failures."
)
SOURCE_V4 = Path(
r"C:\Users\danrh\Desktop\14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_DUAL_IDENTITY_P3_RELAY_GUARDED_V4_20260822.json"
)
SOURCE_V4_SHA256 = (
"fe4261c2dd4fb16df6acf05d8735beb274a8e2d80821936dcda0271d2978d6fa"
)
V5_FILENAME = (
"14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_"
"DUAL_IDENTITY_P3_RELAY_TIMED_LYRICS_V5_20260822.json"
)
DEFAULT_DESKTOP_OUTPUT = Path(r"C:\Users\danrh\Desktop") / V5_FILENAME
DEFAULT_USER_OUTPUT = (
Path(r"C:\ComfyUI\app\user\default\workflows") / V5_FILENAME
)
WHISPER_MODEL_RELATIVE_PATH = "models/whisper/whisper-large-v3-turbo/model.safetensors"
WHISPER_MODEL_BYTES = 1_617_824_864
WHISPER_MODEL_SHA256 = (
"542566a422ae4f3fd23f1ba11add198fca01bbf82e66e6a2857b3f608b1eb9d1"
)
DEMUCS_CHECKPOINT_PATH = (
r"C:\Users\danrh\.cache\torch\hub\torchaudio\models\hdemucs_high_trained.pt"
)
DEMUCS_CHECKPOINT_BYTES = 334_697_255
DEMUCS_CHECKPOINT_SHA256 = (
"a004b2790d73ffeaa535db458a1a79b539dfdbafbccc31f275d07e632ebd7816"
)
START_NODE_ID = 116
DIRECTOR_NODE_ID = 186
VIDEO_DURATION_NODE_ID = 178
LYRICS_NODE_ID = 556
SONG_DURATION_NODE_ID = 562
AUDIO_SELECTOR_NODE_ID = 610
AUDIO_GUIDE_NODE_ID = 611
LEGACY_LYRIC_WINDOW_NODE_ID = 621
PERFORMANCE_MODE_NODE_ID = 675
PLANNER_NODE_ID = 677
SEPARATOR_TYPE = "FL_Audio_Separation"
WHISPER_LOADER_TYPE = "Load Whisper (mtb)"
ANALYZER_TYPE = "DiffusionGemmaTimedLyricsAnalyzer"
PLANNER_TYPE = "DiffusionGemmaAudioAwareMultiShotPlanner"
class WorkflowError(RuntimeError):
"""Raised when a guarded migration invariant is not satisfied."""
def _file_sha256(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def _canonical_sha256(value: Any) -> str:
payload = json.dumps(
value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
).encode("utf-8")
return _file_sha256(payload)
def _json_bytes(value: Any) -> bytes:
return json.dumps(value, ensure_ascii=False, indent=2).encode("utf-8") + b"\n"
def _all_nodes(workflow: dict[str, Any]) -> Iterable[dict[str, Any]]:
yield from workflow.get("nodes", [])
for subgraph in workflow.get("definitions", {}).get("subgraphs", []):
yield from subgraph.get("nodes", [])
class IdAllocator:
def __init__(self, workflow: dict[str, Any]) -> None:
node_ids = [int(node["id"]) for node in _all_nodes(workflow)]
link_ids = [int(link[0]) for link in workflow.get("links", [])]
for subgraph in workflow.get("definitions", {}).get("subgraphs", []):
link_ids.extend(int(link["id"]) for link in subgraph.get("links", []))
self._node = max(node_ids + [int(workflow.get("last_node_id", 0) or 0)])
self._link = max(link_ids + [int(workflow.get("last_link_id", 0) or 0)])
def node(self) -> int:
self._node += 1
return self._node
def link(self) -> int:
self._link += 1
return self._link
@property
def last_node(self) -> int:
return self._node
@property
def last_link(self) -> int:
return self._link
class MainGraph:
def __init__(self, workflow: dict[str, Any], ids: IdAllocator) -> None:
self.workflow = workflow
self.nodes: list[dict[str, Any]] = workflow["nodes"]
self.links: list[list[Any]] = workflow["links"]
self.ids = ids
def node(self, node_id: int) -> dict[str, Any]:
matches = [node for node in self.nodes if int(node.get("id", -1)) == node_id]
if len(matches) != 1:
raise WorkflowError(f"Expected one main-graph node {node_id}; found {len(matches)}.")
return matches[0]
def one(self, node_type: str) -> dict[str, Any]:
matches = [node for node in self.nodes if node.get("type") == node_type]
if len(matches) != 1:
raise WorkflowError(f"Expected one {node_type}; found {len(matches)}.")
return matches[0]
@staticmethod
def input_slot(node: dict[str, Any], name: str) -> int:
matches = [
index
for index, item in enumerate(node.get("inputs", []))
if item.get("name") == name
]
if len(matches) != 1:
raise WorkflowError(
f"Node {node.get('id')} ({node.get('type')}) requires one input {name!r}."
)
return matches[0]
@staticmethod
def output_slot(node: dict[str, Any], name: str) -> int:
matches = [
index
for index, item in enumerate(node.get("outputs", []))
if item.get("name") == name
]
if len(matches) != 1:
raise WorkflowError(
f"Node {node.get('id')} ({node.get('type')}) requires one output {name!r}."
)
return matches[0]
def add(self, node: dict[str, Any]) -> dict[str, Any]:
if any(int(item["id"]) == int(node["id"]) for item in self.nodes):
raise WorkflowError(f"Duplicate node id {node['id']}.")
node["order"] = max(
(int(item.get("order", 0)) for item in self.nodes), default=0
) + 1
self.nodes.append(node)
return node
def connect(
self,
origin: dict[str, Any],
output_name: str,
target: dict[str, Any],
input_name: str,
link_type: str | None = None,
) -> int:
output_slot = self.output_slot(origin, output_name)
input_slot = self.input_slot(target, input_name)
if target["inputs"][input_slot].get("link") is not None:
raise WorkflowError(
f"Refusing to replace existing link on {target.get('id')}:{input_name}."
)
if link_type is None:
link_type = str(
target["inputs"][input_slot].get("type")
or origin["outputs"][output_slot].get("type")
)
link_id = self.ids.link()
self.links.append(
[
link_id,
int(origin["id"]),
output_slot,
int(target["id"]),
input_slot,
link_type,
]
)
target["inputs"][input_slot]["link"] = link_id
output_links = origin["outputs"][output_slot].get("links")
if not isinstance(output_links, list):
output_links = []
origin["outputs"][output_slot]["links"] = output_links
output_links.append(link_id)
return link_id
def _input(name: str, type_name: str, *, optional: bool = False) -> dict[str, Any]:
value: dict[str, Any] = {"name": name, "type": type_name, "link": None}
if optional:
value["shape"] = 7
return value
def _output(name: str, type_name: str) -> dict[str, Any]:
return {"name": name, "type": type_name, "links": []}
def _node(
ids: IdAllocator,
node_type: str,
title: str,
pos: tuple[float, float],
size: tuple[float, float],
inputs: list[dict[str, Any]],
outputs: list[dict[str, Any]],
widgets: list[Any] | None = None,
*,
cnr_id: str | None = "diffusiongemma-prompt-builder",
) -> dict[str, Any]:
properties: dict[str, Any] = {"Node name for S&R": node_type}
if cnr_id:
properties["cnr_id"] = cnr_id
return {
"id": ids.node(),
"type": node_type,
"pos": [float(pos[0]), float(pos[1])],
"size": [float(size[0]), float(size[1])],
"flags": {},
"order": 0,
"mode": 0,
"inputs": inputs,
"outputs": outputs,
"title": title,
"properties": properties,
"widgets_values": list(widgets or []),
}
def _preview_node(
ids: IdAllocator, title: str, pos: tuple[float, float], size: tuple[float, float]
) -> dict[str, Any]:
return _node(
ids,
"PreviewAny",
title,
pos,
size,
[_input("source", "*", optional=True)],
[],
[],
cnr_id="comfyui-easy-use",
)
def _set_first_widget(node: dict[str, Any], value: Any) -> None:
widgets = node.get("widgets_values")
if not isinstance(widgets, list) or not widgets:
raise WorkflowError(
f"Node {node.get('id')} ({node.get('type')}) has no first widget to update."
)
widgets[0] = value
def _set_director_cache_reuse(node: dict[str, Any]) -> None:
widgets = node.get("widgets_values")
if not isinstance(widgets, list) or len(widgets) != 6:
raise WorkflowError(
f"Node {node.get('id')} ({node.get('type')}) has an unexpected Director widget layout."
)
widgets[-1] = "reuse"
def _assert_node_type(node: dict[str, Any], expected: str) -> None:
if node.get("type") != expected:
raise WorkflowError(
f"Node {node.get('id')} is {node.get('type')!r}, expected {expected!r}."
)
def _link_by_id(workflow: dict[str, Any], link_id: int) -> list[Any]:
matches = [link for link in workflow.get("links", []) if int(link[0]) == link_id]
if len(matches) != 1:
raise WorkflowError(f"Expected one main-graph link {link_id}; found {len(matches)}.")
return matches[0]
def _origin(
workflow: dict[str, Any], target: dict[str, Any], input_name: str
) -> tuple[dict[str, Any], str]:
graph = MainGraph(workflow, IdAllocator(workflow))
input_slot = graph.input_slot(target, input_name)
link_id = target["inputs"][input_slot].get("link")
if link_id is None:
raise WorkflowError(f"{target.get('id')}:{input_name} is not connected.")
link = _link_by_id(workflow, int(link_id))
origin = graph.node(int(link[1]))
output = origin["outputs"][int(link[2])]
return origin, str(output.get("name"))
def _assert_origin(
workflow: dict[str, Any],
target: dict[str, Any],
input_name: str,
origin_id: int,
output_name: str,
) -> None:
origin, actual_output = _origin(workflow, target, input_name)
if int(origin["id"]) != origin_id or actual_output != output_name:
raise WorkflowError(
f"{target.get('id')}:{input_name} must come from "
f"{origin_id}:{output_name}, got {origin.get('id')}:{actual_output}."
)
def _sensitive_key_paths(value: Any, prefix: str = "") -> list[str]:
names = {
"api_key",
"apikey",
"authorization",
"access_token",
"refresh_token",
"password",
"secret",
"client_secret",
}
hits: list[str] = []
if isinstance(value, dict):
for key, item in value.items():
path = f"{prefix}.{key}" if prefix else str(key)
if str(key).casefold() in names and item not in (None, "", False):
hits.append(path)
hits.extend(_sensitive_key_paths(item, path))
elif isinstance(value, list):
for index, item in enumerate(value):
hits.extend(_sensitive_key_paths(item, f"{prefix}[{index}]"))
return hits
def _append_start_note(node: dict[str, Any]) -> None:
widgets = node.get("widgets_values")
if not isinstance(widgets, list) or not widgets or not isinstance(widgets[0], str):
raise WorkflowError("The START HERE note is missing its text widget.")
if LEGACY_AUDIO_QC_COPY in widgets[0]:
widgets[0] = widgets[0].replace(LEGACY_AUDIO_QC_COPY, CURRENT_AUDIO_QC_COPY)
addition = f"""
## Music audition + QC (policy revision 4)
{CURRENT_AUDIO_QC_COPY}
## H3 generation-lane boundary policy (revision 2)
{CURRENT_BOUNDARY_POLICY_COPY}
## Music-performance choice (V5)
- **Natural / audio-led sync** is the default. The soundtrack alone decides whether the subject naturally articulates; no lyric text is used as a mouth-motion schedule.
- **Dance / music sync** deliberately suppresses singing/lip-sync and concentrates on whole-body choreography, expressive and physically coherent H3 camera work, and beat/dynamic synchronization.
- **Lyrics + lip sync** first separates the vocal stem and aligns the audible vocal intervals to the authored lyrics. Only verified, hash-locked timing events reach the H3 lane prompts. Instrumental gaps remain explicitly non-vocal. Missing, stale, weak, or failed analysis safely becomes Natural/audio-led behavior instead of forcing singing.
The lyric-window node below remains an internal lexical-candidate helper. Its old local dropdown is informational only; the single authoritative control is **H3 PERFORMANCE MODE**.
""".rstrip()
if "## Music-performance choice (V5)" not in widgets[0]:
widgets[0] = widgets[0].rstrip() + addition + "\n"
node["title"] = "START HERE — dual identity / guarded relay / Natural audio-led default"
def _add_timing_branch(
workflow: dict[str, Any], graph: MainGraph, ids: IdAllocator
) -> dict[str, int]:
selector = graph.node(AUDIO_SELECTOR_NODE_ID)
guide = graph.node(AUDIO_GUIDE_NODE_ID)
lyrics = graph.node(LYRICS_NODE_ID)
song_duration = graph.node(SONG_DURATION_NODE_ID)
video_duration = graph.node(VIDEO_DURATION_NODE_ID)
performance = graph.node(PERFORMANCE_MODE_NODE_ID)
planner = graph.node(PLANNER_NODE_ID)
separator = graph.add(
_node(
ids,
SEPARATOR_TYPE,
"VOCAL STEM — Demucs separation (Lyrics mode only)",
(1788.0, -3680.0),
(760.0, 240.0),
[_input("audio", "AUDIO")],
[
_output("bass", "AUDIO"),
_output("drums", "AUDIO"),
_output("other", "AUDIO"),
_output("vocals", "AUDIO"),
],
[10.0, 0.5, "half_sine"],
cnr_id="comfyui_fill-nodes",
)
)
whisper = graph.add(
_node(
ids,
WHISPER_LOADER_TYPE,
"WHISPER LARGE-V3-TURBO — local / Lyrics mode only",
(1788.0, -3410.0),
(760.0, 160.0),
[],
[_output("pipeline", "WHISPER_PIPELINE")],
["large-v3-turbo", False],
cnr_id="comfy-mtb",
)
)
analyzer = graph.add(
_node(
ids,
ANALYZER_TYPE,
"TIMED LYRICS — audio-gated alignment / safe Natural fallback",
(1788.0, -3220.0),
(760.0, 500.0),
[
_input("final_audio", "AUDIO"),
_input("performance_mode", "STRING"),
_input("lyrics", "STRING"),
_input("master_audio_sha256", "STRING"),
_input("song_duration_seconds", "FLOAT"),
_input("excerpt_start_seconds", "FLOAT"),
_input("excerpt_duration_seconds", "FLOAT"),
_input("vocal_stem", "AUDIO", optional=True),
_input("whisper_pipeline", "WHISPER_PIPELINE", optional=True),
],
[
_output("timed_lyrics_report_json", "STRING"),
_output("status", "STRING"),
_output("safe_ready", "BOOLEAN"),
_output("aligned_preview", "STRING"),
],
[90.0, 0.0, 25.0, "en", 0.55],
)
)
status_preview = graph.add(
_preview_node(
ids,
"TIMED LYRICS STATUS — timing-ready or explicit Natural fallback",
(1788.0, -2690.0),
(760.0, 180.0),
)
)
alignment_preview = graph.add(
_preview_node(
ids,
"ALIGNED VOCAL EVENTS — excerpt-relative, never a lyric guess",
(1788.0, -2480.0),
(760.0, 260.0),
)
)
planner_inputs = planner.get("inputs")
if not isinstance(planner_inputs, list):
raise WorkflowError("Planner inputs are not serialized as a list.")
if any(item.get("name") == "timed_lyrics_report_json" for item in planner_inputs):
raise WorkflowError("The V4 source already contains a timed-lyrics planner input.")
planner_inputs.append(_input("timed_lyrics_report_json", "STRING", optional=True))
graph.connect(guide, "final_audio", separator, "audio")
graph.connect(guide, "final_audio", analyzer, "final_audio")
graph.connect(performance, "performance_mode", analyzer, "performance_mode")
graph.connect(lyrics, "lyrics", analyzer, "lyrics")
graph.connect(selector, "waveform_sha256", analyzer, "master_audio_sha256")
graph.connect(song_duration, "FLOAT", analyzer, "song_duration_seconds")
graph.connect(selector, "suggested_start_seconds", analyzer, "excerpt_start_seconds")
graph.connect(video_duration, "FLOAT", analyzer, "excerpt_duration_seconds")
graph.connect(separator, "vocals", analyzer, "vocal_stem")
graph.connect(whisper, "pipeline", analyzer, "whisper_pipeline")
graph.connect(analyzer, "timed_lyrics_report_json", planner, "timed_lyrics_report_json")
graph.connect(analyzer, "status", status_preview, "source", "*")
graph.connect(analyzer, "aligned_preview", alignment_preview, "source", "*")
return {
"separator": int(separator["id"]),
"whisper_loader": int(whisper["id"]),
"analyzer": int(analyzer["id"]),
"status_preview": int(status_preview["id"]),
"alignment_preview": int(alignment_preview["id"]),
"planner": PLANNER_NODE_ID,
"performance_control": PERFORMANCE_MODE_NODE_ID,
"legacy_lyric_window": LEGACY_LYRIC_WINDOW_NODE_ID,
}
def migrate_workflow(
source: dict[str, Any], *, source_file_sha256: str
) -> dict[str, Any]:
if source_file_sha256.casefold() != SOURCE_V4_SHA256:
raise WorkflowError(
"Timed-lyrics V5 must be derived from the exact verified V4 source: "
f"{source_file_sha256} != {SOURCE_V4_SHA256}."
)
if not isinstance(source.get("nodes"), list) or not isinstance(
source.get("links"), list
):
raise WorkflowError("Source workflow is missing main-graph nodes or links.")
if MIGRATION_SCHEMA in (source.get("extra") or {}):
raise WorkflowError("The supplied source is already a timed-lyrics V5 workflow.")
workflow = copy.deepcopy(source)
ids = IdAllocator(workflow)
graph = MainGraph(workflow, ids)
expected_types = {
START_NODE_ID: "MarkdownNote",
DIRECTOR_NODE_ID: "DiffusionGemmaCoTGenerator",
VIDEO_DURATION_NODE_ID: "PrimitiveFloat",
LYRICS_NODE_ID: "SplatStageBlueprintRouter",
SONG_DURATION_NODE_ID: "PrimitiveFloat",
AUDIO_SELECTOR_NODE_ID: "DiffusionGemmaAudioCandidateSelector",
AUDIO_GUIDE_NODE_ID: "DiffusionGemmaLTXAudioGuide",
LEGACY_LYRIC_WINDOW_NODE_ID: "DiffusionGemmaLTXPerformancePrompt",
PERFORMANCE_MODE_NODE_ID: "DiffusionGemmaMusicVideoPerformanceMode",
PLANNER_NODE_ID: PLANNER_TYPE,
}
for node_id, node_type in expected_types.items():
_assert_node_type(graph.node(node_id), node_type)
_append_start_note(graph.node(START_NODE_ID))
_set_director_cache_reuse(graph.node(DIRECTOR_NODE_ID))
graph.node(PLANNER_NODE_ID)["title"] = (
"AUDIO-AWARE H3 MULTI-LANE PLAN — evidence-preferred / deterministic fallback"
)
performance = graph.node(PERFORMANCE_MODE_NODE_ID)
_set_first_widget(performance, NATURAL_MODE)
performance["title"] = "H3 PERFORMANCE MODE — one authoritative control / Natural default"
legacy = graph.node(LEGACY_LYRIC_WINDOW_NODE_ID)
_set_first_widget(legacy, NATURAL_MODE)
legacy["title"] = (
"INTERNAL LYRIC CANDIDATE WINDOW — mode comes only from H3 PERFORMANCE MODE"
)
legacy["flags"] = {**(legacy.get("flags") or {}), "collapsed": True}
added_nodes = _add_timing_branch(workflow, graph, ids)
planning_groups = [
group
for group in workflow.get("groups", [])
if str(group.get("title", "")).startswith(
"3. Verified H3 project + audio-aware generation-lane plan"
)
]
if len(planning_groups) != 1:
raise WorkflowError(
"Expected one verified H3 planning group to extend for timed lyrics."
)
planning_group = planning_groups[0]
bounding = planning_group.get("bounding")
if not isinstance(bounding, list) or len(bounding) != 4:
raise WorkflowError("Verified H3 planning group has invalid bounds.")
existing_bottom = float(bounding[1]) + float(bounding[3])
bounding[1] = -3730.0
bounding[3] = existing_bottom - float(bounding[1])
planning_group["title"] = (
"3. Verified H3 project + lazy timed lyrics + audio-aware generation-lane plan"
)
extra = workflow.setdefault("extra", {})
if not isinstance(extra, dict):
raise WorkflowError("Workflow extra metadata is not an object.")
extra["workflow_note"] = (
"MiniMax-H3 Ref2VA production graph: versioned Project Master Contract, "
"locked ACE song/QC, evidence-preferred generation-lane planning with a "
"deterministic balanced fallback, four lazy 5–15 second H3 invocations that "
"preserve native multi-shot prompts and carry active shots across non-native "
"seams, retained-tail Picture 2 continuity between lanes, exact-frame assembly, "
"pristine ACE mux audio, and honest multi-format delivery planning."
)
music_marker = extra.get("diffusiongemma.music_video_production")
if isinstance(music_marker, dict):
music_marker["selector_policy"] = copy.deepcopy(AUDIO_SELECTOR_POLICY_MARKER)
performance_marker = music_marker.setdefault("ltx_performance_mode", {})
if isinstance(performance_marker, dict):
performance_marker.update(
{
"default": NATURAL_MODE,
"modes": list(PERFORMANCE_MODES),
"lyrics_policy": (
"authored text is lexical/pronunciation evidence only; "
"visible articulation requires hash-locked vocal timing"
),
"natural_policy": (
"connected audio alone governs articulation; no written-lyric schedule"
),
}
)
expansion_marker = extra.get("diffusiongemma.minimax_music_video_expansion")
if isinstance(expansion_marker, dict):
expansion_marker["generation_lane_boundary_policy"] = copy.deepcopy(
BOUNDARY_POLICY_MARKER
)
expansion_marker["performance_mode_contract"] = {
"default": NATURAL_MODE,
"modes": list(PERFORMANCE_MODES),
"authoritative_node": PERFORMANCE_MODE_NODE_ID,
"lyrics_require_timing_report": True,
}
previous_revision = int(source.get("revision", 0) or 0)
workflow["revision"] = previous_revision + 1
workflow["last_node_id"] = ids.last_node
workflow["last_link_id"] = ids.last_link
extra[MIGRATION_SCHEMA] = {
"version": MIGRATION_VERSION,
"source_workflow_id": str(source.get("id", "")),
"source_workflow_revision": previous_revision,
"source_workflow_file_sha256": source_file_sha256,
"source_workflow_canonical_sha256": _canonical_sha256(source),
"source_is_never_overwritten": True,
"default_performance_mode": NATURAL_MODE,
"performance_modes": list(PERFORMANCE_MODES),
"timing_report": {
"schema": TIMED_REPORT_SCHEMA,
"version": TIMED_REPORT_VERSION,
"audio_hash_locked": True,
"lyrics_hash_locked": True,
"excerpt_clock_locked": True,
"events_are_excerpt_relative": True,
"lyrics_are_not_a_timing_schedule": True,
},
"analysis": {
"vocal_separator": "torchaudio HDEMUCS_HIGH_MUSDB_PLUS via FL_Audio_Separation",
"demucs_checkpoint": {
"path": DEMUCS_CHECKPOINT_PATH,
"bytes": DEMUCS_CHECKPOINT_BYTES,
"sha256": DEMUCS_CHECKPOINT_SHA256,
},
"whisper_model": "openai/whisper-large-v3-turbo",
"whisper_local_directory": "models/whisper/whisper-large-v3-turbo",
"whisper_model_file": {
"relative_path": WHISPER_MODEL_RELATIVE_PATH,
"bytes": WHISPER_MODEL_BYTES,
"sha256": WHISPER_MODEL_SHA256,
},
"transcription_chunk_seconds_max": 15,
"minimum_alignment_confidence": 0.55,
"lazy_for_non_lyrics_modes": True,
},
"fallback_policy": {
"mode": NATURAL_MODE,
"separator_failure": "safe fallback; never original-mix vocal evidence",
"missing_or_stale_report": "safe fallback",
"weak_alignment": "safe fallback",
"instrumental_intervals": "explicit non-vocal behavior",
"generation_blocked_by_analysis_failure": False,
},
"director_cache_mode": {
"node": DIRECTOR_NODE_ID,
"default": "reuse",
"refresh_is_one_run_only": True,
},
"audio_selector_policy": copy.deepcopy(AUDIO_SELECTOR_POLICY_MARKER),
"generation_lane_boundary_policy": copy.deepcopy(BOUNDARY_POLICY_MARKER),
"nodes": added_nodes,
}
validate_workflow(workflow, source=source)
return workflow
def validate_workflow(
workflow: dict[str, Any], *, source: dict[str, Any] | None = None
) -> None:
graph = MainGraph(workflow, IdAllocator(workflow))
marker = (workflow.get("extra") or {}).get(MIGRATION_SCHEMA)
if not isinstance(marker, dict) or marker.get("version") != MIGRATION_VERSION:
raise WorkflowError("Timed-lyrics migration marker is missing or invalid.")
nodes = marker.get("nodes")
if not isinstance(nodes, dict):
raise WorkflowError("Timed-lyrics node map is missing.")
if marker.get("audio_selector_policy") != AUDIO_SELECTOR_POLICY_MARKER:
raise WorkflowError("Timed-lyrics audio selector policy marker is stale.")
if marker.get("generation_lane_boundary_policy") != BOUNDARY_POLICY_MARKER:
raise WorkflowError("Timed-lyrics generation-lane boundary policy is stale.")
music_marker = (workflow.get("extra") or {}).get(
"diffusiongemma.music_video_production"
)
if not isinstance(music_marker, dict) or music_marker.get(
"selector_policy"
) != AUDIO_SELECTOR_POLICY_MARKER:
raise WorkflowError("Inherited music-production selector policy is stale.")
expansion_marker = (workflow.get("extra") or {}).get(
"diffusiongemma.minimax_music_video_expansion"
)
if not isinstance(expansion_marker, dict) or expansion_marker.get(
"generation_lane_boundary_policy"
) != BOUNDARY_POLICY_MARKER:
raise WorkflowError("Inherited generation-lane boundary policy is stale.")
performance = graph.node(PERFORMANCE_MODE_NODE_ID)
legacy = graph.node(LEGACY_LYRIC_WINDOW_NODE_ID)
director = graph.node(DIRECTOR_NODE_ID)
_assert_node_type(director, "DiffusionGemmaCoTGenerator")
if director.get("widgets_values", [None])[-1:] != ["reuse"]:
raise WorkflowError("The V5 Director must reopen in reusable cache mode.")
if performance.get("widgets_values", [None])[0] != NATURAL_MODE:
raise WorkflowError("The authoritative performance control must default to Natural.")
if legacy.get("widgets_values", [None])[0] != NATURAL_MODE:
raise WorkflowError("The legacy hidden dropdown must visually agree with Natural.")
if not bool((legacy.get("flags") or {}).get("collapsed")):
raise WorkflowError("The legacy lyric-window node must be collapsed.")
start_copy = " ".join(
str(value) for value in graph.node(START_NODE_ID).get("widgets_values", [])
)
if CURRENT_AUDIO_QC_COPY not in start_copy or LEGACY_AUDIO_QC_COPY in start_copy:
raise WorkflowError("START HERE does not describe audio selector policy revision 4.")
if CURRENT_BOUNDARY_POLICY_COPY not in start_copy:
raise WorkflowError("START HERE does not describe boundary policy revision 2.")
separator = graph.node(int(nodes["separator"]))
whisper = graph.node(int(nodes["whisper_loader"]))
analyzer = graph.node(int(nodes["analyzer"]))
status_preview = graph.node(int(nodes["status_preview"]))
alignment_preview = graph.node(int(nodes["alignment_preview"]))
planner = graph.node(PLANNER_NODE_ID)
_assert_node_type(separator, SEPARATOR_TYPE)
_assert_node_type(whisper, WHISPER_LOADER_TYPE)
_assert_node_type(analyzer, ANALYZER_TYPE)
if planner.get("title") != (
"AUDIO-AWARE H3 MULTI-LANE PLAN — evidence-preferred / deterministic fallback"
):
raise WorkflowError("The H3 planner boundary-policy title is stale.")
if separator.get("widgets_values") != [10.0, 0.5, "half_sine"]:
raise WorkflowError("Vocal separation must retain the guarded chunk/fade settings.")
if whisper.get("widgets_values") != ["large-v3-turbo", False]:
raise WorkflowError("Whisper must use the local large-v3-turbo model without downloads.")
if analyzer.get("widgets_values") != [90.0, 0.0, 25.0, "en", 0.55]:
raise WorkflowError(
"Timed-lyrics analyzer connected-widget defaults/language/confidence changed."
)
planner_names = [item.get("name") for item in planner.get("inputs", [])]
if planner_names[-1:] != ["timed_lyrics_report_json"]:
raise WorkflowError("Timed report must append at the end of planner inputs.")
timed_input = planner["inputs"][-1]
if timed_input.get("type") != "STRING" or timed_input.get("shape") != 7:
raise WorkflowError("Planner timed-report socket must remain optional STRING shape 7.")
_assert_origin(workflow, separator, "audio", AUDIO_GUIDE_NODE_ID, "final_audio")
_assert_origin(workflow, analyzer, "final_audio", AUDIO_GUIDE_NODE_ID, "final_audio")
_assert_origin(
workflow, analyzer, "performance_mode", PERFORMANCE_MODE_NODE_ID, "performance_mode"
)
_assert_origin(workflow, analyzer, "lyrics", LYRICS_NODE_ID, "lyrics")
_assert_origin(
workflow, analyzer, "master_audio_sha256", AUDIO_SELECTOR_NODE_ID, "waveform_sha256"
)
_assert_origin(
workflow, analyzer, "song_duration_seconds", SONG_DURATION_NODE_ID, "FLOAT"
)
_assert_origin(
workflow,
analyzer,
"excerpt_start_seconds",
AUDIO_SELECTOR_NODE_ID,
"suggested_start_seconds",
)
_assert_origin(
workflow,
analyzer,
"excerpt_duration_seconds",
VIDEO_DURATION_NODE_ID,
"FLOAT",
)
_assert_origin(workflow, analyzer, "vocal_stem", int(separator["id"]), "vocals")
_assert_origin(
workflow, analyzer, "whisper_pipeline", int(whisper["id"]), "pipeline"
)
_assert_origin(
workflow,
planner,
"timed_lyrics_report_json",
int(analyzer["id"]),
"timed_lyrics_report_json",
)
_assert_origin(workflow, status_preview, "source", int(analyzer["id"]), "status")
_assert_origin(
workflow, alignment_preview, "source", int(analyzer["id"]), "aligned_preview"
)
if int(workflow.get("last_node_id", -1)) != max(
int(node["id"]) for node in _all_nodes(workflow)
):
raise WorkflowError("last_node_id is stale.")
main_link_ids = [int(link[0]) for link in workflow.get("links", [])]
nested_link_ids = [
int(link["id"])
for subgraph in workflow.get("definitions", {}).get("subgraphs", [])
for link in subgraph.get("links", [])
]
if int(workflow.get("last_link_id", -1)) != max(main_link_ids + nested_link_ids):
raise WorkflowError("last_link_id is stale.")
node_ids = [int(node["id"]) for node in _all_nodes(workflow)]
if len(node_ids) != len(set(node_ids)):
raise WorkflowError("Workflow contains duplicate node ids.")
if len(main_link_ids + nested_link_ids) != len(set(main_link_ids + nested_link_ids)):
raise WorkflowError("Workflow contains duplicate link ids.")
sensitive = _sensitive_key_paths(workflow)
if sensitive:
raise WorkflowError("Workflow contains sensitive values: " + ", ".join(sensitive))
if source is not None:
if workflow.get("id") != source.get("id"):
raise WorkflowError("V5 must preserve the verified workflow id.")
if int(workflow.get("revision", 0)) != int(source.get("revision", 0)) + 1:
raise WorkflowError("V5 revision must be exactly V4 revision + 1.")
if workflow.get("definitions") != source.get("definitions"):
raise WorkflowError("The H3 subgraph/definitions must remain byte-semantically unchanged.")
def _write_new_or_verify(
path: Path, payload: bytes, *, replace_owned: bool = False
) -> str:
path = path.resolve()
path.parent.mkdir(parents=True, exist_ok=True)
existed = path.exists()
if existed:
existing = path.read_bytes()
if existing != payload:
if not replace_owned:
raise WorkflowError(
f"Refusing to overwrite a different existing V5 artifact: {path}"
)
try:
owned = json.loads(existing.decode("utf-8"))
owned_marker = owned["extra"][MIGRATION_SCHEMA]
except (UnicodeDecodeError, json.JSONDecodeError, KeyError, TypeError) as exc:
raise WorkflowError(
f"Refusing to replace unrecognized artifact: {path}"
) from exc
if (
owned_marker.get("version") != MIGRATION_VERSION
or owned_marker.get("source_workflow_file_sha256") != SOURCE_V4_SHA256
or not bool(owned_marker.get("source_is_never_overwritten"))
):
raise WorkflowError(
f"Refusing to replace artifact without matching V5 ownership: {path}"
)
else:
return "verified"
descriptor, temporary_name = tempfile.mkstemp(
prefix=path.name + ".", suffix=".tmp", dir=path.parent
)
try:
with os.fdopen(descriptor, "wb") as handle:
handle.write(payload)
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary_name, path)
finally:
if os.path.exists(temporary_name):
os.unlink(temporary_name)
return "replaced" if existed else "created"
def build_artifact(source_path: Path = SOURCE_V4) -> tuple[dict[str, Any], bytes, str]:
source_path = source_path.resolve()
source_bytes = source_path.read_bytes()
source_hash = _file_sha256(source_bytes)
if source_hash != SOURCE_V4_SHA256:
raise WorkflowError(
f"Verified V4 source hash changed: {source_hash} != {SOURCE_V4_SHA256}."
)
try:
source = json.loads(source_bytes.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise WorkflowError("Verified V4 source is not valid UTF-8 JSON.") from exc
migrated = migrate_workflow(source, source_file_sha256=source_hash)
return migrated, _json_bytes(migrated), source_hash
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--source", type=Path, default=SOURCE_V4)
parser.add_argument("--desktop-output", type=Path, default=DEFAULT_DESKTOP_OUTPUT)
parser.add_argument("--user-output", type=Path, default=DEFAULT_USER_OUTPUT)
parser.add_argument(
"--check", action="store_true", help="Validate and print hashes without writing."
)
parser.add_argument(
"--replace-owned",
action="store_true",
help="Atomically replace only an existing artifact carrying this migration's exact ownership marker.",
)
args = parser.parse_args(argv)
migrated, payload, source_hash = build_artifact(args.source)
result_hash = _file_sha256(payload)
if args.check:
print(
f"checked source_sha256={source_hash} result_sha256={result_hash} "
f"canonical_sha256={_canonical_sha256(migrated)}"
)
return 0
actions = []
for path in (args.desktop_output, args.user_output):
action = _write_new_or_verify(
path, payload, replace_owned=bool(args.replace_owned)
)
actions.append(f"{action}={path.resolve()}")
print(
" ".join(actions)
+ f" source_sha256={source_hash} result_sha256={result_hash}"
)
return 0
if __name__ == "__main__":
raise SystemExit(main())
tools/migrate_minimax_h3_upload_song_v6.py
+969
View File
@@ -0,0 +1,969 @@
#!/usr/bin/env python3
"""Create the guarded V6 MiniMax-H3 workflow with a lazy uploaded-song path.
The user-supplied timed-lyrics V5 export is immutable source evidence. This
migration adds a dedicated upload loader and one lazy song-source router before
the existing decoded-waveform selector. Generated ACE candidates and uploaded
audio therefore share the same QC, excerpt, hash-lock, guide, planner, and final
mux path without changing Compose, Cover, or production-concept semantics.
"""
from __future__ import annotations
import argparse
import copy
import hashlib
import importlib.util
import json
import os
import tempfile
from pathlib import Path
from types import ModuleType
from typing import Any, Iterable
def _load_tool(filename: str, module_name: str) -> ModuleType:
path = Path(__file__).with_name(filename)
spec = importlib.util.spec_from_file_location(module_name, path)
if spec is None or spec.loader is None: # pragma: no cover - import guard
raise RuntimeError(f"Cannot load migration dependency: {path}")
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
TIMED = _load_tool(
"migrate_minimax_h3_timed_lyrics_v5.py", "diffusiongemma_timed_v5_dependency"
)
PRODUCTION = _load_tool(
"migrate_music_video_production.py", "diffusiongemma_music_production_dependency"
)
EXPANSION = _load_tool(
"migrate_minimax_music_video_expansion.py",
"diffusiongemma_minimax_expansion_dependency",
)
WorkflowError = TIMED.WorkflowError
IdAllocator = TIMED.IdAllocator
MainGraph = TIMED.MainGraph
_input = TIMED._input
_output = TIMED._output
_node = TIMED._node
_origin = TIMED._origin
MIGRATION_SCHEMA = "diffusiongemma.minimax_h3_upload_song"
MIGRATION_VERSION = 1
GENERATE_MODE = "Generate with ACE-Step"
UPLOAD_MODE = "Upload song"
SOURCE_MODES = (GENERATE_MODE, UPLOAD_MODE)
UPLOAD_LOADER_TYPE = "DiffusionGemmaUploadSong"
SOURCE_ROUTER_TYPE = "DiffusionGemmaSongSourceRouter"
_SOURCE_V5_FILENAME = (
"14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_"
"DUAL_IDENTITY_P3_RELAY_TIMED_LYRICS_V5_20260822 (1).json"
)
_SOURCE_V5_CANDIDATES = (
Path(r"C:\Users\danrh\Desktop") / _SOURCE_V5_FILENAME,
Path(r"C:\Users\danrh\Desktop\diffuGemmWkflow") / _SOURCE_V5_FILENAME,
)
SOURCE_V5 = next(
(candidate for candidate in _SOURCE_V5_CANDIDATES if candidate.is_file()),
_SOURCE_V5_CANDIDATES[0],
)
SOURCE_V5_SHA256 = (
"eff3e5a15bcb00b7f70290df5ac972b7f81be85fc9ea6e065f7ca1feb37ed6c3"
)
V6_FILENAME = (
"14_minimax_ref2va+AUDIO_IN_PROCESS_SYNC_organized_"
"DUAL_IDENTITY_P3_RELAY_TIMED_LYRICS_UPLOAD_SONG_V6_20260823.json"
)
DEFAULT_DESKTOP_OUTPUT = Path(r"C:\Users\danrh\Desktop") / V6_FILENAME
DEFAULT_USER_OUTPUT = Path(r"C:\ComfyUI\app\user\default\workflows") / V6_FILENAME
REPOSITORY_EXAMPLE = (
Path(__file__).resolve().parents[1]
/ "examples"
/ "15_minimax_h3_ref2va_music_video_v6.json"
)
REPOSITORY_EXAMPLE_CANONICAL_SHA256 = (
"e3fd1889891fd92f75580a34210fb94e90ae3468d01c8275f06e425e4d46a875"
)
START_NODE_ID = 116
BLUEPRINT_ROUTER_NODE_ID = 556
GENERATED_DURATION_NODE_ID = 562
PRODUCTION_CONCEPT_NODE_ID = 589
ACE_REFERENCE_MODE_NODE_ID = 592
EXPECTED_BPM_NODE_ID = 609
AUDIO_SELECTOR_NODE_ID = 610
LYRIC_WINDOW_NODE_ID = 621
ACE_CANDIDATES_NODE_ID = 623
PROJECT_CONTRACT_NODE_ID = 676
H3_PLANNER_NODE_ID = 677
TIMED_ANALYZER_NODE_ID = 733
AUDIO_GUIDANCE_NODE_ID = 624
LEGACY_DANCE_HELP = (
"- **Dance / music sync** deliberately suppresses singing/lip-sync and concentrates "
"on body, camera-safe scene, and beat/dynamic synchronization."
)
H3_DANCE_HELP = (
"- **Dance / music sync** deliberately suppresses singing/lip-sync and concentrates "
"on whole-body choreography, expressive and physically coherent H3 camera work, "
"and beat/dynamic synchronization."
)
LEGACY_H3_AUDIO_GUIDANCE = (
"Use the generated ACE-Step song as the exact soundtrack and timing authority. "
"Synchronize body motion, scene changes, cuts, and restrained camera accents "
"to measured rhythm, phrasing, dynamics, and recovery intervals. The "
"audio-aware H3 generation-lane planner applies the selected dance or lyric/lip-sync "
"performance contract after validation."
)
H3_AUDIO_GUIDANCE = (
"Use the selected song as the exact soundtrack and timing authority. Synchronize body motion, "
"scene changes, cuts, and expressive, physically coherent H3 camera choreography to measured "
"rhythm, phrasing, dynamics, and recovery intervals. Preserve explicitly requested orbiting, "
"swirling, sweeping or whip movement, pronounced parallax, and coherent compound paths rather "
"than downgrading them. When camera direction is unspecified, let the active Director creativity "
"mode choose camera energy proportional to creative strength. The audio-aware H3 generation-lane "
"planner applies the selected dance or lyric/lip-sync performance contract after validation."
)
H3_CAMERA_POLICY_MARKER = {
"revision": 1,
"ltx_camera_capability_applies": False,
"dynamic_paths_allowed": [
"orbit",
"swirl",
"sweep_or_whip",
"pronounced_parallax",
"coherent_compound_path",
],
"creative_energy_source": "director_creativity_mode_and_strength",
}
UPLOAD_SOURCE_POLICY_MARKER = {
"revision": 1,
"modes": list(SOURCE_MODES),
"default": GENERATE_MODE,
"production_concept_independent": True,
"source_passthrough_semantics": (
"preserves one authored ACE blueprint and still generates audio; "
"it is not waveform passthrough"
),
"ace_mode_semantics": {
"Compose new": "generate a new ACE-Step waveform",
"Cover reference": "generate ACE-Step audio from lazy reference conditioning",
},
"upload_semantics": {
"candidate_count": 1,
"expected_bpm_default": 0.0,
"expected_bpm_zero_means": "signal-derived tempo analysis",
"lyrics": "user supplied; empty is valid outside Lyrics + lip sync",
"duration": "measured from the decoded upload",
"file_is_embedded_in_workflow": False,
"selector_policy": "source_locked_uploaded_song",
"blocking_qc": [
"valid decoded waveform",
"requested excerpt duration",
"non-silent signal",
"clipping limit",
"broadband artifact guard",
],
"advisory_qc": [
"tempo interpretation",
"vocal presence proxy",
"onset pressure",
"tonal movement",
"generated-candidate production score",
],
},
"lazy_contract": {
"upload_mode_requests_generated_metadata_or_candidates": False,
"generate_mode_requests_uploaded_audio_or_metadata": False,
"generate_mode_requests_candidates_only_through_effective_count": True,
},
"shared_downstream_contract": {
"decoded_waveform_qc": "DiffusionGemmaAudioCandidateSelector",
"selector_waveform_sha256_is_authoritative": True,
"selector_excerpt_start_is_authoritative": True,
"router_source_token_drives_selector_policy": True,
"audio_guide_and_pristine_mux_unchanged": True,
},
}
UPLOAD_START_COPY = """## Custom soundtrack upload (V6)
- **Source passthrough** is an ACE planning choice: it still generates one ACE song and does not load an audio file.
- Set **SONG SOURCE** to **Upload song**, choose a file in **UPLOAD SONG**, and optionally enter its known BPM and matching lyrics. BPM `0` uses signal-derived tempo analysis.
- Upload mode requests only the uploaded waveform and its measured metadata. ACE candidates remain lazy. Generate mode requests only the existing ACE branch, so an unselected upload never affects Compose or Cover.
- An uploaded song is source-locked: malformed, too-short, silent, clipped, or artifact-like audio still blocks, while tempo aliases, vocal presence, onset density, tonal movement, and generated-candidate score are reported as advisories instead of rejecting your chosen music. The existing selector remains the authority for waveform SHA-256, excerpt start, sync-safe guide construction, and the pristine soundtrack mux.
- Matching uploaded lyrics are needed only for **Lyrics + lip sync**. Natural/audio-led and Dance/music-sync modes do not require lyric text.
""".strip()
def _sha256_bytes(data: bytes) -> str:
return hashlib.sha256(data).hexdigest()
def _canonical_sha256(value: Any) -> str:
payload = json.dumps(
value, ensure_ascii=False, sort_keys=True, separators=(",", ":")
).encode("utf-8")
return _sha256_bytes(payload)
def _json_bytes(value: Any) -> bytes:
return json.dumps(value, ensure_ascii=False, indent=2).encode("utf-8") + b"\n"
def _all_nodes(workflow: dict[str, Any]) -> Iterable[dict[str, Any]]:
yield from workflow.get("nodes", [])
for subgraph in workflow.get("definitions", {}).get("subgraphs", []):
yield from subgraph.get("nodes", [])
def _widget_input(name: str, type_name: str) -> dict[str, Any]:
value = _input(name, type_name)
value["widget"] = {"name": name}
return value
def _lazy_input(name: str, type_name: str) -> dict[str, Any]:
value = _input(name, type_name, optional=True)
return value
def _disconnect_input(
workflow: dict[str, Any], graph: MainGraph, target: dict[str, Any], input_name: str
) -> list[Any]:
slot = graph.input_slot(target, input_name)
link_id = target["inputs"][slot].get("link")
if link_id is None:
raise WorkflowError(
f"Cannot reroute unconnected input {target.get('id')}:{input_name}."
)
matches = [
link for link in workflow.get("links", []) if int(link[0]) == int(link_id)
]
if len(matches) != 1:
raise WorkflowError(f"Expected one link {link_id}; found {len(matches)}.")
link = matches[0]
origin = graph.node(int(link[1]))
output_slot = int(link[2])
links = origin["outputs"][output_slot].get("links")
if isinstance(links, list):
origin["outputs"][output_slot]["links"] = [
item for item in links if int(item) != int(link_id)
]
workflow["links"] = [
item for item in workflow.get("links", []) if int(item[0]) != int(link_id)
]
graph.links = workflow["links"]
target["inputs"][slot]["link"] = None
return link
def _assert_node_type(node: dict[str, Any], expected: str) -> None:
if node.get("type") != expected:
raise WorkflowError(
f"Node {node.get('id')} is {node.get('type')!r}; expected {expected!r}."
)
def _source_node_contract(graph: MainGraph) -> None:
expected = {
START_NODE_ID: "MarkdownNote",
BLUEPRINT_ROUTER_NODE_ID: "SplatStageBlueprintRouter",
GENERATED_DURATION_NODE_ID: "PrimitiveFloat",
PRODUCTION_CONCEPT_NODE_ID: "DiffusionGemmaMusicProductionConcept",
ACE_REFERENCE_MODE_NODE_ID: "DiffusionGemmaACEReferenceMode",
EXPECTED_BPM_NODE_ID: "CM_IntToFloat",
AUDIO_SELECTOR_NODE_ID: "DiffusionGemmaAudioCandidateSelector",
LYRIC_WINDOW_NODE_ID: "DiffusionGemmaLTXPerformancePrompt",
PROJECT_CONTRACT_NODE_ID: "DiffusionGemmaProjectMasterContract",
H3_PLANNER_NODE_ID: "DiffusionGemmaAudioAwareMultiShotPlanner",
TIMED_ANALYZER_NODE_ID: "DiffusionGemmaTimedLyricsAnalyzer",
AUDIO_GUIDANCE_NODE_ID: "PrimitiveStringMultiline",
}
for node_id, node_type in expected.items():
_assert_node_type(graph.node(node_id), node_type)
candidates = graph.node(ACE_CANDIDATES_NODE_ID)
if [item.get("name") for item in candidates.get("outputs", [])][:4] != [
"AUDIO",
"AUDIO_1",
"AUDIO_2",
"AUDIO_3",
]:
raise WorkflowError("The four-output ACE candidate group contract changed.")
def _normalize_inherited_contracts(workflow: dict[str, Any], graph: MainGraph) -> None:
extra = workflow.setdefault("extra", {})
if not isinstance(extra, dict):
raise WorkflowError("Workflow extra metadata is not an object.")
timed_marker = extra.get(TIMED.MIGRATION_SCHEMA)
if not isinstance(timed_marker, dict) or timed_marker.get("version") != 1:
raise WorkflowError("The source is not a recognized timed-lyrics V5 workflow.")
if timed_marker.get("source_workflow_file_sha256") != TIMED.SOURCE_V4_SHA256:
raise WorkflowError("The timed-lyrics V5 source lineage is not the verified V4.")
music_marker = extra.get(PRODUCTION.MIGRATION_SCHEMA)
if not isinstance(music_marker, dict):
raise WorkflowError("The inherited music-production marker is missing.")
if int(music_marker.get("version", 0)) not in {10, PRODUCTION.MIGRATION_VERSION}:
raise WorkflowError("The inherited music-production marker is unsupported.")
music_marker["version"] = PRODUCTION.MIGRATION_VERSION
music_marker["selector_policy"] = copy.deepcopy(
PRODUCTION.SELECTOR_POLICY_MARKER
)
expansion_marker = extra.get(EXPANSION.MIGRATION_SCHEMA)
if not isinstance(expansion_marker, dict) or int(
expansion_marker.get("version", 0)
) != EXPANSION.MIGRATION_VERSION:
raise WorkflowError("The inherited MiniMax expansion marker is unsupported.")
expansion_marker["performance_mode_contract"] = copy.deepcopy(
EXPANSION.PERFORMANCE_MODE_MARKER
)
expansion_marker["generation_lane_boundary_policy"] = copy.deepcopy(
EXPANSION.BOUNDARY_POLICY_MARKER
)
timed_marker["audio_selector_policy"] = copy.deepcopy(
TIMED.AUDIO_SELECTOR_POLICY_MARKER
)
timed_marker["generation_lane_boundary_policy"] = copy.deepcopy(
TIMED.BOUNDARY_POLICY_MARKER
)
planner = graph.node(H3_PLANNER_NODE_ID)
planner["title"] = (
"AUDIO-AWARE H3 MULTI-LANE PLAN — evidence-preferred / deterministic fallback"
)
start = graph.node(START_NODE_ID)
widgets = start.get("widgets_values")
if not isinstance(widgets, list) or not widgets or not isinstance(widgets[0], str):
raise WorkflowError("START HERE is missing its text widget.")
copy_text = widgets[0]
if TIMED.LEGACY_AUDIO_QC_COPY in copy_text:
copy_text = copy_text.replace(
TIMED.LEGACY_AUDIO_QC_COPY, TIMED.CURRENT_AUDIO_QC_COPY
)
if TIMED.CURRENT_AUDIO_QC_COPY not in copy_text:
copy_text = (
copy_text.rstrip()
+ "\n\n## Music audition + QC (policy revision 4)\n\n"
+ TIMED.CURRENT_AUDIO_QC_COPY
)
if TIMED.CURRENT_BOUNDARY_POLICY_COPY not in copy_text:
copy_text = (
copy_text.rstrip()
+ "\n\n## H3 generation-lane boundary policy (revision 2)\n\n"
+ TIMED.CURRENT_BOUNDARY_POLICY_COPY
)
if UPLOAD_START_COPY not in copy_text:
copy_text = copy_text.rstrip() + "\n\n" + UPLOAD_START_COPY + "\n"
copy_text = copy_text.replace(LEGACY_DANCE_HELP, H3_DANCE_HELP)
widgets[0] = copy_text
start["title"] = (
"START HERE — upload or ACE song / dual identity / Natural audio-led default"
)
guidance = graph.node(AUDIO_GUIDANCE_NODE_ID)
guidance_widgets = guidance.get("widgets_values")
if (
not isinstance(guidance_widgets, list)
or len(guidance_widgets) != 1
or not isinstance(guidance_widgets[0], str)
):
raise WorkflowError("H3 audio/performance guidance is missing its text widget.")
if guidance_widgets[0] == LEGACY_H3_AUDIO_GUIDANCE:
guidance_widgets[0] = H3_AUDIO_GUIDANCE
elif guidance_widgets[0] != H3_AUDIO_GUIDANCE:
raise WorkflowError("The inherited H3 audio/performance guidance is unrecognized.")
def _add_upload_branch(
workflow: dict[str, Any], graph: MainGraph, ids: IdAllocator
) -> dict[str, int]:
loader = graph.add(
_node(
ids,
UPLOAD_LOADER_TYPE,
"UPLOAD SONG — choose a local audio file (Upload mode only)",
(235.0, 118.0),
(535.0, 135.0),
[_widget_input("audio_file", "COMBO")],
[
_output("audio", "AUDIO"),
_output("duration_seconds", "FLOAT"),
_output("waveform_sha256", "STRING"),
_output("status", "STRING"),
_output("ready", "BOOLEAN"),
],
[""],
)
)
router = graph.add(
_node(
ids,
SOURCE_ROUTER_TYPE,
"SONG SOURCE — Generate with ACE-Step / Upload song",
(230.0, 285.0),
(545.0, 625.0),
[
_widget_input("source_mode", "COMBO"),
_widget_input("uploaded_expected_bpm", "FLOAT"),
_widget_input("uploaded_lyrics", "STRING"),
_lazy_input("generated_candidate_count", "INT"),
_lazy_input("generated_expected_bpm", "FLOAT"),
_lazy_input("generated_lyrics", "STRING"),
_lazy_input("generated_duration_seconds", "FLOAT"),
_lazy_input("generated_candidate_1", "AUDIO"),
_lazy_input("generated_candidate_2", "AUDIO"),
_lazy_input("generated_candidate_3", "AUDIO"),
_lazy_input("generated_candidate_4", "AUDIO"),
_lazy_input("uploaded_audio", "AUDIO"),
_lazy_input("uploaded_duration_seconds", "FLOAT"),
_lazy_input("uploaded_waveform_sha256", "STRING"),
_lazy_input("uploaded_status", "STRING"),
_lazy_input("uploaded_ready", "BOOLEAN"),
],
[
_output("candidate_1", "AUDIO"),
_output("candidate_2", "AUDIO"),
_output("candidate_3", "AUDIO"),
_output("candidate_4", "AUDIO"),
_output("effective_candidate_count", "INT"),
_output("effective_expected_bpm", "FLOAT"),
_output("selected_lyrics", "STRING"),
_output("selected_duration_seconds", "FLOAT"),
_output("source_token", "STRING"),
_output("status", "STRING"),
_output("ready", "BOOLEAN"),
],
[GENERATE_MODE, 0.0, ""],
)
)
concept = graph.node(PRODUCTION_CONCEPT_NODE_ID)
bpm = graph.node(EXPECTED_BPM_NODE_ID)
lyrics = graph.node(BLUEPRINT_ROUTER_NODE_ID)
duration = graph.node(GENERATED_DURATION_NODE_ID)
candidates = graph.node(ACE_CANDIDATES_NODE_ID)
selector = graph.node(AUDIO_SELECTOR_NODE_ID)
lyric_window = graph.node(LYRIC_WINDOW_NODE_ID)
project = graph.node(PROJECT_CONTRACT_NODE_ID)
timed = graph.node(TIMED_ANALYZER_NODE_ID)
generated_links = [
(concept, "candidate_count", "generated_candidate_count", "INT"),
(bpm, "FLOAT", "generated_expected_bpm", "FLOAT"),
(lyrics, "lyrics", "generated_lyrics", "STRING"),
(duration, "FLOAT", "generated_duration_seconds", "FLOAT"),
(candidates, "AUDIO", "generated_candidate_1", "AUDIO"),
(candidates, "AUDIO_1", "generated_candidate_2", "AUDIO"),
(candidates, "AUDIO_2", "generated_candidate_3", "AUDIO"),
(candidates, "AUDIO_3", "generated_candidate_4", "AUDIO"),
]
for origin, output_name, input_name, link_type in generated_links:
graph.connect(origin, output_name, router, input_name, link_type)
upload_links = [
("audio", "uploaded_audio", "AUDIO"),
("duration_seconds", "uploaded_duration_seconds", "FLOAT"),
("waveform_sha256", "uploaded_waveform_sha256", "STRING"),
("status", "uploaded_status", "STRING"),
("ready", "uploaded_ready", "BOOLEAN"),
]
for output_name, input_name, link_type in upload_links:
graph.connect(loader, output_name, router, input_name, link_type)
for input_name in (
"candidate_1",
"candidate_2",
"candidate_3",
"candidate_4",
"candidate_count",
"expected_bpm",
):
_disconnect_input(workflow, graph, selector, input_name)
if "source_policy" not in [item.get("name") for item in selector.get("inputs", [])]:
selector.setdefault("inputs", []).append(_lazy_input("source_policy", "STRING"))
selector_links = [
("candidate_1", "candidate_1", "AUDIO"),
("candidate_2", "candidate_2", "AUDIO"),
("candidate_3", "candidate_3", "AUDIO"),
("candidate_4", "candidate_4", "AUDIO"),
("effective_candidate_count", "candidate_count", "INT"),
("effective_expected_bpm", "expected_bpm", "FLOAT"),
("source_token", "source_policy", "STRING"),
]
for output_name, input_name, link_type in selector_links:
graph.connect(router, output_name, selector, input_name, link_type)
for target, input_name in (
(lyric_window, "lyrics"),
(project, "lyrics"),
(timed, "lyrics"),
):
_disconnect_input(workflow, graph, target, input_name)
graph.connect(router, "selected_lyrics", target, input_name, "STRING")
for target in (lyric_window, timed):
_disconnect_input(workflow, graph, target, "song_duration_seconds")
graph.connect(
router,
"selected_duration_seconds",
target,
"song_duration_seconds",
"FLOAT",
)
selector["title"] = (
"MUSIC INPUT + QC — ACE candidates or uploaded-song verification"
)
for group in workflow.get("groups", []):
if int(group.get("id", -1)) == 35:
bounds = group.get("bounding")
if not isinstance(bounds, list) or len(bounds) != 4:
raise WorkflowError("The ACE/QC group has invalid bounds.")
bottom = max(float(bounds[1]) + float(bounds[3]), 950.0)
bounds[3] = bottom - float(bounds[1])
group["title"] = (
"ACE generation or uploaded song → waveform QC + locked soundtrack"
)
break
else:
raise WorkflowError("The ACE/QC workflow group is missing.")
return {"upload_loader": int(loader["id"]), "source_router": int(router["id"])}
def migrate_workflow(
source: dict[str, Any], *, source_file_sha256: str
) -> dict[str, Any]:
if not isinstance(source, dict):
raise WorkflowError("Workflow root must be a JSON object.")
existing = (source.get("extra") or {}).get(MIGRATION_SCHEMA)
if existing is not None:
if not isinstance(existing, dict) or existing.get("version") != MIGRATION_VERSION:
raise WorkflowError("The upload-song migration marker is unsupported.")
result = copy.deepcopy(source)
graph = MainGraph(result, IdAllocator(result))
_source_node_contract(graph)
_normalize_inherited_contracts(result, graph)
result["extra"][MIGRATION_SCHEMA]["h3_camera_policy"] = copy.deepcopy(
H3_CAMERA_POLICY_MARKER
)
validate_workflow(result)
return result
if source_file_sha256.casefold() != SOURCE_V5_SHA256:
raise WorkflowError(
"Upload-song V6 must be derived from the exact attached V5 export: "
f"{source_file_sha256} != {SOURCE_V5_SHA256}."
)
if not isinstance(source.get("nodes"), list) or not isinstance(
source.get("links"), list
):
raise WorkflowError("Source workflow is missing nodes or links.")
workflow = copy.deepcopy(source)
ids = IdAllocator(workflow)
graph = MainGraph(workflow, ids)
_source_node_contract(graph)
_normalize_inherited_contracts(workflow, graph)
added = _add_upload_branch(workflow, graph, ids)
extra = workflow["extra"]
extra["workflow_note"] = (
"MiniMax-H3 Ref2VA V6: Generate with ACE-Step and Upload song are lazy, "
"mutually exclusive sources feeding the same decoded-waveform selector. "
"Compose, Cover, Source passthrough, dual identity, timed-lyrics fallback, "
"generation-lane planning, relay, assembly, and pristine soundtrack mux are preserved."
)
previous_revision = int(source.get("revision", 0) or 0)
workflow["revision"] = previous_revision + 1
workflow["last_node_id"] = ids.last_node
workflow["last_link_id"] = ids.last_link
extra[MIGRATION_SCHEMA] = {
"version": MIGRATION_VERSION,
"source_workflow_id": str(source.get("id", "")),
"source_workflow_revision": previous_revision,
"source_workflow_file_sha256": source_file_sha256,
"source_workflow_canonical_sha256": _canonical_sha256(source),
"source_is_never_overwritten": True,
"policy": copy.deepcopy(UPLOAD_SOURCE_POLICY_MARKER),
"h3_camera_policy": copy.deepcopy(H3_CAMERA_POLICY_MARKER),
"nodes": added,
"preserved_controls": {
"production_concept": PRODUCTION_CONCEPT_NODE_ID,
"ace_reference_mode": ACE_REFERENCE_MODE_NODE_ID,
"audio_selector": AUDIO_SELECTOR_NODE_ID,
"performance_mode": TIMED.PERFORMANCE_MODE_NODE_ID,
"project_contract": PROJECT_CONTRACT_NODE_ID,
},
}
validate_workflow(workflow, source=source)
return workflow
def _assert_origin(
workflow: dict[str, Any], target: dict[str, Any], input_name: str,
origin_id: int, output_name: str,
) -> None:
origin, actual_output = _origin(workflow, target, input_name)
if int(origin["id"]) != int(origin_id) or actual_output != output_name:
raise WorkflowError(
f"{target.get('id')}:{input_name} must come from "
f"{origin_id}:{output_name}, got {origin.get('id')}:{actual_output}."
)
def _validate_integrity(workflow: dict[str, Any]) -> None:
node_ids = [int(node["id"]) for node in _all_nodes(workflow)]
if len(node_ids) != len(set(node_ids)):
raise WorkflowError("Workflow contains duplicate node ids.")
main_links = [int(link[0]) for link in workflow.get("links", [])]
nested_links = [
int(link["id"])
for subgraph in workflow.get("definitions", {}).get("subgraphs", [])
for link in subgraph.get("links", [])
]
if len(main_links + nested_links) != len(set(main_links + nested_links)):
raise WorkflowError("Workflow contains duplicate link ids.")
if int(workflow.get("last_node_id", -1)) != max(node_ids):
raise WorkflowError("last_node_id is stale.")
if int(workflow.get("last_link_id", -1)) != max(main_links + nested_links):
raise WorkflowError("last_link_id is stale.")
link_by_id = {int(link[0]): link for link in workflow.get("links", [])}
main_nodes = {int(node["id"]): node for node in workflow.get("nodes", [])}
for link_id, link in link_by_id.items():
origin = main_nodes.get(int(link[1]))
target = main_nodes.get(int(link[3]))
if origin is None or target is None:
raise WorkflowError(f"Main link {link_id} references a missing node.")
if int(link[2]) >= len(origin.get("outputs", [])) or int(link[4]) >= len(
target.get("inputs", [])
):
raise WorkflowError(f"Main link {link_id} references a missing socket.")
if int(target["inputs"][int(link[4])].get("link", -1)) != link_id:
raise WorkflowError(f"Main link {link_id} target backlink is stale.")
output_links = origin["outputs"][int(link[2])].get("links") or []
if link_id not in [int(value) for value in output_links]:
raise WorkflowError(f"Main link {link_id} origin backlink is stale.")
def validate_workflow(
workflow: dict[str, Any], *, source: dict[str, Any] | None = None
) -> None:
graph = MainGraph(workflow, IdAllocator(workflow))
marker = (workflow.get("extra") or {}).get(MIGRATION_SCHEMA)
if not isinstance(marker, dict) or marker.get("version") != MIGRATION_VERSION:
raise WorkflowError("Upload-song V6 marker is missing or invalid.")
if marker.get("policy") != UPLOAD_SOURCE_POLICY_MARKER:
raise WorkflowError("Upload-song source policy marker is stale.")
if marker.get("h3_camera_policy") != H3_CAMERA_POLICY_MARKER:
raise WorkflowError("MiniMax H3 camera policy marker is missing or stale.")
nodes = marker.get("nodes")
if not isinstance(nodes, dict):
raise WorkflowError("Upload-song node map is missing.")
loader = graph.node(int(nodes.get("upload_loader", -1)))
router = graph.node(int(nodes.get("source_router", -1)))
_assert_node_type(loader, UPLOAD_LOADER_TYPE)
_assert_node_type(router, SOURCE_ROUTER_TYPE)
if [item.get("name") for item in loader.get("inputs", [])] != ["audio_file"]:
raise WorkflowError("Upload loader input contract changed.")
if [item.get("name") for item in loader.get("outputs", [])] != [
"audio", "duration_seconds", "waveform_sha256", "status", "ready"
]:
raise WorkflowError("Upload loader output contract changed.")
expected_router_inputs = [
"source_mode", "uploaded_expected_bpm", "uploaded_lyrics",
"generated_candidate_count", "generated_expected_bpm", "generated_lyrics",
"generated_duration_seconds", "generated_candidate_1", "generated_candidate_2",
"generated_candidate_3", "generated_candidate_4", "uploaded_audio",
"uploaded_duration_seconds", "uploaded_waveform_sha256", "uploaded_status",
"uploaded_ready",
]
if [item.get("name") for item in router.get("inputs", [])] != expected_router_inputs:
raise WorkflowError("Song-source router input contract changed.")
router_widgets = router.get("widgets_values")
if not isinstance(router_widgets, list) or len(router_widgets) != 3:
raise WorkflowError("Song-source router widgets are malformed.")
if router_widgets[0] not in SOURCE_MODES:
raise WorkflowError("Song-source router has an unsupported saved source mode.")
try:
uploaded_bpm = float(router_widgets[1])
except (TypeError, ValueError) as exc:
raise WorkflowError("Uploaded-song BPM must be numeric.") from exc
if not 0.0 <= uploaded_bpm <= 300.0:
raise WorkflowError("Uploaded-song BPM must be from 0 to 300.")
if not isinstance(router_widgets[2], str):
raise WorkflowError("Uploaded-song lyrics must remain a string widget.")
expected_router_outputs = [
"candidate_1", "candidate_2", "candidate_3", "candidate_4",
"effective_candidate_count", "effective_expected_bpm", "selected_lyrics",
"selected_duration_seconds", "source_token", "status", "ready",
]
if [item.get("name") for item in router.get("outputs", [])] != expected_router_outputs:
raise WorkflowError("Song-source router output contract changed.")
for item in router.get("inputs", [])[3:]:
if item.get("shape") != 7:
raise WorkflowError(
f"Song-source input {item.get('name')} must remain optional/lazy shape 7."
)
generated_origins = {
"generated_candidate_count": (PRODUCTION_CONCEPT_NODE_ID, "candidate_count"),
"generated_expected_bpm": (EXPECTED_BPM_NODE_ID, "FLOAT"),
"generated_lyrics": (BLUEPRINT_ROUTER_NODE_ID, "lyrics"),
"generated_duration_seconds": (GENERATED_DURATION_NODE_ID, "FLOAT"),
"generated_candidate_1": (ACE_CANDIDATES_NODE_ID, "AUDIO"),
"generated_candidate_2": (ACE_CANDIDATES_NODE_ID, "AUDIO_1"),
"generated_candidate_3": (ACE_CANDIDATES_NODE_ID, "AUDIO_2"),
"generated_candidate_4": (ACE_CANDIDATES_NODE_ID, "AUDIO_3"),
}
for input_name, (origin_id, output_name) in generated_origins.items():
_assert_origin(workflow, router, input_name, origin_id, output_name)
for input_name, output_name in (
("uploaded_audio", "audio"),
("uploaded_duration_seconds", "duration_seconds"),
("uploaded_waveform_sha256", "waveform_sha256"),
("uploaded_status", "status"),
("uploaded_ready", "ready"),
):
_assert_origin(workflow, router, input_name, int(loader["id"]), output_name)
selector = graph.node(AUDIO_SELECTOR_NODE_ID)
selector_origins = {
"candidate_1": "candidate_1", "candidate_2": "candidate_2",
"candidate_3": "candidate_3", "candidate_4": "candidate_4",
"candidate_count": "effective_candidate_count",
"expected_bpm": "effective_expected_bpm",
"source_policy": "source_token",
}
for input_name, output_name in selector_origins.items():
_assert_origin(workflow, selector, input_name, int(router["id"]), output_name)
_assert_origin(
workflow, graph.node(LYRIC_WINDOW_NODE_ID), "lyrics", int(router["id"]),
"selected_lyrics",
)
_assert_origin(
workflow, graph.node(PROJECT_CONTRACT_NODE_ID), "lyrics", int(router["id"]),
"selected_lyrics",
)
_assert_origin(
workflow, graph.node(TIMED_ANALYZER_NODE_ID), "lyrics", int(router["id"]),
"selected_lyrics",
)
for target_id in (LYRIC_WINDOW_NODE_ID, TIMED_ANALYZER_NODE_ID):
_assert_origin(
workflow, graph.node(target_id), "song_duration_seconds", int(router["id"]),
"selected_duration_seconds",
)
# The H3 planner must continue receiving the bounded lyric window rather than
# an unbounded full-song upload.
_assert_origin(
workflow, graph.node(H3_PLANNER_NODE_ID), "lyrics", LYRIC_WINDOW_NODE_ID,
"selected_lyrics",
)
extra = workflow.get("extra") or {}
music = extra.get(PRODUCTION.MIGRATION_SCHEMA)
if not isinstance(music, dict) or int(music.get("version", 0)) != PRODUCTION.MIGRATION_VERSION:
raise WorkflowError("Inherited music-production marker is not current.")
if music.get("selector_policy") != PRODUCTION.SELECTOR_POLICY_MARKER:
raise WorkflowError("Inherited selector policy is stale.")
expansion = extra.get(EXPANSION.MIGRATION_SCHEMA)
if not isinstance(expansion, dict) or expansion.get(
"generation_lane_boundary_policy"
) != EXPANSION.BOUNDARY_POLICY_MARKER:
raise WorkflowError("Inherited boundary policy is stale.")
timed = extra.get(TIMED.MIGRATION_SCHEMA)
if not isinstance(timed, dict) or timed.get(
"audio_selector_policy"
) != TIMED.AUDIO_SELECTOR_POLICY_MARKER:
raise WorkflowError("Timed-lyrics selector policy is stale.")
if timed.get("generation_lane_boundary_policy") != TIMED.BOUNDARY_POLICY_MARKER:
raise WorkflowError("Timed-lyrics boundary policy is stale.")
start_text = " ".join(
str(value) for value in graph.node(START_NODE_ID).get("widgets_values", [])
)
for required in (
TIMED.CURRENT_AUDIO_QC_COPY,
TIMED.CURRENT_BOUNDARY_POLICY_COPY,
UPLOAD_START_COPY,
):
if required not in start_text:
raise WorkflowError("START HERE upload/current-policy help is incomplete.")
if H3_DANCE_HELP not in start_text or LEGACY_DANCE_HELP in start_text:
raise WorkflowError("START HERE still carries the retired H3 camera-safe guidance.")
guidance_text = " ".join(
str(value)
for value in graph.node(AUDIO_GUIDANCE_NODE_ID).get("widgets_values", [])
)
if guidance_text != H3_AUDIO_GUIDANCE:
raise WorkflowError("The expressive H3 audio/performance camera guidance is missing.")
if graph.node(H3_PLANNER_NODE_ID).get("title") != (
"AUDIO-AWARE H3 MULTI-LANE PLAN — evidence-preferred / deterministic fallback"
):
raise WorkflowError("The current H3 boundary-policy title is missing.")
_validate_integrity(workflow)
sensitive = TIMED._sensitive_key_paths(workflow)
if sensitive:
raise WorkflowError("Workflow contains sensitive values: " + ", ".join(sensitive))
if source is not None:
if workflow.get("id") != source.get("id"):
raise WorkflowError("V6 must preserve the attached V5 workflow id.")
if int(workflow.get("revision", 0)) != int(source.get("revision", 0)) + 1:
raise WorkflowError("V6 revision must be exactly attached V5 revision + 1.")
if workflow.get("definitions") != source.get("definitions"):
raise WorkflowError("The H3 subgraphs/definitions must remain unchanged.")
for node_id in (
PRODUCTION_CONCEPT_NODE_ID,
ACE_REFERENCE_MODE_NODE_ID,
AUDIO_SELECTOR_NODE_ID,
TIMED.PERFORMANCE_MODE_NODE_ID,
PROJECT_CONTRACT_NODE_ID,
):
if graph.node(node_id).get("widgets_values") != MainGraph(
source, IdAllocator(source)
).node(node_id).get("widgets_values"):
raise WorkflowError(f"Saved user control widgets changed on node {node_id}.")
def _write_new_or_verify(
path: Path, payload: bytes, *, replace_owned: bool = False
) -> str:
path = path.resolve()
path.parent.mkdir(parents=True, exist_ok=True)
existed = path.exists()
if existed:
current = path.read_bytes()
if current == payload:
return "verified"
if not replace_owned:
raise WorkflowError(f"Refusing to overwrite a different V6 artifact: {path}")
try:
owned = json.loads(current.decode("utf-8"))
marker = owned["extra"][MIGRATION_SCHEMA]
except (UnicodeDecodeError, json.JSONDecodeError, KeyError, TypeError) as exc:
raise WorkflowError(f"Refusing to replace unrecognized artifact: {path}") from exc
if (
marker.get("version") != MIGRATION_VERSION
or marker.get("source_workflow_file_sha256") != SOURCE_V5_SHA256
or not bool(marker.get("source_is_never_overwritten"))
):
raise WorkflowError(
f"Refusing to replace artifact without matching V6 ownership: {path}"
)
descriptor, temporary_name = tempfile.mkstemp(
prefix=path.name + ".", suffix=".tmp", dir=path.parent
)
try:
with os.fdopen(descriptor, "wb") as handle:
handle.write(payload)
handle.flush()
os.fsync(handle.fileno())
os.replace(temporary_name, path)
finally:
if os.path.exists(temporary_name):
os.unlink(temporary_name)
return "replaced" if existed else "created"
def build_artifact(
source_path: Path = SOURCE_V5,
) -> tuple[dict[str, Any], bytes, str]:
source_path = source_path.resolve()
source_bytes = source_path.read_bytes()
source_hash = _sha256_bytes(source_bytes)
if source_hash != SOURCE_V5_SHA256:
raise WorkflowError(
f"Attached V5 source hash changed: {source_hash} != {SOURCE_V5_SHA256}."
)
try:
source = json.loads(source_bytes.decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise WorkflowError("Attached V5 source is not valid UTF-8 JSON.") from exc
migrated = migrate_workflow(source, source_file_sha256=source_hash)
return migrated, _json_bytes(migrated), source_hash
def upgrade_owned_artifact(path: Path) -> tuple[str, str]:
"""Upgrade one recognized V6 artifact when its immutable V5 source is unavailable."""
path = path.resolve()
try:
current = json.loads(path.read_bytes().decode("utf-8"))
marker = current["extra"][MIGRATION_SCHEMA]
except (OSError, UnicodeDecodeError, json.JSONDecodeError, KeyError, TypeError) as exc:
raise WorkflowError(f"Cannot read a recognized V6 artifact: {path}") from exc
if (
marker.get("version") != MIGRATION_VERSION
or marker.get("source_workflow_file_sha256") != SOURCE_V5_SHA256
or not bool(marker.get("source_is_never_overwritten"))
):
raise WorkflowError(f"Refusing to upgrade an unrecognized V6 artifact: {path}")
upgraded = migrate_workflow(current, source_file_sha256="already-owned-v6")
payload = _json_bytes(upgraded)
result = _write_new_or_verify(path, payload, replace_owned=True)
return result, _sha256_bytes(payload)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--source", type=Path, default=SOURCE_V5)
parser.add_argument("--desktop-output", type=Path, default=DEFAULT_DESKTOP_OUTPUT)
parser.add_argument("--user-output", type=Path, default=DEFAULT_USER_OUTPUT)
parser.add_argument("--check", action="store_true")
parser.add_argument("--replace-owned", action="store_true")
parser.add_argument(
"--upgrade-owned-v6",
action="store_true",
help="Upgrade both recognized active V6 copies without requiring the immutable V5 source.",
)
args = parser.parse_args(argv)
if args.upgrade_owned_v6:
desktop_result, desktop_hash = upgrade_owned_artifact(args.desktop_output)
user_result, user_hash = upgrade_owned_artifact(args.user_output)
if desktop_hash != user_hash:
raise WorkflowError("Upgraded Desktop and ComfyUI V6 copies are not byte-identical.")
print(
json.dumps(
{
"desktop": desktop_result,
"user": user_result,
"sha256": desktop_hash,
},
sort_keys=True,
)
)
return 0
migrated, payload, source_hash = build_artifact(args.source)
result_hash = _sha256_bytes(payload)
if args.check:
print(
f"checked source_sha256={source_hash} result_sha256={result_hash} "
f"canonical_sha256={_canonical_sha256(migrated)}"
)
return 0
actions = []
for path in (args.desktop_output, args.user_output):
actions.append(
f"{_write_new_or_verify(path, payload, replace_owned=args.replace_owned)}="
f"{path.resolve()}"
)
print(
f"source_sha256={source_hash} result_sha256={result_hash} "
+ " ".join(actions)
)
return 0
if __name__ == "__main__": # pragma: no cover
raise SystemExit(main())
tools/migrate_minimax_music_video_expansion.py
+2689
View File
File diff suppressed because it is too large Load Diff
tools/migrate_music_video_production.py
+2389
View File
File diff suppressed because it is too large Load Diff