Files
hao-ai-lab-FastVideo/fastvideo
dc3a62e0f2 [fix] api: route LTX-2 audio kwargs through batch.extra; strict update
* Public `VideoGenerator.generate_video(**kwargs)` was silently dropping
  every LTX-2 audio-conditioning kwarg (`audio_num_frames`,
  `ltx2_audio_clean_latent`, `ltx2_audio_denoise_mask`,
  `video_position_offset_sec`, `ltx2_audio_latents`).
  `SamplingParam.update()` `logger.error`'d on unknown fields and
  continued, so `apply_audio()` for continuation segments was
  effectively a no-op: audio was generated for the default num_frames
  duration and `ltx2_audio_latents` was never carried forward across
  segments.

* The downstream symptom on Dreamverse was a confusing ffmpeg
  "frame writer failed: [Errno 32] Broken pipe" on segment 2: trimmed
  audio ended ~1.7s before trimmed video, ffmpeg `-shortest` closed
  stdin, the writer thread's next `proc.stdin.write(...)` hit EPIPE.

* Port the FastVideo-internal routing block into public:
  - `_BATCH_EXTRA_PASSTHROUGH_KEYS` enumerates the 5 LTX-2 audio kwargs.
  - `_generate_video_impl` extracts them from `**kwargs` BEFORE
    `sampling_param.update()` and stashes as `kwargs["_extra_overrides"]`.
  - `_generate_single_video` pops `_extra_overrides` and copies into
    `batch.extra` so `ltx2_denoising` / `ltx2_refine` see them.
  - Surface `output_batch.extra["ltx2_audio_latents"]` in the result
    dict so `Dreamverse._derive_next_audio_latents` can carry latents
    across segments.

* Convert `SamplingParam.update()` to raise `ValueError` on unknown
  keys instead of `logger.error`-and-continue. Future contributors who
  add a pipeline-specific kwarg without registering it will hit a loud
  failure pointing at `_BATCH_EXTRA_PASSTHROUGH_KEYS` rather than
  debug a broken-pipe-shaped symptom hours later.

* Add `fastvideo/tests/api/test_extra_overrides_routing.py` (7 tests)
  pinning the passthrough-key set, the strict-update contract, and the
  error-message hint at the right routing mechanism.

Verified end-to-end on GPU 4 with the dreamverse-deploy skill: segment 2
streams 22 av chunks / 3.8MB clean, no BrokenPipeError, `Cached audio
latents shape=(1, 8, 126, 16)` log line confirms latents now carry.
185/185 fastvideo/tests/api/ tests pass.

Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
2026-05-07 13:37:26 -07:00
..
2026-01-05 15:47:05 -06:00