Commit Graph
46 Commits
Author SHA1 Message Date
William LinandClaude Opus 5.5 500ba51680 [docs] PAN2: review follow-ups
- Guardrail install points to ifm-ai/pan2-guardrail, as in the vLLM-Omni and Diffusers ports.
- The basic example runs the release's own text-to-video structured caption (10 s, 241 frames) instead of a plain prompt.
- Docs and the flow-shift comment name the release checkpoint IFM/PAN2; the docs and cookbook no longer describe its release status or the parity tests' checkpoint.
- FASTVIDEO_FA4 is set only while the PAN2 pipeline loads, instead of for the rest of the process.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 20:31:55 +00:00
William LinandClaude Opus 5.5 946094bc77 [feat] PAN2: default to the released negative prompt; align serving and cookbook with CFG 2.5
An empty negative prompt now selects the release's structured negative caption with its time ranges set to the
clip's duration, as the reference sampler does; a single space still gives the empty unconditional prompt.
The serving config and cookbook now say guidance 2.5, matching the presets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-03 20:31:55 +00:00
William Lin 86ef6bfadf [perf] PAN2 sampling: Wan2.2 VAE decode kernel, split decode, fused DiT kernels
- fastvideo-kernel/wan22_vae_decode: the Wan2.2 (TI2V-5B) VAE decoder on sm_100a / sm_103a at 704x1248, built on
  first use against fastvideo-kernel's CUTLASS submodule. It frees intermediates early and carries 4 latent frames
  per pass: a 5 s video peaks at 20.3 GiB, down from 94.6 GiB in the kernel's first version, with the same output.
- WanVAEConfig.decode_kernel routes AutoencoderKLWan.decode through it where it applies (bf16 with fp32
  accumulation, about 7x faster than the fp32 decode on a B300, PSNR 61-63 dB, SSIM 0.998 against it);
  decode_kernel_parallel splits the decode over the sequence-parallel group (2, 4 or 8 GPUs) and broadcasts the
  video, with the ranks agreeing on whether every one of them could build the kernel.
- PAN2's DiT runs fused Triton kernels for Q/K RMSNorm + RoPE and LayerNorm + modulation (pan2_fusions,
  set_fused_kernels).
- PAN2PipelineConfig turns all three on (vae_decode_kernel, vae_decode_kernel_parallel, dit_fused_kernels).
- VideoGenerator copies a decoded RGB video's frames to the host once instead of once per frame.
- The PAN2 presets sample at guidance 2.5, the reference launcher's default since 2026-10-23.
2026-11-03 20:31:55 +00:00
William Lin fa4e3659c7 [new-model] Re-add PAN2 text-to-video and image-to-video (reverts #2025)
This reverts commit 6743f4b4b, restoring #1961.
2026-11-03 20:31:55 +00:00
Aryan KumarandAryan Kumar ac81198d33 [new-model] Open-H3-Regenerate-2K: 768P H3 draft to 2560x1440, served on the OpenAI video routes (#2035)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-11-01 22:45:09 -08:00
Aryan Kumar 6743f4b4b7 Revert "[new-model] Add PAN2 text-to-video and image-to-video" (#2025) 2026-10-27 18:03:59 -07:00
William Linandaryan5v a0e6cf1f50 [new-model] Add PAN2 text-to-video and image-to-video (#1961)
Co-authored-by: aryan5v <email.aryan1005@gmail.com>
2026-10-25 20:20:00 -07:00
Aryan KumarandAryan Kumar 089a5a55a0 [feat] Cookbook serving, hardware catalog, and landing page (#1999)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-10-20 09:10:29 -07:00
Boyang ZHONG dc72a35f4e [bugfix] Return individual images for batched image API requests (#1917)
For n > 1, the image endpoints returned one preview grid instead of n images.
The endpoints now run one model batch, save each decoded sample as its own file,
and return one URL or base64 item for each image. This applies to generations
and edits.

A keyword-only return_samples field (default False) is added to OutputConfig,
SamplingParam and ForwardBatch. Batched image requests set it and do not build
preview frames. The worker keeps the sample tensor for these requests, and
non-output ranks clear the field. Defaults and positional arguments do not
change.
2026-10-07 15:06:02 -07:00
Boyang ZHONG 6275ac2c8e [bugfix] Validate image parameters before generation and uploads (#1916)
The image endpoints now validate size, output format and count before they
create output directories, save uploads or start generation. An invalid value
returns HTTP 400. Omitted or null values keep their defaults.

Behavior change: an explicit empty size or output format, and a count outside
1-10, now return HTTP 400. Before, the server replaced them with defaults or
clamped the count.
2026-10-07 14:52:15 -07:00
Boyang ZHONG 3907a69727 [bugfix] Preserve image output formats (#1915)
The OpenAI image endpoint defaults to a .jpg output path, but the shared
output-path helper treated every non-PNG image path as a directory, so JPEG and
WebP requests were written somewhere other than where the endpoint read them.
Recognized PNG/JPEG/WebP extensions are now preserved, and existing directories
are checked first so a directory named e.g. renders.jpg still receives its
output inside it. Collision handling and filename sanitization are unchanged.
2026-10-07 14:23:15 -07:00
8530fdc550 [feat] Add external-launcher executor for multi-node inference (#1746)
Co-authored-by: Will Lin <willlin@nvidia.com>
Co-authored-by: William Lin <8941107+SolitaryThinker@users.noreply.github.com>
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-10-07 04:46:26 -07:00
f06d824054 [feat]: add Kandinsky-6.0 TI2VA and VSR pipelines (#1924)
Co-authored-by: leffff <levnovitskiy@gmail.com>
Co-authored-by: Kirill <kykozlov@gmail.com>
Co-authored-by: shaoxiongduan <shaoxiongduan@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 10:50:17 -07:00
8444c0897a [feat] FastH3 Ref2VA PDD inference with reference-video VSA sparsity (#1907)
Co-authored-by: Davids048 <jundasu@ucsd.edu>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 11:28:07 -07:00
Junda Su 9edc8adf5f [misc] Move test-suite environment access into the registry and allowlists (#1898) 2026-09-30 00:15:39 -07:00
Aryan KumarandAryan Kumar 7f03e03dc6 [docs]: FastH3 V2 cookbook serving and MLX install guide (#1884)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-09-29 13:03:18 -07:00
Keith e90be598e5 [perf] MiniMax H3: return uint8 frames from the decode worker (#1828) 2026-09-24 18:08:56 -07:00
Leleand武垚乐 ba5e81083c [bugfix] Preserve video frames when decoded audio is shorter (#1857)
Signed-off-by: 武垚乐 <wuyaole@mininglamp.com>
Co-authored-by: 武垚乐 <wuyaole@mininglamp.com>
2026-09-24 17:20:37 -07:00
YZJF 76ce9c7fd6 [bugfix] Reject mismatched model names on image generation routes (#1881) 2026-09-24 17:19:29 -07:00
Raghav K d265a454bf [bugfix]: Fall back when large pinned output allocation fails (#1759) 2026-09-24 17:05:59 -07:00
c100c66578 [bugfix]: isolate cancelled streaming requests from GPU result readers (#1848)
Co-authored-by: Gxj230958 <222823329+Gxj230958@users.noreply.github.com>
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
2026-09-24 17:04:59 -07:00
sudhirpol522 8839ba8d4d [bugfix] Add OpenAI-compatible image generation endpoint (#1840) 2026-09-14 17:55:56 -07:00
sudhirpol522andSolitaryThinker 61b91220c0 [bugfix] Validate image response format before generation (#1841)
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
2026-09-14 17:01:40 -07:00
Lele 316f3876c2 [bugfix]: allow MiniMax H3 frame padding at the 15-second limit
Accept the causal-VAE-aligned 362-frame bucket (15.083 s) for 15-second H3 requests.

- Hoist MINIMAX_H3_MIN/MAX_ALIGNED_FRAMES next to align_num_frames and reuse them in the CUDA stage, the MLX runtime, and the LoRA example so the bound is consistent across entry points.
- State the accepted frame range in the error messages.
- Cover seconds="15" -> 362 at OpenAI admission and the MLX resolve_geometry bound.
- Update the Spark/openai cookbook frame caps from 345 to 362.

Known gaps: no GPU-lane coverage for the 362 bucket (SSIM runs 124 frames; the golden gate is a fixed-geometry fingerprint), and explicit num_frames=360 still hits the pre-existing grid gate.
2026-09-14 16:54:41 -07:00
Aryan KumarandAryan Kumar 26064193e2 [feat] Add an H3 server cookbook and prompt playground (#1798)
Co-authored-by: Aryan Kumar <aryan5v@users.noreply.github.com>
2026-08-31 22:14:33 -07:00
William Lin a159b63c67 [bugfix] Harden OpenAI serving after post-merge review (#1782) 2026-08-28 22:29:24 -07:00
William Lin 3987b9ddcd [feat] Align multimodal OpenAI serving APIs (#1781) 2026-08-28 10:09:58 -07:00
Raghav K fca45bc8e1 [perf] Quantize frames to uint8 on-device before the post-decode D->H copy (#1362) 2026-08-20 21:55:55 -05:00
William Lin 9028953625 [misc] yapf pass under CI's interpreter (3.12) + pin hook language_version (#1702) 2026-08-08 22:24:17 -07:00
Junda Su 755a4e4470 [new-model] Add LingBot-Video Dense and MoE/refiner T2V inference (#1595) 2026-07-18 20:18:16 -07:00
Mac LeeandSatyam Srivastava 6cbff73687 [bugfix]: skip unused output materialization (#1567)
Co-authored-by: Satyam Srivastava <srivastavasatyam53@gmail.com>
2026-07-16 04:03:07 +00:00
6392bd40a9 [feat] Dreamverse 10/14: Add serving API contracts (#1333)
Co-authored-by: SolitaryThinker <wlsaidhi@gmail.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
2026-05-14 10:07:01 -07:00
William Lin c5088670c8 [misc]: empty __init__.py files with no logic (#1346) 2026-05-13 13:09:40 -07:00
2aaeee2ab8 [feat] Improve API: streaming router (multi-replica load balancer + ws proxy) (#1286)
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
2026-05-05 03:00:08 -07:00
eb3a394224 [feat] Improve API: streaming auxiliaries (safety, rewrite, logger, mock) (#1284)
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
2026-05-05 00:14:34 -07:00
f673423b51 [feat] Improve API: streaming prompt enhancer with LLMProvider abstraction (#1258)
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
2026-05-04 13:44:40 -07:00
eb0a41528a [feat] Improve API: streaming server GpuPool + worker subprocess (#1257)
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
2026-05-04 12:56:31 -07:00
William Lin 95fd29e0cb [feat] Streaming WebSocket server skeleton (single generator + fMP4) (#1251) 2026-04-26 00:33:49 -07:00
William Lin e0dc5f2b0c [feat] Add typed LTX-2 continuation state and streaming session store (#1250) 2026-04-24 01:28:07 -07:00
William Lin 0e3529869c [feat] [5/n] Improve API: wire ServeConfig.default_request into OpenAI serving (#1237) 2026-04-17 13:26:18 -07:00
William Lin 145a3f166b [feat] [4/n] Improve API: refactor sampling param and merge with presets (#1234) 2026-04-16 14:10:02 -07:00
William Lin 88a5a933ab [feat] [3/n] Improve API: extend support to cli (#1226) 2026-04-14 15:20:47 -07:00
William Lin c591d6d2a6 [feat] [2/n] Improve API: add initial support in video_generator (#1220) 2026-04-06 10:33:54 -07:00
Ajay Anubolu e63b7d8ac4 [Feat] Added OpenAI-compatible API server and benchmark script (#1109) 2026-03-02 17:12:32 -05:00
IshanandJinzhe Pan 0aff68f51d [Feat] Add Stable Diffusion 3.5 (#1075)
Co-authored-by: Jinzhe Pan <eigensystem1318@gmail.com>
2026-02-10 14:31:36 +08:00
William Linandainsley 9370234294 [feat] Add gradio local inference demo (#847)
Co-authored-by: ainsley <jzhang2765@wisc.edu>
2025-10-26 07:01:33 -07:00