On GB10 the light H3 VAE's INT8 ConvRot overlay decodes 832x480x124 in
42 s and gains nothing from tile batching. The dense weights take 24 s
with one tile per decoder call, 14 s with 12 tiles per call and 9.8 s
compiled, at higher fidelity (77.7 vs 63.4 dB PSNR against an FP32
decode). FASTVIDEO_H3_VAE_INT8_OVERLAY=0 now skips the overlay (default
on, so other recipes are unchanged), and the single-Spark recipes use it
with FASTVIDEO_H3_VAE_TILE_BATCH=12 and a compiled VAE decoder.
FlashInfer's CUTLASS mm_fp4 is 2.7-3x slower than its cuDNN backend for
the H3 FFN projections on GB10 with identical outputs, so the recipes
select FASTVIDEO_NVFP4_MM_BACKEND=cudnn. FlashInfer autotuning was not
adopted: it made repeated calls non-deterministic.
One Spark, 832x480x124, seed 1234, warm calls: FastH3 Trim 121-122 s to
68 s (denoise 78 -> 58 s, decode 43 -> 9.5 s); FastH3 V2 139-141 s to
79 s. Repeated calls are bit-identical, which the previous V2 recipe was
not.
FP4 attention (FASTVIDEO_H3_VSA_FP4=1, built for sm_121a since #1598) is
documented for GB10 as opt-in and stays off: it is 24-33% faster in
denoise on V2 but changes the output (LPIPS 0.39-0.55 against BF16
attention), and the checkpoints were not trained for it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
_nvfp4_quantize copied every activation into a zero-padded buffer (F.pad
copies even when no rows are added) before FlashInfer quantization. For
the 128x4 swizzled layout FlashInfer already pads the scale factors to
whole row tiles and quantizes the logical rows exactly as it would the
padded copy, so quantize the input directly and keep the padded path as
a fallback when the returned scales do not span the padded tile.
On DGX Spark the pad accounted for about 2.7k device copies plus the
matching fills per FastH3 Trim denoise (about 6 s of 62 s). Quantized
bytes, scale factors and mm_fp4 outputs are bit-identical for the H3
projection shapes, and a FastH3 Trim 832x480x124 clip is bit-identical
with denoise at 72 s instead of 78 s.
The GB10 quantization fence stays: without it, identical seeded requests
still diverge with or without the pad copy, on FlashInfer 0.6.18 and
0.7.0. Its comment and the Spark guide now describe it without the
padded input. The off-Spark fence test skips on GB10, where the fence is
expected.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For n > 1, the image endpoints returned one preview grid instead of n images.
The endpoints now run one model batch, save each decoded sample as its own file,
and return one URL or base64 item for each image. This applies to generations
and edits.
A keyword-only return_samples field (default False) is added to OutputConfig,
SamplingParam and ForwardBatch. Batched image requests set it and do not build
preview frames. The worker keeps the sample tensor for these requests, and
non-output ranks clear the field. Defaults and positional arguments do not
change.
Accept the causal-VAE-aligned 362-frame bucket (15.083 s) for 15-second H3 requests.
- Hoist MINIMAX_H3_MIN/MAX_ALIGNED_FRAMES next to align_num_frames and reuse them in the CUDA stage, the MLX runtime, and the LoRA example so the bound is consistent across entry points.
- State the accepted frame range in the error messages.
- Cover seconds="15" -> 362 at OpenAI admission and the MLX resolve_geometry bound.
- Update the Spark/openai cookbook frame caps from 345 to 362.
Known gaps: no GPU-lane coverage for the 362 bucket (SSIM runs 124 frames; the golden gate is a fixed-geometry fingerprint), and explicit num_frames=360 still hits the pre-existing grid gate.
