fp4_dense_qat routes FlashAttention layers through the FP4 training kernel with every
key block listed (Q/K NVFP4, P and V BF16, straight-through gradients), matching the
FlashAttention-4 NVFP4 deployment; qad.fp4_attention enables it in the Flex profile.
check_fp4_attention_parity.py compares it with the real FA4 NVFP4 kernel on a GPU.
The FL2VA profile reads keyframe-anchor rows (build_packed_sequence anchors, noised
like inference); the coverage builder's --mode fl2va makes them from the v4 rows'
exact first/last frames re-encoded at the canvas size with the FL2VA Qwen3-VL
presentation; the keyframe-fidelity guard covers the FL2VA tasks.
Decode each reference from its stored latents with the H3 VAE / audio VAE, then
prepare and re-encode it at Flex's tier sizing; compose the new cases from rows of
the same asset.
Re-encode references and Qwen3-VL conditioning with Flex's tier sizing (1024 short edge
below 768p) and add audio-reference, mixed image+video, 6-9 image and two-video rows,
reusing the v4 target latents.
Port the FastH3 OmniRef static-scale calibration onto the Flex QAD harness: per-row
FFN input amax on the teacher's six-forward trajectory, clip candidates, deployed
NVFP4 GEMM output error per candidate, and the per-layer MSE-optimal table.
Add MiniMaxH3FlexRef2VAModel / MiniMaxH3FlexFL2VAModel for OmniRefPDDQADMethod:
dense attention, Pika's shifted 1000..40 ladder, re-noise steps whose noise is a
function of (row seed, forward, modality), NVFP4 linears only. Add an offline
fp32 adapter merge script and QAD configs for both adapters. Let model profiles
pick their attention backend and receive the prepared row in schedulers().
Resolve the FP4 attention kernel and backend conflicts with main's #1980 versions
(re-adding install_fp4_vsa_attention), keep main's FP32 FSDP grouping, and keep both
save_inference and save_tagged in the checkpointer.
- V2 config now matches the canonical 16-GPU stage 1: warmup 10,
min LR ratio 0.3, HSDP 16.
- State that the published V2.1 checkpoint is linears-only (BF16 VSA
attention); the FP4 adaptation is a separate, non-public stage.
- Name the released checkpoints and starting models.
- Vendor and pin both calibrated amax tables.
- Add the V2 prompt/embedding and OmniRef eval data-prep scripts.
Aryan Kumar