Multi-node FastH3/Flex QAD runs outlast the default 10-minute collective timeout
while rank 0 loads and scatters the teacher and student; the OmniRef QAD carried
this as an uncommitted patch.
theseus_steps > 0 trains teacher-forced segment reconstruction first, widening
from single blocks to the whole stack (JustQuant, arXiv:2609.33601), then hands off
to the configured objective.
Decode each reference from its stored latents with the H3 VAE / audio VAE, then
prepare and re-encode it at Flex's tier sizing; compose the new cases from rows of
the same asset.
Re-encode references and Qwen3-VL conditioning with Flex's tier sizing (1024 short edge
below 768p) and add audio-reference, mixed image+video, 6-9 image and two-video rows,
reusing the v4 target latents.
Port the FastH3 OmniRef static-scale calibration onto the Flex QAD harness: per-row
FFN input amax on the teacher's six-forward trajectory, clip candidates, deployed
NVFP4 GEMM output error per candidate, and the per-layer MSE-optimal table.
Add MiniMaxH3FlexRef2VAModel / MiniMaxH3FlexFL2VAModel for OmniRefPDDQADMethod:
dense attention, Pika's shifted 1000..40 ladder, re-noise steps whose noise is a
function of (row seed, forward, modality), NVFP4 linears only. Add an offline
fp32 adapter merge script and QAD configs for both adapters. Let model profiles
pick their attention backend and receive the prepared row in schedulers().
Resolve the FP4 attention kernel and backend conflicts with main's #1980 versions
(re-adding install_fp4_vsa_attention), keep main's FP32 FSDP grouping, and keep both
save_inference and save_tagged in the checkpointer.
Three source recipes (T2VA, keyframes, Ref2VA) on one model page under
MiniMax H3, with knobs for task, resolution tier, aspect bucket,
duration, NVFP4 decoder and text encoder, offload mode, and GPU count.
The page uses the official Pika-Labs Hugging Face avatar.
The runbook now gives the FASTVIDEO_ATTENTION_BACKEND=FLASH_ATTN
FASTVIDEO_FA4=1 launch prefix and the SDPA fallback, the tier override,
and measured GB200 timings and memory. The example selects the bucket
with --resolution/--aspect, the frame count with --duration, and
--offload resident|lazy.
An adapter that trains a subset of the wrapped linears (Pika skips the
AdaLN projections) logged one warning per layer, about 200 per run.
Log one summary with the count and a few names.
The sampling param and presets fill 768x1344 before input preparation, so
an exact p768 bucket always won and the configured tier never applied.
Resolve the omitted canvas in apply_request_constraints, where explicit
request fields are still distinguishable; explicit sizes keep their tier
and one-sided sizes are rejected.
Applying the automatic Pika adapter inside the transformer's lazy load ran
_drop_pinned_host_copies, which read __dict__ through every LazyModule
proxy and reloaded the already-released text encoder and both VAEs.
Skip unmaterialized proxies and clear the real module of resident ones.
Expose the exported H3 NVFP4 VAE decoder through ComponentConfig and map it to FastVideoArgs, alongside the existing text-encoder weight override. Add example flags and docs for the 36L quality and 8L fast decoder tiers, including the 8L depth-cut and no-NVFP4-VAE-encoder caveats. Add compat translation tests.
The constructor LoRA adapter (Pika H3 Flex's released adapter) reads
param_names_mapping through the LazyModule proxy while the materialize
transform is still running. Nothing was published yet, so the access
re-entered materialize and reloaded the 66 GB transformer repeatedly,
ending in CUDA OOM with lazy_module_load=True (the example default).
Publish the loaded instance before running the transform.
Remove the unsupported pipeline.preset selection from the example; the compatibility adapter rejects it before model loading. Load the last image as a PIL image because the request adapter does not do it for the stage. Document that the local Ref2VA path enforces reference counts and the 22-frame video floor, but not Pika's hosted per-clip/per-modality duration caps.
Add Pika H3 Flex (MiniMax-H3 plus the released fl2va/ref2va rank-128 adapters) as a FastVideo model family: registry/config/presets, bucket geometry with tiered reference sizing, automatic adapter selection and manifest validation, the fixed six-forward re-noise sampler, T2VA/I2VA/L2VA/FL2VA/Ref2VA pipeline support, an example, docs, and CPU unit tests.
GPU end-to-end validation is still pending. This change does not add PDD, VSA, or new kernel paths.
BaseLayerWithLoRA.forward was decorated with torch.compile, so wrapping a linear also compiled its base layer.
Compiling a packed NVFP4/FP8 linear on its own changes its numerics, so a wrapped layer at strength 0 could differ
from the unwrapped layer. forward now adds no compiled region: the base layer runs exactly as it would unwrapped
(eagerly, or traced with a caller's compiled block), and only the adapter delta is compiled.
The unit lane fails on main in two contract tests. All three failures are
bookkeeping that drifted from the code, not runtime bugs:
- `test_env_policy.py::test_env_doc_table_matches_registry`: the generated
table in docs/contributing/env_vars.md is stale for
FASTVIDEO_MLX_QUANT_GEMM. Regenerated with the documented command
(`python fastvideo/tests/contract/test_env_policy.py`).
- `test_env_policy.py::test_env_access_follows_policy`: two new violations.
- fastvideo/tests/mlx/test_mlx_affine_dq_gemm.py wrote
FASTVIDEO_MLX_QUANT_GEMM through monkeypatch.setenv; now uses the
registered field's override() through the env_overrides fixture.
- fastvideo/tests/train/models/test_load_kandinsky5.py read
FASTVIDEO_ATTENTION_BACKEND with `not in os.environ`; now uses
envs.FASTVIDEO_ATTENTION_BACKEND.is_set() and override(). Its entry in
KNOWN_VIOLATIONS is deleted, as the shrinking-list rule requires.
- `test_schema_parity_inventory.py::..._extension_fields_are_classified`:
MiniMaxH3PipelineConfig declares video_scheduler_shift and
audio_scheduler_shift (training-time noise clocks rejected by
check_pipeline_config at inference); both are added to
pipeline_config_extensions.internal_only.
Test plan (CPU, no GPU required):
python -m pytest fastvideo/tests/contract/test_env_policy.py \
fastvideo/tests/api/test_schema_parity_inventory.py \
fastvideo/tests/mlx/test_mlx_affine_dq_gemm.py -q
# 45 passed
python -m pytest fastvideo/tests/contract/ fastvideo/tests/api/ -q -m "not gpu"
# 501 passed
- Allow the frozen FP32 boundary modules through the FSDP mixed-dtype
guard when a model declares replicate_fp32_modules.
- Make the tile-64 FP4 kernel and its reference handle a listed block
with an empty query half instead of propagating NaN; add a CPU
regression test.
- Compare generated row ids in the row-plan test.
- Consume method checkpoint/stop requests after the pre-loop eval, so
best/ always refers to the evaluated weights.
- Remove the unused audio-stats module and its claim.
- Refresh the V2 model description and drop the dead compression script
reference.
- V2 config now matches the canonical 16-GPU stage 1: warmup 10,
min LR ratio 0.3, HSDP 16.
- State that the published V2.1 checkpoint is linears-only (BF16 VSA
attention); the FP4 adaptation is a separate, non-public stage.
- Name the released checkpoints and starting models.
- Vendor and pin both calibrated amax tables.
- Add the V2 prompt/embedding and OmniRef eval data-prep scripts.
Aryan Kumar