Two new contract test files plus a few stale-name doc fixups so the
LTX-2 NVFP4 path can't silently regress.
* ``test_nvfp4_ltx2_wiring.py`` (6 tests): asserts that
- ``LTXSelfAttention``'s ``to_q/to_k/to_v/to_out`` are
``ReplicatedLinear`` (not plain ``nn.Linear``);
- ``NVFP4Config()`` attaches ``NVFP4QuantizeMethod`` to the
quantized subset and the ``layer_prefix`` matches the
``ltx2.blocks.<i>.<sub>.<proj>`` paths in
``NVFP4Config.fp4_layers``;
- non-tagged projections (cross-attn K/V, audio attn, audio FFN)
fall back to ``UnquantizedLinearMethod`` instead of crashing on
the quant_method assert;
- ``BasicAVTransformerBlock`` propagates ``quant_config`` and
``prefix`` correctly to all 4 attention modules + FFN at once.
These are CPU-only and don't need flashinfer.
* ``test_typed_quant_flow.py`` (4 tests): asserts that
- typed ``engine.quantization.transformer_quant: "NVFP4"`` resolves
through ``compat.py`` to a concrete ``NVFP4Config()`` instance;
- omitting the typed surface leaves the ``transformer_quant``
carrier ``None`` so legacy callers that mutate
``pipeline_config.dit_config.quant_config`` directly keep working;
- ``__post_init__._apply_transformer_quant`` pins the carrier onto
``dit_config.quant_config``;
- explicit ``dit_config.quant_config = …`` is preserved (the
explicit setter wins over the typed carrier).
Plus stale ``FP4Config`` → ``NVFP4Config`` doc updates in
``api/compat.py``, ``fastvideo_args.py``, and ``layers/linear.py``
that the previous rename commit missed (comments and docstrings only,
no behavior change).
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
The FP4 implementation is specifically NVIDIA's block-scaled FP4
(e2m1 mantissa, fp32 alpha, ``layout_128x4`` scale layout, group
size 16) backed by FlashInfer's ``nvfp4_quantize`` and ``mm_fp4``
kernels. Naming the public surface ``FP4`` would collide with other
FP4 variants we may want to support later (OCP-FP4 / MX-FP4 / e3m0).
Mechanical rename, no behavior change:
* ``fp4_config.py`` → ``nvfp4_config.py``
* ``FP4Config`` → ``NVFP4Config``; ``get_name()`` returns ``"nvfp4"``
* ``FP4QuantizeMethod`` → ``NVFP4QuantizeMethod``
* ``convert_model_to_fp4`` → ``convert_model_to_nvfp4``
* ``QuantizationMethods`` literal: ``"FP4"`` → ``"NVFP4"``
* registered buffer names: ``_fp4_weight``/``_fp4_alpha`` →
``_nvfp4_weight``/``_nvfp4_alpha``
* loader helper ``_maybe_convert_model_to_fp4`` →
``_maybe_convert_model_to_nvfp4``
* test file rename + symbol updates
* Dreamverse worker updates its single import + call site
Internal-scope torch op namespace ``fastvideo_fp4::*`` and
``_get_ltx2_fp4_stage_profile`` left as-is (purely internal naming
that mirrors FastVideo-internal).
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
Ports the FP4 quantization config + ops from FastVideo-internal
(fastvideo.layers.quantization.fp4_config). flashinfer is imported
lazily inside _require_flashinfer() rather than at module top so
``import fastvideo`` stays cheap on hosts that don't ship flashinfer
— the FP4 kernels themselves still raise a clear ImportError pointing
at ``pip install flashinfer-python`` when invoked.
Behavior is identical to the internal version:
* FP4Config: LTX-2 layer set (48 transformer blocks across attn1/attn2
/audio_to_video_attn/video_to_audio_attn/ffn plus adaln_single.linear),
layer_profile selects between "base" and "refine" stage subsets.
* FP4QuantizeMethod: per-layer quantize_input + apply with stage-aware
routing for refine-only layers (audio_to_video_attn.to_q,
video_to_audio_attn.to_k, video_to_audio_attn.to_v fall through to
dense F.linear during the base stage).
* convert_model_to_fp4(): pre-quantize weight buffers + alpha onto
module attrs at load time so the forward path doesn't pay the
per-step weight-side quantize cost.
Required for Dreamverse integration: dreamverse-server's GPU worker
runs ``pipeline_config.dit_config.quant_config = FP4Config()`` during
LTX-2 model load, which is the dominant inference path.
test_fp4_config covers the lazy-flashinfer contract: import succeeds
without flashinfer; layer_profile round-trips via from_config; calling
into _require_flashinfer raises ImportError with an actionable
message when flashinfer is missing.
Co-authored-by: Junda (David) Su <90978028+Davids048@users.noreply.github.com>
Co-authored-by: Matthew Noto <99706358+RandNMR73@users.noreply.github.com>
Co-authored-by: XOR-op <17672363+XOR-op@users.noreply.github.com>
Co-authored-by: Zhang Peiyuan <42993249+jzhang38@users.noreply.github.com>
