[bugfix]: H3 encoder truncation review fixes: parity gate, override validation, key-filter bound

Review follow-ups for the layer-50 truncation of the Qwen3-VL conditioner:

- tests/local_tests/encoders/test_minimax_h3_qwen3_vl_parity.py compared all
  65 hidden states with equal lengths and zip(strict=True), so the gate
  failed deterministically once the production stack builds only 51. It now
  pins the production tuple to language_model.num_layers + 1 entries and
  compares every built state bit-exactly against the official value at the
  same index, which still catches a truncated stack that applies the final
  norm, and degrades to the original full comparison when the override is
  None.

- num_hidden_layers_override <= 0 was accepted: 0 built an empty decoder
  stack, and a negative value additionally left num_layers inconsistent with
  the built layers, making _is_above_the_tap drop every layer key so the
  conditioner loaded "successfully" with no transformer at all. The arch
  config __post_init__ now rejects non-positive overrides, both at
  construction and when update_model_arch re-validates a checkpoint config.

- _is_above_the_tap dropped any layer index >= num_layers, including indexes
  at or above the checkpoint's own num_hidden_layers that the full stack
  would reject as unexpected. The filter is now bounded to
  [num_layers, num_hidden_layers) so truncation only drops keys the full
  model would have accepted.

- The CPU truncation test now compares the whole shared hidden-state prefix,
  mirroring the production-loader parity gate, and covers the
  override == num_hidden_layers boundary. The conditioner's
  num_hidden_layers property documents that it reports the checkpoint's
  nominal depth, not the built depth.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

examples/inference/basic/basic_lingbotworld_fast.py

@@ -1,40 +0,0 @@
-from fastvideo import VideoGenerator
-
-OUTPUT_PATH = "video_samples_lingbotworld_fast"
-
-
-def main():
-    generator = VideoGenerator.from_pretrained(
-        "FastVideo/LingBot-World-Fast-Diffusers",
-        num_gpus=1,
-        use_fsdp_inference=False,  # set to True if GPU is out of memory
-        dit_cpu_offload=True,
-        vae_cpu_offload=False,
-        text_encoder_cpu_offload=True,
-        pin_cpu_memory=True,
-    )
-
-    prompt = (
-        "The video presents a soaring journey through a fantasy jungle. The "
-        "wind whips past the rider's blue hands gripping the reins, causing "
-        "the leather straps to vibrate. The ancient gothic castle approaches "
-        "steadily, its stone details becoming clearer against the backdrop of "
-        "floating islands and distant waterfalls.")
-    image_path = ("https://raw.githubusercontent.com/Robbyant/lingbot-world/"
-                  "main/examples/00/image.jpg")
-    action_path = "examples/inference/basic/lingbotworld_examples/00"
-
-    generator.generate_video(
-        prompt,
-        image_path=image_path,
-        action_path=action_path,
-        output_path=OUTPUT_PATH,
-        save_video=True,
-        num_frames=81,
-        height=480,
-        width=832,
-    )
-
-
-if __name__ == "__main__":
-    main()

fastvideo/configs/models/dits/__init__.py

@@ -18,13 +18,12 @@ from fastvideo.configs.models.dits.zimage import ZImageDiTConfig
 from fastvideo.configs.models.dits.hyworld import HYWorldConfig
 from fastvideo.configs.models.dits.kandinsky5 import Kandinsky5VideoConfig
 from fastvideo.configs.models.dits.lingbotworld2 import LingBotWorld2CausalFastVideoConfig
-from fastvideo.configs.models.dits.lingbotworld_fast import LingBotWorldFastVideoConfig
 from fastvideo.configs.models.dits.lingbot_video import LingBotVideoConfig
 
 __all__ = [
     "HunyuanVideoConfig", "HunyuanVideo15Config", "HunyuanGameCraftConfig", "WanVideoConfig", "DreamXWorldConfig",
     "DreamXWorldARConfig", "CosmosVideoConfig", "Cosmos25VideoConfig", "FluxDiTConfig", "Flux2Config",
     "LongCatVideoConfig", "LTX2VideoConfig", "HYWorldConfig", "Kandinsky5VideoConfig", "MagiHumanVideoConfig",
-    "StableAudioConfig", "GlmImageDiTConfig", "LingBotWorld2CausalFastVideoConfig", "LingBotWorldFastVideoConfig",
-    "LingBotVideoConfig", "MiniMaxH3Config", "ZImageDiTConfig", "MMAudioArchConfig", "MMAudioTransformerConfig"
+    "StableAudioConfig", "GlmImageDiTConfig", "LingBotWorld2CausalFastVideoConfig", "LingBotVideoConfig",
+    "MiniMaxH3Config", "ZImageDiTConfig", "MMAudioArchConfig", "MMAudioTransformerConfig"
 ]

fastvideo/configs/models/dits/lingbotworld_fast.py

@@ -1,33 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-from dataclasses import dataclass, field
-
-from fastvideo.configs.models.dits.base import DiTArchConfig, DiTConfig
-from fastvideo.configs.models.dits.lingbotworld2 import (
-    LingBotWorld2CausalFastArchConfig, )
-
-
-@dataclass
-class LingBotWorldFastArchConfig(LingBotWorld2CausalFastArchConfig):
-    """Arch config for the released LingBot-World-Fast checkpoint.
-
-    The tensor layout is identical to the LingBot World 2 causal-fast model, so
-    the parent's shapes and ``param_names_mapping`` are reused verbatim. Only
-    the sampling/attention-window values released with this checkpoint differ.
-    """
-
-    # The released `generate_fast.py` leaves `--local_attn_size` at -1, so
-    # self-attention stays global and the KV cache never evicts. That also makes
-    # `sink_size` unreachable, so it is deliberately not pinned here (the loader
-    # overwrites it from the checkpoint config either way).
-    local_attn_size: int = -1
-    # `chunk_size` and `timesteps_index` are absent from the checkpoint config,
-    # so these values are what the sampling loop actually runs with.
-    chunk_size: int = 3
-    timesteps_index: tuple[int, int, int, int] = (0, 179, 358, 679)
-
-
-@dataclass
-class LingBotWorldFastVideoConfig(DiTConfig):
-    arch_config: DiTArchConfig = field(default_factory=LingBotWorldFastArchConfig)
-
-    prefix: str = "Wan"

fastvideo/configs/models/encoders/minimax_h3_qwen3_vl.py

@@ -62,6 +62,15 @@ class MiniMaxH3Qwen3VLArchConfig(TextEncoderArchConfig):
     hidden_size: int = 5120
     intermediate_size: int = 25600
     num_hidden_layers: int = 64
+    # H3 conditions on one intermediate hidden state and reads nothing above it,
+    # so the remaining layers are built, weight-loaded and then discarded: 14
+    # layers, 13.7 GB in bf16. Building exactly this many leaves that hidden
+    # state bit-identical, because the tuple records each layer's *input*, so
+    # entry N is the output of layer N-1. Set to None to keep the full stack.
+    # Must equal MINIMAX_H3_TEXT_ENCODER_LAYER in
+    # fastvideo/pipelines/basic/minimax_h3/packing.py; a test pins them together
+    # rather than importing across the models -> pipelines boundary.
+    num_hidden_layers_override: int | None = 50
     num_attention_heads: int = 64
     num_key_value_heads: int = 8
     head_dim: int = 128
@@ -118,6 +127,16 @@ class MiniMaxH3Qwen3VLArchConfig(TextEncoderArchConfig):
     ])
 
     def __post_init__(self) -> None:
+        # Runs both at construction and after ``update_model_arch`` merges the
+        # checkpoint's config.json, so it also guards config-file overrides. A
+        # non-positive override would build no decoder layers at all, and a
+        # negative one would additionally make the surplus-key filter drop
+        # every ``language_model.layers.*`` checkpoint key, so the conditioner
+        # would "load" with no transformer stack and only fail at generation.
+        if self.num_hidden_layers_override is not None and self.num_hidden_layers_override < 1:
+            raise ValueError("MiniMax H3 Qwen3-VL num_hidden_layers_override must be a positive layer count "
+                             f"or None for the full stack; got {self.num_hidden_layers_override}.")
+
         rope_scaling = dict(self.rope_scaling or {})
         self.mrope_interleaved = bool(rope_scaling.get("mrope_interleaved", self.mrope_interleaved))
         if not self.mrope_interleaved:

fastvideo/configs/pipelines/__init__.py

@@ -8,7 +8,6 @@ from fastvideo.configs.pipelines.hunyuangamecraft import HunyuanGameCraftPipelin
 from fastvideo.configs.pipelines.hyworld import HYWorldConfig
 from fastvideo.configs.pipelines.kandinsky5 import Kandinsky5DMDConfig, Kandinsky5I2VConfig, Kandinsky5T2VConfig
 from fastvideo.configs.pipelines.lingbotworld2 import LingBotWorld2CausalFastI2V480PConfig
-from fastvideo.configs.pipelines.lingbotworld_fast import LingBotWorldFastI2V480PConfig
 from fastvideo.configs.pipelines.lingbot_video import LingBotVideoT2VConfig
 from fastvideo.configs.pipelines.matrixgame2 import MatrixGame2I2V480PConfig
 from fastvideo.configs.pipelines.matrixgame3 import MatrixGame3I2V720PConfig
@@ -23,7 +22,6 @@ __all__ = [
     "Hunyuan15T2V720PConfig", "WanT2V480PConfig", "WanI2V480PConfig", "WanT2V720PConfig", "WanI2V720PConfig",
     "SelfForcingWanT2V480PConfig", "LucyEditDevConfig", "CosmosConfig", "Cosmos25Config", "LTX2T2VConfig",
     "DreamXWorld5BCamPipelineConfig", "DreamXWorld5BARPipelineConfig", "HYWorldConfig", "Kandinsky5T2VConfig",
-    "Kandinsky5I2VConfig", "Kandinsky5DMDConfig", "LingBotWorld2CausalFastI2V480PConfig",
-    "LingBotWorldFastI2V480PConfig", "LingBotVideoT2VConfig", "MatrixGame2I2V480PConfig", "MatrixGame3I2V720PConfig",
-    "MMAudioV2AConfig", "get_pipeline_config_cls_from_name"
+    "Kandinsky5I2VConfig", "Kandinsky5DMDConfig", "LingBotWorld2CausalFastI2V480PConfig", "LingBotVideoT2VConfig",
+    "MatrixGame2I2V480PConfig", "MatrixGame3I2V720PConfig", "MMAudioV2AConfig", "get_pipeline_config_cls_from_name"
 ]

fastvideo/configs/pipelines/lingbotworld_fast.py

@@ -1,23 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-from dataclasses import dataclass, field
-
-from fastvideo.configs.models import DiTConfig, EncoderConfig
-from fastvideo.configs.models.dits.lingbotworld_fast import (
-    LingBotWorldFastVideoConfig, )
-from fastvideo.configs.models.encoders import T5Config
-from fastvideo.configs.pipelines.lingbotworld2 import (
-    LingBotWorld2CausalFastI2V480PConfig, )
-
-
-@dataclass
-class LingBotWorldFastI2V480PConfig(LingBotWorld2CausalFastI2V480PConfig):
-    """Pipeline config for LingBot-World-Fast 480P image-to-video.
-
-    Shares the LingBot World 2 causal-fast sampling loop and Wan VAE, but this
-    checkpoint ships the stock ``UMT5EncoderModel`` text encoder (``d_model``
-    fields) rather than LingBot World 2's custom one (``dim`` fields), so the
-    standard ``T5Config`` is restored here alongside this DiT's arch config.
-    """
-
-    dit_config: DiTConfig = field(default_factory=LingBotWorldFastVideoConfig)
-    text_encoder_configs: tuple[EncoderConfig, ...] = field(default_factory=lambda: (T5Config(), ))

fastvideo/models/dits/lingbotworld2/causal_fast.py

@@ -389,11 +389,7 @@ class WanCrossAttention(CausalWanSelfAttention):
         else:
             k = self.norm_k(self.k(context)).view(b, -1, n, d)
             v = self.v(context).view(b, -1, n, d)
-        # Dispatch through `attention` so the TORCH_SDPA backend this model
-        # advertises actually works; self-attention already does the same. The
-        # SDPA path ignores `k_lens`, which is safe here because the caller
-        # always passes `context_lens=None`.
-        x = attention(q, k, v, k_lens=context_lens)
+        x = flash_attention(q, k, v, k_lens=context_lens)
         return self.o(x.flatten(2))
 
 

fastvideo/models/encoders/minimax_h3_qwen3_vl.py

@@ -227,10 +227,21 @@ class MiniMaxH3Qwen3VLLanguageModel(nn.Module):
             org_num_embeddings=config.vocab_size,
             quant_config=quant_config,
         )
+        # Build only as far as the consumer reads. The hidden-state tuple records
+        # each layer's input, so stopping after N layers still yields entry N,
+        # the output of layer N-1, unchanged. Everything above it exists only to
+        # feed `last_hidden_state`, which nothing consumes.
+        override = config.num_hidden_layers_override
+        self.num_layers = (config.num_hidden_layers
+                           if override is None else min(config.num_hidden_layers, override))
         self.layers = nn.ModuleList(
             MiniMaxH3Qwen3VLTextDecoderLayer(config, prefix=f"{config.prefix}.language_model.layers.{index}")
-            for index in range(config.num_hidden_layers))
-        self.norm = RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+            for index in range(self.num_layers))
+        # The final norm sits above the tapped layer, so a truncated stack drops
+        # it. Keeping it would overwrite the tapped entry with a normalised
+        # tensor and change conditioning without raising anything.
+        self.norm = (RMSNorm(config.hidden_size, eps=config.rms_norm_eps)
+                     if self.num_layers == config.num_hidden_layers else None)
         self.rotary_emb = MiniMaxH3Qwen3VLTextRotaryEmbedding(config)
 
     def forward(
@@ -258,7 +269,10 @@ class MiniMaxH3Qwen3VLLanguageModel(nn.Module):
                 visual = deepstack_visual_embeds[layer_index].to(hidden_states.device, hidden_states.dtype)
                 updated = hidden_states[mask].clone() + visual
                 hidden_states[mask] = updated
-        hidden_states = self.norm(hidden_states)
+        if self.norm is not None:
+            hidden_states = self.norm(hidden_states)
+        # Truncated or not, the last entry is appended here, so the tapped index
+        # lands in the same place either way.
         if all_hidden_states is not None:
             all_hidden_states += (hidden_states, )
         return BaseEncoderOutput(last_hidden_state=hidden_states, hidden_states=all_hidden_states)
@@ -516,6 +530,13 @@ class MiniMaxH3Qwen3VLConditioner(TextEncoder):
 
     @property
     def num_hidden_layers(self) -> int:
+        """The checkpoint architecture's nominal depth, matching its config.json.
+
+        When ``num_hidden_layers_override`` truncates the stack at the
+        conditioning tap, fewer layers exist; the built count is
+        ``self.language_model.num_layers``, and the hidden-state tuple has
+        ``num_layers + 1`` entries, not ``num_hidden_layers + 1``.
+        """
         return self.config.num_hidden_layers
 
     def _get_rope_index(
@@ -694,6 +715,31 @@ class MiniMaxH3Qwen3VLConditioner(TextEncoder):
         outputs.attention_mask = attention_mask
         return outputs
 
+    def _is_above_the_tap(self, name: str) -> bool:
+        """Whether this checkpoint key belongs to a layer we did not build.
+
+        A truncated language stack still ships every layer in the checkpoint, and
+        the unexpected-key check below is strict on purpose, so the surplus keys
+        have to be dropped here rather than by relaxing it.
+        """
+        language_model = self.language_model
+        # The final norm is dropped exactly when the stack is truncated, so its
+        # absence is the signal.
+        if language_model.norm is not None:
+            return False
+        if name == "language_model.norm.weight":
+            return True
+        prefix = "language_model.layers."
+        if not name.startswith(prefix):
+            return False
+        index = name[len(prefix):].split(".", 1)[0]
+        if not index.isdigit():
+            return False
+        # Only drop indexes the full stack would have built. Anything at or
+        # above the checkpoint's own num_hidden_layers is corrupt and must
+        # still raise below, exactly as it does without truncation.
+        return language_model.num_layers <= int(index) < self.config.num_hidden_layers
+
     def load_weights(self, weights: Iterable[tuple[str, torch.Tensor]]) -> set[str]:
         parameters = dict(self.named_parameters())
         loaded: set[str] = set()
@@ -702,6 +748,8 @@ class MiniMaxH3Qwen3VLConditioner(TextEncoder):
             if source_name == "lm_head.weight":
                 continue
             name = source_name[6:] if source_name.startswith("model.") else source_name
+            if self._is_above_the_tap(name):
+                continue
             if name not in parameters:
                 raise ValueError(f"Unexpected MiniMax-H3 Qwen3-VL checkpoint key: {source_name}")
             parameter = parameters[name]

fastvideo/models/registry.py

@@ -61,13 +61,6 @@ _IMAGE_TO_VIDEO_DIT_MODELS = {
         "lingbotworld2",
         "LingBotWorld2CausalFastTransformer3DModel",
     ),
-    # LingBot-World-Fast ships this class name; its tensor layout is identical
-    # to the LingBot World 2 causal-fast DiT.
-    "CausalLingBotWorldTransformer3DModel": (
-        "dits",
-        "lingbotworld2",
-        "LingBotWorld2CausalFastTransformer3DModel",
-    ),
     "MatrixGame2WanModel": ("dits", "matrixgame2", "MatrixGame2WanModel"),
     "CausalMatrixGame2WanModel": ("dits", "matrixgame2", "CausalMatrixGame2WanModel"),
     # Legacy aliases for older HF model_index.json files

fastvideo/pipelines/basic/lingbotworld_fast/__init__.py

@@ -1,5 +0,0 @@
-from .fast_pipeline import LingBotWorldFastPipeline
-
-__all__ = ["LingBotWorldFastPipeline"]
-
-EntryClass = LingBotWorldFastPipeline

fastvideo/pipelines/basic/lingbotworld_fast/fast_pipeline.py

@@ -1,38 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-"""LingBot-World-Fast causal image-to-video pipeline."""
-
-from fastvideo.fastvideo_args import FastVideoArgs
-from fastvideo.logger import init_logger
-from fastvideo.models.schedulers.scheduling_flow_unipc_multistep import (
-    FlowUniPCMultistepScheduler, )
-from fastvideo.pipelines.basic.lingbotworld2.causal_fast_pipeline import (
-    LingBotWorld2CausalFastPipeline, )
-
-logger = init_logger(__name__)
-
-
-class LingBotWorldFastPipeline(LingBotWorld2CausalFastPipeline):
-    """LingBot-World-Fast I2V generation.
-
-    The released checkpoint uses the same chunked causal sampling loop as
-    LingBot World 2 causal-fast; the chunk size, timestep indices, and
-    attention window come from ``LingBotWorldFastArchConfig``.
-    """
-
-    def initialize_pipeline(self, fastvideo_args: FastVideoArgs) -> None:
-        """Install the flow-matching scheduler the released model samples with.
-
-        This checkpoint ships a stock ``UniPCMultistepScheduler``, whose
-        ``set_timesteps`` takes no ``shift``. The reference ``generate_fast.py``
-        builds a ``FlowUniPCMultistepScheduler`` instead, so replace the loaded
-        one unconditionally rather than only when absent.
-        """
-        arch_config = fastvideo_args.pipeline_config.dit_config.arch_config
-        self.modules["scheduler"] = FlowUniPCMultistepScheduler(
-            num_train_timesteps=arch_config.num_train_timesteps,
-            shift=1,
-            use_dynamic_shifting=False,
-        )
-
-
-EntryClass = LingBotWorldFastPipeline

fastvideo/pipelines/basic/lingbotworld_fast/presets.py

@@ -1,35 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-"""LingBot-World-Fast pipeline preset."""
-
-from fastvideo.api.presets import InferencePreset, PresetStageSpec
-
-_DENOISE_STAGE = PresetStageSpec(
-    name="denoise",
-    kind="denoising",
-    description="Causal-fast denoising pass",
-    allowed_overrides=frozenset({
-        "num_inference_steps",
-        "guidance_scale",
-    }),
-)
-
-LINGBOTWORLD_FAST_I2V = InferencePreset(
-    name="lingbotworld_fast_i2v",
-    version=1,
-    model_family="lingbotworld_fast",
-    description="LingBot-World-Fast 14B causal I2V",
-    workload_type="i2v",
-    stage_schemas=(_DENOISE_STAGE, ),
-    defaults={
-        "guidance_scale": 1.0,
-        "num_inference_steps": 4,
-        "fps": 16,
-        "seed": 42,
-        "num_frames": 81,
-        "height": 480,
-        "width": 832,
-        "negative_prompt": "",
-    },
-)
-
-ALL_PRESETS = (LINGBOTWORLD_FAST_I2V, )

fastvideo/registry.py

@@ -32,7 +32,6 @@ from fastvideo.configs.pipelines.kandinsky5 import Kandinsky5I2VConfig, Kandinsk
 from fastvideo.configs.pipelines.lingbot_video import LingBotVideoT2VConfig
 from fastvideo.configs.pipelines.lingbotworld import LingBotWorldI2V480PConfig
 from fastvideo.configs.pipelines.lingbotworld2 import LingBotWorld2CausalFastI2V480PConfig
-from fastvideo.configs.pipelines.lingbotworld_fast import LingBotWorldFastI2V480PConfig
 from fastvideo.configs.pipelines.longcat import LongCatT2V480PConfig
 from fastvideo.pipelines.basic.ltx2.pipeline_configs import LTX2T2VConfig
 from fastvideo.configs.pipelines.flux_2 import (
@@ -527,27 +526,6 @@ def _register_configs() -> None:
         default_preset="lingbotworld2_causal_fast_i2v",
     )
 
-    # LingBotWorld-Fast — registered BEFORE the LingBotWorld base entry so its
-    # detector wins when both fire. The base detector only excludes the
-    # "causal-fast" spelling, so it also matches this checkpoint's path and its
-    # `lingbotworldcausaldmdpipeline` model_index name; the detector loop keeps
-    # the first match, which must be this more specific entry.
-    register_configs(
-        sampling_param_cls=None,
-        pipeline_config_cls=LingBotWorldFastI2V480PConfig,
-        workload_types=(WorkloadType.I2V, ),
-        hf_model_paths=[
-            "FastVideo/LingBot-World-Fast-Diffusers",
-        ],
-        model_detectors=[
-            lambda path: ("lingbot-world-fast" in path.lower() or "lingbotworldfast" in path.lower() or
-                          "lingbotworldcausaldmdpipeline" in path.lower())
-        ],
-        model_family="lingbotworld_fast",
-        default_preset="lingbotworld_fast_i2v",
-        pipeline_cls_name="LingBotWorldFastPipeline",
-    )
-
     # LingBotWorld
     register_configs(
         sampling_param_cls=None,
@@ -557,9 +535,8 @@ def _register_configs() -> None:
             "FastVideo/LingBot-World-Base-Cam-Diffusers",
         ],
         model_detectors=[
-            lambda path:
-            (("lingbotworld" in path.lower() or "lingbot-world" in path.lower()) and "causal-fast" not in path.lower()
-             and "causalfast" not in path.lower() and "-fast" not in path.lower() and "causaldmd" not in path.lower())
+            lambda path: (("lingbotworld" in path.lower() or "lingbot-world" in path.lower()) and "causal-fast" not in
+                          path.lower() and "causalfast" not in path.lower())
         ],
         model_family="lingbotworld",
         default_preset="lingbotworld_i2v",
@@ -1330,8 +1307,6 @@ def _register_presets() -> None:
         ALL_PRESETS as LINGBOTWORLD_PRESETS, )
     from fastvideo.pipelines.basic.lingbotworld2.presets import (
         ALL_PRESETS as LINGBOTWORLD2_PRESETS, )
-    from fastvideo.pipelines.basic.lingbotworld_fast.presets import (
-        ALL_PRESETS as LINGBOTWORLD_FAST_PRESETS, )
     from fastvideo.pipelines.basic.lingbot_video.presets import (
         ALL_PRESETS as LINGBOT_VIDEO_PRESETS, )
     from fastvideo.pipelines.basic.longcat.presets import (
@@ -1372,7 +1347,6 @@ def _register_presets() -> None:
         LINGBOT_VIDEO_PRESETS,
         LINGBOTWORLD_PRESETS,
         LINGBOTWORLD2_PRESETS,
-        LINGBOTWORLD_FAST_PRESETS,
         LONGCAT_PRESETS,
         LTX2_PRESETS,
         MATRIXGAME2_PRESETS,

fastvideo/tests/encoders/test_minimax_h3_qwen3_vl_truncation.py

@@ -0,0 +1,207 @@
+# SPDX-License-Identifier: Apache-2.0
+"""The Qwen3-VL stack is built only as far as MiniMax H3 reads.
+
+H3 conditions on one intermediate hidden state. The layers above it were built,
+weight-loaded and then discarded, which is 13.7 GB in bf16 and the difference
+between fitting and not fitting on a 121 GB unified-memory device.
+
+The dangerous part is not the truncation, it is getting the tuple index wrong.
+`hidden_states` records each layer's *input*, so entry N is the output of layer
+N-1, and the final entry comes from the norm that sits above the whole stack. A
+truncated stack that still applies that norm puts a normalised tensor where the
+raw one belongs: the length check in the conditioning stage still passes, and
+conditioning silently changes. These tests pin the index, the content, and the
+constant the two sides agree on.
+"""
+from __future__ import annotations
+
+import os
+
+import pytest
+import torch
+
+# Matches the other encoder tests: the module registry these build against wants
+# a process group, and a single-rank one needs a rendezvous address.
+os.environ.setdefault("MASTER_ADDR", "localhost")
+os.environ.setdefault("MASTER_PORT", "29513")
+
+from fastvideo.configs.models.encoders.minimax_h3_qwen3_vl import (
+    MiniMaxH3Qwen3VLArchConfig,
+    MiniMaxH3Qwen3VLConfig,
+)
+from fastvideo.models.encoders.minimax_h3_qwen3_vl import MiniMaxH3Qwen3VLLanguageModel
+from fastvideo.pipelines.basic.minimax_h3.packing import MINIMAX_H3_TEXT_ENCODER_LAYER
+
+
+def _small_arch(**overrides) -> MiniMaxH3Qwen3VLArchConfig:
+    """A stack small enough to run on CPU but shaped like the real one.
+
+    Everything goes through the constructor so ``__post_init__`` validates the
+    small shape the same way it validates the real one.
+    """
+    kwargs: dict = dict(
+        vocab_size=64,
+        hidden_size=16,
+        intermediate_size=32,
+        num_hidden_layers=8,
+        num_attention_heads=2,
+        num_key_value_heads=1,
+        head_dim=8,
+        # __post_init__ reads the sections out of rope_scaling, and they must
+        # cover exactly half of each head.
+        rope_scaling={
+            "mrope_interleaved": True,
+            "mrope_section": [2, 1, 1],
+            "rope_type": "default",
+        },
+        vision_out_hidden_size=16,
+    )
+    kwargs.update(overrides)
+    return MiniMaxH3Qwen3VLArchConfig(**kwargs)
+
+
+def _small_config(**overrides) -> MiniMaxH3Qwen3VLConfig:
+    """The outer config, which is what the modules take.
+
+    ``ModelConfig.__getattr__`` forwards the architecture fields, so the modules
+    read ``prefix`` off this object and everything else off ``arch_config``.
+    """
+    config = MiniMaxH3Qwen3VLConfig()
+    config.arch_config = _small_arch(**overrides)
+    return config
+
+
+def test_default_matches_the_index_the_pipeline_reads() -> None:
+    """The two sides cannot import each other, so pin them here instead.
+
+    `fastvideo/models/` must not import from `fastvideo/pipelines/`, so the tap
+    is written down twice. If they drift, conditioning reads a hidden state that
+    was never built and the run dies with an index error at generation time,
+    after a full model load.
+    """
+    assert MiniMaxH3Qwen3VLArchConfig().num_hidden_layers_override == MINIMAX_H3_TEXT_ENCODER_LAYER
+
+
+def test_builds_only_up_to_the_override(distributed_setup) -> None:
+    model = MiniMaxH3Qwen3VLLanguageModel(_small_config(num_hidden_layers_override=5))
+
+    assert model.num_layers == 5
+    assert len(model.layers) == 5
+    # The norm sits above the tap, so a truncated stack must not keep it.
+    assert model.norm is None
+
+
+def test_override_none_keeps_the_full_stack(distributed_setup) -> None:
+    model = MiniMaxH3Qwen3VLLanguageModel(_small_config(num_hidden_layers_override=None))
+
+    assert model.num_layers == 8
+    assert model.norm is not None
+
+
+def test_override_above_the_stack_does_not_over_build(distributed_setup) -> None:
+    # num_hidden_layers comes from the checkpoint's config.json via
+    # update_model_arch, so a smaller variant must clamp rather than ask for
+    # layers that do not exist.
+    model = MiniMaxH3Qwen3VLLanguageModel(_small_config(num_hidden_layers_override=99))
+
+    assert model.num_layers == 8
+    assert model.norm is not None
+
+
+def test_override_equal_to_the_stack_keeps_the_norm(distributed_setup) -> None:
+    """The exact boundary of the clamp: a stack cut at its own depth is full.
+
+    A checkpoint with exactly ``override`` layers taps its final layer, whose
+    tuple entry sits after the norm in the full model, so the norm must stay
+    and nothing may be filtered from the checkpoint.
+    """
+    model = MiniMaxH3Qwen3VLLanguageModel(_small_config(num_hidden_layers_override=8))
+
+    assert model.num_layers == 8
+    assert model.norm is not None
+
+
+def test_non_positive_override_is_rejected() -> None:
+    """A non-positive override would build no decoder layers at all.
+
+    Worse, a negative one makes ``num_layers`` disagree with the built stack
+    and the surplus-key filter would then drop every layer key, so the
+    conditioner would load "successfully" with no transformer. Reject it at
+    config construction, and again when update_model_arch re-validates.
+    """
+    for override in (0, -1):
+        with pytest.raises(ValueError, match="num_hidden_layers_override"):
+            _small_arch(num_hidden_layers_override=override)
+
+    config = _small_config()
+    with pytest.raises(ValueError, match="num_hidden_layers_override"):
+        config.update_model_arch({"num_hidden_layers_override": 0})
+
+
+def test_tapped_hidden_state_is_unchanged_by_truncation(distributed_setup) -> None:
+    """The whole point: entry `tap` must be bit-identical either way."""
+    tap = 5
+    full = MiniMaxH3Qwen3VLLanguageModel(_small_config(num_hidden_layers_override=None))
+    cut = MiniMaxH3Qwen3VLLanguageModel(_small_config(num_hidden_layers_override=tap))
+
+    # These modules allocate uninitialised storage and expect a checkpoint, so
+    # give them finite weights before running anything through them.
+    torch.manual_seed(0)
+    for parameter in full.parameters():
+        parameter.data.normal_(std=0.02)
+    # Then make the shared prefix identical, which is the only part the tapped
+    # hidden state depends on.
+    for (_, a), (_, b) in zip(full.layers[:tap].named_parameters(),
+                              cut.layers[:tap].named_parameters(),
+                              strict=True):
+        b.data.copy_(a.data)
+    torch.manual_seed(1)
+    inputs_embeds = torch.randn(1, 6, 16)
+    # mRoPE indexes three axes (t, h, w); text tokens share the same position on
+    # all three.
+    position_ids = torch.arange(6).view(1, 1, 6).expand(3, 1, 6)
+    with torch.no_grad():
+        full_out = full(inputs_embeds, position_ids, None, True, None, None)
+        cut_out = cut(inputs_embeds, position_ids, None, True, None, None)
+
+    assert torch.equal(full_out.hidden_states[tap], cut_out.hidden_states[tap])
+    # And the truncated model must not offer states it never computed.
+    assert len(cut_out.hidden_states) == tap + 1
+    # The whole shared prefix must match, not just the tap: this is the same
+    # comparison the production-loader parity gate runs against the official
+    # model, and it is what catches a truncated stack that still applied the
+    # final norm to its last entry.
+    for index, (cut_state, full_state) in enumerate(zip(cut_out.hidden_states, full_out.hidden_states,
+                                                        strict=False)):
+        assert torch.equal(cut_state, full_state), f"hidden state {index} changed under truncation"
+
+
+def test_truncated_model_drops_the_surplus_checkpoint_keys(distributed_setup) -> None:
+    """The unexpected-key check is strict on purpose, so the surplus keys have
+    to be filtered rather than the check relaxed."""
+    from fastvideo.models.encoders.minimax_h3_qwen3_vl import MiniMaxH3Qwen3VLConditioner
+
+    conditioner = MiniMaxH3Qwen3VLConditioner(_small_config(num_hidden_layers_override=5))
+
+    assert conditioner._is_above_the_tap("language_model.layers.5.mlp.gate_proj.weight")
+    assert conditioner._is_above_the_tap("language_model.layers.7.self_attn.q_proj.weight")
+    assert conditioner._is_above_the_tap("language_model.norm.weight")
+    # Kept: layers we built, the embeddings, and the vision tower.
+    assert not conditioner._is_above_the_tap("language_model.layers.4.mlp.gate_proj.weight")
+    assert not conditioner._is_above_the_tap("language_model.embed_tokens.weight")
+    assert not conditioner._is_above_the_tap("visual.blocks.0.attn.qkv.weight")
+    # The filter only drops indexes the full stack would have built. A key at
+    # or above the checkpoint's own num_hidden_layers is corrupt, and it must
+    # keep raising as unexpected exactly as it does without truncation.
+    assert not conditioner._is_above_the_tap("language_model.layers.8.mlp.gate_proj.weight")
+    with pytest.raises(ValueError, match="Unexpected"):
+        conditioner.load_weights([("model.language_model.layers.8.mlp.gate_proj.weight", torch.zeros(1))])
+
+
+def test_full_stack_filters_nothing(distributed_setup) -> None:
+    from fastvideo.models.encoders.minimax_h3_qwen3_vl import MiniMaxH3Qwen3VLConditioner
+
+    conditioner = MiniMaxH3Qwen3VLConditioner(_small_config(num_hidden_layers_override=None))
+
+    assert not conditioner._is_above_the_tap("language_model.layers.7.mlp.gate_proj.weight")
+    assert not conditioner._is_above_the_tap("language_model.norm.weight")

fastvideo/tests/ssim/test_lingbot_fast_similarity.py

@@ -1,189 +0,0 @@
-# SPDX-License-Identifier: Apache-2.0
-"""SSIM-based similarity test for LingBot-World-Fast causal I2V.
-
-The camera trajectory is read straight from the LingBot example npy files
-(poses.npy/intrinsics.npy) by the pipeline itself, matching the official
-`generate_fast.py` workflow.
-
-Note: this checkpoint is 4-step distilled, so the step count is fixed by the
-model rather than reduced for CI.
-"""
-
-import os
-
-import pytest
-import torch
-
-from fastvideo import VideoGenerator
-from fastvideo.logger import init_logger
-from fastvideo.tests.ssim.reference_utils import (
-    build_generated_output_dir,
-    build_reference_folder_path,
-    get_cuda_device_name,
-    resolve_device_reference_folder,
-    select_ssim_params,
-)
-from fastvideo.tests.utils import compute_video_ssim_torchvision, write_ssim_results
-
-logger = init_logger(__name__)
-
-REQUIRED_GPUS = 2
-
-# The released checkpoint is 4-step distilled; see LingBotWorldFastArchConfig.
-NUM_DISTILLED_STEPS = 4
-
-
-def _find_lingbotworld_examples_root() -> str | None:
-    script_dir = os.path.dirname(os.path.abspath(__file__))
-    repo_root = os.path.abspath(os.path.join(script_dir, "..", "..", ".."))
-    candidate = os.path.join(repo_root, "examples", "inference", "basic",
-                             "lingbotworld_examples")
-    if (os.path.exists(os.path.join(candidate, "00", "poses.npy"))
-            and os.path.exists(os.path.join(candidate, "00",
-                                            "intrinsics.npy"))):
-        return os.path.abspath(candidate)
-    return None
-
-
-device_name = get_cuda_device_name()
-device_reference_folder = resolve_device_reference_folder(
-    (
-        ("A40", "A40"),
-        ("L40S", "L40S"),
-        ("H100", "H100"),
-        ("H200", "H200"),
-    ),
-    device_name=device_name,
-    logger=logger,
-)
-
-LINGBOT_FAST_PARAMS = {
-    "model_path": "FastVideo/LingBot-World-Fast-Diffusers",
-    "num_gpus": 2,
-    "height": 480,
-    "width": 832,
-    "num_frames": 25,  # must be 4k+1; trimmed to a whole number of chunks
-    "seed": 42,
-    "fps": 16,
-    "example_case": "00",
-    "image_path": ("https://raw.githubusercontent.com/Robbyant/lingbot-world/"
-                   "main/examples/00/image.jpg"),
-}
-LINGBOT_FAST_FULL_QUALITY_PARAMS = {
-    **LINGBOT_FAST_PARAMS,
-    "num_frames": 81,
-}
-
-TEST_PROMPTS = [
-    "The video presents a soaring journey through a fantasy jungle. The wind "
-    "whips past the rider's blue hands gripping the reins, causing the leather "
-    "straps to vibrate. The ancient gothic castle approaches steadily, its stone "
-    "details becoming clearer against the backdrop of floating islands and "
-    "distant waterfalls.",
-]
-
-
-@pytest.mark.parametrize("prompt", TEST_PROMPTS)
-@pytest.mark.parametrize("ATTENTION_BACKEND", ["FLASH_ATTN"])
-def test_lingbot_fast_i2v_similarity(prompt: str, ATTENTION_BACKEND: str):
-    os.environ["FASTVIDEO_ATTENTION_BACKEND"] = ATTENTION_BACKEND
-
-    params = select_ssim_params(LINGBOT_FAST_PARAMS,
-                               LINGBOT_FAST_FULL_QUALITY_PARAMS)
-
-    if device_reference_folder is None:
-        pytest.skip(
-            f"Unsupported device for LingBot-World-Fast SSIM test: {device_name}"
-        )
-    if torch.cuda.device_count() < params["num_gpus"]:
-        pytest.skip(
-            f"LingBot-World-Fast SSIM test requires {params['num_gpus']} GPUs, "
-            f"but only {torch.cuda.device_count()} detected.")
-
-    examples_root = _find_lingbotworld_examples_root()
-    if examples_root is None:
-        pytest.skip(
-            "lingbotworld_examples not found under examples/inference/basic.")
-
-    action_path = os.path.join(examples_root, params["example_case"])
-
-    script_dir = os.path.dirname(os.path.abspath(__file__))
-    model_id = "LingBot-World-Fast-Diffusers"
-    output_dir = build_generated_output_dir(
-        script_dir,
-        device_reference_folder,
-        model_id,
-        ATTENTION_BACKEND,
-    )
-    output_video_name = f"{prompt[:100].strip()}.mp4"
-    os.makedirs(output_dir, exist_ok=True)
-
-    init_kwargs = {
-        "num_gpus": params["num_gpus"],
-        "use_fsdp_inference": True,
-        "dit_cpu_offload": True,
-        "dit_layerwise_offload": False,
-        "text_encoder_cpu_offload": True,
-        "vae_cpu_offload": False,
-        "pin_cpu_memory": True,
-    }
-    generation_kwargs = {
-        "output_path": output_dir,
-        "image_path": params["image_path"],
-        "action_path": action_path,
-        "height": params["height"],
-        "width": params["width"],
-        "num_frames": params["num_frames"],
-        "seed": params["seed"],
-        "fps": params["fps"],
-    }
-
-    generator: VideoGenerator | None = None
-    try:
-        generator = VideoGenerator.from_pretrained(
-            model_path=params["model_path"], **init_kwargs)
-        generator.generate_video(prompt, **generation_kwargs)
-    finally:
-        if generator is not None:
-            generator.shutdown()
-
-    generated_video_path = os.path.join(output_dir, output_video_name)
-    assert os.path.exists(generated_video_path), (
-        f"Output video was not generated at {generated_video_path}")
-
-    reference_folder = build_reference_folder_path(
-        script_dir,
-        device_reference_folder,
-        model_id,
-        ATTENTION_BACKEND,
-    )
-    if not os.path.exists(reference_folder):
-        raise FileNotFoundError(
-            f"Reference video folder does not exist: {reference_folder}")
-
-    reference_video_name = None
-    for filename in os.listdir(reference_folder):
-        if filename.endswith(".mp4") and prompt[:100].strip() in filename:
-            reference_video_name = filename
-            break
-    if not reference_video_name:
-        raise FileNotFoundError(
-            f"Reference video missing for prompt/backend under {reference_folder}"
-        )
-
-    reference_video_path = os.path.join(reference_folder, reference_video_name)
-    logger.info("Computing SSIM between %s and %s", reference_video_path,
-                generated_video_path)
-    ssim_values = compute_video_ssim_torchvision(reference_video_path,
-                                                 generated_video_path,
-                                                 use_ms_ssim=True)
-    mean_ssim = ssim_values[0]
-    logger.info("SSIM mean value: %s", mean_ssim)
-
-    write_ssim_results(output_dir, ssim_values, reference_video_path,
-                       generated_video_path, NUM_DISTILLED_STEPS, prompt)
-
-    min_acceptable_ssim = 0.70
-    assert mean_ssim >= min_acceptable_ssim, (
-        f"SSIM value {mean_ssim} is below threshold {min_acceptable_ssim} "
-        f"for {model_id} with backend {ATTENTION_BACKEND}")

tests/local_tests/encoders/test_minimax_h3_qwen3_vl_parity.py

@@ -5,6 +5,12 @@ The test compares the exact Transformers base model used by the official
 pipeline with FastVideo's production ``TextEncoderLoader`` path.  It covers
 the three numerical branches the H3 pipelines exercise: text-only tokens,
 image features, and video features.
+
+The production encoder is built only as far as the layer-50 conditioning tap
+by default (``num_hidden_layers_override``), so it returns fewer hidden states
+than the official full stack.  Every state it does build is compared
+bit-exactly against the official value at the same index, which pins the tap
+and would catch a truncated stack that still applied the final norm.
 """
 
 from __future__ import annotations
@@ -220,9 +226,20 @@ def test_minimax_h3_qwen3_vl_parity() -> None:
     actual = _run_cases(production, cases, device)
 
     assert actual.keys() == expected.keys()
+    # The production stack is built only as far as the conditioning tap by
+    # default (``num_hidden_layers_override``), so it yields one hidden state
+    # per built layer plus the embeddings, while the official model always
+    # yields the full tuple. Every state the production model produces must be
+    # bit-identical to the official value at the same index; the shared-prefix
+    # comparison would in particular catch a truncated stack that still
+    # applied the final norm, which is the failure mode that silently changes
+    # conditioning. With the override set to None the lengths are equal and
+    # this remains the original full comparison, final normed state included.
+    built_layers = int(production.language_model.num_layers)
     for name in expected:
-        assert len(actual[name]) == len(expected[name])
-        for layer, (result, reference) in enumerate(zip(actual[name], expected[name], strict=True)):
+        assert len(actual[name]) == built_layers + 1
+        assert len(actual[name]) <= len(expected[name])
+        for layer, (result, reference) in enumerate(zip(actual[name], expected[name], strict=False)):
             assert_close(result, reference, atol=0.0, rtol=0.0, msg=lambda message: f"{name} layer {layer}: {message}")
         result = actual[name][MINIMAX_H3_TEXT_ENCODER_LAYER]
         reference = expected[name][MINIMAX_H3_TEXT_ENCODER_LAYER]

tests/local_tests/minimax_h3/README.md

@@ -49,7 +49,10 @@ pytest \
 ```
 
 With a gate enabled, missing CUDA, source, or weights is a failure. Recorded component evidence is exact for both DiT
-partitions, the video VAE, and all Qwen3-VL hidden states; audio decode has maximum absolute drift `2.4e-7`.
+partitions, the video VAE, and all Qwen3-VL hidden states; audio decode has maximum absolute drift `2.4e-7`. The
+production Qwen3-VL stack is now built only to the layer-50 conditioning tap by default
+(`num_hidden_layers_override`), so the encoder gate compares every hidden state the production model builds
+bit-exactly against the official full stack at the same index.
 
 FastVideo joint audio/video generation and SP=1/SP=4 latent consistency have been validated. T2VA, FL2VA, and
 Ref2VA video/audio latents match the pinned Diffusers pipeline exactly.