[fix]: init PackedExpertLinear params with zeros, not empty (review)

Addresses review thread on fastvideo/models/dits/magi_human.py:263. torch.empty leaves memory uninitialized; if a future caller skips load_state_dict or strict=False loses entries silently, the network would output NaNs. zero-init costs nothing at runtime (overwritten by load) and matches sibling MultiModalityRMSNorm which also uses torch.zeros.
2026-05-12 14:09:55 -07:00
parent 746d28f16d
commit 345d2acb19
1 changed files with 2 additions and 2 deletions
@@ -259,11 +259,11 @@ class PackedExpertLinear(nn.Module):
        self.num_experts = num_experts
        self.use_bias = bias
        self.weight = nn.Parameter(
-            torch.empty(out_features * num_experts, in_features, dtype=dtype)
+            torch.zeros(out_features * num_experts, in_features, dtype=dtype)
        )
        if bias:
            self.bias = nn.Parameter(
-                torch.empty(out_features * num_experts, dtype=dtype)
+                torch.zeros(out_features * num_experts, dtype=dtype)
            )
        else:
            self.register_parameter("bias", None)