- VSA-H3 backend + DistributedAttention_VSA: cast autocast-fp32 activations
to bf16 before comm/tiling (kernels are bf16-only; RMSNorm emits fp32
under autocast), restore output dtype.
- minimax_h3 DiT: recompute gate_compress activity under grad instead of
trusting a cached no-grad answer.
- Scheduler: shift_sigmas() helper; denoising stage honors
pipeline_config.dmd_denoising_steps with per-modality shifts so validation
samples at the exact training sigmas; batch.VSA_sparsity fallback.
- ValidationCallback: never .to('cpu') FSDP-managed modules, inject the
method's dmd_denoising_steps, post-validation off-CUDA param scan;
make_inference_args no longer defaults dit_cpu_offload=True (this offloaded
the live training student after every validation).
- AdamWBeta1Zero: beta1=0 AdamW without the exp_avg buffer (mathematically
identical; saves one full parameter-sized state per trainable model),
auto-selected when betas[0] == 0.
- methods/base + train models: per-role attn kind for VIDEO_SPARSE_ATTN_H3,
VSA metadata built in the base H3 model's prepare_batch (DMD wrapper
inherits), device-scan probe compares against the model's own device.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
