hao-ai-lab-FastVideo

Files

T

Will LinandClaude Fable 5 33e104ee65 [train] enforce fp32 master weights in every training entry point

Optimizer steps applied in-place to bf16 parameters round away updates
below ~half an ulp of each weight's magnitude: after 1000 DMD2 steps,
205/211 of the H3 student's norm-scale params were bit-identical to the
base checkpoint, and bf16 exp_avg_sq starves the same way. Guard every
path:

- fastvideo/train: Trainer.run refuses to start unless all trainable
  role params are fp32; opt out via
  training.model.allow_low_precision_master_weights
- fastvideo/training (legacy): same check at the top of the three
  train() funnels (base / distillation / self-forcing), opt out via
  FASTVIDEO_ALLOW_LOW_PRECISION_MASTER_WEIGHTS=1. Shipped recipes
  already pass --dit_precision fp32 and are unaffected.
- AdamWBeta1Zero: keep exp_avg_sq and update math in fp32 when the
  param storage is not fp32 (fp32 path stays bitwise-identical)

Also carries the checkpointing_start_step / reset_lr_on_resume config
plumbing for the previous commit (same dataclass/parse files).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

2026-08-14 02:08:05 +00:00

__init__.py

[misc]: empty __init__.py files with no logic (#1346 )

2026-05-13 13:09:40 -07:00

activation_checkpoint.py

[feat] add Minimax H3 sft pipeline (#1688 )

2026-08-07 16:12:04 -07:00

builder.py

[feat] Refactor training framework into fastvideo/train (#1159 )

2026-03-09 15:16:42 -07:00

checkpoint.py

[train] run ops: checkpoint start-step gating, LR reset on resume, W&B fallback, weights-only DCP export