SolitaryThinker
d81f1f166b
- MiniMaxH3Model: allow per-role attention backends (TORCH_SDPA/FLASH_ATTN/ VIDEO_SPARSE_ATTN_H3); predict_noise routes attn_metadata_vsa for attn_kind='vsa' (dense roles keep None). - MiniMaxH3DMDModel.prepare_batch builds real VSA-H3 metadata (packed text|cond|audio prefix segments via _h3_vsa_prefix_segments, sparsity from training.vsa.sparsity) when the role runs VIDEO_SPARSE_ATTN_H3. - VSA-H3 tile(): fresh buffer for grad-tracking forwards; shared-buffer reuse trips autograd's in-place version check at student backward. - build_tracker: degrade to DummyTracker when wandb is unimportable or has no credentials (modular stack already logs per-step loss dicts via the wandb tracker; no new callback needed). - dmd2_vsa0_overfit.yaml: student VSA-H3 @ sparsity 0.0, teacher/critic FLASH_ATTN, 3-step DMD schedule, wandb project h3-dmd2-vsa. - preprocess_minimax_h3_overfit: --video/--prompt/--output-dir/--model-path CLI; overfit_vsa0.sh: preprocess + 4-GPU torchrun with FASTVIDEO_FA4=1. - CPU tests: VSA metadata build/routing, per-role backend override, wandb guard + loss-dict logging, tile-under-grad regression.
Training Configs
Single-step training configurations organized by method and model.
configs/
├── fine_tuning/ # Standard finetuning and DFSFT
├── distribution_matching/ # DMD2 and Self-Forcing
├── knowledge_distillation/ # KD from teacher to student
└── example.yaml # Annotated reference config with all fields
Each method directory contains per-model subdirectories (e.g. wan/, hunyuan/).
Launch any config with:
bash examples/train/run.sh examples/train/configs/<method>/<model>/<config>.yaml
For multi-step training pipelines, see examples/train/scenario/.