World-Model: Matrix-Game 2.0 I2V
Training scenarios for the Matrix-Game 2.0 I2V world model on Solaris (Minecraft)
data and Zelda data, using the new YAML-driven trainer
(fastvideo/train/entrypoint/train.py).
Solaris Configs
| Config | Method | Student | Notes |
|---|---|---|---|
solaris/finetune_i2v.yaml |
FineTuneMethod |
MatrixGame2Model (bidirectional) |
Multi-step SFT from mg_bidirectional_Solaris. |
solaris/dfsft_causal_i2v.yaml |
DiffusionForcingSFTMethod |
MatrixGame2CausalModel |
Diffusion-Forcing SFT with chunkwise timesteps. |
solaris/self_forcing_causal_i2v.yaml |
SelfForcingMethod |
MatrixGame2CausalModel |
Matrix-Game 2.0 DMD/Self-Forcing distillation; teacher = bidirectional, critic = bidirectional. |
Zelda Configs
| Config | Method | Student | Notes |
|---|---|---|---|
zelda/finetune_i2v.yaml |
FineTuneMethod |
MatrixGame2Model (bidirectional) |
Zelda bidirectional I2V finetuning from FastVideo/Matrix-Game-2.0-Base-Diffusers. Uses 33-frame clips and Zelda validation with action overlays. |
zelda/dfsft_causal_i2v.yaml |
DiffusionForcingSFTMethod |
MatrixGame2CausalModel |
Zelda causal Diffusion-Forcing SFT from mignonjia/mg_bidirectional_zelda. Uses the same Zelda data, resolution, optimizer, and validation defaults as the Zelda finetune config. |
zelda/self_forcing_causal_i2v.yaml |
SelfForcingMethod |
MatrixGame2CausalModel |
Zelda DMD/Self-Forcing distillation; student init = mignonjia/mg_causal_zelda, teacher = bidirectional (mignonjia/mg_bidirectional_zelda), critic = bidirectional. |
zelda/streaming_long_tuning_causal_i2v.yaml |
StreamingLongTuningMethod |
MatrixGame2CausalModel |
LongLive-style streaming long tuning from the 1k-step Zelda self-forcing checkpoint. |
Zelda world-model distillation is a two-run workflow: first run
zelda/self_forcing_causal_i2v.yaml to train or load the 1k-step
self-forcing checkpoint (mignonjia/mg_sf_distilled_zelda_1k_steps), then run
zelda/streaming_long_tuning_causal_i2v.yaml for the 3k-step streaming
long-tuning stage. The long-tuning YAML starts from that 1k-step checkpoint; it
does not run the short self-forcing stage inside the same config.
Zelda Training Data
The Zelda training configs use data/zeldam2-clean as a suggested local path.
Download the dataset from Hugging Face before running those configs:
python scripts/huggingface/download_hf.py \
--repo_id mignonjia/zeldam2-clean \
--local_dir data/zeldam2-clean \
--repo_type dataset
You can store the dataset elsewhere; update training.data.data_path in the
YAML to point at that location.
Multi3D Training Data
zelda/finetune_i2v.yaml includes an optional, commented-out Multi3D entry.
Enable it only when you want to mix Zelda with multi-game data from
data/multi3d_games. You can store this dataset anywhere; before enabling it,
update the matching commented training.data.data_path key in the YAML to the
correct location.
To mix datasets in a training YAML, set training.data.data_path to a
path-to-repeat-count mapping. For example, zelda/finetune_i2v.yaml can use
data/zeldam2-clean: 1 and # data/multi3d_games: 10; uncommenting the
Multi3D entry repeats the multi-game parquet list ten times before training
samples are shuffled.
World Model Validation Data
The Zelda validation configs expect a small public validation bundle under
data/zelda_validation_data.
Download it from Hugging Face before running the Zelda scenarios:
python scripts/huggingface/download_hf.py \
--repo_id mignonjia/zelda_validation_data \
--local_dir data/zelda_validation_data \
--repo_type dataset
The bundle contains validation_zelda.json, images/, and actions/.
The Zelda configs point
callbacks.validation.dataset_file at
data/zelda_validation_data/validation_zelda.json.
Usage
Solaris
bash examples/train/run.sh \
examples/train/scenario/worldmodel/solaris/finetune_i2v.yaml
bash examples/train/run.sh \
examples/train/scenario/worldmodel/solaris/dfsft_causal_i2v.yaml
bash examples/train/run.sh \
examples/train/scenario/worldmodel/solaris/self_forcing_causal_i2v.yaml
Zelda
# Finetuning / DFSFT
bash examples/train/run.sh \
examples/train/scenario/worldmodel/zelda/finetune_i2v.yaml
bash examples/train/run.sh \
examples/train/scenario/worldmodel/zelda/dfsft_causal_i2v.yaml
# Distillation / long tuning
bash examples/train/run.sh \
examples/train/scenario/worldmodel/zelda/self_forcing_causal_i2v.yaml
bash examples/train/run.sh \
examples/train/scenario/worldmodel/zelda/streaming_long_tuning_causal_i2v.yaml
Override any field on the command line:
bash examples/train/run.sh \
examples/train/scenario/worldmodel/solaris/dfsft_causal_i2v.yaml \
--training.distributed.num_gpus 8 \
--training.optimizer.learning_rate 1e-5