nodes/aigc-apps-VideoX-Fun
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Fix Bug in Self Forcing in Multi-Gpus Infernece && Update Training Code and Docs && Update Reward Models (#498)

Browse Source
This commit is contained in:
Bubbliiiing
2026-07-14 10:30:54 +08:00
committed by GitHub
parent 1fd9ed9208
commit 403f1f7b78
183 changed files with 5524 additions and 1832 deletions
Download Patch File Download Diff File Expand all files Collapse all files
scripts/flashhead/README_TRAIN_S2V.md
+7 -6
View File
@@ -43,9 +43,10 @@ pip install -r requirements.txt
pip install Pillow einops safetensors timm tomesd librosa "torch>=2.1.2" torchdiffeq torchsde decord datasets numpy scikit-image
pip install omegaconf SentencePiece imageio[ffmpeg] imageio[pyav] tensorboard beautifulsoup4 ftfy func_timeout onnxruntime
pip install "peft>=0.17.0" "accelerate>=0.25.0" "gradio>=3.41.2" "diffusers>=0.30.1" "transformers>=4.46.2"
pip install yunchang xfuser modelscope openpyxl deepspeed==0.17.0 numpy==1.26.4
pip install yunchang xfuser modelscope openpyxl
pip uninstall opencv-python opencv-contrib-python opencv-python-headless -y
pip install opencv-python-headless
pip install deepspeed==0.17.0 numpy==1.26.4
```
**Method 3: Using Docker**
@@ -154,8 +155,8 @@ modelscope download --dataset PAI/X-Fun-Videos-Audios-Demo --local_dir ./dataset
If your data uses relative paths, configure the training script as follows:
```bash
export DATASET_NAME="datasets/internal_datasets/"
export DATASET_META_NAME="datasets/internal_datasets/metadata.json"
export DATASET_NAME="datasets/X-Fun-Videos-Audios-Demo/"
export DATASET_META_NAME="datasets/X-Fun-Videos-Audios-Demo/metadata_add_width_height.json"
```
**Absolute Paths**:
@@ -164,7 +165,7 @@ If your data uses absolute paths, configure the training script as follows:
```bash
export DATASET_NAME=""
export DATASET_META_NAME="/mnt/data/metadata.json"
export DATASET_META_NAME="/mnt/data/metadata_add_width_height.json"
```
> 💡 **Recommendation**: If the dataset is small and stored locally, use relative paths. If the dataset is stored on external storage (e.g., NAS, OSS) or shared across multiple machines, use absolute paths.
@@ -247,8 +248,8 @@ accelerate launch --use_deepspeed --deepspeed_config_file config/zero_stage2_con
| `--config_path` | Model configuration file path | `config/wan2.1/wan_civitai.yaml` |
| `--pretrained_model_name_or_path` | Path to pretrained model | `models/Diffusion_Transformer/SoulX-FlashHead-1_3B` |
| `--audio_encoder_path` | Audio encoder path (**FlashHead-S2V specific**) | `models/Diffusion_Transformer/wav2vec2-base-960h` |
| `--train_data_dir` | Training data directory | `datasets/internal_datasets/` |
| `--train_data_meta` | Training data metadata file | `datasets/internal_datasets/metadata.json` |
| `--train_data_dir` | Training data directory | `datasets/X-Fun-Videos-Audios-Demo/` |
| `--train_data_meta` | Training data metadata file | `datasets/X-Fun-Videos-Audios-Demo/metadata_add_width_height.json` |
| `--train_batch_size` | Samples per batch | 1 |
| `--video_sample_size` | Maximum video resolution for training | 512 |
| `--token_sample_size` | Token length sampling size | 512 |
scripts/flashhead/README_TRAIN_S2V_zh-CN.md
+7 -6
View File
@@ -43,9 +43,10 @@ pip install -r requirements.txt
pip install Pillow einops safetensors timm tomesd librosa "torch>=2.1.2" torchdiffeq torchsde decord datasets numpy scikit-image
pip install omegaconf SentencePiece imageio[ffmpeg] imageio[pyav] tensorboard beautifulsoup4 ftfy func_timeout onnxruntime
pip install "peft>=0.17.0" "accelerate>=0.25.0" "gradio>=3.41.2" "diffusers>=0.30.1" "transformers>=4.46.2"
pip install yunchang xfuser modelscope openpyxl deepspeed==0.17.0 numpy==1.26.4
pip install yunchang xfuser modelscope openpyxl
pip uninstall opencv-python opencv-contrib-python opencv-python-headless -y
pip install opencv-python-headless
pip install deepspeed==0.17.0 numpy==1.26.4
```
**方式三：使用 Docker**
@@ -147,8 +148,8 @@ modelscope download --dataset PAI/X-Fun-Videos-Audios-Demo --local_dir ./dataset
如果你的数据使用的是相对路径，训练脚本中请这样配置：
```bash
export DATASET_NAME="datasets/internal_datasets/"
export DATASET_META_NAME="datasets/internal_datasets/metadata.json"
export DATASET_NAME="datasets/X-Fun-Videos-Audios-Demo/"
export DATASET_META_NAME="datasets/X-Fun-Videos-Audios-Demo/metadata_add_width_height.json"
```
**绝对路径**：
@@ -157,7 +158,7 @@ export DATASET_META_NAME="datasets/internal_datasets/metadata.json"
```bash
export DATASET_NAME=""
export DATASET_META_NAME="/mnt/data/metadata.json"
export DATASET_META_NAME="/mnt/data/metadata_add_width_height.json"
```
> 💡 **建议**：如果数据集较小且存放在本地，请使用相对路径。如果数据集存放在外部存储（如 NAS、OSS）或多机共享，请使用绝对路径。
@@ -240,8 +241,8 @@ accelerate launch --use_deepspeed --deepspeed_config_file config/zero_stage2_con
| `--config_path` | 模型配置文件路径 | `config/wan2.1/wan_civitai.yaml` |
| `--pretrained_model_name_or_path` | 预训练模型路径 | `models/Diffusion_Transformer/SoulX-FlashHead-1_3B` |
| `--audio_encoder_path` | 音频编码器路径（**FlashHead-S2V 特有**） | `models/Diffusion_Transformer/wav2vec2-base-960h` |
| `--train_data_dir` | 训练数据目录 | `datasets/internal_datasets/` |
| `--train_data_meta` | 训练数据元数据文件 | `datasets/internal_datasets/metadata.json` |
| `--train_data_dir` | 训练数据目录 | `datasets/X-Fun-Videos-Audios-Demo/` |
| `--train_data_meta` | 训练数据元数据文件 | `datasets/X-Fun-Videos-Audios-Demo/metadata_add_width_height.json` |
| `--train_batch_size` | 每批训练的样本数 | 1 |
| `--video_sample_size` | 视频最大训练分辨率 | 512 |
| `--token_sample_size` | Token 长度采样大小 | 512 |