Davids048andClaude Opus 5.5
d051603677
- v1_preprocess.py and v1_preprocessing_new.py load a PreprocessRunConfig
from --config <yaml> plus dotted overrides; their argparse flags are
gone. The preprocessing pipelines, workflows, datasets, and stages read
resolved_config.preprocess.* and typed paths instead of
preprocess_config and an argparse namespace.
- The overfit preprocessors build resolved configs instead of FastVideoArgs.
- Each preprocessing launcher has a YAML file next to it with the values
its flags held; shell variables stay as dotted overrides. Against the
3f6893a0 snapshots of the 17 launchers, every value and PipelineConfig
matches, except mode (preprocess instead of inference), ltx2_vae_tiling,
and text_only's vae_config.load_encoder. Five launchers that the old
parser rejected now carry their intended values; the never-defined
--model_type and --validation_dataset_file flags are dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
13 lines
501 B
Bash
13 lines
501 B
Bash
# export WANDB_MODE="offline"
|
|
GPU_NUM=1 # 2,4,8
|
|
MODEL_PATH="Wan-AI/Wan2.1-I2V-14B-480P-Diffusers"
|
|
DATA_MERGE_PATH="mini_i2v_dataset/crush-smol_raw/merge.txt"
|
|
OUTPUT_DIR="mini_i2v_dataset/crush-smol_preprocessed"
|
|
|
|
torchrun --nproc_per_node=$GPU_NUM \
|
|
fastvideo/pipelines/preprocess/v1_preprocess.py \
|
|
--config scripts/preprocess/v1_preprocess_wan_data_i2v.yaml \
|
|
--model_path $MODEL_PATH \
|
|
--preprocess.data_merge_path $DATA_MERGE_PATH \
|
|
--preprocess.dataset_output_dir=$OUTPUT_DIR
|