Davids048andClaude Opus 5.5
d051603677
- v1_preprocess.py and v1_preprocessing_new.py load a PreprocessRunConfig
from --config <yaml> plus dotted overrides; their argparse flags are
gone. The preprocessing pipelines, workflows, datasets, and stages read
resolved_config.preprocess.* and typed paths instead of
preprocess_config and an argparse namespace.
- The overfit preprocessors build resolved configs instead of FastVideoArgs.
- Each preprocessing launcher has a YAML file next to it with the values
its flags held; shell variables stay as dotted overrides. Against the
3f6893a0 snapshots of the 17 launchers, every value and PipelineConfig
matches, except mode (preprocess instead of inference), ltx2_vae_tiling,
and text_only's vae_config.load_encoder. Five launchers that the old
parser rejected now carry their intended values; the never-defined
--model_type and --validation_dataset_file flags are dropped.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012M91wnVFmPEvJ9h39r5BH7
19 lines
509 B
YAML
19 lines
509 B
YAML
# Read by v1_preprocess_wan_data_i2v.sh, which passes model_path, preprocess.data_merge_path, and
|
|
# preprocess.dataset_output_dir.
|
|
# Offload settings of the per-task preprocessing pipelines (v1_preprocess.py).
|
|
engine:
|
|
offload:
|
|
dit_layerwise: true
|
|
image_encoder: true
|
|
pin_cpu_memory: true
|
|
preprocess:
|
|
preprocess_video_batch_size: 8
|
|
max_height: 480
|
|
max_width: 832
|
|
num_frames: 77
|
|
dataloader_num_workers: 0
|
|
train_fps: 16
|
|
samples_per_file: 8
|
|
flush_frequency: 8
|
|
preprocess_task: i2v
|