 shaoxiongduan
|
ac919b1c87
|
[feat] WanTrack: OpenVid preprocess + Wan2.2 synthetic-gen pipeline, training configs, first-frame saturation fix
- data_pipeline: SAM+CoTracker preprocessing, sharded Slurm launchers, OpenVid download/filter,
and a data-parallel Wan2.2-T2V-A14B synthetic video generator with the first-frame
over-brightness fix (over-generate 129 -> drop first 8 -> keep 121).
- examples/train: OpenVid bidir/stage2/stage3 launchers + finetune YAMLs (sparse, merged,
merged_bias, stage2v2 hi-lr, stage3A/B/C).
- research_log: first-frame saturation root-cause writeup (DiT position-0 latent, not VAE decode).
- fastvideo: trackwan model / track-encoder / preprocessing updates, dcp_to_diffusers export, FA4.
|
2026-07-14 08:58:22 +00:00 |
|
|
shaoxiongduan
|
34a0a81be1
|
[feat] WanTrack: bidirectional I2V + point-track (MotionStream) finetune + eval
TrackWan = Wan2.2 + MotionStream point-track motion control. Stage-1 bidirectional
I2V finetune (text kept), built on the new modular trainer.
- DiT: TrackWanTransformer3DModel (52ch in = 16 noisy + 20 I2V + 16 track), TrackEncoder
(sinusoidal track-ID embed, scatter, 4x temporal conv, zero-init head); config + registry.
- Trainer: WanTrackModel wrapper, i2v_track preprocess pipeline + pyarrow schema,
TrackValidationCallback, finetune_wantrack_i2v.yaml.
- data_pipeline/: video gen (Wan2.2-A14B), CoTracker3 track extraction, 52ch init
conversion, synthetic track authoring, controllability (CoTracker EPE) eval, standalone
inference, Gradio demo.
- eval: motion.cotracker_epe metric. research_log/ lab notebook.
- fixes: FA3 flash_attn_func tuple unwrap; decord fallback for removed torchvision.io.read_video;
preprocess flush write_remainder.
|
2026-07-01 06:36:40 +00:00 |
|