 shaoxiongduan
|
34a0a81be1
|
[feat] WanTrack: bidirectional I2V + point-track (MotionStream) finetune + eval
TrackWan = Wan2.2 + MotionStream point-track motion control. Stage-1 bidirectional
I2V finetune (text kept), built on the new modular trainer.
- DiT: TrackWanTransformer3DModel (52ch in = 16 noisy + 20 I2V + 16 track), TrackEncoder
(sinusoidal track-ID embed, scatter, 4x temporal conv, zero-init head); config + registry.
- Trainer: WanTrackModel wrapper, i2v_track preprocess pipeline + pyarrow schema,
TrackValidationCallback, finetune_wantrack_i2v.yaml.
- data_pipeline/: video gen (Wan2.2-A14B), CoTracker3 track extraction, 52ch init
conversion, synthetic track authoring, controllability (CoTracker EPE) eval, standalone
inference, Gradio demo.
- eval: motion.cotracker_epe metric. research_log/ lab notebook.
- fixes: FA3 flash_attn_func tuple unwrap; decord fallback for removed torchvision.io.read_video;
preprocess flush write_remainder.
|
2026-07-01 06:36:40 +00:00 |
|