Commit Graph
182 Commits
Author SHA1 Message Date
Will Lin 6b5b2dc6e7 mp training 2025-05-27 11:38:47 -07:00
Will Lin 91b7cc1be8 move utils into training_utils 2025-05-25 18:23:47 -07:00
Zihang-He d6365373b4 added gradient clipping 2025-05-25 23:54:10 +00:00
Will Lin 50fb94b902 gradient checking 2025-05-25 15:20:41 -07:00
Will Lin 2caa0d4d0b cleanup 2025-05-24 17:07:17 -07:00
Will Lin 24db823998 add validation 2025-05-24 16:27:05 -07:00
Will Lin 8c4704edf5 update train dependecies 2025-05-23 12:48:06 -07:00
JerryZhou54 dfba7ec833 Small fix 2025-05-23 19:03:58 +00:00
JerryZhou54 7a2e171f1b Small fix 2025-05-23 19:01:59 +00:00
JerryZhou54 a8aac6090a Integrate the new parquet dataloader into training pipeline 2025-05-23 18:27:52 +00:00
191d1be3b4 Will/training (#425)
Co-authored-by: Kevin Lin <42618777+kevin314@users.noreply.github.com>
Co-authored-by: William Lin <SolitaryThinker@users.noreply.github.com>
Co-authored-by: Will Lin <wlsaidhi@gmail.com>
2025-05-23 02:06:54 -04:00
JerryZhou54 338ea1e5f2 Add script to upload preprocessed dataset to HF 2025-05-23 06:05:40 +00:00
JerryZhou54 42a2f272d5 Preprocessing Stage: 1. Save to parquets periodically 2. Allow resuming from the middle 3. Doesn't support multi-gpu for now. Data Loader Stage: 1. Allow multi-gpu dataloader 2. Doesn't drop files even if number of parquet files is not divisible by num_gpus 3. Able to resume training 2025-05-23 00:57:39 +00:00
JerryZhou54 982bfcfdc8 Fix small issues when loading mask 2025-05-21 22:46:32 +00:00
JerryZhou54 2ac06379a7 Finish data preprocessing and loading 2025-05-21 22:34:03 +00:00
JerryZhou54 baaa1673f7 Add data preprocessing script for WAN 2025-05-19 15:09:26 +00:00
William Lin ace6e971e5 [V1] Remove vLLM dependency (#413) 2025-05-18 00:57:28 -07:00
William Lin b4f6758253 [Teacache] allow None for forward_context batch when using teacache (#412) 2025-05-17 18:43:14 -07:00
William Lin 535d29b392 [Docs] Fix image (#407) 2025-05-12 14:30:05 -07:00
William Lin b4255517e0 [Docs] Add CLI docs (#406) 2025-05-12 14:16:48 -07:00
William Lin 6eeb60613f Release 0.1.0 (#405) v0.1.0 2025-05-12 11:54:23 -07:00
William Lin 53d2c7791f [V1] Update where num_frame rounding is done (#403) 2025-05-12 11:52:49 -07:00
William Lin 53cb693dca [V1] Docs Update (#402) 2025-05-12 11:52:09 -07:00
Kevin Lin 6f72d24876 [CLI] Default to pipeline config (#401) 2025-05-12 00:22:53 -07:00
William Lin d1459e9976 [V1] Update README (#400) 2025-05-11 22:32:03 -07:00
William Lin 59ab481eb1 release 0.0.5 (#399) v0.0.5 2025-05-11 16:23:11 -07:00
William Lin 0cf001986a [Docs] More docs update (#394) 2025-05-11 16:19:47 -07:00
applesaucethebunandBrayden Zhong 51956369a5 [Misc] Replace instances of time.time() with time.perf_counter() (#396)
Co-authored-by: Brayden Zhong <b8zhong@uwaterloo.ca>
2025-05-11 15:56:39 -07:00
Kevin Lin 4b0970cbbf [CI] Set volume_size required to false (#398) 2025-05-11 15:55:34 -07:00
Kevin Lin 94bf47a572 [CI] Set default disk size (#397) 2025-05-11 15:34:17 -07:00
River (Zihang He)andWill Lin 1a3ac9074b Zihang stepvideo v1 (#389)
Co-authored-by: Will Lin <wlsaidhi@gmail.com>
2025-05-11 14:41:42 -07:00
Kevin Lin fb0581d5b0 [CLI] Update cli to support new api/model config (#384) 2025-05-11 13:50:33 -07:00
Kevin Lin 6c74ab4132 [CI] Use python 3.10/3.11 for SSIM test (#392) 2025-05-10 13:48:29 -07:00
William Lin 9a91021c56 [Docs] Add collect_env.py and various docs update (#393) 2025-05-10 13:42:42 -07:00
Wei Zhou 51c94d6a73 [Misc] Small Fixes & Features (#390) 2025-05-07 15:40:49 -07:00
William Lin dba38dbc03 Release 0.0.4 (#388) 2025-05-07 02:40:42 -07:00
William Lin 2034cc3c4f [misc] Improve worker cleanup (#387) 2025-05-06 22:45:22 -07:00
William Lin c69afce2f6 [Docs] Update for V1 (#381) 2025-05-06 16:34:44 -07:00
Wei Zhou b08e758eb3 Small Fixes & Features (#378) 2025-05-06 15:52:34 -07:00
William Lin 3f3462d7ce Cleanup Teacache params (#386) 2025-05-06 15:52:05 -07:00
William Lin c9c47dd89c Add Teacache to V1 (#371) 2025-05-06 11:58:56 -07:00
William Lin 9c4ef7c2f1 [Lint] fix (#382) 2025-05-06 00:42:55 -07:00
Kevin Lin f25eb4b905 [CI] Add write permissions to build-image workflow (#379) 2025-05-05 19:21:03 -07:00
Kevin Lin 048d55ccbb [CI] Add new images for different Python versions (#377) 2025-05-05 10:28:33 -07:00
Wei Zhou a271c55fe4 Fix FSDP issues when using cpu_offload flag (#376) 2025-05-02 15:54:52 -07:00
William Lin f663ae0d8a change gradio example to use model configs (#375) 2025-05-02 10:42:03 -07:00
William Lin c0911aa3dd release 0.0.3 (#374) 2025-05-02 03:50:41 -07:00
William Lin 5f59687ae7 Fix model config for python 3.11+ (#373) 2025-05-02 03:18:56 -07:00
Wei Zhou 5adbc81cdc Refactor encoder (#370) 2025-05-02 02:41:12 -07:00
Kevin Lin f26d5c37c1 Update SSIM tests to use new API (#369) 2025-05-01 04:14:24 -07:00