 JerryZhou54
|
42a2f272d5
|
Preprocessing Stage: 1. Save to parquets periodically 2. Allow resuming from the middle 3. Doesn't support multi-gpu for now. Data Loader Stage: 1. Allow multi-gpu dataloader 2. Doesn't drop files even if number of parquet files is not divisible by num_gpus 3. Able to resume training
|
2025-05-23 00:57:39 +00:00 |
|
|
JerryZhou54
|
2ac06379a7
|
Finish data preprocessing and loading
|
2025-05-21 22:34:03 +00:00 |
|
|
JerryZhou54
|
baaa1673f7
|
Add data preprocessing script for WAN
|
2025-05-19 15:09:26 +00:00 |
|
|
Zhang Peiyuanandrlsu9
|
58cfd71b5e
|
Cleanup
Co-authored-by: rlsu9 <r3su@ucsd.edu>
|
2024-12-15 17:03:29 -08:00 |
|
|
Hangliang Dingandrlsu9
|
3bf892b6ab
|
update release readme (#81)
Co-authored-by: rlsu9 <r3su@ucsd.edu>
|
2024-12-15 22:24:13 +08:00 |
|