Update README.md

Former-commit-id: 6ecaba8e03d7c7f99c068a57c923f11c5456a8c9
This commit is contained in:
lb203
2024-03-05 14:54:38 +08:00
committed by GitHub
parent 86332d9c0a
commit 2ca658809a
+16 -4
View File
@@ -2,14 +2,26 @@
[[Project Page]](https://pku-yuangroup.github.io/Open-Sora-Plan/) [[中文主页]](https://pku-yuangroup.github.io/Open-Sora-Plan/blog_cn.html)
## Goal
This project aims to create a simple and scalable repo, to reproduce [Sora](https://openai.com/sora) (OpenAI, but we prefer to call it "CloseAI" ) and build knowledge about Video-VQVAE (VideoGPT) + DiT at scale. However, we have limited resources, we deeply wish all open-source community can contribute to this project. Pull request are welcome!!!
本项目希望通过开源社区的力量复现Sora,由北大-兔展AIGC联合实验室共同发起,当前我们资源有限仅搭建了基础架构,无法进行完整训练,希望通过开源社区逐步增加模块并筹集资源进行训练,当前版本离目标差距巨大,仍需持续完善和快速迭代,欢迎Pull request!!!
<p align="center">
<img src="assets/we_want_you.jpg" width=27%>
</p>
![The architecture of Open-Sora-Plan](assets/framework.jpg)
Project stages:
- Primary
1. Setup the codebase and train a un-conditional model on landscape dataset.
2. Train models that boost resolution and duration.
- Extensions
3. Conduct text2video experiments on landscape dataset.
4. Train the 1080p model on video2text dataset.
5. Control model with more condition.
<div style="display: flex; justify-content: center;">
<img src="assets/we_want_you.jpg" width=200>
<img src="assets/framework.jpg" width=600>
</div>
## News