Compare commits

..
3 Commits
Author SHA1 Message Date
jiangzeyinzi 30afc0a1de Merge pull request #39 from yaosheng216/patch-2
Update readme.md
2024-07-18 17:51:53 +08:00
Great aeabef75b4 Update readme.md 2024-07-18 17:40:28 +08:00
jiangzeyinzi 9376149415 Merge pull request #38 from modelscope/v1.0.3_dev
v1.0.3
2024-07-18 17:31:44 +08:00
4 changed files with 36 additions and 8 deletions
Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.3 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 8.8 MiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 2.9 KiB

+36 -8
View File
@@ -18,7 +18,6 @@ SCEPTER offers 3 core components:
## 🎉 News
- [2024.09]: We introduce **ACE**, an **A**ll-round **C**reator and **E**ditor adept at executing a diverse array of image editing tasks tailored to your specifications. Built upon the cutting-edge Diffusion Transformer architecture, ACE has been extensively trained on a comprehensive dataset to seamlessly interpret and execute any natural language instruction. For further information, please consult the [project page]().
- [2024.07]: Support the inference and training of open-source generative models based on the [DiT](https://arxiv.org/abs/2212.09748) architecture, such as [SD3](https://arxiv.org/pdf/2403.03206) and [PixArt](https://arxiv.org/abs/2310.00426).
- [2024.05]: Introducing SCEPTER v1, supporting customized image edit tasks! Simply provide 10 image pairs, SCEPTER will tune an edit tuner for your own Image-to-Image tasks, like `Clay Style`, `De-Text`, `Segmentation`, etc.
- [2024.04]: New [StyleBooth](https://ali-vilab.github.io/stylebooth-page/) demo on SCEPTER Studio for`Text-Based Style Editing`.
@@ -32,16 +31,45 @@ SCEPTER offers 3 core components:
## 🖼 Gallery for Recent Works
### <img src="asset/images/ace/logo.png" height=20> <img src="asset/images/ace/text.png" height=20>
### Edit Tuners
ACE is a unified foundational model framework that supports a wide range of visual generation tasks. By defining CU for unifying multi-modal inputs across different tasks and incorporating long-
context CU, we introduce historical contextual information into visual generation tasks, paving
the way for ChatGPT-like dialog systems in visual generation.
Simply provide 10 image pairs, SCEPTER will tune an edit tuner for your own Image-to-Image tasks, like `Clay Style`, `De-Text`, `Segmentation`, etc.
Try our official few-shot datasets: [De-Text](https://www.modelscope.cn/api/v1/models/iic/scepter/repo?Revision=master&FilePath=datasets%2Fdetext.zip), [Image2Hed](https://www.modelscope.cn/api/v1/models/iic/scepter/repo?Revision=master&FilePath=datasets%2Fhed_pair.zip), [Image2Depth](https://www.modelscope.cn/api/v1/models/iic/scepter/repo?Revision=master&FilePath=datasets%2Fimage2depth.zip), [Depth2Image](https://www.modelscope.cn/api/v1/models/iic/scepter/repo?Revision=master&FilePath=datasets%2Fdepth2image.zip).
<a href="https://ali-vilab.github.io/ace-page/">
<img src="asset/images/ace/teaser_dy.gif" width=1024>
</a>
<table><tbody>
<tr>
<th align="center" colspan="4">Clay Style<br>Prompt: "Convert this image into clay style"</th>
</tr>
<tr>
<td><img src="asset/images/edit_tuner/vermeer.jpeg" width="300"></td>
<td><img src="asset/images/edit_tuner/clay_vermeer.jpeg" width="300"></td>
<td><img src="asset/images/edit_tuner/cat_512.jpg" width="300"></td>
<td><img src="asset/images/edit_tuner/clay_cat.jpeg" width="300"></td>
</tr>
<tr>
<th align="center" colspan="2">De-Text<br>Prompt: "Remove the texts"</th>
<th align="center" colspan="2">Image2Hed<br>Prompt: "Convert to an edge map"</th>
</tr>
<tr>
<td><img src="asset/images/edit_tuner/text.jpg" width="300"></td>
<td><img src="asset/images/edit_tuner/detext.jpeg" width="300"></td>
<td><img src="asset/images/edit_tuner/cat_512.jpg" width="300"></td>
<td><img src="asset/images/edit_tuner/hed.jpeg" width="300"></td>
</tr>
<tr>
<th align="center" colspan="2">Image2Depth<br>Prompt: "Calculate the depth map"</th>
<th align="center" colspan="2">Depth2Image<br>Prompt: "Convert depth map into color image"</th>
</tr>
<tr>
<td><img src="asset/images/edit_tuner/house.jpg" width="300"></td>
<td><img src="asset/images/edit_tuner/image2depth.jpeg" width="300"></td>
<td><img src="asset/images/edit_tuner/depth.jpg" width="300"></td>
<td><img src="asset/images/edit_tuner/depth2image.jpeg" width="300"></td>
</tr>
</tbody>
</table>
Note: Left image is input and right image is output.
## 🛠️ Installation