diff --git a/README.md b/README.md index cef9916..12289e5 100644 --- a/README.md +++ b/README.md @@ -1,143 +1,5 @@ -# PCDMs -Implementation code:[Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Models](https://arxiv.org/pdf/2310.06313.pdf), accepted at International Conference on Learning Representations (ICLR) 2024. +原项目:https://github.com/tencent-ailab/PCDMs +![f88da31cf992a48eed11c93b13285c6](https://github.com/StartHua/ComfyUI_PCDMs/assets/22284244/dee8343b-07f8-4732-9489-f23c26c2e5be) -## Generated Results -![PCDMs Motivation](imgs/compare_sota.png) - ---- -![PCDMs results1](imgs/demo.png) - ---- -![PCDMs results2](imgs/demo2.png) - -You can directly download our test results from Google Drive: (1) [PCDMs vs SOTA](https://drive.google.com/drive/folders/1q21tA3VsQqScecQ7m3_eUFxIPWUGYKAa?usp=drive_link) (2) [PCDMs Results](https://drive.google.com/drive/folders/1sjqMhZ79pugk2IHhW-whg_NASpx3BSew?usp=drive_link). - -The [PCDMs vs SOTA](https://drive.google.com/drive/folders/1q21tA3VsQqScecQ7m3_eUFxIPWUGYKAa?usp=drive_link) compares our method with several state-of-the-art methods e.g. ADGAN, PISE, GFLA, DPTN, CASD, NTED, PIDM. -Each row contains target_pose, source_image, ground_truth, ADGAN, PISE, GFLA, DPTN, CASD, NTED, PIDM, and PCDMs (ours) respectively. - - -We present a simplified version of PCDMs and only use stage2, utilizing training data from TikTok and DeepFashion. -The weights can be obtained from [Google drive](https://drive.google.com/drive/folders/1hnPm-GzaAtQKjgG_glhlQZieJ5g5Xd-y?usp=drive_link). - -Download dwpose weights (`dw-ll_ucoco_384.pth`, `yolox_l_8x8_300e_coco_20211126_140236-d3bd2b23.pth`) following [this](https://github.com/IDEA-Research/DWPose?tab=readme-ov-file#-dwpose-for-controlnet). -## Installation -``` -# install diffusers & pose extractor -pip install diffusers==0.24.0 -pip install controlnet-aux==0.0.7 -pip install transformers==4.32.1 -pip install accelerate==0.24.1 - -# install DWPose which is dependent on MMDetection, MMCV and MMPose -pip install -U openmim -mim install mmengine -mim install "mmcv>=2.0.1" -mim install "mmdet>=3.1.0" -mim install "mmpose>=1.1.0" - -# clone code -git clone https://github.com/tencent-ailab/PCDMs.git - -# download the models -cd PCDMs -mv {weights} ./PCDMs_ckpt.pt - -# then you can use the notebook -{pcdms_demo.ipynb} -``` - - -## Methods -![PCDMs inpainting](imgs/inpainting.png) - ---- -![PCDMs Motivation](imgs/diagram.png) - ---- -![PCDMs Framework](imgs/frameworkv2.png) - -## Dataset -### Processed Data -This [link](https://drive.google.com/drive/folders/17yqf2iV2y7Hwx-__uV_-cRaOm2XX8lWS?usp=drive_link) contains processed and prepared data that is ready for use. -The data has been processed in the following ways: - -• Rename image - -• Split the train/test set - -• keypoints extracted with Openpose - -The folder structure of dataset should be as follows: - -``` -Deepfashion/ -├── all_data_png # including train and test images -│ ├── img1.png -│ ├── ... -│ ├── img52712.png -├── train_lst_256_png # including train images of 256 size -│ ├── img1.png -│ ├── ... -│ ├── img48674.png -├── train_lst_512_png # including train images of 512 size -│ ├── img1.png -│ ├── ... -│ ├── img48674.png -├── test_lst_256_png # including test images of 256 size -│ ├── img1.png -│ ├── ... -│ ├── img4038.png -├── test_lst_512_png # including test images of 512 size -│ ├── img1.png -│ ├── ... -│ ├── img4038.png -├── normalized_pose_txt.zip # including pose coordinate of train and test set -│ ├── pose_coordinate1.txt -│ ├── ... -│ ├── pose_coordinate40160.txt -├── train_data.json -├── test_data.json -``` - -### Original Data - Download ```img_highres.zip``` of the DeepFashion Dataset from [In-shop Clothes Retrieval Benchmark](https://drive.google.com/drive/folders/0B7EVK8r0v71pYkd5TzBiclMzR00?resourcekey=0-fsjVShvqXP2517KnwaZ0zw). - - Unzip ```img_highres.zip```. You will need to ask for password from the dataset maintainers. - - -## Checkpoints Links -We provide 3 stage checkpoints available [here](https://drive.google.com/drive/folders/1xDpSIBdP11UoXe0HoWGPWBfEnk1zRMVw?usp=drive_link). - - - -## How to Train/Test -1. train/test stage1-prior -``` -sh run_stage1.sh & sh run_test_stage1.sh -``` - -2. train/test stage2-inpaint -``` -sh run_stage2.sh & sh run_test_stage2.sh -``` - -3. train/test stage3-refined -``` -sh run_stage3.sh & sh run_test_stage3.sh -``` - -## Citation -If this work is useful to you, please consider citing our paper: -``` -@article{shen2023advancing, - title={Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Models}, - author={Shen, Fei and Ye, Hu and Zhang, Jun and Wang, Cong and Han, Xiao and Yang, Wei}, - journal={arXiv preprint arXiv:2310.06313}, - year={2023} -} - -``` - -## Contact -If you have any questions, please feel free to contact with me at shenfei140721@126.com. +![b7de2de3fd00aff5085b6a5e54a5bca](https://github.com/StartHua/ComfyUI_PCDMs/assets/22284244/8c4d79f8-ba68-43c2-8bad-47a5ad0ddd00)