add new

2024-09-06 13:28:23 +08:00
parent 91c0828ca1
commit d0545f90fe
27 changed files with 0 additions and 1225 deletions
@@ -1,262 +0,0 @@
-<div align="center">
-<h1>CSGO: Content-Style Composition in Text-to-Image Generation</h1>
-
-[**Peng Xing**](https://github.com/xingp-ng)<sup>12*</sup> · [**Haofan Wang**](https://haofanwang.github.io/)<sup>1*</sup> · [**Yanpeng Sun**](https://scholar.google.com.hk/citations?user=a3FI8c4AAAAJ&hl=zh-CN&oi=ao/)<sup>2</sup> · [**Qixun Wang**](https://github.com/wangqixun)<sup>1</sup> · [**Xu Bai**](https://huggingface.co/baymin0220)<sup>1</sup> · [**Hao Ai**](https://github.com/aihao2000)<sup>13</sup> · [**Renyuan Huang**](https://github.com/DannHuang)<sup>14</sup> · [**Zechao Li**](https://zechao-li.github.io/)<sup>2✉</sup>
-
-<sup>1</sup>InstantX Team · <sup>2</sup>Nanjing University of Science and Technology · <sup>3</sup>Beihang University · <sup>4</sup>Peking University
-
-<sup>*</sup>equal contributions, <sup>✉</sup>corresponding authors
-
-<a href='https://csgo-gen.github.io/'><img src='https://img.shields.io/badge/Project-Page-green'></a>
-<a href='https://arxiv.org/abs/2404.02733'><img src='https://img.shields.io/badge/Technique-Report-red'></a>
-[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-blue)](https://huggingface.co/InstantX/CSGO)
-[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-App-red)](https://huggingface.co/spaces/xingpng/CSGO/)
-[![GitHub](https://img.shields.io/github/stars/instantX-research/CSGO?style=social)](https://github.com/instantX-research/CSGO)
-</div>
-
-
-##  Updates 🔥
-
-[//]: # (- **`2024/07/19`**: ✨ We support 🎞️ portrait video editing &#40;aka v2v&#41;! More to see [here]&#40;assets/docs/changelog/2024-07-19.md&#41;.)
-
-[//]: # (- **`2024/07/17`**: 🍎 We support macOS with Apple Silicon, modified from [jeethu]&#40;https://github.com/jeethu&#41;'s PR [#143]&#40;https://github.com/KwaiVGI/LivePortrait/pull/143&#41;.)
-
-[//]: # (- **`2024/07/10`**: 💪 We support audio and video concatenating, driving video auto-cropping, and template making to protect privacy. More to see [here]&#40;assets/docs/changelog/2024-07-10.md&#41;.)
-
-[//]: # (- **`2024/07/09`**: 🤗 We released the [HuggingFace Space]&#40;https://huggingface.co/spaces/KwaiVGI/liveportrait&#41;, thanks to the HF team and [Gradio]&#40;https://github.com/gradio-app/gradio&#41;!)
-[//]: # (Continuous updates, stay tuned!)
- **`2024/09/04`**: 🔥 We released the gradio code. You can simply configure it and use it directly.
- **`2024/09/03`**: 🔥 We released the online demo on [Hugggingface](https://huggingface.co/spaces/xingpng/CSGO/).
- **`2024/09/03`**: 🔥 We released the [pre-trained weight](https://huggingface.co/InstantX/CSGO).
- **`2024/09/03`**: 🔥 We released the initial version of the inference code.
- **`2024/08/30`**: 🔥 We released the technical report on [arXiv](https://arxiv.org/pdf/2408.16766)
- **`2024/07/15`**: 🔥 We released the [homepage](https://csgo-gen.github.io).
-
-##   Plan 💪
- [x]  technical report
- [x]  inference code
- [x]  pre-trained weight [4_16]
- [x]  pre-trained weight [4_32]
- [x]  online demo
- [ ]  IMAGStyle dataset
- [ ]  training code
- [ ]  more pre-trained weight 
-
-## Introduction 📖
-This repo, named **CSGO**, contains the official PyTorch implementation of our paper [CSGO: Content-Style Composition in Text-to-Image Generation](https://arxiv.org/pdf/).
-We are actively updating and improving this repository. If you find any bugs or have suggestions, welcome to raise issues or submit pull requests (PR) 💖.
-
-## Pipeline 	💻
-<p align="center">
-  <img src="assets/image3_1.jpg">
-</p>
-
-## Capabilities 🚅 
-
-  🔥 Our CSGO achieves **image-driven style transfer, text-driven stylized synthesis, and text editing-driven stylized synthesis**.
-
-  🔥 For more results, visit our <a href="https://csgo-gen.github.io"><strong>homepage</strong></a> 🔥
-
-<p align="center">
-  <img src="assets/vis.jpg">
-</p>
-
-
-## Getting Started 🏁
-### 1. Clone the code and prepare the environment
-```bash
-git clone https://github.com/instantX-research/CSGO
-cd CSGO
-
-# create env using conda
-conda create -n CSGO python=3.9
-conda activate CSGO
-
-# install dependencies with pip
-# for Linux and Windows users
-pip install -r requirements.txt
-```
-
-### 2. Download pretrained weights
-
-The easiest way to download the pretrained weights is from HuggingFace:
-```bash
-# first, ensure git-lfs is installed, see: https://docs.github.com/en/repositories/working-with-files/managing-large-files/installing-git-large-file-storage
-git lfs install
-# clone and move the weights
-git clone https://huggingface.co/InstanX/CSGO CSGO
-```
-Our method is fully compatible with [SDXL](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0), [VAE](https://huggingface.co/madebyollin/sdxl-vae-fp16-fix), [ControlNet](https://huggingface.co/TTPlanet/TTPLanet_SDXL_Controlnet_Tile_Realistic), and [Image Encoder](https://huggingface.co/h94/IP-Adapter/tree/main/sdxl_models/image_encoder).
-Please download them and place them in the ./base_models folder.
-
-tips:If you expect to load Controlnet directly using ControlNetPipeline as in CSGO, do the following:
-```bash
-git clone https://huggingface.co/TTPlanet/TTPLanet_SDXL_Controlnet_Tile_Realistic
-mv TTPLanet_SDXL_Controlnet_Tile_Realistic/TTPLANET_Controlnet_Tile_realistic_v2_fp16.safetensors TTPLanet_SDXL_Controlnet_Tile_Realistic/diffusion_pytorch_model.safetensors
-```
-### 3. Inference 🚀
-
-```python
-import torch
-from ip_adapter.utils import BLOCKS as BLOCKS
-from ip_adapter.utils import controlnet_BLOCKS as controlnet_BLOCKS
-from PIL import Image
-from diffusers import (
-    AutoencoderKL,
-    ControlNetModel,
-    StableDiffusionXLControlNetPipeline,
-
-)
-from ip_adapter import CSGO
-
-
-device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
-
-base_model_path =  "./base_models/stable-diffusion-xl-base-1.0"  
-image_encoder_path = "./base_models/IP-Adapter/sdxl_models/image_encoder"
-csgo_ckpt = "./CSGO/csgo.bin"
-pretrained_vae_name_or_path ='./base_models/sdxl-vae-fp16-fix'
-controlnet_path = "./base_models/TTPLanet_SDXL_Controlnet_Tile_Realistic"
-weight_dtype = torch.float16
-
-
-vae = AutoencoderKL.from_pretrained(pretrained_vae_name_or_path,torch_dtype=torch.float16)
-controlnet = ControlNetModel.from_pretrained(controlnet_path, torch_dtype=torch.float16,use_safetensors=True)
-pipe = StableDiffusionXLControlNetPipeline.from_pretrained(
-    base_model_path,
-    controlnet=controlnet,
-    torch_dtype=torch.float16,
-    add_watermarker=False,
-    vae=vae
-)
-pipe.enable_vae_tiling()
-
-
-target_content_blocks = BLOCKS['content']
-target_style_blocks = BLOCKS['style']
-controlnet_target_content_blocks = controlnet_BLOCKS['content']
-controlnet_target_style_blocks = controlnet_BLOCKS['style']
-
-csgo = CSGO(pipe, image_encoder_path, csgo_ckpt, device, num_content_tokens=4,num_style_tokens=32,
-                          target_content_blocks=target_content_blocks, target_style_blocks=target_style_blocks,controlnet=False,controlnet_adapter=True,
-                              controlnet_target_content_blocks=controlnet_target_content_blocks, 
-                              controlnet_target_style_blocks=controlnet_target_style_blocks,
-                              content_model_resampler=True,
-                              style_model_resampler=True,
-                              load_controlnet=False,
-
-                              )
-
-style_name = 'img_0.png'
-content_name = 'img_0.png'
-style_image = "../assets/{}".format(style_name)
-content_image = Image.open('../assets/{}'.format(content_name)).convert('RGB')
-
-caption ='a small house with a sheep statue on top of it'
-
-num_sample=4
-
-#image-driven style transfer
-images = csgo.generate(pil_content_image= content_image, pil_style_image=style_image,
-                           prompt=caption,
-                           negative_prompt= "text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry",
-                           content_scale=1.0,
-                           style_scale=1.0,
-                           guidance_scale=10,
-                           num_images_per_prompt=num_sample,
-                           num_samples=1,
-                           num_inference_steps=50,
-                           seed=42,
-                           image=content_image.convert('RGB'),
-                           controlnet_conditioning_scale=0.6,
-                          )
-
-#text-driven stylized synthesis
-caption='a cat'
-images = csgo.generate(pil_content_image= content_image, pil_style_image=style_image,
-                           prompt=caption,
-                           negative_prompt= "text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry",
-                           content_scale=1.0,
-                           style_scale=1.0,
-                           guidance_scale=10,
-                           num_images_per_prompt=num_sample,
-                           num_samples=1,
-                           num_inference_steps=50,
-                           seed=42,
-                           image=content_image.convert('RGB'),
-                           controlnet_conditioning_scale=0.01,
-                          )
-
-#text editing-driven stylized synthesis
-caption='a small house'
-images = csgo.generate(pil_content_image= content_image, pil_style_image=style_image,
-                           prompt=caption,
-                           negative_prompt= "text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry",
-                           content_scale=1.0,
-                           style_scale=1.0,
-                           guidance_scale=10,
-                           num_images_per_prompt=num_sample,
-                           num_samples=1,
-                           num_inference_steps=50,
-                           seed=42,
-                           image=content_image.convert('RGB'),
-                           controlnet_conditioning_scale=0.4,
-                          )
-```
-### 4 Gradio interface ⚙️
-
-We also provide a Gradio <a href='https://github.com/gradio-app/gradio'><img src='https://img.shields.io/github/stars/gradio-app/gradio'></a> interface for a better experience, just run by:
-
-```bash
-# For Linux and Windows users (and macOS)
-python gradio/app.py 
-```
-If you don't have the resources to configure it, we provide an online [demo](https://huggingface.co/spaces/xingpng/CSGO/).
-## Demos
-<p align="center">
-  <br>
-  🔥 For more results, visit our <a href="https://csgo-gen.github.io"><strong>homepage</strong></a> 🔥
-</p>
-
-### Content-Style Composition
-<p align="center">
-  <img src="assets/page1.png">
-</p>
-
-<p align="center">
-  <img src="assets/page4.png">
-</p>
-
-### Cycle Translation
-<p align="center">
-  <img src="assets/page8.png">
-</p>
-
-### Text-Driven Style Synthesis
-<p align="center">
-  <img src="assets/page10.png">
-</p>
-
-### Text Editing-Driven Style Synthesis
-<p align="center">
-  <img src="assets/page11.jpg">
-</p>
-
-## Star History
-[![Star History Chart](https://api.star-history.com/svg?repos=instantX-research/CSGO&type=Date)](https://star-history.com/#instantX-research/CSGO&Date)
-
-
-
-## Acknowledgements
-This project is developed by InstantX Team, all copyright reserved.
-
-## Citation 💖
-If you find CSGO useful for your research, welcome to 🌟 this repo and cite our work using the following BibTeX:
-```bibtex
-@article{xing2024csgo,
-       title={CSGO: Content-Style Composition in Text-to-Image Generation}, 
-       author={Peng Xing and Haofan Wang and Yanpeng Sun and Qixun Wang and Xu Bai and Hao Ai and Renyuan Huang and Zechao Li},
-       year={2024},
-       journal = {arXiv 2408.16766},
-}
-```
@@ -1,343 +0,0 @@
-import sys
-# sys.path.append("../")
-sys.path.append("./")
-import gradio as gr
-import torch
-from ip_adapter.utils import BLOCKS as BLOCKS
-from ip_adapter.utils import controlnet_BLOCKS as controlnet_BLOCKS
-from ip_adapter.utils import resize_content
-import cv2
-import numpy as np
-import random
-from PIL import Image
-from transformers import AutoImageProcessor, AutoModel
-from diffusers import (
-    AutoencoderKL,
-    ControlNetModel,
-    StableDiffusionXLControlNetPipeline,
-
-)
-from ip_adapter import CSGO
-from transformers import BlipProcessor, BlipForConditionalGeneration
-
-device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
-
-base_model_path = "stabilityai/stable-diffusion-xl-base-1.0"
-image_encoder_path = "h94/IP-Adapter/sdxl_models/image_encoder"
-csgo_ckpt ='InstantX/CSGO/csgo_4_32.bin'
-pretrained_vae_name_or_path ='madebyollin/sdxl-vae-fp16-fix'
-controlnet_path = "TTPlanet/TTPLanet_SDXL_Controlnet_Tile_Realistic"
-weight_dtype = torch.float16
-
-
-
-
-vae = AutoencoderKL.from_pretrained(pretrained_vae_name_or_path,torch_dtype=torch.float16)
-controlnet = ControlNetModel.from_pretrained(controlnet_path, torch_dtype=torch.float16,use_safetensors=True)
-pipe = StableDiffusionXLControlNetPipeline.from_pretrained(
-    base_model_path,
-    controlnet=controlnet,
-    torch_dtype=torch.float16,
-    add_watermarker=False,
-    vae=vae
-)
-pipe.enable_vae_tiling()
-
-
-target_content_blocks = BLOCKS['content']
-target_style_blocks = BLOCKS['style']
-controlnet_target_content_blocks = controlnet_BLOCKS['content']
-controlnet_target_style_blocks = controlnet_BLOCKS['style']
-
-
-
-MAX_SEED = np.iinfo(np.int32).max
-
-def randomize_seed_fn(seed: int, randomize_seed: bool) -> int:
-    if randomize_seed:
-        seed = random.randint(0, MAX_SEED)
-    return seed
-
-
-
-
-
-def get_example():
-    case = [
-        [
-            "./assets/img_0.png",
-            './assets/img_1.png',
-            "Image-Driven Style Transfer",
-            "there is a small house with a sheep statue on top of it",
-            0.6,
-            1.0,
-        ],
-        [
-         None,
-         './assets/img_1.png',
-            "Text-Driven Style Synthesis",
-         "a cat",
-         0.01,1.0
-         ],
-        [
-            None,
-            './assets/img_2.png',
-            "Text-Driven Style Synthesis",
-            "a building",
-            0.01, 1.0
-        ],
-        [
-            "./assets/img_0.png",
-            './assets/img_1.png',
-            "Text Edit-Driven Style Synthesis",
-            "there is a small house",
-            0.4,
-            1.0
-        ],
-    ]
-    return case
-
-
-def run_for_examples(content_image_pil,style_image_pil,target, prompt, scale_c, scale_s):
-    return create_image(
-        content_image_pil=content_image_pil,
-        style_image_pil=style_image_pil,
-        prompt=prompt,
-        scale_c=scale_c,
-        scale_s=scale_s,
-        guidance_scale=7.0,
-        num_samples=3,
-        num_inference_steps=50,
-        seed=42,
-        target=target,
-    )
-def randomize_seed_fn(seed: int, randomize_seed: bool) -> int:
-    if randomize_seed:
-        seed = random.randint(0, MAX_SEED)
-    return seed
-
-def image_grid(imgs, rows, cols):
-    assert len(imgs) == rows * cols
-
-    w, h = imgs[0].size
-    grid = Image.new('RGB', size=(cols * w, rows * h))
-    grid_w, grid_h = grid.size
-
-    for i, img in enumerate(imgs):
-        grid.paste(img, box=(i % cols * w, i // cols * h))
-    return grid
-def create_image(content_image_pil,
-                 style_image_pil,
-                 prompt,
-                 scale_c,
-                 scale_s,
-                 guidance_scale,
-                 num_samples,
-                 num_inference_steps,
-                 seed,
-                 target="Image-Driven Style Transfer",
-):
-
-
-    csgo = CSGO(pipe, image_encoder_path, csgo_ckpt, device, num_content_tokens=4, num_style_tokens=32,
-                target_content_blocks=target_content_blocks, target_style_blocks=target_style_blocks,
-                controlnet_adapter=True,
-                controlnet_target_content_blocks=controlnet_target_content_blocks,
-                controlnet_target_style_blocks=controlnet_target_style_blocks,
-                content_model_resampler=True,
-                style_model_resampler=True,
-                )
-
-
-    if content_image_pil is None:
-        content_image_pil = Image.fromarray(
-            np.zeros((1024, 1024, 3), dtype=np.uint8)).convert('RGB')
-
-    if prompt is None:
-        blip_processor = BlipProcessor.from_pretrained("Salesforce/blip-image-captioning-large")
-        blip_model = BlipForConditionalGeneration.from_pretrained("Salesforce/blip-image-captioning-large").to(device)
-        with torch.no_grad():
-            inputs = blip_processor(content_image_pil, return_tensors="pt").to(device)
-            out = blip_model.generate(**inputs)
-            prompt = blip_processor.decode(out[0], skip_special_tokens=True)
-    width, height, content_image = resize_content(content_image_pil)
-    style_image = style_image_pil
-    neg_content_prompt='text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry'
-    if target =="Image-Driven Style Transfer":
-        with torch.no_grad():
-            images = csgo.generate(pil_content_image=content_image, pil_style_image=style_image,
-                                   prompt=prompt,
-                                   negative_prompt=neg_content_prompt,
-                                   height=height,
-                                   width=width,
-                                   content_scale=1.0,
-                                   style_scale=scale_s,
-                                   guidance_scale=guidance_scale,
-                                   num_images_per_prompt=num_samples,
-                                   num_inference_steps=num_inference_steps,
-                                   num_samples=1,
-                                   seed=seed,
-                                   image=content_image.convert('RGB'),
-                                   controlnet_conditioning_scale=scale_c,
-                                   )
-
-    elif target =="Text-Driven Style Synthesis":
-        content_image = Image.fromarray(
-            np.zeros((1024, 1024, 3), dtype=np.uint8)).convert('RGB')
-        with torch.no_grad():
-            images = csgo.generate(pil_content_image=content_image, pil_style_image=style_image,
-                                   prompt=prompt,
-                                   negative_prompt="text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry",
-                                   height=height,
-                                   width=width,
-                                   content_scale=0.5,
-                                   style_scale=scale_s,
-                                   guidance_scale=7,
-                                   num_images_per_prompt=num_samples,
-                                   num_inference_steps=num_inference_steps,
-                                   num_samples=1,
-                                   seed=42,
-                                   image=content_image.convert('RGB'),
-                                   controlnet_conditioning_scale=scale_c,
-                                   )
-    elif target =="Text Edit-Driven Style Synthesis":
-
-        with torch.no_grad():
-            images = csgo.generate(pil_content_image=content_image, pil_style_image=style_image,
-                                   prompt=prompt,
-                                   negative_prompt=neg_content_prompt,
-                                   height=height,
-                                   width=width,
-                                   content_scale=1.0,
-                                   style_scale=scale_s,
-                                   guidance_scale=guidance_scale,
-                                   num_images_per_prompt=num_samples,
-                                   num_inference_steps=num_inference_steps,
-                                   num_samples=1,
-                                   seed=seed,
-                                   image=content_image.convert('RGB'),
-                                   controlnet_conditioning_scale=scale_c,
-                                   )
-
-    return [image_grid(images, 1, num_samples)]
-
-
-def pil_to_cv2(image_pil):
-    image_np = np.array(image_pil)
-    image_cv2 = cv2.cvtColor(image_np, cv2.COLOR_RGB2BGR)
-    return image_cv2
-
-
-# Description
-title = r"""
-<h1 align="center">CSGO: Content-Style Composition in Text-to-Image Generation</h1>
-"""
-
-description = r"""
-<b>Official 🤗 Gradio demo</b> for <a href='https://github.com/instantX-research/CSGO' target='_blank'><b>CSGO: Content-Style Composition in Text-to-Image Generation</b></a>.<br>
-How to use:<br>
-1. Upload a content image if you want to use image-driven style transfer.
-2. Upload a style image.
-3. Sets the type of task to perform, by default image-driven style transfer is performed. Options are <b>Image-driven style transfer, Text-driven style synthesis, and Text editing-driven style synthesis<b>.
-4. <b>If you choose a text-driven task, enter your desired prompt<b>.
-5.If you don't provide a prompt, the default is to use the BLIP model to generate the caption.
-6. Click the <b>Submit</b> button to begin customization.
-7. Share your stylized photo with your friends and enjoy! 😊
-
-Advanced usage:<br>
-1. Click advanced options.
-2. Choose different guidance and steps.
-"""
-
-article = r"""
---
-📝 **Tips**
-In CSGO, the more accurate the text prompts for content images, the better the content retention.
-Text-driven style synthesis and text-edit-driven style synthesis are expected to be more stable in the next release.
---
-📝 **Citation**
-<br>
-If our work is helpful for your research or applications, please cite us via:
-```bibtex
-@article{xing2024csgo,
-       title={CSGO: Content-Style Composition in Text-to-Image Generation}, 
-       author={Peng Xing and Haofan Wang and Yanpeng Sun and Qixun Wang and Xu Bai and Hao Ai and Renyuan Huang and Zechao Li},
-       year={2024},
-       journal = {arXiv 2408.16766},
-}
-```
-📧 **Contact**
-<br>
-If you have any questions, please feel free to open an issue or directly reach us out at <b>xingp_ng@njust.edu.cn</b>.
-"""
-
-block = gr.Blocks(css="footer {visibility: hidden}").queue(max_size=10, api_open=False)
-with block:
-    # description
-    gr.Markdown(title)
-    gr.Markdown(description)
-
-    with gr.Tabs():
-        with gr.Row():
-            with gr.Column():
-                with gr.Row():
-                    with gr.Column():
-                        content_image_pil = gr.Image(label="Content Image (optional)", type='pil')
-                        style_image_pil = gr.Image(label="Style Image", type='pil')
-
-                target = gr.Radio(["Image-Driven Style Transfer", "Text-Driven Style Synthesis", "Text Edit-Driven Style Synthesis"],
-                                  value="Image-Driven Style Transfer",
-                                  label="task")
-
-                prompt = gr.Textbox(label="Prompt",
-                                    value="there is a small house with a sheep statue on top of it")
-
-
-                scale_c = gr.Slider(minimum=0, maximum=2.0, step=0.01, value=0.6, label="Content Scale")
-                scale_s = gr.Slider(minimum=0, maximum=2.0, step=0.01, value=1.0, label="Style Scale")
-                with gr.Accordion(open=False, label="Advanced Options"):
-
-                    guidance_scale = gr.Slider(minimum=1, maximum=15.0, step=0.01, value=7.0, label="guidance scale")
-                    num_samples = gr.Slider(minimum=1, maximum=4.0, step=1.0, value=1.0, label="num samples")
-                    num_inference_steps = gr.Slider(minimum=5, maximum=100.0, step=1.0, value=50,
-                                                    label="num inference steps")
-                    seed = gr.Slider(minimum=-1000000, maximum=1000000, value=1, step=1, label="Seed Value")
-                    randomize_seed = gr.Checkbox(label="Randomize seed", value=True)
-
-                generate_button = gr.Button("Generate Image")
-
-            with gr.Column():
-                generated_image = gr.Gallery(label="Generated Image")
-
-        generate_button.click(
-            fn=randomize_seed_fn,
-            inputs=[seed, randomize_seed],
-            outputs=seed,
-            queue=False,
-            api_name=False,
-        ).then(
-            fn=create_image,
-            inputs=[content_image_pil,
-                    style_image_pil,
-                    prompt,
-
-                    scale_c,
-                    scale_s,
-                    guidance_scale,
-                    num_samples,
-                    num_inference_steps,
-                    seed,
-                    target,],
-            outputs=[generated_image])
-
-    gr.Examples(
-        examples=get_example(),
-        inputs=[content_image_pil,style_image_pil,target, prompt, scale_c, scale_s],
-        fn=run_for_examples,
-        outputs=[generated_image],
-        cache_examples=True,
-    )
-
-    gr.Markdown(article)
-
-block.launch(server_name="0.0.0.0", server_port=1234)
@@ -1,18 +0,0 @@
-diffusers==0.25.1
-torch==2.0.1
-torchaudio==2.0.2
-torchvision==0.15.2
-transformers==4.40.2
-accelerate
-safetensors
-einops
-spaces==0.19.4
-omegaconf
-peft
-huggingface-hub==0.24.5
-opencv-python
-insightface
-gradio
-controlnet_aux
-gdown
-peft
@@ -1,86 +0,0 @@
-import os
-os.environ['HF_ENDPOINT']='https://hf-mirror.com'
-import torch
-from ip_adapter.utils import BLOCKS as BLOCKS
-from ip_adapter.utils import controlnet_BLOCKS as controlnet_BLOCKS
-from ip_adapter.utils import resize_content
-import cv2
-from PIL import Image
-from transformers import AutoImageProcessor, AutoModel
-from diffusers import (
-    AutoencoderKL,
-    ControlNetModel,
-    StableDiffusionXLControlNetPipeline,
-
-)
-from ip_adapter import CSGO
-from transformers import BlipProcessor, BlipForConditionalGeneration
-
-device = torch.device("cuda:1" if torch.cuda.is_available() else "cpu")
-
-base_model_path =  "../../../base_models/stable-diffusion-xl-base-1.0"
-image_encoder_path = "../../../base_models/IP-Adapter/sdxl_models/image_encoder"
-csgo_ckpt = "/share2/xingpeng/DATA/blora/outputs/content_style_checkpoints_2/base_train_free_controlnet_S12_alldata_C_0_S_I_zero2_style_res_32_content_res4_drop/checkpoint-220000/ip_adapter.bin"
-pretrained_vae_name_or_path ='../../../base_models/sdxl-vae-fp16-fix'
-controlnet_path = "../../../base_models/TTPLanet_SDXL_Controlnet_Tile_Realistic"
-weight_dtype = torch.float16
-
-blip_processor = BlipProcessor.from_pretrained("Salesforce/blip-image-captioning-large")
-blip_model = BlipForConditionalGeneration.from_pretrained("Salesforce/blip-image-captioning-large").to(device)
-
-
-vae = AutoencoderKL.from_pretrained(pretrained_vae_name_or_path,torch_dtype=torch.float16)
-controlnet = ControlNetModel.from_pretrained(controlnet_path, torch_dtype=torch.float16,use_safetensors=True)
-pipe = StableDiffusionXLControlNetPipeline.from_pretrained(
-    base_model_path,
-    controlnet=controlnet,
-    torch_dtype=torch.float16,
-    add_watermarker=False,
-    vae=vae
-)
-pipe.enable_vae_tiling()
-
-
-target_content_blocks = BLOCKS['content']
-target_style_blocks = BLOCKS['style']
-controlnet_target_content_blocks = controlnet_BLOCKS['content']
-controlnet_target_style_blocks = controlnet_BLOCKS['style']
-
-csgo = CSGO(pipe, image_encoder_path, csgo_ckpt, device, num_content_tokens=4,num_style_tokens=32,
-                          target_content_blocks=target_content_blocks, target_style_blocks=target_style_blocks,controlnet_adapter=True,
-                              controlnet_target_content_blocks=controlnet_target_content_blocks,
-                              controlnet_target_style_blocks=controlnet_target_style_blocks,
-                              content_model_resampler=True,
-                              style_model_resampler=True,
-                              )
-
-style_name = 'img_1.png'
-content_name = 'img_0.png'
-style_image = Image.open("../assets/{}".format(style_name)).convert('RGB')
-content_image = Image.open('../assets/{}'.format(content_name)).convert('RGB')
-
-
-with torch.no_grad():
-    inputs = blip_processor(content_image, return_tensors="pt").to(device)
-    out = blip_model.generate(**inputs)
-    caption = blip_processor.decode(out[0], skip_special_tokens=True)
-
-num_sample=1
-
-width,height,content_image  = resize_content(content_image)
-images = csgo.generate(pil_content_image= content_image, pil_style_image=style_image,
-                           prompt=caption,
-                           negative_prompt= "text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry",
-                           height=height,
-                           width=width,
-                           content_scale=0.5,
-                           style_scale=1.0,
-                           guidance_scale=10,
-                           num_images_per_prompt=num_sample,
-                           num_samples=1,
-                           num_inference_steps=50,
-                           seed=42,
-                           image=content_image.convert('RGB'),
-                           controlnet_conditioning_scale=0.6,
-                          )
-images[0].save("../assets/content_img_0_style_imag_1.png")
@@ -1,18 +0,0 @@
-diffusers==0.25.1
-torch==2.0.1
-torchaudio==2.0.2
-torchvision==0.15.2
-transformers==4.40.2
-accelerate
-safetensors
-einops
-spaces==0.19.4
-omegaconf
-peft
-huggingface-hub==0.24.5
-opencv-python
-insightface
-gradio
-controlnet_aux
-gdown
-peft