diff --git a/README.md b/README.md index 1d8dfaf..6aeebd3 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,20 @@ # SD-Latent-Interposer A small neural network to provide interoperability between the latents generated by the different Stable Diffusion models. -## Usage I wanted to see if it was possible to pass latents generated by the new SDXL model directly into SDv1.5 models without decoding and re-encoding them using a VAE first. -See below for an example on how to use it. xl=>v1 conversion is almost flawless, **v1=>xl seems to produce artifacts.** +## Installation +To install it, simply download [comfy_latent_interposer.py](https://github.com/city96/SD-Latent-Interposer/raw/main/comfy_latent_interposer.py) to your `ComfyUI/custom_nodes` folder. You may need to install hfhub using the command `pip install huggingface-hub` inside your venv. + +If you need the model weights for something else, they are [hosted on HF](https://huggingface.co/city96/SD-Latent-Interposer/tree/main) under the same Apache2 license as the rest of the repo. + +## Usage +See the image below for an example on how to use it. xl=>v1 conversion is almost flawless, **v1=>xl seems to produce artifacts.** ![LATENT_INTERPOSER_V3_TEST](https://github.com/city96/SD-Latent-Interposer/assets/125218114/4e15f7b6-e853-417d-ab58-205c1c99e507) +Without the interposer, the two latent spaces are incompatible: + ![LATENT_INTERPOSER_V3 1](https://github.com/city96/SD-Latent-Interposer/assets/125218114/24e2864e-d20f-4977-b218-dff0bf0fdc9f) ## Training @@ -17,6 +24,8 @@ Not sure why the training loss is so different, it might be due to the """highly I probably should've just grabbed LAION. +I also trained a v1-to-v2 mode, which + ### v1.0 Training loss/progress ![loss](https://github.com/city96/SD-Latent-Interposer/assets/125218114/89e996b9-3baa-4027-930c-38dc6ec5ec24)