nodes/city96-SD-Latent-Interposer
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update README.md

Browse Source
This commit is contained in:
City
2023-07-30 06:26:01 +02:00
committed by GitHub
parent a4e3fe9561
commit 57067ff2f8
1 changed files with 22 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+22
View File
@@ -1,2 +1,24 @@
# SD-Latent-Interposer
A small neural network to provide interoperability between the latents generated by the different Stable Diffusion models.
## Usage
I wanted to see if it was possible to pass latents generated by the new SDXL model directly into SDv1.5 models without decoding and re-encoding them using a VAE first.
See below for an example on how to use it. xl=>v1 conversion is almost flawless, **v1=>xl seems to produce artifacts.**
![LATENT_INTERPOSER_V3_TEST](https://github.com/city96/SD-Latent-Interposer/assets/125218114/4e15f7b6-e853-417d-ab58-205c1c99e507)
![LATENT_INTERPOSER_V3 1](https://github.com/city96/SD-Latent-Interposer/assets/125218114/24e2864e-d20f-4977-b218-dff0bf0fdc9f)
## Training
The training script should spit out a working model, nn layout is probably not optimal but I'm pretty short on VRAM to trial and error a better layout. PRs welcome.
Not sure why the training loss is so different, it might be due to the """highly curated""" dataset of 1000 random images from my Downloads folder that I used to train it.
I probably should've just grabbed LAION.
### v1.0 Training loss/progress
![loss](https://github.com/city96/SD-Latent-Interposer/assets/125218114/89e996b9-3baa-4027-930c-38dc6ec5ec24)
![xl-to-v1_interposer](https://github.com/city96/SD-Latent-Interposer/assets/125218114/d94ee64b-5417-40f4-a582-bcd2d0ac4365)