diff --git a/README.md b/README.md index 88a1d3b..1d8dfaf 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,24 @@ # SD-Latent-Interposer A small neural network to provide interoperability between the latents generated by the different Stable Diffusion models. + +## Usage +I wanted to see if it was possible to pass latents generated by the new SDXL model directly into SDv1.5 models without decoding and re-encoding them using a VAE first. + +See below for an example on how to use it. xl=>v1 conversion is almost flawless, **v1=>xl seems to produce artifacts.** + +![LATENT_INTERPOSER_V3_TEST](https://github.com/city96/SD-Latent-Interposer/assets/125218114/4e15f7b6-e853-417d-ab58-205c1c99e507) + +![LATENT_INTERPOSER_V3 1](https://github.com/city96/SD-Latent-Interposer/assets/125218114/24e2864e-d20f-4977-b218-dff0bf0fdc9f) + +## Training +The training script should spit out a working model, nn layout is probably not optimal but I'm pretty short on VRAM to trial and error a better layout. PRs welcome. + +Not sure why the training loss is so different, it might be due to the """highly curated""" dataset of 1000 random images from my Downloads folder that I used to train it. + +I probably should've just grabbed LAION. + +### v1.0 Training loss/progress + +![loss](https://github.com/city96/SD-Latent-Interposer/assets/125218114/89e996b9-3baa-4027-930c-38dc6ec5ec24) + +![xl-to-v1_interposer](https://github.com/city96/SD-Latent-Interposer/assets/125218114/d94ee64b-5417-40f4-a582-bcd2d0ac4365)