From 7a37319826a0ad34b9d602958075a6bc840ed6f5 Mon Sep 17 00:00:00 2001 From: City <125218114+city96@users.noreply.github.com> Date: Sat, 28 Oct 2023 00:05:55 +0200 Subject: [PATCH] Formatting / order Models should be at the top --- README.md | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index 34bf6de..f336e5d 100644 --- a/README.md +++ b/README.md @@ -6,6 +6,7 @@ This repository aims to add support for various random image diffusion models to Simply clone this repo to your custom_nodes folder using the following command: `git clone https://github.com/city96/ComfyUI_ExtraModels custom_nodes/ComfyUI_ExtraModels`. + ## DiT ### Model info / implementation - Uses class labels instead of prompts @@ -26,6 +27,22 @@ ConditioningCombine nodes *should* work for combining multiple labels. The area ![DIT_WORKFLOW_IMG](https://github.com/city96/ComfyUI_ExtraModels/assets/125218114/cdd4ec94-b0eb-436a-bf23-a3bcef8d7b90) + +## PixArt + +This is mostly a proof of concept, as the model weights have not been released officially. [Sample workflow here](https://github.com/city96/ComfyUI_ExtraModels/files/13192747/PixArt.json) + +Make sure to `pip install timm==0.6.13`. xformers is optional but strongly recommended as torch SDP is only partially implemented, if that. + +Limitations: +- The default `KSampler` uses a different noise schedule/sampling algo (I think), so it most likely won't work as expected. +- `PixArt DPM Sampler` requires the negative prompt to be shorter than the positive prompt. +- `PixArt DPM Sampler` can only work with a batch size of 1. +- `PixArt T5 Text Encode` is from the reference implementation, therefore it doesn't support weights. `T5 Text Encode` support weights, but I can't attest to the correctness of the implementation. + +PixArt uses the same T5v1.1-xxl text encoder as DeepFloyd, so the T5 section of the readme also applies. + + ## T5 ### Model @@ -49,18 +66,6 @@ On windows, you may need a newer version of bitsandbytes for 4bit. Try `python - You may also need to upgrade transformers. `pip install --upgrade transformers` -## PixArt - -This is mostly a proof of concept, as the model weights have not been released officially. [Sample workflow here](https://github.com/city96/ComfyUI_ExtraModels/files/13192747/PixArt.json) - -Make sure to `pip install timm==0.6.13`. xformers is optional but strongly recommended as torch SDP is only partially implemented, if that. - -Limitations: -- The default `KSampler` uses a different noise schedule/sampling algo (I think), so it most likely won't work as expected. -- `PixArt DPM Sampler` requires the negative prompt to be shorter than the positive prompt. -- `PixArt DPM Sampler` can only work with a batch size of 1. -- `PixArt T5 Text Encode` is from the reference implementation, therefore it doesn't support weights. `T5 Text Encode` support weights, but I can't attest to the correctness of the implementation. - ## VAE