Merge branch 'main' of https://github.com/kijai/ComfyUI-PyramidFlowWrapper

2024-10-30 16:08:08 +02:00
parent ecd00cf903 eb5668b88b
commit bc8d4360fe
1 changed files with 13 additions and 64 deletions
@@ -1,72 +1,21 @@
 # ComfyUI wrapper nodes for [Pyramid-Flow](https://github.com/jy0205/Pyramid-Flow)

+## UPDATE
+As the first Flux version is out, I'm dropping the SD3 support and refactored the whole thing, if you still want to use the old nodes they are archived in the legacy branch

-https://github.com/user-attachments/assets/9592bfed-9cf8-438a-b1e6-969d6b13ec12
+The fluxmini version can run with 7GB VRAM, currently it only supports 5 second videos (temp 16). 
+Fp8 severely reduces quality and is not recommended, only use it if you must.
+
+Download models from:
+
+https://huggingface.co/Kijai/pyramid-flow-comfy/tree/main
+
+To `ComfyUI/models/diffusion_models` and `ComfyUI/models/vae`
+
+https://github.com/user-attachments/assets/1372549a-4b4e-4569-a062-8f72880e8c4e


-todo
- refactor to use comfy text encoding instead
- optimize memory use
+https://github.com/user-attachments/assets/d0bd38eb-6378-4cfa-ae55-1b4498b7ce84

-Besides text encoder, which can peak at ~12GB VRAM use, this should run at 9-10GB VRAM when using 1280x768.
-With fp8 and 384p model it can fit under 6GB too. Note that these tests were done on 4090, older cards may not support every optimization.
-
-Resolutions outside the model defaults perform poorly.
-
-Model loading has not been optimized at all yet, currently needs everything (choose either of the transformers) from here:
-
-https://huggingface.co/rain1011/pyramid-flow-sd3/tree/main
-
-to:
-
-`ComfyUI/models/pyramidflow/pyramid-flow-sd3`
-
-So that the directory structure is as follows:
-```
-\ComfyUI\models\pyramidflow\pyramid-flow-sd3
-├───causal_video_vae
-│       config.json
-│       diffusion_pytorch_model.safetensors
-│
-├───diffusion_transformer_384p
-│       config.json
-│       diffusion_pytorch_model.safetensors
-│
-├───diffusion_transformer_768p
-│       config.json
-│       diffusion_pytorch_model.safetensors
-│
-├───text_encoder
-│       config.json
-│       model.safetensors
-│
-├───text_encoder_2
-│       config.json
-│       model.safetensors
-│
-├───text_encoder_3
-│       config.json
-│       model-00001-of-00002.safetensors
-│       model-00002-of-00002.safetensors
-│       model.safetensors.index.json
-│
-├───tokenizer
-│       merges.txt
-│       special_tokens_map.json
-│       tokenizer_config.json
-│       vocab.json
-│
-├───tokenizer_2
-│       merges.txt
-│       special_tokens_map.json
-│       tokenizer_config.json
-│       vocab.json
-│
-└───tokenizer_3
-        special_tokens_map.json
-        spiece.model
-        tokenizer.json
-        tokenizer_config.json
-```

 Original repo: https://github.com/jy0205/Pyramid-Flow