From 7387ea5de5a1435b292dfc7ad87da84cc6362d5f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jukka=20Sepp=C3=A4nen?= <40791699+kijai@users.noreply.github.com> Date: Wed, 30 Oct 2024 15:52:46 +0200 Subject: [PATCH 1/2] Update README.md --- README.md | 76 +++++++++---------------------------------------------- 1 file changed, 12 insertions(+), 64 deletions(-) diff --git a/README.md b/README.md index 83ca21a..99f96e8 100644 --- a/README.md +++ b/README.md @@ -1,72 +1,20 @@ # ComfyUI wrapper nodes for [Pyramid-Flow](https://github.com/jy0205/Pyramid-Flow) +## UPDATE +As the first Flux version is out, I'm dropping the SD3 support and refactored the whole thing, if you still want to use the old nodes they are archived in the legacy branch -https://github.com/user-attachments/assets/9592bfed-9cf8-438a-b1e6-969d6b13ec12 +The fluxmini version can run with 7GB VRAM, currently it only supports 5 second videos (temp 16) + +Download models from: + +https://huggingface.co/Kijai/pyramid-flow-comfy/tree/main + +To `ComfyUI/models/diffusion_models` and `ComfyUI/models/vae` + +https://github.com/user-attachments/assets/1372549a-4b4e-4569-a062-8f72880e8c4e -todo -- refactor to use comfy text encoding instead -- optimize memory use +https://github.com/user-attachments/assets/d0bd38eb-6378-4cfa-ae55-1b4498b7ce84 -Besides text encoder, which can peak at ~12GB VRAM use, this should run at 9-10GB VRAM when using 1280x768. -With fp8 and 384p model it can fit under 6GB too. Note that these tests were done on 4090, older cards may not support every optimization. - -Resolutions outside the model defaults perform poorly. - -Model loading has not been optimized at all yet, currently needs everything (choose either of the transformers) from here: - -https://huggingface.co/rain1011/pyramid-flow-sd3/tree/main - -to: - -`ComfyUI/models/pyramidflow/pyramid-flow-sd3` - -So that the directory structure is as follows: -``` -\ComfyUI\models\pyramidflow\pyramid-flow-sd3 -├───causal_video_vae -│ config.json -│ diffusion_pytorch_model.safetensors -│ -├───diffusion_transformer_384p -│ config.json -│ diffusion_pytorch_model.safetensors -│ -├───diffusion_transformer_768p -│ config.json -│ diffusion_pytorch_model.safetensors -│ -├───text_encoder -│ config.json -│ model.safetensors -│ -├───text_encoder_2 -│ config.json -│ model.safetensors -│ -├───text_encoder_3 -│ config.json -│ model-00001-of-00002.safetensors -│ model-00002-of-00002.safetensors -│ model.safetensors.index.json -│ -├───tokenizer -│ merges.txt -│ special_tokens_map.json -│ tokenizer_config.json -│ vocab.json -│ -├───tokenizer_2 -│ merges.txt -│ special_tokens_map.json -│ tokenizer_config.json -│ vocab.json -│ -└───tokenizer_3 - special_tokens_map.json - spiece.model - tokenizer.json - tokenizer_config.json -``` Original repo: https://github.com/jy0205/Pyramid-Flow From eb5668b88b92a95cfc1ba230a9babc94a16828d8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jukka=20Sepp=C3=A4nen?= <40791699+kijai@users.noreply.github.com> Date: Wed, 30 Oct 2024 15:53:19 +0200 Subject: [PATCH 2/2] Update README.md --- README.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 99f96e8..00cc234 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,8 @@ ## UPDATE As the first Flux version is out, I'm dropping the SD3 support and refactored the whole thing, if you still want to use the old nodes they are archived in the legacy branch -The fluxmini version can run with 7GB VRAM, currently it only supports 5 second videos (temp 16) +The fluxmini version can run with 7GB VRAM, currently it only supports 5 second videos (temp 16). +Fp8 severely reduces quality and is not recommended, only use it if you must. Download models from: