Author SHA1 Message Date
Nuked 96b175ba26 Update README.md 2026-09-27 13:52:38 +02:00
Nuked 6d470d8a16 Gate stable publishing on nightly validation 2026-09-27 13:33:46 +02:00
Nuked ae7cc84808 Update pyproject.toml 2024-08-15 23:07:32 +02:00
Nuked e7a424818f Merge pull request #74 from ComfyNodePRs/licence-update
Update PyProject Toml - License
2024-08-15 23:05:44 +02:00
snomiao 580af2f8e2 chore(licence-update): Update PyProject Toml - License 2024-08-15 21:03:33 +00:00
Nuked 59ecf71663 - real_path in init
- implemented use_ram in video_node_advanced
2024-07-25 22:12:58 +02:00
Nuked bfc27bb91e Merge pull request #72 from Nuked88/revert-69-patch-1
Revert "Update extended_widgets.js"
2024-07-25 20:52:48 +02:00
Nuked 114cac7b63 Revert "Update extended_widgets.js" 2024-07-25 20:51:35 +02:00
Nuked 128f00610c Update pyproject.toml 2024-07-22 20:07:44 +02:00
Nuked 0e5cf8a96c Merge pull request #69 from markknol/patch-1
Update extended_widgets.js
2024-07-22 20:07:12 +02:00
Mark Knol 9a0bc9f6d1 Update extended_widgets.js
Fix runtime error on missing "graph" (should be app.graph)
2024-07-22 16:07:09 +02:00
Nuked ab4af9de4f Update pyproject.toml
Update PublisherId
2024-05-21 21:52:44 +02:00
Nuked b7c364e7ff Merge pull request #58 from haohaocreates/pyproject
Add pyproject.toml for Custom Node Registry
2024-05-21 19:52:11 +00:00
Nuked a6d034619d Merge pull request #57 from haohaocreates/publish
Add Github Action for Publishing to Comfy Registry
2024-05-21 19:52:01 +00:00
haohaocreates 87822f4943 Update pyproject.toml description 2024-05-21 15:16:31 -04:00
haohaocreates 3a4b12d374 chore(pyproject): Add pyproject.toml for Custom Node Registry 2024-05-21 19:15:21 +00:00
haohaocreates 77ac2e8662 chore(publish): Add Github Action for Publishing to Comfy Registry 2024-05-21 19:15:20 +00:00
Nuked dbeb9ae951 fix: git called before installation 2024-03-10 21:51:51 +01:00
Nuked 7bd098f4e4 - some fixes
- added image list management
2024-03-02 11:59:11 +01:00
Nuked ac47bc1679 - some graphic fix
- reboot alert
- Added some parameter to loadframefromfolder
2024-02-29 20:29:18 +01:00
Nuked 08d3efce6e Merge pull request #41 from mastfissh/filename-prefixes
Allow passing filename_prefix
2024-02-27 17:40:35 +01:00
Nuked eb3e4959f7 Hot Fix: load js issue in the last version of ComfyUI 27/02/2024 2024-02-27 17:15:26 +01:00
Justin f27a3d85a6 allow passing filename_prefix 2024-02-27 16:11:53 +11:00
Nuked b9cb62c0d4 new outpainting node 2024-02-25 11:01:35 +01:00
Nuked 26353714f0 experimental 2024-02-17 10:56:58 +01:00
Nuked dc265df9de fix moondream output 2024-02-16 05:06:00 +01:00
Nuked 3abbce4e70 fix dependencies 2024-02-15 17:49:13 +01:00
nuked88 8d7b09a51b addedd video example 2024-02-15 09:36:12 +00:00
nuked88 d2d8cfc398 fix img 2024-02-15 09:05:50 +00:00
nuked88 99c79b351e fix images 2024-02-15 08:59:09 +00:00
Nuked e8cc9af684 fix 2024-02-14 22:27:39 +01:00
Nuked aca90cee7d Merge branch 'main' of https://github.com/Nuked88/ComfyUI-N-Nodes 2024-02-14 22:23:42 +01:00
Nuked 86c8e00f8e - Changed all node names in order to avoid any conflicts with other extensions in the future
- Introduced Llava, moondream, joytag models
- Introduced new cliptextencodeAdvanced
- Added new output field to video node
2024-02-14 22:23:36 +01:00
Nuked 29b2e43bab Update video_node.py
addedd jpg
2024-01-15 21:26:59 +01:00
Nuked eb285e220d Update issue templates 2023-12-22 19:34:01 +01:00
Nuked 185ffdac03 Update issue templates 2023-12-22 18:48:16 +01:00
Nuked 8a142f7e0e Update README.md 2023-12-11 15:00:06 +01:00
Nuked 33d956d4a1 Fixed installation on cpu only of llama_cpp_python wheel 2023-12-10 19:42:50 +01:00
Nuked df512ef7e6 some fixes 2023-11-19 08:53:31 +01:00
Nuked c72970a917 tests for manager 2023-11-19 08:42:14 +01:00
Nuked a61712b5ec test for manager 2023-11-19 08:28:49 +01:00
Nuked 0f1d68f252 updated readme + some corrections 2023-11-14 20:09:05 +01:00
Nuked c6b88dee7e Merge pull request #18 from picobyte/patch-1
Delete py/__pycache__ directory, to allow ComfyUI updates
2023-11-11 21:03:33 +01:00
R d230b375e2 Delete py/__pycache__ directory, to allow ComfyUI updates
When you run ComfyUI update all, these files are always modified and prevent the update, they require a `git stash` and to manually `git pull -v -- origin` to check if there were updates, but really they are compiled python and should not be part off the repository, they are created while running.
2023-11-11 20:46:24 +01:00
Nuked 017bf2e47a troubleshoting for savevideo 2023-11-11 15:25:27 +01:00
Nuked bebeb61ec4 GGUF support 2023-11-11 08:10:46 +01:00
Nuked ae71244bb0 fix 2023-11-11 08:07:22 +01:00
Nuked 7da375e2e4 fix readme 2023-11-11 07:53:38 +01:00
Nuked c9a06b63ab readme update 2023-11-11 07:52:15 +01:00
Nuked 7810ca29b4 Made installation llama-cpp-python automatic
with CPU, GPU and CUDA detection
Other fixes.
2023-11-11 07:50:20 +01:00
Nuked 66546b96ad -Fixed some bugs
-Added autplay button in LoadVideo
-Added starting_frame setting in LoadVideo
2023-10-29 15:26:39 +01:00
Nuked e50fe2fe98 Fixed Bug that was causing comfyui to run
more slowly on the first ksampler job due to benckmark enabled
2023-10-27 21:03:57 +02:00
Nuked ae64a47e83 added cpu batch for llama_cpp 2023-10-26 19:50:29 +02:00
Nuked 833203dc02 Fixed images_limit and batch_size when
over the limit
Now it should be possible to use multiple LoadVideo in the same workflow
2023-10-20 21:56:42 +02:00
Nuked 0e4e77379e added important notice 2023-10-09 19:56:36 +02:00
Nuked f4d5b048c7 fix for was and
fixed llama-cpp-python to 0.1.84 for new models batch
2023-10-08 21:44:32 +02:00
Nuked 01422a0bbe added trackback for exceptions 2023-10-07 18:36:36 +02:00
Nuked 31897e469d added a generic metadata node 2023-10-07 12:47:43 +02:00
Nuked 932d9a3236 fixed nodes specifics 2023-10-06 19:54:21 +02:00
Nuked f9261b5546 - Fix for installing dependencies
without using the cached version.
- Readme Updated
- Added Save Frames
2023-10-05 22:18:40 +02:00
Nuked 6e28f925c0 renamed nodes,
auto install dependencies at startup for skbuild
2023-09-30 19:20:39 +02:00
Nuked 1379f4a94c dependencies downloading included in init 2023-09-30 16:20:57 +02:00
Nuked 5498b2051d fix + new interpolation node 2023-09-30 15:09:17 +02:00
Nuked 20e182ebf8 Revert "start implementing gif"
This reverts commit 11c48ad52e.
2023-09-29 19:58:22 +02:00
54 changed files with 3969 additions and 1275 deletions
+3
View File
@@ -0,0 +1,3 @@
# Development-only files are not needed in the Registry package.
tests/
pytest.ini
+37
View File
@@ -0,0 +1,37 @@
---
name: Bug report
about: Create a report to help us improve
title: "[BUG]"
labels: bug
assignees: ''
---
**Describe the bug**
A clear and concise description of what the bug is.
**To Reproduce**
Steps to reproduce the behavior:
1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error
**Expected behavior**
A clear and concise description of what you expected to happen.
**Full log**
This is MANDATORY. By log I mean all the text in the console from the time ComfyUI was started until the time of the reported bug.
>>Bug reports that do not have this log will be closed.<<
**Screenshots**
If applicable, add screenshots to help explain your problem.
**Desktop (please complete the following information):**
- OS: [e.g. iOS]
- Browser [e.g. chrome, safari]
- Version [e.g. 22]
**Additional context**
Add any other context about the problem here.
+20
View File
@@ -0,0 +1,20 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: enhancement
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
**Describe the solution you'd like**
A clear and concise description of what you want to happen.
**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.
**Additional context**
Add any other context or screenshots about the feature request here.
+20
View File
@@ -0,0 +1,20 @@
name: Publish to Comfy registry
on:
workflow_dispatch:
# Merging into main makes the commit available as the Manager's "nightly"
# version. Stable Registry releases are intentionally published only by
# manually running this workflow after nightly validation.
jobs:
publish-node:
name: Publish Custom Node to registry
runs-on: ubuntu-latest
steps:
- name: Check out code
uses: actions/checkout@v4
- name: Publish Custom Node
uses: Comfy-Org/publish-node-action@main
with:
## Add your own personal access token to your Github Repository secrets and reference it here.
personal_access_token: ${{ secrets.REGISTRY_ACCESS_TOKEN }}
+1 -1
View File
@@ -2,7 +2,7 @@
__pycache__/ __pycache__/
*.py[cod] *.py[cod]
*$py.class *$py.class
libs/moondream_repo
# C extensions # C extensions
*.so *.so
+200 -45
View File
@@ -1,5 +1,12 @@
# ComfyUI-N-Nodes [![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/C0C0AJECJ)
A suite of custom nodes for ComfyUI, for now i just put Integer, string and float variable nodes
# ComfyUI-N-Suite
A suite of custom nodes for ComfyUI that includes integer, string and float variable nodes, image-captioning nodes and video nodes.
> [!IMPORTANT]
> These nodes were tested primarily in Windows in the default environment provided by ComfyUI and in the environment created by the [notebook](https://github.com/comfyanonymous/ComfyUI/blob/master/notebooks/comfyui_colab.ipynb) for paperspace specifically with the cyberes/gradient-base-py3.10:latest docker image.
**Any other environment has not been tested.**
# Installation # Installation
@@ -7,16 +14,34 @@ A suite of custom nodes for ComfyUI, for now i just put Integer, string and floa
`git clone https://github.com/Nuked88/ComfyUI-N-Nodes.git` `git clone https://github.com/Nuked88/ComfyUI-N-Nodes.git`
to your ComfyUI `custom_nodes` directory to your ComfyUI `custom_nodes` directory
2. **IMPORTANT**: For the GPT node you need to run **install_dependency bat file**. 2. Install it through **ComfyUI Manager** (recommended), which installs the dependencies declared by the project, or run `python -m pip install -r requirements.txt` in ComfyUI's Python environment after a manual clone.
There are 2 versions: ***install_dependency_ggml_models.bat*** for the old ggmlv3 models and ***install_dependency_new_models.bat*** for all the new models (GGUF). 3. Restart ComfyUI.
YOU CAN ONLY USE ONE OF THEM AT A TIME!
Since _llama-cpp-python_ needs to be compiled from source code to enable it to use the GPU, you will first need to have [CUDA](https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64) and visual studio 2019 or 2022 (in the case of my bat) installed to compile it. For details and the full guide you can go [HERE](https://github.com/abetlen/llama-cpp-python) . This bats are made for the official portable windows version of ComfyUI
ComfyUI will then automatically load all custom scripts and nodes at the start. ComfyUI automatically loads all custom scripts and nodes at startup.
> [!IMPORTANT]
> **Breaking change in 1.2.0:** `llama-cpp-python` integration has been removed because its platform-specific installation was the main source of installation and startup failures. GGUF text-generation models and LLaVA nodes are therefore no longer available in N-Suite. Existing workflows using `Llava Clip Loader` must remove that node; `GPT Loader Simple` and `GPT Sampler` now support only Moondream and JoyTag. N-Suite no longer detects, downloads, or installs `llama-cpp-python`.
> [!WARNING]
> **The `Llava Clip Loader` node and the GGUF text-generation path of `GPT Loader Simple` / `GPT Sampler` have been removed.** To keep using those legacy nodes, install the last revision that contains them with `git checkout ae7cc84`. That revision is unsupported and retains the `llama-cpp-python` installation problems; use it in a separate ComfyUI installation or Python environment.
> [!NOTE]
> Since 14/02/2024, the node has undergone a massive rewrite, which also led to the change of all node names in order to avoid any conflicts with other extensions in the future (or at least I hope so). Consequently, the old workflows are no longer compatible and will require manual replacement of each node.
> To avoid this, I have created a tool that allows for automatic replacement.
> On Windows, simply drag any *.json workflow onto the migrate.bat file located in (custom_nodes/ComfyUI-N-Nodes), and another workflow with the suffix _migrated will be created in the same folder as the current workflow.
> On Linux, you can use the script in the following way: python libs/migrate.py path/to/original/workflow/.
> For security reasons, the original workflow will not be deleted."
> For install the last version of this repository before this changes from the Comfyui-N-Suite execute **git checkout 29b2e43baba81ee556b2930b0ca0a9c978c47083**
- For uninstallation: - For uninstallation:
- Delete the `ComfyUI-N-Nodes` folder in `custom_nodes` - Delete the `ComfyUI-N-Nodes` folder in `custom_nodes`
- Delete the `comfyui-n-nodes` folder in `ComfyUI\web\extensions`
- Delete the `n-styles.csv` and `n-styles.csv.backup` file in `ComfyUI\styles`
- Delete the `GPTcheckpoints` folder in `ComfyUI\models`
# Update # Update
@@ -25,6 +50,113 @@ Since _llama-cpp-python_ needs to be compiled from source code to enable it to u
# Features # Features
## 📽️ Video Nodes 📽️
### LoadVideo
![alt text](./img/image-13.png)
The LoadVideoAdvanced node allows loading a video file and extracting frames from it.
The name has been changed from `LoadVideo` to `LoadVideoAdvanced` in order to avoid conflicts with the `LoadVideo` animatediff node.
#### Input Fields
- `video`: Select the video file to load.
- `framerate`: Choose whether to keep the original framerate or reduce to half or quarter speed.
- `resize_by`: Select how to resize frames - 'none', 'height', or 'width'.
- `size`: Target size if resizing by height or width.
- `images_limit`: Limit number of frames to extract.
- `batch_size`: Batch size for encoding frames.
- `starting_frame`: Select which frame to start from.
- `autoplay`: Select whether to autoplay the video.
- `use_ram`: Use RAM instead of disk for decompressing video frames.
#### Output
- `IMAGES`: Extracted frame images as PyTorch tensors.
- `LATENT`: Empty latent vectors.
- `METADATA`: Video metadata - FPS and number of frames.
- `WIDTH:` Frame width.
- `HEIGHT`: Frame height.
- `META_FPS`: Frame rate.
- `META_N_FRAMES`: Number of frames.
The node extracts frames from the input video at the specified framerate. It resizes frames if chosen and returns them as batches of PyTorch image tensors along with latent vectors, metadata, and frame dimensions.
### SaveVideo
The SaveVideo node takes in extracted frames and saves them back as a video file.
![alt text](./img/image-3.png)
#### Input Fields
- `images`: Frame images as tensors.
- `METADATA`: Metadata from LoadVideo node.
- `SaveVideo`: Toggle saving output video file.
- `SaveFrames`: Toggle saving frames to a folder.
- `CompressionLevel`: PNG compression level for saving frames.
#### Output
Saves output video file and/or extracted frames.
The node takes extracted frames and metadata and can save them as a new video file and/or individual frame images. Video compression and frame PNG compression can be configured.
NOTE: If you are using **LoadVideo** as source of the frames, the audio of the original file will be maintained but only in case **images_limit** and **starting_frame** are equal to Zero.
### LoadFramesFromFolder
![alt text](./img/image.png)
The LoadFramesFromFolder node allows loading image frames from a folder and returning them as a batch.
#### Input Fields
- `folder`: Path to the folder containing the frame images.Must be png format, named with a number (eg. 1.png or even 0001.png).The images will be loaded sequentially.
- `fps`: Frames per second to assign to the loaded frames.
#### Output
- `IMAGES`: Batch of loaded frame images as PyTorch tensors.
- `METADATA`: Metadata containing the set FPS value.
- `MAX_WIDTH`: Maximum frame width.
- `MAX_HEIGHT`: Maximum frame height.
- `FRAME COUNT`: Number of frames in the folder.
- `PATH`: Path to the folder containing the frame images.
- `IMAGE LIST`: List of frame images in the folder (not a real list just a string divided by \n).
The node loads all image files from the specified folder, converts them to PyTorch tensors, and returns them as a batched tensor along with simple metadata containing the set FPS value.
This allows easily loading a set of frames that were extracted and saved previously, for example, to reload and process them again. By setting the FPS value, the frames can be properly interpreted as a video sequence.
### SetMetadataForSaveVideo
![alt text](./img/image-1.png)
The SetMetadataForSaveVideo node allows setting metadata for the SaveVideo node.
### FrameInterpolator
![alt text](./img/image-4.png)
The FrameInterpolator node allows interpolating between extracted video frames to increase the frame rate and smooth motion.
#### Input Fields
- `images`: Extracted frame images as tensors.
- `METADATA`: Metadata from video - FPS and number of frames.
- `multiplier`: Factor by which to increase frame rate.
#### Output
- `IMAGES`: Interpolated frames as image tensors.
- `METADATA`: Updated metadata with new frame rate.
The node takes extracted frames and metadata as input. It uses an interpolation model (RIFE) to generate additional in-between frames at a higher frame rate.
The original frame rate in the metadata is multiplied by the `multiplier` value to get the new interpolated frame rate.
The interpolated frames are returned as a batch of image tensors, along with updated metadata containing the new frame rate.
This allows increasing the frame rate of an existing video to achieve smoother motion and slower playback. The interpolation model creates new realistic frames to fill in the gaps rather than just duplicating existing frames.
The original code has been taken from [HERE](https://github.com/hzwer/Practical-RIFE/tree/main)
## Variables ## Variables
Since the primitive node has limitations in links (for example at the time i'm writing you cannot link "start_at_step" and "steps" of another ksampler toghether), I decided to create these simple node-variables to bypass this limitation Since the primitive node has limitations in links (for example at the time i'm writing you cannot link "start_at_step" and "steps" of another ksampler toghether), I decided to create these simple node-variables to bypass this limitation
The node-variables are: The node-variables are:
@@ -33,62 +165,56 @@ The node-variables are:
- String - String
## GPTLoaderSimple and GPTSampler ## 🤖 Image captioning: GPTLoaderSimple and GPTSampler 🤖
These custom nodes are designed to enhance the capabilities of the ConfyUI framework by enabling text generation using GPTQ GPT models. This README provides an overview of the two custom nodes and their usage within ConfyUI. The legacy node identifiers are retained so existing Moondream and JoyTag workflows continue to load, but these nodes are now limited to image captioning:
You can add in the _extra_model_paths.yaml_ the path where your model GPTQ are in this way (example): - **Moondream:** its `config.json`, `model.safetensors`, and `tokenizer.json` files are downloaded automatically from the `vikhyatk/moondream1` Hugging Face repository the first time Moondream is loaded.
- **JoyTag:** its model snapshot is downloaded automatically from the `fancyfeast/joytag` Hugging Face repository the first time JoyTag is loaded.
- **LLaVA:** was never downloaded automatically. Its GGUF model and projector had to be installed manually; support has now been removed together with `llama-cpp-python`.
`other_ui: Downloads happen on first model use, not merely when ComfyUI starts. An internet connection and sufficient disk space are required for that initial load. Models are stored under `ComfyUI/models/GPTcheckpoints/moondream` and `ComfyUI/models/GPTcheckpoints/joytag`.
base_path: I:\\text-generation-webui
GPTcheckpoints: models/`
Otherwise it will create a GPTcheckpoints folder in the model folder of ComfyUI where you can place your .bin models.
### GPTLoaderSimple ### GPTLoaderSimple
The `GPTLoaderSimple` node is responsible for loading GPT model checkpoints and creating an instance of the Llama library for text generation. It provides an interface to configure GPU layers, the number of threads, and maximum context for text generation. `GPTLoaderSimple` loads either Moondream or JoyTag. The `gpu_layers` field is retained for workflow compatibility: set it to `0` for CPU, or to a value greater than zero for GPU. The old `n_threads` and `max_ctx` fields are also retained so saved workflows continue to deserialize, but they do not affect these image-captioning models.
#### Input Fields
- `ckpt_name`: Select the GPT checkpoint name from the available options.
- `gpu_layers`: Specify the number of GPU layers to use (default: 27).
- `n_threads`: Specify the number of threads for text generation (default: 8).
- `max_ctx`: Specify the maximum context length for text generation (default: 2048).
#### Output
The node returns an instance of the Llama library (MODEL) and the path to the loaded checkpoint (STRING).
### GPTSampler ### GPTSampler
The `GPTSampler` node facilitates text generation using GPT models based on the input prompt and various generation parameters. It allows you to control aspects like temperature, top-p sampling, penalties, and more. Connect an image and, for Moondream, a question or instruction in `prompt`. JoyTag uses `max_tags` to limit the number of returned tags. The advanced text-generation controls remain visible for workflow compatibility but no longer apply to GGUF text generation.
## Image Pad For Outpainting Advanced
![alt text](./img/image-14.png)
The `ImagePadForOutpaintingAdvanced` node is an alternative to the `ImagePadForOutpainting` node that applies the technique seen in [this video](https://www.youtube.com/@robadams2451) under the outpainting mask.
The color correction part was taken from [this](https://github.com/sipherxyz/comfyui-art-venture) custom node from Sipherxyz
#### Input Fields #### Input Fields
- `prompt`: Enter the input prompt for text generation. - `image`: Image input.
- `model`: Choose the GPT model to use for text generation. - `left`: pixel to extend from left,
- `model_path`: Specify the path to the GPT model checkpoint. - `top`: pixel to extend from top,
- `max_tokens`: Set the maximum number of tokens in the generated text (default: 128). - `right`: pixel to extend from right,
- `temperature`: Set the temperature parameter for randomness (default: 0.7). - `bottom`: pixel to extend from bottom.
- `top_p`: Set the top-p probability for nucleus sampling (default: 0.5). - `feathering`: feathering strength
- `logprobs`: Specify the number of log probabilities to output (default: 0). - `noise`: blend strenght from noise and the copied border
- `echo`: Enable or disable printing the input prompt alongside the generated text. - `pixel_size`: how big will be the pixel in the pixellated effect
- `stop_token`: Specify the token at which text generation stops. - `pixel_to_copy`: how many pixels to copy (from each side)
- `frequency_penalty`, `presence_penalty`, `repeat_penalty`: Control word generation penalties. - `temperature`: color correction setting that is only applied to the mask part.
- `top_k`: Set the top-k tokens to consider during generation (default: 40). - `hue`: color correction setting that is only applied to the mask part.
- `tfs_z`: Set the temperature scaling factor for top frequent samples (default: 1.0). - `brightness`: color correction setting that is only applied to the mask part.
- `print_output`: Enable or disable printing the generated text to the console. - `contrast`: color correction setting that is only applied to the mask part.
- `cached`: Choose whether to use cached generation (default: NO). - `saturation`: color correction setting that is only applied to the mask part.
- `prefix`, `suffix`: Specify text to prepend and append to the prompt. - `gamma`: color correction setting that is only applied to the mask part.
#### Output #### Output
The node returns the generated text along with a UI-friendly representation. The node returns the processed image and the mask.
## Dynamic Prompt ## Dynamic Prompt
![alt text](./img/image-9.png)
The `DynamicPrompt` node generates prompts by combining a fixed prompt with a random selection of tags from a variable prompt. This enables flexible and dynamic prompt generation for various use cases. The `DynamicPrompt` node generates prompts by combining a fixed prompt with a random selection of tags from a variable prompt. This enables flexible and dynamic prompt generation for various use cases.
@@ -109,6 +235,35 @@ The node returns the generated prompt, which is a combination of the fixed promp
- Just fill the `variable_prompt` field with tag comma separated, the `fixed_prompt` is optional - Just fill the `variable_prompt` field with tag comma separated, the `fixed_prompt` is optional
## CLIP Text Encode Advanced (Experimental)
![alt text](./img/image-10.png)
The `CLIP Text Encode Advanced` node is an alternative to the standard `CLIP Text Encode` node. It offers support for Add/Replace/Delete styles, allowing for the inclusion of both positive and negative prompts within a single node.
The base style file is called `n-styles.csv` and is located in the `ComfyUI\styles` folder.
The styles file follows the same format as the current `styles.csv` file utilized in A1111 (at the time of writing).
NOTE: this note is experimental and still have alot of bugs
#### Input Fields
- `clip`: clip input
- `style`: it will automatically fill the positive and negative prompts based on the choosen style
#### Output
- `positive`: positive conditions
- `negative`: negative conditions
## Troubleshooting
- ~~**SaveVideo - Preview not working**: is related to a conflict with animateDiff, i've already opened a [PR](https://github.com/ArtVentureX/comfyui-animatediff/pull/64) to solve this issue. Meanwhile you can download my patched version from [here](https://github.com/Nuked88/comfyui-animatediff)~~ pull has been merged so this problem should be fixed now!
## Contributing ## Contributing
Feel free to contribute to this project by reporting issues or suggesting improvements. Open an issue or submit a pull request on the GitHub repository. Feel free to contribute to this project by reporting issues or suggesting improvements. Open an issue or submit a pull request on the GitHub repository.
+61 -39
View File
@@ -1,51 +1,73 @@
# code based on pysssss repo
import importlib.util import importlib.util
import glob
import os import os
import sys import sys
from .nnodes import init, get_ext_dir import traceback
from pathlib import Path
NODE_CLASS_MAPPINGS = {} NODE_CLASS_MAPPINGS = {}
NODE_DISPLAY_NAME_MAPPINGS = {} NODE_DISPLAY_NAME_MAPPINGS = {}
WEB_DIRECTORY = "./js"
def install_and_import(package): RIFE_REPOSITORY = "https://github.com/hzwer/Practical-RIFE.git"
import importlib RIFE_REVISION = "a8a8035323b1c1a4a20753c751780e5b0a879455"
try: MOONDREAM_REPOSITORY = "https://github.com/Nuked88/moondream.git"
print("Detected: ", package) MOONDREAM_REVISION = "38af98596e59f2a6c25c6b52b2bd5a672dab4144"
importlib.import_module(package) RIFE_MODEL_REVISION = "572480112b87f9bfbff7579b8a38b483766e455f"
except ImportError:
import pip
pip.main(['install', package])
finally:
globals()[package] = importlib.import_module(package)
def check_module(package):
import importlib
try:
print("Detected: ", package)
importlib.import_module(package)
return True
except ImportError:
return False
def clone_at_revision(repo_class, repository, destination, revision):
"""Clone an external repository once and keep it on a tested revision."""
repo = repo_class.clone_from(repository, destination) if not os.path.exists(destination) else repo_class(destination)
if repo.head.commit.hexsha != revision:
repo.git.checkout(revision)
return repo
if init(): # Pytest imports repository-level __init__.py files while discovering tests. A
py = get_ext_dir("py") # standalone import has no package context and, unlike ComfyUI, cannot resolve
files = glob.glob("*.py", root_dir=py, recursive=False) # the extension's relative imports. Leave the mappings empty in that context.
install_and_import('moviepy') if __package__:
for file in files: from .nnodes import color, downloader, get_commit, get_ext_dir, init
try:
name = os.path.splitext(file)[0] if init():
spec = importlib.util.spec_from_file_location(name, os.path.join(py, file)) print("------------------------------------------")
module = importlib.util.module_from_spec(spec) print(f"{color.BLUE}### N-Suite Revision:{color.END} {color.GREEN}{get_commit()} {color.END}")
sys.modules[name] = module py = Path(get_ext_dir("py"))
spec.loader.exec_module(module) files = list(py.glob("*.py"))
if hasattr(module, "NODE_CLASS_MAPPINGS") and getattr(module, "NODE_CLASS_MAPPINGS") is not None: print(
NODE_CLASS_MAPPINGS.update(module.NODE_CLASS_MAPPINGS) f"{color.YELLOW}N-Suite 1.2 removed the llama.cpp/GGUF and LLaVA nodes. "
if hasattr(module, "NODE_DISPLAY_NAME_MAPPINGS") and getattr(module, "NODE_DISPLAY_NAME_MAPPINGS") is not None: f"Use commit ae7cc84 to keep the legacy nodes.{color.END}"
NODE_DISPLAY_NAME_MAPPINGS.update(module.NODE_DISPLAY_NAME_MAPPINGS) )
except Exception as e:
print(e) from git import Repo
rife_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), "libs", "rifle")
clone_at_revision(Repo, RIFE_REPOSITORY, rife_path, RIFE_REVISION)
moondream_path = os.path.join(
os.path.dirname(os.path.realpath(__file__)), "libs", "moondream_repo"
)
clone_at_revision(Repo, MOONDREAM_REPOSITORY, moondream_path, MOONDREAM_REVISION)
if not os.path.exists(os.path.join(rife_path, "train_log")):
downloader(
f"https://raw.githubusercontent.com/Nuked88/DreamingAI/{RIFE_MODEL_REVISION}/RIFE_trained_model_v4.7.zip"
)
# Code based on pysssss's repository.
for file in files:
try:
name = os.path.splitext(file)[0]
spec = importlib.util.spec_from_file_location(name, os.path.join(py, file))
module = importlib.util.module_from_spec(spec)
sys.modules[name] = module
spec.loader.exec_module(module)
mappings = getattr(module, "NODE_CLASS_MAPPINGS", None)
if mappings is not None:
NODE_CLASS_MAPPINGS.update(mappings)
display_mappings = getattr(module, "NODE_DISPLAY_NAME_MAPPINGS", None)
if display_mappings is not None:
NODE_DISPLAY_NAME_MAPPINGS.update(display_mappings)
except Exception:
traceback.print_exc()
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"] __all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"]
+4
View File
@@ -0,0 +1,4 @@
{
"name": "N-Suite",
"logging": false
}
BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 76 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 26 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 8.7 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 168 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 139 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 123 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 35 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 14 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 20 KiB

BIN
View File
Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

-18
View File
@@ -1,18 +0,0 @@
@echo off
SET CMAKE_ARGS=-DLLAMA_CUBLAS=on
SET FORCE_CMAKE=1
xcopy "%CUDA_PATH%\extras\visual_studio_integration\MSBuildExtensions\" "C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\BuildCustomizations\" /E /I /Y
xcopy "%CUDA_PATH%\extras\visual_studio_integration\MSBuildExtensions\" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\BuildCustomizations\" /E /I /Y
cd /d %~dp0
git pull
cd ../../../python_embeded
python.exe -s -m pip install scikit-build
python.exe -s -m pip install cmake moviepy
python.exe -s -m pip install llama-cpp-python==0.1.78 --force-reinstall --upgrade --no-cache-dir
PAUSE
-18
View File
@@ -1,18 +0,0 @@
@echo off
SET CMAKE_ARGS=-DLLAMA_CUBLAS=on
SET FORCE_CMAKE=1
xcopy "%CUDA_PATH%\extras\visual_studio_integration\MSBuildExtensions\" "C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\BuildCustomizations\" /E /I /Y
xcopy "%CUDA_PATH%\extras\visual_studio_integration\MSBuildExtensions\" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\BuildCustomizations\" /E /I /Y
cd /d %~dp0
git pull
cd ../../../python_embeded
python.exe -s -m pip install scikit-build
python.exe -s -m pip install cmake moviepy
python.exe -s -m pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir
PAUSE
+214
View File
@@ -0,0 +1,214 @@
import { app } from "/scripts/app.js";
import { api } from "/scripts/api.js"
const MultilineSymbol = Symbol();
const MultilineResizeSymbol = Symbol();
function getStyles(name) {
//console.log("getStyles called " + name);
return api.fetchApi('/nsuite/styles')
.then(response => response.json())
.then(data => {
// Eseguire l'elaborazione dei dati
const styles = data.styles;
//console.log('Styles:', styles);
let positive_prompt = "";
let negative_prompt = "";
// Funzione per ottenere positive_prompt e negative_prompt dato il name
for (let i = 0; i < styles[0].length; i++) {
const style = styles[0][i];
if (style.name === name) {
positive_prompt = style.prompt;
negative_prompt = style.negative_prompt;
//console.log('Style:', style.name);
break;
}
}
if (positive_prompt !== "") {
//console.log("Positive prompt:", positive_prompt);
//console.log("Negative prompt:", negative_prompt);
return { positive_prompt: positive_prompt, negative_prompt: negative_prompt };
} else {
return { positive_prompt: "", negative_prompt: "" };
}
})
.catch(error => {
console.error('Error:', error);
throw error; // Rilancia l'errore per consentire al chiamante di gestirlo
});
}
function addStyles(name, positive_prompt, negative_prompt) {
return api.fetchApi('/nsuite/styles/add', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
name: name,
positive_prompt: positive_prompt,
negative_prompt: negative_prompt
}),
})
}
function updateStyles(name, positive_prompt, negative_prompt) {
return api.fetchApi('/nsuite/styles/update', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
name: name,
positive_prompt: positive_prompt,
negative_prompt: negative_prompt
}),
})
}
function removeStyles(name) {
//confirmation
let ok = confirm("Are you sure you want to remove this style?");
if (!ok) {
return;
}
return api.fetchApi('/nsuite/styles/remove', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({
name: name
}),
})
}
app.registerExtension({
name: "n.CLIPTextEncodeAdvancedNSuite",
async beforeRegisterNodeDef(nodeType, nodeData, app) {
const onAdded = nodeType.prototype.onAdded;
if (nodeData.name === "CLIPTextEncodeAdvancedNSuite [n-suite]") {
nodeType.prototype.onAdded = function () {
onAdded?.apply(this, arguments);
const styles = this.widgets.find((w) => w.name === "styles");
const p_prompt = this.widgets.find((w) => w.name === "positive_prompt");
const n_prompt = this.widgets.find((w) => w.name === "negative_prompt");
if (!styles || !p_prompt || !n_prompt) return;
const cb = styles.callback;
let addedd_positive_prompt = "";
let addedd_negative_prompt = "";
styles.callback = function () {
let index = styles.options.values.indexOf(styles.value);
if (addedd_positive_prompt == "" && addedd_negative_prompt == "") {
getStyles(styles.options.values[index-1]).then(style_prompts => {
//wait 4 seconds
console.log(style_prompts);
addedd_positive_prompt = style_prompts.positive_prompt;
addedd_negative_prompt = style_prompts.negative_prompt;
//alert("Addedd positive prompt: " + addedd_positive_prompt + "\nAddedd negative prompt: " + addedd_negative_prompt);
})
}
let current_positive_prompt = p_prompt.value;
let current_negative_prompt = n_prompt.value;
getStyles(styles.value).then(style_prompts => {
//console.log(style_prompts)
if ((current_positive_prompt.trim() != addedd_positive_prompt.trim() || current_negative_prompt.trim() != addedd_negative_prompt.trim())) {
let ok = confirm("Style has been changed. Do you want to change style without saving?");
if (!ok) {
if (styles.value === styles.options.values[0]) {
styles.value = styles.options.values[0];
}
styles.value = styles.options.values[index-1];
return;
}
}
// add the addedd prompt to the current prompt
p_prompt.value = style_prompts.positive_prompt;
n_prompt.value = style_prompts.negative_prompt;
addedd_positive_prompt = style_prompts.positive_prompt;
addedd_negative_prompt = style_prompts.negative_prompt;
if (cb) {
return cb.apply(this, arguments);
}
})
.catch(error => {
console.error('Error:', error);
});
};
let savestyle;
let replacestyle;
let deletestyle;
// Create the button widget for selecting the files
savestyle = this.addWidget("button", "New", "image", () => {
////console.log("Save called");
//ask input name style
let inputName = prompt("Enter a name for the style:", styles.value);
if (inputName === null) {
return;
}
addStyles(inputName, p_prompt.value, n_prompt.value);
// Add the file to the dropdown list and update the widget value
if (!styles.options.values.includes(inputName)) {
styles.options.values.push(inputName);
}
},{
cursor: "grab",
},);
replacestyle = this.addWidget("button", "Replace", "image", () => {
//console.log("Replace called");
updateStyles(styles.value, p_prompt.value, n_prompt.value);
},{
cursor: "grab",
},);
deletestyle = this.addWidget("button", "Delete", "image", () => {
//console.log("Delete called");
removeStyles(styles.value);
// Remove the file from the dropdown list
styles.options.values = styles.options.values.filter((value) => value !== styles.value);
},{
cursor: "grab",
},);
savestyle.serialize = false;
}
};
},
});
+20 -36
View File
@@ -3,42 +3,26 @@ import { ComfyWidgets } from "/scripts/widgets.js";
app.registerExtension({ app.registerExtension({
name: "n.DynamicPrompt", name: "n.DynamicPrompt",
async beforeRegisterNodeDef(nodeType, nodeData, app) { async beforeRegisterNodeDef(nodeType, nodeData) {
if (nodeData.name !== "DynamicPrompt [n-suite]") return;
if (nodeData.name === "DynamicPrompt") { const onExecuted = nodeType.prototype.onExecuted;
console.warn("DynamicPrompt detected") nodeType.prototype.onExecuted = function () {
onExecuted?.apply(this, arguments);
const onExecuted = nodeType.prototype.onExecuted; const widgets = this.widgets ?? [];
const cached = widgets.find((widget) => widget.name === "cached");
for (const widget of widgets.filter((item) => item.name === "text")) {
nodeType.prototype.onExecuted = function (message) { this.removeWidget(widget);
onExecuted?.apply(this, arguments); }
if (cached?.value === "NO") {
const pos_cached = this.widgets.findIndex((w) => w.name === "cached"); const result = ComfyWidgets.STRING(
console.warn("value:"+pos_cached) this,
"text",
if (this.widgets) { ["STRING", { multiline: true }],
const pos_text = this.widgets.findIndex((w) => w.name === "text"); app,
if (pos_text !== -1) { );
for (let i = pos_text; i < this.widgets.length; i++) { result.widget.value = Math.floor(Math.random() * 10000);
this.widgets[i].onRemove?.(); }
} };
this.widgets.length = pos_text;
}
}
if (this.widgets[pos_cached].value === "NO") {
const w = ComfyWidgets["STRING"](this, "text", ["STRING", { multiline: true }], app);
//random seed
var rnm = Math.floor(Math.random() * 10000)
w.widget.value = rnm;
}
};
}
}, },
}); });
+63 -412
View File
@@ -1,430 +1,81 @@
//extended_widgets.js import { app } from "/scripts/app.js";
import { api } from "/scripts/api.js" import { api } from "/scripts/api.js";
import { ComfyWidgets } from "/scripts/widgets.js"; import { ComfyWidgets } from "/scripts/widgets.js";
const MultilineSymbol = Symbol(); function buildViewUrl(name, type, defaultSubfolder) {
const MultilineResizeSymbol = Symbol(); const separator = name.lastIndexOf("/");
async function uploadFile(file, updateNode, node, pasted = false) { const subfolder = separator >= 0 ? name.slice(0, separator) : defaultSubfolder;
const videoWidget = node.widgets.find((w) => w.name === "video"); const filename = separator >= 0 ? name.slice(separator + 1) : name;
const params = new URLSearchParams({ filename, type, subfolder });
return api.apiURL(`/view?${params.toString()}`);
try {
// Wrap file in formdata so it includes filename
const body = new FormData();
body.append("image", file);
if (pasted) {
body.append("subfolder", "pasted");
}
else {
body.append("subfolder", "n-suite");
}
const resp = await api.fetchApi("/upload/image", {
method: "POST",
body,
});
if (resp.status === 200) {
const data = await resp.json();
// Add the file to the dropdown list and update the widget value
let path = data.name;
if (!videoWidget.options.values.includes(path)) {
videoWidget.options.values.push(path);
}
if (updateNode) {
videoWidget.value = path;
if (data.subfolder) path = data.subfolder + "/" + path;
showVideoInput(path,node);
}
} else {
alert(resp.status + " - " + resp.statusText);
}
} catch (error) {
alert(error);
}
} }
function addVideo(node, name,src, app) { function updateVideoWidget(node, widgetName, url) {
console.log(src) const widget = node.widgets?.find((item) => item.name === widgetName);
if (!widget?.element) return;
widget.element.src = url;
}
const MIN_SIZE = 50; function addVideo(node, name, src, autoplayValue) {
function computeSize(size) { const video = document.createElement("video");
try{ video.controls = true;
video.loop = true;
if (node.widgets[0].last_y == null) return; video.muted = true;
video.autoplay = autoplayValue;
let y = node.widgets[0].last_y; video.playsInline = true;
let freeSpace = size[1] - y; video.src = src || "";
video.style.width = "100%";
// Compute the height of all non customvideo widgets video.style.height = "100%";
let widgetHeight = 0; video.style.objectFit = "contain";
const multi = [];
for (let i = 0; i < node.widgets.length; i++) {
const w = node.widgets[i];
if (w.type === "customvideo") {
multi.push(w);
} else {
if (w.computeSize) {
widgetHeight += w.computeSize()[1] + 4;
} else {
widgetHeight += LiteGraph.NODE_WIDGET_HEIGHT + 4;
}
}
}
// See how large each text input can be
freeSpace -= widgetHeight;
freeSpace /= multi.length + (!!node.imgs?.length);
if (freeSpace < MIN_SIZE) {
// There isnt enough space for all the widgets, increase the size of the node
freeSpace = MIN_SIZE;
node.size[1] = y + widgetHeight + freeSpace * (multi.length + (!!node.imgs?.length));
node.graph.setDirtyCanvas(true);
}
// Position each of the widgets
for (const w of node.widgets) {
w.y = y;
if (w.type === "customvideo") {
y += freeSpace;
w.computedHeight = freeSpace - multi.length*4;
} else if (w.computeSize) {
y += w.computeSize()[1] + 4;
} else {
y += LiteGraph.NODE_WIDGET_HEIGHT + 4;
}
}
node.inputHeight = freeSpace;
}catch(e){
}
}
const widget = {
type: "customvideo",
name,
get value() {
return this.inputEl.value;
},
set value(x) {
this.inputEl.value = x;
},
draw: function (ctx, _, widgetWidth, y, widgetHeight) {
if (!this.parent.inputHeight) {
// If we are initially offscreen when created we wont have received a resize event
// Calculate it here instead
node.setSizeForImage?.();
}
const visible = app.canvas.ds.scale > 0.5 && this.type === "customvideo";
const margin = 10;
const elRect = ctx.canvas.getBoundingClientRect();
const transform = new DOMMatrix()
.scaleSelf(elRect.width / ctx.canvas.width, elRect.height / ctx.canvas.height)
.multiplySelf(ctx.getTransform())
.translateSelf(margin, margin + y);
const scale = new DOMMatrix().scaleSelf(transform.a, transform.d)
Object.assign(this.inputEl.style, {
transformOrigin: "0 0",
transform: scale,
left: `${transform.a + transform.e}px`,
top: `${transform.d + transform.f}px`,
width: `${widgetWidth - (margin * 2)}px`,
height: `${this.parent.inputHeight - (margin * 2)}px`,
position: "absolute",
background: (!node.color)?'':node.color,
color: (!node.color)?'':'white',
zIndex: app.graph._nodes.indexOf(node),
});
this.inputEl.hidden = !visible;
},
};
let type_file="mp4";
const regex = /\.gif&type/;
if (regex.test(src)) {
type_file="gif";
}
widget.inputEl = document.createElement("div");
Object.assign(widget.inputEl, {
id: "videoContainer",
width: 400,
height: 300
})
if (type_file=="gif"){
let img_element = document.createElement("img");
Object.assign(img_element, {
id:"mediaContainer",
src: src,
style: "width: 100%; height: 100%;",
type : "image/gif"
})
widget.inputEl.appendChild(img_element);
}
else{
let video_element = document.createElement("video");
// Set the video attributes
Object.assign(video_element, {
id:"mediaContainer",
controls: true,
src: src,
poster: "",
style: "width: 100%; height: 100%;",
loop: true,
muted: true,
autoplay:true,
type : "video/mp4"
});
widget.inputEl.appendChild(video_element);
}
// Add video element to the body
document.body.appendChild(widget.inputEl);
widget.parent = node;
//document.body.appendChild(widget.inputEl);
node.addCustomWidget(widget);
app.canvas.onDrawBackground = function () {
// Draw node isnt fired once the node is off the screen
// if it goes off screen quickly, the input may not be removed
// this shifts it off screen so it can be moved back if the node is visible.
for (let n in app.graph._nodes) {
n = graph._nodes[n];
for (let w in n.widgets) {
let wid = n.widgets[w];
if (Object.hasOwn(wid, "inputEl")) {
wid.inputEl.style.left = -8000 + "px";
wid.inputEl.style.position = "absolute";
}
}
}
};
node.onRemoved = function () {
// When removing this node we need to remove the input from the DOM
for (let y in this.widgets) {
if (this.widgets[y].inputEl) {
this.widgets[y].inputEl.remove();
}
}
};
widget.onRemove = () => {
widget.inputEl?.remove();
// Restore original size handler if we are the last
if (!--node[MultilineSymbol]) {
node.onResize = node[MultilineResizeSymbol];
delete node[MultilineSymbol];
delete node[MultilineResizeSymbol];
}
};
if (node[MultilineSymbol]) {
node[MultilineSymbol]++;
} else {
node[MultilineSymbol] = 1;
const onResize = (node[MultilineResizeSymbol] = node.onResize);
node.onResize = function (size) {
computeSize(size);
// Call original resizer handler
if (onResize) {
onResize.apply(this, arguments);
}
};
}
const widget = node.addDOMWidget(name, "video", video, {
hideOnZoom: false,
getMinHeight: () => 200,
getHeight: () => 240,
});
widget.serialize = false;
widget.options.serialize = false;
return { minWidth: 400, minHeight: 200, widget }; return { minWidth: 400, minHeight: 200, widget };
} }
export function showVideoInput(name, node) {
export function showVideoInput(name,node) { const url = buildViewUrl(name, "input", "n-suite");
const videoWidget = node.widgets.find((w) => w.name === "videoWidget"); updateVideoWidget(node, "videoWidget", url);
const temp_web_url = node.widgets.find((w) => w.name === "local_url"); const localUrl = node.widgets?.find((item) => item.name === "local_url");
///const videoContainer = videoWidget.inputEl.widgets.find((w) => w.name === "videoWidget"); if (localUrl) localUrl.value = url;
return url;
let folder_separator = name.lastIndexOf("/");
let subfolder = "n-suite";
if (folder_separator > -1) {
subfolder = name.substring(0, folder_separator);
name = name.substring(folder_separator + 1);
}
let url_video = api.apiURL(`/view?filename=${encodeURIComponent(name)}&type=input&subfolder=${subfolder}${app.getPreviewFormatParam()}`);
const regex = /\.gif&type/;
let prev_format = "mp4"
if (document.getElementById("mediaContainer").tagName=="IMG"){
prev_format="gif";
}
let current_format = "mp4"
if (regex.test(url_video)){
current_format="gif";
}
if (prev_format == current_format) {
//update
console.log(videoWidget.inputEl)//..getElementById("mediaContainer")
//videoWidget.inputEl.children[1].src = url_video
}
else{
let newElement;
if (current_format=="gif"){
newElement = document.createElement("img");
Object.assign(newElement, {
id:"mediaContainer",
src: url_video,
style: "width: 100%; height: 100%;",
type : "image/gif"
})
}
else{
newElement = document.createElement("video");
// Set the video attributes
Object.assign(newElement, {
id:"mediaContainer",
controls: true,
src: url_video,
poster: "",
style: "width: 100%; height: 100%;",
loop: true,
muted: true,
autoplay:true,
type : "video/mp4"
});
}
let newEl= document.createElement("div");
Object.assign(newEl, {
id: "videoContainer",
width: 400,
height: 300
})
newEl.appendChild(newElement);
videoWidget.lastChilds = newEl;
console.log(videoWidget)
console.log("ssssssssss")
//document.getElementById("videoContainer")
//videoWidget.inputEl.children[0].replaceChild(newElement,videoWidget.inputEl.children[1]);
}
temp_web_url.value = url_video
} }
export function showVideoOutput(name,node) { export function showVideoOutput(name, node) {
const videoWidget = node.widgets.find((w) => w.name === "videoOutWidget"); const url = buildViewUrl(name, "output", "n-suite/videos");
console.log(name) updateVideoWidget(node, "videoOutWidget", url);
return url;
let folder_separator = name.lastIndexOf("/");
let subfolder = "videos";
if (folder_separator > -1) {
subfolder = name.substring(0, folder_separator);
name = name.substring(folder_separator + 1);
}
let url_video = api.apiURL(`/view?filename=${encodeURIComponent(name)}&type=output&subfolder=${subfolder}${app.getPreviewFormatParam()}`);
videoWidget.inputEl.src = url_video
return url_video;
} }
export const ExtendedComfyWidgets = { export const ExtendedComfyWidgets = {
...ComfyWidgets, // Copy all the functions from ComfyWidgets ...ComfyWidgets,
VIDEO(node, inputName, _inputData, src, _app, type = "input", autoplayValue = true) {
const result = addVideo(node, inputName, src, autoplayValue);
if (type !== "input") return result;
VIDEO(node, inputName, inputData, src, app,type="input") { const video = node.widgets?.find((item) => item.name === "video");
try { const autoplay = node.widgets?.find((item) => item.name === "autoplay");
if (video) {
const videoWidget = node.widgets.find((w) => w.name === "video"); const callback = video.callback;
const defaultVal = ""; video.callback = function () {
let res; showVideoInput(video.value, node);
res = addVideo(node, inputName, src, app); return callback?.apply(this, arguments);
if (type == "input"){
const cb = node.callback;
videoWidget.callback = function () {
showVideoInput(videoWidget.value, node);
if (cb) {
return cb.apply(this, arguments);
}
}; };
} }
if (autoplay) {
const callback = autoplay.callback;
if (node.type =="VideoLoader"){ autoplay.callback = function () {
// do this only on VideoLoad node! const preview = node.widgets?.find((item) => item.name === "videoWidget");
let uploadWidget; if (preview?.element) preview.element.autoplay = autoplay.value;
const fileInput = document.createElement("input"); if (video?.value) showVideoInput(video.value, node);
Object.assign(fileInput, { return callback?.apply(this, arguments);
type: "file", };
accept: "video/mp4,image/gif", }
style: "display: none", return result;
onchange: async () => { },
if (fileInput.files.length) {
await uploadFile(fileInput.files[0], true,node);
}
},
});
document.body.append(fileInput);
// Create the button widget for selecting the files
uploadWidget = node.addWidget("button", "choose file to upload", "image", () => {
fileInput.click();
});
uploadWidget.serialize = false;
}
return res;
}
catch (error) {
console.error("Errore in extended_widgets.js:", error);
throw error;
}
},
}; };
+20 -35
View File
@@ -3,41 +3,26 @@ import { ComfyWidgets } from "/scripts/widgets.js";
app.registerExtension({ app.registerExtension({
name: "n.GPTSampler", name: "n.GPTSampler",
async beforeRegisterNodeDef(nodeType, nodeData, app) { async beforeRegisterNodeDef(nodeType, nodeData) {
if (nodeData.name !== "GPT Sampler [n-suite]") return;
if (nodeData.name === "GPTSampler") { const onExecuted = nodeType.prototype.onExecuted;
console.warn("GPTSampler detected") nodeType.prototype.onExecuted = function () {
onExecuted?.apply(this, arguments);
const onExecuted = nodeType.prototype.onExecuted; const widgets = this.widgets ?? [];
const cached = widgets.find((widget) => widget.name === "cached");
nodeType.prototype.onExecuted = function (message) { for (const widget of widgets.filter((item) => item.name === "text")) {
onExecuted?.apply(this, arguments); this.removeWidget(widget);
}
const pos_cached = this.widgets.findIndex((w) => w.name === "cached"); if (cached?.value === "NO") {
console.warn("value:"+pos_cached) const result = ComfyWidgets.STRING(
this,
if (this.widgets) { "text",
const pos_text = this.widgets.findIndex((w) => w.name === "text"); ["STRING", { multiline: true }],
if (pos_text !== -1) { app,
for (let i = pos_text; i < this.widgets.length; i++) { );
this.widgets[i].onRemove?.(); result.widget.value = Math.floor(Math.random() * 10000);
} }
this.widgets.length = pos_text; };
}
}
if (this.widgets[pos_cached].value === "NO") {
const w = ComfyWidgets["STRING"](this, "text", ["STRING", { multiline: true }], app);
//random seed
var rnm = Math.floor(Math.random() * 10000)
w.widget.value = rnm;
}
};
}
}, },
}); });
+18
View File
@@ -0,0 +1,18 @@
function addStylesheet(url) {
if (url.endsWith(".js")) {
url = url.substr(0, url.length - 2) + "css";
}
const link = document.createElement("link");
link.rel = "stylesheet";
link.href = url.startsWith("http") ? url : getUrl(url);
document.head.append(link);
}
function getUrl(path, baseUrl) {
if (baseUrl) {
return new URL(path, baseUrl).toString();
} else {
return new URL("../" + path, import.meta.url).toString();
}
}
addStylesheet(getUrl("styles.css", import.meta.url));
+20
View File
@@ -0,0 +1,20 @@
textarea[placeholder="positive_prompt"] {
border: 1px solid #64d509;
}
textarea[placeholder="positive_prompt"]:focus-visible {
border: 1px solid #72eb0f;
}
textarea[placeholder="negative_prompt"] {
border: 1px solid #a94442;
border-color: #a94442;
}
textarea[placeholder="negative_prompt"]:focus-visible {
border: 1px solid #de5755;
border-color: #de5755;
}
-122
View File
@@ -1,122 +0,0 @@
import { app } from "/scripts/app.js";
import { api } from "/scripts/api.js"
import { ExtendedComfyWidgets,showVideoInput } from "./extended_widgets.js";
const MultilineSymbol = Symbol();
const MultilineResizeSymbol = Symbol();
async function uploadFile(file, updateNode, node, pasted = false) {
const videoWidget = node.widgets.find((w) => w.name === "video");
try {
// Wrap file in formdata so it includes filename
const body = new FormData();
body.append("image", file);
if (pasted) {
body.append("subfolder", "pasted");
}
else {
body.append("subfolder", "n-suite");
}
const resp = await api.fetchApi("/upload/image", {
method: "POST",
body,
});
if (resp.status === 200) {
const data = await resp.json();
// Add the file to the dropdown list and update the widget value
let path = data.name;
if (!videoWidget.options.values.includes(path)) {
videoWidget.options.values.push(path);
}
if (updateNode) {
videoWidget.value = path;
if (data.subfolder) path = data.subfolder + "/" + path;
showVideoInput(path,node);
}
} else {
alert(resp.status + " - " + resp.statusText);
}
} catch (error) {
alert(error);
}
}
let uploadWidget = "";
app.registerExtension({
name: "Comfy.VideoLoad",
async beforeRegisterNodeDef(nodeType, nodeData, app) {
const onAdded = nodeType.prototype.onAdded;
if (nodeData.name === "VideoLoader") {
nodeType.prototype.onAdded = function () {
onAdded?.apply(this, arguments);
const temp_web_url = this.widgets.find((w) => w.name === "local_url");
setTimeout(() => {
ExtendedComfyWidgets["VIDEO"](this, "videoWidget", ["STRING"], temp_web_url.value, app);
}, 100);
}
nodeType.prototype.onDragOver = function (e) {
if (e.dataTransfer && e.dataTransfer.items) {
const image = [...e.dataTransfer.items].find((f) => f.kind === "file");
return !!image;
}
return false;
};
// On drop upload files
nodeType.prototype.onDragDrop = function (e) {
console.log("onDragDrop called");
let handled = false;
for (const file of e.dataTransfer.files) {
if (file.type.startsWith("video/mp4") || file.type.startsWith("image/gif")) {
const filePath = file.path || (file.webkitRelativePath || '').split('/').slice(1).join('/');
uploadFile(file, !handled,this ); // Dont await these, any order is fine, only update on first one
handled = true;
}
}
return handled;
};
nodeType.prototype.pasteFile = function(file) {
if (file.type.startsWith("image/")) {
const is_pasted = (file.name === "image.png") &&
(file.lastModified - Date.now() < 2000);
//uploadFile(file, true, is_pasted);
return true;
}
return false;
}
};
},
});
+96
View File
@@ -0,0 +1,96 @@
import { app } from "/scripts/app.js";
import { api } from "/scripts/api.js";
import { ExtendedComfyWidgets, showVideoInput } from "./extended_widgets.js";
const VIDEO_TYPES = new Set(["video/mp4", "video/webm", "image/gif"]);
async function uploadFile(file, updateNode, node, pasted = false) {
const videoWidget = node.widgets?.find((widget) => widget.name === "video");
if (!videoWidget) return false;
try {
const body = new FormData();
body.append("image", file);
body.append("subfolder", pasted ? "pasted" : "n-suite");
const response = await api.fetchApi("/upload/image", { method: "POST", body });
if (!response.ok) {
alert(`${response.status} - ${response.statusText}`);
return false;
}
const data = await response.json();
const value = data.name;
const previewPath = data.subfolder ? `${data.subfolder}/${value}` : value;
if (!videoWidget.options.values.includes(value)) videoWidget.options.values.push(value);
if (updateNode) {
const oldValue = videoWidget.value;
videoWidget.value = value;
videoWidget.callback?.(value);
node.onWidgetChanged?.(videoWidget.name, value, oldValue, videoWidget);
showVideoInput(previewPath, node);
}
return true;
} catch (error) {
console.error("N-Suite video upload failed", error);
alert(String(error));
return false;
}
}
app.registerExtension({
name: "Comfy.VideoLoadAdvanced",
async beforeRegisterNodeDef(nodeType, nodeData) {
if (nodeData.name !== "LoadVideo [n-suite]") return;
const onAdded = nodeType.prototype.onAdded;
const onRemoved = nodeType.prototype.onRemoved;
nodeType.prototype.onAdded = function () {
onAdded?.apply(this, arguments);
const localUrl = this.widgets?.find((widget) => widget.name === "local_url");
const autoplay = this.widgets?.find((widget) => widget.name === "autoplay");
const fileInput = document.createElement("input");
fileInput.type = "file";
fileInput.accept = "video/mp4,video/webm,image/gif";
fileInput.hidden = true;
fileInput.onchange = async () => {
if (fileInput.files?.length) await uploadFile(fileInput.files[0], true, this);
};
document.body.append(fileInput);
this.__nSuiteVideoFileInput = fileInput;
const uploadWidget = this.addWidget("button", "choose file to upload", "image", () => fileInput.click());
uploadWidget.serialize = false;
ExtendedComfyWidgets.VIDEO(
this,
"videoWidget",
["STRING"],
localUrl?.value ?? "",
app,
"input",
autoplay?.value ?? true,
);
};
nodeType.prototype.onRemoved = function () {
this.__nSuiteVideoFileInput?.remove();
delete this.__nSuiteVideoFileInput;
onRemoved?.apply(this, arguments);
};
nodeType.prototype.onDragOver = function (event) {
return [...(event.dataTransfer?.items ?? [])].some((item) => item.kind === "file");
};
nodeType.prototype.onDragDrop = function (event) {
let handled = false;
for (const file of event.dataTransfer?.files ?? []) {
if (!VIDEO_TYPES.has(file.type)) continue;
uploadFile(file, !handled, this);
handled = true;
}
return handled;
};
nodeType.prototype.pasteFile = function (file) {
if (!VIDEO_TYPES.has(file.type)) return false;
uploadFile(file, true, this, true);
return true;
};
},
});
+9 -74
View File
@@ -1,87 +1,22 @@
import { app } from "/scripts/app.js"; import { app } from "/scripts/app.js";
import { api } from "/scripts/api.js" import { ExtendedComfyWidgets, showVideoOutput } from "./extended_widgets.js";
import { ExtendedComfyWidgets,showVideoOutput } from "./extended_widgets.js";
const MultilineSymbol = Symbol();
const MultilineResizeSymbol = Symbol();
async function uploadFile(file, updateNode, node, pasted = false) {
const videoWidget = node.widgets.find((w) => w.name === "video");
try {
// Wrap file in formdata so it includes filename
const body = new FormData();
body.append("image", file);
if (pasted) {
body.append("subfolder", "pasted");
}
else {
body.append("subfolder", "n-suite");
}
const resp = await api.fetchApi("/upload/image", {
method: "POST",
body,
});
if (resp.status === 200) {
const data = await resp.json();
// Add the file to the dropdown list and update the widget value
let path = data.name;
if (!videoWidget.options.values.includes(path)) {
videoWidget.options.values.push(path);
}
if (updateNode) {
// showVideo(path,node);
videoWidget.value = path;
if (data.subfolder) path = data.subfolder + "/" + path;
showVideo(path,node);
}
} else {
alert(resp.status + " - " + resp.statusText);
}
} catch (error) {
alert(error);
}
}
let uploadWidget = "";
app.registerExtension({ app.registerExtension({
name: "Comfy.VideoSave", name: "Comfy.VideoSave",
async beforeRegisterNodeDef(nodeType, nodeData, app) { async beforeRegisterNodeDef(nodeType, nodeData) {
if (nodeData.name !== "SaveVideo [n-suite]") return;
const onExecuted = nodeType.prototype.onExecuted;
const onAdded = nodeType.prototype.onAdded; const onAdded = nodeType.prototype.onAdded;
if (nodeData.name === "VideoSaver") { const onExecuted = nodeType.prototype.onExecuted;
nodeType.prototype.onAdded = function () { nodeType.prototype.onAdded = function () {
onAdded?.apply(this, arguments);
ExtendedComfyWidgets["VIDEO"](this, "videoOutWidget", ["STRING"], "", app,"output"); ExtendedComfyWidgets.VIDEO(this, "videoOutWidget", ["STRING"], "", app, "output");
}; };
nodeType.prototype.onExecuted = function (message) { nodeType.prototype.onExecuted = function (message) {
onExecuted?.apply(this, arguments); onExecuted?.apply(this, arguments);
console.log(nodeData) const paths = message?.text?.flat?.(Infinity) ?? message?.text ?? [];
const fullPath = Array.isArray(paths) ? paths.at(-1) : paths;
let full_path=""; if (fullPath) showVideoOutput(fullPath, this);
for (const list of message.text) {
full_path = list;
}
let fullweb= showVideoOutput(full_path,this)
}
}; };
}, },
}); });
File diff suppressed because it is too large Load Diff
+43
View File
@@ -0,0 +1,43 @@
import sys
import os
def migrate_workflow(input_file_path):
try:
file_name, file_extension = os.path.splitext(input_file_path)
output_file_path = f"{file_name}_migrated.json"
pre_list = ('LoadVideo', 'SaveVideo','FrameInterpolator', 'LoadFramesFromFolder','SetMetadataForSaveVideo','GPT Loader Simple','GPTSampler','String Variable','Integer Variable','Float Variable','DynamicPrompt')
post_list= ('LoadVideo [n-suite]', 'SaveVideo [n-suite]','FrameInterpolator [n-suite]', 'LoadFramesFromFolder [n-suite]','SetMetadataForSaveVideo [n-suite]','GPT Loader Simple [n-suite]','GPT Sampler [n-suite]','String Variable [n-suite]','Integer Variable [n-suite]','Float Variable [n-suite]','DynamicPrompt [n-suite]')
replacements = list(zip(pre_list, post_list))
with open(input_file_path, 'r') as input_file:
content = input_file.read()
# s&r
for old, new in replacements:
content = content.replace(f'"Node name for S&R": "{old}"', f'"Node name for S&R": "{new}"')
#type
for old, new in replacements:
content = content.replace(f'"type": "{old}"', f'"type": "{new}"')
with open(output_file_path, 'w') as output_file:
output_file.write(content)
print("Replacement completed successfully.")
except Exception as e:
print(f"An error occurred: {str(e)}")
if __name__ == "__main__":
print(len(sys.argv))
if len(sys.argv) != 2:
print("Error: Provide the path of the text file to migrate.")
sys.exit(1)
file_path = sys.argv[1]
if not os.path.isfile(file_path):
print(f"Error: The file {file_path} does not exist.")
sys.exit(1)
migrate_workflow(file_path)
+21
View File
@@ -0,0 +1,21 @@
@echo off
setlocal
rem Check if the file path is provided
if "%1"=="" (
echo Error: provide the path of the text file to migrate.
exit /b 1
)
rem Check if the file exists
if not exist "%1" (
echo Error: the file %1 does not exist.
exit /b 1
)
rem Run the Python script to migrate the file
python %~dp0libs\migrate.py "%~f1"
echo Replacement completed successfully.
pause
-4
View File
@@ -1,4 +0,0 @@
{
"name": "CustomScripts",
"logging": false
}
+87 -63
View File
@@ -1,14 +1,96 @@
import asyncio import asyncio
import os import os
import json import json
import shutil
import inspect import inspect
import aiohttp import aiohttp
from server import PromptServer from server import PromptServer
from tqdm import tqdm from tqdm import tqdm
import requests
import folder_paths
config = None config = None
class color:
END = '\33[0m'
BOLD = '\33[1m'
ITALIC = '\33[3m'
UNDERLINE = '\33[4m'
BLINK = '\33[5m'
BLINK2 = '\33[6m'
SELECTED = '\33[7m'
BLACK = '\33[30m'
RED = '\33[31m'
GREEN = '\33[32m'
YELLOW = '\33[33m'
BLUE = '\33[34m'
VIOLET = '\33[35m'
BEIGE = '\33[36m'
WHITE = '\33[37m'
BLACKBG = '\33[40m'
REDBG = '\33[41m'
GREENBG = '\33[42m'
YELLOWBG = '\33[43m'
BLUEBG = '\33[44m'
VIOLETBG = '\33[45m'
BEIGEBG = '\33[46m'
WHITEBG = '\33[47m'
GREY = '\33[90m'
LIGHTRED = '\33[91m'
LIGHTGREEN = '\33[92m'
LIGHTYELLOW = '\33[93m'
LIGHTBLUE = '\33[94m'
LIGHTVIOLET = '\33[95m'
LIGHTBEIGE = '\33[96m'
LIGHTWHITE = '\33[97m'
GREYBG = '\33[100m'
LIGHTREDBG = '\33[101m'
LIGHTGREENBG = '\33[102m'
LIGHTYELLOWBG = '\33[103m'
LIGHTBLUEBG = '\33[104m'
LIGHTVIOLETBG = '\33[105m'
LIGHTBEIGEBG = '\33[106m'
LIGHTWHITEBG = '\33[107m'
def get_commit():
try:
import git
repo = git.Repo( os.path.join(folder_paths.folder_names_and_paths["custom_nodes"][0][0],"ComfyUI-N-Nodes"))
return repo.head.object.hexsha[:8]
except:
return 0
import zipfile
def downloader(link):
print("Downloading dependencies...")
response = requests.get(link, stream=True)
try:
os.makedirs(folder_paths.get_temp_directory())
except:
pass
temp_file = os.path.join(folder_paths.get_temp_directory(), "file.zip")
with open(temp_file, "wb") as f:
for chunk in response.iter_content(chunk_size=1024):
if chunk:
f.write(chunk)
zip_file = zipfile.ZipFile(temp_file)
target_dir = os.path.join(folder_paths.folder_names_and_paths["custom_nodes"][0][0],"ComfyUI-N-Nodes","libs","rifle") # Cartella dove estrarre lo zip
zip_file.extractall(target_dir)
def is_logging_enabled(): def is_logging_enabled():
config = get_extension_config() config = get_extension_config()
if "logging" not in config: if "logging" not in config:
@@ -26,7 +108,7 @@ def log(message, type=None, always=False, name=None):
if name is None: if name is None:
name = get_extension_config()["name"] name = get_extension_config()["name"]
print(f"(nnodes:{name}) {message}") print(f"{name}: {message}")
def get_ext_dir(subpath=None, mkdir=False): def get_ext_dir(subpath=None, mkdir=False):
@@ -53,24 +135,15 @@ def get_comfy_dir(subpath=None, mkdir=False):
return dir return dir
def get_web_ext_dir():
config = get_extension_config()
name = config["name"]
dir = get_comfy_dir("web/extensions/comfyui-n-nodes")
if not os.path.exists(dir):
os.makedirs(dir)
dir = os.path.join(dir, name)
return dir
def get_extension_config(reload=False): def get_extension_config(reload=False):
global config global config
if reload == False and config is not None: if reload == False and config is not None:
return config return config
config_path = get_ext_dir("nnodes.json") config_path = get_ext_dir("config.json")
if not os.path.exists(config_path): if not os.path.exists(config_path):
log("Missing nnodes.json, this extension may not work correctly. Please reinstall the extension.", log("Missing config.json, this extension may not work correctly. Please reinstall the extension.", type="ERROR", always=True, name="???")
type="ERROR", always=True, name="???")
print(f"Extension path: {get_ext_dir()}") print(f"Extension path: {get_ext_dir()}")
return {"name": "Unknown", "version": -1} return {"name": "Unknown", "version": -1}
with open(config_path, "r") as f: with open(config_path, "r") as f:
@@ -78,53 +151,6 @@ def get_extension_config(reload=False):
return config return config
def link_js(src, dst):
src = os.path.abspath(src)
dst = os.path.abspath(dst)
if os.name == "nt":
try:
import _winapi
_winapi.CreateJunction(src, dst)
return True
except:
pass
try:
os.symlink(src, dst)
return True
except:
import logging
logging.exception('')
return False
def is_junction(path):
if os.name != "nt":
return False
try:
return bool(os.readlink(path))
except OSError:
return False
def install_js():
src_dir = get_ext_dir("js")
if not os.path.exists(src_dir):
log("No JS")
return
dst_dir = get_web_ext_dir()
if os.path.exists(dst_dir):
if os.path.islink(dst_dir) or is_junction(dst_dir):
log("JS already linked")
return
elif link_js(src_dir, dst_dir):
log("JS linked")
return
log("Copying JS files")
shutil.copytree(src_dir, dst_dir, dirs_exist_ok=True)
def init(check_imports=None): def init(check_imports=None):
log("Init") log("Init")
@@ -137,7 +163,7 @@ def init(check_imports=None):
type="ERROR", always=True) type="ERROR", always=True)
return False return False
install_js()
return True return True
@@ -190,8 +216,6 @@ async def download_to_file(url, destination, update_callback=None, is_ext_subpat
download(url, f, update_callback, session) download(url, f, update_callback, session)
def is_inside_dir(root_dir, check_path): def is_inside_dir(root_dir, check_path):
root_dir = os.path.abspath(root_dir) root_dir = os.path.abspath(root_dir)
if not os.path.isabs(check_path): if not os.path.isabs(check_path):
+390
View File
@@ -0,0 +1,390 @@
import torch
import numpy as np
from PIL import Image, ImageOps, ImageEnhance
import cv2
MAX_RESOLUTION = 4096
def tensor2pil(image):
return Image.fromarray(np.clip(255. * image.cpu().numpy().squeeze(), 0, 255).astype(np.uint8))
# Adapt from https://github.com/sipherxyz/comfyui-art-venture
def color_correct(
image,
temperature: float,
hue: float,
brightness: float,
contrast: float,
saturation: float,
gamma: float,
):
brightness /= 100
contrast /= 100
saturation /= 100
temperature /= 100
brightness = 1 + brightness
contrast = 1 + contrast
saturation = 1 + saturation
modified_image = image
# brightness
modified_image = ImageEnhance.Brightness(modified_image).enhance(brightness)
# contrast
modified_image = ImageEnhance.Contrast(modified_image).enhance(contrast)
modified_image = np.array(modified_image).astype(np.float32)
# temperature
if temperature > 0:
modified_image[:, :, 0] *= 1 + temperature
modified_image[:, :, 1] *= 1 + temperature * 0.4
elif temperature < 0:
modified_image[:, :, 2] *= 1 - temperature
modified_image = np.clip(modified_image, 0, 255) / 255
# gamma
modified_image = np.clip(np.power(modified_image, gamma), 0, 1)
# saturation
hls_img = cv2.cvtColor(modified_image, cv2.COLOR_RGB2HLS)
hls_img[:, :, 2] = np.clip(saturation * hls_img[:, :, 2], 0, 1)
modified_image = cv2.cvtColor(hls_img, cv2.COLOR_HLS2RGB) * 255
# hue
hsv_img = cv2.cvtColor(modified_image, cv2.COLOR_RGB2HSV)
hsv_img[:, :, 0] = (hsv_img[:, :, 0] + hue) % 360
modified_image = cv2.cvtColor(hsv_img, cv2.COLOR_HSV2RGB)
modified_image = modified_image.astype(np.uint8)
#modified_image = modified_image / 255
#modified_image = torch.from_numpy(modified_image).unsqueeze(0)
return modified_image
def extract_pixels(image, side, num_pixels):
# Ottieni le dimensioni dell'immagine
width, height = image.size
# Determina la regione di ritaglio in base al lato specificato
if side == "l":
crop_box = (0, 0, num_pixels, height)
elif side == "r":
crop_box = (width - num_pixels, 0, width, height)
elif side == "t":
crop_box = (0, 0, width, num_pixels)
elif side == "b":
crop_box = (0, height - num_pixels, width, height)
else:
raise ValueError("Il lato specificato non è valido. Utilizzare 'sinistro', 'destro', 'alto' o 'basso'.")
# Esegui il ritaglio dell'immagine
cropped_image = image.crop(crop_box)
return cropped_image
from PIL import Image
def make_pixelated(image, pixel_size):
if pixel_size > image.width or pixel_size > image.height:
raise ValueError("Top, bottom, left, and right padding must higher than the pixel_size!")
small_image = image.resize((image.width // pixel_size, image.height // pixel_size), Image.Resampling.NEAREST)
pixelated_image = small_image.resize(image.size, Image.Resampling.NEAREST)
return pixelated_image
def flip_and_stretch(image, flip_direction, stretch_value):
# Inverti l'immagine in base alla direzione specificata
# Calcola le nuove dimensioni dell'immagine con stretching
original_width, original_height = image.size
if flip_direction == "h":
flipped_image = image.transpose(Image.Transpose.FLIP_LEFT_RIGHT)
stretched_height = original_height
stretched_width = stretch_value
elif flip_direction == "v":
flipped_image = image.transpose(Image.Transpose.FLIP_TOP_BOTTOM)
stretched_width = original_width
stretched_height = stretch_value
else:
raise ValueError("La direzione specificata non è valida. Utilizzare 'orizzontale' o 'verticale'.")
# "Stretcha" l'immagine alle nuove dimensioni
stretched_image = flipped_image.resize((stretched_width, stretched_height))
return stretched_image
def create_noise_image(width, height):
noise_array = np.random.randint(0, 256, (height, width, 3), dtype=np.uint8)
# Crea un'immagine PIL utilizzando i valori dei pixel generati
noise_image = Image.fromarray(noise_array)
return noise_image
def blend_images(image1, image2, blend_percentage):
# Assicurati che le due immagini abbiano le stesse dimensioni
if image1.size != image2.size:
raise ValueError("Le dimensioni delle due immagini devono essere uguali.")
# Blend delle due immagini in base alla percentuale specificata
blended_image = Image.blend(image1, image2, blend_percentage)
return blended_image
def image_paste(main_image, image_to_paste,side):
# Ottieni le dimensioni delle immagini
width_main, height_main = main_image.size
width_paste, height_paste = image_to_paste.size
# Calcola le coordinate di incollaggio in base alla posizione desiderata
if side == "t":
# Crea una nuova immagine che sarà la combinazione delle due immagini
new_width = width_main
new_height = height_main + height_paste
new_image = Image.new("RGB", (new_width, new_height))
new_image.paste(image_to_paste, (0,0))
new_image.paste(main_image, (0,height_paste))
elif side == "b":
new_width = width_main
new_height = height_main + height_paste
new_image = Image.new("RGB", (new_width, new_height))
new_image.paste(image_to_paste, (0,height_main))
new_image.paste(main_image, (0,0))
elif side == "r":
new_width = width_main + width_paste
new_height = height_main
new_image = Image.new("RGB", (new_width, new_height))
new_image.paste(image_to_paste, (width_main, 0))
new_image.paste(main_image, (0, 0))
elif side == "l":
new_width = width_main + width_paste
new_height = height_main
new_image = Image.new("RGB", (new_width, new_height))
new_image.paste(image_to_paste, (0, 0))
new_image.paste(main_image, (width_paste, 0))
return new_image
def resize_image(image,noise,pixel_size, pixel_to_copy, left, right, top, bottom, temperature=5.0,hue=0,brightness=32,contrast=0,saturation=0,gamma=2):
# RIGHT SIDE
if right != 0:
r_image = extract_pixels(image, "r", pixel_to_copy)
r_image= flip_and_stretch(r_image, "h", right)
r_image = make_pixelated(r_image, pixel_size)
r_noise = create_noise_image(r_image.size[0], r_image.size[1])
r_image = blend_images(r_image, r_noise, noise)
r_image = color_correct(r_image, temperature,hue,brightness,contrast,saturation,gamma)
r_image= Image.fromarray(r_image)
r_image = image_paste(image, r_image,"r")
else:
r_image = image
# LEFT SIDE
if left != 0:
l_image = extract_pixels(r_image, "l", pixel_to_copy)
l_image = flip_and_stretch(l_image, "h", left)
l_image = make_pixelated(l_image, pixel_size)
l_noise = create_noise_image(l_image.size[0], l_image.size[1])
l_image = blend_images(l_image, l_noise, noise)
l_image = color_correct(l_image, temperature,hue,brightness,contrast,saturation,gamma)
l_image= Image.fromarray(l_image)
l_image = image_paste(r_image, l_image,"l")
else:
l_image = r_image
# TOP
if top != 0:
t_image = extract_pixels(l_image, "t", pixel_to_copy)
t_image = flip_and_stretch(t_image, "v", top)
t_image = make_pixelated(t_image, pixel_size)
t_noise = create_noise_image(t_image.size[0], t_image.size[1])
t_image = blend_images(t_image, t_noise, noise)
t_image = color_correct(t_image, temperature,hue,brightness,contrast,saturation,gamma)
t_image= Image.fromarray(t_image)
t_image = image_paste(l_image, t_image,"t")
else:
t_image = l_image
# BOTTOM
if bottom != 0:
b_image = extract_pixels(t_image, "b", pixel_to_copy)
b_image = flip_and_stretch(b_image, "v", bottom)
b_image = make_pixelated(b_image, pixel_size)
b_noise = create_noise_image(b_image.size[0], b_image.size[1])
b_image = blend_images(b_image, b_noise, noise)
b_image = color_correct(b_image, temperature,hue,brightness,contrast,saturation,gamma)
b_image= Image.fromarray(b_image)
b_image = image_paste(t_image, b_image,"b")
else:
b_image = t_image
final_image = b_image
return final_image
class ImagePadForOutpaintAdvanced:
@classmethod
def INPUT_TYPES(s):
return {
"required": {
"image": ("IMAGE",),
"left": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}),
"top": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}),
"right": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}),
"bottom": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}),
"feathering": ("INT", {"default": 40, "min": 0, "max": MAX_RESOLUTION, "step": 1}),
"noise": ("FLOAT", {"default": 0.1, "min": 0, "max": 1.0, "step": 0.01}),
"pixel_size": ("INT", {"default": 8, "min": 8, "max": 64, "step": 8}),
"pixel_to_copy": ("INT", {"default": 32, "min": 0, "max": MAX_RESOLUTION, "step": 1}),
"temperature": ("FLOAT",{"default": 0, "min": -100, "max": 100, "step": 5},),
"hue": ("FLOAT", {"default": 0, "min": -90, "max": 90, "step": 5}),
"brightness": ("FLOAT",{"default": 0, "min": -100, "max": 100, "step": 5},),
"contrast": ("FLOAT",{"default": 0, "min": -100, "max": 100, "step": 5},),
"saturation": ("FLOAT",{"default": 0, "min": -100, "max": 100, "step": 5},),
"gamma": ("FLOAT", {"default": 1, "min": 0.2, "max": 2.2, "step": 0.1}),
},
}
RETURN_TYPES = ("IMAGE", "MASK")
FUNCTION = "expand_image"
CATEGORY = "image"
def expand_image(self,image,feathering,noise,pixel_size, pixel_to_copy, left, right, top, bottom, temperature=5.0,hue=0,brightness=32,contrast=0,saturation=0,gamma=2):
d1, d2, d3, d4 = image.size()
#new_image = torch.zeros(
# (d1, d2 + top + bottom, d3 + left + right, d4),
# dtype=torch.float32,
#)
#new_image[:, top:top + d2, left:left + d3, :] = image
image = tensor2pil(image)
#image = Image.fromarray(image.astype(np.uint8))
new_image = resize_image(image,noise,pixel_size, pixel_to_copy, left, right, top, bottom, temperature,hue,brightness,contrast,saturation,gamma)
i = ImageOps.exif_transpose(new_image)
image = i.convert("RGB")
image = np.array(image).astype(np.float32) / 255.0
new_image = torch.from_numpy(image)[None,]
mask = torch.ones(
(d2 + top + bottom, d3 + left + right),
dtype=torch.float32,
)
t = torch.zeros(
(d2, d3),
dtype=torch.float32
)
if feathering > 0 and feathering * 2 < d2 and feathering * 2 < d3:
for i in range(d2):
for j in range(d3):
dt = i if top != 0 else d2
db = d2 - i if bottom != 0 else d2
dl = j if left != 0 else d3
dr = d3 - j if right != 0 else d3
d = min(dt, db, dl, dr)
if d >= feathering:
continue
v = (feathering - d) / feathering
t[i, j] = v * v
mask[top:top + d2, left:left + d3] = t
return (new_image, mask)
# A dictionary that contains all nodes you want to export with their names
# NOTE: names should be globally unique
NODE_CLASS_MAPPINGS = {
"ImagePadForOutpaintAdvanced [n-suite]": ImagePadForOutpaintAdvanced
}
# A dictionary that contains the friendly/humanly readable titles for the nodes
NODE_DISPLAY_NAME_MAPPINGS = {
"ImagePadForOutpaintAdvanced [n-suite]": "Image Pad For Outpainting Advanced [🅝-🅢🅤🅘🅣🅔]"
}
Binary file not shown.
Binary file not shown.
Binary file not shown.
+174
View File
@@ -0,0 +1,174 @@
import random
import folder_paths
import os
import json
import csv
import server
from aiohttp import web
_choice = ["YES", "NO"]
_range = ["Fixed", "Random"]
def loadCSVStyle():
csv_dir = os.path.join(folder_paths.base_path,"styles")
csv_path = os.path.join(csv_dir,"n-styles.csv")
#make directory if it doesn't exist
if not os.path.exists(csv_dir):
os.makedirs(csv_dir)
styles = []
if os.path.exists(csv_path):
with open(csv_path, newline='', encoding='utf-8') as csvfile:
reader = csv.DictReader(csvfile)
for row in reader:
styles.append(row)
else:
#create a file containing name,prompt,negative_prompt\n
with open(csv_path, "w", encoding="utf-8") as f:
f.write("name,prompt,negative_prompt\n")
f.write('NAI,"masterpiece, best quality, masterpiece, asuka langley sitting cross legged on a chair","lowres, bad anatomy, bad hands"\n')
styles.append({'name': 'NAI', 'prompt': 'masterpiece, best quality, masterpiece, asuka langley sitting cross legged on a chair', 'negative_prompt': 'lowres, bad anatomy, bad hands'})
if len(styles) == 0:
with open(csv_path, "w", encoding="utf-8") as f:
f.write("name,prompt,negative_prompt\n")
f.write('NAI,"masterpiece, best quality, masterpiece, asuka langley sitting cross legged on a chair","lowres, bad anatomy, bad hands"\n')
styles.append({'name': 'NAI', 'prompt': 'masterpiece, best quality, masterpiece, asuka langley sitting cross legged on a chair', 'negative_prompt': 'lowres, bad anatomy, bad hands'})
return (styles, )
def addStyle(name, positive_prompt, negative_prompt):
csv_dir = os.path.join(folder_paths.base_path,"styles","n-styles.csv")
#make directory if it doesn't exist
if not os.path.exists(csv_dir):
os.makedirs(csv_dir)
#backup file
backup_dir = os.path.join(folder_paths.base_path,"styles","n-styles-backup.csv")
if os.path.exists(backup_dir):
os.remove(backup_dir)
os.rename(csv_dir, backup_dir)
#edit style if it already exists else add it
with open(backup_dir, "r", encoding="utf-8") as f:
lines = f.readlines()
with open(csv_dir, "w", encoding="utf-8") as f:
for line in lines:
name_style = line.split(",")[0]
if name == name_style:
f.write(f'{name},"{positive_prompt}","{negative_prompt}"\n')
continue
f.write(line)
f.write(f'{name},"{positive_prompt}","{negative_prompt}"\n')
styles = loadCSVStyle()
def deleteStyle(name):
csv_dir = os.path.join(folder_paths.base_path,"styles","n-styles.csv")
#make directory if it doesn't exist
if not os.path.exists(csv_dir):
os.makedirs(csv_dir)
#backup file
backup_dir = os.path.join(folder_paths.base_path,"styles","n-styles-backup.csv")
if os.path.exists(backup_dir):
os.remove(backup_dir)
os.rename(csv_dir, backup_dir)
with open(csv_dir, "r", encoding="utf-8") as f:
lines = f.readlines()
with open(backup_dir, "w", encoding="utf-8") as f:
for line in lines:
if name in line:
continue
f.write(line)
@server.PromptServer.instance.routes.get("/nsuite/styles" )
async def style_get(request):
result = {"styles": loadCSVStyle()}
return web.json_response(result, content_type='application/json')
@server.PromptServer.instance.routes.post("/nsuite/styles/add" )
async def style_add(request):
data = await request.json()
name = data["name"]
positive_prompt = data["positive_prompt"]
negative_prompt = data["negative_prompt"]
addStyle(name, positive_prompt, negative_prompt)
result = {"error": "none"}
return web.json_response(result, content_type='application/json')
@server.PromptServer.instance.routes.post("/nsuite/styles/update" )
async def style_add(request):
data = await request.json()
name = data["name"]
positive_prompt = data["positive_prompt"]
negative_prompt = data["negative_prompt"]
addStyle(name, positive_prompt, negative_prompt)
result = {"error": "none"}
return web.json_response(result, content_type='application/json')
@server.PromptServer.instance.routes.post("/nsuite/styles/remove" )
async def style_delete(request):
data = await request.json()
name = data["name"]
deleteStyle(name)
result = {"error": "none"}
return web.json_response(result, content_type='application/json')
class CLIPTextEncodeAdvancedNSuite:
@classmethod
def INPUT_TYPES(s):
return {"required":
{ "styles": ([x['name'] for x in styles[0]],),
"positive_prompt": ("STRING", {"multiline": True}),
"negative_prompt": ("STRING", {"multiline": True}),
"clip": ("CLIP", )}
}
RETURN_TYPES = ("CONDITIONING","CONDITIONING")
RETURN_NAMES = ("positive", "negative")
FUNCTION = "encode"
CATEGORY = "N-Suite/Experimental"
def encode(self, clip, positive_prompt, negative_prompt,styles):
p_tokens = clip.tokenize(positive_prompt)
n_tokens = clip.tokenize(negative_prompt)
p_cond, p_pooled = clip.encode_from_tokens(p_tokens, return_pooled=True)
n_cond, n_pooled = clip.encode_from_tokens(n_tokens, return_pooled=True)
return ([[p_cond, {"pooled_output": p_pooled}]],[[n_cond, {"pooled_output": n_pooled}]], )
# A dictionary that contains all nodes you want to export with their names
# NOTE: names should be globally unique
NODE_CLASS_MAPPINGS = {
"CLIPTextEncodeAdvancedNSuite [n-suite]": CLIPTextEncodeAdvancedNSuite
}
# A dictionary that contains the friendly/humanly readable titles for the nodes
NODE_DISPLAY_NAME_MAPPINGS = {
"CLIPTextEncodeAdvancedNSuite [n-suite]": "CLIP Text Encode Advanced [🅝-🅢🅤🅘🅣🅔]"
}
+3 -3
View File
@@ -23,7 +23,7 @@ class DynamicPrompt:
} }
RETURN_TYPES = ("STRING",) RETURN_TYPES = ("STRING",)
FUNCTION = "prompt_generator" FUNCTION = "prompt_generator"
CATEGORY = "conditioning" CATEGORY = "N-Suite/Conditioning"
OUTPUT_NODE = True OUTPUT_NODE = True
""" """
@@ -82,10 +82,10 @@ class DynamicPrompt:
# A dictionary that contains all nodes you want to export with their names # A dictionary that contains all nodes you want to export with their names
# NOTE: names should be globally unique # NOTE: names should be globally unique
NODE_CLASS_MAPPINGS = { NODE_CLASS_MAPPINGS = {
"DynamicPrompt": DynamicPrompt "DynamicPrompt [n-suite]": DynamicPrompt
} }
# A dictionary that contains the friendly/humanly readable titles for the nodes # A dictionary that contains the friendly/humanly readable titles for the nodes
NODE_DISPLAY_NAME_MAPPINGS = { NODE_DISPLAY_NAME_MAPPINGS = {
"DynamicPrompt": "Dynamic Prompt" "DynamicPrompt [n-suite]": "Dynamic Prompt [🅝-🅢🅤🅘🅣🅔]"
} }
+324
View File
@@ -0,0 +1,324 @@
import os
import cv2
import sys
import torch
import argparse
from PIL import Image, ImageOps
import folder_paths
import numpy as np
from tqdm import tqdm
from torch.nn import functional as F
import _thread
from queue import Queue, Empty
from pathlib import Path
sys.path.append(os.path.join(str(Path(__file__).parent.parent),"libs","rifle"))
from model.pytorch_msssim import ssim_matlab
interpolation_temp_input_folder = os.path.join(folder_paths.get_temp_directory(),"n-suite","interpolation_input")
interpolation_temp_output_folder = os.path.join(folder_paths.get_temp_directory(),"n-suite","interpolation_output")
try:
os.makedirs(interpolation_temp_input_folder)
except:
pass
try:
os.makedirs(interpolation_temp_output_folder)
except:
pass
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
scale=1
torch.set_grad_enabled(False)
if torch.cuda.is_available():
torch.backends.cudnn.enabled = True
try:
from train_log.RIFE_HDv3 import Model
except:
print("Please download our model from model list")
model = Model()
if not hasattr(model, 'version'):
model.version = 0
model_folder= os.path.join(folder_paths.folder_names_and_paths["custom_nodes"][0][0],'ComfyUI-N-Nodes','libs','rifle','train_log')
output_frames = []
def clear_write_buffer(user_args, write_buffer,output_folder):
cnt = 0
while True:
item = write_buffer.get()
if item is None:
break
cv2.imwrite(os.path.join(output_folder, '{:0>7d}.png'.format(cnt)), item[:, :, ::-1])
cnt += 1
def build_read_buffer(img, read_buffer, videogen):
try:
for frame in videogen:
if not img is None:
frame = cv2.imread(os.path.join(img, frame), cv2.IMREAD_UNCHANGED)[:, :, ::-1].copy()
read_buffer.put(frame)
except:
pass
read_buffer.put(None)
def make_inference(I0, I1, n):
global model
if model.version >= 3.9:
res = []
for i in range(n):
res.append(model.inference(I0, I1, (i+1) * 1. / (n+1), scale))
return res
else:
middle = model.inference(I0, I1, scale)
if n == 1:
return [middle]
first_half = make_inference(I0, middle, n=n//2)
second_half = make_inference(middle, I1, n=n//2)
if n%2:
return [*first_half, middle, *second_half]
else:
return [*first_half, *second_half]
def get_output_filename(input_file_path, output_folder, file_extension,suffix="") :
existing_files = [f for f in os.listdir(output_folder)]
max_progressive = 0
for filename in existing_files:
parts_ext = filename.split(".")
parts = parts_ext[0]
if len(parts) > 2 and parts.isdigit():
progressive = int(parts)
max_progressive = max(max_progressive, progressive)
new_progressive = max_progressive + 1
new_filename = f"{new_progressive:07d}{suffix}{file_extension}"
return os.path.join(output_folder, new_filename), new_filename
def image_preprocessing(i):
i = ImageOps.exif_transpose(i)
image = i.convert("RGB")
image = np.array(image).astype(np.float32) / 255.0
image = torch.from_numpy(image)[None,]
return image
_choice = ["YES", "NO"]
_range = ["Fixed", "Random"]
class FrameInterpolator:
def __init__(self):
model.load_model(model_folder, -1)
print("Loaded 3.x/4.x HD model.")
model.eval()
model.device()
self.type = "output"
@classmethod
def INPUT_TYPES(s):
#clear directory
try:
for file in os.listdir(interpolation_temp_input_folder):
os.remove(os.path.join(interpolation_temp_input_folder,file))
for file in os.listdir(interpolation_temp_output_folder):
os.remove(os.path.join(interpolation_temp_output_folder,file))
except:
pass
return {"required":
{"images": ("IMAGE", ),
"METADATA": ("STRING", {"default": "", "forceInput": True} ),
"multiplier": ("INT", {"default": 2, "min": 1, "step": 1}),
},
}
RETURN_TYPES = ()
FUNCTION = "save_video"
OUTPUT_NODE = True
CATEGORY = "N-Suite/Video"
RETURN_TYPES = ("IMAGE","STRING",)
OUTPUT_IS_LIST = (True, False, )
RETURN_NAMES = ("IMAGES","METADATA",)
FUNCTION = "interpolate"
def interpolate(self,images,multiplier,METADATA):
fps = METADATA[0]*multiplier
frame_number = METADATA[1]
video_name = METADATA[2]
for image in images:
full_input_temp_frame_folder,file = get_output_filename("", interpolation_temp_input_folder, ".png")
file_name = file
i = 255. * image.cpu().numpy()
img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
metadata = None
#file = f"frame_{counter:05}_.png"
img.save(full_input_temp_frame_folder, pnginfo=metadata, compress_level=0)
try:
file_name_number = int(file.split(".")[0])
except:
file_name_number = 0
image_list = []
if(file_name_number >= frame_number):
videogen = []
for f in os.listdir(interpolation_temp_input_folder):
if 'png' in f:
videogen.append(f)
tot_frame = len(videogen)
videogen.sort(key= lambda x:int(x[:-4]))
lastframe = cv2.imread(os.path.join(interpolation_temp_input_folder, videogen[0]), cv2.IMREAD_UNCHANGED)[:, :, ::-1].copy()
videogen = videogen[1:]
h, w, _ = lastframe.shape
tmp = max(128, int(128 / scale))
ph = ((h - 1) // tmp + 1) * tmp
pw = ((w - 1) // tmp + 1) * tmp
padding = (0, pw - w, 0, ph - h)
pbar = tqdm(total=tot_frame)
write_buffer = Queue(maxsize=500)
read_buffer = Queue(maxsize=500)
_thread.start_new_thread(build_read_buffer, (interpolation_temp_input_folder, read_buffer, videogen))
_thread.start_new_thread(clear_write_buffer, (interpolation_temp_input_folder, write_buffer, interpolation_temp_output_folder))
I1 = torch.from_numpy(np.transpose(lastframe, (2,0,1))).to(device, non_blocking=True).unsqueeze(0).float() / 255.
I1 = F.pad(I1, padding)
temp = None # save lastframe when processing static frame
while True:
if temp is not None:
frame = temp
temp = None
else:
frame = read_buffer.get()
if frame is None:
break
I0 = I1
I1 = torch.from_numpy(np.transpose(frame, (2,0,1))).to(device, non_blocking=True).unsqueeze(0).float() / 255.
I1 = F.pad(I1, padding)
I0_small = F.interpolate(I0, (32, 32), mode='bilinear', align_corners=False)
I1_small = F.interpolate(I1, (32, 32), mode='bilinear', align_corners=False)
ssim = ssim_matlab(I0_small[:, :3], I1_small[:, :3])
break_flag = False
if ssim > 0.996:
frame = read_buffer.get() # read a new frame
if frame is None:
break_flag = True
frame = lastframe
else:
temp = frame
I1 = torch.from_numpy(np.transpose(frame, (2,0,1))).to(device, non_blocking=True).unsqueeze(0).float() / 255.
I1 = F.pad(I1, padding)
I1 = model.inference(I0, I1, scale)
I1_small = F.interpolate(I1, (32, 32), mode='bilinear', align_corners=False)
ssim = ssim_matlab(I0_small[:, :3], I1_small[:, :3])
frame = (I1[0] * 255).byte().cpu().numpy().transpose(1, 2, 0)[:h, :w]
if ssim < 0.2:
output = []
for i in range(multiplier - 1):
output.append(I0)
else:
output = make_inference(I0, I1, multiplier-1)
write_buffer.put(lastframe)
for mid in output:
mid = (((mid[0] * 255.).byte().cpu().numpy().transpose(1, 2, 0)))
write_buffer.put(mid[:h, :w])
pbar.update(1)
lastframe = frame
if break_flag:
break
write_buffer.put(lastframe)
import time
while(not write_buffer.empty()):
time.sleep(0.1)
pbar.close()
METADATA = [fps, len(os.listdir(interpolation_temp_output_folder)),video_name]
images = [os.path.join(interpolation_temp_output_folder, filename) for filename in os.listdir(interpolation_temp_output_folder) if filename.endswith(".png")]
images.sort(key=lambda f: int(''.join(filter(str.isdigit, f))))
for image in images:
image_list.append(image_preprocessing(Image.open(image)))
return ( image_list,METADATA)
# NOTE: names should be globally unique
NODE_CLASS_MAPPINGS = {
"FrameInterpolator [n-suite]": FrameInterpolator,
}
# A dictionary that contains the friendly/humanly readable titles for the nodes
NODE_DISPLAY_NAME_MAPPINGS = {
"FrameInterpolator [n-suite]": "FrameInterpolator [🅝-🅢🅤🅘🅣🅔]"
}
-152
View File
@@ -1,152 +0,0 @@
import folder_paths
import os
from llama_cpp import Llama
import copy
from typing_extensions import TypedDict, Literal
from typing import List, Optional
_choice = ["YES", "NO"]
def env_or_def(env, default):
if (env in os.environ):
return os.environ[env]
return default
supported_gpt_extensions = set([ '.bin','.gguf'])
try:
folder_paths.folder_names_and_paths["GPTcheckpoints"] = (folder_paths.folder_names_and_paths["GPTcheckpoints"][0], supported_gpt_extensions)
except:
# check if GPTcheckpoints exists otherwise create
if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints")):
os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints"))
folder_paths.folder_names_and_paths["GPTcheckpoints"] = ([os.path.join(folder_paths.models_dir, "GPTcheckpoints")], supported_gpt_extensions)
class GPTLoaderSimple:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"ckpt_name": (folder_paths.get_filename_list("GPTcheckpoints"), ),
"gpu_layers": ("INT", {"default": 27, "min": 0, "max": 100, "step": 1}),
"n_threads": ("INT", {"default": 8, "min": 1, "max": 100, "step": 1}),
"max_ctx": ("INT", {"default": 2048, "min": 300, "max": 100000, "step": 64}),
}}
RETURN_TYPES = ("CUSTOM","STRING")
RETURN_NAMES = ("model", "model_path")
FUNCTION = "load_gpt_checkpoint"
CATEGORY = "loaders"
print()
def load_gpt_checkpoint(self, ckpt_name, gpu_layers,n_threads,max_ctx):
ckpt_path = folder_paths.get_full_path("GPTcheckpoints", ckpt_name)
llm = Llama(model_path=ckpt_path,n_gpu_layers=gpu_layers,verbose=False,n_threads=n_threads, n_ctx=max_ctx, )
return llm, ckpt_path
class GPTSampler:
"""
A custom node for text generation using GPT
Attributes
----------
max_tokens (`int`): Maximum number of tokens in the generated text.
temperature (`float`): Temperature parameter for controlling randomness (0.2 to 1.0).
top_p (`float`): Top-p probability for nucleus sampling.
logprobs (`int`|`None`): Number of log probabilities to output alongside the generated text.
echo (`bool`): Whether to print the input prompt alongside the generated text.
stop (`str`|`List[str]`|`None`): Tokens at which to stop generation.
frequency_penalty (`float`): Frequency penalty for word repetition.
presence_penalty (`float`): Presence penalty for word diversity.
repeat_penalty (`float`): Penalty for repeating a prompt's output.
top_k (`int`): Top-k tokens to consider during generation.
stream (`bool`): Whether to generate the text in a streaming fashion.
tfs_z (`float`): Temperature scaling factor for top frequent samples.
model (`str`): The GPT model to use for text generation.
"""
def __init__(self):
self.temp_prompt = ""
pass
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"prompt": ("STRING",{"forceInput": True} ),
"model": ("CUSTOM", {"default": ""}),
"model_path": ("STRING", {"default": "","forceInput": True}),
"max_tokens": ("INT", {"default": 2048}),
"temperature": ("FLOAT", {"default": 0.7, "min": 0.2, "max": 1.0}),
"top_p": ("FLOAT", {"default": 0.5, "min": 0.1, "max": 1.0}),
"logprobs": ("INT", {"default": 0}),
"echo": (["enable", "disable"], {"default": "disable"}),
"stop_token": ("STRING", {"default": "STOPTOKEN"}),
"frequency_penalty": ("FLOAT", {"default": 0.0}),
"presence_penalty": ("FLOAT", {"default": 0.0}),
"repeat_penalty": ("FLOAT", {"default": 1.17647}),
"top_k": ("INT", {"default": 40}),
"tfs_z": ("FLOAT", {"default": 1.0}),
"print_output": (["enable", "disable"], {"default": "disable"}),
"cached": (_choice,{"default": "NO"} ),
"prefix": ("STRING", {"default": "### Instruction: "}),
"suffix": ("STRING", {"default": "### Response: "}),
}
}
RETURN_TYPES = ("STRING",)
FUNCTION = "generate_text"
CATEGORY = "sampling"
def generate_text(self,prompt, max_tokens, temperature, top_p, logprobs, echo, stop_token, frequency_penalty, presence_penalty, repeat_penalty, top_k, tfs_z, model,model_path,print_output,cached,prefix,suffix):
if cached == "NO":
# Call your GPT generation function here using the provided parameters
composed_prompt = f"{prefix} {prompt} {suffix}"
cont =""
stream = model( max_tokens=max_tokens, stop=[stop_token], stream=False,frequency_penalty=frequency_penalty,presence_penalty=presence_penalty ,repeat_penalty=repeat_penalty,temperature=temperature,top_k=top_k,top_p=top_p,model=model_path,prompt=composed_prompt)
print(len(stream))
print(stream)
cont= stream["choices"][0]["text"]
self.temp_prompt = cont
else:
cont = self.temp_prompt
#remove fist 30 characters of cont
try:
if print_output == "enable":
print(f"Input: {prompt}\nGenerated Text: {cont}")
return {"ui": {"text": cont}, "result": (cont,)}
except:
if print_output == "enable":
print(f"Input: {prompt}\nGenerated Text: ")
return {"ui": {"text": " "}, "result": (" ",)}
NODE_CLASS_MAPPINGS = {
"GPT Loader Simple": GPTLoaderSimple,
"GPTSampler": GPTSampler
}
# A dictionary that contains the friendly/humanly readable titles for the nodes
NODE_DISPLAY_NAME_MAPPINGS = {
"GPT Loader Simple": "GPT Loader Simple",
"GPTSampler": "GPT Text Sampler"
}
+413
View File
@@ -0,0 +1,413 @@
import folder_paths
import os
from pathlib import Path
import sys
import torch
from huggingface_hub import snapshot_download, hf_hub_download
sys.path.append(os.path.join(str(Path(__file__).parent.parent),"libs"))
import joytag_models
try:
from moondream_repo.moondream.moondream import Moondream
moondream_loaded = True
except Exception as e:
moondream_loaded = False
print(f"Moondream error: reinstall N-Suite dependencies with ComfyUI Manager or requirements.txt.\nTorch must be >= 2.1.0 (ERROR: {e})")
from PIL import Image
from transformers import CodeGenTokenizerFast as Tokenizer
#,AutoTokenizer, AutoModelForCausalLM
import numpy as np
models_base_path = os.path.join(folder_paths.models_dir, "GPTcheckpoints")
_choice = ["YES", "NO"]
_folders_whitelist = ["moondream","joytag"]#,"internlm"]
def env_or_def(env, default):
if (env in os.environ):
return os.environ[env]
return default
def get_model_path(folder_list, model_name):
for folder_path in folder_list:
if folder_path.endswith(model_name):
return folder_path
def get_model_list(models_base_path,supported_gpt_extensions):
all_models = []
try:
for file in os.listdir(models_base_path):
if os.path.isdir(os.path.join(models_base_path, file)):
if file in _folders_whitelist:
all_models.append(os.path.join(models_base_path, file))
else:
if file.endswith(tuple(supported_gpt_extensions)):
all_models.append(os.path.join(models_base_path, file))
except:
print(f"Path {models_base_path} not valid.")
return all_models
def tensor2pil(image):
return Image.fromarray(np.clip(255. * image.cpu().numpy().squeeze(), 0, 255).astype(np.uint8))
# Convert PIL to Tensor
def pil2tensor(image):
return torch.from_numpy(np.array(image).astype(np.float32) / 255.0).unsqueeze(0)
def detect_device():
"""
Detects the appropriate device to run on, and return the device and dtype.
"""
if torch.cuda.is_available():
return torch.device("cuda"), torch.float16
elif torch.backends.mps.is_available():
return torch.device("mps"), torch.float16
else:
return torch.device("cpu"), torch.float32
def load_joytag(ckpt_path,cpu=False):
print("JOYTAG MODEL DETECTED")
jt_config = os.path.join(models_base_path,"joytag","config.json")
jt_readme= os.path.join(models_base_path,"joytag","README.md")
jt_top_tags= os.path.join(models_base_path,"joytag","top_tags.txt")
jt_model= os.path.join(models_base_path,"joytag","model.safetensors")
if os.path.exists(jt_config)==False or os.path.exists(jt_readme)==False or os.path.exists(jt_top_tags)==False or os.path.exists(jt_model)==False:
snapshot_download("fancyfeast/joytag",local_dir = os.path.join(models_base_path,"joytag"),local_dir_use_symlinks = False,)
model = joytag_models.VisionModel.load_model(ckpt_path)
model.eval()
if cpu:
return model.to('cpu')
else:
return model.to('cuda')
def run_joytag(images, prompt, max_tags, model_funct):
with open(os.path.join(models_base_path,'joytag','top_tags.txt') , 'r') as f:
top_tags = [line.strip() for line in f.readlines() if line.strip()]
if images is None:
raise ValueError("No image provided")
top_tags_processed = []
for image in images:
_, scores = joytag_models.predict(image, model_funct, top_tags)
top_tags_scores = sorted(scores.items(), key=lambda x: x[1], reverse=True)[:max_tags]
# Extract the tags from the pairs
top_tags_processed.append(', '.join([tag for tag, _ in top_tags_scores]))
return top_tags_processed
def load_moondream(ckpt_path,cpu=False):
dtype = torch.float32
if cpu:
device=torch.device("cpu")
else:
device = torch.device("cuda")
config_json=os.path.join(os.path.join(models_base_path,"moondream"),'config.json')
if os.path.exists(config_json)==False:
hf_hub_download("vikhyatk/moondream1",
local_dir=os.path.join(models_base_path,"moondream"),
local_dir_use_symlinks=True,
filename="config.json",
endpoint='https://hf-mirror.com')
model_safetensors=os.path.join(models_base_path,"moondream",'model.safetensors')
if os.path.exists(model_safetensors)==False:
hf_hub_download("vikhyatk/moondream1",
local_dir=os.path.join(models_base_path,"moondream"),
local_dir_use_symlinks=True,
filename="model.safetensors",
endpoint='https://hf-mirror.com')
tokenizer_json=os.path.join(models_base_path,"moondream",'tokenizer.json')
if os.path.exists(tokenizer_json)==False:
hf_hub_download("vikhyatk/moondream1",
local_dir=os.path.join(models_base_path,"moondream"),
local_dir_use_symlinks=True,
filename="tokenizer.json",
endpoint='https://hf-mirror.com')
if moondream_loaded:
tokenizer = Tokenizer.from_pretrained(os.path.join(models_base_path,"moondream"))
moondream = Moondream.from_pretrained(os.path.join(models_base_path,"moondream")).to(device=device, dtype=dtype)
moondream.eval()
else:
tokenizer=None
moondream=None
return ([moondream, tokenizer])
def run_moondream(images, prompt, max_tags, model_funct):
from PIL import Image
moondream = model_funct[0]
tokenizer = model_funct[1]
list_descriptions = []
for image in images:
im=tensor2pil(image)
image_embeds = moondream.encode_image(im)
try:
list_descriptions.append(moondream.answer_question(image_embeds, prompt,tokenizer))
except ValueError:
print("\n\n\n")
raise ModuleNotFoundError("Moondream requires the dependency versions declared in N-Suite requirements.txt. Reinstall dependencies with ComfyUI Manager.")
return list_descriptions
"""
def load_internlm(ckpt_path,cpu=False):
local_dir=os.path.join(os.path.join(models_base_path,"internlm"))
local_model_1 = os.path.join(local_dir,"pytorch_model-00001-of-00002.bin")
local_model_2 = os.path.join(local_dir,"pytorch_model-00002-of-00002.bin")
if os.path.exists(local_model_1) and os.path.exists(local_model_2):
model_path = local_dir
else:
model_path = snapshot_download("internlm/internlm-xcomposer2-vl-7b", local_dir=local_dir, revision="f8e6ab8d7ff14dbd6b53335c93ff8377689040bf", local_dir_use_symlinks=False)
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
if torch.cuda.is_available() and cpu == False:
model = AutoModelForCausalLM.from_pretrained(
model_path,
torch_dtype="auto",
trust_remote_code=True,
device_map="auto"
).eval()
else:
model = model.cpu().float().eval()
model.tokenizer = tokenizer
#device = device
#dtype = dtype
name = "internlm"
#low_memory = low_memory
return ([model, tokenizer])
def run_internlm(image, prompt, max_tags, model_funct):
model = model_funct[0]
tokenizer = model_funct[1]
low_memory = True
import tempfile
image = Image.fromarray(np.clip(255. * image[0].cpu().numpy(),0,255).astype(np.uint8))
#image = model.vis_processor(image)
temp_dir = tempfile.mkdtemp()
image_path = os.path.join(temp_dir,"input.jpg")
image.save(image_path)
#image = tensor2pil(image)
if torch.cuda.is_available():
with torch.cuda.amp.autocast():
response, _ = model.chat(
query=prompt,
image=image_path,
tokenizer= tokenizer,
history=[],
do_sample=True
)
if low_memory:
torch.cuda.empty_cache()
print(f"Memory usage: {torch.cuda.memory_allocated() / 1024 ** 3:.2f} GB")
model.to("cpu", dtype=torch.float16)
print(f"Memory usage: {torch.cuda.memory_allocated() / 1024 ** 3:.2f} GB")
else:
response, _ = model.chat(
query=prompt,
image=image,
tokenizer= tokenizer,
history=[],
do_sample=True
)
return response
"""
if not os.path.isdir(models_base_path):
os.mkdir(models_base_path)
#create folder if it doesn't exist
if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","joytag")):
os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","joytag"))
if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","moondream")):
os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","moondream"))
"""#internlm
if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","internlm")):
os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","internlm"))
"""
#folder_paths.folder_names_and_paths["GPTcheckpoints"] += (os.listdir(models_base_path),)
MODEL_FUNCTIONS = {
'joytag': run_joytag,
'moondream': run_moondream
}
MODEL_LOAD_FUNCTIONS = {
'joytag': load_joytag,
'moondream': load_moondream
}
all_models = get_model_list(models_base_path, set())
all_models_names = [os.path.basename(model) for model in all_models]
class GPTLoaderSimple:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"ckpt_name": (all_models_names, ),
"gpu_layers": ("INT", {"default": 27, "min": 0, "max": 100, "step": 1}),
"n_threads": ("INT", {"default": 8, "min": 1, "max": 100, "step": 1}),
"max_ctx": ("INT", {"default": 2048, "min": 300, "max": 100000, "step": 64}),
}}
RETURN_TYPES = ("CUSTOM", )
RETURN_NAMES = ("model",)
FUNCTION = "load_gpt_checkpoint"
DESCRIPTION = "Loads a Moondream or JoyTag image-captioning model. GGUF and LLaVA support was removed in version 1.2.0."
CATEGORY = "N-Suite/loaders"
def load_gpt_checkpoint(self, ckpt_name, gpu_layers, n_threads, max_ctx):
ckpt_path = get_model_path(all_models,ckpt_name)
if ckpt_name not in MODEL_LOAD_FUNCTIONS:
raise ValueError(f"Unsupported model: {ckpt_name}")
cpu = gpu_layers == 0
llm = MODEL_LOAD_FUNCTIONS[ckpt_name](ckpt_path, cpu)
return ([llm, ckpt_name, ckpt_path],)
class GPTSampler:
"""
A custom node for text generation using GPT
Attributes
----------
max_tokens (`int`): Maximum number of tokens in the generated text.
temperature (`float`): Temperature parameter for controlling randomness (0.2 to 1.0).
top_p (`float`): Top-p probability for nucleus sampling.
logprobs (`int`|`None`): Number of log probabilities to output alongside the generated text.
echo (`bool`): Whether to print the input prompt alongside the generated text.
stop (`str`|`List[str]`|`None`): Tokens at which to stop generation.
frequency_penalty (`float`): Frequency penalty for word repetition.
presence_penalty (`float`): Presence penalty for word diversity.
repeat_penalty (`float`): Penalty for repeating a prompt's output.
top_k (`int`): Top-k tokens to consider during generation.
stream (`bool`): Whether to generate the text in a streaming fashion.
tfs_z (`float`): Temperature scaling factor for top frequent samples.
model (`str`): The GPT model to use for text generation.
"""
def __init__(self):
self.temp_prompt = ""
pass
@classmethod
def INPUT_TYPES(cls):
return {
"required": {
"model": ("CUSTOM", {"default": ""}),
"max_tokens": ("INT", {"default": 2048}),
"temperature": ("FLOAT", {"default": 0.7, "min": 0.2, "max": 1.0}),
"top_p": ("FLOAT", {"default": 0.5, "min": 0.1, "max": 1.0}),
"logprobs": ("INT", {"default": 0}),
"echo": (["enable", "disable"], {"default": "disable"}),
"stop_token": ("STRING", {"default": "STOPTOKEN"}),
"frequency_penalty": ("FLOAT", {"default": 0.0}),
"presence_penalty": ("FLOAT", {"default": 0.0}),
"repeat_penalty": ("FLOAT", {"default": 1.17647}),
"top_k": ("INT", {"default": 40}),
"tfs_z": ("FLOAT", {"default": 1.0}),
"print_output": (["enable", "disable"], {"default": "disable"}),
"cached": (_choice,{"default": "NO"} ),
"prefix": ("STRING", {"default": "### Instruction: "}),
"suffix": ("STRING", {"default": "### Response: "}),
"max_tags": ("INT", {"default": 50}),
},
"optional": {
"prompt": ("STRING",{"forceInput": True} ),
"image": ("IMAGE",),
}
}
RETURN_TYPES = ("STRING",)
OUTPUT_IS_LIST = (True,)
FUNCTION = "generate_text"
CATEGORY = "N-Suite/Sampling"
def generate_text(self, max_tokens, temperature, top_p, logprobs, echo, stop_token, frequency_penalty, presence_penalty, repeat_penalty, top_k, tfs_z, model,print_output,cached,prefix,suffix,max_tags,image=None,prompt=None):
model_funct = model[0]
model_name = model[1]
model_path = model[2]
if cached == "NO":
if model_name in MODEL_FUNCTIONS and os.path.isdir(model_path):
cont = MODEL_FUNCTIONS[model_name](image, prompt, max_tags, model_funct)
else:
raise ValueError(f"Unsupported model: {model_name}")
else:
cont = self.temp_prompt
#remove fist 30 characters of cont
try:
if print_output == "enable":
print(f"Input: {prompt}\nGenerated Text: {cont}")
return {"ui": {"text": cont}, "result": (cont,)}
except:
if print_output == "enable":
print(f"Input: {prompt}\nGenerated Text: ")
return {"ui": {"text": " "}, "result": (" ",)}
NODE_CLASS_MAPPINGS = {
"GPT Loader Simple [n-suite]": GPTLoaderSimple,
"GPT Sampler [n-suite]": GPTSampler
}
# A dictionary that contains the friendly/humanly readable titles for the nodes
NODE_DISPLAY_NAME_MAPPINGS = {
"GPT Loader Simple [n-suite]": "GPT Loader Simple [🅝-🅢🅤🅘🅣🅔]",
"GPT Sampler [n-suite]": "Image Caption Sampler [🅝-🅢🅤🅘🅣🅔]"
}
+124
View File
@@ -0,0 +1,124 @@
import os
import cv2
import sys
import torch
import argparse
from PIL import Image, ImageOps
import folder_paths
import numpy as np
from tqdm import tqdm
from torch.nn import functional as F
import _thread
from queue import Queue, Empty
from pathlib import Path
def image_preprocessing(i):
i = ImageOps.exif_transpose(i)
image = i.convert("RGB")
image = np.array(image).astype(np.float32) / 255.0
image = torch.from_numpy(image)[None,]
return image
class LoadImageFromFolder:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(s):
return {"required": { "folder":("STRING", {"default": ""} ),
"fps":("INT", {"default": 30})
}}
RETURN_TYPES = ("IMAGE","INT","INT","INT","STRING","STRING",)
RETURN_NAMES = ("IMAGES","MAX WIDTH","MAX HEIGHT","IMAGE COUNT","PATH","IMAGE LIST")
FUNCTION = "load_images"
OUTPUT_IS_LIST = (True,False,False,False,False,False,)
CATEGORY = "N-Suite/Experimental"
def load_images(self, folder,fps):
image_list = []
image_names = []
max_width = 0
max_height = 0
frame_count = 0
images = [os.path.join(folder, filename) for filename in os.listdir(folder) if filename.endswith(".png") or filename.endswith(".jpg")]
for image_path in images:
#get image name
image_names.append(image_path.split("/")[-1])
image = Image.open(image_path)
width, height = image.size
max_width = max(max_width, width)
max_height = max(max_height, height)
image_list.append((image_preprocessing(image)))
frame_count += 1
image_names_final='\n'.join(image_names)
print (f"Details: {frame_count} frames, {max_width}x{max_height}")
return (image_list, max_width, max_height,frame_count,folder,image_names_final,)
class SaveCaptionsFromImageList:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(s):
return {"required": { "folder":("STRING", {"default": ""} ),
"fps":("INT", {"default": 30})
}}
RETURN_TYPES = ("IMAGE","INT","INT","INT","STRING","STRING",)
RETURN_NAMES = ("IMAGES","MAX WIDTH","MAX HEIGHT","IMAGE COUNT","PATH","IMAGE LIST")
FUNCTION = "load_images"
OUTPUT_IS_LIST = (True,False,False,False,False,False,)
CATEGORY = "LJRE/Loader"
def load_images(self, folder,fps):
image_list = []
image_names = []
max_width = 0
max_height = 0
frame_count = 0
images = [os.path.join(folder, filename) for filename in os.listdir(folder) if filename.endswith(".png") or filename.endswith(".jpg")]
for image_path in images:
#get image name
image_names.append(image_path.split("/")[-1])
image = Image.open(image_path)
width, height = image.size
max_width = max(max_width, width)
max_height = max(max_height, height)
image_list.append((image_preprocessing(image)))
frame_count += 1
image_names_final='\n'.join(image_names)
print (f"Details: {frame_count} frames, {max_width}x{max_height}")
return (image_list, max_width, max_height,frame_count,folder,image_names_final,)
# NOTE: names should be globally unique
NODE_CLASS_MAPPINGS = {
"LoadImageFromFolder [n-suite]": LoadImageFromFolder,
}
# A dictionary that contains the friendly/humanly readable titles for the nodes
NODE_DISPLAY_NAME_MAPPINGS = {
"LoadImageFromFolder [n-suite]": "Load Image From Folder [🅝-🅢🅤🅘🅣🅔]"
}
+21 -26
View File
@@ -19,7 +19,7 @@ class IntVariable:
RETURN_TYPES = ("INT",) RETURN_TYPES = ("INT",)
FUNCTION = "check_int" FUNCTION = "check_int"
CATEGORY = "Variables" CATEGORY = "N-Suite/Variables"
def check_int(self, value): def check_int(self, value):
if value == "": if value == "":
@@ -53,7 +53,7 @@ class FloatVariable:
RETURN_TYPES = ("FLOAT",) RETURN_TYPES = ("FLOAT",)
FUNCTION = "check" FUNCTION = "check"
CATEGORY = "Variables" CATEGORY = "N-Suite/Variables"
def check(self, value): def check(self, value):
if value == "": if value == "":
@@ -67,41 +67,36 @@ class FloatVariable:
return (value,) return (value,)
class StringVariable: class StringVariable:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(s):
return {"required": {"value": ("STRING", {"multiline": True})}}
RETURN_TYPES = ("STRING",) RETURN_TYPES = ("STRING",)
FUNCTION = "check" FUNCTION = "check"
CATEGORY = "Variables" CATEGORY = "N-Suite/Variables"
@classmethod
def INPUT_TYPES(s):
return {"required": {"string": ("STRING", {"default": "", "multiline": True})}}
def check(self, value):
if value == "undefined":
value = ""
def check(self, string):
if string == "undefined":
string = ""
#if not an int #if not an int
if not str(value): if not str(string):
value = "" string = ""
return (string,)
return (value,)
# A dictionary that contains all nodes you want to export with their names # A dictionary that contains all nodes you want to export with their names
# NOTE: names should be globally unique # NOTE: names should be globally unique
NODE_CLASS_MAPPINGS = { NODE_CLASS_MAPPINGS = {
"Integer Variable": IntVariable, "Integer Variable [n-suite]": IntVariable,
"Float Variable": FloatVariable, "Float Variable [n-suite]": FloatVariable,
"String Variable": StringVariable "String Variable [n-suite]": StringVariable
} }
# A dictionary that contains the friendly/humanly readable titles for the nodes # A dictionary that contains the friendly/humanly readable titles for the nodes
NODE_DISPLAY_NAME_MAPPINGS = { NODE_DISPLAY_NAME_MAPPINGS = {
"Variables": "Integer Variable", "Integer Variable [n-suite]": "Integer Variable [🅝-🅢🅤🅘🅣🅔]",
"Variables": "Float Variable", "Float Variable [n-suite]": "Float Variable [🅝-🅢🅤🅘🅣🅔]",
"Variables": "String Variable" "String Variable [n-suite]": "String Variable [🅝-🅢🅤🅘🅣🅔]"
} }
+290 -208
View File
@@ -16,6 +16,13 @@ import json
from comfy.cli_args import args from comfy.cli_args import args
import time import time
import concurrent.futures import concurrent.futures
import skbuild
YELLOW = '\33[33m'
END = '\33[0m'
# Brutally copied from comfy_extras/nodes_rebatch.py and modified # Brutally copied from comfy_extras/nodes_rebatch.py and modified
class LatentRebatch: class LatentRebatch:
@@ -107,32 +114,42 @@ class LatentRebatch:
input_dir = os.path.join(folder_paths.get_input_directory(),"n-suite") input_dir = os.path.join(folder_paths.get_input_directory(),"n-suite")
output_dir = os.path.join(folder_paths.get_output_directory(),"n-suite") output_dir = os.path.join(folder_paths.get_output_directory(),"n-suite","frames_out")
frames_output_dir = os.path.join(folder_paths.get_output_directory(),"frames") temp_output_dir = os.path.join(folder_paths.get_temp_directory(),"n-suite","frames_out")
videos_output_dir = os.path.join(folder_paths.get_output_directory(),"videos") frames_output_dir = os.path.join(folder_paths.get_temp_directory(),"n-suite","frames")
videos_output_dir = os.path.join(folder_paths.get_output_directory(),"n-suite","videos")
audios_output_temp_dir = os.path.join(folder_paths.get_temp_directory(),"audio.mp3") audios_output_temp_dir = os.path.join(folder_paths.get_temp_directory(),"audio.mp3")
videos_output_temp_dir = os.path.join(folder_paths.get_temp_directory(),"video.mp4") videos_output_temp_dir = os.path.join(folder_paths.get_temp_directory(),"video.mp4")
video_preview_output_temp_dir = os.path.join(folder_paths.get_output_directory(),"videos") video_preview_output_temp_dir = os.path.join(folder_paths.get_output_directory(),"n-suite","videos")
_resize_type = ["none","width", "height"] _resize_type = ["none","width", "height"]
_framerate = ["original","half", "quarter"] _framerate = ["original","half", "quarter"]
_choice = ["Yes", "No"] _choice = ["Yes", "No"]
try: try:
os.mkdir(input_dir) os.makedirs(input_dir)
except:
pass
try:
os.makedirs(output_dir)
except: except:
pass pass
try: try:
os.mkdir(videos_output_dir) os.makedirs(temp_output_dir)
except: except:
pass pass
try: try:
os.mkdir(frames_output_dir) os.makedirs(videos_output_dir)
except: except:
pass pass
try: try:
os.mkdir(folder_paths.get_temp_directory()) os.makedirs(frames_output_dir)
except:
pass
try:
os.makedirs(folder_paths.get_temp_directory())
except: except:
pass pass
@@ -142,25 +159,33 @@ def calc_resize_image(input_path, target_size, resize_by):
height, width = image.shape[:2] height, width = image.shape[:2]
if resize_by == 'width': if resize_by == 'width':
new_width = target_size new_width = target_size
new_height = int(height * (target_size / width)) new_height = int(height * (target_size / width))
elif resize_by == 'height': elif resize_by == 'height':
new_height = target_size new_height = target_size
new_width = int(width * (target_size / height)) new_width = int(width * (target_size / height))
else: else:
new_height = height new_height = height
new_width = width new_width = width
return new_width, new_height return new_width, new_height
def calc_resize_image_from_ram(input_frame, target_size, resize_by):
height, width = input_frame.shape[:2]
if resize_by == 'width':
new_width = target_size
new_height = int(height * (target_size / width))
elif resize_by == 'height':
new_height = target_size
new_width = int(width * (target_size / height))
else:
new_height = height
new_width = width
return new_width, new_height
def resize_image(input_path, new_width, new_height): def resize_image(input_path, new_width, new_height):
image = cv2.imread(input_path) image = cv2.imread(input_path)
height, width = image.shape[:2] height, width = image.shape[:2]
@@ -169,37 +194,23 @@ def resize_image(input_path, new_width, new_height):
else: else:
resized_image = image resized_image = image
pil_image = Image.fromarray(cv2.cvtColor(resized_image, cv2.COLOR_BGR2RGB)) pil_image = Image.fromarray(cv2.cvtColor(resized_image, cv2.COLOR_BGR2RGB))
return pil_image return pil_image
""" def extract_frames_from_video(video_path, output_folder): def resize_image_from_ram(image, new_width, new_height):
height, width = image.shape[:2]
if height != new_height or width != new_width:
resized_image = cv2.resize(image, (new_width, new_height))
else:
resized_image = image
pil_image = Image.fromarray(cv2.cvtColor(resized_image, cv2.COLOR_BGR2RGB))
return pil_image
def extract_frames_from_video(video_path, output_folder=None, target_fps=30, use_ram=True):
frames = []
list_files = [] list_files = []
os.makedirs(output_folder, exist_ok=True)
cap = cv2.VideoCapture(video_path)
frame_count = 0
while True:
ret, frame = cap.read()
if not ret:
break
frame_count += 1
frame_filename = os.path.join(output_folder, f"frame_{frame_count:04d}.png")
list_files.append(frame_filename)
cv2.imwrite(frame_filename, frame)
cap.release()
print(f"{frame_count} frames have been extracted from the video and saved in {output_folder}")
return list_files """
def extract_frames_from_video(video_path, output_folder, target_fps=30):
list_files = []
os.makedirs(output_folder, exist_ok=True)
cap = cv2.VideoCapture(video_path) cap = cv2.VideoCapture(video_path)
frame_count = 0 frame_count = 0
@@ -208,6 +219,13 @@ def extract_frames_from_video(video_path, output_folder, target_fps=30):
# Calcola il rapporto per ridurre il framerate # Calcola il rapporto per ridurre il framerate
frame_skip_ratio = original_fps // target_fps frame_skip_ratio = original_fps // target_fps
real_frame_count = 0 real_frame_count = 0
if not use_ram:
if output_folder is None:
raise ValueError("output_folder must be specified if use_ram is False")
if output_folder is not None:
os.makedirs(output_folder, exist_ok=True)
while True: while True:
ret, frame = cap.read() ret, frame = cap.read()
if not ret: if not ret:
@@ -217,40 +235,60 @@ def extract_frames_from_video(video_path, output_folder, target_fps=30):
# Estrai solo ogni "frame_skip_ratio"-esimo fotogramma # Estrai solo ogni "frame_skip_ratio"-esimo fotogramma
if frame_count % frame_skip_ratio == 0: if frame_count % frame_skip_ratio == 0:
frame_filename = os.path.join(output_folder, f"frame_{frame_count:04d}.png") if use_ram:
list_files.append(frame_filename) frames.append(frame)
cv2.imwrite(frame_filename, frame) else:
frame_filename = os.path.join(output_folder, f"{frame_count:07d}.png")
list_files.append(frame_filename)
cv2.imwrite(frame_filename, frame)
real_frame_count += 1 real_frame_count += 1
cap.release() cap.release()
print(f"{real_frame_count} frames have been extracted from the video and saved in {output_folder}") print(f"{real_frame_count} frames have been extracted from the video")
return list_files
if use_ram:
return frames
else:
return list_files
def extract_frames_from_gif(gif_path, output_folder, target_fps=30):
def extract_frames_from_gif(gif_path, output_folder):
list_files = [] list_files = []
os.makedirs(output_folder, exist_ok=True) os.makedirs(output_folder, exist_ok=True)
real_frame_count = 0
metadata = imageio.v3.immeta(gif_path)
gif_frames = imageio.mimread(gif_path) gif_frames = imageio.mimread(gif_path, memtest=False)
original_fps = len(gif_frames)
frame_skip_ratio = original_fps // original_fps
frame_count = 0 frame_count = 0
for frame in gif_frames: for frame in gif_frames:
frame_count += 1 frame_count += 1
frame_filename = os.path.join(output_folder, f"{frame_count:07d}.png")
if frame_count % frame_skip_ratio == 0: list_files.append(frame_filename)
frame_filename = os.path.join(output_folder, f"frame_{frame_count:04d}.png") cv2.imwrite(frame_filename, cv2.cvtColor(frame, cv2.COLOR_RGB2BGR))
cv2.imwrite(frame_filename, cv2.cvtColor(frame, cv2.COLOR_RGB2BGR))
list_files.append(frame_filename)
real_frame_count += 1
print(f"{frame_count} frames have been extracted from the GIF and saved in {output_folder}") print(f"{frame_count} frames have been extracted from the GIF and saved in {output_folder}")
return list_files,metadata return list_files
def get_output_filename(input_file_path, output_folder, file_extension,suffix="") : def get_output_filename(input_file_path, output_folder, file_extension,suffix="") :
existing_files = [f for f in os.listdir(output_folder)]
max_progressive = 0
for filename in existing_files:
parts_ext = filename.split(".")
parts = parts_ext[0]
if len(parts) > 2 and parts.isdigit():
progressive = int(parts)
max_progressive = max(max_progressive, progressive)
new_progressive = max_progressive + 1
new_filename = f"{new_progressive:07d}{suffix}{file_extension}"
return os.path.join(output_folder, new_filename), new_filename
def get_output_filename_video(input_file_path, output_folder, file_extension,suffix="") :
input_filename = os.path.basename(input_file_path) input_filename = os.path.basename(input_file_path)
input_filename_without_extension = os.path.splitext(input_filename)[0] input_filename_without_extension = os.path.splitext(input_filename)[0]
@@ -280,7 +318,6 @@ def image_preprocessing(i):
def create_video_from_frames(frame_folder, output_video, frame_rate = 30.0): def create_video_from_frames(frame_folder, output_video, frame_rate = 30.0):
frame_filenames = [os.path.join(frame_folder, filename) for filename in os.listdir(frame_folder) if filename.endswith(".png")] frame_filenames = [os.path.join(frame_folder, filename) for filename in os.listdir(frame_folder) if filename.endswith(".png")]
frame_filenames.sort(key=lambda f: int(''.join(filter(str.isdigit, f)))) frame_filenames.sort(key=lambda f: int(''.join(filter(str.isdigit, f))))
first_frame = cv2.imread(frame_filenames[0]) first_frame = cv2.imread(frame_filenames[0])
height, width, layers = first_frame.shape height, width, layers = first_frame.shape
@@ -294,14 +331,14 @@ def create_video_from_frames(frame_folder, output_video, frame_rate = 30.0):
out.release() out.release()
print(f"Frames have been successfully reassembled into {output_video}") print(f"Frames have been successfully reassembled into {output_video}")
def create_gif_from_frames(frame_folder, output_gif, metadata): def create_gif_from_frames(frame_folder, output_gif):
frame_filenames = [os.path.join(frame_folder, filename) for filename in os.listdir(frame_folder) if filename.endswith(".png")] frame_filenames = [os.path.join(frame_folder, filename) for filename in os.listdir(frame_folder) if filename.endswith(".png")]
frame_filenames.sort() frame_filenames.sort()
frames = [imageio.imread(frame_filename) for frame_filename in frame_filenames] frames = [imageio.imread(frame_filename) for frame_filename in frame_filenames]
# imageio # imageio
imageio.mimsave(output_gif, frames, loop=metadata[3], duration=metadata[4]) imageio.mimsave(output_gif, frames, duration=0.1)
print(f"Frames have been successfully assembled into {output_gif}") print(f"Frames have been successfully assembled into {output_gif}")
@@ -310,29 +347,33 @@ def create_gif_from_frames(frame_folder, output_gif, metadata):
temp_dir= folder_paths.temp_directory temp_dir= folder_paths.temp_directory
class VideoLoader: class LoadVideoAdvanced:
def __init__(self): def __init__(self):
pass pass
@classmethod @classmethod
def INPUT_TYPES(s): def INPUT_TYPES(s):
files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))] files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))]
return {"required": {"video": (sorted(files), {"image_upload": True} ), return {"required": {"video": (sorted(files), ),
"local_url": ("STRING", {"default": ""} ), "local_url": ("STRING", {"default": ""} ),
"framerate": (_framerate, {"default": "original"} ), "framerate": (_framerate, {"default": "original"} ),
"resize_by": (_resize_type,{"default": "none"} ), "resize_by": (_resize_type,{"default": "none"} ),
"size": ("INT", {"default": 512, "min": 512, "step": 64}), "size": ("INT", {"default": 512, "min": 512, "step": 64}),
"images_limit": ("INT", {"default": 0, "min": 0, "step": 1}), "images_limit": ("INT", {"default": 0, "min": 0, "step": 1}),
"batch_size": ("INT", {"default": 0, "min": 0, "step": 1}) "batch_size": ("INT", {"default": 0, "min": 0, "step": 1}),
"starting_frame": ("INT", {"default": 0, "min": 0, "step": 1}),
"autoplay":("BOOLEAN",{"default": True} ),
"use_ram": ("BOOLEAN", {"default": False}),
},} },}
RETURN_TYPES = ("IMAGE","LATENT","STRING","INT","INT",) RETURN_TYPES = ("IMAGE","LATENT","STRING","INT","INT","INT","INT",)
OUTPUT_IS_LIST = (True, True, False, False,False, ) OUTPUT_IS_LIST = (True, True, False, False,False,False,False, )
RETURN_NAMES = ("IMAGES","EMPTY LATENT","METADATA","WIDTH","HEIGHT") RETURN_NAMES = ("IMAGES","EMPTY LATENTS","METADATA","WIDTH","HEIGHT","META_FPS","META_N_FRAMES")
CATEGORY = "video" CATEGORY = "N-Suite/Video"
FUNCTION = "encode" FUNCTION = "encode"
TYPE="N-Suite"
@staticmethod @staticmethod
@@ -345,9 +386,8 @@ class VideoLoader:
pixels = pixels[:, x_offset:x + x_offset, y_offset:y + y_offset, :] pixels = pixels[:, x_offset:x + x_offset, y_offset:y + y_offset, :]
return pixels return pixels
def load_video(self, video,framerate, local_url): def load_video(self, video, framerate, local_url, use_ram):
file_path = folder_paths.get_annotated_filepath(os.path.join("n-suite", video))
file_path = folder_paths.get_annotated_filepath(os.path.join("n-suite",video))
cap = cv2.VideoCapture(file_path) cap = cv2.VideoCapture(file_path)
# Check if the video was opened successfully # Check if the video was opened successfully
if not cap.isOpened(): if not cap.isOpened():
@@ -357,129 +397,139 @@ class VideoLoader:
fps = int(cap.get(cv2.CAP_PROP_FPS)) fps = int(cap.get(cv2.CAP_PROP_FPS))
print(f"The video has {fps} frames per second.") print(f"The video has {fps} frames per second.")
#shutil.rmtree(output_dir) try:
#print(f"Temporary folder {output_dir} has been emptied.") shutil.rmtree(os.path.join(temp_output_dir, video.split(".")[0]))
except:
print("Video Path already deleted")
#set new framerate full_temp_output_dir = os.path.join(temp_output_dir, video.split(".")[0])
# Set new framerate
if "half" in framerate: if "half" in framerate:
fps = fps // 2 fps = fps // 2
print (f"The video has been reduced to {fps} frames per second.") print(f"The video has been reduced to {fps} frames per second.")
elif "quarter" in framerate: elif "quarter" in framerate:
fps = fps // 4 fps = fps // 4
print (f"The video has been reduced to {fps} frames per second.") print(f"The video has been reduced to {fps} frames per second.")
# Estract frames
file_extension = os.path.splitext(file_path)[1].lower() file_extension = os.path.splitext(file_path)[1].lower()
if file_extension == ".mp4": if file_extension in [".mp4", ".webm"]:
list_files = extract_frames_from_video(file_path, output_dir, target_fps=fps) list_files = extract_frames_from_video(file_path, full_temp_output_dir, fps, use_ram)
meta = {"loop": 0, "duration": 0}
audio_clip = VideoFileClip(file_path).audio audio_clip = VideoFileClip(file_path).audio
try: try:
#save audio # Save audio
audio_clip.write_audiofile(audios_output_temp_dir) audio_clip.write_audiofile(os.path.join(temp_output_dir, video.split(".")[0], "audio.mp3"))
except: except:
print("Could not save audio")
pass pass
elif file_extension == ".gif": elif file_extension == ".gif":
list_files,meta = extract_frames_from_gif(file_path, output_dir) list_files = extract_frames_from_gif(file_path, output_dir)
#create_gif_from_frames(output_dir, output_video2)
else: else:
print("Format not supported. Please provide an MP4 or GIF file.") print("Format not supported. Please provide an MP4 or GIF file.")
return list_files, fps
return list_files,fps,file_extension,meta["loop"],meta["duration"]
def generate_latent(self, width, height, batch_size=1): def generate_latent(self, width, height, batch_size=1):
latent = torch.zeros([batch_size, 4, height // 8, width // 8]) latent = torch.zeros([batch_size, 4, height // 8, width // 8])
return {"samples":latent} return {"samples": latent}
def process_image(self,args): def process_image(self, args):
image_path, width, height = args image, width, height, use_ram = args
# Funzione per ridimensionare e pre-elaborare un'immagine # Funzione per ridimensionare e pre-elaborare un'immagine
image = resize_image(image_path, width, height) if use_ram:
image = resize_image_from_ram(image, width, height)
else:
image = resize_image(image, width, height)
image = image_preprocessing(image) image = image_preprocessing(image)
return torch.tensor(image) return torch.tensor(image)
def encode(self,video,framerate, local_url, resize_by, size, images_limit,batch_size): def encode(self, video, framerate, local_url, resize_by, size, images_limit, batch_size, starting_frame, autoplay, use_ram):
metadata = [] metadata = []
FRAMES,fps,file_extension,loop,duration = self.load_video(video,framerate, local_url) FRAMES, fps = self.load_video(video, framerate, local_url, use_ram)
pool_size=5 max_frames = len(FRAMES)
t_list = []
if images_limit > 0 and starting_frame > 0:
images_limit += starting_frame
print(f"images_limit {images_limit}")
if starting_frame > max_frames:
starting_frame = max_frames - 1
print(f"WARNING: The starting frame is greater than the number of frames in the video. Only the last frame of the video will be used ({starting_frame}).")
if images_limit > max_frames:
images_limit = max_frames
print(f"WARNING: The number of images to extract is greater than the number of frames in the video. Images_limit has been reduced to the number of frames ({images_limit}).")
if batch_size > max_frames:
print(f"WARNING: The batch size is greater than the number of frames requested. Batch size has been reduced.")
batch_size = max_frames
if images_limit != 0 and batch_size > images_limit:
print(f"WARNING: The batch size is greater than the number of frames requested. Batch size has been reduced to the number of images_limit.")
batch_size = images_limit
pool_size = 5
i_list = [] i_list = []
i = 0 final_count_frame = 0
o = 0
with concurrent.futures.ThreadPoolExecutor() as executor: with concurrent.futures.ThreadPoolExecutor() as executor:
futures = [] futures = []
width, height = calc_resize_image(FRAMES[0], size, resize_by) if use_ram:
width, height = calc_resize_image_from_ram(FRAMES[0], size, resize_by)
else:
width, height = calc_resize_image(FRAMES[0], size, resize_by)
for batch_start in range(0, len(FRAMES), pool_size): for batch_start in range(0, len(FRAMES), pool_size):
batch_images = FRAMES[batch_start:batch_start + pool_size] batch_images = FRAMES[batch_start:batch_start + pool_size]
#remove audio if image_limit > 0
if images_limit != 0: if images_limit != 0 or starting_frame != 0:
try: try:
os.remove(audios_output_temp_dir) os.remove(os.path.join(temp_output_dir, video.split(".")[0], "audio.mp3"))
except: except:
pass pass
if o >= images_limit: for idx, image in enumerate(batch_images):
break if final_count_frame >= starting_frame and (final_count_frame < images_limit or images_limit == 0):
args = (image, width, height, use_ram)
futures.append(executor.submit(self.process_image, args))
final_count_frame += 1
for image_path in batch_images:
args = (image_path, width, height)
futures.append(executor.submit(self.process_image, args))
o += 1
if images_limit != 0:
if o >= images_limit:
break
i += len(batch_images)
# Attendi il completamento delle operazioni in parallelo
concurrent.futures.wait(futures) concurrent.futures.wait(futures)
# Recupera i risultati
for future in futures: for future in futures:
batch_i_tensors = future.result() batch_i_tensors = future.result()
i_list.extend(batch_i_tensors) i_list.extend(batch_i_tensors)
i_tensor = torch.stack(i_list, dim=0) i_tensor = torch.stack(i_list, dim=0)
if images_limit != 0 or starting_frame != 0:
if images_limit != 0: b_size = final_count_frame
b_size=images_limit
else: else:
b_size=len(FRAMES) b_size = len(FRAMES)
latent = self.generate_latent(width, height, batch_size=b_size)
latent = self.generate_latent( width, height, batch_size=b_size)
metadata.append(fps) metadata.append(fps)
metadata.append(b_size) metadata.append(b_size)
metadata.append(file_extension) try:
metadata.append(loop) metadata.append(video.split(".")[0])
metadata.append(duration) except:
print("No video name")
if batch_size != 0: if batch_size != 0:
rebatcher = LatentRebatch() rebatcher = LatentRebatch()
rebatched_latent = rebatcher.rebatch([latent], [batch_size]) rebatched_latent = rebatcher.rebatch([latent], [batch_size])
n_chunks = b_size//batch_size n_chunks = b_size // batch_size
i_tensor_batches = torch.chunk(i_tensor, n_chunks, dim=0) i_tensor_batches = torch.chunk(i_tensor, n_chunks, dim=0)
return i_tensor_batches, rebatched_latent, metadata, width, height
return (i_tensor_batches,rebatched_latent,metadata, width, height,) return [i_tensor], [latent], metadata, width, height, fps, b_size
return ( [i_tensor],[latent],metadata, width, height,)
class VideoSaver: class SaveVideo:
def __init__(self): def __init__(self):
self.type = "output" self.type = "output"
@@ -487,8 +537,6 @@ class VideoSaver:
@classmethod @classmethod
def INPUT_TYPES(s): def INPUT_TYPES(s):
s.video_file_path,s.video_filename = get_output_filename("video", videos_output_dir, ".mp4")
s.gif_file_path,s.gif_filename = get_output_filename("gif", videos_output_dir, ".gif")
try: try:
shutil.rmtree(frames_output_dir) shutil.rmtree(frames_output_dir)
@@ -497,11 +545,15 @@ class VideoSaver:
pass pass
print(f"Temporary folder {frames_output_dir} has been emptied.") #print(f"Temporary folder {frames_output_dir} has been emptied.")
return {"required": return {"required":
{"images": ("IMAGE", ), {"images": ("IMAGE", ),
"METADATA": ("STRING", {"default": "", "forceInput": True} ), "METADATA": ("STRING", {"default": "", "forceInput": True} ),
"SaveVideo": (_choice,{"default": "No"} ), "SaveVideo": ("BOOLEAN",{"default": False} ),
"SaveFrames": ("BOOLEAN",{"default": False} ),
"filename_prefix": ("STRING",{"default": "video"} ),
"CompressionLevel": ("INT", {"default": 2, "min": 0, "max":9, "step": 1}),
}, },
"hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"}, "hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"},
} }
@@ -512,77 +564,71 @@ class VideoSaver:
OUTPUT_NODE = True OUTPUT_NODE = True
CATEGORY = "video" CATEGORY = "N-Suite/Video"
def save_video(self, images,METADATA,SaveVideo, prompt=None, extra_pnginfo=None): def save_video(self, images,METADATA,SaveVideo,SaveFrames,filename_prefix, CompressionLevel, prompt=None, extra_pnginfo=None):
self.video_file_path,self.video_filename = get_output_filename_video(filename_prefix, videos_output_dir, ".mp4")
fps = METADATA[0] fps = METADATA[0]
frame_number = METADATA[1] frame_number = METADATA[1]
file_extension = METADATA[2] video_filename_original = METADATA[2]
results = list()
#full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path("", frames_output_dir, images[0].shape[1], images[0].shape[0]) #full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path("", frames_output_dir, images[0].shape[1], images[0].shape[0])
results = list() results = list()
for image in images: for image in images:
full_output_folder,file = get_output_filename("frame", frames_output_dir, ".png") full_output_folder,file = get_output_filename("", frames_output_dir, ".png")
file_name = file
i = 255. * image.cpu().numpy() i = 255. * image.cpu().numpy()
img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8)) img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
metadata = None metadata = None
#file = f"frame_{counter:05}_.png" #file = f"frame_{counter:05}_.png"
img.save(full_output_folder, pnginfo=metadata, compress_level=4) img.save(full_output_folder, pnginfo=metadata, compress_level=CompressionLevel)
results.append({ results.append({
"filename": file, "filename": file,
"subfolder": "frames", "subfolder": "frames",
"type": self.type "type": self.type
}) })
try: try:
file_name_number = int(file.split(".")[0].split("_")[1]) file_name_number = int(file.split(".")[0])
except: except:
file_name_number = 0 file_name_number = 0
if(file_name_number >= frame_number): if(file_name_number >= frame_number):
if file_extension == ".mp4": create_video_from_frames(frames_output_dir, videos_output_temp_dir,frame_rate=fps)
create_video_from_frames(frames_output_dir, videos_output_temp_dir,frame_rate=fps)
video_clip = VideoFileClip(videos_output_temp_dir) video_clip = VideoFileClip(videos_output_temp_dir)
try: try:
audio_clip = AudioFileClip(audios_output_temp_dir) audio_clip = AudioFileClip(os.path.join(temp_output_dir,video_filename_original,"audio.mp3"))
video_clip = video_clip.set_audio(audio_clip) video_clip = video_clip.set_audio(audio_clip)
except: except:
pass print("No audio found")
if SaveVideo == "Yes": pass
video_clip.write_videofile(self.video_file_path)
file_name = self.video_filename
else:
#delete all temporary files that start with video_preview
for file in os.listdir(video_preview_output_temp_dir):
if file.startswith("video_preview"):
os.remove(os.path.join(video_preview_output_temp_dir,file))
#random number
suffix = str(random.randint(1,100000))
file_name = f"video_preview_{suffix}.mp4"
video_clip.write_videofile(os.path.join(video_preview_output_temp_dir,file_name))
elif file_extension == ".gif":
if SaveVideo == "Yes": if SaveFrames == True:
create_gif_from_frames(frames_output_dir, os.path.join(video_preview_output_temp_dir,self.gif_filename),METADATA) #copy frames_output_dir to self.video_file_path/self.video_filename
file_name = self.gif_filename frame_folder=os.path.join(videos_output_dir,self.video_filename.split(".")[0])
else:
#delete all temporary files that start with video_preview
for file in os.listdir(video_preview_output_temp_dir):
if file.startswith("gif_preview"):
os.remove(os.path.join(video_preview_output_temp_dir,file))
#random number
suffix = str(random.randint(1,100000))
file_name = f"gif_preview_{suffix}.gif"
create_gif_from_frames(frames_output_dir,os.path.join(video_preview_output_temp_dir,file_name),METADATA)
shutil.copytree(frames_output_dir, frame_folder)
if SaveVideo == True:
video_clip.write_videofile(self.video_file_path)
file_name = self.video_filename
else:
#delete all temporary files that start with video_preview
for file in os.listdir(video_preview_output_temp_dir):
if file.startswith("video_preview"):
os.remove(os.path.join(video_preview_output_temp_dir,file))
#random number
suffix = str(random.randint(1,100000))
file_name = f"video_preview_{suffix}.mp4"
video_clip.write_videofile(os.path.join(video_preview_output_temp_dir,file_name))
@@ -597,47 +643,83 @@ class LoadFramesFromFolder:
@classmethod @classmethod
def INPUT_TYPES(s): def INPUT_TYPES(s):
return {"required": { "folder":("STRING", {"default": ""} ), return {"required": { "folder":("STRING", {"default": ""} ),
"fps":("INT", {"default": 30}), "fps":("INT", {"default": 30})
"loop_for_gif": ("INT", {"default": 0}), }}
"duration_for_gif":("INT", {"default": 0}),
RETURN_TYPES = ("IMAGE","STRING","INT","INT","INT","STRING","STRING",)
RETURN_NAMES = ("IMAGES","METADATA","MAX WIDTH","MAX HEIGHT","FRAME COUNT","PATH","IMAGE LIST")
FUNCTION = "load_images"
OUTPUT_IS_LIST = (True,False,False,False,False,False,False,)
CATEGORY = "N-Suite/Video"
def load_images(self, folder,fps):
image_list = []
image_names = []
max_width = 0
max_height = 0
frame_count = 0
METADATA = [fps, len(os.listdir(folder)),"load"]
images = [os.path.join(folder, filename) for filename in os.listdir(folder) if filename.endswith(".png") or filename.endswith(".jpg")]
images.sort(key=lambda f: int(''.join(filter(str.isdigit, f))))
for image_path in images:
#get image name
image_names.append(image_path.split("/")[-1])
image = Image.open(image_path)
width, height = image.size
max_width = max(max_width, width)
max_height = max(max_height, height)
image_list.append((image_preprocessing(image)))
frame_count += 1
image_names_final='\n'.join(image_names)
print (f"Details: {frame_count} frames, {max_width}x{max_height}")
return (image_list,METADATA, max_width, max_height,frame_count,folder,image_names_final,)
class SetMetadata:
def __init__(self):
pass
@classmethod
def INPUT_TYPES(s):
return {"required": { "number_of_frames":("INT", {"default": 1, "min": 1, "step": 1}),
"fps":("INT", {"default": 30, "min": 1, "step": 1}),
"VideoName": ("STRING", {"default": "manual"} )
}} }}
RETURN_TYPES = ("IMAGE","STRING",) RETURN_TYPES = ("STRING",)
RETURN_NAMES = ("IMAGES","METADATA") RETURN_NAMES = ("METADATA",)
FUNCTION = "set_metadata"
OUTPUT_IS_LIST = (False,)
CATEGORY = "N-Suite/Video"
FUNCTION = "load_images" def set_metadata(self, number_of_frames,fps,VideoName):
OUTPUT_IS_LIST = (True,False,)
CATEGORY = "video"
def load_images(self, folder,fps,loop_for_gif,duration_for_gif): METADATA = [fps, number_of_frames,VideoName]
image_list = [] return (METADATA,)
METADATA = [fps, len(os.listdir(folder)),loop_for_gif,duration_for_gif]
images = [os.path.join(folder, filename) for filename in os.listdir(folder) if filename.endswith(".png")]
images.sort(key=lambda f: int(''.join(filter(str.isdigit, f))))
for image in images:
image_list.append(image_preprocessing(Image.open(image)))
#i_tensor = torch.stack(image_list, dim=0)
return (image_list,METADATA,)
# A dictionary that contains all nodes you want to export with their names # A dictionary that contains all nodes you want to export with their names
# NOTE: names should be globally unique # NOTE: names should be globally unique
NODE_CLASS_MAPPINGS = { NODE_CLASS_MAPPINGS = {
"VideoLoader": VideoLoader, "LoadVideo [n-suite]": LoadVideoAdvanced,
"VideoSaver":VideoSaver, "SaveVideo [n-suite]":SaveVideo,
"LoadFramesFromFolder": LoadFramesFromFolder "LoadFramesFromFolder [n-suite]": LoadFramesFromFolder,
"SetMetadataForSaveVideo [n-suite]": SetMetadata
} }
# A dictionary that contains the friendly/humanly readable titles for the nodes # A dictionary that contains the friendly/humanly readable titles for the nodes
NODE_DISPLAY_NAME_MAPPINGS = { NODE_DISPLAY_NAME_MAPPINGS = {
"Video": "VideoLoader", "LoadVideo [n-suite]": "LoadVideo [🅝-🅢🅤🅘🅣🅔]",
"Video": "VideoSaver", "SaveVideo [n-suite]": "SaveVideo [🅝-🅢🅤🅘🅣🅔]",
"Video": "LoadFramesFromFolder" "LoadFramesFromFolder [n-suite]": "LoadFramesFromFolder [🅝-🅢🅤🅘🅣🅔]",
"SetMetadataForSaveVideo [n-suite]": "SetMetadataForSaveVideo [🅝-🅢🅤🅘🅣🅔]"
} }
+23
View File
@@ -0,0 +1,23 @@
[project]
name = "comfyui-n-nodes"
description = "A suite of custom nodes for ComfyUI that includes image captioning, LoadVideo, SaveVideo, LoadFramesFromFolder and FrameInterpolator"
version = "1.2.0"
license = { file = "LICENSE" }
dependencies = [
"gitpython",
"huggingface-hub",
"moviepy<2",
"opencv-python",
"scikit-build",
"timm==0.9.12",
"transformers==4.36.2",
]
[project.urls]
Repository = "https://github.com/Nuked88/ComfyUI-N-Nodes"
# Used by Comfy Registry https://comfyregistry.org
[tool.comfy]
PublisherId = "nuked"
DisplayName = "ComfyUI-N-Nodes"
Icon = ""
+3
View File
@@ -0,0 +1,3 @@
[pytest]
testpaths = tests
addopts = --import-mode=importlib
+7
View File
@@ -0,0 +1,7 @@
gitpython
huggingface-hub
moviepy<2
opencv-python
scikit-build
timm==0.9.12
transformers==4.36.2
+30
View File
@@ -0,0 +1,30 @@
import importlib.util
from pathlib import Path
MODULE_PATH = Path(__file__).resolve().parents[1] / "py" / "dynamic_prompt_node.py"
SPEC = importlib.util.spec_from_file_location("dynamic_prompt_node", MODULE_PATH)
MODULE = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(MODULE)
DynamicPrompt = MODULE.DynamicPrompt
def generate(variable, mode="Fixed", count=1, fixed=""):
return DynamicPrompt().prompt_generator(variable, "NO", mode, count, fixed)["result"][0]
def test_fixed_prompt_is_combined_with_requested_number_of_unique_tags():
result = generate("red, green, blue", count=2, fixed="portrait")
parts = result.split(",")
assert parts[0] == "portrait"
assert len(parts[1:]) == 2
assert len(set(parts[1:])) == 2
assert set(parts[1:]) <= {"red", "green", "blue"}
def test_requested_tag_count_is_capped_to_available_tags():
assert set(generate("red,blue", count=20).split(",")) == {"red", "blue"}
def test_empty_variable_prompt_returns_empty_result():
assert generate("", fixed="portrait") == ""
+82
View File
@@ -0,0 +1,82 @@
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
RUNTIME_FILES = [
ROOT / "__init__.py",
ROOT / "nnodes.py",
ROOT / "py" / "image_captioning_node.py",
ROOT / "requirements.txt",
ROOT / "pyproject.toml",
]
def test_runtime_has_no_llama_cpp_dependency():
for path in RUNTIME_FILES:
assert "llama_cpp" not in path.read_text(), path
def test_llava_node_is_no_longer_registered():
source = (ROOT / "py" / "image_captioning_node.py").read_text()
assert '"Llava Clip Loader [n-suite]"' not in source
assert "Llava15ChatHandler" not in source
def test_readme_announces_breaking_change_and_model_downloads():
readme = (ROOT / "README.md").read_text()
assert "Breaking change in 1.2.0" in readme
assert "was never downloaded automatically" in readme
assert "first time Moondream is loaded" in readme
assert "first time JoyTag is loaded" in readme
assert "git checkout ae7cc84" in readme
assert "old LLaVA implementation was not independent" in readme
def test_dependencies_are_manager_installable_and_moviepy_is_compatible():
requirements = (ROOT / "requirements.txt").read_text().splitlines()
pyproject = (ROOT / "pyproject.toml").read_text()
assert "moviepy<2" in requirements
assert '"moviepy<2"' in pyproject
assert "huggingface-hub" in requirements
assert "transformers==4.36.2" in requirements
assert "timm==0.9.12" in requirements
def test_extension_does_not_install_packages_during_import():
bootstrap = (ROOT / "__init__.py").read_text()
assert "check_and_install" not in bootstrap
def test_frontend_uses_managed_dom_widgets():
widgets = (ROOT / "js" / "extended_widgets.js").read_text()
assert "addDOMWidget" in widgets
assert "addCustomWidget" not in widgets
assert "onDrawBackground" not in widgets
assert "graph._nodes" not in widgets
def test_dynamic_widgets_use_current_removal_api_and_node_id():
dynamic_prompt = (ROOT / "js" / "dynamicPrompt.js").read_text()
gpt_sampler = (ROOT / "js" / "gptSampler.js").read_text()
assert 'nodeData.name !== "DynamicPrompt [n-suite]"' in dynamic_prompt
assert "removeWidget(widget)" in dynamic_prompt
assert "removeWidget(widget)" in gpt_sampler
def test_external_repositories_and_model_archive_are_pinned():
bootstrap = (ROOT / "__init__.py").read_text()
assert 'RIFE_REVISION = "a8a8035323b1c1a4a20753c751780e5b0a879455"' in bootstrap
assert 'MOONDREAM_REVISION = "38af98596e59f2a6c25c6b52b2bd5a672dab4144"' in bootstrap
assert 'RIFE_MODEL_REVISION = "572480112b87f9bfbff7579b8a38b483766e455f"' in bootstrap
assert "/raw/main/RIFE_trained_model" not in bootstrap
assert "repo.git.checkout(revision)" in bootstrap
def test_registry_publish_is_manual_after_nightly_validation():
workflow = (ROOT / ".github" / "workflows" / "publish.yml").read_text()
assert "workflow_dispatch:" in workflow
assert "push:" not in workflow
readme = (ROOT / "README.md").read_text()
assert "Testing this release through Manager Nightly" in readme
assert "a pull request cannot be selected as nightly while it is still unmerged" in readme