Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1803f7ca5f |
@@ -1,3 +0,0 @@
|
||||
# Development-only files are not needed in the Registry package.
|
||||
tests/
|
||||
pytest.ini
|
||||
@@ -1,37 +0,0 @@
|
||||
---
|
||||
name: Bug report
|
||||
about: Create a report to help us improve
|
||||
title: "[BUG]"
|
||||
labels: bug
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
**Describe the bug**
|
||||
A clear and concise description of what the bug is.
|
||||
|
||||
**To Reproduce**
|
||||
Steps to reproduce the behavior:
|
||||
1. Go to '...'
|
||||
2. Click on '....'
|
||||
3. Scroll down to '....'
|
||||
4. See error
|
||||
|
||||
**Expected behavior**
|
||||
A clear and concise description of what you expected to happen.
|
||||
|
||||
**Full log**
|
||||
This is MANDATORY. By log I mean all the text in the console from the time ComfyUI was started until the time of the reported bug.
|
||||
>>Bug reports that do not have this log will be closed.<<
|
||||
|
||||
**Screenshots**
|
||||
If applicable, add screenshots to help explain your problem.
|
||||
|
||||
**Desktop (please complete the following information):**
|
||||
- OS: [e.g. iOS]
|
||||
- Browser [e.g. chrome, safari]
|
||||
- Version [e.g. 22]
|
||||
|
||||
|
||||
**Additional context**
|
||||
Add any other context about the problem here.
|
||||
@@ -1,20 +0,0 @@
|
||||
---
|
||||
name: Feature request
|
||||
about: Suggest an idea for this project
|
||||
title: ''
|
||||
labels: enhancement
|
||||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
**Is your feature request related to a problem? Please describe.**
|
||||
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
|
||||
|
||||
**Describe the solution you'd like**
|
||||
A clear and concise description of what you want to happen.
|
||||
|
||||
**Describe alternatives you've considered**
|
||||
A clear and concise description of any alternative solutions or features you've considered.
|
||||
|
||||
**Additional context**
|
||||
Add any other context or screenshots about the feature request here.
|
||||
@@ -1,24 +0,0 @@
|
||||
name: Publish to Comfy registry
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
# Merging into main makes the commit available as the Manager's "nightly"
|
||||
# version. Stable Registry releases are intentionally published only by
|
||||
# manually running this workflow after nightly validation.
|
||||
|
||||
permissions:
|
||||
issues: write
|
||||
|
||||
jobs:
|
||||
publish-node:
|
||||
name: Publish Custom Node to registry
|
||||
runs-on: ubuntu-latest
|
||||
if: ${{ github.repository_owner == 'Nuked88' }}
|
||||
steps:
|
||||
- name: Check out code
|
||||
uses: actions/checkout@v4
|
||||
- name: Publish Custom Node
|
||||
uses: Comfy-Org/publish-node-action@v1
|
||||
with:
|
||||
## Add your own personal access token to your Github Repository secrets and reference it here.
|
||||
personal_access_token: ${{ secrets.REGISTRY_ACCESS_TOKEN }}
|
||||
@@ -2,7 +2,7 @@
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
libs/moondream_repo
|
||||
|
||||
# C extensions
|
||||
*.so
|
||||
|
||||
|
||||
@@ -1,156 +1,30 @@
|
||||
[](https://ko-fi.com/C0C0AJECJ)
|
||||
|
||||
# ComfyUI-N-Suite
|
||||
A suite of custom nodes for ComfyUI that includes integer, string and float variable nodes, image-captioning nodes and video nodes.
|
||||
|
||||
The nodes support ComfyUI's Python environment on Windows and Linux. The current dependencies include MoviePy 2, timm 1.0.22 or newer, accelerate 1.x, and transformers 4.36.2 through 4.x.
|
||||
|
||||
# ComfyUI-N-Nodes
|
||||
A suite of custom nodes for ComfyUI, for now i just put Integer, string and float variable nodes
|
||||
|
||||
# Installation
|
||||
|
||||
1. Install **ComfyUI-N-Nodes** through ComfyUI Manager (recommended). For a manual install, clone `https://github.com/Nuked88/ComfyUI-N-Nodes.git` into ComfyUI's `custom_nodes` directory and run `python -m pip install -r requirements.txt` using the same Python environment that runs ComfyUI.
|
||||
2. Ensure `ComfyUI/models/GPTcheckpoints` is writable by the ComfyUI process so Moondream and JoyTag can download their models.
|
||||
3. Restart ComfyUI. The extension clones pinned RIFE code and downloads its pinned model at startup on a fresh install; an internet connection is needed for that first startup.
|
||||
1. Clone the repository:
|
||||
`git clone https://github.com/Nuked88/ComfyUI-N-Nodes.git`
|
||||
to your ComfyUI `custom_nodes` directory
|
||||
|
||||
ComfyUI automatically loads all custom scripts and nodes at startup.
|
||||
|
||||
> [!IMPORTANT]
|
||||
> **Breaking change in 1.2.0:** `llama-cpp-python` integration has been removed because its platform-specific installation was the main source of installation and startup failures. GGUF text-generation models and LLaVA nodes are therefore no longer available in N-Suite. Existing workflows using `Llava Clip Loader` must remove that node; `GPT Loader Simple` and `GPT Sampler` now support only Moondream and JoyTag. N-Suite no longer detects, downloads, or installs `llama-cpp-python`.
|
||||
|
||||
> [!WARNING]
|
||||
> **The `Llava Clip Loader` node and the GGUF text-generation path of `GPT Loader Simple` / `GPT Sampler` have been removed.** To keep using those legacy nodes, install the last revision that contains them with `git checkout ae7cc84`. That revision is unsupported and retains the `llama-cpp-python` installation problems; use it in a separate ComfyUI installation or Python environment.
|
||||
|
||||
> [!NOTE]
|
||||
> Since 14/02/2024, the node has undergone a massive rewrite, which also led to the change of all node names in order to avoid any conflicts with other extensions in the future (or at least I hope so). Consequently, the old workflows are no longer compatible and will require manual replacement of each node.
|
||||
> To avoid this, I have created a tool that allows for automatic replacement.
|
||||
> On Windows, simply drag any *.json workflow onto the migrate.bat file located in (custom_nodes/ComfyUI-N-Nodes), and another workflow with the suffix _migrated will be created in the same folder as the current workflow.
|
||||
> On Linux, you can use the script in the following way: python libs/migrate.py path/to/original/workflow/.
|
||||
> For security reasons, the original workflow will not be deleted."
|
||||
> For install the last version of this repository before this changes from the Comfyui-N-Suite execute **git checkout 29b2e43baba81ee556b2930b0ca0a9c978c47083**
|
||||
|
||||
|
||||
For uninstallation, remove the extension through ComfyUI Manager or delete its folder from `custom_nodes`, then restart ComfyUI. Model files in `models/GPTcheckpoints` are user data and can be kept for a later reinstall.
|
||||
2. **IMPORTANT**: For the GPT node you need to run **install_dependency bat file**.
|
||||
There are 2 versions: ***install_dependency_ggml_models.bat*** for the old ggmlv3 models and ***install_dependency_new_models.bat*** for all the new models (GGUF).
|
||||
YOU CAN ONLY USE ONE OF THEM AT A TIME!
|
||||
Since _llama-cpp-python_ needs to be compiled from source code to enable it to use the GPU, you will first need to have [CUDA](https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64) and visual studio 2019 or 2022 (in the case of my bat) installed to compile it. For details and the full guide you can go [HERE](https://github.com/abetlen/llama-cpp-python) . This bats are made for the official portable windows version of ComfyUI
|
||||
|
||||
ComfyUI will then automatically load all custom scripts and nodes at the start.
|
||||
|
||||
|
||||
- For uninstallation:
|
||||
- Delete the `ComfyUI-N-Nodes` folder in `custom_nodes`
|
||||
|
||||
|
||||
# Update
|
||||
Update through ComfyUI Manager. For a manual install, run `git pull` in the cloned extension directory, install `requirements.txt` again in ComfyUI's Python environment, and restart ComfyUI.
|
||||
|
||||
## Test workflow
|
||||
|
||||
[`examples/N-Suite-all-nodes-test.json`](examples/N-Suite-all-nodes-test.json) connects all 14 N-Suite node types in one workflow. Follow the [test instructions](examples/README.md) to add an image, a short MP4, and numbered PNG frames before running it.
|
||||
1. Navigate to the cloned repo e.g. `custom_nodes/ComfyUI-N-Nodes`
|
||||
2. `git pull`
|
||||
|
||||
# Features
|
||||
|
||||
## 📽️ Video Nodes 📽️
|
||||
|
||||
### LoadVideo
|
||||
|
||||

|
||||
|
||||
The LoadVideoAdvanced node allows loading a video file and extracting frames from it.
|
||||
The name has been changed from `LoadVideo` to `LoadVideoAdvanced` in order to avoid conflicts with the `LoadVideo` animatediff node.
|
||||
|
||||
|
||||
#### Input Fields
|
||||
- `video`: Select the video file to load.
|
||||
- `framerate`: Choose whether to keep the original framerate or reduce to half or quarter speed.
|
||||
- `resize_by`: Select how to resize frames - 'none', 'height', or 'width'.
|
||||
- `size`: Target size if resizing by height or width.
|
||||
- `images_limit`: Limit number of frames to extract.
|
||||
- `batch_size`: Batch size for encoding frames.
|
||||
- `starting_frame`: Select which frame to start from.
|
||||
- `autoplay`: Select whether to autoplay the video.
|
||||
- `use_ram`: Use RAM instead of disk for decompressing video frames.
|
||||
|
||||
#### Output
|
||||
|
||||
- `IMAGES`: Extracted frame images as PyTorch tensors.
|
||||
- `LATENT`: Empty latent vectors.
|
||||
- `METADATA`: Video metadata - FPS and number of frames.
|
||||
- `WIDTH:` Frame width.
|
||||
- `HEIGHT`: Frame height.
|
||||
- `META_FPS`: Frame rate.
|
||||
- `META_N_FRAMES`: Number of frames.
|
||||
|
||||
|
||||
The node extracts frames from the input video at the specified framerate. It resizes frames if chosen and returns them as batches of PyTorch image tensors along with latent vectors, metadata, and frame dimensions.
|
||||
|
||||
### SaveVideo
|
||||
The SaveVideo node takes in extracted frames and saves them back as a video file.
|
||||

|
||||
|
||||
#### Input Fields
|
||||
- `images`: Frame images as tensors.
|
||||
- `METADATA`: Metadata from LoadVideo node.
|
||||
- `SaveVideo`: Toggle saving output video file.
|
||||
- `SaveFrames`: Toggle saving frames to a folder.
|
||||
- `CompressionLevel`: PNG compression level for saving frames.
|
||||
#### Output
|
||||
Saves output video file and/or extracted frames.
|
||||
|
||||
The node takes extracted frames and metadata and can save them as a new video file and/or individual frame images. Video compression and frame PNG compression can be configured.
|
||||
NOTE: If you are using **LoadVideo** as source of the frames, the audio of the original file will be maintained but only in case **images_limit** and **starting_frame** are equal to Zero.
|
||||
|
||||
### LoadFramesFromFolder
|
||||

|
||||
|
||||
The LoadFramesFromFolder node allows loading image frames from a folder and returning them as a batch.
|
||||
|
||||
|
||||
#### Input Fields
|
||||
- `folder`: Path to the folder containing the frame images.Must be png format, named with a number (eg. 1.png or even 0001.png).The images will be loaded sequentially.
|
||||
- `fps`: Frames per second to assign to the loaded frames.
|
||||
|
||||
#### Output
|
||||
- `IMAGES`: Batch of loaded frame images as PyTorch tensors.
|
||||
- `METADATA`: Metadata containing the set FPS value.
|
||||
- `MAX_WIDTH`: Maximum frame width.
|
||||
- `MAX_HEIGHT`: Maximum frame height.
|
||||
- `FRAME COUNT`: Number of frames in the folder.
|
||||
- `PATH`: Path to the folder containing the frame images.
|
||||
- `IMAGE LIST`: List of frame images in the folder (not a real list just a string divided by \n).
|
||||
|
||||
The node loads all image files from the specified folder, converts them to PyTorch tensors, and returns them as a batched tensor along with simple metadata containing the set FPS value.
|
||||
|
||||
This allows easily loading a set of frames that were extracted and saved previously, for example, to reload and process them again. By setting the FPS value, the frames can be properly interpreted as a video sequence.
|
||||
|
||||
### SetMetadataForSaveVideo
|
||||
|
||||

|
||||
|
||||
The SetMetadataForSaveVideo node allows setting metadata for the SaveVideo node.
|
||||
|
||||
### FrameInterpolator
|
||||
|
||||

|
||||
|
||||
The FrameInterpolator node allows interpolating between extracted video frames to increase the frame rate and smooth motion.
|
||||
|
||||
|
||||
#### Input Fields
|
||||
|
||||
- `images`: Extracted frame images as tensors.
|
||||
- `METADATA`: Metadata from video - FPS and number of frames.
|
||||
- `multiplier`: Factor by which to increase frame rate.
|
||||
|
||||
#### Output
|
||||
|
||||
- `IMAGES`: Interpolated frames as image tensors.
|
||||
- `METADATA`: Updated metadata with new frame rate.
|
||||
|
||||
The node takes extracted frames and metadata as input. It uses an interpolation model (RIFE) to generate additional in-between frames at a higher frame rate.
|
||||
|
||||
The original frame rate in the metadata is multiplied by the `multiplier` value to get the new interpolated frame rate.
|
||||
|
||||
The interpolated frames are returned as a batch of image tensors, along with updated metadata containing the new frame rate.
|
||||
|
||||
This allows increasing the frame rate of an existing video to achieve smoother motion and slower playback. The interpolation model creates new realistic frames to fill in the gaps rather than just duplicating existing frames.
|
||||
|
||||
The original code has been taken from [HERE](https://github.com/hzwer/Practical-RIFE/tree/main)
|
||||
|
||||
## Variables
|
||||
Since the primitive node has limitations in links (for example at the time i'm writing you cannot link "start_at_step" and "steps" of another ksampler toghether), I decided to create these simple node-variables to bypass this limitation
|
||||
The node-variables are:
|
||||
@@ -159,69 +33,62 @@ The node-variables are:
|
||||
- String
|
||||
|
||||
|
||||
## 🤖 Image captioning: GPTLoaderSimple and GPTSampler 🤖
|
||||
## GPTLoaderSimple and GPTSampler
|
||||
|
||||
#### Moondream
|
||||
The model will be automatically downloaded when you run the first time.Only the required code, tokenizer, and single `model.safetensors` file are downloaded from a pinned revision of `vikhyatk/moondream1` on Hugging Face. The model file is about **3.72 GB**; the repository also contains larger alternative weights that are not downloaded.
|
||||
Anyway, it is available [HERE](https://huggingface.co/vikhyatk/moondream1/tree/main)
|
||||
The code taken from [this repository](https://github.com/vikhyat/moondream)
|
||||
These custom nodes are designed to enhance the capabilities of the ConfyUI framework by enabling text generation using GPTQ GPT models. This README provides an overview of the two custom nodes and their usage within ConfyUI.
|
||||
|
||||
#### Example with Moondream model:
|
||||

|
||||
You can add in the _extra_model_paths.yaml_ the path where your model GPTQ are in this way (example):
|
||||
|
||||
#### Joytag
|
||||
The model will be automatically downloaded when you run the first time.Only the required configuration, tags, and `model.safetensors` are downloaded from a pinned revision of `fancyfeast/joytag`. The model file is about **0.37 GB**; the unused ONNX file is not downloaded.
|
||||
Anyway, it is available [HERE](https://huggingface.co/fancyfeast/joytag/tree/main)
|
||||
The code taken from [this repository](https://github.com/fpgaminer/joytag)
|
||||
|
||||
#### Example with Joytag model:
|
||||

|
||||
|
||||
|
||||
Downloads happen on first model use, not merely when ComfyUI starts. An internet connection and sufficient disk space are required for that initial load. Models are stored under `ComfyUI/models/GPTcheckpoints/moondream` and `ComfyUI/models/GPTcheckpoints/joytag`.
|
||||
GPT Loader Simple displays a download notice when the selected model file is missing. The ComfyUI console shows the download progress.
|
||||
Moondream1 uses legacy Phi model code; N-Suite applies a small compatibility patch to its downloaded `modeling_phi.py` file and adapts the text model for image embedding generation with newer Transformers releases. The model weights are not changed.
|
||||
`other_ui:
|
||||
base_path: I:\\text-generation-webui
|
||||
GPTcheckpoints: models/`
|
||||
|
||||
Otherwise it will create a GPTcheckpoints folder in the model folder of ComfyUI where you can place your .bin models.
|
||||
|
||||
### GPTLoaderSimple
|
||||
|
||||
`GPTLoaderSimple` loads either Moondream or JoyTag. The `gpu_layers` field is retained for workflow compatibility: set it to `0` for CPU, or to a value greater than zero for GPU. The old `n_threads` and `max_ctx` fields are also retained so saved workflows continue to deserialize, but they do not affect these image-captioning models.
|
||||
|
||||
### GPTSampler
|
||||
|
||||
Connect an image and, for Moondream, a question or instruction in `prompt`. JoyTag uses `max_tags` to limit the number of returned tags. The advanced text-generation controls remain visible for workflow compatibility but no longer apply to GGUF text generation.
|
||||
|
||||
|
||||
## Image Pad For Outpainting Advanced
|
||||

|
||||
|
||||
The `ImagePadForOutpaintingAdvanced` node is an alternative to the `ImagePadForOutpainting` node that applies the technique seen in [this video](https://www.youtube.com/@robadams2451) under the outpainting mask.
|
||||
The color correction part was taken from [this](https://github.com/sipherxyz/comfyui-art-venture) custom node from Sipherxyz
|
||||
The `GPTLoaderSimple` node is responsible for loading GPT model checkpoints and creating an instance of the Llama library for text generation. It provides an interface to configure GPU layers, the number of threads, and maximum context for text generation.
|
||||
|
||||
#### Input Fields
|
||||
|
||||
- `image`: Image input.
|
||||
- `left`: pixel to extend from left,
|
||||
- `top`: pixel to extend from top,
|
||||
- `right`: pixel to extend from right,
|
||||
- `bottom`: pixel to extend from bottom.
|
||||
- `feathering`: feathering strength
|
||||
- `noise`: blend strenght from noise and the copied border
|
||||
- `pixel_size`: how big will be the pixel in the pixellated effect
|
||||
- `pixel_to_copy`: how many pixels to copy (from each side)
|
||||
- `temperature`: color correction setting that is only applied to the mask part.
|
||||
- `hue`: color correction setting that is only applied to the mask part.
|
||||
- `brightness`: color correction setting that is only applied to the mask part.
|
||||
- `contrast`: color correction setting that is only applied to the mask part.
|
||||
- `saturation`: color correction setting that is only applied to the mask part.
|
||||
- `gamma`: color correction setting that is only applied to the mask part.
|
||||
- `ckpt_name`: Select the GPT checkpoint name from the available options.
|
||||
- `gpu_layers`: Specify the number of GPU layers to use (default: 27).
|
||||
- `n_threads`: Specify the number of threads for text generation (default: 8).
|
||||
- `max_ctx`: Specify the maximum context length for text generation (default: 2048).
|
||||
|
||||
#### Output
|
||||
|
||||
The node returns the processed image and the mask.
|
||||
The node returns an instance of the Llama library (MODEL) and the path to the loaded checkpoint (STRING).
|
||||
|
||||
### GPTSampler
|
||||
|
||||
The `GPTSampler` node facilitates text generation using GPT models based on the input prompt and various generation parameters. It allows you to control aspects like temperature, top-p sampling, penalties, and more.
|
||||
|
||||
#### Input Fields
|
||||
|
||||
- `prompt`: Enter the input prompt for text generation.
|
||||
- `model`: Choose the GPT model to use for text generation.
|
||||
- `model_path`: Specify the path to the GPT model checkpoint.
|
||||
- `max_tokens`: Set the maximum number of tokens in the generated text (default: 128).
|
||||
- `temperature`: Set the temperature parameter for randomness (default: 0.7).
|
||||
- `top_p`: Set the top-p probability for nucleus sampling (default: 0.5).
|
||||
- `logprobs`: Specify the number of log probabilities to output (default: 0).
|
||||
- `echo`: Enable or disable printing the input prompt alongside the generated text.
|
||||
- `stop_token`: Specify the token at which text generation stops.
|
||||
- `frequency_penalty`, `presence_penalty`, `repeat_penalty`: Control word generation penalties.
|
||||
- `top_k`: Set the top-k tokens to consider during generation (default: 40).
|
||||
- `tfs_z`: Set the temperature scaling factor for top frequent samples (default: 1.0).
|
||||
- `print_output`: Enable or disable printing the generated text to the console.
|
||||
- `cached`: Choose whether to use cached generation (default: NO).
|
||||
- `prefix`, `suffix`: Specify text to prepend and append to the prompt.
|
||||
|
||||
#### Output
|
||||
|
||||
The node returns the generated text along with a UI-friendly representation.
|
||||
|
||||
|
||||
## Dynamic Prompt
|
||||
|
||||

|
||||
|
||||
The `DynamicPrompt` node generates prompts by combining a fixed prompt with a random selection of tags from a variable prompt. This enables flexible and dynamic prompt generation for various use cases.
|
||||
|
||||
@@ -242,39 +109,10 @@ The node returns the generated prompt, which is a combination of the fixed promp
|
||||
- Just fill the `variable_prompt` field with tag comma separated, the `fixed_prompt` is optional
|
||||
|
||||
|
||||
## CLIP Text Encode Advanced (Experimental)
|
||||
|
||||

|
||||
|
||||
The `CLIP Text Encode Advanced` node is an alternative to the standard `CLIP Text Encode` node. It offers support for Add/Replace/Delete styles, allowing for the inclusion of both positive and negative prompts within a single node.
|
||||
|
||||
The base style file is called `n-styles.csv` and is located in the `ComfyUI\styles` folder.
|
||||
The styles file follows the same format as the current `styles.csv` file utilized in A1111 (at the time of writing).
|
||||
|
||||
NOTE: this note is experimental and still have alot of bugs
|
||||
|
||||
#### Input Fields
|
||||
|
||||
- `clip`: clip input
|
||||
- `style`: it will automatically fill the positive and negative prompts based on the choosen style
|
||||
|
||||
#### Output
|
||||
- `positive`: positive conditions
|
||||
- `negative`: negative conditions
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- ~~**SaveVideo - Preview not working**: is related to a conflict with animateDiff, i've already opened a [PR](https://github.com/ArtVentureX/comfyui-animatediff/pull/64) to solve this issue. Meanwhile you can download my patched version from [here](https://github.com/Nuked88/comfyui-animatediff)~~ pull has been merged so this problem should be fixed now!
|
||||
|
||||
## Contributing
|
||||
|
||||
Feel free to contribute to this project by reporting issues or suggesting improvements. Open an issue or submit a pull request on the GitHub repository.
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
|
||||
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
|
||||
@@ -1,66 +1,51 @@
|
||||
# code based on pysssss repo
|
||||
import importlib.util
|
||||
import glob
|
||||
import os
|
||||
import sys
|
||||
import traceback
|
||||
from pathlib import Path
|
||||
from .nnodes import init, get_ext_dir
|
||||
|
||||
NODE_CLASS_MAPPINGS = {}
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {}
|
||||
WEB_DIRECTORY = "./js"
|
||||
|
||||
RIFE_REPOSITORY = "https://github.com/hzwer/Practical-RIFE.git"
|
||||
RIFE_REVISION = "a8a8035323b1c1a4a20753c751780e5b0a879455"
|
||||
RIFE_MODEL_REVISION = "572480112b87f9bfbff7579b8a38b483766e455f"
|
||||
def install_and_import(package):
|
||||
import importlib
|
||||
try:
|
||||
print("Detected: ", package)
|
||||
importlib.import_module(package)
|
||||
except ImportError:
|
||||
import pip
|
||||
pip.main(['install', package])
|
||||
finally:
|
||||
globals()[package] = importlib.import_module(package)
|
||||
|
||||
def check_module(package):
|
||||
import importlib
|
||||
try:
|
||||
print("Detected: ", package)
|
||||
importlib.import_module(package)
|
||||
return True
|
||||
except ImportError:
|
||||
return False
|
||||
|
||||
|
||||
def clone_at_revision(repo_class, repository, destination, revision):
|
||||
"""Clone an external repository once and keep it on a tested revision."""
|
||||
repo = repo_class.clone_from(repository, destination) if not os.path.exists(destination) else repo_class(destination)
|
||||
if repo.head.commit.hexsha != revision:
|
||||
repo.git.checkout(revision)
|
||||
return repo
|
||||
|
||||
# Pytest imports repository-level __init__.py files while discovering tests. A
|
||||
# standalone import has no package context and, unlike ComfyUI, cannot resolve
|
||||
# the extension's relative imports. Leave the mappings empty in that context.
|
||||
if __package__:
|
||||
from .nnodes import color, downloader, get_commit, get_ext_dir, init
|
||||
|
||||
if init():
|
||||
print("------------------------------------------")
|
||||
print(f"{color.BLUE}### N-Suite Revision:{color.END} {color.GREEN}{get_commit()} {color.END}")
|
||||
py = Path(get_ext_dir("py"))
|
||||
files = list(py.glob("*.py"))
|
||||
print(
|
||||
f"{color.YELLOW}N-Suite 1.2 removed the llama.cpp/GGUF and LLaVA nodes. "
|
||||
f"Use commit ae7cc84 to keep the legacy nodes.{color.END}"
|
||||
)
|
||||
|
||||
from git import Repo
|
||||
|
||||
rife_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), "libs", "rifle")
|
||||
clone_at_revision(Repo, RIFE_REPOSITORY, rife_path, RIFE_REVISION)
|
||||
|
||||
if not os.path.exists(os.path.join(rife_path, "train_log")):
|
||||
downloader(
|
||||
f"https://raw.githubusercontent.com/Nuked88/DreamingAI/{RIFE_MODEL_REVISION}/RIFE_trained_model_v4.7.zip"
|
||||
)
|
||||
|
||||
# Code based on pysssss's repository.
|
||||
for file in files:
|
||||
try:
|
||||
name = os.path.splitext(file)[0]
|
||||
spec = importlib.util.spec_from_file_location(name, os.path.join(py, file))
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[name] = module
|
||||
spec.loader.exec_module(module)
|
||||
mappings = getattr(module, "NODE_CLASS_MAPPINGS", None)
|
||||
if mappings is not None:
|
||||
NODE_CLASS_MAPPINGS.update(mappings)
|
||||
display_mappings = getattr(module, "NODE_DISPLAY_NAME_MAPPINGS", None)
|
||||
if display_mappings is not None:
|
||||
NODE_DISPLAY_NAME_MAPPINGS.update(display_mappings)
|
||||
except Exception:
|
||||
traceback.print_exc()
|
||||
if init():
|
||||
py = get_ext_dir("py")
|
||||
files = glob.glob("*.py", root_dir=py, recursive=False)
|
||||
install_and_import('moviepy')
|
||||
for file in files:
|
||||
try:
|
||||
name = os.path.splitext(file)[0]
|
||||
spec = importlib.util.spec_from_file_location(name, os.path.join(py, file))
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[name] = module
|
||||
spec.loader.exec_module(module)
|
||||
if hasattr(module, "NODE_CLASS_MAPPINGS") and getattr(module, "NODE_CLASS_MAPPINGS") is not None:
|
||||
NODE_CLASS_MAPPINGS.update(module.NODE_CLASS_MAPPINGS)
|
||||
if hasattr(module, "NODE_DISPLAY_NAME_MAPPINGS") and getattr(module, "NODE_DISPLAY_NAME_MAPPINGS") is not None:
|
||||
NODE_DISPLAY_NAME_MAPPINGS.update(module.NODE_DISPLAY_NAME_MAPPINGS)
|
||||
except Exception as e:
|
||||
print(e)
|
||||
|
||||
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"]
|
||||
|
||||
@@ -1,4 +0,0 @@
|
||||
{
|
||||
"name": "N-Suite",
|
||||
"logging": false
|
||||
}
|
||||
@@ -1,14 +0,0 @@
|
||||
# N-Suite: test di tutti i nodi
|
||||
|
||||
Apri `N-Suite-all-nodes-test.json` in ComfyUI. Il workflow contiene tutti i 14 tipi di nodo N-Suite presenti in questa versione, con anteprime dei risultati e tre prove di salvataggio video.
|
||||
|
||||
Prima di premere **Queue Prompt**:
|
||||
|
||||
1. Scegli una tua immagine nel nodo **LoadImage** della sezione 01. Il loader GPT usa Moondream, già selezionato nel workflow.
|
||||
2. Copia un MP4 breve nella cartella `ComfyUI/input/n-suite`, ricarica la pagina e selezionalo nel nodo **LoadVideo** della sezione 04. Un video di pochi secondi riduce il tempo necessario per RIFE.
|
||||
3. Metti almeno due immagini PNG della stessa dimensione, con nomi numerati come `0001.png` e `0002.png`, nella cartella `ComfyUI/input/n-suite/test_frames`. Il nodo **String Variable** della sezione 05 contiene il percorso visto dal container: `/workspace/ComfyUI/input/n-suite/test_frames`.
|
||||
4. Premi **Queue Prompt**. Il nodo CLIP usa `clip_l.safetensors`, già disponibile nell'installazione per cui è stato creato il workflow.
|
||||
|
||||
La risposta Moondream e i condizionamenti CLIP compaiono nei nodi **Preview as Text**. Le immagini e la maschera compaiono nelle anteprime. I video vengono scritti in `ComfyUI/output/n-suite/videos` con prefissi `n_suite_test_*`. Se un ramo fallisce, ComfyUI evidenzia il nodo che ha generato l'errore.
|
||||
|
||||
Il file è generato da `generate_test_workflow.py` usando gli schemi `/object_info` di ComfyUI. Su un'installazione diversa, rigeneralo con `python examples/generate_test_workflow.py http://127.0.0.1:8188` dalla cartella del repository.
|
||||
@@ -1,144 +0,0 @@
|
||||
"""Generate the all-node smoke test from a running ComfyUI instance.
|
||||
|
||||
Usage: python examples/generate_test_workflow.py http://127.0.0.1:8188
|
||||
"""
|
||||
|
||||
import json
|
||||
import sys
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
from urllib.request import urlopen
|
||||
|
||||
|
||||
url = sys.argv[1].rstrip("/") if len(sys.argv) > 1 else "http://127.0.0.1:8188"
|
||||
schema = json.load(urlopen(f"{url}/object_info"))
|
||||
nodes = []
|
||||
links = []
|
||||
|
||||
|
||||
def add(kind, pos, values=None, title=None, size=None):
|
||||
info = schema[kind]
|
||||
values = values or {}
|
||||
inputs, widgets = [], []
|
||||
for name, spec in {**info["input"].get("required", {}), **info["input"].get("optional", {})}.items():
|
||||
raw_type = spec[0]
|
||||
input_type = "COMBO" if isinstance(raw_type, list) else raw_type
|
||||
options = spec[1] if len(spec) > 1 and isinstance(spec[1], dict) else {}
|
||||
entry = {"name": name, "type": input_type, "link": None}
|
||||
if input_type in ("COMBO", "STRING", "INT", "FLOAT", "BOOLEAN") and not options.get("forceInput"):
|
||||
entry["widget"] = {"name": name}
|
||||
default = options.get("default", raw_type[0] if isinstance(raw_type, list) and raw_type else "")
|
||||
widgets.append(values.get(name, default))
|
||||
inputs.append(entry)
|
||||
if kind == "LoadImage":
|
||||
inputs.append({"name": "upload", "type": "IMAGEUPLOAD", "widget": {"name": "upload"}, "link": None})
|
||||
widgets.append("image")
|
||||
node_id = len(nodes) + 1
|
||||
node = {
|
||||
"id": node_id, "type": kind, "pos": pos, "size": size or [350, 180],
|
||||
"flags": {}, "order": node_id - 1, "mode": 0, "inputs": inputs,
|
||||
"outputs": [{"name": name, "type": typ, "links": []} for name, typ in
|
||||
zip(info.get("output_name", info["output"]), info["output"])],
|
||||
"properties": {"Node name for S&R": kind}, "widgets_values": widgets,
|
||||
}
|
||||
if title:
|
||||
node["title"] = title
|
||||
nodes.append(node)
|
||||
return node_id
|
||||
|
||||
|
||||
def connect(source, slot, target, input_name):
|
||||
origin = nodes[source - 1]
|
||||
dest = nodes[target - 1]
|
||||
dest_slot = next(i for i, item in enumerate(dest["inputs"]) if item["name"] == input_name)
|
||||
assert dest["inputs"][dest_slot]["link"] is None
|
||||
link_id = len(links) + 1
|
||||
links.append([link_id, source, slot, target, dest_slot, origin["outputs"][slot]["type"]])
|
||||
origin["outputs"][slot]["links"].append(link_id)
|
||||
dest["inputs"][dest_slot]["link"] = link_id
|
||||
|
||||
|
||||
image = add("LoadImage", [80, 100], {"image": "example.png"}, "Scegli la tua immagine", [380, 330])
|
||||
questions = add("String Variable [n-suite]", [80, 500],
|
||||
{"string": "What is in this image?,What colors are in this image?"}, "Domande di prova")
|
||||
dynamic = add("DynamicPrompt [n-suite]", [520, 490],
|
||||
{"cached": "NO", "number_of_random_tag": "Fixed", "fixed_number_of_random_tag": 1})
|
||||
caption_model = add("GPT Loader Simple [n-suite]", [520, 100], {"ckpt_name": "moondream"})
|
||||
caption = add("GPT Sampler [n-suite]", [930, 100],
|
||||
{"max_tokens": 128, "cached": "NO", "print_output": "enable"}, size=[390, 700])
|
||||
caption_preview = add("PreviewAny", [1400, 150], title="Risposta Moondream")
|
||||
noise = add("Float Variable [n-suite]", [80, 1040], {"value": 0.1})
|
||||
pad = add("ImagePadForOutpaintAdvanced [n-suite]", [500, 930],
|
||||
{"left": 32, "right": 32, "top": 32, "bottom": 32}, size=[430, 590])
|
||||
padded_preview = add("PreviewImage", [1030, 970], title="Immagine con bordo", size=[350, 300])
|
||||
mask_to_image = add("MaskToImage", [1030, 1330])
|
||||
mask_preview = add("PreviewImage", [1410, 1320], title="Maschera del bordo", size=[350, 300])
|
||||
clip = add("CLIPLoader", [2030, 100], {"clip_name": "clip_l.safetensors", "type": "stable_diffusion"})
|
||||
encode = add("CLIPTextEncodeAdvancedNSuite [n-suite]", [2460, 100],
|
||||
{"styles": "NAI", "positive_prompt": "a small test image", "negative_prompt": "blurry"}, size=[400, 350])
|
||||
positive_preview = add("PreviewAny", [2940, 100], title="Condizionamento positivo")
|
||||
negative_preview = add("PreviewAny", [2940, 400], title="Condizionamento negativo")
|
||||
multiplier = add("Integer Variable [n-suite]", [80, 2060], {"value": 2})
|
||||
video = add("LoadVideo [n-suite]", [430, 1950],
|
||||
{"video": "SELECT_VIDEO.mp4", "framerate": "original", "resize_by": "none",
|
||||
"images_limit": 0, "batch_size": 0, "starting_frame": 0, "autoplay": False, "use_ram": False},
|
||||
size=[420, 570])
|
||||
interpolator = add("FrameInterpolator [n-suite]", [940, 1990])
|
||||
interpolated_video = add("SaveVideo [n-suite]", [1430, 1970],
|
||||
{"SaveVideo": True, "SaveFrames": False, "filename_prefix": "n_suite_test_interpolated"})
|
||||
video_info = add("PreviewAny", [940, 2290], title="Metadati video originale")
|
||||
folder = add("String Variable [n-suite]", [80, 2990],
|
||||
{"string": "/workspace/ComfyUI/input/n-suite/test_frames"}, "Cartella frame: cambia qui", [500, 130])
|
||||
image_folder = add("LoadImageFromFolder [n-suite]", [660, 2860])
|
||||
image_folder_preview = add("PreviewImage", [1100, 2840], title="Immagini dalla cartella", size=[330, 260])
|
||||
manual_metadata = add("SetMetadataForSaveVideo [n-suite]", [1100, 3210],
|
||||
{"fps": 24, "VideoName": "n_suite_folder"})
|
||||
manual_video = add("SaveVideo [n-suite]", [1550, 2890],
|
||||
{"SaveVideo": True, "SaveFrames": False, "filename_prefix": "n_suite_test_manual_metadata"})
|
||||
frame_folder = add("LoadFramesFromFolder [n-suite]", [660, 3530], {"fps": 24})
|
||||
frame_folder_preview = add("PreviewImage", [1100, 3520], title="Frame numerati", size=[330, 260])
|
||||
frame_video = add("SaveVideo [n-suite]", [1550, 3510],
|
||||
{"SaveVideo": True, "SaveFrames": False, "filename_prefix": "n_suite_test_folder_frames"})
|
||||
|
||||
for args in [
|
||||
(questions, 0, dynamic, "variable_prompt"), (dynamic, 0, caption, "prompt"),
|
||||
(caption_model, 0, caption, "model"), (image, 0, caption, "image"),
|
||||
(caption, 0, caption_preview, "source"), (image, 0, pad, "image"),
|
||||
(noise, 0, pad, "noise"), (pad, 0, padded_preview, "images"),
|
||||
(pad, 1, mask_to_image, "mask"), (mask_to_image, 0, mask_preview, "images"),
|
||||
(clip, 0, encode, "clip"), (encode, 0, positive_preview, "source"),
|
||||
(encode, 1, negative_preview, "source"), (video, 0, interpolator, "images"),
|
||||
(video, 2, interpolator, "METADATA"), (multiplier, 0, interpolator, "multiplier"),
|
||||
(interpolator, 0, interpolated_video, "images"),
|
||||
(interpolator, 1, interpolated_video, "METADATA"), (video, 2, video_info, "source"),
|
||||
(folder, 0, image_folder, "folder"), (folder, 0, frame_folder, "folder"),
|
||||
(image_folder, 0, image_folder_preview, "images"),
|
||||
(image_folder, 0, manual_video, "images"),
|
||||
(image_folder, 3, manual_metadata, "number_of_frames"),
|
||||
(manual_metadata, 0, manual_video, "METADATA"),
|
||||
(frame_folder, 0, frame_folder_preview, "images"),
|
||||
(frame_folder, 0, frame_video, "images"),
|
||||
(frame_folder, 1, frame_video, "METADATA"),
|
||||
]:
|
||||
connect(*args)
|
||||
|
||||
groups = [
|
||||
("01 FOTO + MOONDREAM: scegli la foto in LoadImage", [40, 40, 1750, 790], "#3f789e"),
|
||||
("02 IMAGE PAD: controlla immagine e maschera", [40, 870, 1760, 790], "#637c49"),
|
||||
("03 CLIP: usa il modello clip_l presente", [1980, 40, 1400, 680], "#76578e"),
|
||||
("04 VIDEO: copia un MP4 in input/n-suite, ricarica e selezionalo", [40, 1880, 1790, 690], "#896a3c"),
|
||||
("05 CARTELLA: aggiungi 0001.png e 0002.png in test_frames", [40, 2780, 1920, 1050], "#3f789e"),
|
||||
]
|
||||
used = {node["type"] for node in nodes if "[n-suite]" in node["type"].lower()}
|
||||
expected = {name for name in schema if "[n-suite]" in name.lower()}
|
||||
assert used == expected, f"Missing N-Suite nodes: {sorted(expected - used)}"
|
||||
workflow = {
|
||||
"id": str(uuid.uuid4()), "revision": 0, "last_node_id": len(nodes), "last_link_id": len(links),
|
||||
"nodes": nodes, "links": links,
|
||||
"groups": [{"id": i, "title": title, "bounding": bounds, "color": color, "flags": {}}
|
||||
for i, (title, bounds, color) in enumerate(groups, 1)],
|
||||
"config": {}, "extra": {"ds": {"scale": 0.55, "offset": [70, 70]}}, "version": 0.4,
|
||||
}
|
||||
destination = Path(__file__).with_name("N-Suite-all-nodes-test.json")
|
||||
destination.write_text(json.dumps(workflow, ensure_ascii=False, indent=2) + "\n")
|
||||
print(f"Saved {destination}: {len(nodes)} nodes, {len(links)} links, {len(used)} N-Suite types")
|
||||
|
Before Width: | Height: | Size: 13 KiB |
|
Before Width: | Height: | Size: 13 KiB |
|
Before Width: | Height: | Size: 76 KiB |
|
Before Width: | Height: | Size: 26 KiB |
|
Before Width: | Height: | Size: 351 KiB |
|
Before Width: | Height: | Size: 289 KiB |
|
Before Width: | Height: | Size: 20 KiB |
|
Before Width: | Height: | Size: 8.7 KiB |
|
Before Width: | Height: | Size: 168 KiB |
|
Before Width: | Height: | Size: 139 KiB |
|
Before Width: | Height: | Size: 123 KiB |
|
Before Width: | Height: | Size: 35 KiB |
|
Before Width: | Height: | Size: 14 KiB |
|
Before Width: | Height: | Size: 20 KiB |
|
Before Width: | Height: | Size: 12 KiB |
@@ -0,0 +1,18 @@
|
||||
@echo off
|
||||
|
||||
SET CMAKE_ARGS=-DLLAMA_CUBLAS=on
|
||||
SET FORCE_CMAKE=1
|
||||
xcopy "%CUDA_PATH%\extras\visual_studio_integration\MSBuildExtensions\" "C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\BuildCustomizations\" /E /I /Y
|
||||
xcopy "%CUDA_PATH%\extras\visual_studio_integration\MSBuildExtensions\" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\BuildCustomizations\" /E /I /Y
|
||||
|
||||
|
||||
cd /d %~dp0
|
||||
git pull
|
||||
cd ../../../python_embeded
|
||||
|
||||
python.exe -s -m pip install scikit-build
|
||||
python.exe -s -m pip install cmake moviepy
|
||||
python.exe -s -m pip install llama-cpp-python==0.1.78 --force-reinstall --upgrade --no-cache-dir
|
||||
|
||||
|
||||
PAUSE
|
||||
@@ -0,0 +1,18 @@
|
||||
@echo off
|
||||
|
||||
SET CMAKE_ARGS=-DLLAMA_CUBLAS=on
|
||||
SET FORCE_CMAKE=1
|
||||
xcopy "%CUDA_PATH%\extras\visual_studio_integration\MSBuildExtensions\" "C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\BuildCustomizations\" /E /I /Y
|
||||
xcopy "%CUDA_PATH%\extras\visual_studio_integration\MSBuildExtensions\" "C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\BuildCustomizations\" /E /I /Y
|
||||
|
||||
|
||||
cd /d %~dp0
|
||||
git pull
|
||||
cd ../../../python_embeded
|
||||
|
||||
python.exe -s -m pip install scikit-build
|
||||
python.exe -s -m pip install cmake moviepy
|
||||
python.exe -s -m pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir
|
||||
|
||||
|
||||
PAUSE
|
||||
@@ -1,214 +0,0 @@
|
||||
import { app } from "/scripts/app.js";
|
||||
import { api } from "/scripts/api.js"
|
||||
const MultilineSymbol = Symbol();
|
||||
const MultilineResizeSymbol = Symbol();
|
||||
|
||||
function getStyles(name) {
|
||||
//console.log("getStyles called " + name);
|
||||
|
||||
return api.fetchApi('/nsuite/styles')
|
||||
.then(response => response.json())
|
||||
.then(data => {
|
||||
// Eseguire l'elaborazione dei dati
|
||||
const styles = data.styles;
|
||||
//console.log('Styles:', styles);
|
||||
let positive_prompt = "";
|
||||
let negative_prompt = "";
|
||||
|
||||
// Funzione per ottenere positive_prompt e negative_prompt dato il name
|
||||
for (let i = 0; i < styles[0].length; i++) {
|
||||
const style = styles[0][i];
|
||||
if (style.name === name) {
|
||||
positive_prompt = style.prompt;
|
||||
negative_prompt = style.negative_prompt;
|
||||
//console.log('Style:', style.name);
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
if (positive_prompt !== "") {
|
||||
//console.log("Positive prompt:", positive_prompt);
|
||||
//console.log("Negative prompt:", negative_prompt);
|
||||
return { positive_prompt: positive_prompt, negative_prompt: negative_prompt };
|
||||
} else {
|
||||
return { positive_prompt: "", negative_prompt: "" };
|
||||
}
|
||||
})
|
||||
.catch(error => {
|
||||
console.error('Error:', error);
|
||||
throw error; // Rilancia l'errore per consentire al chiamante di gestirlo
|
||||
});
|
||||
}
|
||||
|
||||
function addStyles(name, positive_prompt, negative_prompt) {
|
||||
return api.fetchApi('/nsuite/styles/add', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
name: name,
|
||||
positive_prompt: positive_prompt,
|
||||
negative_prompt: negative_prompt
|
||||
}),
|
||||
|
||||
})
|
||||
}
|
||||
|
||||
function updateStyles(name, positive_prompt, negative_prompt) {
|
||||
return api.fetchApi('/nsuite/styles/update', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
name: name,
|
||||
positive_prompt: positive_prompt,
|
||||
negative_prompt: negative_prompt
|
||||
}),
|
||||
})
|
||||
}
|
||||
|
||||
function removeStyles(name) {
|
||||
//confirmation
|
||||
let ok = confirm("Are you sure you want to remove this style?");
|
||||
if (!ok) {
|
||||
return;
|
||||
}
|
||||
|
||||
return api.fetchApi('/nsuite/styles/remove', {
|
||||
method: 'POST',
|
||||
headers: {
|
||||
'Content-Type': 'application/json',
|
||||
},
|
||||
body: JSON.stringify({
|
||||
name: name
|
||||
}),
|
||||
})
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "n.CLIPTextEncodeAdvancedNSuite",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData, app) {
|
||||
|
||||
const onAdded = nodeType.prototype.onAdded;
|
||||
if (nodeData.name === "CLIPTextEncodeAdvancedNSuite [n-suite]") {
|
||||
nodeType.prototype.onAdded = function () {
|
||||
onAdded?.apply(this, arguments);
|
||||
const styles = this.widgets.find((w) => w.name === "styles");
|
||||
const p_prompt = this.widgets.find((w) => w.name === "positive_prompt");
|
||||
const n_prompt = this.widgets.find((w) => w.name === "negative_prompt");
|
||||
if (!styles || !p_prompt || !n_prompt) return;
|
||||
const cb = styles.callback;
|
||||
let addedd_positive_prompt = "";
|
||||
let addedd_negative_prompt = "";
|
||||
styles.callback = function () {
|
||||
let index = styles.options.values.indexOf(styles.value);
|
||||
|
||||
|
||||
if (addedd_positive_prompt == "" && addedd_negative_prompt == "") {
|
||||
getStyles(styles.options.values[index-1]).then(style_prompts => {
|
||||
//wait 4 seconds
|
||||
|
||||
console.log(style_prompts);
|
||||
|
||||
addedd_positive_prompt = style_prompts.positive_prompt;
|
||||
addedd_negative_prompt = style_prompts.negative_prompt;
|
||||
//alert("Addedd positive prompt: " + addedd_positive_prompt + "\nAddedd negative prompt: " + addedd_negative_prompt);
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
let current_positive_prompt = p_prompt.value;
|
||||
let current_negative_prompt = n_prompt.value;
|
||||
|
||||
getStyles(styles.value).then(style_prompts => {
|
||||
//console.log(style_prompts)
|
||||
|
||||
if ((current_positive_prompt.trim() != addedd_positive_prompt.trim() || current_negative_prompt.trim() != addedd_negative_prompt.trim())) {
|
||||
|
||||
let ok = confirm("Style has been changed. Do you want to change style without saving?");
|
||||
|
||||
|
||||
if (!ok) {
|
||||
if (styles.value === styles.options.values[0]) {
|
||||
styles.value = styles.options.values[0];
|
||||
}
|
||||
styles.value = styles.options.values[index-1];
|
||||
|
||||
|
||||
return;
|
||||
}
|
||||
}
|
||||
|
||||
// add the addedd prompt to the current prompt
|
||||
p_prompt.value = style_prompts.positive_prompt;
|
||||
n_prompt.value = style_prompts.negative_prompt;
|
||||
|
||||
|
||||
addedd_positive_prompt = style_prompts.positive_prompt;
|
||||
addedd_negative_prompt = style_prompts.negative_prompt;
|
||||
if (cb) {
|
||||
return cb.apply(this, arguments);
|
||||
}
|
||||
})
|
||||
.catch(error => {
|
||||
console.error('Error:', error);
|
||||
});
|
||||
|
||||
};
|
||||
|
||||
|
||||
|
||||
let savestyle;
|
||||
let replacestyle;
|
||||
let deletestyle;
|
||||
|
||||
|
||||
// Create the button widget for selecting the files
|
||||
savestyle = this.addWidget("button", "New", "image", () => {
|
||||
////console.log("Save called");
|
||||
//ask input name style
|
||||
let inputName = prompt("Enter a name for the style:", styles.value);
|
||||
if (inputName === null) {
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
addStyles(inputName, p_prompt.value, n_prompt.value);
|
||||
// Add the file to the dropdown list and update the widget value
|
||||
|
||||
if (!styles.options.values.includes(inputName)) {
|
||||
styles.options.values.push(inputName);
|
||||
}
|
||||
|
||||
},{
|
||||
cursor: "grab",
|
||||
},);
|
||||
replacestyle = this.addWidget("button", "Replace", "image", () => {
|
||||
//console.log("Replace called");
|
||||
updateStyles(styles.value, p_prompt.value, n_prompt.value);
|
||||
},{
|
||||
cursor: "grab",
|
||||
},);
|
||||
deletestyle = this.addWidget("button", "Delete", "image", () => {
|
||||
//console.log("Delete called");
|
||||
removeStyles(styles.value);
|
||||
|
||||
// Remove the file from the dropdown list
|
||||
styles.options.values = styles.options.values.filter((value) => value !== styles.value);
|
||||
},{
|
||||
cursor: "grab",
|
||||
},);
|
||||
savestyle.serialize = false;
|
||||
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
};
|
||||
|
||||
},
|
||||
});
|
||||
@@ -3,26 +3,42 @@ import { ComfyWidgets } from "/scripts/widgets.js";
|
||||
|
||||
app.registerExtension({
|
||||
name: "n.DynamicPrompt",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData) {
|
||||
if (nodeData.name !== "DynamicPrompt [n-suite]") return;
|
||||
async beforeRegisterNodeDef(nodeType, nodeData, app) {
|
||||
|
||||
if (nodeData.name === "DynamicPrompt") {
|
||||
console.warn("DynamicPrompt detected")
|
||||
|
||||
const onExecuted = nodeType.prototype.onExecuted;
|
||||
|
||||
const onExecuted = nodeType.prototype.onExecuted;
|
||||
nodeType.prototype.onExecuted = function () {
|
||||
onExecuted?.apply(this, arguments);
|
||||
const widgets = this.widgets ?? [];
|
||||
const cached = widgets.find((widget) => widget.name === "cached");
|
||||
for (const widget of widgets.filter((item) => item.name === "text")) {
|
||||
this.removeWidget(widget);
|
||||
}
|
||||
if (cached?.value === "NO") {
|
||||
const result = ComfyWidgets.STRING(
|
||||
this,
|
||||
"text",
|
||||
["STRING", { multiline: true }],
|
||||
app,
|
||||
);
|
||||
result.widget.value = Math.floor(Math.random() * 10000);
|
||||
}
|
||||
};
|
||||
|
||||
nodeType.prototype.onExecuted = function (message) {
|
||||
onExecuted?.apply(this, arguments);
|
||||
|
||||
const pos_cached = this.widgets.findIndex((w) => w.name === "cached");
|
||||
console.warn("value:"+pos_cached)
|
||||
|
||||
if (this.widgets) {
|
||||
const pos_text = this.widgets.findIndex((w) => w.name === "text");
|
||||
if (pos_text !== -1) {
|
||||
for (let i = pos_text; i < this.widgets.length; i++) {
|
||||
this.widgets[i].onRemove?.();
|
||||
}
|
||||
this.widgets.length = pos_text;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
if (this.widgets[pos_cached].value === "NO") {
|
||||
|
||||
const w = ComfyWidgets["STRING"](this, "text", ["STRING", { multiline: true }], app);
|
||||
//random seed
|
||||
var rnm = Math.floor(Math.random() * 10000)
|
||||
w.widget.value = rnm;
|
||||
|
||||
|
||||
}
|
||||
|
||||
};
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
@@ -1,81 +1,430 @@
|
||||
import { app } from "/scripts/app.js";
|
||||
import { api } from "/scripts/api.js";
|
||||
//extended_widgets.js
|
||||
import { api } from "/scripts/api.js"
|
||||
import { ComfyWidgets } from "/scripts/widgets.js";
|
||||
|
||||
function buildViewUrl(name, type, defaultSubfolder) {
|
||||
const separator = name.lastIndexOf("/");
|
||||
const subfolder = separator >= 0 ? name.slice(0, separator) : defaultSubfolder;
|
||||
const filename = separator >= 0 ? name.slice(separator + 1) : name;
|
||||
const params = new URLSearchParams({ filename, type, subfolder });
|
||||
return api.apiURL(`/view?${params.toString()}`);
|
||||
const MultilineSymbol = Symbol();
|
||||
const MultilineResizeSymbol = Symbol();
|
||||
async function uploadFile(file, updateNode, node, pasted = false) {
|
||||
const videoWidget = node.widgets.find((w) => w.name === "video");
|
||||
|
||||
|
||||
try {
|
||||
// Wrap file in formdata so it includes filename
|
||||
const body = new FormData();
|
||||
body.append("image", file);
|
||||
if (pasted) {
|
||||
body.append("subfolder", "pasted");
|
||||
}
|
||||
else {
|
||||
body.append("subfolder", "n-suite");
|
||||
}
|
||||
|
||||
const resp = await api.fetchApi("/upload/image", {
|
||||
method: "POST",
|
||||
body,
|
||||
});
|
||||
|
||||
if (resp.status === 200) {
|
||||
const data = await resp.json();
|
||||
// Add the file to the dropdown list and update the widget value
|
||||
let path = data.name;
|
||||
|
||||
|
||||
if (!videoWidget.options.values.includes(path)) {
|
||||
videoWidget.options.values.push(path);
|
||||
}
|
||||
|
||||
if (updateNode) {
|
||||
|
||||
videoWidget.value = path;
|
||||
if (data.subfolder) path = data.subfolder + "/" + path;
|
||||
showVideoInput(path,node);
|
||||
|
||||
}
|
||||
} else {
|
||||
alert(resp.status + " - " + resp.statusText);
|
||||
}
|
||||
} catch (error) {
|
||||
alert(error);
|
||||
}
|
||||
}
|
||||
|
||||
function updateVideoWidget(node, widgetName, url) {
|
||||
const widget = node.widgets?.find((item) => item.name === widgetName);
|
||||
if (!widget?.element) return;
|
||||
widget.element.src = url;
|
||||
}
|
||||
function addVideo(node, name,src, app) {
|
||||
console.log(src)
|
||||
|
||||
const MIN_SIZE = 50;
|
||||
function computeSize(size) {
|
||||
try{
|
||||
|
||||
if (node.widgets[0].last_y == null) return;
|
||||
|
||||
function addVideo(node, name, src, autoplayValue) {
|
||||
const video = document.createElement("video");
|
||||
video.controls = true;
|
||||
video.loop = true;
|
||||
video.muted = true;
|
||||
video.autoplay = autoplayValue;
|
||||
video.playsInline = true;
|
||||
video.src = src || "";
|
||||
video.style.width = "100%";
|
||||
video.style.height = "100%";
|
||||
video.style.objectFit = "contain";
|
||||
let y = node.widgets[0].last_y;
|
||||
let freeSpace = size[1] - y;
|
||||
|
||||
// Compute the height of all non customvideo widgets
|
||||
let widgetHeight = 0;
|
||||
const multi = [];
|
||||
for (let i = 0; i < node.widgets.length; i++) {
|
||||
const w = node.widgets[i];
|
||||
if (w.type === "customvideo") {
|
||||
multi.push(w);
|
||||
} else {
|
||||
if (w.computeSize) {
|
||||
widgetHeight += w.computeSize()[1] + 4;
|
||||
} else {
|
||||
widgetHeight += LiteGraph.NODE_WIDGET_HEIGHT + 4;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// See how large each text input can be
|
||||
freeSpace -= widgetHeight;
|
||||
freeSpace /= multi.length + (!!node.imgs?.length);
|
||||
|
||||
if (freeSpace < MIN_SIZE) {
|
||||
// There isnt enough space for all the widgets, increase the size of the node
|
||||
freeSpace = MIN_SIZE;
|
||||
node.size[1] = y + widgetHeight + freeSpace * (multi.length + (!!node.imgs?.length));
|
||||
node.graph.setDirtyCanvas(true);
|
||||
}
|
||||
|
||||
// Position each of the widgets
|
||||
for (const w of node.widgets) {
|
||||
w.y = y;
|
||||
if (w.type === "customvideo") {
|
||||
y += freeSpace;
|
||||
w.computedHeight = freeSpace - multi.length*4;
|
||||
} else if (w.computeSize) {
|
||||
y += w.computeSize()[1] + 4;
|
||||
} else {
|
||||
y += LiteGraph.NODE_WIDGET_HEIGHT + 4;
|
||||
}
|
||||
}
|
||||
|
||||
node.inputHeight = freeSpace;
|
||||
}catch(e){
|
||||
|
||||
}
|
||||
}
|
||||
const widget = {
|
||||
type: "customvideo",
|
||||
name,
|
||||
get value() {
|
||||
return this.inputEl.value;
|
||||
},
|
||||
set value(x) {
|
||||
this.inputEl.value = x;
|
||||
},
|
||||
draw: function (ctx, _, widgetWidth, y, widgetHeight) {
|
||||
if (!this.parent.inputHeight) {
|
||||
// If we are initially offscreen when created we wont have received a resize event
|
||||
// Calculate it here instead
|
||||
node.setSizeForImage?.();
|
||||
|
||||
}
|
||||
const visible = app.canvas.ds.scale > 0.5 && this.type === "customvideo";
|
||||
const margin = 10;
|
||||
const elRect = ctx.canvas.getBoundingClientRect();
|
||||
const transform = new DOMMatrix()
|
||||
.scaleSelf(elRect.width / ctx.canvas.width, elRect.height / ctx.canvas.height)
|
||||
.multiplySelf(ctx.getTransform())
|
||||
.translateSelf(margin, margin + y);
|
||||
|
||||
const scale = new DOMMatrix().scaleSelf(transform.a, transform.d)
|
||||
Object.assign(this.inputEl.style, {
|
||||
transformOrigin: "0 0",
|
||||
transform: scale,
|
||||
left: `${transform.a + transform.e}px`,
|
||||
top: `${transform.d + transform.f}px`,
|
||||
width: `${widgetWidth - (margin * 2)}px`,
|
||||
height: `${this.parent.inputHeight - (margin * 2)}px`,
|
||||
position: "absolute",
|
||||
background: (!node.color)?'':node.color,
|
||||
color: (!node.color)?'':'white',
|
||||
zIndex: app.graph._nodes.indexOf(node),
|
||||
});
|
||||
this.inputEl.hidden = !visible;
|
||||
},
|
||||
};
|
||||
let type_file="mp4";
|
||||
const regex = /\.gif&type/;
|
||||
|
||||
if (regex.test(src)) {
|
||||
type_file="gif";
|
||||
}
|
||||
|
||||
widget.inputEl = document.createElement("div");
|
||||
Object.assign(widget.inputEl, {
|
||||
id: "videoContainer",
|
||||
width: 400,
|
||||
height: 300
|
||||
})
|
||||
|
||||
|
||||
if (type_file=="gif"){
|
||||
|
||||
let img_element = document.createElement("img");
|
||||
Object.assign(img_element, {
|
||||
id:"mediaContainer",
|
||||
src: src,
|
||||
style: "width: 100%; height: 100%;",
|
||||
type : "image/gif"
|
||||
})
|
||||
|
||||
widget.inputEl.appendChild(img_element);
|
||||
}
|
||||
else{
|
||||
let video_element = document.createElement("video");
|
||||
// Set the video attributes
|
||||
Object.assign(video_element, {
|
||||
id:"mediaContainer",
|
||||
controls: true,
|
||||
src: src,
|
||||
poster: "",
|
||||
style: "width: 100%; height: 100%;",
|
||||
loop: true,
|
||||
muted: true,
|
||||
autoplay:true,
|
||||
type : "video/mp4"
|
||||
|
||||
});
|
||||
widget.inputEl.appendChild(video_element);
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
// Add video element to the body
|
||||
document.body.appendChild(widget.inputEl);
|
||||
|
||||
|
||||
|
||||
widget.parent = node;
|
||||
//document.body.appendChild(widget.inputEl);
|
||||
|
||||
node.addCustomWidget(widget);
|
||||
|
||||
app.canvas.onDrawBackground = function () {
|
||||
// Draw node isnt fired once the node is off the screen
|
||||
// if it goes off screen quickly, the input may not be removed
|
||||
// this shifts it off screen so it can be moved back if the node is visible.
|
||||
for (let n in app.graph._nodes) {
|
||||
n = graph._nodes[n];
|
||||
for (let w in n.widgets) {
|
||||
let wid = n.widgets[w];
|
||||
if (Object.hasOwn(wid, "inputEl")) {
|
||||
wid.inputEl.style.left = -8000 + "px";
|
||||
wid.inputEl.style.position = "absolute";
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
node.onRemoved = function () {
|
||||
// When removing this node we need to remove the input from the DOM
|
||||
for (let y in this.widgets) {
|
||||
if (this.widgets[y].inputEl) {
|
||||
this.widgets[y].inputEl.remove();
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
widget.onRemove = () => {
|
||||
widget.inputEl?.remove();
|
||||
|
||||
// Restore original size handler if we are the last
|
||||
if (!--node[MultilineSymbol]) {
|
||||
node.onResize = node[MultilineResizeSymbol];
|
||||
delete node[MultilineSymbol];
|
||||
delete node[MultilineResizeSymbol];
|
||||
}
|
||||
};
|
||||
|
||||
if (node[MultilineSymbol]) {
|
||||
node[MultilineSymbol]++;
|
||||
} else {
|
||||
node[MultilineSymbol] = 1;
|
||||
const onResize = (node[MultilineResizeSymbol] = node.onResize);
|
||||
|
||||
node.onResize = function (size) {
|
||||
|
||||
computeSize(size);
|
||||
// Call original resizer handler
|
||||
if (onResize) {
|
||||
onResize.apply(this, arguments);
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
const widget = node.addDOMWidget(name, "video", video, {
|
||||
hideOnZoom: false,
|
||||
getMinHeight: () => 200,
|
||||
getHeight: () => 240,
|
||||
});
|
||||
widget.serialize = false;
|
||||
widget.options.serialize = false;
|
||||
return { minWidth: 400, minHeight: 200, widget };
|
||||
}
|
||||
|
||||
export function showVideoInput(name, node) {
|
||||
const url = buildViewUrl(name, "input", "n-suite");
|
||||
updateVideoWidget(node, "videoWidget", url);
|
||||
const localUrl = node.widgets?.find((item) => item.name === "local_url");
|
||||
if (localUrl) localUrl.value = url;
|
||||
return url;
|
||||
|
||||
export function showVideoInput(name,node) {
|
||||
const videoWidget = node.widgets.find((w) => w.name === "videoWidget");
|
||||
const temp_web_url = node.widgets.find((w) => w.name === "local_url");
|
||||
///const videoContainer = videoWidget.inputEl.widgets.find((w) => w.name === "videoWidget");
|
||||
|
||||
let folder_separator = name.lastIndexOf("/");
|
||||
let subfolder = "n-suite";
|
||||
if (folder_separator > -1) {
|
||||
subfolder = name.substring(0, folder_separator);
|
||||
name = name.substring(folder_separator + 1);
|
||||
}
|
||||
|
||||
let url_video = api.apiURL(`/view?filename=${encodeURIComponent(name)}&type=input&subfolder=${subfolder}${app.getPreviewFormatParam()}`);
|
||||
|
||||
|
||||
const regex = /\.gif&type/;
|
||||
|
||||
let prev_format = "mp4"
|
||||
if (document.getElementById("mediaContainer").tagName=="IMG"){
|
||||
prev_format="gif";
|
||||
}
|
||||
let current_format = "mp4"
|
||||
if (regex.test(url_video)){
|
||||
current_format="gif";
|
||||
}
|
||||
|
||||
if (prev_format == current_format) {
|
||||
//update
|
||||
|
||||
console.log(videoWidget.inputEl)//..getElementById("mediaContainer")
|
||||
//videoWidget.inputEl.children[1].src = url_video
|
||||
}
|
||||
else{
|
||||
let newElement;
|
||||
if (current_format=="gif"){
|
||||
|
||||
newElement = document.createElement("img");
|
||||
Object.assign(newElement, {
|
||||
id:"mediaContainer",
|
||||
src: url_video,
|
||||
style: "width: 100%; height: 100%;",
|
||||
type : "image/gif"
|
||||
})
|
||||
}
|
||||
else{
|
||||
newElement = document.createElement("video");
|
||||
// Set the video attributes
|
||||
Object.assign(newElement, {
|
||||
id:"mediaContainer",
|
||||
controls: true,
|
||||
src: url_video,
|
||||
poster: "",
|
||||
style: "width: 100%; height: 100%;",
|
||||
loop: true,
|
||||
muted: true,
|
||||
autoplay:true,
|
||||
type : "video/mp4"
|
||||
|
||||
});
|
||||
}
|
||||
|
||||
let newEl= document.createElement("div");
|
||||
Object.assign(newEl, {
|
||||
id: "videoContainer",
|
||||
width: 400,
|
||||
height: 300
|
||||
})
|
||||
|
||||
newEl.appendChild(newElement);
|
||||
|
||||
videoWidget.lastChilds = newEl;
|
||||
console.log(videoWidget)
|
||||
console.log("ssssssssss")
|
||||
//document.getElementById("videoContainer")
|
||||
//videoWidget.inputEl.children[0].replaceChild(newElement,videoWidget.inputEl.children[1]);
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
}
|
||||
|
||||
|
||||
temp_web_url.value = url_video
|
||||
}
|
||||
|
||||
export function showVideoOutput(name, node) {
|
||||
const url = buildViewUrl(name, "output", "n-suite/videos");
|
||||
updateVideoWidget(node, "videoOutWidget", url);
|
||||
return url;
|
||||
export function showVideoOutput(name,node) {
|
||||
const videoWidget = node.widgets.find((w) => w.name === "videoOutWidget");
|
||||
console.log(name)
|
||||
|
||||
|
||||
let folder_separator = name.lastIndexOf("/");
|
||||
let subfolder = "videos";
|
||||
if (folder_separator > -1) {
|
||||
subfolder = name.substring(0, folder_separator);
|
||||
name = name.substring(folder_separator + 1);
|
||||
}
|
||||
|
||||
|
||||
let url_video = api.apiURL(`/view?filename=${encodeURIComponent(name)}&type=output&subfolder=${subfolder}${app.getPreviewFormatParam()}`);
|
||||
videoWidget.inputEl.src = url_video
|
||||
|
||||
return url_video;
|
||||
}
|
||||
|
||||
|
||||
|
||||
export const ExtendedComfyWidgets = {
|
||||
...ComfyWidgets,
|
||||
VIDEO(node, inputName, _inputData, src, _app, type = "input", autoplayValue = true) {
|
||||
const result = addVideo(node, inputName, src, autoplayValue);
|
||||
if (type !== "input") return result;
|
||||
...ComfyWidgets, // Copy all the functions from ComfyWidgets
|
||||
|
||||
VIDEO(node, inputName, inputData, src, app,type="input") {
|
||||
try {
|
||||
|
||||
const videoWidget = node.widgets.find((w) => w.name === "video");
|
||||
const defaultVal = "";
|
||||
let res;
|
||||
res = addVideo(node, inputName, src, app);
|
||||
|
||||
if (type == "input"){
|
||||
|
||||
const video = node.widgets?.find((item) => item.name === "video");
|
||||
const autoplay = node.widgets?.find((item) => item.name === "autoplay");
|
||||
if (video) {
|
||||
const callback = video.callback;
|
||||
video.callback = function () {
|
||||
showVideoInput(video.value, node);
|
||||
return callback?.apply(this, arguments);
|
||||
const cb = node.callback;
|
||||
videoWidget.callback = function () {
|
||||
showVideoInput(videoWidget.value, node);
|
||||
if (cb) {
|
||||
return cb.apply(this, arguments);
|
||||
}
|
||||
};
|
||||
}
|
||||
if (autoplay) {
|
||||
const callback = autoplay.callback;
|
||||
autoplay.callback = function () {
|
||||
const preview = node.widgets?.find((item) => item.name === "videoWidget");
|
||||
if (preview?.element) preview.element.autoplay = autoplay.value;
|
||||
if (video?.value) showVideoInput(video.value, node);
|
||||
return callback?.apply(this, arguments);
|
||||
};
|
||||
}
|
||||
return result;
|
||||
},
|
||||
|
||||
|
||||
if (node.type =="VideoLoader"){
|
||||
// do this only on VideoLoad node!
|
||||
let uploadWidget;
|
||||
const fileInput = document.createElement("input");
|
||||
Object.assign(fileInput, {
|
||||
type: "file",
|
||||
accept: "video/mp4,image/gif",
|
||||
style: "display: none",
|
||||
onchange: async () => {
|
||||
if (fileInput.files.length) {
|
||||
await uploadFile(fileInput.files[0], true,node);
|
||||
}
|
||||
},
|
||||
});
|
||||
document.body.append(fileInput);
|
||||
// Create the button widget for selecting the files
|
||||
uploadWidget = node.addWidget("button", "choose file to upload", "image", () => {
|
||||
fileInput.click();
|
||||
});
|
||||
uploadWidget.serialize = false;
|
||||
|
||||
}
|
||||
|
||||
return res;
|
||||
}
|
||||
catch (error) {
|
||||
|
||||
console.error("Errore in extended_widgets.js:", error);
|
||||
throw error;
|
||||
|
||||
}
|
||||
|
||||
},
|
||||
|
||||
|
||||
};
|
||||
|
||||
@@ -3,26 +3,41 @@ import { ComfyWidgets } from "/scripts/widgets.js";
|
||||
|
||||
app.registerExtension({
|
||||
name: "n.GPTSampler",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData) {
|
||||
if (nodeData.name !== "GPT Sampler [n-suite]") return;
|
||||
async beforeRegisterNodeDef(nodeType, nodeData, app) {
|
||||
|
||||
if (nodeData.name === "GPTSampler") {
|
||||
console.warn("GPTSampler detected")
|
||||
|
||||
const onExecuted = nodeType.prototype.onExecuted;
|
||||
|
||||
nodeType.prototype.onExecuted = function (message) {
|
||||
onExecuted?.apply(this, arguments);
|
||||
|
||||
const pos_cached = this.widgets.findIndex((w) => w.name === "cached");
|
||||
console.warn("value:"+pos_cached)
|
||||
|
||||
if (this.widgets) {
|
||||
const pos_text = this.widgets.findIndex((w) => w.name === "text");
|
||||
if (pos_text !== -1) {
|
||||
for (let i = pos_text; i < this.widgets.length; i++) {
|
||||
this.widgets[i].onRemove?.();
|
||||
}
|
||||
this.widgets.length = pos_text;
|
||||
}
|
||||
}
|
||||
|
||||
const onExecuted = nodeType.prototype.onExecuted;
|
||||
nodeType.prototype.onExecuted = function () {
|
||||
onExecuted?.apply(this, arguments);
|
||||
const widgets = this.widgets ?? [];
|
||||
const cached = widgets.find((widget) => widget.name === "cached");
|
||||
for (const widget of widgets.filter((item) => item.name === "text")) {
|
||||
this.removeWidget(widget);
|
||||
}
|
||||
if (cached?.value === "NO") {
|
||||
const result = ComfyWidgets.STRING(
|
||||
this,
|
||||
"text",
|
||||
["STRING", { multiline: true }],
|
||||
app,
|
||||
);
|
||||
result.widget.value = Math.floor(Math.random() * 10000);
|
||||
}
|
||||
};
|
||||
|
||||
if (this.widgets[pos_cached].value === "NO") {
|
||||
|
||||
const w = ComfyWidgets["STRING"](this, "text", ["STRING", { multiline: true }], app);
|
||||
//random seed
|
||||
var rnm = Math.floor(Math.random() * 10000)
|
||||
w.widget.value = rnm;
|
||||
|
||||
|
||||
}
|
||||
|
||||
};
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
@@ -1,18 +0,0 @@
|
||||
function addStylesheet(url) {
|
||||
if (url.endsWith(".js")) {
|
||||
url = url.substr(0, url.length - 2) + "css";
|
||||
}
|
||||
const link = document.createElement("link");
|
||||
link.rel = "stylesheet";
|
||||
link.href = url.startsWith("http") ? url : getUrl(url);
|
||||
document.head.append(link);
|
||||
}
|
||||
function getUrl(path, baseUrl) {
|
||||
if (baseUrl) {
|
||||
return new URL(path, baseUrl).toString();
|
||||
} else {
|
||||
return new URL("../" + path, import.meta.url).toString();
|
||||
}
|
||||
}
|
||||
|
||||
addStylesheet(getUrl("styles.css", import.meta.url));
|
||||
@@ -1,17 +0,0 @@
|
||||
import { app } from "/scripts/app.js";
|
||||
import { api } from "/scripts/api.js";
|
||||
|
||||
app.registerExtension({
|
||||
name: "n.ModelDownloadNotice",
|
||||
setup() {
|
||||
api.addEventListener("n-suite-model-download", (event) => {
|
||||
const { model, message } = event.detail;
|
||||
app.extensionManager.toast.add({
|
||||
severity: "info",
|
||||
summary: `${model} download started`,
|
||||
detail: message,
|
||||
life: 15000,
|
||||
});
|
||||
});
|
||||
},
|
||||
});
|
||||
@@ -1,20 +0,0 @@
|
||||
textarea[placeholder="positive_prompt"] {
|
||||
border: 1px solid #64d509;
|
||||
|
||||
|
||||
}
|
||||
textarea[placeholder="positive_prompt"]:focus-visible {
|
||||
border: 1px solid #72eb0f;
|
||||
|
||||
}
|
||||
|
||||
|
||||
textarea[placeholder="negative_prompt"] {
|
||||
border: 1px solid #a94442;
|
||||
border-color: #a94442;
|
||||
}
|
||||
|
||||
textarea[placeholder="negative_prompt"]:focus-visible {
|
||||
border: 1px solid #de5755;
|
||||
border-color: #de5755;
|
||||
}
|
||||
@@ -0,0 +1,122 @@
|
||||
import { app } from "/scripts/app.js";
|
||||
import { api } from "/scripts/api.js"
|
||||
import { ExtendedComfyWidgets,showVideoInput } from "./extended_widgets.js";
|
||||
const MultilineSymbol = Symbol();
|
||||
const MultilineResizeSymbol = Symbol();
|
||||
|
||||
|
||||
async function uploadFile(file, updateNode, node, pasted = false) {
|
||||
const videoWidget = node.widgets.find((w) => w.name === "video");
|
||||
|
||||
|
||||
try {
|
||||
// Wrap file in formdata so it includes filename
|
||||
const body = new FormData();
|
||||
body.append("image", file);
|
||||
if (pasted) {
|
||||
body.append("subfolder", "pasted");
|
||||
}
|
||||
else {
|
||||
body.append("subfolder", "n-suite");
|
||||
}
|
||||
|
||||
const resp = await api.fetchApi("/upload/image", {
|
||||
method: "POST",
|
||||
body,
|
||||
});
|
||||
|
||||
if (resp.status === 200) {
|
||||
const data = await resp.json();
|
||||
// Add the file to the dropdown list and update the widget value
|
||||
let path = data.name;
|
||||
|
||||
|
||||
if (!videoWidget.options.values.includes(path)) {
|
||||
videoWidget.options.values.push(path);
|
||||
}
|
||||
|
||||
if (updateNode) {
|
||||
|
||||
videoWidget.value = path;
|
||||
if (data.subfolder) path = data.subfolder + "/" + path;
|
||||
showVideoInput(path,node);
|
||||
|
||||
}
|
||||
} else {
|
||||
alert(resp.status + " - " + resp.statusText);
|
||||
}
|
||||
} catch (error) {
|
||||
alert(error);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
let uploadWidget = "";
|
||||
app.registerExtension({
|
||||
name: "Comfy.VideoLoad",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData, app) {
|
||||
|
||||
const onAdded = nodeType.prototype.onAdded;
|
||||
if (nodeData.name === "VideoLoader") {
|
||||
|
||||
|
||||
|
||||
nodeType.prototype.onAdded = function () {
|
||||
onAdded?.apply(this, arguments);
|
||||
const temp_web_url = this.widgets.find((w) => w.name === "local_url");
|
||||
|
||||
|
||||
setTimeout(() => {
|
||||
ExtendedComfyWidgets["VIDEO"](this, "videoWidget", ["STRING"], temp_web_url.value, app);
|
||||
}, 100);
|
||||
|
||||
|
||||
}
|
||||
|
||||
|
||||
nodeType.prototype.onDragOver = function (e) {
|
||||
if (e.dataTransfer && e.dataTransfer.items) {
|
||||
const image = [...e.dataTransfer.items].find((f) => f.kind === "file");
|
||||
return !!image;
|
||||
}
|
||||
|
||||
return false;
|
||||
};
|
||||
|
||||
// On drop upload files
|
||||
nodeType.prototype.onDragDrop = function (e) {
|
||||
console.log("onDragDrop called");
|
||||
let handled = false;
|
||||
for (const file of e.dataTransfer.files) {
|
||||
if (file.type.startsWith("video/mp4") || file.type.startsWith("image/gif")) {
|
||||
|
||||
const filePath = file.path || (file.webkitRelativePath || '').split('/').slice(1).join('/');
|
||||
|
||||
|
||||
uploadFile(file, !handled,this ); // Dont await these, any order is fine, only update on first one
|
||||
|
||||
handled = true;
|
||||
}
|
||||
}
|
||||
|
||||
return handled;
|
||||
};
|
||||
|
||||
nodeType.prototype.pasteFile = function(file) {
|
||||
if (file.type.startsWith("image/")) {
|
||||
const is_pasted = (file.name === "image.png") &&
|
||||
(file.lastModified - Date.now() < 2000);
|
||||
//uploadFile(file, true, is_pasted);
|
||||
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
|
||||
};
|
||||
|
||||
},
|
||||
});
|
||||
@@ -1,96 +0,0 @@
|
||||
import { app } from "/scripts/app.js";
|
||||
import { api } from "/scripts/api.js";
|
||||
import { ExtendedComfyWidgets, showVideoInput } from "./extended_widgets.js";
|
||||
|
||||
const VIDEO_TYPES = new Set(["video/mp4", "video/webm", "image/gif"]);
|
||||
|
||||
async function uploadFile(file, updateNode, node, pasted = false) {
|
||||
const videoWidget = node.widgets?.find((widget) => widget.name === "video");
|
||||
if (!videoWidget) return false;
|
||||
|
||||
try {
|
||||
const body = new FormData();
|
||||
body.append("image", file);
|
||||
body.append("subfolder", pasted ? "pasted" : "n-suite");
|
||||
const response = await api.fetchApi("/upload/image", { method: "POST", body });
|
||||
if (!response.ok) {
|
||||
alert(`${response.status} - ${response.statusText}`);
|
||||
return false;
|
||||
}
|
||||
|
||||
const data = await response.json();
|
||||
const value = data.name;
|
||||
const previewPath = data.subfolder ? `${data.subfolder}/${value}` : value;
|
||||
if (!videoWidget.options.values.includes(value)) videoWidget.options.values.push(value);
|
||||
if (updateNode) {
|
||||
const oldValue = videoWidget.value;
|
||||
videoWidget.value = value;
|
||||
videoWidget.callback?.(value);
|
||||
node.onWidgetChanged?.(videoWidget.name, value, oldValue, videoWidget);
|
||||
showVideoInput(previewPath, node);
|
||||
}
|
||||
return true;
|
||||
} catch (error) {
|
||||
console.error("N-Suite video upload failed", error);
|
||||
alert(String(error));
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
app.registerExtension({
|
||||
name: "Comfy.VideoLoadAdvanced",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData) {
|
||||
if (nodeData.name !== "LoadVideo [n-suite]") return;
|
||||
|
||||
const onAdded = nodeType.prototype.onAdded;
|
||||
const onRemoved = nodeType.prototype.onRemoved;
|
||||
nodeType.prototype.onAdded = function () {
|
||||
onAdded?.apply(this, arguments);
|
||||
const localUrl = this.widgets?.find((widget) => widget.name === "local_url");
|
||||
const autoplay = this.widgets?.find((widget) => widget.name === "autoplay");
|
||||
const fileInput = document.createElement("input");
|
||||
fileInput.type = "file";
|
||||
fileInput.accept = "video/mp4,video/webm,image/gif";
|
||||
fileInput.hidden = true;
|
||||
fileInput.onchange = async () => {
|
||||
if (fileInput.files?.length) await uploadFile(fileInput.files[0], true, this);
|
||||
};
|
||||
document.body.append(fileInput);
|
||||
this.__nSuiteVideoFileInput = fileInput;
|
||||
|
||||
const uploadWidget = this.addWidget("button", "choose file to upload", "image", () => fileInput.click());
|
||||
uploadWidget.serialize = false;
|
||||
ExtendedComfyWidgets.VIDEO(
|
||||
this,
|
||||
"videoWidget",
|
||||
["STRING"],
|
||||
localUrl?.value ?? "",
|
||||
app,
|
||||
"input",
|
||||
autoplay?.value ?? true,
|
||||
);
|
||||
};
|
||||
nodeType.prototype.onRemoved = function () {
|
||||
this.__nSuiteVideoFileInput?.remove();
|
||||
delete this.__nSuiteVideoFileInput;
|
||||
onRemoved?.apply(this, arguments);
|
||||
};
|
||||
nodeType.prototype.onDragOver = function (event) {
|
||||
return [...(event.dataTransfer?.items ?? [])].some((item) => item.kind === "file");
|
||||
};
|
||||
nodeType.prototype.onDragDrop = function (event) {
|
||||
let handled = false;
|
||||
for (const file of event.dataTransfer?.files ?? []) {
|
||||
if (!VIDEO_TYPES.has(file.type)) continue;
|
||||
uploadFile(file, !handled, this);
|
||||
handled = true;
|
||||
}
|
||||
return handled;
|
||||
};
|
||||
nodeType.prototype.pasteFile = function (file) {
|
||||
if (!VIDEO_TYPES.has(file.type)) return false;
|
||||
uploadFile(file, true, this, true);
|
||||
return true;
|
||||
};
|
||||
},
|
||||
});
|
||||
@@ -1,22 +1,87 @@
|
||||
import { app } from "/scripts/app.js";
|
||||
import { ExtendedComfyWidgets, showVideoOutput } from "./extended_widgets.js";
|
||||
import { api } from "/scripts/api.js"
|
||||
import { ExtendedComfyWidgets,showVideoOutput } from "./extended_widgets.js";
|
||||
const MultilineSymbol = Symbol();
|
||||
const MultilineResizeSymbol = Symbol();
|
||||
|
||||
|
||||
async function uploadFile(file, updateNode, node, pasted = false) {
|
||||
const videoWidget = node.widgets.find((w) => w.name === "video");
|
||||
|
||||
|
||||
try {
|
||||
// Wrap file in formdata so it includes filename
|
||||
const body = new FormData();
|
||||
body.append("image", file);
|
||||
if (pasted) {
|
||||
body.append("subfolder", "pasted");
|
||||
}
|
||||
else {
|
||||
body.append("subfolder", "n-suite");
|
||||
}
|
||||
|
||||
const resp = await api.fetchApi("/upload/image", {
|
||||
method: "POST",
|
||||
body,
|
||||
});
|
||||
|
||||
if (resp.status === 200) {
|
||||
const data = await resp.json();
|
||||
// Add the file to the dropdown list and update the widget value
|
||||
let path = data.name;
|
||||
|
||||
|
||||
if (!videoWidget.options.values.includes(path)) {
|
||||
videoWidget.options.values.push(path);
|
||||
}
|
||||
|
||||
if (updateNode) {
|
||||
// showVideo(path,node);
|
||||
videoWidget.value = path;
|
||||
if (data.subfolder) path = data.subfolder + "/" + path;
|
||||
showVideo(path,node);
|
||||
|
||||
}
|
||||
} else {
|
||||
alert(resp.status + " - " + resp.statusText);
|
||||
}
|
||||
} catch (error) {
|
||||
alert(error);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
let uploadWidget = "";
|
||||
app.registerExtension({
|
||||
name: "Comfy.VideoSave",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData) {
|
||||
if (nodeData.name !== "SaveVideo [n-suite]") return;
|
||||
async beforeRegisterNodeDef(nodeType, nodeData, app) {
|
||||
|
||||
const onAdded = nodeType.prototype.onAdded;
|
||||
const onExecuted = nodeType.prototype.onExecuted;
|
||||
|
||||
|
||||
const onAdded = nodeType.prototype.onAdded;
|
||||
if (nodeData.name === "VideoSaver") {
|
||||
nodeType.prototype.onAdded = function () {
|
||||
onAdded?.apply(this, arguments);
|
||||
ExtendedComfyWidgets.VIDEO(this, "videoOutWidget", ["STRING"], "", app, "output");
|
||||
|
||||
ExtendedComfyWidgets["VIDEO"](this, "videoOutWidget", ["STRING"], "", app,"output");
|
||||
|
||||
};
|
||||
nodeType.prototype.onExecuted = function (message) {
|
||||
onExecuted?.apply(this, arguments);
|
||||
const paths = message?.text?.flat?.(Infinity) ?? message?.text ?? [];
|
||||
const fullPath = Array.isArray(paths) ? paths.at(-1) : paths;
|
||||
if (fullPath) showVideoOutput(fullPath, this);
|
||||
console.log(nodeData)
|
||||
|
||||
let full_path="";
|
||||
|
||||
for (const list of message.text) {
|
||||
full_path = list;
|
||||
}
|
||||
|
||||
let fullweb= showVideoOutput(full_path,this)
|
||||
|
||||
}
|
||||
};
|
||||
|
||||
},
|
||||
});
|
||||
|
||||
@@ -1,43 +0,0 @@
|
||||
import sys
|
||||
import os
|
||||
|
||||
def migrate_workflow(input_file_path):
|
||||
try:
|
||||
file_name, file_extension = os.path.splitext(input_file_path)
|
||||
|
||||
output_file_path = f"{file_name}_migrated.json"
|
||||
|
||||
pre_list = ('LoadVideo', 'SaveVideo','FrameInterpolator', 'LoadFramesFromFolder','SetMetadataForSaveVideo','GPT Loader Simple','GPTSampler','String Variable','Integer Variable','Float Variable','DynamicPrompt')
|
||||
post_list= ('LoadVideo [n-suite]', 'SaveVideo [n-suite]','FrameInterpolator [n-suite]', 'LoadFramesFromFolder [n-suite]','SetMetadataForSaveVideo [n-suite]','GPT Loader Simple [n-suite]','GPT Sampler [n-suite]','String Variable [n-suite]','Integer Variable [n-suite]','Float Variable [n-suite]','DynamicPrompt [n-suite]')
|
||||
replacements = list(zip(pre_list, post_list))
|
||||
|
||||
with open(input_file_path, 'r') as input_file:
|
||||
content = input_file.read()
|
||||
|
||||
# s&r
|
||||
for old, new in replacements:
|
||||
content = content.replace(f'"Node name for S&R": "{old}"', f'"Node name for S&R": "{new}"')
|
||||
#type
|
||||
for old, new in replacements:
|
||||
content = content.replace(f'"type": "{old}"', f'"type": "{new}"')
|
||||
|
||||
with open(output_file_path, 'w') as output_file:
|
||||
output_file.write(content)
|
||||
|
||||
print("Replacement completed successfully.")
|
||||
except Exception as e:
|
||||
print(f"An error occurred: {str(e)}")
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(len(sys.argv))
|
||||
if len(sys.argv) != 2:
|
||||
print("Error: Provide the path of the text file to migrate.")
|
||||
sys.exit(1)
|
||||
|
||||
file_path = sys.argv[1]
|
||||
|
||||
if not os.path.isfile(file_path):
|
||||
print(f"Error: The file {file_path} does not exist.")
|
||||
sys.exit(1)
|
||||
|
||||
migrate_workflow(file_path)
|
||||
@@ -1,21 +0,0 @@
|
||||
@echo off
|
||||
setlocal
|
||||
|
||||
rem Check if the file path is provided
|
||||
if "%1"=="" (
|
||||
echo Error: provide the path of the text file to migrate.
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
rem Check if the file exists
|
||||
if not exist "%1" (
|
||||
echo Error: the file %1 does not exist.
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
rem Run the Python script to migrate the file
|
||||
python %~dp0libs\migrate.py "%~f1"
|
||||
|
||||
echo Replacement completed successfully.
|
||||
|
||||
pause
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"name": "CustomScripts",
|
||||
"logging": false
|
||||
}
|
||||
@@ -1,96 +1,14 @@
|
||||
import asyncio
|
||||
import os
|
||||
import json
|
||||
import shutil
|
||||
import inspect
|
||||
import aiohttp
|
||||
from server import PromptServer
|
||||
from tqdm import tqdm
|
||||
import requests
|
||||
import folder_paths
|
||||
|
||||
|
||||
|
||||
config = None
|
||||
|
||||
class color:
|
||||
END = '\33[0m'
|
||||
BOLD = '\33[1m'
|
||||
ITALIC = '\33[3m'
|
||||
UNDERLINE = '\33[4m'
|
||||
BLINK = '\33[5m'
|
||||
BLINK2 = '\33[6m'
|
||||
SELECTED = '\33[7m'
|
||||
|
||||
BLACK = '\33[30m'
|
||||
RED = '\33[31m'
|
||||
GREEN = '\33[32m'
|
||||
YELLOW = '\33[33m'
|
||||
BLUE = '\33[34m'
|
||||
VIOLET = '\33[35m'
|
||||
BEIGE = '\33[36m'
|
||||
WHITE = '\33[37m'
|
||||
|
||||
BLACKBG = '\33[40m'
|
||||
REDBG = '\33[41m'
|
||||
GREENBG = '\33[42m'
|
||||
YELLOWBG = '\33[43m'
|
||||
BLUEBG = '\33[44m'
|
||||
VIOLETBG = '\33[45m'
|
||||
BEIGEBG = '\33[46m'
|
||||
WHITEBG = '\33[47m'
|
||||
|
||||
GREY = '\33[90m'
|
||||
LIGHTRED = '\33[91m'
|
||||
LIGHTGREEN = '\33[92m'
|
||||
LIGHTYELLOW = '\33[93m'
|
||||
LIGHTBLUE = '\33[94m'
|
||||
LIGHTVIOLET = '\33[95m'
|
||||
LIGHTBEIGE = '\33[96m'
|
||||
LIGHTWHITE = '\33[97m'
|
||||
|
||||
GREYBG = '\33[100m'
|
||||
LIGHTREDBG = '\33[101m'
|
||||
LIGHTGREENBG = '\33[102m'
|
||||
LIGHTYELLOWBG = '\33[103m'
|
||||
LIGHTBLUEBG = '\33[104m'
|
||||
LIGHTVIOLETBG = '\33[105m'
|
||||
LIGHTBEIGEBG = '\33[106m'
|
||||
LIGHTWHITEBG = '\33[107m'
|
||||
|
||||
|
||||
def get_commit():
|
||||
try:
|
||||
import git
|
||||
repo = git.Repo(get_ext_dir())
|
||||
return repo.head.object.hexsha[:8]
|
||||
except:
|
||||
return 0
|
||||
|
||||
import zipfile
|
||||
|
||||
|
||||
def downloader(link):
|
||||
print("Downloading dependencies...")
|
||||
response = requests.get(link, stream=True)
|
||||
try:
|
||||
os.makedirs(folder_paths.get_temp_directory())
|
||||
except:
|
||||
pass
|
||||
temp_file = os.path.join(folder_paths.get_temp_directory(), "file.zip")
|
||||
with open(temp_file, "wb") as f:
|
||||
for chunk in response.iter_content(chunk_size=1024):
|
||||
if chunk:
|
||||
f.write(chunk)
|
||||
|
||||
zip_file = zipfile.ZipFile(temp_file)
|
||||
target_dir = get_ext_dir(os.path.join("libs", "rifle"))
|
||||
|
||||
zip_file.extractall(target_dir)
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
def is_logging_enabled():
|
||||
config = get_extension_config()
|
||||
if "logging" not in config:
|
||||
@@ -108,7 +26,7 @@ def log(message, type=None, always=False, name=None):
|
||||
if name is None:
|
||||
name = get_extension_config()["name"]
|
||||
|
||||
print(f"{name}: {message}")
|
||||
print(f"(nnodes:{name}) {message}")
|
||||
|
||||
|
||||
def get_ext_dir(subpath=None, mkdir=False):
|
||||
@@ -135,15 +53,24 @@ def get_comfy_dir(subpath=None, mkdir=False):
|
||||
return dir
|
||||
|
||||
|
||||
def get_web_ext_dir():
|
||||
config = get_extension_config()
|
||||
name = config["name"]
|
||||
dir = get_comfy_dir("web/extensions/comfyui-n-nodes")
|
||||
if not os.path.exists(dir):
|
||||
os.makedirs(dir)
|
||||
dir = os.path.join(dir, name)
|
||||
return dir
|
||||
|
||||
def get_extension_config(reload=False):
|
||||
global config
|
||||
if reload == False and config is not None:
|
||||
return config
|
||||
|
||||
config_path = get_ext_dir("config.json")
|
||||
config_path = get_ext_dir("nnodes.json")
|
||||
if not os.path.exists(config_path):
|
||||
log("Missing config.json, this extension may not work correctly. Please reinstall the extension.", type="ERROR", always=True, name="???")
|
||||
log("Missing nnodes.json, this extension may not work correctly. Please reinstall the extension.",
|
||||
type="ERROR", always=True, name="???")
|
||||
print(f"Extension path: {get_ext_dir()}")
|
||||
return {"name": "Unknown", "version": -1}
|
||||
with open(config_path, "r") as f:
|
||||
@@ -151,6 +78,53 @@ def get_extension_config(reload=False):
|
||||
return config
|
||||
|
||||
|
||||
|
||||
def link_js(src, dst):
|
||||
src = os.path.abspath(src)
|
||||
dst = os.path.abspath(dst)
|
||||
if os.name == "nt":
|
||||
try:
|
||||
import _winapi
|
||||
_winapi.CreateJunction(src, dst)
|
||||
return True
|
||||
except:
|
||||
pass
|
||||
try:
|
||||
os.symlink(src, dst)
|
||||
return True
|
||||
except:
|
||||
import logging
|
||||
logging.exception('')
|
||||
return False
|
||||
|
||||
def is_junction(path):
|
||||
if os.name != "nt":
|
||||
return False
|
||||
try:
|
||||
return bool(os.readlink(path))
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
def install_js():
|
||||
src_dir = get_ext_dir("js")
|
||||
if not os.path.exists(src_dir):
|
||||
log("No JS")
|
||||
return
|
||||
|
||||
dst_dir = get_web_ext_dir()
|
||||
|
||||
if os.path.exists(dst_dir):
|
||||
if os.path.islink(dst_dir) or is_junction(dst_dir):
|
||||
log("JS already linked")
|
||||
return
|
||||
elif link_js(src_dir, dst_dir):
|
||||
log("JS linked")
|
||||
return
|
||||
|
||||
log("Copying JS files")
|
||||
shutil.copytree(src_dir, dst_dir, dirs_exist_ok=True)
|
||||
|
||||
|
||||
def init(check_imports=None):
|
||||
log("Init")
|
||||
|
||||
@@ -163,7 +137,7 @@ def init(check_imports=None):
|
||||
type="ERROR", always=True)
|
||||
return False
|
||||
|
||||
|
||||
install_js()
|
||||
return True
|
||||
|
||||
|
||||
@@ -216,6 +190,8 @@ async def download_to_file(url, destination, update_callback=None, is_ext_subpat
|
||||
download(url, f, update_callback, session)
|
||||
|
||||
|
||||
|
||||
|
||||
def is_inside_dir(root_dir, check_path):
|
||||
root_dir = os.path.abspath(root_dir)
|
||||
if not os.path.isabs(check_path):
|
||||
|
||||
@@ -1,390 +0,0 @@
|
||||
import torch
|
||||
import numpy as np
|
||||
from PIL import Image, ImageOps, ImageEnhance
|
||||
import cv2
|
||||
MAX_RESOLUTION = 4096
|
||||
|
||||
|
||||
def tensor2pil(image):
|
||||
return Image.fromarray(np.clip(255. * image.cpu().numpy().squeeze(), 0, 255).astype(np.uint8))
|
||||
|
||||
# Adapt from https://github.com/sipherxyz/comfyui-art-venture
|
||||
def color_correct(
|
||||
image,
|
||||
temperature: float,
|
||||
hue: float,
|
||||
brightness: float,
|
||||
contrast: float,
|
||||
saturation: float,
|
||||
gamma: float,
|
||||
):
|
||||
|
||||
|
||||
|
||||
brightness /= 100
|
||||
contrast /= 100
|
||||
saturation /= 100
|
||||
temperature /= 100
|
||||
|
||||
brightness = 1 + brightness
|
||||
contrast = 1 + contrast
|
||||
saturation = 1 + saturation
|
||||
|
||||
|
||||
modified_image = image
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
# brightness
|
||||
modified_image = ImageEnhance.Brightness(modified_image).enhance(brightness)
|
||||
|
||||
# contrast
|
||||
modified_image = ImageEnhance.Contrast(modified_image).enhance(contrast)
|
||||
modified_image = np.array(modified_image).astype(np.float32)
|
||||
|
||||
# temperature
|
||||
if temperature > 0:
|
||||
modified_image[:, :, 0] *= 1 + temperature
|
||||
modified_image[:, :, 1] *= 1 + temperature * 0.4
|
||||
elif temperature < 0:
|
||||
modified_image[:, :, 2] *= 1 - temperature
|
||||
modified_image = np.clip(modified_image, 0, 255) / 255
|
||||
|
||||
# gamma
|
||||
modified_image = np.clip(np.power(modified_image, gamma), 0, 1)
|
||||
|
||||
# saturation
|
||||
hls_img = cv2.cvtColor(modified_image, cv2.COLOR_RGB2HLS)
|
||||
hls_img[:, :, 2] = np.clip(saturation * hls_img[:, :, 2], 0, 1)
|
||||
modified_image = cv2.cvtColor(hls_img, cv2.COLOR_HLS2RGB) * 255
|
||||
|
||||
# hue
|
||||
hsv_img = cv2.cvtColor(modified_image, cv2.COLOR_RGB2HSV)
|
||||
hsv_img[:, :, 0] = (hsv_img[:, :, 0] + hue) % 360
|
||||
modified_image = cv2.cvtColor(hsv_img, cv2.COLOR_HSV2RGB)
|
||||
|
||||
|
||||
|
||||
modified_image = modified_image.astype(np.uint8)
|
||||
#modified_image = modified_image / 255
|
||||
|
||||
#modified_image = torch.from_numpy(modified_image).unsqueeze(0)
|
||||
|
||||
|
||||
return modified_image
|
||||
|
||||
|
||||
|
||||
|
||||
def extract_pixels(image, side, num_pixels):
|
||||
|
||||
|
||||
# Ottieni le dimensioni dell'immagine
|
||||
width, height = image.size
|
||||
|
||||
# Determina la regione di ritaglio in base al lato specificato
|
||||
if side == "l":
|
||||
crop_box = (0, 0, num_pixels, height)
|
||||
elif side == "r":
|
||||
crop_box = (width - num_pixels, 0, width, height)
|
||||
elif side == "t":
|
||||
crop_box = (0, 0, width, num_pixels)
|
||||
elif side == "b":
|
||||
crop_box = (0, height - num_pixels, width, height)
|
||||
else:
|
||||
raise ValueError("Il lato specificato non è valido. Utilizzare 'sinistro', 'destro', 'alto' o 'basso'.")
|
||||
|
||||
# Esegui il ritaglio dell'immagine
|
||||
cropped_image = image.crop(crop_box)
|
||||
|
||||
return cropped_image
|
||||
|
||||
|
||||
from PIL import Image
|
||||
|
||||
|
||||
|
||||
def make_pixelated(image, pixel_size):
|
||||
if pixel_size > image.width or pixel_size > image.height:
|
||||
raise ValueError("Top, bottom, left, and right padding must higher than the pixel_size!")
|
||||
|
||||
small_image = image.resize((image.width // pixel_size, image.height // pixel_size), Image.Resampling.NEAREST)
|
||||
|
||||
pixelated_image = small_image.resize(image.size, Image.Resampling.NEAREST)
|
||||
|
||||
return pixelated_image
|
||||
|
||||
|
||||
|
||||
def flip_and_stretch(image, flip_direction, stretch_value):
|
||||
|
||||
# Inverti l'immagine in base alla direzione specificata
|
||||
# Calcola le nuove dimensioni dell'immagine con stretching
|
||||
original_width, original_height = image.size
|
||||
|
||||
if flip_direction == "h":
|
||||
flipped_image = image.transpose(Image.Transpose.FLIP_LEFT_RIGHT)
|
||||
stretched_height = original_height
|
||||
stretched_width = stretch_value
|
||||
elif flip_direction == "v":
|
||||
flipped_image = image.transpose(Image.Transpose.FLIP_TOP_BOTTOM)
|
||||
stretched_width = original_width
|
||||
stretched_height = stretch_value
|
||||
else:
|
||||
raise ValueError("La direzione specificata non è valida. Utilizzare 'orizzontale' o 'verticale'.")
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
# "Stretcha" l'immagine alle nuove dimensioni
|
||||
stretched_image = flipped_image.resize((stretched_width, stretched_height))
|
||||
|
||||
return stretched_image
|
||||
|
||||
|
||||
def create_noise_image(width, height):
|
||||
|
||||
noise_array = np.random.randint(0, 256, (height, width, 3), dtype=np.uint8)
|
||||
|
||||
# Crea un'immagine PIL utilizzando i valori dei pixel generati
|
||||
noise_image = Image.fromarray(noise_array)
|
||||
|
||||
return noise_image
|
||||
|
||||
|
||||
def blend_images(image1, image2, blend_percentage):
|
||||
|
||||
|
||||
|
||||
# Assicurati che le due immagini abbiano le stesse dimensioni
|
||||
if image1.size != image2.size:
|
||||
raise ValueError("Le dimensioni delle due immagini devono essere uguali.")
|
||||
|
||||
# Blend delle due immagini in base alla percentuale specificata
|
||||
blended_image = Image.blend(image1, image2, blend_percentage)
|
||||
|
||||
return blended_image
|
||||
|
||||
|
||||
def image_paste(main_image, image_to_paste,side):
|
||||
|
||||
|
||||
# Ottieni le dimensioni delle immagini
|
||||
width_main, height_main = main_image.size
|
||||
width_paste, height_paste = image_to_paste.size
|
||||
|
||||
|
||||
|
||||
|
||||
# Calcola le coordinate di incollaggio in base alla posizione desiderata
|
||||
if side == "t":
|
||||
# Crea una nuova immagine che sarà la combinazione delle due immagini
|
||||
new_width = width_main
|
||||
new_height = height_main + height_paste
|
||||
|
||||
new_image = Image.new("RGB", (new_width, new_height))
|
||||
new_image.paste(image_to_paste, (0,0))
|
||||
new_image.paste(main_image, (0,height_paste))
|
||||
|
||||
elif side == "b":
|
||||
new_width = width_main
|
||||
new_height = height_main + height_paste
|
||||
|
||||
new_image = Image.new("RGB", (new_width, new_height))
|
||||
new_image.paste(image_to_paste, (0,height_main))
|
||||
new_image.paste(main_image, (0,0))
|
||||
|
||||
elif side == "r":
|
||||
new_width = width_main + width_paste
|
||||
new_height = height_main
|
||||
new_image = Image.new("RGB", (new_width, new_height))
|
||||
new_image.paste(image_to_paste, (width_main, 0))
|
||||
new_image.paste(main_image, (0, 0))
|
||||
|
||||
|
||||
elif side == "l":
|
||||
new_width = width_main + width_paste
|
||||
new_height = height_main
|
||||
|
||||
new_image = Image.new("RGB", (new_width, new_height))
|
||||
new_image.paste(image_to_paste, (0, 0))
|
||||
new_image.paste(main_image, (width_paste, 0))
|
||||
|
||||
|
||||
return new_image
|
||||
|
||||
|
||||
|
||||
def resize_image(image,noise,pixel_size, pixel_to_copy, left, right, top, bottom, temperature=5.0,hue=0,brightness=32,contrast=0,saturation=0,gamma=2):
|
||||
|
||||
|
||||
# RIGHT SIDE
|
||||
if right != 0:
|
||||
r_image = extract_pixels(image, "r", pixel_to_copy)
|
||||
r_image= flip_and_stretch(r_image, "h", right)
|
||||
r_image = make_pixelated(r_image, pixel_size)
|
||||
r_noise = create_noise_image(r_image.size[0], r_image.size[1])
|
||||
r_image = blend_images(r_image, r_noise, noise)
|
||||
r_image = color_correct(r_image, temperature,hue,brightness,contrast,saturation,gamma)
|
||||
r_image= Image.fromarray(r_image)
|
||||
|
||||
r_image = image_paste(image, r_image,"r")
|
||||
else:
|
||||
r_image = image
|
||||
|
||||
# LEFT SIDE
|
||||
if left != 0:
|
||||
l_image = extract_pixels(r_image, "l", pixel_to_copy)
|
||||
l_image = flip_and_stretch(l_image, "h", left)
|
||||
l_image = make_pixelated(l_image, pixel_size)
|
||||
l_noise = create_noise_image(l_image.size[0], l_image.size[1])
|
||||
l_image = blend_images(l_image, l_noise, noise)
|
||||
l_image = color_correct(l_image, temperature,hue,brightness,contrast,saturation,gamma)
|
||||
l_image= Image.fromarray(l_image)
|
||||
|
||||
l_image = image_paste(r_image, l_image,"l")
|
||||
else:
|
||||
l_image = r_image
|
||||
|
||||
# TOP
|
||||
if top != 0:
|
||||
t_image = extract_pixels(l_image, "t", pixel_to_copy)
|
||||
t_image = flip_and_stretch(t_image, "v", top)
|
||||
t_image = make_pixelated(t_image, pixel_size)
|
||||
t_noise = create_noise_image(t_image.size[0], t_image.size[1])
|
||||
t_image = blend_images(t_image, t_noise, noise)
|
||||
t_image = color_correct(t_image, temperature,hue,brightness,contrast,saturation,gamma)
|
||||
t_image= Image.fromarray(t_image)
|
||||
|
||||
t_image = image_paste(l_image, t_image,"t")
|
||||
else:
|
||||
t_image = l_image
|
||||
|
||||
# BOTTOM
|
||||
if bottom != 0:
|
||||
b_image = extract_pixels(t_image, "b", pixel_to_copy)
|
||||
b_image = flip_and_stretch(b_image, "v", bottom)
|
||||
b_image = make_pixelated(b_image, pixel_size)
|
||||
b_noise = create_noise_image(b_image.size[0], b_image.size[1])
|
||||
b_image = blend_images(b_image, b_noise, noise)
|
||||
b_image = color_correct(b_image, temperature,hue,brightness,contrast,saturation,gamma)
|
||||
b_image= Image.fromarray(b_image)
|
||||
|
||||
b_image = image_paste(t_image, b_image,"b")
|
||||
else:
|
||||
b_image = t_image
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
final_image = b_image
|
||||
|
||||
return final_image
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
class ImagePadForOutpaintAdvanced:
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {
|
||||
"required": {
|
||||
"image": ("IMAGE",),
|
||||
"left": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}),
|
||||
"top": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}),
|
||||
"right": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}),
|
||||
"bottom": ("INT", {"default": 0, "min": 0, "max": MAX_RESOLUTION, "step": 8}),
|
||||
"feathering": ("INT", {"default": 40, "min": 0, "max": MAX_RESOLUTION, "step": 1}),
|
||||
"noise": ("FLOAT", {"default": 0.1, "min": 0, "max": 1.0, "step": 0.01}),
|
||||
"pixel_size": ("INT", {"default": 8, "min": 8, "max": 64, "step": 8}),
|
||||
"pixel_to_copy": ("INT", {"default": 32, "min": 0, "max": MAX_RESOLUTION, "step": 1}),
|
||||
"temperature": ("FLOAT",{"default": 0, "min": -100, "max": 100, "step": 5},),
|
||||
"hue": ("FLOAT", {"default": 0, "min": -90, "max": 90, "step": 5}),
|
||||
"brightness": ("FLOAT",{"default": 0, "min": -100, "max": 100, "step": 5},),
|
||||
"contrast": ("FLOAT",{"default": 0, "min": -100, "max": 100, "step": 5},),
|
||||
"saturation": ("FLOAT",{"default": 0, "min": -100, "max": 100, "step": 5},),
|
||||
"gamma": ("FLOAT", {"default": 1, "min": 0.2, "max": 2.2, "step": 0.1}),
|
||||
},
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("IMAGE", "MASK")
|
||||
FUNCTION = "expand_image"
|
||||
|
||||
CATEGORY = "image"
|
||||
|
||||
|
||||
def expand_image(self,image,feathering,noise,pixel_size, pixel_to_copy, left, right, top, bottom, temperature=5.0,hue=0,brightness=32,contrast=0,saturation=0,gamma=2):
|
||||
d1, d2, d3, d4 = image.size()
|
||||
|
||||
|
||||
#new_image = torch.zeros(
|
||||
# (d1, d2 + top + bottom, d3 + left + right, d4),
|
||||
# dtype=torch.float32,
|
||||
#)
|
||||
#new_image[:, top:top + d2, left:left + d3, :] = image
|
||||
|
||||
image = tensor2pil(image)
|
||||
#image = Image.fromarray(image.astype(np.uint8))
|
||||
|
||||
new_image = resize_image(image,noise,pixel_size, pixel_to_copy, left, right, top, bottom, temperature,hue,brightness,contrast,saturation,gamma)
|
||||
|
||||
|
||||
|
||||
i = ImageOps.exif_transpose(new_image)
|
||||
image = i.convert("RGB")
|
||||
image = np.array(image).astype(np.float32) / 255.0
|
||||
new_image = torch.from_numpy(image)[None,]
|
||||
|
||||
|
||||
|
||||
|
||||
mask = torch.ones(
|
||||
(d2 + top + bottom, d3 + left + right),
|
||||
dtype=torch.float32,
|
||||
)
|
||||
|
||||
t = torch.zeros(
|
||||
(d2, d3),
|
||||
dtype=torch.float32
|
||||
)
|
||||
|
||||
if feathering > 0 and feathering * 2 < d2 and feathering * 2 < d3:
|
||||
|
||||
for i in range(d2):
|
||||
for j in range(d3):
|
||||
dt = i if top != 0 else d2
|
||||
db = d2 - i if bottom != 0 else d2
|
||||
|
||||
dl = j if left != 0 else d3
|
||||
dr = d3 - j if right != 0 else d3
|
||||
|
||||
d = min(dt, db, dl, dr)
|
||||
|
||||
if d >= feathering:
|
||||
continue
|
||||
|
||||
v = (feathering - d) / feathering
|
||||
|
||||
t[i, j] = v * v
|
||||
|
||||
mask[top:top + d2, left:left + d3] = t
|
||||
|
||||
return (new_image, mask)
|
||||
|
||||
|
||||
# A dictionary that contains all nodes you want to export with their names
|
||||
# NOTE: names should be globally unique
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"ImagePadForOutpaintAdvanced [n-suite]": ImagePadForOutpaintAdvanced
|
||||
}
|
||||
|
||||
# A dictionary that contains the friendly/humanly readable titles for the nodes
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"ImagePadForOutpaintAdvanced [n-suite]": "Image Pad For Outpainting Advanced [🅝-🅢🅤🅘🅣🅔]"
|
||||
}
|
||||
@@ -1,174 +0,0 @@
|
||||
import random
|
||||
import folder_paths
|
||||
import os
|
||||
import json
|
||||
import csv
|
||||
import server
|
||||
from aiohttp import web
|
||||
_choice = ["YES", "NO"]
|
||||
_range = ["Fixed", "Random"]
|
||||
|
||||
|
||||
def loadCSVStyle():
|
||||
csv_dir = os.path.join(folder_paths.base_path,"styles")
|
||||
csv_path = os.path.join(csv_dir,"n-styles.csv")
|
||||
#make directory if it doesn't exist
|
||||
if not os.path.exists(csv_dir):
|
||||
os.makedirs(csv_dir)
|
||||
|
||||
|
||||
styles = []
|
||||
if os.path.exists(csv_path):
|
||||
with open(csv_path, newline='', encoding='utf-8') as csvfile:
|
||||
reader = csv.DictReader(csvfile)
|
||||
for row in reader:
|
||||
styles.append(row)
|
||||
else:
|
||||
#create a file containing name,prompt,negative_prompt\n
|
||||
with open(csv_path, "w", encoding="utf-8") as f:
|
||||
f.write("name,prompt,negative_prompt\n")
|
||||
f.write('NAI,"masterpiece, best quality, masterpiece, asuka langley sitting cross legged on a chair","lowres, bad anatomy, bad hands"\n')
|
||||
styles.append({'name': 'NAI', 'prompt': 'masterpiece, best quality, masterpiece, asuka langley sitting cross legged on a chair', 'negative_prompt': 'lowres, bad anatomy, bad hands'})
|
||||
|
||||
if len(styles) == 0:
|
||||
with open(csv_path, "w", encoding="utf-8") as f:
|
||||
f.write("name,prompt,negative_prompt\n")
|
||||
f.write('NAI,"masterpiece, best quality, masterpiece, asuka langley sitting cross legged on a chair","lowres, bad anatomy, bad hands"\n')
|
||||
|
||||
|
||||
styles.append({'name': 'NAI', 'prompt': 'masterpiece, best quality, masterpiece, asuka langley sitting cross legged on a chair', 'negative_prompt': 'lowres, bad anatomy, bad hands'})
|
||||
return (styles, )
|
||||
|
||||
|
||||
|
||||
|
||||
def addStyle(name, positive_prompt, negative_prompt):
|
||||
csv_dir = os.path.join(folder_paths.base_path,"styles","n-styles.csv")
|
||||
#make directory if it doesn't exist
|
||||
if not os.path.exists(csv_dir):
|
||||
os.makedirs(csv_dir)
|
||||
|
||||
#backup file
|
||||
backup_dir = os.path.join(folder_paths.base_path,"styles","n-styles-backup.csv")
|
||||
if os.path.exists(backup_dir):
|
||||
os.remove(backup_dir)
|
||||
os.rename(csv_dir, backup_dir)
|
||||
|
||||
#edit style if it already exists else add it
|
||||
|
||||
with open(backup_dir, "r", encoding="utf-8") as f:
|
||||
lines = f.readlines()
|
||||
|
||||
with open(csv_dir, "w", encoding="utf-8") as f:
|
||||
for line in lines:
|
||||
name_style = line.split(",")[0]
|
||||
if name == name_style:
|
||||
f.write(f'{name},"{positive_prompt}","{negative_prompt}"\n')
|
||||
continue
|
||||
f.write(line)
|
||||
f.write(f'{name},"{positive_prompt}","{negative_prompt}"\n')
|
||||
|
||||
|
||||
|
||||
|
||||
styles = loadCSVStyle()
|
||||
|
||||
def deleteStyle(name):
|
||||
csv_dir = os.path.join(folder_paths.base_path,"styles","n-styles.csv")
|
||||
#make directory if it doesn't exist
|
||||
if not os.path.exists(csv_dir):
|
||||
os.makedirs(csv_dir)
|
||||
|
||||
#backup file
|
||||
backup_dir = os.path.join(folder_paths.base_path,"styles","n-styles-backup.csv")
|
||||
if os.path.exists(backup_dir):
|
||||
os.remove(backup_dir)
|
||||
os.rename(csv_dir, backup_dir)
|
||||
|
||||
with open(csv_dir, "r", encoding="utf-8") as f:
|
||||
lines = f.readlines()
|
||||
|
||||
|
||||
with open(backup_dir, "w", encoding="utf-8") as f:
|
||||
for line in lines:
|
||||
if name in line:
|
||||
continue
|
||||
f.write(line)
|
||||
|
||||
@server.PromptServer.instance.routes.get("/nsuite/styles" )
|
||||
async def style_get(request):
|
||||
result = {"styles": loadCSVStyle()}
|
||||
return web.json_response(result, content_type='application/json')
|
||||
|
||||
|
||||
|
||||
@server.PromptServer.instance.routes.post("/nsuite/styles/add" )
|
||||
async def style_add(request):
|
||||
data = await request.json()
|
||||
name = data["name"]
|
||||
positive_prompt = data["positive_prompt"]
|
||||
negative_prompt = data["negative_prompt"]
|
||||
addStyle(name, positive_prompt, negative_prompt)
|
||||
|
||||
result = {"error": "none"}
|
||||
return web.json_response(result, content_type='application/json')
|
||||
|
||||
|
||||
@server.PromptServer.instance.routes.post("/nsuite/styles/update" )
|
||||
async def style_add(request):
|
||||
data = await request.json()
|
||||
name = data["name"]
|
||||
positive_prompt = data["positive_prompt"]
|
||||
negative_prompt = data["negative_prompt"]
|
||||
addStyle(name, positive_prompt, negative_prompt)
|
||||
|
||||
result = {"error": "none"}
|
||||
return web.json_response(result, content_type='application/json')
|
||||
|
||||
|
||||
@server.PromptServer.instance.routes.post("/nsuite/styles/remove" )
|
||||
async def style_delete(request):
|
||||
data = await request.json()
|
||||
name = data["name"]
|
||||
|
||||
deleteStyle(name)
|
||||
result = {"error": "none"}
|
||||
return web.json_response(result, content_type='application/json')
|
||||
|
||||
|
||||
|
||||
class CLIPTextEncodeAdvancedNSuite:
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {"required":
|
||||
{ "styles": ([x['name'] for x in styles[0]],),
|
||||
"positive_prompt": ("STRING", {"multiline": True}),
|
||||
"negative_prompt": ("STRING", {"multiline": True}),
|
||||
"clip": ("CLIP", )}
|
||||
}
|
||||
RETURN_TYPES = ("CONDITIONING","CONDITIONING")
|
||||
RETURN_NAMES = ("positive", "negative")
|
||||
FUNCTION = "encode"
|
||||
|
||||
CATEGORY = "N-Suite/Experimental"
|
||||
|
||||
def encode(self, clip, positive_prompt, negative_prompt,styles):
|
||||
p_tokens = clip.tokenize(positive_prompt)
|
||||
n_tokens = clip.tokenize(negative_prompt)
|
||||
p_cond, p_pooled = clip.encode_from_tokens(p_tokens, return_pooled=True)
|
||||
n_cond, n_pooled = clip.encode_from_tokens(n_tokens, return_pooled=True)
|
||||
return ([[p_cond, {"pooled_output": p_pooled}]],[[n_cond, {"pooled_output": n_pooled}]], )
|
||||
|
||||
|
||||
|
||||
# A dictionary that contains all nodes you want to export with their names
|
||||
# NOTE: names should be globally unique
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"CLIPTextEncodeAdvancedNSuite [n-suite]": CLIPTextEncodeAdvancedNSuite
|
||||
}
|
||||
|
||||
# A dictionary that contains the friendly/humanly readable titles for the nodes
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"CLIPTextEncodeAdvancedNSuite [n-suite]": "CLIP Text Encode Advanced [🅝-🅢🅤🅘🅣🅔]"
|
||||
}
|
||||
@@ -23,7 +23,7 @@ class DynamicPrompt:
|
||||
}
|
||||
RETURN_TYPES = ("STRING",)
|
||||
FUNCTION = "prompt_generator"
|
||||
CATEGORY = "N-Suite/Conditioning"
|
||||
CATEGORY = "conditioning"
|
||||
OUTPUT_NODE = True
|
||||
|
||||
"""
|
||||
@@ -82,10 +82,10 @@ class DynamicPrompt:
|
||||
# A dictionary that contains all nodes you want to export with their names
|
||||
# NOTE: names should be globally unique
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"DynamicPrompt [n-suite]": DynamicPrompt
|
||||
"DynamicPrompt": DynamicPrompt
|
||||
}
|
||||
|
||||
# A dictionary that contains the friendly/humanly readable titles for the nodes
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"DynamicPrompt [n-suite]": "Dynamic Prompt [🅝-🅢🅤🅘🅣🅔]"
|
||||
"DynamicPrompt": "Dynamic Prompt"
|
||||
}
|
||||
|
||||
@@ -1,324 +0,0 @@
|
||||
import os
|
||||
import cv2
|
||||
import sys
|
||||
import torch
|
||||
import argparse
|
||||
from PIL import Image, ImageOps
|
||||
import folder_paths
|
||||
import numpy as np
|
||||
from tqdm import tqdm
|
||||
from torch.nn import functional as F
|
||||
import _thread
|
||||
from queue import Queue, Empty
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
rife_dir = Path(__file__).resolve().parent.parent / "libs" / "rifle"
|
||||
sys.path.append(str(rife_dir))
|
||||
|
||||
from model.pytorch_msssim import ssim_matlab
|
||||
interpolation_temp_input_folder = os.path.join(folder_paths.get_temp_directory(),"n-suite","interpolation_input")
|
||||
interpolation_temp_output_folder = os.path.join(folder_paths.get_temp_directory(),"n-suite","interpolation_output")
|
||||
|
||||
try:
|
||||
os.makedirs(interpolation_temp_input_folder)
|
||||
except:
|
||||
pass
|
||||
|
||||
|
||||
try:
|
||||
os.makedirs(interpolation_temp_output_folder)
|
||||
except:
|
||||
pass
|
||||
|
||||
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
||||
|
||||
scale=1
|
||||
torch.set_grad_enabled(False)
|
||||
if torch.cuda.is_available():
|
||||
torch.backends.cudnn.enabled = True
|
||||
|
||||
|
||||
try:
|
||||
from train_log.RIFE_HDv3 import Model
|
||||
except:
|
||||
print("Please download our model from model list")
|
||||
|
||||
|
||||
model = Model()
|
||||
if not hasattr(model, 'version'):
|
||||
model.version = 0
|
||||
|
||||
model_folder = str(rife_dir / "train_log")
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
output_frames = []
|
||||
def clear_write_buffer(user_args, write_buffer,output_folder):
|
||||
cnt = 0
|
||||
|
||||
while True:
|
||||
item = write_buffer.get()
|
||||
if item is None:
|
||||
break
|
||||
|
||||
cv2.imwrite(os.path.join(output_folder, '{:0>7d}.png'.format(cnt)), item[:, :, ::-1])
|
||||
cnt += 1
|
||||
|
||||
|
||||
|
||||
def build_read_buffer(img, read_buffer, videogen):
|
||||
try:
|
||||
for frame in videogen:
|
||||
if not img is None:
|
||||
frame = cv2.imread(os.path.join(img, frame), cv2.IMREAD_UNCHANGED)[:, :, ::-1].copy()
|
||||
|
||||
read_buffer.put(frame)
|
||||
except:
|
||||
pass
|
||||
read_buffer.put(None)
|
||||
|
||||
def make_inference(I0, I1, n):
|
||||
global model
|
||||
if model.version >= 3.9:
|
||||
res = []
|
||||
for i in range(n):
|
||||
res.append(model.inference(I0, I1, (i+1) * 1. / (n+1), scale))
|
||||
return res
|
||||
else:
|
||||
middle = model.inference(I0, I1, scale)
|
||||
if n == 1:
|
||||
return [middle]
|
||||
first_half = make_inference(I0, middle, n=n//2)
|
||||
second_half = make_inference(middle, I1, n=n//2)
|
||||
if n%2:
|
||||
return [*first_half, middle, *second_half]
|
||||
else:
|
||||
return [*first_half, *second_half]
|
||||
|
||||
|
||||
def get_output_filename(input_file_path, output_folder, file_extension,suffix="") :
|
||||
existing_files = [f for f in os.listdir(output_folder)]
|
||||
max_progressive = 0
|
||||
for filename in existing_files:
|
||||
parts_ext = filename.split(".")
|
||||
parts = parts_ext[0]
|
||||
|
||||
if len(parts) > 2 and parts.isdigit():
|
||||
progressive = int(parts)
|
||||
max_progressive = max(max_progressive, progressive)
|
||||
|
||||
|
||||
|
||||
new_progressive = max_progressive + 1
|
||||
new_filename = f"{new_progressive:07d}{suffix}{file_extension}"
|
||||
|
||||
return os.path.join(output_folder, new_filename), new_filename
|
||||
|
||||
def image_preprocessing(i):
|
||||
i = ImageOps.exif_transpose(i)
|
||||
image = i.convert("RGB")
|
||||
image = np.array(image).astype(np.float32) / 255.0
|
||||
image = torch.from_numpy(image)[None,]
|
||||
return image
|
||||
|
||||
|
||||
|
||||
|
||||
_choice = ["YES", "NO"]
|
||||
_range = ["Fixed", "Random"]
|
||||
class FrameInterpolator:
|
||||
def __init__(self):
|
||||
model.load_model(model_folder, -1)
|
||||
print("Loaded 3.x/4.x HD model.")
|
||||
model.eval()
|
||||
model.device()
|
||||
self.type = "output"
|
||||
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
#clear directory
|
||||
try:
|
||||
for file in os.listdir(interpolation_temp_input_folder):
|
||||
os.remove(os.path.join(interpolation_temp_input_folder,file))
|
||||
for file in os.listdir(interpolation_temp_output_folder):
|
||||
os.remove(os.path.join(interpolation_temp_output_folder,file))
|
||||
except:
|
||||
pass
|
||||
|
||||
return {"required":
|
||||
{"images": ("IMAGE", ),
|
||||
"METADATA": ("STRING", {"default": "", "forceInput": True} ),
|
||||
"multiplier": ("INT", {"default": 2, "min": 1, "step": 1}),
|
||||
|
||||
},
|
||||
|
||||
}
|
||||
|
||||
|
||||
|
||||
RETURN_TYPES = ()
|
||||
FUNCTION = "save_video"
|
||||
|
||||
OUTPUT_NODE = True
|
||||
|
||||
CATEGORY = "N-Suite/Video"
|
||||
|
||||
RETURN_TYPES = ("IMAGE","STRING",)
|
||||
OUTPUT_IS_LIST = (True, False, )
|
||||
RETURN_NAMES = ("IMAGES","METADATA",)
|
||||
|
||||
FUNCTION = "interpolate"
|
||||
|
||||
|
||||
|
||||
def interpolate(self,images,multiplier,METADATA):
|
||||
fps = METADATA[0]*multiplier
|
||||
frame_number = METADATA[1]
|
||||
video_name = METADATA[2]
|
||||
|
||||
|
||||
|
||||
for image in images:
|
||||
|
||||
full_input_temp_frame_folder,file = get_output_filename("", interpolation_temp_input_folder, ".png")
|
||||
file_name = file
|
||||
i = 255. * image.cpu().numpy()
|
||||
img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
|
||||
metadata = None
|
||||
|
||||
|
||||
#file = f"frame_{counter:05}_.png"
|
||||
img.save(full_input_temp_frame_folder, pnginfo=metadata, compress_level=0)
|
||||
|
||||
|
||||
try:
|
||||
file_name_number = int(file.split(".")[0])
|
||||
except:
|
||||
file_name_number = 0
|
||||
image_list = []
|
||||
if(file_name_number >= frame_number):
|
||||
|
||||
videogen = []
|
||||
for f in os.listdir(interpolation_temp_input_folder):
|
||||
if 'png' in f:
|
||||
videogen.append(f)
|
||||
tot_frame = len(videogen)
|
||||
videogen.sort(key= lambda x:int(x[:-4]))
|
||||
lastframe = cv2.imread(os.path.join(interpolation_temp_input_folder, videogen[0]), cv2.IMREAD_UNCHANGED)[:, :, ::-1].copy()
|
||||
videogen = videogen[1:]
|
||||
h, w, _ = lastframe.shape
|
||||
|
||||
|
||||
|
||||
tmp = max(128, int(128 / scale))
|
||||
ph = ((h - 1) // tmp + 1) * tmp
|
||||
pw = ((w - 1) // tmp + 1) * tmp
|
||||
padding = (0, pw - w, 0, ph - h)
|
||||
pbar = tqdm(total=tot_frame)
|
||||
|
||||
write_buffer = Queue(maxsize=500)
|
||||
read_buffer = Queue(maxsize=500)
|
||||
_thread.start_new_thread(build_read_buffer, (interpolation_temp_input_folder, read_buffer, videogen))
|
||||
_thread.start_new_thread(clear_write_buffer, (interpolation_temp_input_folder, write_buffer, interpolation_temp_output_folder))
|
||||
|
||||
I1 = torch.from_numpy(np.transpose(lastframe, (2,0,1))).to(device, non_blocking=True).unsqueeze(0).float() / 255.
|
||||
I1 = F.pad(I1, padding)
|
||||
temp = None # save lastframe when processing static frame
|
||||
|
||||
|
||||
|
||||
|
||||
while True:
|
||||
if temp is not None:
|
||||
frame = temp
|
||||
temp = None
|
||||
else:
|
||||
frame = read_buffer.get()
|
||||
if frame is None:
|
||||
break
|
||||
I0 = I1
|
||||
I1 = torch.from_numpy(np.transpose(frame, (2,0,1))).to(device, non_blocking=True).unsqueeze(0).float() / 255.
|
||||
I1 = F.pad(I1, padding)
|
||||
I0_small = F.interpolate(I0, (32, 32), mode='bilinear', align_corners=False)
|
||||
I1_small = F.interpolate(I1, (32, 32), mode='bilinear', align_corners=False)
|
||||
ssim = ssim_matlab(I0_small[:, :3], I1_small[:, :3])
|
||||
|
||||
break_flag = False
|
||||
if ssim > 0.996:
|
||||
frame = read_buffer.get() # read a new frame
|
||||
if frame is None:
|
||||
break_flag = True
|
||||
frame = lastframe
|
||||
else:
|
||||
temp = frame
|
||||
I1 = torch.from_numpy(np.transpose(frame, (2,0,1))).to(device, non_blocking=True).unsqueeze(0).float() / 255.
|
||||
I1 = F.pad(I1, padding)
|
||||
I1 = model.inference(I0, I1, scale)
|
||||
I1_small = F.interpolate(I1, (32, 32), mode='bilinear', align_corners=False)
|
||||
ssim = ssim_matlab(I0_small[:, :3], I1_small[:, :3])
|
||||
frame = (I1[0] * 255).byte().cpu().numpy().transpose(1, 2, 0)[:h, :w]
|
||||
|
||||
if ssim < 0.2:
|
||||
output = []
|
||||
for i in range(multiplier - 1):
|
||||
output.append(I0)
|
||||
|
||||
else:
|
||||
output = make_inference(I0, I1, multiplier-1)
|
||||
|
||||
|
||||
write_buffer.put(lastframe)
|
||||
for mid in output:
|
||||
mid = (((mid[0] * 255.).byte().cpu().numpy().transpose(1, 2, 0)))
|
||||
write_buffer.put(mid[:h, :w])
|
||||
|
||||
pbar.update(1)
|
||||
lastframe = frame
|
||||
if break_flag:
|
||||
break
|
||||
|
||||
|
||||
write_buffer.put(lastframe)
|
||||
|
||||
|
||||
import time
|
||||
while(not write_buffer.empty()):
|
||||
time.sleep(0.1)
|
||||
pbar.close()
|
||||
|
||||
|
||||
|
||||
METADATA = [fps, len(os.listdir(interpolation_temp_output_folder)),video_name]
|
||||
|
||||
images = [os.path.join(interpolation_temp_output_folder, filename) for filename in os.listdir(interpolation_temp_output_folder) if filename.endswith(".png")]
|
||||
images.sort(key=lambda f: int(''.join(filter(str.isdigit, f))))
|
||||
|
||||
for image in images:
|
||||
|
||||
image_list.append(image_preprocessing(Image.open(image)))
|
||||
|
||||
|
||||
|
||||
|
||||
return ( image_list,METADATA)
|
||||
|
||||
|
||||
|
||||
# NOTE: names should be globally unique
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"FrameInterpolator [n-suite]": FrameInterpolator,
|
||||
|
||||
}
|
||||
|
||||
# A dictionary that contains the friendly/humanly readable titles for the nodes
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"FrameInterpolator [n-suite]": "FrameInterpolator [🅝-🅢🅤🅘🅣🅔]"
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,152 @@
|
||||
import folder_paths
|
||||
import os
|
||||
from llama_cpp import Llama
|
||||
import copy
|
||||
from typing_extensions import TypedDict, Literal
|
||||
from typing import List, Optional
|
||||
|
||||
|
||||
_choice = ["YES", "NO"]
|
||||
def env_or_def(env, default):
|
||||
if (env in os.environ):
|
||||
return os.environ[env]
|
||||
return default
|
||||
|
||||
|
||||
supported_gpt_extensions = set([ '.bin','.gguf'])
|
||||
|
||||
|
||||
|
||||
try:
|
||||
folder_paths.folder_names_and_paths["GPTcheckpoints"] = (folder_paths.folder_names_and_paths["GPTcheckpoints"][0], supported_gpt_extensions)
|
||||
except:
|
||||
# check if GPTcheckpoints exists otherwise create
|
||||
if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints")):
|
||||
os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints"))
|
||||
|
||||
folder_paths.folder_names_and_paths["GPTcheckpoints"] = ([os.path.join(folder_paths.models_dir, "GPTcheckpoints")], supported_gpt_extensions)
|
||||
|
||||
|
||||
|
||||
class GPTLoaderSimple:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {"required": {
|
||||
"ckpt_name": (folder_paths.get_filename_list("GPTcheckpoints"), ),
|
||||
"gpu_layers": ("INT", {"default": 27, "min": 0, "max": 100, "step": 1}),
|
||||
"n_threads": ("INT", {"default": 8, "min": 1, "max": 100, "step": 1}),
|
||||
"max_ctx": ("INT", {"default": 2048, "min": 300, "max": 100000, "step": 64}),
|
||||
}}
|
||||
|
||||
|
||||
|
||||
RETURN_TYPES = ("CUSTOM","STRING")
|
||||
RETURN_NAMES = ("model", "model_path")
|
||||
FUNCTION = "load_gpt_checkpoint"
|
||||
|
||||
CATEGORY = "loaders"
|
||||
print()
|
||||
def load_gpt_checkpoint(self, ckpt_name, gpu_layers,n_threads,max_ctx):
|
||||
ckpt_path = folder_paths.get_full_path("GPTcheckpoints", ckpt_name)
|
||||
llm = Llama(model_path=ckpt_path,n_gpu_layers=gpu_layers,verbose=False,n_threads=n_threads, n_ctx=max_ctx, )
|
||||
|
||||
return llm, ckpt_path
|
||||
|
||||
|
||||
class GPTSampler:
|
||||
|
||||
"""
|
||||
A custom node for text generation using GPT
|
||||
|
||||
Attributes
|
||||
----------
|
||||
max_tokens (`int`): Maximum number of tokens in the generated text.
|
||||
temperature (`float`): Temperature parameter for controlling randomness (0.2 to 1.0).
|
||||
top_p (`float`): Top-p probability for nucleus sampling.
|
||||
logprobs (`int`|`None`): Number of log probabilities to output alongside the generated text.
|
||||
echo (`bool`): Whether to print the input prompt alongside the generated text.
|
||||
stop (`str`|`List[str]`|`None`): Tokens at which to stop generation.
|
||||
frequency_penalty (`float`): Frequency penalty for word repetition.
|
||||
presence_penalty (`float`): Presence penalty for word diversity.
|
||||
repeat_penalty (`float`): Penalty for repeating a prompt's output.
|
||||
top_k (`int`): Top-k tokens to consider during generation.
|
||||
stream (`bool`): Whether to generate the text in a streaming fashion.
|
||||
tfs_z (`float`): Temperature scaling factor for top frequent samples.
|
||||
model (`str`): The GPT model to use for text generation.
|
||||
"""
|
||||
def __init__(self):
|
||||
self.temp_prompt = ""
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
"prompt": ("STRING",{"forceInput": True} ),
|
||||
"model": ("CUSTOM", {"default": ""}),
|
||||
"model_path": ("STRING", {"default": "","forceInput": True}),
|
||||
"max_tokens": ("INT", {"default": 2048}),
|
||||
"temperature": ("FLOAT", {"default": 0.7, "min": 0.2, "max": 1.0}),
|
||||
"top_p": ("FLOAT", {"default": 0.5, "min": 0.1, "max": 1.0}),
|
||||
"logprobs": ("INT", {"default": 0}),
|
||||
"echo": (["enable", "disable"], {"default": "disable"}),
|
||||
"stop_token": ("STRING", {"default": "STOPTOKEN"}),
|
||||
"frequency_penalty": ("FLOAT", {"default": 0.0}),
|
||||
"presence_penalty": ("FLOAT", {"default": 0.0}),
|
||||
"repeat_penalty": ("FLOAT", {"default": 1.17647}),
|
||||
"top_k": ("INT", {"default": 40}),
|
||||
"tfs_z": ("FLOAT", {"default": 1.0}),
|
||||
"print_output": (["enable", "disable"], {"default": "disable"}),
|
||||
"cached": (_choice,{"default": "NO"} ),
|
||||
"prefix": ("STRING", {"default": "### Instruction: "}),
|
||||
"suffix": ("STRING", {"default": "### Response: "}),
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("STRING",)
|
||||
FUNCTION = "generate_text"
|
||||
CATEGORY = "sampling"
|
||||
|
||||
def generate_text(self,prompt, max_tokens, temperature, top_p, logprobs, echo, stop_token, frequency_penalty, presence_penalty, repeat_penalty, top_k, tfs_z, model,model_path,print_output,cached,prefix,suffix):
|
||||
|
||||
|
||||
if cached == "NO":
|
||||
# Call your GPT generation function here using the provided parameters
|
||||
composed_prompt = f"{prefix} {prompt} {suffix}"
|
||||
cont =""
|
||||
stream = model( max_tokens=max_tokens, stop=[stop_token], stream=False,frequency_penalty=frequency_penalty,presence_penalty=presence_penalty ,repeat_penalty=repeat_penalty,temperature=temperature,top_k=top_k,top_p=top_p,model=model_path,prompt=composed_prompt)
|
||||
print(len(stream))
|
||||
print(stream)
|
||||
cont= stream["choices"][0]["text"]
|
||||
self.temp_prompt = cont
|
||||
else:
|
||||
cont = self.temp_prompt
|
||||
#remove fist 30 characters of cont
|
||||
try:
|
||||
if print_output == "enable":
|
||||
print(f"Input: {prompt}\nGenerated Text: {cont}")
|
||||
return {"ui": {"text": cont}, "result": (cont,)}
|
||||
|
||||
except:
|
||||
if print_output == "enable":
|
||||
print(f"Input: {prompt}\nGenerated Text: ")
|
||||
return {"ui": {"text": " "}, "result": (" ",)}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"GPT Loader Simple": GPTLoaderSimple,
|
||||
"GPTSampler": GPTSampler
|
||||
}
|
||||
# A dictionary that contains the friendly/humanly readable titles for the nodes
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"GPT Loader Simple": "GPT Loader Simple",
|
||||
"GPTSampler": "GPT Text Sampler"
|
||||
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -1,482 +0,0 @@
|
||||
import folder_paths
|
||||
import os
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import torch
|
||||
from huggingface_hub import snapshot_download
|
||||
sys.path.append(os.path.join(str(Path(__file__).parent.parent),"libs"))
|
||||
import joytag_models
|
||||
from PIL import Image
|
||||
from transformers import AutoModelForCausalLM, CodeGenTokenizerFast as Tokenizer, GenerationConfig, GenerationMixin, PreTrainedModel
|
||||
from transformers.dynamic_module_utils import HF_MODULES_CACHE
|
||||
from server import PromptServer
|
||||
#,AutoTokenizer, AutoModelForCausalLM
|
||||
import numpy as np
|
||||
|
||||
models_base_path = os.path.join(folder_paths.models_dir, "GPTcheckpoints")
|
||||
MOONDREAM_REVISION = "f6e9da68e8f1b78b8f3ee10905d56826db7a5802"
|
||||
JOYTAG_REVISION = "6b7f16331a6ccf0fdce37d5a9564715f6e772b22"
|
||||
MODEL_DOWNLOADS = {
|
||||
"moondream": ("Moondream", 3.72),
|
||||
"joytag": ("JoyTag", 0.37),
|
||||
}
|
||||
_choice = ["YES", "NO"]
|
||||
_folders_whitelist = ["moondream","joytag"]#,"internlm"]
|
||||
|
||||
|
||||
def env_or_def(env, default):
|
||||
if (env in os.environ):
|
||||
return os.environ[env]
|
||||
return default
|
||||
|
||||
def get_model_path(folder_list, model_name):
|
||||
for folder_path in folder_list:
|
||||
if folder_path.endswith(model_name):
|
||||
return folder_path
|
||||
|
||||
def get_model_list(models_base_path,supported_gpt_extensions):
|
||||
all_models = []
|
||||
try:
|
||||
for file in os.listdir(models_base_path):
|
||||
|
||||
if os.path.isdir(os.path.join(models_base_path, file)):
|
||||
if file in _folders_whitelist:
|
||||
all_models.append(os.path.join(models_base_path, file))
|
||||
|
||||
else:
|
||||
if file.endswith(tuple(supported_gpt_extensions)):
|
||||
all_models.append(os.path.join(models_base_path, file))
|
||||
except:
|
||||
print(f"Path {models_base_path} not valid.")
|
||||
return all_models
|
||||
|
||||
|
||||
def tensor2pil(image):
|
||||
return Image.fromarray(np.clip(255. * image.cpu().numpy().squeeze(), 0, 255).astype(np.uint8))
|
||||
|
||||
# Convert PIL to Tensor
|
||||
def pil2tensor(image):
|
||||
return torch.from_numpy(np.array(image).astype(np.float32) / 255.0).unsqueeze(0)
|
||||
|
||||
|
||||
def detect_device():
|
||||
"""
|
||||
Detects the appropriate device to run on, and return the device and dtype.
|
||||
"""
|
||||
if torch.cuda.is_available():
|
||||
return torch.device("cuda"), torch.float16
|
||||
elif torch.backends.mps.is_available():
|
||||
return torch.device("mps"), torch.float16
|
||||
else:
|
||||
return torch.device("cpu"), torch.float32
|
||||
|
||||
|
||||
def load_joytag(ckpt_path,cpu=False):
|
||||
print("JOYTAG MODEL DETECTED")
|
||||
jt_config = os.path.join(models_base_path,"joytag","config.json")
|
||||
jt_readme= os.path.join(models_base_path,"joytag","README.md")
|
||||
jt_top_tags= os.path.join(models_base_path,"joytag","top_tags.txt")
|
||||
jt_model= os.path.join(models_base_path,"joytag","model.safetensors")
|
||||
|
||||
|
||||
if os.path.exists(jt_config)==False or os.path.exists(jt_readme)==False or os.path.exists(jt_top_tags)==False or os.path.exists(jt_model)==False:
|
||||
snapshot_download(
|
||||
"fancyfeast/joytag",
|
||||
revision=JOYTAG_REVISION,
|
||||
local_dir=os.path.join(models_base_path, "joytag"),
|
||||
allow_patterns=["README.md", "config.json", "model.safetensors", "top_tags.txt"],
|
||||
)
|
||||
model = joytag_models.VisionModel.load_model(ckpt_path)
|
||||
model.eval()
|
||||
if cpu:
|
||||
return model.to('cpu')
|
||||
else:
|
||||
return model.to('cuda')
|
||||
|
||||
def run_joytag(images, prompt, max_tags, model_funct):
|
||||
with open(os.path.join(models_base_path,'joytag','top_tags.txt') , 'r') as f:
|
||||
top_tags = [line.strip() for line in f.readlines() if line.strip()]
|
||||
|
||||
if images is None:
|
||||
raise ValueError("No image provided")
|
||||
top_tags_processed = []
|
||||
for image in images:
|
||||
_, scores = joytag_models.predict(image, model_funct, top_tags)
|
||||
top_tags_scores = sorted(scores.items(), key=lambda x: x[1], reverse=True)[:max_tags]
|
||||
# Extract the tags from the pairs
|
||||
top_tags_processed.append(', '.join([tag for tag, _ in top_tags_scores]))
|
||||
|
||||
return top_tags_processed
|
||||
|
||||
|
||||
def load_moondream(ckpt_path,cpu=False):
|
||||
|
||||
dtype = torch.float32
|
||||
|
||||
if cpu:
|
||||
device=torch.device("cpu")
|
||||
else:
|
||||
device = torch.device("cuda")
|
||||
|
||||
|
||||
model_dir = os.path.join(models_base_path, "moondream")
|
||||
snapshot_download(
|
||||
"vikhyatk/moondream1",
|
||||
revision=MOONDREAM_REVISION,
|
||||
local_dir=model_dir,
|
||||
allow_patterns=[
|
||||
"config.json", "configuration_moondream.py", "moondream.py",
|
||||
"modeling_phi.py", "text_model.py", "vision_encoder.py",
|
||||
"model.safetensors", "tokenizer.json", "tokenizer_config.json",
|
||||
"special_tokens_map.json", "added_tokens.json", "merges.txt", "vocab.json",
|
||||
],
|
||||
)
|
||||
patch_moondream_model_code(model_dir)
|
||||
tokenizer = Tokenizer.from_pretrained(model_dir)
|
||||
moondream = AutoModelForCausalLM.from_pretrained(model_dir, trust_remote_code=True)
|
||||
enable_moondream_generation(moondream)
|
||||
moondream = moondream.to(device=device, dtype=dtype)
|
||||
moondream.eval()
|
||||
return [moondream, tokenizer]
|
||||
|
||||
|
||||
def enable_moondream_generation(moondream):
|
||||
"""Restore generation for Moondream1's legacy Phi model on Transformers 4.50+."""
|
||||
text_model = moondream.text_model
|
||||
if getattr(text_model, "_n_suite_generation_compat", False):
|
||||
return
|
||||
|
||||
model_class = type(text_model)
|
||||
original_prepare = model_class.prepare_inputs_for_generation
|
||||
|
||||
def prepare_inputs_for_generation(
|
||||
self, input_ids=None, inputs_embeds=None, past_key_values=None,
|
||||
attention_mask=None, **kwargs,
|
||||
):
|
||||
prepared = original_prepare(
|
||||
self, input_ids=input_ids, inputs_embeds=inputs_embeds,
|
||||
past_key_values=past_key_values, attention_mask=attention_mask,
|
||||
**kwargs,
|
||||
)
|
||||
# Moondream supplies image embeddings without padding. The newer
|
||||
# generation API otherwise builds a mask one token too long.
|
||||
prepared["attention_mask"] = None
|
||||
return prepared
|
||||
|
||||
bases = (model_class,) if isinstance(text_model, GenerationMixin) else (model_class, GenerationMixin)
|
||||
text_model.__class__ = type(
|
||||
"GeneratingPhiForCausalLM", bases,
|
||||
{"prepare_inputs_for_generation": prepare_inputs_for_generation},
|
||||
)
|
||||
text_model._n_suite_generation_compat = True
|
||||
if text_model.generation_config is None:
|
||||
text_model.generation_config = GenerationConfig.from_model_config(text_model.config)
|
||||
|
||||
|
||||
def patch_moondream_model_code(model_dir):
|
||||
"""Add GenerationMixin to the pinned Phi source before Transformers imports it."""
|
||||
original_import = "from transformers import PretrainedConfig, PreTrainedModel"
|
||||
parent_base = "class PhiPreTrainedModel(PreTrainedModel):"
|
||||
previous_patch = "class PhiPreTrainedModel(PreTrainedModel, GenerationMixin):"
|
||||
model_bases = (
|
||||
("class PhiModel(PhiPreTrainedModel):", "class PhiModel(PhiPreTrainedModel, GenerationMixin):"),
|
||||
("class PhiForCausalLM(PhiPreTrainedModel):", "class PhiForCausalLM(PhiPreTrainedModel, GenerationMixin):"),
|
||||
)
|
||||
needs_patch = not issubclass(PreTrainedModel, GenerationMixin)
|
||||
|
||||
def patch_source(path):
|
||||
source = path.read_text()
|
||||
if original_import not in source:
|
||||
raise RuntimeError("Unexpected Moondream1 Phi source; cannot apply the generation compatibility patch")
|
||||
if previous_patch in source:
|
||||
source = source.replace(previous_patch, parent_base, 1)
|
||||
for original, patched in model_bases:
|
||||
if original not in source and patched not in source:
|
||||
raise RuntimeError("Unexpected Moondream1 Phi source; cannot apply the generation compatibility patch")
|
||||
if needs_patch:
|
||||
source = source.replace(original, patched, 1)
|
||||
else:
|
||||
source = source.replace(patched, original, 1)
|
||||
patched_import = original_import + ", GenerationMixin"
|
||||
if needs_patch:
|
||||
source = source.replace(original_import, patched_import, 1) if patched_import not in source else source
|
||||
else:
|
||||
source = source.replace(patched_import, original_import, 1)
|
||||
if source != path.read_text():
|
||||
path.write_text(source)
|
||||
|
||||
patch_source(Path(model_dir) / "modeling_phi.py")
|
||||
cached_source = Path(HF_MODULES_CACHE) / "transformers_modules" / Path(model_dir).name / "modeling_phi.py"
|
||||
if cached_source.is_file():
|
||||
patch_source(cached_source)
|
||||
|
||||
|
||||
|
||||
def run_moondream(images, prompt, max_tags, model_funct):
|
||||
from PIL import Image
|
||||
moondream = model_funct[0]
|
||||
tokenizer = model_funct[1]
|
||||
list_descriptions = []
|
||||
for image in images:
|
||||
im=tensor2pil(image)
|
||||
|
||||
image_embeds = moondream.encode_image(im)
|
||||
try:
|
||||
list_descriptions.append(moondream.answer_question(image_embeds, prompt,tokenizer))
|
||||
except ValueError:
|
||||
print("\n\n\n")
|
||||
raise ModuleNotFoundError("Moondream requires the dependency versions declared in N-Suite requirements.txt. Reinstall dependencies with ComfyUI Manager.")
|
||||
|
||||
|
||||
|
||||
return list_descriptions
|
||||
|
||||
"""
|
||||
def load_internlm(ckpt_path,cpu=False):
|
||||
|
||||
|
||||
|
||||
local_dir=os.path.join(os.path.join(models_base_path,"internlm"))
|
||||
local_model_1 = os.path.join(local_dir,"pytorch_model-00001-of-00002.bin")
|
||||
local_model_2 = os.path.join(local_dir,"pytorch_model-00002-of-00002.bin")
|
||||
|
||||
if os.path.exists(local_model_1) and os.path.exists(local_model_2):
|
||||
model_path = local_dir
|
||||
else:
|
||||
model_path = snapshot_download("internlm/internlm-xcomposer2-vl-7b", local_dir=local_dir, revision="f8e6ab8d7ff14dbd6b53335c93ff8377689040bf", local_dir_use_symlinks=False)
|
||||
|
||||
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)
|
||||
|
||||
if torch.cuda.is_available() and cpu == False:
|
||||
|
||||
model = AutoModelForCausalLM.from_pretrained(
|
||||
model_path,
|
||||
torch_dtype="auto",
|
||||
trust_remote_code=True,
|
||||
device_map="auto"
|
||||
).eval()
|
||||
|
||||
else:
|
||||
model = model.cpu().float().eval()
|
||||
|
||||
model.tokenizer = tokenizer
|
||||
|
||||
#device = device
|
||||
#dtype = dtype
|
||||
name = "internlm"
|
||||
#low_memory = low_memory
|
||||
|
||||
return ([model, tokenizer])
|
||||
|
||||
|
||||
def run_internlm(image, prompt, max_tags, model_funct):
|
||||
model = model_funct[0]
|
||||
tokenizer = model_funct[1]
|
||||
low_memory = True
|
||||
import tempfile
|
||||
image = Image.fromarray(np.clip(255. * image[0].cpu().numpy(),0,255).astype(np.uint8))
|
||||
#image = model.vis_processor(image)
|
||||
temp_dir = tempfile.mkdtemp()
|
||||
image_path = os.path.join(temp_dir,"input.jpg")
|
||||
image.save(image_path)
|
||||
#image = tensor2pil(image)
|
||||
if torch.cuda.is_available():
|
||||
with torch.cuda.amp.autocast():
|
||||
response, _ = model.chat(
|
||||
query=prompt,
|
||||
image=image_path,
|
||||
tokenizer= tokenizer,
|
||||
history=[],
|
||||
do_sample=True
|
||||
)
|
||||
if low_memory:
|
||||
torch.cuda.empty_cache()
|
||||
print(f"Memory usage: {torch.cuda.memory_allocated() / 1024 ** 3:.2f} GB")
|
||||
model.to("cpu", dtype=torch.float16)
|
||||
print(f"Memory usage: {torch.cuda.memory_allocated() / 1024 ** 3:.2f} GB")
|
||||
else:
|
||||
response, _ = model.chat(
|
||||
query=prompt,
|
||||
image=image,
|
||||
tokenizer= tokenizer,
|
||||
history=[],
|
||||
do_sample=True
|
||||
)
|
||||
|
||||
return response
|
||||
"""
|
||||
|
||||
|
||||
|
||||
|
||||
os.makedirs(models_base_path, exist_ok=True)
|
||||
|
||||
#create folder if it doesn't exist
|
||||
os.makedirs(os.path.join(models_base_path, "joytag"), exist_ok=True)
|
||||
|
||||
os.makedirs(os.path.join(models_base_path, "moondream"), exist_ok=True)
|
||||
|
||||
"""#internlm
|
||||
if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","internlm")):
|
||||
os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","internlm"))
|
||||
"""
|
||||
#folder_paths.folder_names_and_paths["GPTcheckpoints"] += (os.listdir(models_base_path),)
|
||||
|
||||
|
||||
|
||||
MODEL_FUNCTIONS = {
|
||||
'joytag': run_joytag,
|
||||
'moondream': run_moondream
|
||||
}
|
||||
MODEL_LOAD_FUNCTIONS = {
|
||||
'joytag': load_joytag,
|
||||
'moondream': load_moondream
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
all_models = get_model_list(models_base_path, set())
|
||||
all_models_names = [os.path.basename(model) for model in all_models]
|
||||
|
||||
|
||||
|
||||
class GPTLoaderSimple:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {"required": {
|
||||
"ckpt_name": (all_models_names, ),
|
||||
"gpu_layers": ("INT", {"default": 27, "min": 0, "max": 100, "step": 1}),
|
||||
"n_threads": ("INT", {"default": 8, "min": 1, "max": 100, "step": 1}),
|
||||
"max_ctx": ("INT", {"default": 2048, "min": 300, "max": 100000, "step": 64}),
|
||||
},
|
||||
"hidden": {"unique_id": "UNIQUE_ID"}}
|
||||
|
||||
|
||||
|
||||
RETURN_TYPES = ("CUSTOM", )
|
||||
RETURN_NAMES = ("model",)
|
||||
FUNCTION = "load_gpt_checkpoint"
|
||||
DESCRIPTION = "Loads Moondream (~3.72 GB) or JoyTag (~0.37 GB). The first use downloads the selected model; watch the ComfyUI console for progress."
|
||||
|
||||
CATEGORY = "N-Suite/loaders"
|
||||
|
||||
def load_gpt_checkpoint(self, ckpt_name, gpu_layers, n_threads, max_ctx, unique_id=None):
|
||||
ckpt_path = get_model_path(all_models,ckpt_name)
|
||||
if ckpt_name not in MODEL_LOAD_FUNCTIONS:
|
||||
raise ValueError(f"Unsupported model: {ckpt_name}")
|
||||
model_dir = os.path.join(models_base_path, ckpt_name)
|
||||
if not os.path.isfile(os.path.join(model_dir, "model.safetensors")):
|
||||
model_name, size_gb = MODEL_DOWNLOADS[ckpt_name]
|
||||
message = (f"{model_name}: downloading approximately {size_gb:.2f} GB on first use. "
|
||||
"This may take a while; watch the ComfyUI console for progress.")
|
||||
print(f"[N-Suite] {message}", flush=True)
|
||||
if PromptServer.instance is not None:
|
||||
PromptServer.instance.send_sync(
|
||||
"n-suite-model-download",
|
||||
{"node_id": unique_id, "model": model_name, "size_gb": size_gb, "message": message},
|
||||
)
|
||||
cpu = gpu_layers == 0
|
||||
llm = MODEL_LOAD_FUNCTIONS[ckpt_name](ckpt_path, cpu)
|
||||
|
||||
return ([llm, ckpt_name, ckpt_path],)
|
||||
|
||||
|
||||
class GPTSampler:
|
||||
|
||||
"""
|
||||
A custom node for text generation using GPT
|
||||
|
||||
Attributes
|
||||
----------
|
||||
max_tokens (`int`): Maximum number of tokens in the generated text.
|
||||
temperature (`float`): Temperature parameter for controlling randomness (0.2 to 1.0).
|
||||
top_p (`float`): Top-p probability for nucleus sampling.
|
||||
logprobs (`int`|`None`): Number of log probabilities to output alongside the generated text.
|
||||
echo (`bool`): Whether to print the input prompt alongside the generated text.
|
||||
stop (`str`|`List[str]`|`None`): Tokens at which to stop generation.
|
||||
frequency_penalty (`float`): Frequency penalty for word repetition.
|
||||
presence_penalty (`float`): Presence penalty for word diversity.
|
||||
repeat_penalty (`float`): Penalty for repeating a prompt's output.
|
||||
top_k (`int`): Top-k tokens to consider during generation.
|
||||
stream (`bool`): Whether to generate the text in a streaming fashion.
|
||||
tfs_z (`float`): Temperature scaling factor for top frequent samples.
|
||||
model (`str`): The GPT model to use for text generation.
|
||||
"""
|
||||
def __init__(self):
|
||||
self.temp_prompt = ""
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls):
|
||||
return {
|
||||
"required": {
|
||||
|
||||
"model": ("CUSTOM", {"default": ""}),
|
||||
"max_tokens": ("INT", {"default": 2048}),
|
||||
"temperature": ("FLOAT", {"default": 0.7, "min": 0.2, "max": 1.0}),
|
||||
"top_p": ("FLOAT", {"default": 0.5, "min": 0.1, "max": 1.0}),
|
||||
"logprobs": ("INT", {"default": 0}),
|
||||
"echo": (["enable", "disable"], {"default": "disable"}),
|
||||
"stop_token": ("STRING", {"default": "STOPTOKEN"}),
|
||||
"frequency_penalty": ("FLOAT", {"default": 0.0}),
|
||||
"presence_penalty": ("FLOAT", {"default": 0.0}),
|
||||
"repeat_penalty": ("FLOAT", {"default": 1.17647}),
|
||||
"top_k": ("INT", {"default": 40}),
|
||||
"tfs_z": ("FLOAT", {"default": 1.0}),
|
||||
"print_output": (["enable", "disable"], {"default": "disable"}),
|
||||
"cached": (_choice,{"default": "NO"} ),
|
||||
"prefix": ("STRING", {"default": "### Instruction: "}),
|
||||
"suffix": ("STRING", {"default": "### Response: "}),
|
||||
"max_tags": ("INT", {"default": 50}),
|
||||
|
||||
},
|
||||
"optional": {
|
||||
"prompt": ("STRING",{"forceInput": True} ),
|
||||
"image": ("IMAGE",),
|
||||
}
|
||||
}
|
||||
|
||||
RETURN_TYPES = ("STRING",)
|
||||
OUTPUT_IS_LIST = (True,)
|
||||
FUNCTION = "generate_text"
|
||||
CATEGORY = "N-Suite/Sampling"
|
||||
|
||||
|
||||
|
||||
def generate_text(self, max_tokens, temperature, top_p, logprobs, echo, stop_token, frequency_penalty, presence_penalty, repeat_penalty, top_k, tfs_z, model,print_output,cached,prefix,suffix,max_tags,image=None,prompt=None):
|
||||
model_funct = model[0]
|
||||
model_name = model[1]
|
||||
model_path = model[2]
|
||||
|
||||
|
||||
if cached == "NO":
|
||||
if model_name in MODEL_FUNCTIONS and os.path.isdir(model_path):
|
||||
cont = MODEL_FUNCTIONS[model_name](image, prompt, max_tags, model_funct)
|
||||
else:
|
||||
raise ValueError(f"Unsupported model: {model_name}")
|
||||
else:
|
||||
cont = self.temp_prompt
|
||||
#remove fist 30 characters of cont
|
||||
try:
|
||||
if print_output == "enable":
|
||||
print(f"Input: {prompt}\nGenerated Text: {cont}")
|
||||
return {"ui": {"text": cont}, "result": (cont,)}
|
||||
|
||||
except:
|
||||
if print_output == "enable":
|
||||
print(f"Input: {prompt}\nGenerated Text: ")
|
||||
return {"ui": {"text": " "}, "result": (" ",)}
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"GPT Loader Simple [n-suite]": GPTLoaderSimple,
|
||||
"GPT Sampler [n-suite]": GPTSampler
|
||||
}
|
||||
# A dictionary that contains the friendly/humanly readable titles for the nodes
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"GPT Loader Simple [n-suite]": "GPT Loader Simple [🅝-🅢🅤🅘🅣🅔]",
|
||||
"GPT Sampler [n-suite]": "Image Caption Sampler [🅝-🅢🅤🅘🅣🅔]"
|
||||
|
||||
}
|
||||
@@ -1,124 +0,0 @@
|
||||
import os
|
||||
import cv2
|
||||
import sys
|
||||
import torch
|
||||
import argparse
|
||||
from PIL import Image, ImageOps
|
||||
import folder_paths
|
||||
import numpy as np
|
||||
from tqdm import tqdm
|
||||
from torch.nn import functional as F
|
||||
import _thread
|
||||
from queue import Queue, Empty
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def image_preprocessing(i):
|
||||
i = ImageOps.exif_transpose(i)
|
||||
image = i.convert("RGB")
|
||||
image = np.array(image).astype(np.float32) / 255.0
|
||||
image = torch.from_numpy(image)[None,]
|
||||
return image
|
||||
class LoadImageFromFolder:
|
||||
def __init__(self):
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {"required": { "folder":("STRING", {"default": ""} ),
|
||||
"fps":("INT", {"default": 30})
|
||||
}}
|
||||
|
||||
|
||||
RETURN_TYPES = ("IMAGE","INT","INT","INT","STRING","STRING",)
|
||||
RETURN_NAMES = ("IMAGES","MAX WIDTH","MAX HEIGHT","IMAGE COUNT","PATH","IMAGE LIST")
|
||||
FUNCTION = "load_images"
|
||||
OUTPUT_IS_LIST = (True,False,False,False,False,False,)
|
||||
|
||||
CATEGORY = "N-Suite/Experimental"
|
||||
|
||||
def load_images(self, folder,fps):
|
||||
image_list = []
|
||||
image_names = []
|
||||
max_width = 0
|
||||
max_height = 0
|
||||
frame_count = 0
|
||||
|
||||
|
||||
images = [os.path.join(folder, filename) for filename in os.listdir(folder) if filename.endswith(".png") or filename.endswith(".jpg")]
|
||||
|
||||
|
||||
for image_path in images:
|
||||
#get image name
|
||||
image_names.append(image_path.split("/")[-1])
|
||||
image = Image.open(image_path)
|
||||
width, height = image.size
|
||||
max_width = max(max_width, width)
|
||||
max_height = max(max_height, height)
|
||||
image_list.append((image_preprocessing(image)))
|
||||
frame_count += 1
|
||||
|
||||
image_names_final='\n'.join(image_names)
|
||||
print (f"Details: {frame_count} frames, {max_width}x{max_height}")
|
||||
|
||||
return (image_list, max_width, max_height,frame_count,folder,image_names_final,)
|
||||
|
||||
|
||||
class SaveCaptionsFromImageList:
|
||||
def __init__(self):
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {"required": { "folder":("STRING", {"default": ""} ),
|
||||
"fps":("INT", {"default": 30})
|
||||
}}
|
||||
|
||||
|
||||
RETURN_TYPES = ("IMAGE","INT","INT","INT","STRING","STRING",)
|
||||
RETURN_NAMES = ("IMAGES","MAX WIDTH","MAX HEIGHT","IMAGE COUNT","PATH","IMAGE LIST")
|
||||
FUNCTION = "load_images"
|
||||
OUTPUT_IS_LIST = (True,False,False,False,False,False,)
|
||||
|
||||
CATEGORY = "LJRE/Loader"
|
||||
|
||||
def load_images(self, folder,fps):
|
||||
image_list = []
|
||||
image_names = []
|
||||
max_width = 0
|
||||
max_height = 0
|
||||
frame_count = 0
|
||||
|
||||
|
||||
images = [os.path.join(folder, filename) for filename in os.listdir(folder) if filename.endswith(".png") or filename.endswith(".jpg")]
|
||||
|
||||
|
||||
for image_path in images:
|
||||
#get image name
|
||||
image_names.append(image_path.split("/")[-1])
|
||||
image = Image.open(image_path)
|
||||
width, height = image.size
|
||||
max_width = max(max_width, width)
|
||||
max_height = max(max_height, height)
|
||||
image_list.append((image_preprocessing(image)))
|
||||
frame_count += 1
|
||||
|
||||
image_names_final='\n'.join(image_names)
|
||||
print (f"Details: {frame_count} frames, {max_width}x{max_height}")
|
||||
|
||||
return (image_list, max_width, max_height,frame_count,folder,image_names_final,)
|
||||
|
||||
# NOTE: names should be globally unique
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"LoadImageFromFolder [n-suite]": LoadImageFromFolder,
|
||||
|
||||
}
|
||||
|
||||
# A dictionary that contains the friendly/humanly readable titles for the nodes
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"LoadImageFromFolder [n-suite]": "Load Image From Folder [🅝-🅢🅤🅘🅣🅔]"
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -19,7 +19,7 @@ class IntVariable:
|
||||
|
||||
RETURN_TYPES = ("INT",)
|
||||
FUNCTION = "check_int"
|
||||
CATEGORY = "N-Suite/Variables"
|
||||
CATEGORY = "Variables"
|
||||
|
||||
def check_int(self, value):
|
||||
if value == "":
|
||||
@@ -53,7 +53,7 @@ class FloatVariable:
|
||||
|
||||
RETURN_TYPES = ("FLOAT",)
|
||||
FUNCTION = "check"
|
||||
CATEGORY = "N-Suite/Variables"
|
||||
CATEGORY = "Variables"
|
||||
|
||||
def check(self, value):
|
||||
if value == "":
|
||||
@@ -67,36 +67,41 @@ class FloatVariable:
|
||||
return (value,)
|
||||
|
||||
class StringVariable:
|
||||
RETURN_TYPES = ("STRING",)
|
||||
FUNCTION = "check"
|
||||
CATEGORY = "N-Suite/Variables"
|
||||
|
||||
def __init__(self):
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {"required": {"string": ("STRING", {"default": "", "multiline": True})}}
|
||||
return {"required": {"value": ("STRING", {"multiline": True})}}
|
||||
|
||||
|
||||
RETURN_TYPES = ("STRING",)
|
||||
FUNCTION = "check"
|
||||
CATEGORY = "Variables"
|
||||
|
||||
|
||||
def check(self, string):
|
||||
if string == "undefined":
|
||||
string = ""
|
||||
def check(self, value):
|
||||
if value == "undefined":
|
||||
value = ""
|
||||
#if not an int
|
||||
if not str(string):
|
||||
string = ""
|
||||
if not str(value):
|
||||
value = ""
|
||||
|
||||
return (string,)
|
||||
|
||||
|
||||
return (value,)
|
||||
|
||||
# A dictionary that contains all nodes you want to export with their names
|
||||
# NOTE: names should be globally unique
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"Integer Variable [n-suite]": IntVariable,
|
||||
"Float Variable [n-suite]": FloatVariable,
|
||||
"String Variable [n-suite]": StringVariable
|
||||
"Integer Variable": IntVariable,
|
||||
"Float Variable": FloatVariable,
|
||||
"String Variable": StringVariable
|
||||
}
|
||||
|
||||
# A dictionary that contains the friendly/humanly readable titles for the nodes
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"Integer Variable [n-suite]": "Integer Variable [🅝-🅢🅤🅘🅣🅔]",
|
||||
"Float Variable [n-suite]": "Float Variable [🅝-🅢🅤🅘🅣🅔]",
|
||||
"String Variable [n-suite]": "String Variable [🅝-🅢🅤🅘🅣🅔]"
|
||||
"Variables": "Integer Variable",
|
||||
"Variables": "Float Variable",
|
||||
"Variables": "String Variable"
|
||||
}
|
||||
|
||||
@@ -9,8 +9,7 @@ import cv2
|
||||
import os
|
||||
import imageio
|
||||
import shutil
|
||||
from moviepy import VideoFileClip, AudioFileClip
|
||||
from contextlib import ExitStack
|
||||
from moviepy.editor import VideoFileClip, AudioFileClip
|
||||
import random
|
||||
import math
|
||||
import json
|
||||
@@ -18,12 +17,6 @@ from comfy.cli_args import args
|
||||
import time
|
||||
import concurrent.futures
|
||||
|
||||
|
||||
|
||||
|
||||
YELLOW = '\33[33m'
|
||||
END = '\33[0m'
|
||||
|
||||
# Brutally copied from comfy_extras/nodes_rebatch.py and modified
|
||||
class LatentRebatch:
|
||||
|
||||
@@ -114,42 +107,32 @@ class LatentRebatch:
|
||||
|
||||
|
||||
input_dir = os.path.join(folder_paths.get_input_directory(),"n-suite")
|
||||
output_dir = os.path.join(folder_paths.get_output_directory(),"n-suite","frames_out")
|
||||
temp_output_dir = os.path.join(folder_paths.get_temp_directory(),"n-suite","frames_out")
|
||||
frames_output_dir = os.path.join(folder_paths.get_temp_directory(),"n-suite","frames")
|
||||
videos_output_dir = os.path.join(folder_paths.get_output_directory(),"n-suite","videos")
|
||||
output_dir = os.path.join(folder_paths.get_output_directory(),"n-suite")
|
||||
frames_output_dir = os.path.join(folder_paths.get_output_directory(),"frames")
|
||||
videos_output_dir = os.path.join(folder_paths.get_output_directory(),"videos")
|
||||
audios_output_temp_dir = os.path.join(folder_paths.get_temp_directory(),"audio.mp3")
|
||||
videos_output_temp_dir = os.path.join(folder_paths.get_temp_directory(),"video.mp4")
|
||||
video_preview_output_temp_dir = os.path.join(folder_paths.get_output_directory(),"n-suite","videos")
|
||||
video_preview_output_temp_dir = os.path.join(folder_paths.get_output_directory(),"videos")
|
||||
_resize_type = ["none","width", "height"]
|
||||
_framerate = ["original","half", "quarter"]
|
||||
_choice = ["Yes", "No"]
|
||||
try:
|
||||
os.makedirs(input_dir)
|
||||
except:
|
||||
pass
|
||||
try:
|
||||
os.makedirs(output_dir)
|
||||
except:
|
||||
pass
|
||||
|
||||
try:
|
||||
os.makedirs(temp_output_dir)
|
||||
os.mkdir(input_dir)
|
||||
except:
|
||||
pass
|
||||
|
||||
try:
|
||||
os.makedirs(videos_output_dir)
|
||||
os.mkdir(videos_output_dir)
|
||||
except:
|
||||
pass
|
||||
|
||||
try:
|
||||
os.makedirs(frames_output_dir)
|
||||
os.mkdir(frames_output_dir)
|
||||
except:
|
||||
pass
|
||||
|
||||
try:
|
||||
os.makedirs(folder_paths.get_temp_directory())
|
||||
os.mkdir(folder_paths.get_temp_directory())
|
||||
except:
|
||||
pass
|
||||
|
||||
@@ -159,33 +142,25 @@ def calc_resize_image(input_path, target_size, resize_by):
|
||||
height, width = image.shape[:2]
|
||||
|
||||
if resize_by == 'width':
|
||||
|
||||
new_width = target_size
|
||||
new_height = int(height * (target_size / width))
|
||||
|
||||
elif resize_by == 'height':
|
||||
|
||||
new_height = target_size
|
||||
new_width = int(width * (target_size / height))
|
||||
|
||||
else:
|
||||
|
||||
new_height = height
|
||||
new_width = width
|
||||
|
||||
return new_width, new_height
|
||||
|
||||
def calc_resize_image_from_ram(input_frame, target_size, resize_by):
|
||||
height, width = input_frame.shape[:2]
|
||||
|
||||
if resize_by == 'width':
|
||||
new_width = target_size
|
||||
new_height = int(height * (target_size / width))
|
||||
elif resize_by == 'height':
|
||||
new_height = target_size
|
||||
new_width = int(width * (target_size / height))
|
||||
else:
|
||||
new_height = height
|
||||
new_width = width
|
||||
|
||||
return new_width, new_height
|
||||
|
||||
return new_width, new_height
|
||||
|
||||
|
||||
def resize_image(input_path, new_width, new_height):
|
||||
|
||||
image = cv2.imread(input_path)
|
||||
height, width = image.shape[:2]
|
||||
|
||||
@@ -193,24 +168,38 @@ def resize_image(input_path, new_width, new_height):
|
||||
resized_image = cv2.resize(image, (new_width, new_height))
|
||||
else:
|
||||
resized_image = image
|
||||
|
||||
|
||||
|
||||
pil_image = Image.fromarray(cv2.cvtColor(resized_image, cv2.COLOR_BGR2RGB))
|
||||
|
||||
|
||||
return pil_image
|
||||
|
||||
def resize_image_from_ram(image, new_width, new_height):
|
||||
height, width = image.shape[:2]
|
||||
""" def extract_frames_from_video(video_path, output_folder):
|
||||
|
||||
if height != new_height or width != new_width:
|
||||
resized_image = cv2.resize(image, (new_width, new_height))
|
||||
else:
|
||||
resized_image = image
|
||||
|
||||
pil_image = Image.fromarray(cv2.cvtColor(resized_image, cv2.COLOR_BGR2RGB))
|
||||
return pil_image
|
||||
|
||||
def extract_frames_from_video(video_path, output_folder=None, target_fps=30, use_ram=True):
|
||||
frames = []
|
||||
list_files = []
|
||||
os.makedirs(output_folder, exist_ok=True)
|
||||
cap = cv2.VideoCapture(video_path)
|
||||
frame_count = 0
|
||||
|
||||
while True:
|
||||
ret, frame = cap.read()
|
||||
if not ret:
|
||||
break
|
||||
|
||||
frame_count += 1
|
||||
frame_filename = os.path.join(output_folder, f"frame_{frame_count:04d}.png")
|
||||
list_files.append(frame_filename)
|
||||
cv2.imwrite(frame_filename, frame)
|
||||
|
||||
cap.release()
|
||||
print(f"{frame_count} frames have been extracted from the video and saved in {output_folder}")
|
||||
return list_files """
|
||||
|
||||
|
||||
def extract_frames_from_video(video_path, output_folder, target_fps=30):
|
||||
list_files = []
|
||||
os.makedirs(output_folder, exist_ok=True)
|
||||
cap = cv2.VideoCapture(video_path)
|
||||
frame_count = 0
|
||||
|
||||
@@ -219,13 +208,6 @@ def extract_frames_from_video(video_path, output_folder=None, target_fps=30, use
|
||||
# Calcola il rapporto per ridurre il framerate
|
||||
frame_skip_ratio = original_fps // target_fps
|
||||
real_frame_count = 0
|
||||
|
||||
if not use_ram:
|
||||
if output_folder is None:
|
||||
raise ValueError("output_folder must be specified if use_ram is False")
|
||||
if output_folder is not None:
|
||||
os.makedirs(output_folder, exist_ok=True)
|
||||
|
||||
while True:
|
||||
ret, frame = cap.read()
|
||||
if not ret:
|
||||
@@ -235,60 +217,40 @@ def extract_frames_from_video(video_path, output_folder=None, target_fps=30, use
|
||||
|
||||
# Estrai solo ogni "frame_skip_ratio"-esimo fotogramma
|
||||
if frame_count % frame_skip_ratio == 0:
|
||||
if use_ram:
|
||||
frames.append(frame)
|
||||
else:
|
||||
frame_filename = os.path.join(output_folder, f"{frame_count:07d}.png")
|
||||
list_files.append(frame_filename)
|
||||
cv2.imwrite(frame_filename, frame)
|
||||
frame_filename = os.path.join(output_folder, f"frame_{frame_count:04d}.png")
|
||||
list_files.append(frame_filename)
|
||||
cv2.imwrite(frame_filename, frame)
|
||||
real_frame_count += 1
|
||||
|
||||
cap.release()
|
||||
print(f"{real_frame_count} frames have been extracted from the video")
|
||||
|
||||
if use_ram:
|
||||
return frames
|
||||
else:
|
||||
return list_files
|
||||
print(f"{real_frame_count} frames have been extracted from the video and saved in {output_folder}")
|
||||
return list_files
|
||||
|
||||
|
||||
|
||||
def extract_frames_from_gif(gif_path, output_folder):
|
||||
def extract_frames_from_gif(gif_path, output_folder, target_fps=30):
|
||||
list_files = []
|
||||
os.makedirs(output_folder, exist_ok=True)
|
||||
|
||||
gif_frames = imageio.mimread(gif_path, memtest=False)
|
||||
|
||||
real_frame_count = 0
|
||||
metadata = imageio.v3.immeta(gif_path)
|
||||
|
||||
gif_frames = imageio.mimread(gif_path)
|
||||
original_fps = len(gif_frames)
|
||||
frame_skip_ratio = original_fps // original_fps
|
||||
frame_count = 0
|
||||
for frame in gif_frames:
|
||||
frame_count += 1
|
||||
frame_filename = os.path.join(output_folder, f"{frame_count:07d}.png")
|
||||
list_files.append(frame_filename)
|
||||
cv2.imwrite(frame_filename, cv2.cvtColor(frame, cv2.COLOR_RGB2BGR))
|
||||
|
||||
if frame_count % frame_skip_ratio == 0:
|
||||
frame_filename = os.path.join(output_folder, f"frame_{frame_count:04d}.png")
|
||||
cv2.imwrite(frame_filename, cv2.cvtColor(frame, cv2.COLOR_RGB2BGR))
|
||||
list_files.append(frame_filename)
|
||||
real_frame_count += 1
|
||||
|
||||
print(f"{frame_count} frames have been extracted from the GIF and saved in {output_folder}")
|
||||
return list_files
|
||||
return list_files,metadata
|
||||
|
||||
|
||||
def get_output_filename(input_file_path, output_folder, file_extension,suffix="") :
|
||||
existing_files = [f for f in os.listdir(output_folder)]
|
||||
max_progressive = 0
|
||||
for filename in existing_files:
|
||||
parts_ext = filename.split(".")
|
||||
parts = parts_ext[0]
|
||||
|
||||
if len(parts) > 2 and parts.isdigit():
|
||||
progressive = int(parts)
|
||||
max_progressive = max(max_progressive, progressive)
|
||||
|
||||
|
||||
|
||||
new_progressive = max_progressive + 1
|
||||
new_filename = f"{new_progressive:07d}{suffix}{file_extension}"
|
||||
|
||||
return os.path.join(output_folder, new_filename), new_filename
|
||||
|
||||
|
||||
def get_output_filename_video(input_file_path, output_folder, file_extension,suffix="") :
|
||||
input_filename = os.path.basename(input_file_path)
|
||||
input_filename_without_extension = os.path.splitext(input_filename)[0]
|
||||
|
||||
@@ -318,6 +280,7 @@ def image_preprocessing(i):
|
||||
def create_video_from_frames(frame_folder, output_video, frame_rate = 30.0):
|
||||
frame_filenames = [os.path.join(frame_folder, filename) for filename in os.listdir(frame_folder) if filename.endswith(".png")]
|
||||
frame_filenames.sort(key=lambda f: int(''.join(filter(str.isdigit, f))))
|
||||
|
||||
first_frame = cv2.imread(frame_filenames[0])
|
||||
height, width, layers = first_frame.shape
|
||||
|
||||
@@ -331,14 +294,14 @@ def create_video_from_frames(frame_folder, output_video, frame_rate = 30.0):
|
||||
out.release()
|
||||
print(f"Frames have been successfully reassembled into {output_video}")
|
||||
|
||||
def create_gif_from_frames(frame_folder, output_gif):
|
||||
def create_gif_from_frames(frame_folder, output_gif, metadata):
|
||||
frame_filenames = [os.path.join(frame_folder, filename) for filename in os.listdir(frame_folder) if filename.endswith(".png")]
|
||||
frame_filenames.sort()
|
||||
|
||||
frames = [imageio.imread(frame_filename) for frame_filename in frame_filenames]
|
||||
|
||||
# imageio
|
||||
imageio.mimsave(output_gif, frames, duration=0.1)
|
||||
imageio.mimsave(output_gif, frames, loop=metadata[3], duration=metadata[4])
|
||||
|
||||
|
||||
print(f"Frames have been successfully assembled into {output_gif}")
|
||||
@@ -347,33 +310,29 @@ def create_gif_from_frames(frame_folder, output_gif):
|
||||
temp_dir= folder_paths.temp_directory
|
||||
|
||||
|
||||
class LoadVideoAdvanced:
|
||||
class VideoLoader:
|
||||
def __init__(self):
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
files = [f for f in os.listdir(input_dir) if os.path.isfile(os.path.join(input_dir, f))]
|
||||
return {"required": {"video": (sorted(files), ),
|
||||
return {"required": {"video": (sorted(files), {"image_upload": True} ),
|
||||
"local_url": ("STRING", {"default": ""} ),
|
||||
"framerate": (_framerate, {"default": "original"} ),
|
||||
"resize_by": (_resize_type,{"default": "none"} ),
|
||||
"size": ("INT", {"default": 512, "min": 512, "step": 64}),
|
||||
"images_limit": ("INT", {"default": 0, "min": 0, "step": 1}),
|
||||
"batch_size": ("INT", {"default": 0, "min": 0, "step": 1}),
|
||||
"starting_frame": ("INT", {"default": 0, "min": 0, "step": 1}),
|
||||
"autoplay":("BOOLEAN",{"default": True} ),
|
||||
"use_ram": ("BOOLEAN", {"default": False}),
|
||||
|
||||
"batch_size": ("INT", {"default": 0, "min": 0, "step": 1})
|
||||
|
||||
},}
|
||||
|
||||
|
||||
RETURN_TYPES = ("IMAGE","LATENT","STRING","INT","INT","INT","INT",)
|
||||
OUTPUT_IS_LIST = (True, True, False, False,False,False,False, )
|
||||
RETURN_NAMES = ("IMAGES","EMPTY LATENTS","METADATA","WIDTH","HEIGHT","META_FPS","META_N_FRAMES")
|
||||
CATEGORY = "N-Suite/Video"
|
||||
RETURN_TYPES = ("IMAGE","LATENT","STRING","INT","INT",)
|
||||
OUTPUT_IS_LIST = (True, True, False, False,False, )
|
||||
RETURN_NAMES = ("IMAGES","EMPTY LATENT","METADATA","WIDTH","HEIGHT")
|
||||
CATEGORY = "video"
|
||||
FUNCTION = "encode"
|
||||
TYPE="N-Suite"
|
||||
|
||||
|
||||
@staticmethod
|
||||
@@ -386,8 +345,9 @@ class LoadVideoAdvanced:
|
||||
pixels = pixels[:, x_offset:x + x_offset, y_offset:y + y_offset, :]
|
||||
return pixels
|
||||
|
||||
def load_video(self, video, framerate, local_url, use_ram):
|
||||
file_path = folder_paths.get_annotated_filepath(os.path.join("n-suite", video))
|
||||
def load_video(self, video,framerate, local_url):
|
||||
|
||||
file_path = folder_paths.get_annotated_filepath(os.path.join("n-suite",video))
|
||||
cap = cv2.VideoCapture(file_path)
|
||||
# Check if the video was opened successfully
|
||||
if not cap.isOpened():
|
||||
@@ -397,138 +357,129 @@ class LoadVideoAdvanced:
|
||||
fps = int(cap.get(cv2.CAP_PROP_FPS))
|
||||
print(f"The video has {fps} frames per second.")
|
||||
|
||||
try:
|
||||
shutil.rmtree(os.path.join(temp_output_dir, video.split(".")[0]))
|
||||
except:
|
||||
print("Video Path already deleted")
|
||||
|
||||
full_temp_output_dir = os.path.join(temp_output_dir, video.split(".")[0])
|
||||
#shutil.rmtree(output_dir)
|
||||
#print(f"Temporary folder {output_dir} has been emptied.")
|
||||
|
||||
#set new framerate
|
||||
|
||||
# Set new framerate
|
||||
if "half" in framerate:
|
||||
fps = fps // 2
|
||||
print(f"The video has been reduced to {fps} frames per second.")
|
||||
print (f"The video has been reduced to {fps} frames per second.")
|
||||
elif "quarter" in framerate:
|
||||
fps = fps // 4
|
||||
print(f"The video has been reduced to {fps} frames per second.")
|
||||
print (f"The video has been reduced to {fps} frames per second.")
|
||||
|
||||
|
||||
# Estract frames
|
||||
file_extension = os.path.splitext(file_path)[1].lower()
|
||||
|
||||
if file_extension in [".mp4", ".webm"]:
|
||||
list_files = extract_frames_from_video(file_path, full_temp_output_dir, fps, use_ram)
|
||||
|
||||
if file_extension == ".mp4":
|
||||
list_files = extract_frames_from_video(file_path, output_dir, target_fps=fps)
|
||||
meta = {"loop": 0, "duration": 0}
|
||||
|
||||
audio_clip = VideoFileClip(file_path).audio
|
||||
try:
|
||||
with VideoFileClip(file_path) as video_clip:
|
||||
if video_clip.audio is not None:
|
||||
video_clip.audio.write_audiofile(os.path.join(temp_output_dir, video.split(".")[0], "audio.mp3"))
|
||||
except Exception as exc:
|
||||
print(f"Could not save audio: {exc}")
|
||||
#save audio
|
||||
audio_clip.write_audiofile(audios_output_temp_dir)
|
||||
except:
|
||||
pass
|
||||
|
||||
|
||||
elif file_extension == ".gif":
|
||||
list_files = extract_frames_from_gif(file_path, output_dir)
|
||||
list_files,meta = extract_frames_from_gif(file_path, output_dir)
|
||||
#create_gif_from_frames(output_dir, output_video2)
|
||||
|
||||
else:
|
||||
print("Format not supported. Please provide an MP4 or GIF file.")
|
||||
|
||||
return list_files, fps
|
||||
|
||||
return list_files,fps,file_extension,meta["loop"],meta["duration"]
|
||||
|
||||
def generate_latent(self, width, height, batch_size=1):
|
||||
latent = torch.zeros([batch_size, 4, height // 8, width // 8])
|
||||
return {"samples": latent}
|
||||
|
||||
def process_image(self, args):
|
||||
image, width, height, use_ram = args
|
||||
return {"samples":latent}
|
||||
|
||||
def process_image(self,args):
|
||||
image_path, width, height = args
|
||||
# Funzione per ridimensionare e pre-elaborare un'immagine
|
||||
if use_ram:
|
||||
image = resize_image_from_ram(image, width, height)
|
||||
else:
|
||||
image = resize_image(image, width, height)
|
||||
image = resize_image(image_path, width, height)
|
||||
image = image_preprocessing(image)
|
||||
return torch.tensor(image)
|
||||
|
||||
def encode(self, video, framerate, local_url, resize_by, size, images_limit, batch_size, starting_frame, autoplay, use_ram):
|
||||
|
||||
def encode(self,video,framerate, local_url, resize_by, size, images_limit,batch_size):
|
||||
metadata = []
|
||||
FRAMES, fps = self.load_video(video, framerate, local_url, use_ram)
|
||||
max_frames = len(FRAMES)
|
||||
|
||||
if images_limit > 0 and starting_frame > 0:
|
||||
images_limit += starting_frame
|
||||
|
||||
print(f"images_limit {images_limit}")
|
||||
|
||||
if starting_frame > max_frames:
|
||||
starting_frame = max_frames - 1
|
||||
print(f"WARNING: The starting frame is greater than the number of frames in the video. Only the last frame of the video will be used ({starting_frame}).")
|
||||
|
||||
if images_limit > max_frames:
|
||||
images_limit = max_frames
|
||||
print(f"WARNING: The number of images to extract is greater than the number of frames in the video. Images_limit has been reduced to the number of frames ({images_limit}).")
|
||||
|
||||
if batch_size > max_frames:
|
||||
print(f"WARNING: The batch size is greater than the number of frames requested. Batch size has been reduced.")
|
||||
batch_size = max_frames
|
||||
|
||||
if images_limit != 0 and batch_size > images_limit:
|
||||
print(f"WARNING: The batch size is greater than the number of frames requested. Batch size has been reduced to the number of images_limit.")
|
||||
batch_size = images_limit
|
||||
|
||||
pool_size = 5
|
||||
i_list = []
|
||||
final_count_frame = 0
|
||||
FRAMES,fps,file_extension,loop,duration = self.load_video(video,framerate, local_url)
|
||||
pool_size=5
|
||||
t_list = []
|
||||
i_list = []
|
||||
i = 0
|
||||
o = 0
|
||||
|
||||
with concurrent.futures.ThreadPoolExecutor() as executor:
|
||||
futures = []
|
||||
if use_ram:
|
||||
width, height = calc_resize_image_from_ram(FRAMES[0], size, resize_by)
|
||||
else:
|
||||
width, height = calc_resize_image(FRAMES[0], size, resize_by)
|
||||
width, height = calc_resize_image(FRAMES[0], size, resize_by)
|
||||
|
||||
|
||||
for batch_start in range(0, len(FRAMES), pool_size):
|
||||
batch_images = FRAMES[batch_start:batch_start + pool_size]
|
||||
|
||||
if images_limit != 0 or starting_frame != 0:
|
||||
#remove audio if image_limit > 0
|
||||
if images_limit != 0:
|
||||
try:
|
||||
os.remove(os.path.join(temp_output_dir, video.split(".")[0], "audio.mp3"))
|
||||
os.remove(audios_output_temp_dir)
|
||||
except:
|
||||
pass
|
||||
|
||||
if o >= images_limit:
|
||||
break
|
||||
|
||||
for idx, image in enumerate(batch_images):
|
||||
if final_count_frame >= starting_frame and (final_count_frame < images_limit or images_limit == 0):
|
||||
args = (image, width, height, use_ram)
|
||||
futures.append(executor.submit(self.process_image, args))
|
||||
final_count_frame += 1
|
||||
for image_path in batch_images:
|
||||
args = (image_path, width, height)
|
||||
futures.append(executor.submit(self.process_image, args))
|
||||
o += 1
|
||||
if images_limit != 0:
|
||||
if o >= images_limit:
|
||||
break
|
||||
|
||||
|
||||
i += len(batch_images)
|
||||
|
||||
# Attendi il completamento delle operazioni in parallelo
|
||||
concurrent.futures.wait(futures)
|
||||
|
||||
# Recupera i risultati
|
||||
for future in futures:
|
||||
batch_i_tensors = future.result()
|
||||
i_list.extend(batch_i_tensors)
|
||||
|
||||
i_tensor = torch.stack(i_list, dim=0)
|
||||
|
||||
|
||||
if images_limit != 0 or starting_frame != 0:
|
||||
b_size = final_count_frame
|
||||
if images_limit != 0:
|
||||
b_size=images_limit
|
||||
else:
|
||||
b_size = len(FRAMES)
|
||||
b_size=len(FRAMES)
|
||||
|
||||
latent = self.generate_latent(width, height, batch_size=b_size)
|
||||
|
||||
latent = self.generate_latent( width, height, batch_size=b_size)
|
||||
|
||||
metadata.append(fps)
|
||||
metadata.append(b_size)
|
||||
try:
|
||||
metadata.append(video.split(".")[0])
|
||||
except:
|
||||
print("No video name")
|
||||
metadata.append(file_extension)
|
||||
metadata.append(loop)
|
||||
metadata.append(duration)
|
||||
|
||||
if batch_size != 0:
|
||||
rebatcher = LatentRebatch()
|
||||
rebatched_latent = rebatcher.rebatch([latent], [batch_size])
|
||||
n_chunks = b_size // batch_size
|
||||
n_chunks = b_size//batch_size
|
||||
i_tensor_batches = torch.chunk(i_tensor, n_chunks, dim=0)
|
||||
return i_tensor_batches, rebatched_latent, metadata, width, height
|
||||
|
||||
return [i_tensor], [latent], metadata, width, height, fps, b_size
|
||||
|
||||
return (i_tensor_batches,rebatched_latent,metadata, width, height,)
|
||||
|
||||
return ( [i_tensor],[latent],metadata, width, height,)
|
||||
|
||||
|
||||
class SaveVideo:
|
||||
class VideoSaver:
|
||||
def __init__(self):
|
||||
|
||||
self.type = "output"
|
||||
@@ -536,6 +487,8 @@ class SaveVideo:
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
s.video_file_path,s.video_filename = get_output_filename("video", videos_output_dir, ".mp4")
|
||||
s.gif_file_path,s.gif_filename = get_output_filename("gif", videos_output_dir, ".gif")
|
||||
|
||||
try:
|
||||
shutil.rmtree(frames_output_dir)
|
||||
@@ -544,15 +497,11 @@ class SaveVideo:
|
||||
pass
|
||||
|
||||
|
||||
#print(f"Temporary folder {frames_output_dir} has been emptied.")
|
||||
print(f"Temporary folder {frames_output_dir} has been emptied.")
|
||||
return {"required":
|
||||
{"images": ("IMAGE", ),
|
||||
"METADATA": ("STRING", {"default": "", "forceInput": True} ),
|
||||
"SaveVideo": ("BOOLEAN",{"default": False} ),
|
||||
"SaveFrames": ("BOOLEAN",{"default": False} ),
|
||||
"filename_prefix": ("STRING",{"default": "video"} ),
|
||||
"CompressionLevel": ("INT", {"default": 2, "min": 0, "max":9, "step": 1}),
|
||||
|
||||
"SaveVideo": (_choice,{"default": "No"} ),
|
||||
},
|
||||
"hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"},
|
||||
}
|
||||
@@ -563,67 +512,77 @@ class SaveVideo:
|
||||
|
||||
OUTPUT_NODE = True
|
||||
|
||||
CATEGORY = "N-Suite/Video"
|
||||
|
||||
def save_video(self, images,METADATA,SaveVideo,SaveFrames,filename_prefix, CompressionLevel, prompt=None, extra_pnginfo=None):
|
||||
|
||||
self.video_file_path,self.video_filename = get_output_filename_video(filename_prefix, videos_output_dir, ".mp4")
|
||||
CATEGORY = "video"
|
||||
|
||||
def save_video(self, images,METADATA,SaveVideo, prompt=None, extra_pnginfo=None):
|
||||
|
||||
fps = METADATA[0]
|
||||
frame_number = METADATA[1]
|
||||
video_filename_original = METADATA[2]
|
||||
|
||||
file_extension = METADATA[2]
|
||||
|
||||
|
||||
results = list()
|
||||
|
||||
#full_output_folder, filename, counter, subfolder, filename_prefix = folder_paths.get_save_image_path("", frames_output_dir, images[0].shape[1], images[0].shape[0])
|
||||
results = list()
|
||||
|
||||
for image in images:
|
||||
|
||||
full_output_folder,file = get_output_filename("", frames_output_dir, ".png")
|
||||
file_name = file
|
||||
|
||||
full_output_folder,file = get_output_filename("frame", frames_output_dir, ".png")
|
||||
|
||||
i = 255. * image.cpu().numpy()
|
||||
img = Image.fromarray(np.clip(i, 0, 255).astype(np.uint8))
|
||||
metadata = None
|
||||
|
||||
|
||||
#file = f"frame_{counter:05}_.png"
|
||||
img.save(full_output_folder, pnginfo=metadata, compress_level=CompressionLevel)
|
||||
img.save(full_output_folder, pnginfo=metadata, compress_level=4)
|
||||
results.append({
|
||||
"filename": file,
|
||||
"subfolder": "frames",
|
||||
"type": self.type
|
||||
})
|
||||
|
||||
try:
|
||||
file_name_number = int(file.split(".")[0])
|
||||
file_name_number = int(file.split(".")[0].split("_")[1])
|
||||
except:
|
||||
file_name_number = 0
|
||||
|
||||
if(file_name_number >= frame_number):
|
||||
create_video_from_frames(frames_output_dir, videos_output_temp_dir,frame_rate=fps)
|
||||
|
||||
with ExitStack() as clips:
|
||||
video_clip = clips.enter_context(VideoFileClip(videos_output_temp_dir))
|
||||
audio_path = os.path.join(temp_output_dir, video_filename_original, "audio.mp3")
|
||||
if os.path.isfile(audio_path):
|
||||
audio_clip = clips.enter_context(AudioFileClip(audio_path))
|
||||
video_clip = video_clip.with_audio(audio_clip)
|
||||
|
||||
if SaveFrames == True:
|
||||
#copy frames_output_dir to self.video_file_path/self.video_filename
|
||||
frame_folder=os.path.join(videos_output_dir,self.video_filename.split(".")[0])
|
||||
shutil.copytree(frames_output_dir, frame_folder)
|
||||
|
||||
if SaveVideo == True:
|
||||
if file_extension == ".mp4":
|
||||
create_video_from_frames(frames_output_dir, videos_output_temp_dir,frame_rate=fps)
|
||||
|
||||
video_clip = VideoFileClip(videos_output_temp_dir)
|
||||
try:
|
||||
audio_clip = AudioFileClip(audios_output_temp_dir)
|
||||
video_clip = video_clip.set_audio(audio_clip)
|
||||
except:
|
||||
pass
|
||||
if SaveVideo == "Yes":
|
||||
video_clip.write_videofile(self.video_file_path)
|
||||
file_name = self.video_filename
|
||||
else:
|
||||
#delete all temporary files that start with video_preview
|
||||
for file in os.listdir(video_preview_output_temp_dir):
|
||||
if file.startswith("video_preview"):
|
||||
os.remove(os.path.join(video_preview_output_temp_dir,file))
|
||||
#random number
|
||||
suffix = str(random.randint(1,100000))
|
||||
file_name = f"video_preview_{suffix}.mp4"
|
||||
video_clip.write_videofile(os.path.join(video_preview_output_temp_dir,file_name))
|
||||
elif file_extension == ".gif":
|
||||
|
||||
if SaveVideo == "Yes":
|
||||
create_gif_from_frames(frames_output_dir, os.path.join(video_preview_output_temp_dir,self.gif_filename),METADATA)
|
||||
file_name = self.gif_filename
|
||||
else:
|
||||
#delete all temporary files that start with video_preview
|
||||
for file in os.listdir(video_preview_output_temp_dir):
|
||||
if file.startswith("gif_preview"):
|
||||
os.remove(os.path.join(video_preview_output_temp_dir,file))
|
||||
#random number
|
||||
suffix = str(random.randint(1,100000))
|
||||
file_name = f"gif_preview_{suffix}.gif"
|
||||
create_gif_from_frames(frames_output_dir,os.path.join(video_preview_output_temp_dir,file_name),METADATA)
|
||||
|
||||
|
||||
|
||||
|
||||
@@ -638,83 +597,47 @@ class LoadFramesFromFolder:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {"required": { "folder":("STRING", {"default": ""} ),
|
||||
"fps":("INT", {"default": 30})
|
||||
"fps":("INT", {"default": 30}),
|
||||
"loop_for_gif": ("INT", {"default": 0}),
|
||||
"duration_for_gif":("INT", {"default": 0}),
|
||||
|
||||
}}
|
||||
|
||||
|
||||
RETURN_TYPES = ("IMAGE","STRING","INT","INT","INT","STRING","STRING",)
|
||||
RETURN_NAMES = ("IMAGES","METADATA","MAX WIDTH","MAX HEIGHT","FRAME COUNT","PATH","IMAGE LIST")
|
||||
RETURN_TYPES = ("IMAGE","STRING",)
|
||||
RETURN_NAMES = ("IMAGES","METADATA")
|
||||
|
||||
FUNCTION = "load_images"
|
||||
OUTPUT_IS_LIST = (True,False,False,False,False,False,False,)
|
||||
CATEGORY = "N-Suite/Video"
|
||||
OUTPUT_IS_LIST = (True,False,)
|
||||
CATEGORY = "video"
|
||||
|
||||
def load_images(self, folder,fps):
|
||||
def load_images(self, folder,fps,loop_for_gif,duration_for_gif):
|
||||
image_list = []
|
||||
image_names = []
|
||||
max_width = 0
|
||||
max_height = 0
|
||||
frame_count = 0
|
||||
METADATA = [fps, len(os.listdir(folder)),"load"]
|
||||
METADATA = [fps, len(os.listdir(folder)),loop_for_gif,duration_for_gif]
|
||||
|
||||
images = [os.path.join(folder, filename) for filename in os.listdir(folder) if filename.endswith(".png") or filename.endswith(".jpg")]
|
||||
images = [os.path.join(folder, filename) for filename in os.listdir(folder) if filename.endswith(".png")]
|
||||
images.sort(key=lambda f: int(''.join(filter(str.isdigit, f))))
|
||||
for image in images:
|
||||
|
||||
for image_path in images:
|
||||
#get image name
|
||||
image_names.append(image_path.split("/")[-1])
|
||||
image = Image.open(image_path)
|
||||
width, height = image.size
|
||||
max_width = max(max_width, width)
|
||||
max_height = max(max_height, height)
|
||||
image_list.append((image_preprocessing(image)))
|
||||
frame_count += 1
|
||||
|
||||
image_names_final='\n'.join(image_names)
|
||||
print (f"Details: {frame_count} frames, {max_width}x{max_height}")
|
||||
image_list.append(image_preprocessing(Image.open(image)))
|
||||
|
||||
return (image_list,METADATA, max_width, max_height,frame_count,folder,image_names_final,)
|
||||
|
||||
class SetMetadata:
|
||||
def __init__(self):
|
||||
pass
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {"required": { "number_of_frames":("INT", {"default": 1, "min": 1, "step": 1}),
|
||||
"fps":("INT", {"default": 30, "min": 1, "step": 1}),
|
||||
"VideoName": ("STRING", {"default": "manual"} )
|
||||
|
||||
|
||||
}}
|
||||
|
||||
|
||||
RETURN_TYPES = ("STRING",)
|
||||
RETURN_NAMES = ("METADATA",)
|
||||
FUNCTION = "set_metadata"
|
||||
OUTPUT_IS_LIST = (False,)
|
||||
CATEGORY = "N-Suite/Video"
|
||||
|
||||
def set_metadata(self, number_of_frames,fps,VideoName):
|
||||
|
||||
METADATA = [fps, number_of_frames,VideoName]
|
||||
return (METADATA,)
|
||||
#i_tensor = torch.stack(image_list, dim=0)
|
||||
|
||||
return (image_list,METADATA,)
|
||||
|
||||
|
||||
|
||||
# A dictionary that contains all nodes you want to export with their names
|
||||
# NOTE: names should be globally unique
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"LoadVideo [n-suite]": LoadVideoAdvanced,
|
||||
"SaveVideo [n-suite]":SaveVideo,
|
||||
"LoadFramesFromFolder [n-suite]": LoadFramesFromFolder,
|
||||
"SetMetadataForSaveVideo [n-suite]": SetMetadata
|
||||
"VideoLoader": VideoLoader,
|
||||
"VideoSaver":VideoSaver,
|
||||
"LoadFramesFromFolder": LoadFramesFromFolder
|
||||
}
|
||||
|
||||
# A dictionary that contains the friendly/humanly readable titles for the nodes
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"LoadVideo [n-suite]": "LoadVideo [🅝-🅢🅤🅘🅣🅔]",
|
||||
"SaveVideo [n-suite]": "SaveVideo [🅝-🅢🅤🅘🅣🅔]",
|
||||
"LoadFramesFromFolder [n-suite]": "LoadFramesFromFolder [🅝-🅢🅤🅘🅣🅔]",
|
||||
"SetMetadataForSaveVideo [n-suite]": "SetMetadataForSaveVideo [🅝-🅢🅤🅘🅣🅔]"
|
||||
"Video": "VideoLoader",
|
||||
"Video": "VideoSaver",
|
||||
"Video": "LoadFramesFromFolder"
|
||||
}
|
||||
@@ -1,23 +0,0 @@
|
||||
[project]
|
||||
name = "comfyui-n-nodes"
|
||||
description = "A suite of custom nodes for ComfyUI that includes image captioning, LoadVideo, SaveVideo, LoadFramesFromFolder and FrameInterpolator"
|
||||
version = "1.2.0"
|
||||
license = { file = "LICENSE" }
|
||||
dependencies = [
|
||||
"gitpython",
|
||||
"huggingface-hub",
|
||||
"moviepy>=2.2.1,<3",
|
||||
"opencv-python",
|
||||
"accelerate>=1.0,<2",
|
||||
"timm>=1.0.22",
|
||||
"transformers>=4.36.2,<5",
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
Repository = "https://github.com/Nuked88/ComfyUI-N-Nodes"
|
||||
# Used by Comfy Registry https://comfyregistry.org
|
||||
|
||||
[tool.comfy]
|
||||
PublisherId = "nuked"
|
||||
DisplayName = "ComfyUI-N-Nodes"
|
||||
Icon = ""
|
||||
@@ -1,3 +0,0 @@
|
||||
[pytest]
|
||||
testpaths = tests
|
||||
addopts = --import-mode=importlib
|
||||
@@ -1,7 +0,0 @@
|
||||
gitpython
|
||||
huggingface-hub
|
||||
moviepy>=2.2.1,<3
|
||||
opencv-python
|
||||
accelerate>=1.0,<2
|
||||
timm>=1.0.22
|
||||
transformers>=4.36.2,<5
|
||||
@@ -1,30 +0,0 @@
|
||||
import importlib.util
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
MODULE_PATH = Path(__file__).resolve().parents[1] / "py" / "dynamic_prompt_node.py"
|
||||
SPEC = importlib.util.spec_from_file_location("dynamic_prompt_node", MODULE_PATH)
|
||||
MODULE = importlib.util.module_from_spec(SPEC)
|
||||
SPEC.loader.exec_module(MODULE)
|
||||
DynamicPrompt = MODULE.DynamicPrompt
|
||||
|
||||
|
||||
def generate(variable, mode="Fixed", count=1, fixed=""):
|
||||
return DynamicPrompt().prompt_generator(variable, "NO", mode, count, fixed)["result"][0]
|
||||
|
||||
|
||||
def test_fixed_prompt_is_combined_with_requested_number_of_unique_tags():
|
||||
result = generate("red, green, blue", count=2, fixed="portrait")
|
||||
parts = result.split(",")
|
||||
assert parts[0] == "portrait"
|
||||
assert len(parts[1:]) == 2
|
||||
assert len(set(parts[1:])) == 2
|
||||
assert set(parts[1:]) <= {"red", "green", "blue"}
|
||||
|
||||
|
||||
def test_requested_tag_count_is_capped_to_available_tags():
|
||||
assert set(generate("red,blue", count=20).split(",")) == {"red", "blue"}
|
||||
|
||||
|
||||
def test_empty_variable_prompt_returns_empty_result():
|
||||
assert generate("", fixed="portrait") == ""
|
||||
@@ -1,83 +0,0 @@
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
RUNTIME_FILES = [
|
||||
ROOT / "__init__.py",
|
||||
ROOT / "nnodes.py",
|
||||
ROOT / "py" / "image_captioning_node.py",
|
||||
ROOT / "requirements.txt",
|
||||
ROOT / "pyproject.toml",
|
||||
]
|
||||
|
||||
|
||||
def test_runtime_has_no_llama_cpp_dependency():
|
||||
for path in RUNTIME_FILES:
|
||||
assert "llama_cpp" not in path.read_text(), path
|
||||
|
||||
|
||||
def test_llava_node_is_no_longer_registered():
|
||||
source = (ROOT / "py" / "image_captioning_node.py").read_text()
|
||||
assert '"Llava Clip Loader [n-suite]"' not in source
|
||||
assert "Llava15ChatHandler" not in source
|
||||
|
||||
|
||||
def test_readme_announces_breaking_change_and_model_downloads():
|
||||
readme = (ROOT / "README.md").read_text()
|
||||
assert "Breaking change in 1.2.0" in readme
|
||||
assert "was never downloaded automatically" in readme
|
||||
assert "Downloads happen on first model use" in readme
|
||||
assert "3.72 GB" in readme
|
||||
assert "0.37 GB" in readme
|
||||
assert "git checkout ae7cc84" in readme
|
||||
|
||||
|
||||
def test_dependencies_allow_current_comfyui_versions():
|
||||
requirements = (ROOT / "requirements.txt").read_text().splitlines()
|
||||
pyproject = (ROOT / "pyproject.toml").read_text()
|
||||
assert "moviepy>=2.2.1,<3" in requirements
|
||||
assert '"moviepy>=2.2.1,<3"' in pyproject
|
||||
assert "huggingface-hub" in requirements
|
||||
assert "transformers>=4.36.2,<5" in requirements
|
||||
assert "timm>=1.0.22" in requirements
|
||||
assert "accelerate>=1.0,<2" in requirements
|
||||
assert "scikit-build" not in requirements
|
||||
|
||||
|
||||
def test_extension_does_not_install_packages_during_import():
|
||||
bootstrap = (ROOT / "__init__.py").read_text()
|
||||
assert "check_and_install" not in bootstrap
|
||||
|
||||
|
||||
def test_frontend_uses_managed_dom_widgets():
|
||||
widgets = (ROOT / "js" / "extended_widgets.js").read_text()
|
||||
assert "addDOMWidget" in widgets
|
||||
assert "addCustomWidget" not in widgets
|
||||
assert "onDrawBackground" not in widgets
|
||||
assert "graph._nodes" not in widgets
|
||||
|
||||
|
||||
def test_dynamic_widgets_use_current_removal_api_and_node_id():
|
||||
dynamic_prompt = (ROOT / "js" / "dynamicPrompt.js").read_text()
|
||||
gpt_sampler = (ROOT / "js" / "gptSampler.js").read_text()
|
||||
assert 'nodeData.name !== "DynamicPrompt [n-suite]"' in dynamic_prompt
|
||||
assert "removeWidget(widget)" in dynamic_prompt
|
||||
assert "removeWidget(widget)" in gpt_sampler
|
||||
|
||||
|
||||
def test_external_repositories_and_model_archive_are_pinned():
|
||||
bootstrap = (ROOT / "__init__.py").read_text()
|
||||
assert 'RIFE_REVISION = "a8a8035323b1c1a4a20753c751780e5b0a879455"' in bootstrap
|
||||
captioning = (ROOT / "py" / "image_captioning_node.py").read_text()
|
||||
assert 'MOONDREAM_REVISION = "f6e9da68e8f1b78b8f3ee10905d56826db7a5802"' in captioning
|
||||
assert 'RIFE_MODEL_REVISION = "572480112b87f9bfbff7579b8a38b483766e455f"' in bootstrap
|
||||
assert "/raw/main/RIFE_trained_model" not in bootstrap
|
||||
assert "repo.git.checkout(revision)" in bootstrap
|
||||
|
||||
|
||||
def test_registry_publish_is_manual_after_nightly_validation():
|
||||
workflow = (ROOT / ".github" / "workflows" / "publish.yml").read_text()
|
||||
assert "workflow_dispatch:" in workflow
|
||||
assert "push:" not in workflow
|
||||
|
||||
readme = (ROOT / "README.md").read_text()
|
||||