nodes/Nuked88-ComfyUI-N-Nodes
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Gate stable publishing on nightly validation

Browse Source
This commit is contained in:
Nuked
2026-09-27 13:33:46 +02:00
parent ae7cc84808
commit 6d470d8a16
21 changed files with 442 additions and 1233 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.comfyignore
+3
View File
@@ -0,0 +1,3 @@
# Development-only files are not needed in the Registry package.
tests/
pytest.ini
.github/workflows/publish.yml
+5 -6
View File
@@ -1,11 +1,10 @@
name: Publish to Comfy registry
on:
workflow_dispatch:
push:
branches:
- main
paths:
- "pyproject.toml"
# Merging into main makes the commit available as the Manager's "nightly"
# version. Stable Registry releases are intentionally published only by
# manually running this workflow after nightly validation.
jobs:
publish-node:
@@ -18,4 +17,4 @@ jobs:
uses: Comfy-Org/publish-node-action@main
with:
## Add your own personal access token to your Github Repository secrets and reference it here.
personal_access_token: ${{ secrets.REGISTRY_ACCESS_TOKEN }}
personal_access_token: ${{ secrets.REGISTRY_ACCESS_TOKEN }}
README.md
+46 -106
View File
@@ -1,7 +1,7 @@
[![ko-fi](https://ko-fi.com/img/githubbutton_sm.svg)](https://ko-fi.com/C0C0AJECJ)
# ComfyUI-N-Suite
A suite of custom nodes for ComfyUI that includes Integer, string and float variable nodes, GPT nodes and video nodes.
A suite of custom nodes for ComfyUI that includes integer, string and float variable nodes, image-captioning nodes and video nodes.
> [!IMPORTANT]
> These nodes were tested primarily in Windows in the default environment provided by ComfyUI and in the environment created by the [notebook](https://github.com/comfyanonymous/ComfyUI/blob/master/notebooks/comfyui_colab.ipynb) for paperspace specifically with the cyberes/gradient-base-py3.10:latest docker image.
@@ -14,21 +14,16 @@ A suite of custom nodes for ComfyUI that includes Integer, string and float vari
`git clone https://github.com/Nuked88/ComfyUI-N-Nodes.git`
to your ComfyUI `custom_nodes` directory
2. ~~IMPORTANT: If you want the GPT nodes on GPU you'll need to run **install_dependency bat files**.
There are 2 versions: ***install_dependency_ggml_models.bat*** for the old ggmlv3 models and ***install_dependency_gguf_models.bat*** for all the new models (GGUF).
YOU CAN ONLY USE ONE OF THEM AT A TIME!
Since _llama-cpp-python_ needs to be compiled from source code to enable it to use the GPU, you will first need to have [CUDA](https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64) and visual studio 2019 or 2022 (in the case of my bat) installed to compile it. For details and the full guide you can go [HERE](https://github.com/abetlen/llama-cpp-python).~~
2. Install it through **ComfyUI Manager** (recommended), which installs the dependencies declared by the project, or run `python -m pip install -r requirements.txt` in ComfyUI's Python environment after a manual clone.
3. Restart ComfyUI.
3. If you intend to use GPTLoaderSimple with the Moondream model, you'll need to execute the 'install_extra.bat' script, which will install transformers version 4.36.2.
4. Reboot ComfyUI
ComfyUI automatically loads all custom scripts and nodes at startup.
In case you need to revert these changes (due to incompatibility with other nodes), you can utilize the 'remove_extra.bat' script.
ComfyUI will automatically load all custom scripts and nodes at startup.
> [!NOTE]
> The llama-cpp-python installation will be done automatically by the script. If you have an NVIDIA GPU NO MORE CUDA BUILD IS NECESSARY thanks to [jllllll](https://github.com/jllllll/llama-cpp-python-cuBLAS-wheels/) repo. I've also dropped the support to GGMLv3 models since all notable models should have switched to the latest version of GGUF by now.
> [!IMPORTANT]
> **Breaking change in 1.2.0:** `llama-cpp-python` integration has been removed because its platform-specific installation was the main source of installation and startup failures. GGUF text-generation models and LLaVA nodes are therefore no longer available in N-Suite. Existing workflows using `Llava Clip Loader` must remove that node; `GPT Loader Simple` and `GPT Sampler` now support only Moondream and JoyTag. N-Suite no longer detects, downloads, or installs `llama-cpp-python`.
> [!WARNING]
> **The `Llava Clip Loader` node and the GGUF text-generation path of `GPT Loader Simple` / `GPT Sampler` have been removed.** To keep using those legacy nodes, install the last revision that contains them with `git checkout ae7cc84`. That revision is unsupported and retains the `llama-cpp-python` installation problems; use it in a separate ComfyUI installation or Python environment.
> [!NOTE]
> Since 14/02/2024, the node has undergone a massive rewrite, which also led to the change of all node names in order to avoid any conflicts with other extensions in the future (or at least I hope so). Consequently, the old workflows are no longer compatible and will require manual replacement of each node.
@@ -170,114 +165,60 @@ The node-variables are:
- String
## 🤖 GPTLoaderSimple and GPTSampler 🤖
## 🤖 Image captioning: GPTLoaderSimple and GPTSampler 🤖
These custom nodes are designed to enhance the capabilities of the ConfyUI framework by enabling text generation using GGUF GPT models. This README provides an overview of the two custom nodes and their usage within ConfyUI.
The legacy node identifiers are retained so existing Moondream and JoyTag workflows continue to load, but these nodes are now limited to image captioning:
You can add in the _extra_model_paths.yaml_ the path where your model GGUF are in this way (example):
- **Moondream:** its `config.json`, `model.safetensors`, and `tokenizer.json` files are downloaded automatically from the `vikhyatk/moondream1` Hugging Face repository the first time Moondream is loaded.
- **JoyTag:** its model snapshot is downloaded automatically from the `fancyfeast/joytag` Hugging Face repository the first time JoyTag is loaded.
- **LLaVA:** was never downloaded automatically. Its GGUF model and projector had to be installed manually; support has now been removed together with `llama-cpp-python`.
`other_ui:
base_path: I:\\text-generation-webui
GPTcheckpoints: models/`
Otherwise it will create a GPTcheckpoints folder in the model folder of ComfyUI where you can place your .gguf models.
Two folders have also been created within the 'Llava' directory in the 'GPTcheckpoints' folder for the LLava model:
`clips`: This folder is designated for storing the clips for your LLava models (usually, files that start with **mm** in the repository).
`models`: This folder is designated for storing the LLava models.
This nodes actually supports 4 different models:
- All the GGUF supported by [llama.cpp](https://github.com/ggerganov/llama.cpp)
- Llava
- Moondream
- Joytag
#### GGUF LLM
The GGUF models can be downloaded from the [Huggingface Hub](https://huggingface.co/models?search=gguf)
[HERE](https://www.youtube.com/watch?v=gzTqXbF0S-w) a video of an example of how to use the GGUF models by [boricuapab](https://github.com/boricuapab)
#### Llava
Here a small list of the models supported by this nodes:
[LlaVa 1.5 7B](https://huggingface.co/mys/ggml_llava-v1.5-7b/)
[LlaVa 1.5 13B](https://huggingface.co/mys/ggml_llava-v1.5-13b)
[LlaVa 1.6 Mistral 7B](https://huggingface.co/cjpais/llava-1.6-mistral-7b-gguf/)
[BakLLaVa](https://huggingface.co/mys/ggml_bakllava-1)
[Nous Hermes 2 Vision](https://huggingface.co/billborkowski/llava-NousResearch_Nous-Hermes-2-Vision-GGUF)
####Example with Llava model:
![alt text](./img/image-5.png)
#### Moondream
The model will be automatically downloaded when you run the first time.
Anyway, it is available [HERE](https://huggingface.co/vikhyatk/moondream1/tree/main)
The code taken from [this repository](https://github.com/vikhyat/moondream)
####Example with Moondream model:
![alt text](./img/image-6.png)
#### Joytag
The model will be automatically downloaded when you run the first time.
Anyway, it is available [HERE](https://huggingface.co/fancyfeast/joytag/tree/main)
The code taken from [this repository](https://github.com/fpgaminer/joytag)
####Example with Joytag model:
![alt text](./img/image-7.png)
Downloads happen on first model use, not merely when ComfyUI starts. An internet connection and sufficient disk space are required for that initial load. Models are stored under `ComfyUI/models/GPTcheckpoints/moondream` and `ComfyUI/models/GPTcheckpoints/joytag`.
### GPTLoaderSimple
![alt text](./img/image11.png)
The `GPTLoaderSimple` node is responsible for loading GPT model checkpoints and creating an instance of the Llama library for text generation. It provides an interface to configure GPU layers, the number of threads, and maximum context for text generation.
#### Input Fields
- `ckpt_name`: Select the GPT checkpoint name from the available options (joytag and moondream will be automatically downloaded used the first time).
- `gpu_layers`: Specify the number of GPU layers to use (default: 27).
- `n_threads`: Specify the number of threads for text generation (default: 8).
- `max_ctx`: Specify the maximum context length for text generation (default: 2048).
#### Output
The node returns an instance of the Llama library (MODEL) and the path to the loaded checkpoint (STRING).
`GPTLoaderSimple` loads either Moondream or JoyTag. The `gpu_layers` field is retained for workflow compatibility: set it to `0` for CPU, or to a value greater than zero for GPU. The old `n_threads` and `max_ctx` fields are also retained so saved workflows continue to deserialize, but they do not affect these image-captioning models.
### GPTSampler
![alt text](./img/image-8.png)
Connect an image and, for Moondream, a question or instruction in `prompt`. JoyTag uses `max_tags` to limit the number of returned tags. The advanced text-generation controls remain visible for workflow compatibility but no longer apply to GGUF text generation.
The `GPTSampler` node facilitates text generation using GPT models based on the input prompt and various generation parameters. It allows you to control aspects like temperature, top-p sampling, penalties, and more.
### Why LLaVA was removed
The old LLaVA implementation was not independent: both `Llama` and `Llava15ChatHandler` came from `llama-cpp-python`. Automatically downloading the LLaVA GGUF files would therefore not solve the native-library installation failure. Restoring LLaVA without restoring `llama-cpp-python` requires a new backend (for example Transformers) and a migration path for existing workflows. Such a replacement should download model weights only when the user executes the loader, show the repository and approximate download size, and use the normal Hugging Face cache rather than downloading during N-Suite import.
#### Input Fields
## Installation and maintenance status
- `prompt`: Enter the input prompt for text generation.
- `image`: Image input for Joytag, moondream and llava models.
- `model`: Choose the GPT model to use for text generation.
- `max_tokens`: Set the maximum number of tokens in the generated text (default: 128).
- `temperature`: Set the temperature parameter for randomness (default: 0.7).
- `top_p`: Set the top-p probability for nucleus sampling (default: 0.5).
- `logprobs`: Specify the number of log probabilities to output (default: 0).
- `echo`: Enable or disable printing the input prompt alongside the generated text.
- `stop_token`: Specify the token at which text generation stops.
- `frequency_penalty`, `presence_penalty`, `repeat_penalty`: Control word generation penalties.
- `top_k`: Set the top-k tokens to consider during generation (default: 40).
- `tfs_z`: Set the temperature scaling factor for top frequent samples (default: 1.0).
- `print_output`: Enable or disable printing the generated text to the console.
- `cached`: Choose whether to use cached generation (default: NO).
- `prefix`, `suffix`: Specify text to prepend and append to the prompt.
- `max_tags`: This only affect the max number of tags generated by joydag.
N-Suite is published through the Comfy Registry and is installable by current versions of ComfyUI Manager. Dependencies are declared in both `pyproject.toml` and `requirements.txt`; the extension no longer runs `pip install` itself during import. A manual Git installation remains supported, but the user must install `requirements.txt` into the exact Python environment used by ComfyUI.
#### Output
### Testing this release through Manager Nightly
The node returns the generated text along with a UI-friendly representation.
ComfyUI Manager's **nightly** entry is the current Git revision from the repository's default branch; it is not a separate version published to the Comfy Registry. Consequently, a pull request cannot be selected as nightly while it is still unmerged. The test flow for this release is:
1. merge the pull request into `main`;
2. open ComfyUI Manager and select N-Suite's `nightly` version;
3. restart ComfyUI and validate the nodes and existing workflows;
4. only after validation, manually run the `Publish to Comfy registry` GitHub Actions workflow to publish version `1.2.0` as stable.
The Registry workflow is deliberately manual: merging a change to `pyproject.toml` no longer publishes an untested stable version automatically. Before the merge, testers can still clone the pull-request branch manually under `custom_nodes`, but Manager will not label that branch as `nightly`.
Practical-RIFE and the Moondream helper remain intentionally separate upstream repositories: their code is **not** copied, merged, or vendored into N-Suite. N-Suite clones each repository into its own directory under `libs/`, as before. To avoid silently running newer upstream code that has not been tested with this suite, each checkout is fixed to a known revision:
- Practical-RIFE: `a8a8035323b1c1a4a20753c751780e5b0a879455` (12 August 2024);
- N-Suite Moondream helper: `38af98596e59f2a6c25c6b52b2bd5a672dab4144` (29 January 2024);
- the RIFE 4.7 model archive URL is fixed to DreamingAI revision `572480112b87f9bfbff7579b8a38b483766e455f` instead of the moving `main` branch.
On startup, an existing managed checkout is returned to the corresponding revision. Local changes inside `libs/rifle` or `libs/moondream_repo` must therefore be committed or moved elsewhere before starting ComfyUI; Git deliberately refuses the checkout rather than overwriting them.
### Known open issues checked on 27 September 2026
- [#90](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/90) reports that MoviePy 2.x removed `moviepy.editor`. N-Suite currently uses that API, so dependencies are constrained to `moviepy<2` until the video nodes are migrated.
- [#87](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/87), [#84](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/84), [#83](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/83), [#63](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/63), and [#56](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/56) report missing nodes or installation/import failures. Declaring all direct Python dependencies and removing runtime `pip` calls addresses part, but not necessarily all, of this group.
- [#80](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/80), [#79](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/79), and [#75](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/75) are specifically about missing or incompatible `llama_cpp`; version 1.2 removes that failing integration.
- [#89](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/89) reports creation of an output folder at startup; [#78](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/78), [#62](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/62), [#60](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/60), and [#59](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/59) cover video frame, memory and path handling and remain separate work.
- [#66](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/66) and [#65](https://github.com/Nuked88/ComfyUI-N-Nodes/issues/65) report frontend compatibility problems. The video previews now use ComfyUI's managed `addDOMWidget` lifecycle, the undefined global from #65 has been removed, and widget removal uses `removeWidget`; further reports should be checked against a current frontend build.
This list is a triage summary, not a claim that the referenced issues are fixed merely by the packaging changes above.
## Image Pad For Outpainting Advanced
![alt text](./img/image-14.png)
@@ -366,4 +307,3 @@ Feel free to contribute to this project by reporting issues or suggesting improv
## License
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
__init__.py
+55 -56
View File
@@ -1,74 +1,73 @@
import importlib.util
import os
import sys
from .nnodes import init, get_ext_dir,check_and_install,downloader,get_commit,color
import folder_paths
import traceback
from pathlib import Path
NODE_CLASS_MAPPINGS = {}
NODE_DISPLAY_NAME_MAPPINGS = {}
WEB_DIRECTORY = "./js"
RIFE_REPOSITORY = "https://github.com/hzwer/Practical-RIFE.git"
RIFE_REVISION = "a8a8035323b1c1a4a20753c751780e5b0a879455"
MOONDREAM_REPOSITORY = "https://github.com/Nuked88/moondream.git"
MOONDREAM_REVISION = "38af98596e59f2a6c25c6b52b2bd5a672dab4144"
RIFE_MODEL_REVISION = "572480112b87f9bfbff7579b8a38b483766e455f"
if init():
print("------------------------------------------")
print(f"{color.BLUE}### N-Suite Revision:{color.END} {color.GREEN}{get_commit()} {color.END}")
py = Path(get_ext_dir("py"))
files = list(py.glob("*.py"))
check_and_install('packaging')
check_and_install('py-cpuinfo',"cpuinfo")
check_and_install('gitpython','git')
check_and_install('moviepy')
check_and_install("opencv-python","cv2")
check_and_install('scikit-build',"skbuild")
#LLAMA DEPENTENCIES
check_and_install('typing')
check_and_install('diskcache')
check_and_install('llama_cpp')
check_and_install('timm',"timm","0.9.12",reboot=True)
#check_and_install('gitpython',"git")
#check_and_install('sentencepiece')
#check_and_install("accelerate")
#check_and_install('transformers','transformers',"4.36.2")
def clone_at_revision(repo_class, repository, destination, revision):
"""Clone an external repository once and keep it on a tested revision."""
repo = repo_class.clone_from(repository, destination) if not os.path.exists(destination) else repo_class(destination)
if repo.head.commit.hexsha != revision:
repo.git.checkout(revision)
return repo
# Pytest imports repository-level __init__.py files while discovering tests. A
# standalone import has no package context and, unlike ComfyUI, cannot resolve
# the extension's relative imports. Leave the mappings empty in that context.
if __package__:
from .nnodes import color, downloader, get_commit, get_ext_dir, init
#git clone https://github.com/hzwer/Practical-RIFE.git
from git import Repo
if not os.path.exists(os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","rifle")):
Repo.clone_from("https://github.com/hzwer/Practical-RIFE.git", os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","rifle"))
if init():
print("------------------------------------------")
print(f"{color.BLUE}### N-Suite Revision:{color.END} {color.GREEN}{get_commit()} {color.END}")
py = Path(get_ext_dir("py"))
files = list(py.glob("*.py"))
print(
f"{color.YELLOW}N-Suite 1.2 removed the llama.cpp/GGUF and LLaVA nodes. "
f"Use commit ae7cc84 to keep the legacy nodes.{color.END}"
)
if not os.path.exists(os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","moondream_repo")):
repo = Repo.clone_from("https://github.com/Nuked88/moondream.git", os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","moondream_repo"))
from git import Repo
#commit_hash = "38af98596e59f2a6c25c6b52b2bd5a672dab4144"
#repo.git.checkout(commit_hash)
rife_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), "libs", "rifle")
clone_at_revision(Repo, RIFE_REPOSITORY, rife_path, RIFE_REVISION)
#if file moondream.py not exist
#if not os.path.exists(os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","moondream_repo","moondream","moondream.py")):
# #delete moondream_repo and download repo again
# shutil.rmtree(os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","moondream_repo"))
# repo = Repo.clone_from("https://github.com/Nuked88/moondream.git", os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","moondream_repo"))
moondream_path = os.path.join(
os.path.dirname(os.path.realpath(__file__)), "libs", "moondream_repo"
)
clone_at_revision(Repo, MOONDREAM_REPOSITORY, moondream_path, MOONDREAM_REVISION)
#if train_log folder not exists
if not os.path.exists(os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","rifle","train_log")):
downloader("https://github.com/Nuked88/DreamingAI/raw/main/RIFE_trained_model_v4.7.zip")
# code based on pysssss repo
for file in files:
try:
name = os.path.splitext(file)[0]
spec = importlib.util.spec_from_file_location(name, os.path.join(py, file))
module = importlib.util.module_from_spec(spec)
sys.modules[name] = module
spec.loader.exec_module(module)
if hasattr(module, "NODE_CLASS_MAPPINGS") and getattr(module, "NODE_CLASS_MAPPINGS") is not None:
NODE_CLASS_MAPPINGS.update(module.NODE_CLASS_MAPPINGS)
if hasattr(module, "NODE_DISPLAY_NAME_MAPPINGS") and getattr(module, "NODE_DISPLAY_NAME_MAPPINGS") is not None:
NODE_DISPLAY_NAME_MAPPINGS.update(module.NODE_DISPLAY_NAME_MAPPINGS)
except Exception as e:
traceback.print_exc()
if not os.path.exists(os.path.join(rife_path, "train_log")):
downloader(
f"https://raw.githubusercontent.com/Nuked88/DreamingAI/{RIFE_MODEL_REVISION}/RIFE_trained_model_v4.7.zip"
)
# Code based on pysssss's repository.
for file in files:
try:
name = os.path.splitext(file)[0]
spec = importlib.util.spec_from_file_location(name, os.path.join(py, file))
module = importlib.util.module_from_spec(spec)
sys.modules[name] = module
spec.loader.exec_module(module)
mappings = getattr(module, "NODE_CLASS_MAPPINGS", None)
if mappings is not None:
NODE_CLASS_MAPPINGS.update(mappings)
display_mappings = getattr(module, "NODE_DISPLAY_NAME_MAPPINGS", None)
if display_mappings is not None:
NODE_DISPLAY_NAME_MAPPINGS.update(display_mappings)
except Exception:
traceback.print_exc()
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"]
WEB_DIRECTORY = "./js"
install_extra.bat
-15
View File
@@ -1,15 +0,0 @@
@echo off
set "python_exec=..\..\..\python_embeded\python.exe"
echo Installing dependency for moondream_repo...
if exist "%python_exec%" (
echo Installing with ComfyUI Portable
"%python_exec%" -s -m pip install transformers==4.36.2
echo Done. Please reboot ComfyUI.
) else (
echo Installing with system Python
pip install transformers==4.36.2
echo Done. Please reboot ComfyUI.
)
pause
js/cte_advanced.js
+3 -2
View File
@@ -98,7 +98,8 @@ app.registerExtension({
const styles = this.widgets.find((w) => w.name === "styles");
const p_prompt = this.widgets.find((w) => w.name === "positive_prompt");
const n_prompt = this.widgets.find((w) => w.name === "negative_prompt");
const cb = nodeData.callback;
if (!styles || !p_prompt || !n_prompt) return;
const cb = styles.callback;
let addedd_positive_prompt = "";
let addedd_negative_prompt = "";
styles.callback = function () {
@@ -131,7 +132,7 @@ app.registerExtension({
if (!ok) {
if (styles.value === styles.options.values[0]) {
value = styles.options.values[0];
styles.value = styles.options.values[0];
}
styles.value = styles.options.values[index-1];
js/dynamicPrompt.js
+20 -36
View File
@@ -3,42 +3,26 @@ import { ComfyWidgets } from "/scripts/widgets.js";
app.registerExtension({
name: "n.DynamicPrompt",
async beforeRegisterNodeDef(nodeType, nodeData, app) {
if (nodeData.name === "DynamicPrompt") {
console.warn("DynamicPrompt detected")
const onExecuted = nodeType.prototype.onExecuted;
async beforeRegisterNodeDef(nodeType, nodeData) {
if (nodeData.name !== "DynamicPrompt [n-suite]") return;
nodeType.prototype.onExecuted = function (message) {
onExecuted?.apply(this, arguments);
const pos_cached = this.widgets.findIndex((w) => w.name === "cached");
console.warn("value:"+pos_cached)
if (this.widgets) {
const pos_text = this.widgets.findIndex((w) => w.name === "text");
if (pos_text !== -1) {
for (let i = pos_text; i < this.widgets.length; i++) {
this.widgets[i].onRemove?.();
}
this.widgets.length = pos_text;
}
}
if (this.widgets[pos_cached].value === "NO") {
const w = ComfyWidgets["STRING"](this, "text", ["STRING", { multiline: true }], app);
//random seed
var rnm = Math.floor(Math.random() * 10000)
w.widget.value = rnm;
}
};
}
const onExecuted = nodeType.prototype.onExecuted;
nodeType.prototype.onExecuted = function () {
onExecuted?.apply(this, arguments);
const widgets = this.widgets ?? [];
const cached = widgets.find((widget) => widget.name === "cached");
for (const widget of widgets.filter((item) => item.name === "text")) {
this.removeWidget(widget);
}
if (cached?.value === "NO") {
const result = ComfyWidgets.STRING(
this,
"text",
["STRING", { multiline: true }],
app,
);
result.widget.value = Math.floor(Math.random() * 10000);
}
};
},
});
js/extended_widgets.js
+62 -310
View File
@@ -1,329 +1,81 @@
//extended_widgets.js
import { api } from "/scripts/api.js"
import { app } from "/scripts/app.js";
import { api } from "/scripts/api.js";
import { ComfyWidgets } from "/scripts/widgets.js";
const MultilineSymbol = Symbol();
const MultilineResizeSymbol = Symbol();
async function uploadFile(file, updateNode, node, pasted = false) {
const videoWidget = node.widgets.find((w) => w.name === "video");
try {
// Wrap file in formdata so it includes filename
const body = new FormData();
body.append("image", file);
if (pasted) {
body.append("subfolder", "pasted");
}
else {
body.append("subfolder", "n-suite");
}
const resp = await api.fetchApi("/upload/image", {
method: "POST",
body,
});
if (resp.status === 200) {
const data = await resp.json();
// Add the file to the dropdown list and update the widget value
let path = data.name;
if (!videoWidget.options.values.includes(path)) {
videoWidget.options.values.push(path);
}
if (updateNode) {
videoWidget.value = path;
if (data.subfolder) path = data.subfolder + "/" + path;
showVideoInput(path,node);
}
} else {
alert(resp.status + " - " + resp.statusText);
}
} catch (error) {
alert(error);
}
function buildViewUrl(name, type, defaultSubfolder) {
const separator = name.lastIndexOf("/");
const subfolder = separator >= 0 ? name.slice(0, separator) : defaultSubfolder;
const filename = separator >= 0 ? name.slice(separator + 1) : name;
const params = new URLSearchParams({ filename, type, subfolder });
return api.apiURL(`/view?${params.toString()}`);
}
function addVideo(node, name,src, app,autoplay_value) {
const MIN_SIZE = 50;
function computeSize(size) {
try{
if (node.widgets[0].last_y == null) return;
function updateVideoWidget(node, widgetName, url) {
const widget = node.widgets?.find((item) => item.name === widgetName);
if (!widget?.element) return;
widget.element.src = url;
}
let y = node.widgets[0].last_y;
let freeSpace = size[1] - y;
function addVideo(node, name, src, autoplayValue) {
const video = document.createElement("video");
video.controls = true;
video.loop = true;
video.muted = true;
video.autoplay = autoplayValue;
video.playsInline = true;
video.src = src || "";
video.style.width = "100%";
video.style.height = "100%";
video.style.objectFit = "contain";
// Compute the height of all non customvideo widgets
let widgetHeight = 0;
const multi = [];
for (let i = 0; i < node.widgets.length; i++) {
const w = node.widgets[i];
if (w.type === "customvideo") {
multi.push(w);
} else {
if (w.computeSize) {
widgetHeight += w.computeSize()[1] + 4;
} else {
widgetHeight += LiteGraph.NODE_WIDGET_HEIGHT + 4;
}
}
}
// See how large each text input can be
freeSpace -= widgetHeight;
freeSpace /= multi.length + (!!node.imgs?.length);
if (freeSpace < MIN_SIZE) {
// There isnt enough space for all the widgets, increase the size of the node
freeSpace = MIN_SIZE;
node.size[1] = y + widgetHeight + freeSpace * (multi.length + (!!node.imgs?.length));
node.graph.setDirtyCanvas(true);
}
// Position each of the widgets
for (const w of node.widgets) {
w.y = y;
if (w.type === "customvideo") {
y += freeSpace;
w.computedHeight = freeSpace - multi.length*4;
} else if (w.computeSize) {
y += w.computeSize()[1] + 4;
} else {
y += LiteGraph.NODE_WIDGET_HEIGHT + 4;
}
}
node.inputHeight = freeSpace;
}catch(e){
}
}
const widget = {
type: "customvideo",
name,
get value() {
return this.inputEl.value;
},
set value(x) {
this.inputEl.value = x;
},
draw: function (ctx, _, widgetWidth, y, widgetHeight) {
if (!this.parent.inputHeight) {
// If we are initially offscreen when created we wont have received a resize event
// Calculate it here instead
node.setSizeForImage?.();
}
const visible = app.canvas.ds.scale > 0.5 && this.type === "customvideo";
const margin = 10;
let top_offset = 5
//hack for top menu
if (localStorage.getItem("Comfy.Settings.Comfy.UseNewMenu") === '"Top"') {
top_offset = 40;
}
const elRect = ctx.canvas.getBoundingClientRect();
const transform = new DOMMatrix()
.scaleSelf(elRect.width / ctx.canvas.width, elRect.height / ctx.canvas.height)
.multiplySelf(ctx.getTransform())
.translateSelf(margin, margin + y);
const scale = new DOMMatrix().scaleSelf(transform.a, transform.d)
Object.assign(this.inputEl.style, {
transformOrigin: "0 0",
transform: scale,
left: `${transform.a + transform.e}px`,
top: `${transform.d +top_offset+ transform.f}px`,
width: `${widgetWidth - (margin * 2)}px`,
height: `${this.parent.inputHeight - (margin * 2)}px`,
position: "absolute",
background: (!node.color)?'':node.color,
color: (!node.color)?'':'white',
zIndex: app.graph._nodes.indexOf(node),
});
this.inputEl.hidden = !visible;
},
};
widget.inputEl = document.createElement("video");
// Set the video attributes
Object.assign(widget.inputEl, {
controls: true,
src: src,
poster: "",
width: 400,
height: 300,
loop: true,
muted: true,
autoplay: autoplay_value,
type : "video/mp4"
const widget = node.addDOMWidget(name, "video", video, {
hideOnZoom: false,
getMinHeight: () => 200,
getHeight: () => 240,
});
// Add video element to the body
document.body.appendChild(widget.inputEl);
widget.parent = node;
document.body.appendChild(widget.inputEl);
node.addCustomWidget(widget);
app.canvas.onDrawBackground = function () {
// Draw node isnt fired once the node is off the screen
// if it goes off screen quickly, the input may not be removed
// this shifts it off screen so it can be moved back if the node is visible.
for (let n in app.graph._nodes) {
n = graph._nodes[n];
for (let w in n.widgets) {
let wid = n.widgets[w];
if (Object.hasOwn(wid, "inputEl")) {
wid.inputEl.style.left = -8000 + "px";
wid.inputEl.style.position = "absolute";
}
}
}
};
node.onRemoved = function () {
// When removing this node we need to remove the input from the DOM
for (let y in this.widgets) {
if (this.widgets[y].inputEl) {
this.widgets[y].inputEl.remove();
}
}
};
widget.onRemove = () => {
widget.inputEl?.remove();
// Restore original size handler if we are the last
if (!--node[MultilineSymbol]) {
node.onResize = node[MultilineResizeSymbol];
delete node[MultilineSymbol];
delete node[MultilineResizeSymbol];
}
};
if (node[MultilineSymbol]) {
node[MultilineSymbol]++;
} else {
node[MultilineSymbol] = 1;
const onResize = (node[MultilineResizeSymbol] = node.onResize);
node.onResize = function (size) {
computeSize(size);
// Call original resizer handler
if (onResize) {
onResize.apply(this, arguments);
}
};
}
widget.serialize = false;
widget.options.serialize = false;
return { minWidth: 400, minHeight: 200, widget };
}
export function showVideoInput(name,node) {
const videoWidget = node.widgets.find((w) => w.name === "videoWidget");
const temp_web_url = node.widgets.find((w) => w.name === "local_url");
let folder_separator = name.lastIndexOf("/");
let subfolder = "n-suite";
if (folder_separator > -1) {
subfolder = name.substring(0, folder_separator);
name = name.substring(folder_separator + 1);
}
let url_video = api.apiURL(`/view?filename=${encodeURIComponent(name)}&type=input&subfolder=${subfolder}${app.getPreviewFormatParam()}`);
videoWidget.inputEl.src = url_video
temp_web_url.value = url_video
export function showVideoInput(name, node) {
const url = buildViewUrl(name, "input", "n-suite");
updateVideoWidget(node, "videoWidget", url);
const localUrl = node.widgets?.find((item) => item.name === "local_url");
if (localUrl) localUrl.value = url;
return url;
}
export function showVideoOutput(name,node) {
const videoWidget = node.widgets.find((w) => w.name === "videoOutWidget");
let folder_separator = name.lastIndexOf("/");
let subfolder = "n-suite/videos";
if (folder_separator > -1) {
subfolder = name.substring(0, folder_separator);
name = name.substring(folder_separator + 1);
}
let url_video = api.apiURL(`/view?filename=${encodeURIComponent(name)}&type=output&subfolder=${subfolder}${app.getPreviewFormatParam()}`);
videoWidget.inputEl.src = url_video
return url_video;
export function showVideoOutput(name, node) {
const url = buildViewUrl(name, "output", "n-suite/videos");
updateVideoWidget(node, "videoOutWidget", url);
return url;
}
export const ExtendedComfyWidgets = {
...ComfyWidgets, // Copy all the functions from ComfyWidgets
VIDEO(node, inputName, inputData, src, app,type="input",autoplay_value=true) {
try {
const videoWidget = node.widgets.find((w) => w.name === "video");
const autoplay = node.widgets.find((w) => w.name === "autoplay");
const defaultVal = "";
let res;
res = addVideo(node, inputName, src, app,autoplay_value);
if (type == "input"){
...ComfyWidgets,
VIDEO(node, inputName, _inputData, src, _app, type = "input", autoplayValue = true) {
const result = addVideo(node, inputName, src, autoplayValue);
if (type !== "input") return result;
const cb = node.callback;
videoWidget.callback = function () {
showVideoInput(videoWidget.value, node);
if (cb) {
return cb.apply(this, arguments);
}
const video = node.widgets?.find((item) => item.name === "video");
const autoplay = node.widgets?.find((item) => item.name === "autoplay");
if (video) {
const callback = video.callback;
video.callback = function () {
showVideoInput(video.value, node);
return callback?.apply(this, arguments);
};
autoplay.callback = function () {
const videoWidgetz = node.widgets.find((w) => w.name === "videoWidget");
videoWidgetz.inputEl.autoplay = autoplay.value;
showVideoInput(videoWidget.value, node);
if (cb) {
return cb.apply(this, arguments);
}
}
}
if (node.type =="LoadVideoAdvanced"){
}
return res;
}
catch (error) {
console.error("Errore in extended_widgets.js:", error);
throw error;
}
},
if (autoplay) {
const callback = autoplay.callback;
autoplay.callback = function () {
const preview = node.widgets?.find((item) => item.name === "videoWidget");
if (preview?.element) preview.element.autoplay = autoplay.value;
if (video?.value) showVideoInput(video.value, node);
return callback?.apply(this, arguments);
};
}
return result;
},
};
js/gptSampler.js
+20 -35
View File
@@ -3,41 +3,26 @@ import { ComfyWidgets } from "/scripts/widgets.js";
app.registerExtension({
name: "n.GPTSampler",
async beforeRegisterNodeDef(nodeType, nodeData, app) {
if (nodeData.name === "GPT Sampler [n-suite]") {
console.warn("GPTSampler detected")
const onExecuted = nodeType.prototype.onExecuted;
nodeType.prototype.onExecuted = function (message) {
onExecuted?.apply(this, arguments);
const pos_cached = this.widgets.findIndex((w) => w.name === "cached");
console.warn("value:"+pos_cached)
if (this.widgets) {
const pos_text = this.widgets.findIndex((w) => w.name === "text");
if (pos_text !== -1) {
for (let i = pos_text; i < this.widgets.length; i++) {
this.widgets[i].onRemove?.();
}
this.widgets.length = pos_text;
}
}
async beforeRegisterNodeDef(nodeType, nodeData) {
if (nodeData.name !== "GPT Sampler [n-suite]") return;
if (this.widgets[pos_cached].value === "NO") {
const w = ComfyWidgets["STRING"](this, "text", ["STRING", { multiline: true }], app);
//random seed
var rnm = Math.floor(Math.random() * 10000)
w.widget.value = rnm;
}
};
}
const onExecuted = nodeType.prototype.onExecuted;
nodeType.prototype.onExecuted = function () {
onExecuted?.apply(this, arguments);
const widgets = this.widgets ?? [];
const cached = widgets.find((widget) => widget.name === "cached");
for (const widget of widgets.filter((item) => item.name === "text")) {
this.removeWidget(widget);
}
if (cached?.value === "NO") {
const result = ComfyWidgets.STRING(
this,
"text",
["STRING", { multiline: true }],
app,
);
result.widget.value = Math.floor(Math.random() * 10000);
}
};
},
});
js/include_css.js
+5 -9
View File
@@ -1,15 +1,11 @@
import { $el } from "../../../scripts/ui.js";
function addStylesheet(url) {
if (url.endsWith(".js")) {
url = url.substr(0, url.length - 2) + "css";
}
$el("link", {
parent: document.head,
rel: "stylesheet",
type: "text/css",
href: url.startsWith("http") ? url : getUrl(url),
});
const link = document.createElement("link");
link.rel = "stylesheet";
link.href = url.startsWith("http") ? url : getUrl(url);
document.head.append(link);
}
function getUrl(path, baseUrl) {
if (baseUrl) {
@@ -19,4 +15,4 @@ function getUrl(path, baseUrl) {
}
}
addStylesheet(getUrl("styles.css", import.meta.url));
addStylesheet(getUrl("styles.css", import.meta.url));
js/videoLoadAdvanced.js
+69 -115
View File
@@ -1,142 +1,96 @@
import { app } from "/scripts/app.js";
import { api } from "/scripts/api.js"
import { ExtendedComfyWidgets,showVideoInput } from "./extended_widgets.js";
const MultilineSymbol = Symbol();
const MultilineResizeSymbol = Symbol();
import { api } from "/scripts/api.js";
import { ExtendedComfyWidgets, showVideoInput } from "./extended_widgets.js";
const VIDEO_TYPES = new Set(["video/mp4", "video/webm", "image/gif"]);
async function uploadFile(file, updateNode, node, pasted = false) {
const videoWidget = node.widgets.find((w) => w.name === "video");
const videoWidget = node.widgets?.find((widget) => widget.name === "video");
if (!videoWidget) return false;
try {
// Wrap file in formdata so it includes filename
const body = new FormData();
body.append("image", file);
if (pasted) {
body.append("subfolder", "pasted");
body.append("subfolder", pasted ? "pasted" : "n-suite");
const response = await api.fetchApi("/upload/image", { method: "POST", body });
if (!response.ok) {
alert(`${response.status} - ${response.statusText}`);
return false;
}
else {
body.append("subfolder", "n-suite");
}
const resp = await api.fetchApi("/upload/image", {
method: "POST",
body,
});
if (resp.status === 200) {
const data = await resp.json();
// Add the file to the dropdown list and update the widget value
let path = data.name;
if (!videoWidget.options.values.includes(path)) {
videoWidget.options.values.push(path);
}
if (updateNode) {
videoWidget.value = path;
if (data.subfolder) path = data.subfolder + "/" + path;
showVideoInput(path,node);
}
} else {
alert(resp.status + " - " + resp.statusText);
const data = await response.json();
const value = data.name;
const previewPath = data.subfolder ? `${data.subfolder}/${value}` : value;
if (!videoWidget.options.values.includes(value)) videoWidget.options.values.push(value);
if (updateNode) {
const oldValue = videoWidget.value;
videoWidget.value = value;
videoWidget.callback?.(value);
node.onWidgetChanged?.(videoWidget.name, value, oldValue, videoWidget);
showVideoInput(previewPath, node);
}
return true;
} catch (error) {
alert(error);
console.error("N-Suite video upload failed", error);
alert(String(error));
return false;
}
}
let uploadWidget = "";
app.registerExtension({
name: "Comfy.VideoLoadAdvanced",
async beforeRegisterNodeDef(nodeType, nodeData, app) {
async beforeRegisterNodeDef(nodeType, nodeData) {
if (nodeData.name !== "LoadVideo [n-suite]") return;
const onAdded = nodeType.prototype.onAdded;
if (nodeData.name === "LoadVideo [n-suite]") {
const onRemoved = nodeType.prototype.onRemoved;
nodeType.prototype.onAdded = function () {
onAdded?.apply(this, arguments);
const temp_web_url = this.widgets.find((w) => w.name === "local_url");
const autoplay_value = this.widgets.find((w) => w.name === "autoplay");
let uploadWidget;
const localUrl = this.widgets?.find((widget) => widget.name === "local_url");
const autoplay = this.widgets?.find((widget) => widget.name === "autoplay");
const fileInput = document.createElement("input");
Object.assign(fileInput, {
type: "file",
accept: "video/mp4,image/gif,video/webm",
style: "display: none",
onchange: async () => {
if (fileInput.files.length) {
await uploadFile(fileInput.files[0], true,this);
}
},
});
fileInput.type = "file";
fileInput.accept = "video/mp4,video/webm,image/gif";
fileInput.hidden = true;
fileInput.onchange = async () => {
if (fileInput.files?.length) await uploadFile(fileInput.files[0], true, this);
};
document.body.append(fileInput);
// Create the button widget for selecting the files
uploadWidget = this.addWidget("button", "choose file to upload", "image", () => {
fileInput.click();
},{
cursor: "grab",
},);
this.__nSuiteVideoFileInput = fileInput;
const uploadWidget = this.addWidget("button", "choose file to upload", "image", () => fileInput.click());
uploadWidget.serialize = false;
setTimeout(() => {
ExtendedComfyWidgets["VIDEO"](this, "videoWidget", ["STRING"], temp_web_url.value, app,"input", autoplay_value.value);
}, 100);
}
nodeType.prototype.onDragOver = function (e) {
if (e.dataTransfer && e.dataTransfer.items) {
const image = [...e.dataTransfer.items].find((f) => f.kind === "file");
return !!image;
}
return false;
};
// On drop upload files
nodeType.prototype.onDragDrop = function (e) {
console.log("onDragDrop called");
let handled = false;
for (const file of e.dataTransfer.files) {
if (file.type.startsWith("video/mp4")) {
const filePath = file.path || (file.webkitRelativePath || '').split('/').slice(1).join('/');
uploadFile(file, !handled,this ); // Dont await these, any order is fine, only update on first one
handled = true;
}
}
return handled;
};
nodeType.prototype.pasteFile = function(file) {
if (file.type.startsWith("video/mp4")) {
//uploadFile(file, true, is_pasted);
return true;
}
return false;
ExtendedComfyWidgets.VIDEO(
this,
"videoWidget",
["STRING"],
localUrl?.value ?? "",
app,
"input",
autoplay?.value ?? true,
);
};
nodeType.prototype.onRemoved = function () {
this.__nSuiteVideoFileInput?.remove();
delete this.__nSuiteVideoFileInput;
onRemoved?.apply(this, arguments);
};
nodeType.prototype.onDragOver = function (event) {
return [...(event.dataTransfer?.items ?? [])].some((item) => item.kind === "file");
};
nodeType.prototype.onDragDrop = function (event) {
let handled = false;
for (const file of event.dataTransfer?.files ?? []) {
if (!VIDEO_TYPES.has(file.type)) continue;
uploadFile(file, !handled, this);
handled = true;
}
return handled;
};
nodeType.prototype.pasteFile = function (file) {
if (!VIDEO_TYPES.has(file.type)) return false;
uploadFile(file, true, this, true);
return true;
};
},
});
js/videoSave.js
+9 -74
View File
@@ -1,87 +1,22 @@
import { app } from "/scripts/app.js";
import { api } from "/scripts/api.js"
import { ExtendedComfyWidgets,showVideoOutput } from "./extended_widgets.js";
const MultilineSymbol = Symbol();
const MultilineResizeSymbol = Symbol();
import { ExtendedComfyWidgets, showVideoOutput } from "./extended_widgets.js";
async function uploadFile(file, updateNode, node, pasted = false) {
const videoWidget = node.widgets.find((w) => w.name === "video");
try {
// Wrap file in formdata so it includes filename
const body = new FormData();
body.append("image", file);
if (pasted) {
body.append("subfolder", "pasted");
}
else {
body.append("subfolder", "n-suite");
}
const resp = await api.fetchApi("/upload/image", {
method: "POST",
body,
});
if (resp.status === 200) {
const data = await resp.json();
// Add the file to the dropdown list and update the widget value
let path = data.name;
if (!videoWidget.options.values.includes(path)) {
videoWidget.options.values.push(path);
}
if (updateNode) {
// showVideo(path,node);
videoWidget.value = path;
if (data.subfolder) path = data.subfolder + "/" + path;
showVideo(path,node);
}
} else {
alert(resp.status + " - " + resp.statusText);
}
} catch (error) {
alert(error);
}
}
let uploadWidget = "";
app.registerExtension({
name: "Comfy.VideoSave",
async beforeRegisterNodeDef(nodeType, nodeData, app) {
async beforeRegisterNodeDef(nodeType, nodeData) {
if (nodeData.name !== "SaveVideo [n-suite]") return;
const onExecuted = nodeType.prototype.onExecuted;
const onAdded = nodeType.prototype.onAdded;
if (nodeData.name === "SaveVideo [n-suite]") {
const onExecuted = nodeType.prototype.onExecuted;
nodeType.prototype.onAdded = function () {
ExtendedComfyWidgets["VIDEO"](this, "videoOutWidget", ["STRING"], "", app,"output");
onAdded?.apply(this, arguments);
ExtendedComfyWidgets.VIDEO(this, "videoOutWidget", ["STRING"], "", app, "output");
};
nodeType.prototype.onExecuted = function (message) {
onExecuted?.apply(this, arguments);
console.log(nodeData)
let full_path="";
for (const list of message.text) {
full_path = list;
}
let fullweb= showVideoOutput(full_path,this)
}
const paths = message?.text?.flat?.(Infinity) ?? message?.text ?? [];
const fullPath = Array.isArray(paths) ? paths.at(-1) : paths;
if (fullPath) showVideoOutput(fullPath, this);
};
},
});
nnodes.py
-148
View File
@@ -1,18 +1,12 @@
import asyncio
import os
import json
import shutil
import inspect
import aiohttp
from server import PromptServer
from tqdm import tqdm
import requests
import subprocess
import platform
import importlib.util
import torch
import folder_paths
import sys
@@ -72,148 +66,6 @@ def get_commit():
except:
return 0
def check_nvidia_gpu():
try:
# Utilizza torch per verificare la presenza di una GPU NVIDIA
return torch.cuda.is_available() and 'NVIDIA' in torch.cuda.get_device_name(0)
except Exception as e:
print(f"Error while checking for NVIDIA GPU: {e}")
return False
def get_cuda_version():
try:
if torch.cuda.is_available():
cuda_version = torch.version.cuda.replace(".","").strip()
return "cu"+cuda_version
else:
return "No NVIDIA GPU available"
except Exception as e:
print(f"Error while checking CUDA version: {e}")
return "Unable to determine CUDA version"
def check_avx2_support():
import cpuinfo
try:
info = cpuinfo.get_cpu_info()
return 'avx2' in info['flags']
except Exception as e:
print(f"Error while checking AVX2 support: {e}")
return False
def get_python_version():
if "3.9" in platform.python_version():
return "39"
elif "3.10" in platform.python_version():
return "310"
elif "3.11" in platform.python_version():
return "311"
else:
return None
def get_os():
return platform.system()
def get_os_bit():
return platform.architecture()[0].replace("bit","")
def get_platform_tag():
#return the first tag in the list of tags
try:
import packaging.tags
return list(packaging.tags.sys_tags())[0]
except:
return None
def get_last_llcpppy_version():
try:
import requests
response = requests.get("https://api.github.com/repos/abetlen/llama-cpp-python/releases/latest")
return response.json()["tag_name"].replace("v","")
except:
return "0.2.20"
from packaging import version
def check_and_install(package, import_name="", desired_version=None,reboot=False):
if import_name == "":
import_name = package
try:
library_module = importlib.import_module(import_name)
current_version = getattr(library_module, '__version__', None)
if current_version :
if current_version:
print(f"Current version of {import_name}: {current_version}")
if desired_version:
if version.parse(current_version) < version.parse(desired_version):
print(f"Updating {import_name} to version {desired_version}...")
install_package(f"{package}=={desired_version}")
print(f"{import_name} updated successfully to version {desired_version}")
#else:
# print(f"{import_name} is already up-to-date with version {current_version}")
else:
print(f"Version of {import_name}: Not found")
except ImportError:
print(f"Installing {import_name}...")
if package == "llama_cpp":
install_llama()
else:
install_package(package)
if reboot:
print(f"{color.RED}------------------------------------------{color.END}")
print(f"{color.RED}IMPORTANT: Please reboot ComfyUI!{color.END}")
print(f"{color.RED}------------------------------------------{color.END}")
def install_package(package):
subprocess.check_call([sys.executable, "-m", "pip", "install", "--no-cache-dir", package])
def install_llama():
try:
gpu = check_nvidia_gpu()
avx2 = check_avx2_support()
lcpVersion = get_last_llcpppy_version()
python_version = get_python_version()
os = get_os()
os_bit = get_os_bit()
platform_tag = get_platform_tag()
print(f"Python version: {python_version}")
print(f"OS: {os}")
print(f"OS bit: {os_bit}")
print(f"Platform tag: {platform_tag}")
if python_version == None:
print("Unsupported Python version. Please use Python 3.9, 3.10 or 3.11.")
return
#python -m pip install llama-cpp-python --force-reinstall --no-deps --index-url=https://jllllll.github.io/llama-cpp-python-cuBLAS-wheels/AVX2/cu117
if avx2:
avx="AVX2"
else:
avx="AVX"
if gpu:
cuda = get_cuda_version()
subprocess.check_call([sys.executable, "-m", "pip", "install", "llama-cpp-python", "--no-cache-dir", "--force-reinstall", "--no-deps" , f"--index-url=https://jllllll.github.io/llama-cpp-python-cuBLAS-wheels/{avx}/{cuda}"])
else:
subprocess.check_call([sys.executable, "-m", "pip", "install", f"https://github.com/abetlen/llama-cpp-python/releases/download/v{lcpVersion}/llama_cpp_python-{lcpVersion}-{platform_tag}.whl"])
except Exception as e:
print(f"Error while installing LLAMA: {e}")
# llama wheels https://github.com/jllllll/llama-cpp-python-cuBLAS-wheels
def check_module(package):
import importlib
try:
print("Detected: ", package)
importlib.import_module(package)
return True
except ImportError:
return False
import zipfile
py/gptcpp_node.py → py/image_captioning_node.py
+15 -128
View File
@@ -1,8 +1,5 @@
import folder_paths
import os
from io import BytesIO
from llama_cpp import Llama
from llama_cpp.llama_chat_format import Llava15ChatHandler
from pathlib import Path
import sys
import torch
@@ -14,12 +11,11 @@ try:
moondream_loaded = True
except Exception as e:
moondream_loaded = False
print(f"Moondream error: You should probably run install_extra.bat (windows) or install transformers==4.36.2 in the enviroment.\n Also torch must be >= 2.1.0 (ERROR: {e})")
print(f"Moondream error: reinstall N-Suite dependencies with ComfyUI Manager or requirements.txt.\nTorch must be >= 2.1.0 (ERROR: {e})")
from PIL import Image
from transformers import CodeGenTokenizerFast as Tokenizer
#,AutoTokenizer, AutoModelForCausalLM
import numpy as np
import base64
models_base_path = os.path.join(folder_paths.models_dir, "GPTcheckpoints")
_choice = ["YES", "NO"]
@@ -169,7 +165,7 @@ def run_moondream(images, prompt, max_tags, model_funct):
list_descriptions.append(moondream.answer_question(image_embeds, prompt,tokenizer))
except ValueError:
print("\n\n\n")
raise ModuleNotFoundError("Please run install_extra.bat in custom_nodes/ComfyUI-N-Nodes folder to make sure to have the required verision of Transformers installed (4.36.2).")
raise ModuleNotFoundError("Moondream requires the dependency versions declared in N-Suite requirements.txt. Reinstall dependencies with ComfyUI Manager.")
@@ -253,33 +249,6 @@ def run_internlm(image, prompt, max_tags, model_funct):
def llava_inference(model_funct,prompt,images,max_tokens,stop_token,frequency_penalty,presence_penalty,repeat_penalty,temperature,top_k,top_p):
list_descriptions = []
for image in images:
pil_image = tensor2pil(image)
# Convert the PIL image to a bytes buffer
buffer = BytesIO()
pil_image.save(buffer, format="JPEG") # You can change the format if needed
image_bytes = buffer.getvalue()
base64_string = f"data:image/jpeg;base64,{base64.b64encode(image_bytes).decode('utf-8')}"
response = model_funct.create_chat_completion( max_tokens=max_tokens, stop=[stop_token], stream=False,frequency_penalty=frequency_penalty,presence_penalty=presence_penalty ,repeat_penalty=repeat_penalty,
temperature=temperature,top_k=top_k,top_p=top_p,
messages = [
{"role": "system", "content": "You are an assistant who perfectly describes images."},
{
"role": "user",
"content": [
{"type": "image_url", "image_url": {"url": base64_string}},
{"type" : "text", "text": prompt}
]
}
]
)
list_descriptions.append(response['choices'][0]['message']['content'])
return list_descriptions
if not os.path.isdir(models_base_path):
os.mkdir(models_base_path)
@@ -294,16 +263,6 @@ if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","moo
if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","internlm")):
os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","internlm"))
"""
if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava")):
os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava"))
if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava","models")):
os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava","models"))
if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava","clips")):
os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava","clips"))
#folder_paths.folder_names_and_paths["GPTcheckpoints"] += (os.listdir(models_base_path),)
@@ -321,35 +280,9 @@ MODEL_LOAD_FUNCTIONS = {
supported_gpt_extensions = set(['.gguf'])
supported_clip_extensions = set(['.gguf','.bin'])
model_external_path = None
all_models = []
try:
model_external_path = folder_paths.folder_names_and_paths["GPTcheckpoints"][0][0]
except:
# no external folder
pass
all_llava_models = get_model_list(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava","models"),supported_gpt_extensions)
all_llava_clips = get_model_list(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava","clips"),supported_clip_extensions)
all_models = get_model_list(models_base_path,supported_gpt_extensions)
if model_external_path is not None:
all_models += get_model_list(model_external_path,supported_gpt_extensions)
all_models += all_llava_models
#extract only names
all_models = get_model_list(models_base_path, set())
all_models_names = [os.path.basename(model) for model in all_models]
all_clips_names = [os.path.basename(model) for model in all_llava_clips]
class GPTLoaderSimple:
@@ -360,38 +293,23 @@ class GPTLoaderSimple:
"gpu_layers": ("INT", {"default": 27, "min": 0, "max": 100, "step": 1}),
"n_threads": ("INT", {"default": 8, "min": 1, "max": 100, "step": 1}),
"max_ctx": ("INT", {"default": 2048, "min": 300, "max": 100000, "step": 64}),
},
"optional": {
"llava_clip": ("LLAVA_CLIP", ),
}}
}}
RETURN_TYPES = ("CUSTOM", )
RETURN_NAMES = ("model",)
FUNCTION = "load_gpt_checkpoint"
DESCRIPTION = "Loads a GPT checkpoint (GGUF format)<img src='https://compote.slate.com/images/697b023b-64a5-49a0-8059-27b963453fb1.gif?crop=780%2C520%2Cx0%2Cy0&width=1280' />"
DESCRIPTION = "Loads a Moondream or JoyTag image-captioning model. GGUF and LLaVA support was removed in version 1.2.0."
CATEGORY = "N-Suite/loaders"
def load_gpt_checkpoint(self, ckpt_name, gpu_layers,n_threads,max_ctx,llava_clip=None):
def load_gpt_checkpoint(self, ckpt_name, gpu_layers, n_threads, max_ctx):
ckpt_path = get_model_path(all_models,ckpt_name)
llm = None
#if is path
if os.path.isfile(ckpt_path):
print("GPT MODEL DETECTED")
if "llava" in ckpt_path:
if llava_clip is None:
raise ValueError("Please provide a llava clip")
llm = Llama(model_path=ckpt_path,n_gpu_layers=gpu_layers,verbose=False,n_threads=n_threads, n_ctx=max_ctx, logits_all=True,chat_handler=llava_clip)
else:
llm = Llama(model_path=ckpt_path,n_gpu_layers=gpu_layers,verbose=False,n_threads=n_threads, n_ctx=max_ctx )
else:
if ckpt_name in MODEL_LOAD_FUNCTIONS :
cpu = False if gpu_layers > 0 else True
llm = MODEL_LOAD_FUNCTIONS[ckpt_name](ckpt_path,cpu)
if ckpt_name not in MODEL_LOAD_FUNCTIONS:
raise ValueError(f"Unsupported model: {ckpt_name}")
cpu = gpu_layers == 0
llm = MODEL_LOAD_FUNCTIONS[ckpt_name](ckpt_path, cpu)
return ([llm, ckpt_name, ckpt_path],)
@@ -465,21 +383,10 @@ class GPTSampler:
if cached == "NO":
if model_name in MODEL_FUNCTIONS and os.path.isdir(model_path):
if model_name in MODEL_FUNCTIONS and os.path.isdir(model_path):
cont = MODEL_FUNCTIONS[model_name](image, prompt, max_tags, model_funct)
else:
if "llava" in model_path:
cont = llava_inference(model_funct,prompt,image,max_tokens,stop_token,frequency_penalty,presence_penalty,repeat_penalty,temperature,top_k,top_p)
else:
# Call your GPT generation function here using the provided parameters
composed_prompt = f"{prefix} {prompt} {suffix}"
cont =""
stream = model_funct( max_tokens=max_tokens, stop=[stop_token], stream=False,frequency_penalty=frequency_penalty,presence_penalty=presence_penalty ,repeat_penalty=repeat_penalty,temperature=temperature,top_k=top_k,top_p=top_p,model=model_path,prompt=composed_prompt)
cont= [stream["choices"][0]["text"]]
self.temp_prompt = cont
raise ValueError(f"Unsupported model: {model_name}")
else:
cont = self.temp_prompt
#remove fist 30 characters of cont
@@ -494,33 +401,13 @@ class GPTSampler:
return {"ui": {"text": " "}, "result": (" ",)}
class LlavaClipLoader:
@classmethod
def INPUT_TYPES(s):
return {"required": {
"clip_name": (all_clips_names, ),
}}
RETURN_TYPES = ("LLAVA_CLIP", )
RETURN_NAMES = ("llava_clip", )
FUNCTION = "load_clip_checkpoint"
CATEGORY = "N-Suite/LLava"
def load_clip_checkpoint(self, clip_name):
clip_path = get_model_path(all_llava_clips,clip_name)
clip = Llava15ChatHandler(clip_model_path = clip_path, verbose=False)
return (clip, )
NODE_CLASS_MAPPINGS = {
"GPT Loader Simple [n-suite]": GPTLoaderSimple,
"GPT Sampler [n-suite]": GPTSampler,
"Llava Clip Loader [n-suite]": LlavaClipLoader
"GPT Sampler [n-suite]": GPTSampler
}
# A dictionary that contains the friendly/humanly readable titles for the nodes
NODE_DISPLAY_NAME_MAPPINGS = {
"GPT Loader Simple [n-suite]": "GPT Loader Simple [🅝-🅢🅤🅘🅣🅔]",
"GPT Sampler [n-suite]": "GPT Text Sampler [🅝-🅢🅤🅘🅣🅔]",
"Llava Clip Loader [n-suite]": "Llava Clip Loader [🅝-🅢🅤🅘🅣🅔]"
"GPT Sampler [n-suite]": "Image Caption Sampler [🅝-🅢🅤🅘🅣🅔]"
}
}
pyproject.toml
+11 -3
View File
@@ -1,9 +1,17 @@
[project]
name = "comfyui-n-nodes"
description = "A suite of custom nodes for ConfyUI that includes GPT text-prompt generation, LoadVideo,SaveVideo,LoadFramesFromFolder and FrameInterpolator"
version = "1.1.1"
description = "A suite of custom nodes for ComfyUI that includes image captioning, LoadVideo, SaveVideo, LoadFramesFromFolder and FrameInterpolator"
version = "1.2.0"
license = { file = "LICENSE" }
dependencies = ["py-cpuinfo", "gitpython", "moviepy", "opencv-python", "scikit-build", "typing", "diskcache"]
dependencies = [
"gitpython",
"huggingface-hub",
"moviepy<2",
"opencv-python",
"scikit-build",
"timm==0.9.12",
"transformers==4.36.2",
]
[project.urls]
Repository = "https://github.com/Nuked88/ComfyUI-N-Nodes"
pytest.ini
+3
View File
@@ -0,0 +1,3 @@
[pytest]
testpaths = tests
addopts = --import-mode=importlib
remove_extra.bat
-15
View File
@@ -1,15 +0,0 @@
@echo off
set "python_exec=..\..\..\python_embeded\python.exe"
echo Restore original ComfyUI dependency...
if exist "%python_exec%" (
echo Restore with ComfyUI Portable
"%python_exec%" -s -m pip install transformers==4.26.1
echo Done. Please reboot ComfyUI.
) else (
echo Restore with system Python
pip install transformers==4.26.1
echo Done. Please reboot ComfyUI.
)
pause
requirements.txt
+4 -4
View File
@@ -1,7 +1,7 @@
py-cpuinfo
gitpython
moviepy
huggingface-hub
moviepy<2
opencv-python
scikit-build
typing
diskcache
timm==0.9.12
transformers==4.36.2
tests.py
-171
View File
@@ -1,171 +0,0 @@
import asyncio
import os
import json
import shutil
import inspect
import requests
import subprocess
import platform
import importlib.util
import sys
config = None
def check_nvidia_gpu():
try:
# Utilizza torch per verificare la presenza di una GPU NVIDIA
return torch.cuda.is_available() and 'NVIDIA' in torch.cuda.get_device_name(0)
except Exception as e:
print(f"Error while checking for NVIDIA GPU: {e}")
return False
def get_cuda_version():
try:
if torch.cuda.is_available():
cuda_version = torch.version.cuda.replace(".","").strip()
return "cu"+cuda_version
else:
return "No NVIDIA GPU available"
except Exception as e:
print(f"Error while checking CUDA version: {e}")
return "Unable to determine CUDA version"
def check_avx2_support():
import cpuinfo
try:
info = cpuinfo.get_cpu_info()
return 'avx2' in info['flags']
except Exception as e:
print(f"Error while checking AVX2 support: {e}")
return False
def get_python_version():
if "3.9" in platform.python_version():
return "39"
elif "3.10" in platform.python_version():
return "310"
elif "3.11" in platform.python_version():
return "311"
else:
return None
def get_os():
return platform.system()
def get_os_bit():
return platform.architecture()[0].replace("bit","")
import requests
def get_platform_tag(_os):
#return the first tag in the list of tags
try:
import packaging.tags
response = requests.get("https://api.github.com/repos/abetlen/llama-cpp-python/releases/latest")
jresponse= response.json()
return extract_platform_tag(jresponse,list(packaging.tags.sys_tags())[0],_os)
except:
return None
def install_package(package):
subprocess.check_call([sys.executable, "-m", "pip", "install", "--no-cache-dir", package])
def install_llama():
try:
gpu = check_nvidia_gpu()
avx2 = check_avx2_support()
lcpVersion = get_last_llcpppy_version()
python_version = get_python_version()
_os = get_os()
os_bit = get_os_bit()
platform_tag = get_platform_tag(_os)
print(f"Python version: {python_version}")
print(f"OS: {_os}")
print(f"OS bit: {os_bit}")
print(f"Platform tag: {platform_tag}")
if python_version == None:
print("Unsupported Python version. Please use Python 3.9, 3.10 or 3.11.")
return
#python -m pip install llama-cpp-python --force-reinstall --no-deps --index-url=https://jllllll.github.io/llama-cpp-python-cuBLAS-wheels/AVX2/cu117
if avx2:
avx="AVX2"
else:
avx="AVX"
if gpu:
cuda = get_cuda_version()
print(f"--index-url=https://jllllll.github.io/llama-cpp-python-cuBLAS-wheels/{avx}/{cuda}")
else:
print(f"https://github.com/abetlen/llama-cpp-python/releases/download/v{lcpVersion}/llama_cpp_python-{lcpVersion}-{platform_tag}.whl")
except Exception as e:
print(f"Error while installing LLAMA: {e}")
# llama wheels https://github.com/jllllll/llama-cpp-python-cuBLAS-wheels
def check_module(package):
import importlib
try:
print("Detected: ", package)
importlib.import_module(package)
return True
except ImportError:
return False
import zipfile
import re
def extract_platform_tag(jresponse,tag,_os):
if _os.lower()== "linux" or _os.lower()=="macosx":
print(jresponse)
for res in jresponse["assets"]:
url = res["browser_download_url"]
print(url)
pattern = r'.*-((cp\d+-cp\d+)-(manylinux|macosx)_(\d+_\d+)_(x86_64|i686)\.whl)$'
match = re.match(pattern, url)
pattern_tag = r'((cp\d+-cp\d+)-(manylinux|macosx)_\d+_\d+_(x86_64|i686))$'
match_tag = re.match(pattern_tag, tag)
if match_tag:
rl_platform_tag = f"{match_tag.group(2)}-{match_tag.group(3)}_**_{match_tag.group(4)}"
#print(rl_platform_tag)
if match:
# Estrai il platform tag dal match
url_platform_tag = f"{match.group(2)}-{match.group(3)}_**_{match.group(5)}"
final_platform_tag =f"{match.group(2)}-{match.group(3)}_{match.group(4)}_{match.group(5)}"
if rl_platform_tag == url_platform_tag:
return final_platform_tag
return None
else:
return tag
def get_last_llcpppy_version():
try:
import requests
response = requests.get("https://api.github.com/repos/abetlen/llama-cpp-python/releases/latest")
return response.json()["tag_name"].replace("v","")
except:
return "0.2.20"
install_llama()
tests/test_dynamic_prompt.py
+30
View File
@@ -0,0 +1,30 @@
import importlib.util
from pathlib import Path
MODULE_PATH = Path(__file__).resolve().parents[1] / "py" / "dynamic_prompt_node.py"
SPEC = importlib.util.spec_from_file_location("dynamic_prompt_node", MODULE_PATH)
MODULE = importlib.util.module_from_spec(SPEC)
SPEC.loader.exec_module(MODULE)
DynamicPrompt = MODULE.DynamicPrompt
def generate(variable, mode="Fixed", count=1, fixed=""):
return DynamicPrompt().prompt_generator(variable, "NO", mode, count, fixed)["result"][0]
def test_fixed_prompt_is_combined_with_requested_number_of_unique_tags():
result = generate("red, green, blue", count=2, fixed="portrait")
parts = result.split(",")
assert parts[0] == "portrait"
assert len(parts[1:]) == 2
assert len(set(parts[1:])) == 2
assert set(parts[1:]) <= {"red", "green", "blue"}
def test_requested_tag_count_is_capped_to_available_tags():
assert set(generate("red,blue", count=20).split(",")) == {"red", "blue"}
def test_empty_variable_prompt_returns_empty_result():
assert generate("", fixed="portrait") == ""
tests/test_llama_cpp_removal.py
+82
View File
@@ -0,0 +1,82 @@
from pathlib import Path
ROOT = Path(__file__).resolve().parents[1]
RUNTIME_FILES = [
ROOT / "__init__.py",
ROOT / "nnodes.py",
ROOT / "py" / "image_captioning_node.py",
ROOT / "requirements.txt",
ROOT / "pyproject.toml",
]
def test_runtime_has_no_llama_cpp_dependency():
for path in RUNTIME_FILES:
assert "llama_cpp" not in path.read_text(), path
def test_llava_node_is_no_longer_registered():
source = (ROOT / "py" / "image_captioning_node.py").read_text()
assert '"Llava Clip Loader [n-suite]"' not in source
assert "Llava15ChatHandler" not in source
def test_readme_announces_breaking_change_and_model_downloads():
readme = (ROOT / "README.md").read_text()
assert "Breaking change in 1.2.0" in readme
assert "was never downloaded automatically" in readme
assert "first time Moondream is loaded" in readme
assert "first time JoyTag is loaded" in readme
assert "git checkout ae7cc84" in readme
assert "old LLaVA implementation was not independent" in readme
def test_dependencies_are_manager_installable_and_moviepy_is_compatible():
requirements = (ROOT / "requirements.txt").read_text().splitlines()
pyproject = (ROOT / "pyproject.toml").read_text()
assert "moviepy<2" in requirements
assert '"moviepy<2"' in pyproject
assert "huggingface-hub" in requirements
assert "transformers==4.36.2" in requirements
assert "timm==0.9.12" in requirements
def test_extension_does_not_install_packages_during_import():
bootstrap = (ROOT / "__init__.py").read_text()
assert "check_and_install" not in bootstrap
def test_frontend_uses_managed_dom_widgets():
widgets = (ROOT / "js" / "extended_widgets.js").read_text()
assert "addDOMWidget" in widgets
assert "addCustomWidget" not in widgets
assert "onDrawBackground" not in widgets
assert "graph._nodes" not in widgets
def test_dynamic_widgets_use_current_removal_api_and_node_id():
dynamic_prompt = (ROOT / "js" / "dynamicPrompt.js").read_text()
gpt_sampler = (ROOT / "js" / "gptSampler.js").read_text()
assert 'nodeData.name !== "DynamicPrompt [n-suite]"' in dynamic_prompt
assert "removeWidget(widget)" in dynamic_prompt
assert "removeWidget(widget)" in gpt_sampler
def test_external_repositories_and_model_archive_are_pinned():
bootstrap = (ROOT / "__init__.py").read_text()
assert 'RIFE_REVISION = "a8a8035323b1c1a4a20753c751780e5b0a879455"' in bootstrap
assert 'MOONDREAM_REVISION = "38af98596e59f2a6c25c6b52b2bd5a672dab4144"' in bootstrap
assert 'RIFE_MODEL_REVISION = "572480112b87f9bfbff7579b8a38b483766e455f"' in bootstrap
assert "/raw/main/RIFE_trained_model" not in bootstrap
assert "repo.git.checkout(revision)" in bootstrap
def test_registry_publish_is_manual_after_nightly_validation():
workflow = (ROOT / ".github" / "workflows" / "publish.yml").read_text()
assert "workflow_dispatch:" in workflow
assert "push:" not in workflow
readme = (ROOT / "README.md").read_text()
assert "Testing this release through Manager Nightly" in readme
assert "a pull request cannot be selected as nightly while it is still unmerged" in readme