Merge pull request #91 from Nuked88/codex/review-project-issues-summary
v1.2.0: remove llama‑cpp/GGUF & LLaVA, pin external repos, modernize frontend widgets and packaging
This commit is contained in:
Nuked
@@ -0,0 +1,3 @@
|
||||
# Development-only files are not needed in the Registry package.
|
||||
tests/
|
||||
pytest.ini
|
||||
@@ -1,11 +1,10 @@
|
||||
name: Publish to Comfy registry
|
||||
on:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
paths:
|
||||
- "pyproject.toml"
|
||||
|
||||
# Merging into main makes the commit available as the Manager's "nightly"
|
||||
# version. Stable Registry releases are intentionally published only by
|
||||
# manually running this workflow after nightly validation.
|
||||
|
||||
permissions:
|
||||
issues: write
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
[](https://ko-fi.com/C0C0AJECJ)
|
||||
|
||||
# ComfyUI-N-Suite
|
||||
A suite of custom nodes for ComfyUI that includes Integer, string and float variable nodes, GPT nodes and video nodes.
|
||||
A suite of custom nodes for ComfyUI that includes integer, string and float variable nodes, image-captioning nodes and video nodes.
|
||||
|
||||
> [!IMPORTANT]
|
||||
> These nodes were tested primarily in Windows in the default environment provided by ComfyUI and in the environment created by the [notebook](https://github.com/comfyanonymous/ComfyUI/blob/master/notebooks/comfyui_colab.ipynb) for paperspace specifically with the cyberes/gradient-base-py3.10:latest docker image.
|
||||
@@ -14,21 +14,16 @@ A suite of custom nodes for ComfyUI that includes Integer, string and float vari
|
||||
`git clone https://github.com/Nuked88/ComfyUI-N-Nodes.git`
|
||||
to your ComfyUI `custom_nodes` directory
|
||||
|
||||
2. ~~IMPORTANT: If you want the GPT nodes on GPU you'll need to run **install_dependency bat files**.
|
||||
There are 2 versions: ***install_dependency_ggml_models.bat*** for the old ggmlv3 models and ***install_dependency_gguf_models.bat*** for all the new models (GGUF).
|
||||
YOU CAN ONLY USE ONE OF THEM AT A TIME!
|
||||
Since _llama-cpp-python_ needs to be compiled from source code to enable it to use the GPU, you will first need to have [CUDA](https://developer.nvidia.com/cuda-downloads?target_os=Windows&target_arch=x86_64) and visual studio 2019 or 2022 (in the case of my bat) installed to compile it. For details and the full guide you can go [HERE](https://github.com/abetlen/llama-cpp-python).~~
|
||||
2. Install it through **ComfyUI Manager** (recommended), which installs the dependencies declared by the project, or run `python -m pip install -r requirements.txt` in ComfyUI's Python environment after a manual clone.
|
||||
3. Restart ComfyUI.
|
||||
|
||||
3. If you intend to use GPTLoaderSimple with the Moondream model, you'll need to execute the 'install_extra.bat' script, which will install transformers version 4.36.2.
|
||||
4. Reboot ComfyUI
|
||||
ComfyUI automatically loads all custom scripts and nodes at startup.
|
||||
|
||||
In case you need to revert these changes (due to incompatibility with other nodes), you can utilize the 'remove_extra.bat' script.
|
||||
|
||||
ComfyUI will automatically load all custom scripts and nodes at startup.
|
||||
|
||||
> [!NOTE]
|
||||
> The llama-cpp-python installation will be done automatically by the script. If you have an NVIDIA GPU NO MORE CUDA BUILD IS NECESSARY thanks to [jllllll](https://github.com/jllllll/llama-cpp-python-cuBLAS-wheels/) repo. I've also dropped the support to GGMLv3 models since all notable models should have switched to the latest version of GGUF by now.
|
||||
> [!IMPORTANT]
|
||||
> **Breaking change in 1.2.0:** `llama-cpp-python` integration has been removed because its platform-specific installation was the main source of installation and startup failures. GGUF text-generation models and LLaVA nodes are therefore no longer available in N-Suite. Existing workflows using `Llava Clip Loader` must remove that node; `GPT Loader Simple` and `GPT Sampler` now support only Moondream and JoyTag. N-Suite no longer detects, downloads, or installs `llama-cpp-python`.
|
||||
|
||||
> [!WARNING]
|
||||
> **The `Llava Clip Loader` node and the GGUF text-generation path of `GPT Loader Simple` / `GPT Sampler` have been removed.** To keep using those legacy nodes, install the last revision that contains them with `git checkout ae7cc84`. That revision is unsupported and retains the `llama-cpp-python` installation problems; use it in a separate ComfyUI installation or Python environment.
|
||||
|
||||
> [!NOTE]
|
||||
> Since 14/02/2024, the node has undergone a massive rewrite, which also led to the change of all node names in order to avoid any conflicts with other extensions in the future (or at least I hope so). Consequently, the old workflows are no longer compatible and will require manual replacement of each node.
|
||||
@@ -170,113 +165,23 @@ The node-variables are:
|
||||
- String
|
||||
|
||||
|
||||
## 🤖 GPTLoaderSimple and GPTSampler 🤖
|
||||
## 🤖 Image captioning: GPTLoaderSimple and GPTSampler 🤖
|
||||
|
||||
These custom nodes are designed to enhance the capabilities of the ConfyUI framework by enabling text generation using GGUF GPT models. This README provides an overview of the two custom nodes and their usage within ConfyUI.
|
||||
The legacy node identifiers are retained so existing Moondream and JoyTag workflows continue to load, but these nodes are now limited to image captioning:
|
||||
|
||||
You can add in the _extra_model_paths.yaml_ the path where your model GGUF are in this way (example):
|
||||
- **Moondream:** its `config.json`, `model.safetensors`, and `tokenizer.json` files are downloaded automatically from the `vikhyatk/moondream1` Hugging Face repository the first time Moondream is loaded.
|
||||
- **JoyTag:** its model snapshot is downloaded automatically from the `fancyfeast/joytag` Hugging Face repository the first time JoyTag is loaded.
|
||||
- **LLaVA:** was never downloaded automatically. Its GGUF model and projector had to be installed manually; support has now been removed together with `llama-cpp-python`.
|
||||
|
||||
`other_ui:
|
||||
base_path: I:\\text-generation-webui
|
||||
GPTcheckpoints: models/`
|
||||
|
||||
Otherwise it will create a GPTcheckpoints folder in the model folder of ComfyUI where you can place your .gguf models.
|
||||
|
||||
Two folders have also been created within the 'Llava' directory in the 'GPTcheckpoints' folder for the LLava model:
|
||||
|
||||
`clips`: This folder is designated for storing the clips for your LLava models (usually, files that start with **mm** in the repository).
|
||||
`models`: This folder is designated for storing the LLava models.
|
||||
|
||||
This nodes actually supports 4 different models:
|
||||
- All the GGUF supported by [llama.cpp](https://github.com/ggerganov/llama.cpp)
|
||||
- Llava
|
||||
- Moondream
|
||||
- Joytag
|
||||
|
||||
|
||||
#### GGUF LLM
|
||||
|
||||
The GGUF models can be downloaded from the [Huggingface Hub](https://huggingface.co/models?search=gguf)
|
||||
|
||||
[HERE](https://www.youtube.com/watch?v=gzTqXbF0S-w) a video of an example of how to use the GGUF models by [boricuapab](https://github.com/boricuapab)
|
||||
|
||||
|
||||
#### Llava
|
||||
Here a small list of the models supported by this nodes:
|
||||
|
||||
[LlaVa 1.5 7B](https://huggingface.co/mys/ggml_llava-v1.5-7b/)
|
||||
[LlaVa 1.5 13B](https://huggingface.co/mys/ggml_llava-v1.5-13b)
|
||||
[LlaVa 1.6 Mistral 7B](https://huggingface.co/cjpais/llava-1.6-mistral-7b-gguf/)
|
||||
[BakLLaVa](https://huggingface.co/mys/ggml_bakllava-1)
|
||||
[Nous Hermes 2 Vision](https://huggingface.co/billborkowski/llava-NousResearch_Nous-Hermes-2-Vision-GGUF)
|
||||
|
||||
####Example with Llava model:
|
||||
|
||||
|
||||
#### Moondream
|
||||
The model will be automatically downloaded when you run the first time.
|
||||
Anyway, it is available [HERE](https://huggingface.co/vikhyatk/moondream1/tree/main)
|
||||
The code taken from [this repository](https://github.com/vikhyat/moondream)
|
||||
|
||||
####Example with Moondream model:
|
||||
|
||||
|
||||
#### Joytag
|
||||
The model will be automatically downloaded when you run the first time.
|
||||
Anyway, it is available [HERE](https://huggingface.co/fancyfeast/joytag/tree/main)
|
||||
The code taken from [this repository](https://github.com/fpgaminer/joytag)
|
||||
|
||||
####Example with Joytag model:
|
||||
|
||||
Downloads happen on first model use, not merely when ComfyUI starts. An internet connection and sufficient disk space are required for that initial load. Models are stored under `ComfyUI/models/GPTcheckpoints/moondream` and `ComfyUI/models/GPTcheckpoints/joytag`.
|
||||
|
||||
### GPTLoaderSimple
|
||||
|
||||
|
||||
|
||||
The `GPTLoaderSimple` node is responsible for loading GPT model checkpoints and creating an instance of the Llama library for text generation. It provides an interface to configure GPU layers, the number of threads, and maximum context for text generation.
|
||||
|
||||
|
||||
|
||||
#### Input Fields
|
||||
|
||||
- `ckpt_name`: Select the GPT checkpoint name from the available options (joytag and moondream will be automatically downloaded used the first time).
|
||||
- `gpu_layers`: Specify the number of GPU layers to use (default: 27).
|
||||
- `n_threads`: Specify the number of threads for text generation (default: 8).
|
||||
- `max_ctx`: Specify the maximum context length for text generation (default: 2048).
|
||||
|
||||
#### Output
|
||||
|
||||
The node returns an instance of the Llama library (MODEL) and the path to the loaded checkpoint (STRING).
|
||||
`GPTLoaderSimple` loads either Moondream or JoyTag. The `gpu_layers` field is retained for workflow compatibility: set it to `0` for CPU, or to a value greater than zero for GPU. The old `n_threads` and `max_ctx` fields are also retained so saved workflows continue to deserialize, but they do not affect these image-captioning models.
|
||||
|
||||
### GPTSampler
|
||||
|
||||
|
||||
|
||||
The `GPTSampler` node facilitates text generation using GPT models based on the input prompt and various generation parameters. It allows you to control aspects like temperature, top-p sampling, penalties, and more.
|
||||
|
||||
|
||||
#### Input Fields
|
||||
|
||||
- `prompt`: Enter the input prompt for text generation.
|
||||
- `image`: Image input for Joytag, moondream and llava models.
|
||||
- `model`: Choose the GPT model to use for text generation.
|
||||
- `max_tokens`: Set the maximum number of tokens in the generated text (default: 128).
|
||||
- `temperature`: Set the temperature parameter for randomness (default: 0.7).
|
||||
- `top_p`: Set the top-p probability for nucleus sampling (default: 0.5).
|
||||
- `logprobs`: Specify the number of log probabilities to output (default: 0).
|
||||
- `echo`: Enable or disable printing the input prompt alongside the generated text.
|
||||
- `stop_token`: Specify the token at which text generation stops.
|
||||
- `frequency_penalty`, `presence_penalty`, `repeat_penalty`: Control word generation penalties.
|
||||
- `top_k`: Set the top-k tokens to consider during generation (default: 40).
|
||||
- `tfs_z`: Set the temperature scaling factor for top frequent samples (default: 1.0).
|
||||
- `print_output`: Enable or disable printing the generated text to the console.
|
||||
- `cached`: Choose whether to use cached generation (default: NO).
|
||||
- `prefix`, `suffix`: Specify text to prepend and append to the prompt.
|
||||
- `max_tags`: This only affect the max number of tags generated by joydag.
|
||||
|
||||
#### Output
|
||||
|
||||
The node returns the generated text along with a UI-friendly representation.
|
||||
Connect an image and, for Moondream, a question or instruction in `prompt`. JoyTag uses `max_tags` to limit the number of returned tags. The advanced text-generation controls remain visible for workflow compatibility but no longer apply to GGUF text generation.
|
||||
|
||||
|
||||
## Image Pad For Outpainting Advanced
|
||||
@@ -366,4 +271,3 @@ Feel free to contribute to this project by reporting issues or suggesting improv
|
||||
## License
|
||||
|
||||
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
|
||||
|
||||
|
||||
+55
-56
@@ -1,74 +1,73 @@
|
||||
import importlib.util
|
||||
import os
|
||||
import sys
|
||||
from .nnodes import init, get_ext_dir,check_and_install,downloader,get_commit,color
|
||||
import folder_paths
|
||||
import traceback
|
||||
from pathlib import Path
|
||||
|
||||
NODE_CLASS_MAPPINGS = {}
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {}
|
||||
WEB_DIRECTORY = "./js"
|
||||
|
||||
RIFE_REPOSITORY = "https://github.com/hzwer/Practical-RIFE.git"
|
||||
RIFE_REVISION = "a8a8035323b1c1a4a20753c751780e5b0a879455"
|
||||
MOONDREAM_REPOSITORY = "https://github.com/Nuked88/moondream.git"
|
||||
MOONDREAM_REVISION = "38af98596e59f2a6c25c6b52b2bd5a672dab4144"
|
||||
RIFE_MODEL_REVISION = "572480112b87f9bfbff7579b8a38b483766e455f"
|
||||
|
||||
|
||||
if init():
|
||||
print("------------------------------------------")
|
||||
print(f"{color.BLUE}### N-Suite Revision:{color.END} {color.GREEN}{get_commit()} {color.END}")
|
||||
py = Path(get_ext_dir("py"))
|
||||
files = list(py.glob("*.py"))
|
||||
check_and_install('packaging')
|
||||
check_and_install('py-cpuinfo',"cpuinfo")
|
||||
check_and_install('gitpython','git')
|
||||
check_and_install('moviepy')
|
||||
check_and_install("opencv-python","cv2")
|
||||
check_and_install('scikit-build',"skbuild")
|
||||
#LLAMA DEPENTENCIES
|
||||
check_and_install('typing')
|
||||
check_and_install('diskcache')
|
||||
check_and_install('llama_cpp')
|
||||
check_and_install('timm',"timm","0.9.12",reboot=True)
|
||||
#check_and_install('gitpython',"git")
|
||||
#check_and_install('sentencepiece')
|
||||
#check_and_install("accelerate")
|
||||
#check_and_install('transformers','transformers',"4.36.2")
|
||||
|
||||
|
||||
def clone_at_revision(repo_class, repository, destination, revision):
|
||||
"""Clone an external repository once and keep it on a tested revision."""
|
||||
repo = repo_class.clone_from(repository, destination) if not os.path.exists(destination) else repo_class(destination)
|
||||
if repo.head.commit.hexsha != revision:
|
||||
repo.git.checkout(revision)
|
||||
return repo
|
||||
|
||||
# Pytest imports repository-level __init__.py files while discovering tests. A
|
||||
# standalone import has no package context and, unlike ComfyUI, cannot resolve
|
||||
# the extension's relative imports. Leave the mappings empty in that context.
|
||||
if __package__:
|
||||
from .nnodes import color, downloader, get_commit, get_ext_dir, init
|
||||
|
||||
#git clone https://github.com/hzwer/Practical-RIFE.git
|
||||
from git import Repo
|
||||
if not os.path.exists(os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","rifle")):
|
||||
Repo.clone_from("https://github.com/hzwer/Practical-RIFE.git", os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","rifle"))
|
||||
if init():
|
||||
print("------------------------------------------")
|
||||
print(f"{color.BLUE}### N-Suite Revision:{color.END} {color.GREEN}{get_commit()} {color.END}")
|
||||
py = Path(get_ext_dir("py"))
|
||||
files = list(py.glob("*.py"))
|
||||
print(
|
||||
f"{color.YELLOW}N-Suite 1.2 removed the llama.cpp/GGUF and LLaVA nodes. "
|
||||
f"Use commit ae7cc84 to keep the legacy nodes.{color.END}"
|
||||
)
|
||||
|
||||
if not os.path.exists(os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","moondream_repo")):
|
||||
repo = Repo.clone_from("https://github.com/Nuked88/moondream.git", os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","moondream_repo"))
|
||||
from git import Repo
|
||||
|
||||
#commit_hash = "38af98596e59f2a6c25c6b52b2bd5a672dab4144"
|
||||
#repo.git.checkout(commit_hash)
|
||||
rife_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), "libs", "rifle")
|
||||
clone_at_revision(Repo, RIFE_REPOSITORY, rife_path, RIFE_REVISION)
|
||||
|
||||
#if file moondream.py not exist
|
||||
#if not os.path.exists(os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","moondream_repo","moondream","moondream.py")):
|
||||
# #delete moondream_repo and download repo again
|
||||
# shutil.rmtree(os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","moondream_repo"))
|
||||
# repo = Repo.clone_from("https://github.com/Nuked88/moondream.git", os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","moondream_repo"))
|
||||
moondream_path = os.path.join(
|
||||
os.path.dirname(os.path.realpath(__file__)), "libs", "moondream_repo"
|
||||
)
|
||||
clone_at_revision(Repo, MOONDREAM_REPOSITORY, moondream_path, MOONDREAM_REVISION)
|
||||
|
||||
#if train_log folder not exists
|
||||
if not os.path.exists(os.path.join(os.path.dirname(os.path.realpath(__file__)),"libs","rifle","train_log")):
|
||||
downloader("https://github.com/Nuked88/DreamingAI/raw/main/RIFE_trained_model_v4.7.zip")
|
||||
|
||||
# code based on pysssss repo
|
||||
for file in files:
|
||||
try:
|
||||
name = os.path.splitext(file)[0]
|
||||
spec = importlib.util.spec_from_file_location(name, os.path.join(py, file))
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[name] = module
|
||||
spec.loader.exec_module(module)
|
||||
if hasattr(module, "NODE_CLASS_MAPPINGS") and getattr(module, "NODE_CLASS_MAPPINGS") is not None:
|
||||
NODE_CLASS_MAPPINGS.update(module.NODE_CLASS_MAPPINGS)
|
||||
if hasattr(module, "NODE_DISPLAY_NAME_MAPPINGS") and getattr(module, "NODE_DISPLAY_NAME_MAPPINGS") is not None:
|
||||
NODE_DISPLAY_NAME_MAPPINGS.update(module.NODE_DISPLAY_NAME_MAPPINGS)
|
||||
except Exception as e:
|
||||
traceback.print_exc()
|
||||
if not os.path.exists(os.path.join(rife_path, "train_log")):
|
||||
downloader(
|
||||
f"https://raw.githubusercontent.com/Nuked88/DreamingAI/{RIFE_MODEL_REVISION}/RIFE_trained_model_v4.7.zip"
|
||||
)
|
||||
|
||||
# Code based on pysssss's repository.
|
||||
for file in files:
|
||||
try:
|
||||
name = os.path.splitext(file)[0]
|
||||
spec = importlib.util.spec_from_file_location(name, os.path.join(py, file))
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[name] = module
|
||||
spec.loader.exec_module(module)
|
||||
mappings = getattr(module, "NODE_CLASS_MAPPINGS", None)
|
||||
if mappings is not None:
|
||||
NODE_CLASS_MAPPINGS.update(mappings)
|
||||
display_mappings = getattr(module, "NODE_DISPLAY_NAME_MAPPINGS", None)
|
||||
if display_mappings is not None:
|
||||
NODE_DISPLAY_NAME_MAPPINGS.update(display_mappings)
|
||||
except Exception:
|
||||
traceback.print_exc()
|
||||
|
||||
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"]
|
||||
WEB_DIRECTORY = "./js"
|
||||
@@ -1,15 +0,0 @@
|
||||
@echo off
|
||||
set "python_exec=..\..\..\python_embeded\python.exe"
|
||||
|
||||
echo Installing dependency for moondream_repo...
|
||||
if exist "%python_exec%" (
|
||||
echo Installing with ComfyUI Portable
|
||||
"%python_exec%" -s -m pip install transformers==4.36.2
|
||||
echo Done. Please reboot ComfyUI.
|
||||
) else (
|
||||
echo Installing with system Python
|
||||
pip install transformers==4.36.2
|
||||
echo Done. Please reboot ComfyUI.
|
||||
)
|
||||
|
||||
pause
|
||||
+3
-2
@@ -98,7 +98,8 @@ app.registerExtension({
|
||||
const styles = this.widgets.find((w) => w.name === "styles");
|
||||
const p_prompt = this.widgets.find((w) => w.name === "positive_prompt");
|
||||
const n_prompt = this.widgets.find((w) => w.name === "negative_prompt");
|
||||
const cb = nodeData.callback;
|
||||
if (!styles || !p_prompt || !n_prompt) return;
|
||||
const cb = styles.callback;
|
||||
let addedd_positive_prompt = "";
|
||||
let addedd_negative_prompt = "";
|
||||
styles.callback = function () {
|
||||
@@ -131,7 +132,7 @@ app.registerExtension({
|
||||
|
||||
if (!ok) {
|
||||
if (styles.value === styles.options.values[0]) {
|
||||
value = styles.options.values[0];
|
||||
styles.value = styles.options.values[0];
|
||||
}
|
||||
styles.value = styles.options.values[index-1];
|
||||
|
||||
|
||||
+20
-36
@@ -3,42 +3,26 @@ import { ComfyWidgets } from "/scripts/widgets.js";
|
||||
|
||||
app.registerExtension({
|
||||
name: "n.DynamicPrompt",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData, app) {
|
||||
|
||||
if (nodeData.name === "DynamicPrompt") {
|
||||
console.warn("DynamicPrompt detected")
|
||||
|
||||
const onExecuted = nodeType.prototype.onExecuted;
|
||||
async beforeRegisterNodeDef(nodeType, nodeData) {
|
||||
if (nodeData.name !== "DynamicPrompt [n-suite]") return;
|
||||
|
||||
|
||||
nodeType.prototype.onExecuted = function (message) {
|
||||
onExecuted?.apply(this, arguments);
|
||||
|
||||
const pos_cached = this.widgets.findIndex((w) => w.name === "cached");
|
||||
console.warn("value:"+pos_cached)
|
||||
|
||||
if (this.widgets) {
|
||||
const pos_text = this.widgets.findIndex((w) => w.name === "text");
|
||||
if (pos_text !== -1) {
|
||||
for (let i = pos_text; i < this.widgets.length; i++) {
|
||||
this.widgets[i].onRemove?.();
|
||||
}
|
||||
this.widgets.length = pos_text;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
if (this.widgets[pos_cached].value === "NO") {
|
||||
|
||||
const w = ComfyWidgets["STRING"](this, "text", ["STRING", { multiline: true }], app);
|
||||
//random seed
|
||||
var rnm = Math.floor(Math.random() * 10000)
|
||||
w.widget.value = rnm;
|
||||
|
||||
|
||||
}
|
||||
|
||||
};
|
||||
}
|
||||
const onExecuted = nodeType.prototype.onExecuted;
|
||||
nodeType.prototype.onExecuted = function () {
|
||||
onExecuted?.apply(this, arguments);
|
||||
const widgets = this.widgets ?? [];
|
||||
const cached = widgets.find((widget) => widget.name === "cached");
|
||||
for (const widget of widgets.filter((item) => item.name === "text")) {
|
||||
this.removeWidget(widget);
|
||||
}
|
||||
if (cached?.value === "NO") {
|
||||
const result = ComfyWidgets.STRING(
|
||||
this,
|
||||
"text",
|
||||
["STRING", { multiline: true }],
|
||||
app,
|
||||
);
|
||||
result.widget.value = Math.floor(Math.random() * 10000);
|
||||
}
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
+62
-310
@@ -1,329 +1,81 @@
|
||||
//extended_widgets.js
|
||||
import { api } from "/scripts/api.js"
|
||||
import { app } from "/scripts/app.js";
|
||||
import { api } from "/scripts/api.js";
|
||||
import { ComfyWidgets } from "/scripts/widgets.js";
|
||||
|
||||
const MultilineSymbol = Symbol();
|
||||
const MultilineResizeSymbol = Symbol();
|
||||
async function uploadFile(file, updateNode, node, pasted = false) {
|
||||
const videoWidget = node.widgets.find((w) => w.name === "video");
|
||||
|
||||
|
||||
try {
|
||||
// Wrap file in formdata so it includes filename
|
||||
const body = new FormData();
|
||||
body.append("image", file);
|
||||
if (pasted) {
|
||||
body.append("subfolder", "pasted");
|
||||
}
|
||||
else {
|
||||
body.append("subfolder", "n-suite");
|
||||
}
|
||||
|
||||
const resp = await api.fetchApi("/upload/image", {
|
||||
method: "POST",
|
||||
body,
|
||||
});
|
||||
|
||||
if (resp.status === 200) {
|
||||
const data = await resp.json();
|
||||
// Add the file to the dropdown list and update the widget value
|
||||
let path = data.name;
|
||||
|
||||
|
||||
if (!videoWidget.options.values.includes(path)) {
|
||||
videoWidget.options.values.push(path);
|
||||
}
|
||||
|
||||
if (updateNode) {
|
||||
|
||||
videoWidget.value = path;
|
||||
if (data.subfolder) path = data.subfolder + "/" + path;
|
||||
showVideoInput(path,node);
|
||||
|
||||
}
|
||||
} else {
|
||||
alert(resp.status + " - " + resp.statusText);
|
||||
}
|
||||
} catch (error) {
|
||||
alert(error);
|
||||
}
|
||||
function buildViewUrl(name, type, defaultSubfolder) {
|
||||
const separator = name.lastIndexOf("/");
|
||||
const subfolder = separator >= 0 ? name.slice(0, separator) : defaultSubfolder;
|
||||
const filename = separator >= 0 ? name.slice(separator + 1) : name;
|
||||
const params = new URLSearchParams({ filename, type, subfolder });
|
||||
return api.apiURL(`/view?${params.toString()}`);
|
||||
}
|
||||
|
||||
function addVideo(node, name,src, app,autoplay_value) {
|
||||
const MIN_SIZE = 50;
|
||||
|
||||
function computeSize(size) {
|
||||
try{
|
||||
|
||||
if (node.widgets[0].last_y == null) return;
|
||||
function updateVideoWidget(node, widgetName, url) {
|
||||
const widget = node.widgets?.find((item) => item.name === widgetName);
|
||||
if (!widget?.element) return;
|
||||
widget.element.src = url;
|
||||
}
|
||||
|
||||
let y = node.widgets[0].last_y;
|
||||
let freeSpace = size[1] - y;
|
||||
function addVideo(node, name, src, autoplayValue) {
|
||||
const video = document.createElement("video");
|
||||
video.controls = true;
|
||||
video.loop = true;
|
||||
video.muted = true;
|
||||
video.autoplay = autoplayValue;
|
||||
video.playsInline = true;
|
||||
video.src = src || "";
|
||||
video.style.width = "100%";
|
||||
video.style.height = "100%";
|
||||
video.style.objectFit = "contain";
|
||||
|
||||
// Compute the height of all non customvideo widgets
|
||||
let widgetHeight = 0;
|
||||
const multi = [];
|
||||
for (let i = 0; i < node.widgets.length; i++) {
|
||||
const w = node.widgets[i];
|
||||
if (w.type === "customvideo") {
|
||||
multi.push(w);
|
||||
} else {
|
||||
if (w.computeSize) {
|
||||
widgetHeight += w.computeSize()[1] + 4;
|
||||
} else {
|
||||
widgetHeight += LiteGraph.NODE_WIDGET_HEIGHT + 4;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// See how large each text input can be
|
||||
freeSpace -= widgetHeight;
|
||||
freeSpace /= multi.length + (!!node.imgs?.length);
|
||||
|
||||
if (freeSpace < MIN_SIZE) {
|
||||
// There isnt enough space for all the widgets, increase the size of the node
|
||||
freeSpace = MIN_SIZE;
|
||||
node.size[1] = y + widgetHeight + freeSpace * (multi.length + (!!node.imgs?.length));
|
||||
node.graph.setDirtyCanvas(true);
|
||||
}
|
||||
|
||||
// Position each of the widgets
|
||||
for (const w of node.widgets) {
|
||||
w.y = y;
|
||||
if (w.type === "customvideo") {
|
||||
y += freeSpace;
|
||||
w.computedHeight = freeSpace - multi.length*4;
|
||||
} else if (w.computeSize) {
|
||||
y += w.computeSize()[1] + 4;
|
||||
} else {
|
||||
y += LiteGraph.NODE_WIDGET_HEIGHT + 4;
|
||||
}
|
||||
}
|
||||
|
||||
node.inputHeight = freeSpace;
|
||||
}catch(e){
|
||||
|
||||
}
|
||||
}
|
||||
const widget = {
|
||||
type: "customvideo",
|
||||
name,
|
||||
get value() {
|
||||
return this.inputEl.value;
|
||||
},
|
||||
set value(x) {
|
||||
this.inputEl.value = x;
|
||||
},
|
||||
draw: function (ctx, _, widgetWidth, y, widgetHeight) {
|
||||
if (!this.parent.inputHeight) {
|
||||
// If we are initially offscreen when created we wont have received a resize event
|
||||
// Calculate it here instead
|
||||
node.setSizeForImage?.();
|
||||
|
||||
}
|
||||
const visible = app.canvas.ds.scale > 0.5 && this.type === "customvideo";
|
||||
const margin = 10;
|
||||
let top_offset = 5
|
||||
//hack for top menu
|
||||
if (localStorage.getItem("Comfy.Settings.Comfy.UseNewMenu") === '"Top"') {
|
||||
top_offset = 40;
|
||||
}
|
||||
|
||||
const elRect = ctx.canvas.getBoundingClientRect();
|
||||
const transform = new DOMMatrix()
|
||||
.scaleSelf(elRect.width / ctx.canvas.width, elRect.height / ctx.canvas.height)
|
||||
.multiplySelf(ctx.getTransform())
|
||||
.translateSelf(margin, margin + y);
|
||||
|
||||
const scale = new DOMMatrix().scaleSelf(transform.a, transform.d)
|
||||
Object.assign(this.inputEl.style, {
|
||||
transformOrigin: "0 0",
|
||||
transform: scale,
|
||||
left: `${transform.a + transform.e}px`,
|
||||
top: `${transform.d +top_offset+ transform.f}px`,
|
||||
width: `${widgetWidth - (margin * 2)}px`,
|
||||
height: `${this.parent.inputHeight - (margin * 2)}px`,
|
||||
position: "absolute",
|
||||
background: (!node.color)?'':node.color,
|
||||
color: (!node.color)?'':'white',
|
||||
zIndex: app.graph._nodes.indexOf(node),
|
||||
});
|
||||
this.inputEl.hidden = !visible;
|
||||
},
|
||||
};
|
||||
|
||||
|
||||
widget.inputEl = document.createElement("video");
|
||||
|
||||
|
||||
// Set the video attributes
|
||||
Object.assign(widget.inputEl, {
|
||||
controls: true,
|
||||
src: src,
|
||||
poster: "",
|
||||
width: 400,
|
||||
height: 300,
|
||||
loop: true,
|
||||
muted: true,
|
||||
autoplay: autoplay_value,
|
||||
type : "video/mp4"
|
||||
|
||||
const widget = node.addDOMWidget(name, "video", video, {
|
||||
hideOnZoom: false,
|
||||
getMinHeight: () => 200,
|
||||
getHeight: () => 240,
|
||||
});
|
||||
|
||||
|
||||
|
||||
|
||||
// Add video element to the body
|
||||
document.body.appendChild(widget.inputEl);
|
||||
|
||||
|
||||
|
||||
widget.parent = node;
|
||||
document.body.appendChild(widget.inputEl);
|
||||
|
||||
node.addCustomWidget(widget);
|
||||
|
||||
app.canvas.onDrawBackground = function () {
|
||||
// Draw node isnt fired once the node is off the screen
|
||||
// if it goes off screen quickly, the input may not be removed
|
||||
// this shifts it off screen so it can be moved back if the node is visible.
|
||||
for (let n in app.graph._nodes) {
|
||||
n = graph._nodes[n];
|
||||
for (let w in n.widgets) {
|
||||
let wid = n.widgets[w];
|
||||
if (Object.hasOwn(wid, "inputEl")) {
|
||||
wid.inputEl.style.left = -8000 + "px";
|
||||
wid.inputEl.style.position = "absolute";
|
||||
}
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
node.onRemoved = function () {
|
||||
// When removing this node we need to remove the input from the DOM
|
||||
for (let y in this.widgets) {
|
||||
if (this.widgets[y].inputEl) {
|
||||
this.widgets[y].inputEl.remove();
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
widget.onRemove = () => {
|
||||
widget.inputEl?.remove();
|
||||
|
||||
// Restore original size handler if we are the last
|
||||
if (!--node[MultilineSymbol]) {
|
||||
node.onResize = node[MultilineResizeSymbol];
|
||||
delete node[MultilineSymbol];
|
||||
delete node[MultilineResizeSymbol];
|
||||
}
|
||||
};
|
||||
|
||||
if (node[MultilineSymbol]) {
|
||||
node[MultilineSymbol]++;
|
||||
} else {
|
||||
node[MultilineSymbol] = 1;
|
||||
const onResize = (node[MultilineResizeSymbol] = node.onResize);
|
||||
|
||||
node.onResize = function (size) {
|
||||
|
||||
computeSize(size);
|
||||
// Call original resizer handler
|
||||
if (onResize) {
|
||||
onResize.apply(this, arguments);
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
widget.serialize = false;
|
||||
widget.options.serialize = false;
|
||||
return { minWidth: 400, minHeight: 200, widget };
|
||||
}
|
||||
|
||||
|
||||
export function showVideoInput(name,node) {
|
||||
const videoWidget = node.widgets.find((w) => w.name === "videoWidget");
|
||||
const temp_web_url = node.widgets.find((w) => w.name === "local_url");
|
||||
|
||||
|
||||
let folder_separator = name.lastIndexOf("/");
|
||||
let subfolder = "n-suite";
|
||||
if (folder_separator > -1) {
|
||||
subfolder = name.substring(0, folder_separator);
|
||||
name = name.substring(folder_separator + 1);
|
||||
}
|
||||
|
||||
let url_video = api.apiURL(`/view?filename=${encodeURIComponent(name)}&type=input&subfolder=${subfolder}${app.getPreviewFormatParam()}`);
|
||||
videoWidget.inputEl.src = url_video
|
||||
temp_web_url.value = url_video
|
||||
export function showVideoInput(name, node) {
|
||||
const url = buildViewUrl(name, "input", "n-suite");
|
||||
updateVideoWidget(node, "videoWidget", url);
|
||||
const localUrl = node.widgets?.find((item) => item.name === "local_url");
|
||||
if (localUrl) localUrl.value = url;
|
||||
return url;
|
||||
}
|
||||
|
||||
export function showVideoOutput(name,node) {
|
||||
const videoWidget = node.widgets.find((w) => w.name === "videoOutWidget");
|
||||
|
||||
|
||||
|
||||
let folder_separator = name.lastIndexOf("/");
|
||||
let subfolder = "n-suite/videos";
|
||||
if (folder_separator > -1) {
|
||||
subfolder = name.substring(0, folder_separator);
|
||||
name = name.substring(folder_separator + 1);
|
||||
}
|
||||
|
||||
|
||||
let url_video = api.apiURL(`/view?filename=${encodeURIComponent(name)}&type=output&subfolder=${subfolder}${app.getPreviewFormatParam()}`);
|
||||
videoWidget.inputEl.src = url_video
|
||||
|
||||
return url_video;
|
||||
export function showVideoOutput(name, node) {
|
||||
const url = buildViewUrl(name, "output", "n-suite/videos");
|
||||
updateVideoWidget(node, "videoOutWidget", url);
|
||||
return url;
|
||||
}
|
||||
|
||||
|
||||
|
||||
export const ExtendedComfyWidgets = {
|
||||
...ComfyWidgets, // Copy all the functions from ComfyWidgets
|
||||
|
||||
VIDEO(node, inputName, inputData, src, app,type="input",autoplay_value=true) {
|
||||
try {
|
||||
const videoWidget = node.widgets.find((w) => w.name === "video");
|
||||
const autoplay = node.widgets.find((w) => w.name === "autoplay");
|
||||
const defaultVal = "";
|
||||
let res;
|
||||
res = addVideo(node, inputName, src, app,autoplay_value);
|
||||
|
||||
if (type == "input"){
|
||||
...ComfyWidgets,
|
||||
VIDEO(node, inputName, _inputData, src, _app, type = "input", autoplayValue = true) {
|
||||
const result = addVideo(node, inputName, src, autoplayValue);
|
||||
if (type !== "input") return result;
|
||||
|
||||
const cb = node.callback;
|
||||
videoWidget.callback = function () {
|
||||
|
||||
showVideoInput(videoWidget.value, node);
|
||||
if (cb) {
|
||||
return cb.apply(this, arguments);
|
||||
}
|
||||
const video = node.widgets?.find((item) => item.name === "video");
|
||||
const autoplay = node.widgets?.find((item) => item.name === "autoplay");
|
||||
if (video) {
|
||||
const callback = video.callback;
|
||||
video.callback = function () {
|
||||
showVideoInput(video.value, node);
|
||||
return callback?.apply(this, arguments);
|
||||
};
|
||||
autoplay.callback = function () {
|
||||
const videoWidgetz = node.widgets.find((w) => w.name === "videoWidget");
|
||||
|
||||
videoWidgetz.inputEl.autoplay = autoplay.value;
|
||||
showVideoInput(videoWidget.value, node);
|
||||
if (cb) {
|
||||
return cb.apply(this, arguments);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (node.type =="LoadVideoAdvanced"){
|
||||
|
||||
|
||||
}
|
||||
|
||||
return res;
|
||||
}
|
||||
catch (error) {
|
||||
|
||||
console.error("Errore in extended_widgets.js:", error);
|
||||
throw error;
|
||||
|
||||
}
|
||||
|
||||
},
|
||||
|
||||
|
||||
if (autoplay) {
|
||||
const callback = autoplay.callback;
|
||||
autoplay.callback = function () {
|
||||
const preview = node.widgets?.find((item) => item.name === "videoWidget");
|
||||
if (preview?.element) preview.element.autoplay = autoplay.value;
|
||||
if (video?.value) showVideoInput(video.value, node);
|
||||
return callback?.apply(this, arguments);
|
||||
};
|
||||
}
|
||||
return result;
|
||||
},
|
||||
};
|
||||
|
||||
+20
-35
@@ -3,41 +3,26 @@ import { ComfyWidgets } from "/scripts/widgets.js";
|
||||
|
||||
app.registerExtension({
|
||||
name: "n.GPTSampler",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData, app) {
|
||||
|
||||
if (nodeData.name === "GPT Sampler [n-suite]") {
|
||||
console.warn("GPTSampler detected")
|
||||
|
||||
const onExecuted = nodeType.prototype.onExecuted;
|
||||
|
||||
nodeType.prototype.onExecuted = function (message) {
|
||||
onExecuted?.apply(this, arguments);
|
||||
|
||||
const pos_cached = this.widgets.findIndex((w) => w.name === "cached");
|
||||
console.warn("value:"+pos_cached)
|
||||
|
||||
if (this.widgets) {
|
||||
const pos_text = this.widgets.findIndex((w) => w.name === "text");
|
||||
if (pos_text !== -1) {
|
||||
for (let i = pos_text; i < this.widgets.length; i++) {
|
||||
this.widgets[i].onRemove?.();
|
||||
}
|
||||
this.widgets.length = pos_text;
|
||||
}
|
||||
}
|
||||
async beforeRegisterNodeDef(nodeType, nodeData) {
|
||||
if (nodeData.name !== "GPT Sampler [n-suite]") return;
|
||||
|
||||
|
||||
if (this.widgets[pos_cached].value === "NO") {
|
||||
|
||||
const w = ComfyWidgets["STRING"](this, "text", ["STRING", { multiline: true }], app);
|
||||
//random seed
|
||||
var rnm = Math.floor(Math.random() * 10000)
|
||||
w.widget.value = rnm;
|
||||
|
||||
|
||||
}
|
||||
|
||||
};
|
||||
}
|
||||
const onExecuted = nodeType.prototype.onExecuted;
|
||||
nodeType.prototype.onExecuted = function () {
|
||||
onExecuted?.apply(this, arguments);
|
||||
const widgets = this.widgets ?? [];
|
||||
const cached = widgets.find((widget) => widget.name === "cached");
|
||||
for (const widget of widgets.filter((item) => item.name === "text")) {
|
||||
this.removeWidget(widget);
|
||||
}
|
||||
if (cached?.value === "NO") {
|
||||
const result = ComfyWidgets.STRING(
|
||||
this,
|
||||
"text",
|
||||
["STRING", { multiline: true }],
|
||||
app,
|
||||
);
|
||||
result.widget.value = Math.floor(Math.random() * 10000);
|
||||
}
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
+5
-9
@@ -1,15 +1,11 @@
|
||||
import { $el } from "../../../scripts/ui.js";
|
||||
|
||||
function addStylesheet(url) {
|
||||
if (url.endsWith(".js")) {
|
||||
url = url.substr(0, url.length - 2) + "css";
|
||||
}
|
||||
$el("link", {
|
||||
parent: document.head,
|
||||
rel: "stylesheet",
|
||||
type: "text/css",
|
||||
href: url.startsWith("http") ? url : getUrl(url),
|
||||
});
|
||||
const link = document.createElement("link");
|
||||
link.rel = "stylesheet";
|
||||
link.href = url.startsWith("http") ? url : getUrl(url);
|
||||
document.head.append(link);
|
||||
}
|
||||
function getUrl(path, baseUrl) {
|
||||
if (baseUrl) {
|
||||
@@ -19,4 +15,4 @@ function getUrl(path, baseUrl) {
|
||||
}
|
||||
}
|
||||
|
||||
addStylesheet(getUrl("styles.css", import.meta.url));
|
||||
addStylesheet(getUrl("styles.css", import.meta.url));
|
||||
|
||||
+69
-115
@@ -1,142 +1,96 @@
|
||||
import { app } from "/scripts/app.js";
|
||||
import { api } from "/scripts/api.js"
|
||||
import { ExtendedComfyWidgets,showVideoInput } from "./extended_widgets.js";
|
||||
const MultilineSymbol = Symbol();
|
||||
const MultilineResizeSymbol = Symbol();
|
||||
import { api } from "/scripts/api.js";
|
||||
import { ExtendedComfyWidgets, showVideoInput } from "./extended_widgets.js";
|
||||
|
||||
const VIDEO_TYPES = new Set(["video/mp4", "video/webm", "image/gif"]);
|
||||
|
||||
async function uploadFile(file, updateNode, node, pasted = false) {
|
||||
const videoWidget = node.widgets.find((w) => w.name === "video");
|
||||
|
||||
|
||||
const videoWidget = node.widgets?.find((widget) => widget.name === "video");
|
||||
if (!videoWidget) return false;
|
||||
|
||||
try {
|
||||
// Wrap file in formdata so it includes filename
|
||||
const body = new FormData();
|
||||
body.append("image", file);
|
||||
if (pasted) {
|
||||
body.append("subfolder", "pasted");
|
||||
body.append("subfolder", pasted ? "pasted" : "n-suite");
|
||||
const response = await api.fetchApi("/upload/image", { method: "POST", body });
|
||||
if (!response.ok) {
|
||||
alert(`${response.status} - ${response.statusText}`);
|
||||
return false;
|
||||
}
|
||||
else {
|
||||
body.append("subfolder", "n-suite");
|
||||
}
|
||||
|
||||
const resp = await api.fetchApi("/upload/image", {
|
||||
method: "POST",
|
||||
body,
|
||||
});
|
||||
|
||||
if (resp.status === 200) {
|
||||
const data = await resp.json();
|
||||
// Add the file to the dropdown list and update the widget value
|
||||
let path = data.name;
|
||||
|
||||
|
||||
if (!videoWidget.options.values.includes(path)) {
|
||||
videoWidget.options.values.push(path);
|
||||
}
|
||||
|
||||
if (updateNode) {
|
||||
|
||||
videoWidget.value = path;
|
||||
if (data.subfolder) path = data.subfolder + "/" + path;
|
||||
showVideoInput(path,node);
|
||||
|
||||
}
|
||||
} else {
|
||||
alert(resp.status + " - " + resp.statusText);
|
||||
const data = await response.json();
|
||||
const value = data.name;
|
||||
const previewPath = data.subfolder ? `${data.subfolder}/${value}` : value;
|
||||
if (!videoWidget.options.values.includes(value)) videoWidget.options.values.push(value);
|
||||
if (updateNode) {
|
||||
const oldValue = videoWidget.value;
|
||||
videoWidget.value = value;
|
||||
videoWidget.callback?.(value);
|
||||
node.onWidgetChanged?.(videoWidget.name, value, oldValue, videoWidget);
|
||||
showVideoInput(previewPath, node);
|
||||
}
|
||||
return true;
|
||||
} catch (error) {
|
||||
alert(error);
|
||||
console.error("N-Suite video upload failed", error);
|
||||
alert(String(error));
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
let uploadWidget = "";
|
||||
app.registerExtension({
|
||||
name: "Comfy.VideoLoadAdvanced",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData, app) {
|
||||
async beforeRegisterNodeDef(nodeType, nodeData) {
|
||||
if (nodeData.name !== "LoadVideo [n-suite]") return;
|
||||
|
||||
const onAdded = nodeType.prototype.onAdded;
|
||||
if (nodeData.name === "LoadVideo [n-suite]") {
|
||||
const onRemoved = nodeType.prototype.onRemoved;
|
||||
nodeType.prototype.onAdded = function () {
|
||||
onAdded?.apply(this, arguments);
|
||||
const temp_web_url = this.widgets.find((w) => w.name === "local_url");
|
||||
const autoplay_value = this.widgets.find((w) => w.name === "autoplay");
|
||||
|
||||
|
||||
let uploadWidget;
|
||||
const localUrl = this.widgets?.find((widget) => widget.name === "local_url");
|
||||
const autoplay = this.widgets?.find((widget) => widget.name === "autoplay");
|
||||
const fileInput = document.createElement("input");
|
||||
Object.assign(fileInput, {
|
||||
type: "file",
|
||||
accept: "video/mp4,image/gif,video/webm",
|
||||
style: "display: none",
|
||||
onchange: async () => {
|
||||
if (fileInput.files.length) {
|
||||
await uploadFile(fileInput.files[0], true,this);
|
||||
}
|
||||
},
|
||||
});
|
||||
fileInput.type = "file";
|
||||
fileInput.accept = "video/mp4,video/webm,image/gif";
|
||||
fileInput.hidden = true;
|
||||
fileInput.onchange = async () => {
|
||||
if (fileInput.files?.length) await uploadFile(fileInput.files[0], true, this);
|
||||
};
|
||||
document.body.append(fileInput);
|
||||
// Create the button widget for selecting the files
|
||||
uploadWidget = this.addWidget("button", "choose file to upload", "image", () => {
|
||||
fileInput.click();
|
||||
},{
|
||||
cursor: "grab",
|
||||
},);
|
||||
this.__nSuiteVideoFileInput = fileInput;
|
||||
|
||||
const uploadWidget = this.addWidget("button", "choose file to upload", "image", () => fileInput.click());
|
||||
uploadWidget.serialize = false;
|
||||
|
||||
|
||||
setTimeout(() => {
|
||||
ExtendedComfyWidgets["VIDEO"](this, "videoWidget", ["STRING"], temp_web_url.value, app,"input", autoplay_value.value);
|
||||
|
||||
}, 100);
|
||||
|
||||
|
||||
}
|
||||
|
||||
|
||||
nodeType.prototype.onDragOver = function (e) {
|
||||
if (e.dataTransfer && e.dataTransfer.items) {
|
||||
const image = [...e.dataTransfer.items].find((f) => f.kind === "file");
|
||||
return !!image;
|
||||
}
|
||||
|
||||
return false;
|
||||
};
|
||||
|
||||
// On drop upload files
|
||||
nodeType.prototype.onDragDrop = function (e) {
|
||||
console.log("onDragDrop called");
|
||||
let handled = false;
|
||||
for (const file of e.dataTransfer.files) {
|
||||
if (file.type.startsWith("video/mp4")) {
|
||||
|
||||
const filePath = file.path || (file.webkitRelativePath || '').split('/').slice(1).join('/');
|
||||
|
||||
|
||||
uploadFile(file, !handled,this ); // Dont await these, any order is fine, only update on first one
|
||||
|
||||
handled = true;
|
||||
}
|
||||
}
|
||||
|
||||
return handled;
|
||||
};
|
||||
|
||||
nodeType.prototype.pasteFile = function(file) {
|
||||
if (file.type.startsWith("video/mp4")) {
|
||||
|
||||
//uploadFile(file, true, is_pasted);
|
||||
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
ExtendedComfyWidgets.VIDEO(
|
||||
this,
|
||||
"videoWidget",
|
||||
["STRING"],
|
||||
localUrl?.value ?? "",
|
||||
app,
|
||||
"input",
|
||||
autoplay?.value ?? true,
|
||||
);
|
||||
};
|
||||
nodeType.prototype.onRemoved = function () {
|
||||
this.__nSuiteVideoFileInput?.remove();
|
||||
delete this.__nSuiteVideoFileInput;
|
||||
onRemoved?.apply(this, arguments);
|
||||
};
|
||||
nodeType.prototype.onDragOver = function (event) {
|
||||
return [...(event.dataTransfer?.items ?? [])].some((item) => item.kind === "file");
|
||||
};
|
||||
nodeType.prototype.onDragDrop = function (event) {
|
||||
let handled = false;
|
||||
for (const file of event.dataTransfer?.files ?? []) {
|
||||
if (!VIDEO_TYPES.has(file.type)) continue;
|
||||
uploadFile(file, !handled, this);
|
||||
handled = true;
|
||||
}
|
||||
|
||||
|
||||
return handled;
|
||||
};
|
||||
nodeType.prototype.pasteFile = function (file) {
|
||||
if (!VIDEO_TYPES.has(file.type)) return false;
|
||||
uploadFile(file, true, this, true);
|
||||
return true;
|
||||
};
|
||||
|
||||
},
|
||||
});
|
||||
|
||||
+9
-74
@@ -1,87 +1,22 @@
|
||||
import { app } from "/scripts/app.js";
|
||||
import { api } from "/scripts/api.js"
|
||||
import { ExtendedComfyWidgets,showVideoOutput } from "./extended_widgets.js";
|
||||
const MultilineSymbol = Symbol();
|
||||
const MultilineResizeSymbol = Symbol();
|
||||
import { ExtendedComfyWidgets, showVideoOutput } from "./extended_widgets.js";
|
||||
|
||||
|
||||
async function uploadFile(file, updateNode, node, pasted = false) {
|
||||
const videoWidget = node.widgets.find((w) => w.name === "video");
|
||||
|
||||
|
||||
try {
|
||||
// Wrap file in formdata so it includes filename
|
||||
const body = new FormData();
|
||||
body.append("image", file);
|
||||
if (pasted) {
|
||||
body.append("subfolder", "pasted");
|
||||
}
|
||||
else {
|
||||
body.append("subfolder", "n-suite");
|
||||
}
|
||||
|
||||
const resp = await api.fetchApi("/upload/image", {
|
||||
method: "POST",
|
||||
body,
|
||||
});
|
||||
|
||||
if (resp.status === 200) {
|
||||
const data = await resp.json();
|
||||
// Add the file to the dropdown list and update the widget value
|
||||
let path = data.name;
|
||||
|
||||
|
||||
if (!videoWidget.options.values.includes(path)) {
|
||||
videoWidget.options.values.push(path);
|
||||
}
|
||||
|
||||
if (updateNode) {
|
||||
// showVideo(path,node);
|
||||
videoWidget.value = path;
|
||||
if (data.subfolder) path = data.subfolder + "/" + path;
|
||||
showVideo(path,node);
|
||||
|
||||
}
|
||||
} else {
|
||||
alert(resp.status + " - " + resp.statusText);
|
||||
}
|
||||
} catch (error) {
|
||||
alert(error);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
let uploadWidget = "";
|
||||
app.registerExtension({
|
||||
name: "Comfy.VideoSave",
|
||||
async beforeRegisterNodeDef(nodeType, nodeData, app) {
|
||||
async beforeRegisterNodeDef(nodeType, nodeData) {
|
||||
if (nodeData.name !== "SaveVideo [n-suite]") return;
|
||||
|
||||
const onExecuted = nodeType.prototype.onExecuted;
|
||||
|
||||
|
||||
const onAdded = nodeType.prototype.onAdded;
|
||||
if (nodeData.name === "SaveVideo [n-suite]") {
|
||||
const onExecuted = nodeType.prototype.onExecuted;
|
||||
nodeType.prototype.onAdded = function () {
|
||||
|
||||
ExtendedComfyWidgets["VIDEO"](this, "videoOutWidget", ["STRING"], "", app,"output");
|
||||
|
||||
onAdded?.apply(this, arguments);
|
||||
ExtendedComfyWidgets.VIDEO(this, "videoOutWidget", ["STRING"], "", app, "output");
|
||||
};
|
||||
nodeType.prototype.onExecuted = function (message) {
|
||||
onExecuted?.apply(this, arguments);
|
||||
console.log(nodeData)
|
||||
|
||||
let full_path="";
|
||||
|
||||
for (const list of message.text) {
|
||||
full_path = list;
|
||||
}
|
||||
|
||||
let fullweb= showVideoOutput(full_path,this)
|
||||
|
||||
}
|
||||
const paths = message?.text?.flat?.(Infinity) ?? message?.text ?? [];
|
||||
const fullPath = Array.isArray(paths) ? paths.at(-1) : paths;
|
||||
if (fullPath) showVideoOutput(fullPath, this);
|
||||
};
|
||||
|
||||
},
|
||||
});
|
||||
|
||||
@@ -1,18 +1,12 @@
|
||||
import asyncio
|
||||
import os
|
||||
import json
|
||||
import shutil
|
||||
import inspect
|
||||
import aiohttp
|
||||
from server import PromptServer
|
||||
from tqdm import tqdm
|
||||
import requests
|
||||
import subprocess
|
||||
import platform
|
||||
import importlib.util
|
||||
import torch
|
||||
import folder_paths
|
||||
import sys
|
||||
|
||||
|
||||
|
||||
@@ -72,148 +66,6 @@ def get_commit():
|
||||
except:
|
||||
return 0
|
||||
|
||||
def check_nvidia_gpu():
|
||||
try:
|
||||
# Utilizza torch per verificare la presenza di una GPU NVIDIA
|
||||
return torch.cuda.is_available() and 'NVIDIA' in torch.cuda.get_device_name(0)
|
||||
except Exception as e:
|
||||
print(f"Error while checking for NVIDIA GPU: {e}")
|
||||
return False
|
||||
|
||||
def get_cuda_version():
|
||||
try:
|
||||
if torch.cuda.is_available():
|
||||
cuda_version = torch.version.cuda.replace(".","").strip()
|
||||
|
||||
return "cu"+cuda_version
|
||||
else:
|
||||
return "No NVIDIA GPU available"
|
||||
except Exception as e:
|
||||
print(f"Error while checking CUDA version: {e}")
|
||||
return "Unable to determine CUDA version"
|
||||
|
||||
def check_avx2_support():
|
||||
import cpuinfo
|
||||
try:
|
||||
info = cpuinfo.get_cpu_info()
|
||||
return 'avx2' in info['flags']
|
||||
except Exception as e:
|
||||
print(f"Error while checking AVX2 support: {e}")
|
||||
return False
|
||||
|
||||
def get_python_version():
|
||||
if "3.9" in platform.python_version():
|
||||
return "39"
|
||||
elif "3.10" in platform.python_version():
|
||||
return "310"
|
||||
elif "3.11" in platform.python_version():
|
||||
return "311"
|
||||
else:
|
||||
return None
|
||||
|
||||
|
||||
def get_os():
|
||||
return platform.system()
|
||||
|
||||
def get_os_bit():
|
||||
return platform.architecture()[0].replace("bit","")
|
||||
|
||||
def get_platform_tag():
|
||||
#return the first tag in the list of tags
|
||||
try:
|
||||
import packaging.tags
|
||||
return list(packaging.tags.sys_tags())[0]
|
||||
except:
|
||||
return None
|
||||
def get_last_llcpppy_version():
|
||||
try:
|
||||
import requests
|
||||
|
||||
response = requests.get("https://api.github.com/repos/abetlen/llama-cpp-python/releases/latest")
|
||||
|
||||
|
||||
return response.json()["tag_name"].replace("v","")
|
||||
except:
|
||||
return "0.2.20"
|
||||
|
||||
from packaging import version
|
||||
|
||||
def check_and_install(package, import_name="", desired_version=None,reboot=False):
|
||||
if import_name == "":
|
||||
import_name = package
|
||||
try:
|
||||
library_module = importlib.import_module(import_name)
|
||||
current_version = getattr(library_module, '__version__', None)
|
||||
if current_version :
|
||||
if current_version:
|
||||
print(f"Current version of {import_name}: {current_version}")
|
||||
if desired_version:
|
||||
if version.parse(current_version) < version.parse(desired_version):
|
||||
print(f"Updating {import_name} to version {desired_version}...")
|
||||
install_package(f"{package}=={desired_version}")
|
||||
print(f"{import_name} updated successfully to version {desired_version}")
|
||||
#else:
|
||||
# print(f"{import_name} is already up-to-date with version {current_version}")
|
||||
|
||||
else:
|
||||
print(f"Version of {import_name}: Not found")
|
||||
|
||||
|
||||
except ImportError:
|
||||
print(f"Installing {import_name}...")
|
||||
if package == "llama_cpp":
|
||||
install_llama()
|
||||
else:
|
||||
install_package(package)
|
||||
if reboot:
|
||||
print(f"{color.RED}------------------------------------------{color.END}")
|
||||
print(f"{color.RED}IMPORTANT: Please reboot ComfyUI!{color.END}")
|
||||
print(f"{color.RED}------------------------------------------{color.END}")
|
||||
|
||||
def install_package(package):
|
||||
subprocess.check_call([sys.executable, "-m", "pip", "install", "--no-cache-dir", package])
|
||||
|
||||
def install_llama():
|
||||
try:
|
||||
gpu = check_nvidia_gpu()
|
||||
avx2 = check_avx2_support()
|
||||
lcpVersion = get_last_llcpppy_version()
|
||||
python_version = get_python_version()
|
||||
os = get_os()
|
||||
os_bit = get_os_bit()
|
||||
platform_tag = get_platform_tag()
|
||||
print(f"Python version: {python_version}")
|
||||
print(f"OS: {os}")
|
||||
print(f"OS bit: {os_bit}")
|
||||
print(f"Platform tag: {platform_tag}")
|
||||
if python_version == None:
|
||||
print("Unsupported Python version. Please use Python 3.9, 3.10 or 3.11.")
|
||||
return
|
||||
|
||||
|
||||
#python -m pip install llama-cpp-python --force-reinstall --no-deps --index-url=https://jllllll.github.io/llama-cpp-python-cuBLAS-wheels/AVX2/cu117
|
||||
if avx2:
|
||||
avx="AVX2"
|
||||
else:
|
||||
avx="AVX"
|
||||
|
||||
if gpu:
|
||||
cuda = get_cuda_version()
|
||||
subprocess.check_call([sys.executable, "-m", "pip", "install", "llama-cpp-python", "--no-cache-dir", "--force-reinstall", "--no-deps" , f"--index-url=https://jllllll.github.io/llama-cpp-python-cuBLAS-wheels/{avx}/{cuda}"])
|
||||
else:
|
||||
subprocess.check_call([sys.executable, "-m", "pip", "install", f"https://github.com/abetlen/llama-cpp-python/releases/download/v{lcpVersion}/llama_cpp_python-{lcpVersion}-{platform_tag}.whl"])
|
||||
except Exception as e:
|
||||
print(f"Error while installing LLAMA: {e}")
|
||||
# llama wheels https://github.com/jllllll/llama-cpp-python-cuBLAS-wheels
|
||||
|
||||
def check_module(package):
|
||||
import importlib
|
||||
try:
|
||||
print("Detected: ", package)
|
||||
importlib.import_module(package)
|
||||
return True
|
||||
except ImportError:
|
||||
return False
|
||||
import zipfile
|
||||
|
||||
|
||||
|
||||
@@ -1,8 +1,5 @@
|
||||
import folder_paths
|
||||
import os
|
||||
from io import BytesIO
|
||||
from llama_cpp import Llama
|
||||
from llama_cpp.llama_chat_format import Llava15ChatHandler
|
||||
from pathlib import Path
|
||||
import sys
|
||||
import torch
|
||||
@@ -14,12 +11,11 @@ try:
|
||||
moondream_loaded = True
|
||||
except Exception as e:
|
||||
moondream_loaded = False
|
||||
print(f"Moondream error: You should probably run install_extra.bat (windows) or install transformers==4.36.2 in the enviroment.\n Also torch must be >= 2.1.0 (ERROR: {e})")
|
||||
print(f"Moondream error: reinstall N-Suite dependencies with ComfyUI Manager or requirements.txt.\nTorch must be >= 2.1.0 (ERROR: {e})")
|
||||
from PIL import Image
|
||||
from transformers import CodeGenTokenizerFast as Tokenizer
|
||||
#,AutoTokenizer, AutoModelForCausalLM
|
||||
import numpy as np
|
||||
import base64
|
||||
|
||||
models_base_path = os.path.join(folder_paths.models_dir, "GPTcheckpoints")
|
||||
_choice = ["YES", "NO"]
|
||||
@@ -169,7 +165,7 @@ def run_moondream(images, prompt, max_tags, model_funct):
|
||||
list_descriptions.append(moondream.answer_question(image_embeds, prompt,tokenizer))
|
||||
except ValueError:
|
||||
print("\n\n\n")
|
||||
raise ModuleNotFoundError("Please run install_extra.bat in custom_nodes/ComfyUI-N-Nodes folder to make sure to have the required verision of Transformers installed (4.36.2).")
|
||||
raise ModuleNotFoundError("Moondream requires the dependency versions declared in N-Suite requirements.txt. Reinstall dependencies with ComfyUI Manager.")
|
||||
|
||||
|
||||
|
||||
@@ -253,33 +249,6 @@ def run_internlm(image, prompt, max_tags, model_funct):
|
||||
|
||||
|
||||
|
||||
def llava_inference(model_funct,prompt,images,max_tokens,stop_token,frequency_penalty,presence_penalty,repeat_penalty,temperature,top_k,top_p):
|
||||
list_descriptions = []
|
||||
for image in images:
|
||||
pil_image = tensor2pil(image)
|
||||
# Convert the PIL image to a bytes buffer
|
||||
buffer = BytesIO()
|
||||
pil_image.save(buffer, format="JPEG") # You can change the format if needed
|
||||
image_bytes = buffer.getvalue()
|
||||
base64_string = f"data:image/jpeg;base64,{base64.b64encode(image_bytes).decode('utf-8')}"
|
||||
|
||||
response = model_funct.create_chat_completion( max_tokens=max_tokens, stop=[stop_token], stream=False,frequency_penalty=frequency_penalty,presence_penalty=presence_penalty ,repeat_penalty=repeat_penalty,
|
||||
temperature=temperature,top_k=top_k,top_p=top_p,
|
||||
messages = [
|
||||
{"role": "system", "content": "You are an assistant who perfectly describes images."},
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "image_url", "image_url": {"url": base64_string}},
|
||||
{"type" : "text", "text": prompt}
|
||||
]
|
||||
}
|
||||
]
|
||||
)
|
||||
list_descriptions.append(response['choices'][0]['message']['content'])
|
||||
return list_descriptions
|
||||
|
||||
|
||||
if not os.path.isdir(models_base_path):
|
||||
os.mkdir(models_base_path)
|
||||
|
||||
@@ -294,16 +263,6 @@ if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","moo
|
||||
if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","internlm")):
|
||||
os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","internlm"))
|
||||
"""
|
||||
if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava")):
|
||||
os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava"))
|
||||
|
||||
if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava","models")):
|
||||
os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava","models"))
|
||||
|
||||
if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava","clips")):
|
||||
os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava","clips"))
|
||||
|
||||
|
||||
#folder_paths.folder_names_and_paths["GPTcheckpoints"] += (os.listdir(models_base_path),)
|
||||
|
||||
|
||||
@@ -321,35 +280,9 @@ MODEL_LOAD_FUNCTIONS = {
|
||||
|
||||
|
||||
|
||||
supported_gpt_extensions = set(['.gguf'])
|
||||
supported_clip_extensions = set(['.gguf','.bin'])
|
||||
model_external_path = None
|
||||
|
||||
all_models = []
|
||||
|
||||
try:
|
||||
model_external_path = folder_paths.folder_names_and_paths["GPTcheckpoints"][0][0]
|
||||
except:
|
||||
# no external folder
|
||||
pass
|
||||
|
||||
|
||||
|
||||
all_llava_models = get_model_list(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava","models"),supported_gpt_extensions)
|
||||
all_llava_clips = get_model_list(os.path.join(folder_paths.models_dir, "GPTcheckpoints","llava","clips"),supported_clip_extensions)
|
||||
|
||||
all_models = get_model_list(models_base_path,supported_gpt_extensions)
|
||||
if model_external_path is not None:
|
||||
all_models += get_model_list(model_external_path,supported_gpt_extensions)
|
||||
all_models += all_llava_models
|
||||
|
||||
|
||||
|
||||
#extract only names
|
||||
all_models = get_model_list(models_base_path, set())
|
||||
all_models_names = [os.path.basename(model) for model in all_models]
|
||||
|
||||
all_clips_names = [os.path.basename(model) for model in all_llava_clips]
|
||||
|
||||
|
||||
|
||||
class GPTLoaderSimple:
|
||||
@@ -360,38 +293,23 @@ class GPTLoaderSimple:
|
||||
"gpu_layers": ("INT", {"default": 27, "min": 0, "max": 100, "step": 1}),
|
||||
"n_threads": ("INT", {"default": 8, "min": 1, "max": 100, "step": 1}),
|
||||
"max_ctx": ("INT", {"default": 2048, "min": 300, "max": 100000, "step": 64}),
|
||||
},
|
||||
"optional": {
|
||||
"llava_clip": ("LLAVA_CLIP", ),
|
||||
|
||||
}}
|
||||
}}
|
||||
|
||||
|
||||
|
||||
RETURN_TYPES = ("CUSTOM", )
|
||||
RETURN_NAMES = ("model",)
|
||||
FUNCTION = "load_gpt_checkpoint"
|
||||
DESCRIPTION = "Loads a GPT checkpoint (GGUF format)<img src='https://compote.slate.com/images/697b023b-64a5-49a0-8059-27b963453fb1.gif?crop=780%2C520%2Cx0%2Cy0&width=1280' />"
|
||||
DESCRIPTION = "Loads a Moondream or JoyTag image-captioning model. GGUF and LLaVA support was removed in version 1.2.0."
|
||||
|
||||
CATEGORY = "N-Suite/loaders"
|
||||
|
||||
def load_gpt_checkpoint(self, ckpt_name, gpu_layers,n_threads,max_ctx,llava_clip=None):
|
||||
def load_gpt_checkpoint(self, ckpt_name, gpu_layers, n_threads, max_ctx):
|
||||
ckpt_path = get_model_path(all_models,ckpt_name)
|
||||
llm = None
|
||||
#if is path
|
||||
if os.path.isfile(ckpt_path):
|
||||
print("GPT MODEL DETECTED")
|
||||
if "llava" in ckpt_path:
|
||||
if llava_clip is None:
|
||||
raise ValueError("Please provide a llava clip")
|
||||
llm = Llama(model_path=ckpt_path,n_gpu_layers=gpu_layers,verbose=False,n_threads=n_threads, n_ctx=max_ctx, logits_all=True,chat_handler=llava_clip)
|
||||
else:
|
||||
llm = Llama(model_path=ckpt_path,n_gpu_layers=gpu_layers,verbose=False,n_threads=n_threads, n_ctx=max_ctx )
|
||||
else:
|
||||
if ckpt_name in MODEL_LOAD_FUNCTIONS :
|
||||
|
||||
cpu = False if gpu_layers > 0 else True
|
||||
llm = MODEL_LOAD_FUNCTIONS[ckpt_name](ckpt_path,cpu)
|
||||
if ckpt_name not in MODEL_LOAD_FUNCTIONS:
|
||||
raise ValueError(f"Unsupported model: {ckpt_name}")
|
||||
cpu = gpu_layers == 0
|
||||
llm = MODEL_LOAD_FUNCTIONS[ckpt_name](ckpt_path, cpu)
|
||||
|
||||
return ([llm, ckpt_name, ckpt_path],)
|
||||
|
||||
@@ -465,21 +383,10 @@ class GPTSampler:
|
||||
|
||||
|
||||
if cached == "NO":
|
||||
if model_name in MODEL_FUNCTIONS and os.path.isdir(model_path):
|
||||
if model_name in MODEL_FUNCTIONS and os.path.isdir(model_path):
|
||||
cont = MODEL_FUNCTIONS[model_name](image, prompt, max_tags, model_funct)
|
||||
|
||||
else:
|
||||
if "llava" in model_path:
|
||||
cont = llava_inference(model_funct,prompt,image,max_tokens,stop_token,frequency_penalty,presence_penalty,repeat_penalty,temperature,top_k,top_p)
|
||||
|
||||
|
||||
else:
|
||||
# Call your GPT generation function here using the provided parameters
|
||||
composed_prompt = f"{prefix} {prompt} {suffix}"
|
||||
cont =""
|
||||
stream = model_funct( max_tokens=max_tokens, stop=[stop_token], stream=False,frequency_penalty=frequency_penalty,presence_penalty=presence_penalty ,repeat_penalty=repeat_penalty,temperature=temperature,top_k=top_k,top_p=top_p,model=model_path,prompt=composed_prompt)
|
||||
cont= [stream["choices"][0]["text"]]
|
||||
self.temp_prompt = cont
|
||||
raise ValueError(f"Unsupported model: {model_name}")
|
||||
else:
|
||||
cont = self.temp_prompt
|
||||
#remove fist 30 characters of cont
|
||||
@@ -494,33 +401,13 @@ class GPTSampler:
|
||||
return {"ui": {"text": " "}, "result": (" ",)}
|
||||
|
||||
|
||||
class LlavaClipLoader:
|
||||
@classmethod
|
||||
def INPUT_TYPES(s):
|
||||
return {"required": {
|
||||
"clip_name": (all_clips_names, ),
|
||||
}}
|
||||
|
||||
RETURN_TYPES = ("LLAVA_CLIP", )
|
||||
RETURN_NAMES = ("llava_clip", )
|
||||
FUNCTION = "load_clip_checkpoint"
|
||||
|
||||
CATEGORY = "N-Suite/LLava"
|
||||
def load_clip_checkpoint(self, clip_name):
|
||||
clip_path = get_model_path(all_llava_clips,clip_name)
|
||||
clip = Llava15ChatHandler(clip_model_path = clip_path, verbose=False)
|
||||
return (clip, )
|
||||
|
||||
|
||||
NODE_CLASS_MAPPINGS = {
|
||||
"GPT Loader Simple [n-suite]": GPTLoaderSimple,
|
||||
"GPT Sampler [n-suite]": GPTSampler,
|
||||
"Llava Clip Loader [n-suite]": LlavaClipLoader
|
||||
"GPT Sampler [n-suite]": GPTSampler
|
||||
}
|
||||
# A dictionary that contains the friendly/humanly readable titles for the nodes
|
||||
NODE_DISPLAY_NAME_MAPPINGS = {
|
||||
"GPT Loader Simple [n-suite]": "GPT Loader Simple [🅝-🅢🅤🅘🅣🅔]",
|
||||
"GPT Sampler [n-suite]": "GPT Text Sampler [🅝-🅢🅤🅘🅣🅔]",
|
||||
"Llava Clip Loader [n-suite]": "Llava Clip Loader [🅝-🅢🅤🅘🅣🅔]"
|
||||
"GPT Sampler [n-suite]": "Image Caption Sampler [🅝-🅢🅤🅘🅣🅔]"
|
||||
|
||||
}
|
||||
}
|
||||
+11
-3
@@ -1,9 +1,17 @@
|
||||
[project]
|
||||
name = "comfyui-n-nodes"
|
||||
description = "A suite of custom nodes for ConfyUI that includes GPT text-prompt generation, LoadVideo,SaveVideo,LoadFramesFromFolder and FrameInterpolator"
|
||||
version = "1.1.1"
|
||||
description = "A suite of custom nodes for ComfyUI that includes image captioning, LoadVideo, SaveVideo, LoadFramesFromFolder and FrameInterpolator"
|
||||
version = "1.2.0"
|
||||
license = { file = "LICENSE" }
|
||||
dependencies = ["py-cpuinfo", "gitpython", "moviepy", "opencv-python", "scikit-build", "typing", "diskcache"]
|
||||
dependencies = [
|
||||
"gitpython",
|
||||
"huggingface-hub",
|
||||
"moviepy<2",
|
||||
"opencv-python",
|
||||
"scikit-build",
|
||||
"timm==0.9.12",
|
||||
"transformers==4.36.2",
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
Repository = "https://github.com/Nuked88/ComfyUI-N-Nodes"
|
||||
|
||||
@@ -0,0 +1,3 @@
|
||||
[pytest]
|
||||
testpaths = tests
|
||||
addopts = --import-mode=importlib
|
||||
@@ -1,15 +0,0 @@
|
||||
@echo off
|
||||
set "python_exec=..\..\..\python_embeded\python.exe"
|
||||
|
||||
echo Restore original ComfyUI dependency...
|
||||
if exist "%python_exec%" (
|
||||
echo Restore with ComfyUI Portable
|
||||
"%python_exec%" -s -m pip install transformers==4.26.1
|
||||
echo Done. Please reboot ComfyUI.
|
||||
) else (
|
||||
echo Restore with system Python
|
||||
pip install transformers==4.26.1
|
||||
echo Done. Please reboot ComfyUI.
|
||||
)
|
||||
|
||||
pause
|
||||
+4
-4
@@ -1,7 +1,7 @@
|
||||
py-cpuinfo
|
||||
gitpython
|
||||
moviepy
|
||||
huggingface-hub
|
||||
moviepy<2
|
||||
opencv-python
|
||||
scikit-build
|
||||
typing
|
||||
diskcache
|
||||
timm==0.9.12
|
||||
transformers==4.36.2
|
||||
|
||||
@@ -1,171 +0,0 @@
|
||||
import asyncio
|
||||
import os
|
||||
import json
|
||||
import shutil
|
||||
import inspect
|
||||
|
||||
import requests
|
||||
import subprocess
|
||||
import platform
|
||||
import importlib.util
|
||||
|
||||
import sys
|
||||
|
||||
config = None
|
||||
|
||||
|
||||
|
||||
|
||||
def check_nvidia_gpu():
|
||||
try:
|
||||
# Utilizza torch per verificare la presenza di una GPU NVIDIA
|
||||
return torch.cuda.is_available() and 'NVIDIA' in torch.cuda.get_device_name(0)
|
||||
except Exception as e:
|
||||
print(f"Error while checking for NVIDIA GPU: {e}")
|
||||
return False
|
||||
|
||||
def get_cuda_version():
|
||||
try:
|
||||
if torch.cuda.is_available():
|
||||
cuda_version = torch.version.cuda.replace(".","").strip()
|
||||
|
||||
return "cu"+cuda_version
|
||||
else:
|
||||
return "No NVIDIA GPU available"
|
||||
except Exception as e:
|
||||
print(f"Error while checking CUDA version: {e}")
|
||||
return "Unable to determine CUDA version"
|
||||
|
||||
def check_avx2_support():
|
||||
import cpuinfo
|
||||
try:
|
||||
info = cpuinfo.get_cpu_info()
|
||||
return 'avx2' in info['flags']
|
||||
except Exception as e:
|
||||
print(f"Error while checking AVX2 support: {e}")
|
||||
return False
|
||||
|
||||
def get_python_version():
|
||||
if "3.9" in platform.python_version():
|
||||
return "39"
|
||||
elif "3.10" in platform.python_version():
|
||||
return "310"
|
||||
elif "3.11" in platform.python_version():
|
||||
return "311"
|
||||
else:
|
||||
return None
|
||||
|
||||
|
||||
def get_os():
|
||||
return platform.system()
|
||||
|
||||
def get_os_bit():
|
||||
return platform.architecture()[0].replace("bit","")
|
||||
import requests
|
||||
|
||||
def get_platform_tag(_os):
|
||||
#return the first tag in the list of tags
|
||||
try:
|
||||
import packaging.tags
|
||||
response = requests.get("https://api.github.com/repos/abetlen/llama-cpp-python/releases/latest")
|
||||
jresponse= response.json()
|
||||
|
||||
return extract_platform_tag(jresponse,list(packaging.tags.sys_tags())[0],_os)
|
||||
|
||||
except:
|
||||
return None
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
def install_package(package):
|
||||
subprocess.check_call([sys.executable, "-m", "pip", "install", "--no-cache-dir", package])
|
||||
|
||||
def install_llama():
|
||||
try:
|
||||
gpu = check_nvidia_gpu()
|
||||
avx2 = check_avx2_support()
|
||||
lcpVersion = get_last_llcpppy_version()
|
||||
python_version = get_python_version()
|
||||
_os = get_os()
|
||||
os_bit = get_os_bit()
|
||||
platform_tag = get_platform_tag(_os)
|
||||
print(f"Python version: {python_version}")
|
||||
print(f"OS: {_os}")
|
||||
print(f"OS bit: {os_bit}")
|
||||
print(f"Platform tag: {platform_tag}")
|
||||
if python_version == None:
|
||||
print("Unsupported Python version. Please use Python 3.9, 3.10 or 3.11.")
|
||||
return
|
||||
|
||||
|
||||
#python -m pip install llama-cpp-python --force-reinstall --no-deps --index-url=https://jllllll.github.io/llama-cpp-python-cuBLAS-wheels/AVX2/cu117
|
||||
if avx2:
|
||||
avx="AVX2"
|
||||
else:
|
||||
avx="AVX"
|
||||
|
||||
if gpu:
|
||||
cuda = get_cuda_version()
|
||||
print(f"--index-url=https://jllllll.github.io/llama-cpp-python-cuBLAS-wheels/{avx}/{cuda}")
|
||||
else:
|
||||
print(f"https://github.com/abetlen/llama-cpp-python/releases/download/v{lcpVersion}/llama_cpp_python-{lcpVersion}-{platform_tag}.whl")
|
||||
except Exception as e:
|
||||
print(f"Error while installing LLAMA: {e}")
|
||||
# llama wheels https://github.com/jllllll/llama-cpp-python-cuBLAS-wheels
|
||||
|
||||
def check_module(package):
|
||||
import importlib
|
||||
try:
|
||||
print("Detected: ", package)
|
||||
importlib.import_module(package)
|
||||
return True
|
||||
except ImportError:
|
||||
return False
|
||||
import zipfile
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
import re
|
||||
def extract_platform_tag(jresponse,tag,_os):
|
||||
if _os.lower()== "linux" or _os.lower()=="macosx":
|
||||
print(jresponse)
|
||||
for res in jresponse["assets"]:
|
||||
url = res["browser_download_url"]
|
||||
print(url)
|
||||
pattern = r'.*-((cp\d+-cp\d+)-(manylinux|macosx)_(\d+_\d+)_(x86_64|i686)\.whl)$'
|
||||
match = re.match(pattern, url)
|
||||
pattern_tag = r'((cp\d+-cp\d+)-(manylinux|macosx)_\d+_\d+_(x86_64|i686))$'
|
||||
match_tag = re.match(pattern_tag, tag)
|
||||
if match_tag:
|
||||
rl_platform_tag = f"{match_tag.group(2)}-{match_tag.group(3)}_**_{match_tag.group(4)}"
|
||||
#print(rl_platform_tag)
|
||||
if match:
|
||||
# Estrai il platform tag dal match
|
||||
url_platform_tag = f"{match.group(2)}-{match.group(3)}_**_{match.group(5)}"
|
||||
final_platform_tag =f"{match.group(2)}-{match.group(3)}_{match.group(4)}_{match.group(5)}"
|
||||
if rl_platform_tag == url_platform_tag:
|
||||
return final_platform_tag
|
||||
|
||||
return None
|
||||
else:
|
||||
return tag
|
||||
|
||||
def get_last_llcpppy_version():
|
||||
try:
|
||||
import requests
|
||||
|
||||
response = requests.get("https://api.github.com/repos/abetlen/llama-cpp-python/releases/latest")
|
||||
|
||||
|
||||
return response.json()["tag_name"].replace("v","")
|
||||
except:
|
||||
return "0.2.20"
|
||||
|
||||
|
||||
install_llama()
|
||||
@@ -0,0 +1,30 @@
|
||||
import importlib.util
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
MODULE_PATH = Path(__file__).resolve().parents[1] / "py" / "dynamic_prompt_node.py"
|
||||
SPEC = importlib.util.spec_from_file_location("dynamic_prompt_node", MODULE_PATH)
|
||||
MODULE = importlib.util.module_from_spec(SPEC)
|
||||
SPEC.loader.exec_module(MODULE)
|
||||
DynamicPrompt = MODULE.DynamicPrompt
|
||||
|
||||
|
||||
def generate(variable, mode="Fixed", count=1, fixed=""):
|
||||
return DynamicPrompt().prompt_generator(variable, "NO", mode, count, fixed)["result"][0]
|
||||
|
||||
|
||||
def test_fixed_prompt_is_combined_with_requested_number_of_unique_tags():
|
||||
result = generate("red, green, blue", count=2, fixed="portrait")
|
||||
parts = result.split(",")
|
||||
assert parts[0] == "portrait"
|
||||
assert len(parts[1:]) == 2
|
||||
assert len(set(parts[1:])) == 2
|
||||
assert set(parts[1:]) <= {"red", "green", "blue"}
|
||||
|
||||
|
||||
def test_requested_tag_count_is_capped_to_available_tags():
|
||||
assert set(generate("red,blue", count=20).split(",")) == {"red", "blue"}
|
||||
|
||||
|
||||
def test_empty_variable_prompt_returns_empty_result():
|
||||
assert generate("", fixed="portrait") == ""
|
||||
@@ -0,0 +1,82 @@
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
RUNTIME_FILES = [
|
||||
ROOT / "__init__.py",
|
||||
ROOT / "nnodes.py",
|
||||
ROOT / "py" / "image_captioning_node.py",
|
||||
ROOT / "requirements.txt",
|
||||
ROOT / "pyproject.toml",
|
||||
]
|
||||
|
||||
|
||||
def test_runtime_has_no_llama_cpp_dependency():
|
||||
for path in RUNTIME_FILES:
|
||||
assert "llama_cpp" not in path.read_text(), path
|
||||
|
||||
|
||||
def test_llava_node_is_no_longer_registered():
|
||||
source = (ROOT / "py" / "image_captioning_node.py").read_text()
|
||||
assert '"Llava Clip Loader [n-suite]"' not in source
|
||||
assert "Llava15ChatHandler" not in source
|
||||
|
||||
|
||||
def test_readme_announces_breaking_change_and_model_downloads():
|
||||
readme = (ROOT / "README.md").read_text()
|
||||
assert "Breaking change in 1.2.0" in readme
|
||||
assert "was never downloaded automatically" in readme
|
||||
assert "first time Moondream is loaded" in readme
|
||||
assert "first time JoyTag is loaded" in readme
|
||||
assert "git checkout ae7cc84" in readme
|
||||
assert "old LLaVA implementation was not independent" in readme
|
||||
|
||||
|
||||
def test_dependencies_are_manager_installable_and_moviepy_is_compatible():
|
||||
requirements = (ROOT / "requirements.txt").read_text().splitlines()
|
||||
pyproject = (ROOT / "pyproject.toml").read_text()
|
||||
assert "moviepy<2" in requirements
|
||||
assert '"moviepy<2"' in pyproject
|
||||
assert "huggingface-hub" in requirements
|
||||
assert "transformers==4.36.2" in requirements
|
||||
assert "timm==0.9.12" in requirements
|
||||
|
||||
|
||||
def test_extension_does_not_install_packages_during_import():
|
||||
bootstrap = (ROOT / "__init__.py").read_text()
|
||||
assert "check_and_install" not in bootstrap
|
||||
|
||||
|
||||
def test_frontend_uses_managed_dom_widgets():
|
||||
widgets = (ROOT / "js" / "extended_widgets.js").read_text()
|
||||
assert "addDOMWidget" in widgets
|
||||
assert "addCustomWidget" not in widgets
|
||||
assert "onDrawBackground" not in widgets
|
||||
assert "graph._nodes" not in widgets
|
||||
|
||||
|
||||
def test_dynamic_widgets_use_current_removal_api_and_node_id():
|
||||
dynamic_prompt = (ROOT / "js" / "dynamicPrompt.js").read_text()
|
||||
gpt_sampler = (ROOT / "js" / "gptSampler.js").read_text()
|
||||
assert 'nodeData.name !== "DynamicPrompt [n-suite]"' in dynamic_prompt
|
||||
assert "removeWidget(widget)" in dynamic_prompt
|
||||
assert "removeWidget(widget)" in gpt_sampler
|
||||
|
||||
|
||||
def test_external_repositories_and_model_archive_are_pinned():
|
||||
bootstrap = (ROOT / "__init__.py").read_text()
|
||||
assert 'RIFE_REVISION = "a8a8035323b1c1a4a20753c751780e5b0a879455"' in bootstrap
|
||||
assert 'MOONDREAM_REVISION = "38af98596e59f2a6c25c6b52b2bd5a672dab4144"' in bootstrap
|
||||
assert 'RIFE_MODEL_REVISION = "572480112b87f9bfbff7579b8a38b483766e455f"' in bootstrap
|
||||
assert "/raw/main/RIFE_trained_model" not in bootstrap
|
||||
assert "repo.git.checkout(revision)" in bootstrap
|
||||
|
||||
|
||||
def test_registry_publish_is_manual_after_nightly_validation():
|
||||
workflow = (ROOT / ".github" / "workflows" / "publish.yml").read_text()
|
||||
assert "workflow_dispatch:" in workflow
|
||||
assert "push:" not in workflow
|
||||
|
||||
readme = (ROOT / "README.md").read_text()
|
||||
assert "Testing this release through Manager Nightly" in readme
|
||||
assert "a pull request cannot be selected as nightly while it is still unmerged" in readme
|
||||
Reference in New Issue
Block a user