Add all-node N-Suite test workflow

Corrected formatting for example headings in README.

.github/workflows/publish.yml

@@ -6,15 +6,19 @@ on:
 # version. Stable Registry releases are intentionally published only by
 # manually running this workflow after nightly validation.
 
+permissions:
+  issues: write
+
 jobs:
   publish-node:
     name: Publish Custom Node to registry
     runs-on: ubuntu-latest
+    if: ${{ github.repository_owner == 'Nuked88' }}
     steps:
       - name: Check out code
         uses: actions/checkout@v4
       - name: Publish Custom Node
-        uses: Comfy-Org/publish-node-action@main
+        uses: Comfy-Org/publish-node-action@v1
         with:
           ## Add your own personal access token to your Github Repository secrets and reference it here.
           personal_access_token: ${{ secrets.REGISTRY_ACCESS_TOKEN }}

README.md

@@ -3,19 +3,14 @@
 # ComfyUI-N-Suite
 A suite of custom nodes for ComfyUI that includes integer, string and float variable nodes, image-captioning nodes and video nodes.
 
-> [!IMPORTANT]  
-> These nodes were tested primarily in Windows in the default environment provided by ComfyUI and in the environment created by the [notebook](https://github.com/comfyanonymous/ComfyUI/blob/master/notebooks/comfyui_colab.ipynb) for paperspace specifically with the cyberes/gradient-base-py3.10:latest docker image.
-**Any other environment has not been tested.**
+The nodes support ComfyUI's Python environment on Windows and Linux. The current dependencies include MoviePy 2, timm 1.0.22 or newer, accelerate 1.x, and transformers 4.36.2 through 4.x.
 
 
 # Installation
 
-1. Clone the repository:
-`git clone https://github.com/Nuked88/ComfyUI-N-Nodes.git`  
-to your ComfyUI `custom_nodes` directory
-
-2. Install it through **ComfyUI Manager** (recommended), which installs the dependencies declared by the project, or run `python -m pip install -r requirements.txt` in ComfyUI's Python environment after a manual clone.
-3. Restart ComfyUI.
+1. Install **ComfyUI-N-Nodes** through ComfyUI Manager (recommended). For a manual install, clone `https://github.com/Nuked88/ComfyUI-N-Nodes.git` into ComfyUI's `custom_nodes` directory and run `python -m pip install -r requirements.txt` using the same Python environment that runs ComfyUI.
+2. Ensure `ComfyUI/models/GPTcheckpoints` is writable by the ComfyUI process so Moondream and JoyTag can download their models.
+3. Restart ComfyUI. The extension clones pinned RIFE code and downloads its pinned model at startup on a fresh install; an internet connection is needed for that first startup.
 
 ComfyUI automatically loads all custom scripts and nodes at startup.
 
@@ -34,19 +29,18 @@ ComfyUI automatically loads all custom scripts and nodes at startup.
 > For install the last version of this repository before this changes from the Comfyui-N-Suite execute **git checkout 29b2e43baba81ee556b2930b0ca0a9c978c47083**
 
 
-- For uninstallation:
-  - Delete the `ComfyUI-N-Nodes` folder in `custom_nodes`
-  - Delete the `comfyui-n-nodes` folder in  `ComfyUI\web\extensions`
-  - Delete the `n-styles.csv` and `n-styles.csv.backup` file in `ComfyUI\styles`
-  - Delete the `GPTcheckpoints` folder in `ComfyUI\models`
+For uninstallation, remove the extension through ComfyUI Manager or delete its folder from `custom_nodes`, then restart ComfyUI. Model files in `models/GPTcheckpoints` are user data and can be kept for a later reinstall.
 
 
 
 
 
 # Update
-1. Navigate to the cloned repo e.g. `custom_nodes/ComfyUI-N-Nodes`
-2. `git pull`
+Update through ComfyUI Manager. For a manual install, run `git pull` in the cloned extension directory, install `requirements.txt` again in ComfyUI's Python environment, and restart ComfyUI.
+
+## Test workflow
+
+[`examples/N-Suite-all-nodes-test.json`](examples/N-Suite-all-nodes-test.json) connects all 14 N-Suite node types in one workflow. Follow the [test instructions](examples/README.md) to add an image, a short MP4, and numbered PNG frames before running it.
 
 # Features
 
@@ -167,13 +161,26 @@ The node-variables are:
 
 ## 🤖 Image captioning: GPTLoaderSimple and GPTSampler 🤖
 
-The legacy node identifiers are retained so existing Moondream and JoyTag workflows continue to load, but these nodes are now limited to image captioning:
+#### Moondream
+The model will be automatically downloaded when you run the first time.Only the required code, tokenizer, and single `model.safetensors` file are downloaded from a pinned revision of `vikhyatk/moondream1` on Hugging Face. The model file is about **3.72 GB**; the repository also contains larger alternative weights that are not downloaded.
+Anyway, it is available [HERE](https://huggingface.co/vikhyatk/moondream1/tree/main)
+The code taken from [this repository](https://github.com/vikhyat/moondream)
+
+#### Example with Moondream model:
+![alt text](./img/image-15.png)
+
+#### Joytag
+The model will be automatically downloaded when you run the first time.Only the required configuration, tags, and `model.safetensors` are downloaded from a pinned revision of `fancyfeast/joytag`. The model file is about **0.37 GB**; the unused ONNX file is not downloaded.
+Anyway, it is available [HERE](https://huggingface.co/fancyfeast/joytag/tree/main)
+The code taken from [this repository](https://github.com/fpgaminer/joytag)
+
+#### Example with Joytag model:
+![alt text](./img/image-16.png)
 
-- **Moondream:** its `config.json`, `model.safetensors`, and `tokenizer.json` files are downloaded automatically from the `vikhyatk/moondream1` Hugging Face repository the first time Moondream is loaded.
-- **JoyTag:** its model snapshot is downloaded automatically from the `fancyfeast/joytag` Hugging Face repository the first time JoyTag is loaded.
-- **LLaVA:** was never downloaded automatically. Its GGUF model and projector had to be installed manually; support has now been removed together with `llama-cpp-python`.
 
 Downloads happen on first model use, not merely when ComfyUI starts. An internet connection and sufficient disk space are required for that initial load. Models are stored under `ComfyUI/models/GPTcheckpoints/moondream` and `ComfyUI/models/GPTcheckpoints/joytag`.
+GPT Loader Simple displays a download notice when the selected model file is missing. The ComfyUI console shows the download progress.
+Moondream1 uses legacy Phi model code; N-Suite applies a small compatibility patch to its downloaded `modeling_phi.py` file and adapts the text model for image embedding generation with newer Transformers releases. The model weights are not changed.
 
 ### GPTLoaderSimple
 

__init__.py

@@ -10,8 +10,6 @@ WEB_DIRECTORY = "./js"
 
 RIFE_REPOSITORY = "https://github.com/hzwer/Practical-RIFE.git"
 RIFE_REVISION = "a8a8035323b1c1a4a20753c751780e5b0a879455"
-MOONDREAM_REPOSITORY = "https://github.com/Nuked88/moondream.git"
-MOONDREAM_REVISION = "38af98596e59f2a6c25c6b52b2bd5a672dab4144"
 RIFE_MODEL_REVISION = "572480112b87f9bfbff7579b8a38b483766e455f"
 
 
@@ -43,11 +41,6 @@ if __package__:
         rife_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), "libs", "rifle")
         clone_at_revision(Repo, RIFE_REPOSITORY, rife_path, RIFE_REVISION)
 
-        moondream_path = os.path.join(
-            os.path.dirname(os.path.realpath(__file__)), "libs", "moondream_repo"
-        )
-        clone_at_revision(Repo, MOONDREAM_REPOSITORY, moondream_path, MOONDREAM_REVISION)
-
         if not os.path.exists(os.path.join(rife_path, "train_log")):
             downloader(
                 f"https://raw.githubusercontent.com/Nuked88/DreamingAI/{RIFE_MODEL_REVISION}/RIFE_trained_model_v4.7.zip"

examples/README.md

@@ -0,0 +1,14 @@
+# N-Suite: test di tutti i nodi
+
+Apri `N-Suite-all-nodes-test.json` in ComfyUI. Il workflow contiene tutti i 14 tipi di nodo N-Suite presenti in questa versione, con anteprime dei risultati e tre prove di salvataggio video.
+
+Prima di premere **Queue Prompt**:
+
+1. Scegli una tua immagine nel nodo **LoadImage** della sezione 01. Il loader GPT usa Moondream, già selezionato nel workflow.
+2. Copia un MP4 breve nella cartella `ComfyUI/input/n-suite`, ricarica la pagina e selezionalo nel nodo **LoadVideo** della sezione 04. Un video di pochi secondi riduce il tempo necessario per RIFE.
+3. Metti almeno due immagini PNG della stessa dimensione, con nomi numerati come `0001.png` e `0002.png`, nella cartella `ComfyUI/input/n-suite/test_frames`. Il nodo **String Variable** della sezione 05 contiene il percorso visto dal container: `/workspace/ComfyUI/input/n-suite/test_frames`.
+4. Premi **Queue Prompt**. Il nodo CLIP usa `clip_l.safetensors`, già disponibile nell'installazione per cui è stato creato il workflow.
+
+La risposta Moondream e i condizionamenti CLIP compaiono nei nodi **Preview as Text**. Le immagini e la maschera compaiono nelle anteprime. I video vengono scritti in `ComfyUI/output/n-suite/videos` con prefissi `n_suite_test_*`. Se un ramo fallisce, ComfyUI evidenzia il nodo che ha generato l'errore.
+
+Il file è generato da `generate_test_workflow.py` usando gli schemi `/object_info` di ComfyUI. Su un'installazione diversa, rigeneralo con `python examples/generate_test_workflow.py http://127.0.0.1:8188` dalla cartella del repository.

examples/generate_test_workflow.py

@@ -0,0 +1,144 @@
+"""Generate the all-node smoke test from a running ComfyUI instance.
+
+Usage: python examples/generate_test_workflow.py http://127.0.0.1:8188
+"""
+
+import json
+import sys
+import uuid
+from pathlib import Path
+from urllib.request import urlopen
+
+
+url = sys.argv[1].rstrip("/") if len(sys.argv) > 1 else "http://127.0.0.1:8188"
+schema = json.load(urlopen(f"{url}/object_info"))
+nodes = []
+links = []
+
+
+def add(kind, pos, values=None, title=None, size=None):
+    info = schema[kind]
+    values = values or {}
+    inputs, widgets = [], []
+    for name, spec in {**info["input"].get("required", {}), **info["input"].get("optional", {})}.items():
+        raw_type = spec[0]
+        input_type = "COMBO" if isinstance(raw_type, list) else raw_type
+        options = spec[1] if len(spec) > 1 and isinstance(spec[1], dict) else {}
+        entry = {"name": name, "type": input_type, "link": None}
+        if input_type in ("COMBO", "STRING", "INT", "FLOAT", "BOOLEAN") and not options.get("forceInput"):
+            entry["widget"] = {"name": name}
+            default = options.get("default", raw_type[0] if isinstance(raw_type, list) and raw_type else "")
+            widgets.append(values.get(name, default))
+        inputs.append(entry)
+    if kind == "LoadImage":
+        inputs.append({"name": "upload", "type": "IMAGEUPLOAD", "widget": {"name": "upload"}, "link": None})
+        widgets.append("image")
+    node_id = len(nodes) + 1
+    node = {
+        "id": node_id, "type": kind, "pos": pos, "size": size or [350, 180],
+        "flags": {}, "order": node_id - 1, "mode": 0, "inputs": inputs,
+        "outputs": [{"name": name, "type": typ, "links": []} for name, typ in
+                    zip(info.get("output_name", info["output"]), info["output"])],
+        "properties": {"Node name for S&R": kind}, "widgets_values": widgets,
+    }
+    if title:
+        node["title"] = title
+    nodes.append(node)
+    return node_id
+
+
+def connect(source, slot, target, input_name):
+    origin = nodes[source - 1]
+    dest = nodes[target - 1]
+    dest_slot = next(i for i, item in enumerate(dest["inputs"]) if item["name"] == input_name)
+    assert dest["inputs"][dest_slot]["link"] is None
+    link_id = len(links) + 1
+    links.append([link_id, source, slot, target, dest_slot, origin["outputs"][slot]["type"]])
+    origin["outputs"][slot]["links"].append(link_id)
+    dest["inputs"][dest_slot]["link"] = link_id
+
+
+image = add("LoadImage", [80, 100], {"image": "example.png"}, "Scegli la tua immagine", [380, 330])
+questions = add("String Variable [n-suite]", [80, 500],
+                {"string": "What is in this image?,What colors are in this image?"}, "Domande di prova")
+dynamic = add("DynamicPrompt [n-suite]", [520, 490],
+              {"cached": "NO", "number_of_random_tag": "Fixed", "fixed_number_of_random_tag": 1})
+caption_model = add("GPT Loader Simple [n-suite]", [520, 100], {"ckpt_name": "moondream"})
+caption = add("GPT Sampler [n-suite]", [930, 100],
+              {"max_tokens": 128, "cached": "NO", "print_output": "enable"}, size=[390, 700])
+caption_preview = add("PreviewAny", [1400, 150], title="Risposta Moondream")
+noise = add("Float Variable [n-suite]", [80, 1040], {"value": 0.1})
+pad = add("ImagePadForOutpaintAdvanced [n-suite]", [500, 930],
+          {"left": 32, "right": 32, "top": 32, "bottom": 32}, size=[430, 590])
+padded_preview = add("PreviewImage", [1030, 970], title="Immagine con bordo", size=[350, 300])
+mask_to_image = add("MaskToImage", [1030, 1330])
+mask_preview = add("PreviewImage", [1410, 1320], title="Maschera del bordo", size=[350, 300])
+clip = add("CLIPLoader", [2030, 100], {"clip_name": "clip_l.safetensors", "type": "stable_diffusion"})
+encode = add("CLIPTextEncodeAdvancedNSuite [n-suite]", [2460, 100],
+             {"styles": "NAI", "positive_prompt": "a small test image", "negative_prompt": "blurry"}, size=[400, 350])
+positive_preview = add("PreviewAny", [2940, 100], title="Condizionamento positivo")
+negative_preview = add("PreviewAny", [2940, 400], title="Condizionamento negativo")
+multiplier = add("Integer Variable [n-suite]", [80, 2060], {"value": 2})
+video = add("LoadVideo [n-suite]", [430, 1950],
+            {"video": "SELECT_VIDEO.mp4", "framerate": "original", "resize_by": "none",
+             "images_limit": 0, "batch_size": 0, "starting_frame": 0, "autoplay": False, "use_ram": False},
+            size=[420, 570])
+interpolator = add("FrameInterpolator [n-suite]", [940, 1990])
+interpolated_video = add("SaveVideo [n-suite]", [1430, 1970],
+                         {"SaveVideo": True, "SaveFrames": False, "filename_prefix": "n_suite_test_interpolated"})
+video_info = add("PreviewAny", [940, 2290], title="Metadati video originale")
+folder = add("String Variable [n-suite]", [80, 2990],
+             {"string": "/workspace/ComfyUI/input/n-suite/test_frames"}, "Cartella frame: cambia qui", [500, 130])
+image_folder = add("LoadImageFromFolder [n-suite]", [660, 2860])
+image_folder_preview = add("PreviewImage", [1100, 2840], title="Immagini dalla cartella", size=[330, 260])
+manual_metadata = add("SetMetadataForSaveVideo [n-suite]", [1100, 3210],
+                      {"fps": 24, "VideoName": "n_suite_folder"})
+manual_video = add("SaveVideo [n-suite]", [1550, 2890],
+                   {"SaveVideo": True, "SaveFrames": False, "filename_prefix": "n_suite_test_manual_metadata"})
+frame_folder = add("LoadFramesFromFolder [n-suite]", [660, 3530], {"fps": 24})
+frame_folder_preview = add("PreviewImage", [1100, 3520], title="Frame numerati", size=[330, 260])
+frame_video = add("SaveVideo [n-suite]", [1550, 3510],
+                  {"SaveVideo": True, "SaveFrames": False, "filename_prefix": "n_suite_test_folder_frames"})
+
+for args in [
+    (questions, 0, dynamic, "variable_prompt"), (dynamic, 0, caption, "prompt"),
+    (caption_model, 0, caption, "model"), (image, 0, caption, "image"),
+    (caption, 0, caption_preview, "source"), (image, 0, pad, "image"),
+    (noise, 0, pad, "noise"), (pad, 0, padded_preview, "images"),
+    (pad, 1, mask_to_image, "mask"), (mask_to_image, 0, mask_preview, "images"),
+    (clip, 0, encode, "clip"), (encode, 0, positive_preview, "source"),
+    (encode, 1, negative_preview, "source"), (video, 0, interpolator, "images"),
+    (video, 2, interpolator, "METADATA"), (multiplier, 0, interpolator, "multiplier"),
+    (interpolator, 0, interpolated_video, "images"),
+    (interpolator, 1, interpolated_video, "METADATA"), (video, 2, video_info, "source"),
+    (folder, 0, image_folder, "folder"), (folder, 0, frame_folder, "folder"),
+    (image_folder, 0, image_folder_preview, "images"),
+    (image_folder, 0, manual_video, "images"),
+    (image_folder, 3, manual_metadata, "number_of_frames"),
+    (manual_metadata, 0, manual_video, "METADATA"),
+    (frame_folder, 0, frame_folder_preview, "images"),
+    (frame_folder, 0, frame_video, "images"),
+    (frame_folder, 1, frame_video, "METADATA"),
+]:
+    connect(*args)
+
+groups = [
+    ("01 FOTO + MOONDREAM: scegli la foto in LoadImage", [40, 40, 1750, 790], "#3f789e"),
+    ("02 IMAGE PAD: controlla immagine e maschera", [40, 870, 1760, 790], "#637c49"),
+    ("03 CLIP: usa il modello clip_l presente", [1980, 40, 1400, 680], "#76578e"),
+    ("04 VIDEO: copia un MP4 in input/n-suite, ricarica e selezionalo", [40, 1880, 1790, 690], "#896a3c"),
+    ("05 CARTELLA: aggiungi 0001.png e 0002.png in test_frames", [40, 2780, 1920, 1050], "#3f789e"),
+]
+used = {node["type"] for node in nodes if "[n-suite]" in node["type"].lower()}
+expected = {name for name in schema if "[n-suite]" in name.lower()}
+assert used == expected, f"Missing N-Suite nodes: {sorted(expected - used)}"
+workflow = {
+    "id": str(uuid.uuid4()), "revision": 0, "last_node_id": len(nodes), "last_link_id": len(links),
+    "nodes": nodes, "links": links,
+    "groups": [{"id": i, "title": title, "bounding": bounds, "color": color, "flags": {}}
+               for i, (title, bounds, color) in enumerate(groups, 1)],
+    "config": {}, "extra": {"ds": {"scale": 0.55, "offset": [70, 70]}}, "version": 0.4,
+}
+destination = Path(__file__).with_name("N-Suite-all-nodes-test.json")
+destination.write_text(json.dumps(workflow, ensure_ascii=False, indent=2) + "\n")
+print(f"Saved {destination}: {len(nodes)} nodes, {len(links)} links, {len(used)} N-Suite types")

js/modelDownload.js

@@ -0,0 +1,17 @@
+import { app } from "/scripts/app.js";
+import { api } from "/scripts/api.js";
+
+app.registerExtension({
+    name: "n.ModelDownloadNotice",
+    setup() {
+        api.addEventListener("n-suite-model-download", (event) => {
+            const { model, message } = event.detail;
+            app.extensionManager.toast.add({
+                severity: "info",
+                summary: `${model} download started`,
+                detail: message,
+                life: 15000,
+            });
+        });
+    },
+});

nnodes.py

@@ -61,7 +61,7 @@ class color:
 def get_commit():
     try:
         import git
-        repo = git.Repo( os.path.join(folder_paths.folder_names_and_paths["custom_nodes"][0][0],"ComfyUI-N-Nodes"))
+        repo = git.Repo(get_ext_dir())
         return repo.head.object.hexsha[:8]
     except:
         return 0
@@ -83,7 +83,7 @@ def downloader(link):
                 f.write(chunk) 
 
     zip_file = zipfile.ZipFile(temp_file) 
-    target_dir = os.path.join(folder_paths.folder_names_and_paths["custom_nodes"][0][0],"ComfyUI-N-Nodes","libs","rifle") # Cartella dove estrarre lo zip
+    target_dir = get_ext_dir(os.path.join("libs", "rifle"))
 
     zip_file.extractall(target_dir) 
 

py/frame_interpolator_node.py

@@ -13,7 +13,8 @@ from queue import Queue, Empty
 from pathlib import Path
 
 
-sys.path.append(os.path.join(str(Path(__file__).parent.parent),"libs","rifle"))
+rife_dir = Path(__file__).resolve().parent.parent / "libs" / "rifle"
+sys.path.append(str(rife_dir))
 
 from model.pytorch_msssim import ssim_matlab
 interpolation_temp_input_folder = os.path.join(folder_paths.get_temp_directory(),"n-suite","interpolation_input")
@@ -48,7 +49,7 @@ model = Model()
 if not hasattr(model, 'version'):
     model.version = 0
 
-model_folder= os.path.join(folder_paths.folder_names_and_paths["custom_nodes"][0][0],'ComfyUI-N-Nodes','libs','rifle','train_log')
+model_folder = str(rife_dir / "train_log")
 
 
 
@@ -321,4 +322,3 @@ NODE_DISPLAY_NAME_MAPPINGS = {
 
 
 
-

py/image_captioning_node.py

@@ -3,21 +3,23 @@ import os
 from pathlib import Path
 import sys
 import torch
-from huggingface_hub import snapshot_download, hf_hub_download
+from huggingface_hub import snapshot_download
 sys.path.append(os.path.join(str(Path(__file__).parent.parent),"libs"))
 import joytag_models
-try:
-    from moondream_repo.moondream.moondream import Moondream
-    moondream_loaded = True
-except Exception as e:
-    moondream_loaded = False
-    print(f"Moondream error: reinstall N-Suite dependencies with ComfyUI Manager or requirements.txt.\nTorch must be >= 2.1.0 (ERROR: {e})")
 from PIL import Image
-from transformers import CodeGenTokenizerFast as Tokenizer
+from transformers import AutoModelForCausalLM, CodeGenTokenizerFast as Tokenizer, GenerationConfig, GenerationMixin, PreTrainedModel
+from transformers.dynamic_module_utils import HF_MODULES_CACHE
+from server import PromptServer
 #,AutoTokenizer, AutoModelForCausalLM
 import numpy as np
 
 models_base_path = os.path.join(folder_paths.models_dir, "GPTcheckpoints")
+MOONDREAM_REVISION = "f6e9da68e8f1b78b8f3ee10905d56826db7a5802"
+JOYTAG_REVISION = "6b7f16331a6ccf0fdce37d5a9564715f6e772b22"
+MODEL_DOWNLOADS = {
+    "moondream": ("Moondream", 3.72),
+    "joytag": ("JoyTag", 0.37),
+}
 _choice = ["YES", "NO"]
 _folders_whitelist = ["moondream","joytag"]#,"internlm"]
 
@@ -78,7 +80,12 @@ def load_joytag(ckpt_path,cpu=False):
 
 
     if os.path.exists(jt_config)==False or os.path.exists(jt_readme)==False or os.path.exists(jt_top_tags)==False or os.path.exists(jt_model)==False:
-        snapshot_download("fancyfeast/joytag",local_dir = os.path.join(models_base_path,"joytag"),local_dir_use_symlinks = False,)
+        snapshot_download(
+            "fancyfeast/joytag",
+            revision=JOYTAG_REVISION,
+            local_dir=os.path.join(models_base_path, "joytag"),
+            allow_patterns=["README.md", "config.json", "model.safetensors", "top_tags.txt"],
+        )
     model = joytag_models.VisionModel.load_model(ckpt_path)
     model.eval()
     if cpu:
@@ -112,43 +119,96 @@ def load_moondream(ckpt_path,cpu=False):
         device = torch.device("cuda")
  
 
-    config_json=os.path.join(os.path.join(models_base_path,"moondream"),'config.json')
-    if os.path.exists(config_json)==False:
-        hf_hub_download("vikhyatk/moondream1",
-                                    local_dir=os.path.join(models_base_path,"moondream"),
-                                    local_dir_use_symlinks=True,
-                                    filename="config.json",
-                                    endpoint='https://hf-mirror.com')
-    
-    model_safetensors=os.path.join(models_base_path,"moondream",'model.safetensors')
-    if os.path.exists(model_safetensors)==False:
-        hf_hub_download("vikhyatk/moondream1",
-                                   local_dir=os.path.join(models_base_path,"moondream"),
-                                   local_dir_use_symlinks=True,
-                                   filename="model.safetensors",
-                                   endpoint='https://hf-mirror.com')
-    
-    tokenizer_json=os.path.join(models_base_path,"moondream",'tokenizer.json')
-    if os.path.exists(tokenizer_json)==False:
-        hf_hub_download("vikhyatk/moondream1",
-                                   local_dir=os.path.join(models_base_path,"moondream"),
-                                   local_dir_use_symlinks=True,
-                                   filename="tokenizer.json",
-                                   endpoint='https://hf-mirror.com')
-    
-    if moondream_loaded:
-        tokenizer = Tokenizer.from_pretrained(os.path.join(models_base_path,"moondream"))
-        moondream = Moondream.from_pretrained(os.path.join(models_base_path,"moondream")).to(device=device, dtype=dtype)
-        moondream.eval()
-    else:
-        tokenizer=None
-        moondream=None
+    model_dir = os.path.join(models_base_path, "moondream")
+    snapshot_download(
+        "vikhyatk/moondream1",
+        revision=MOONDREAM_REVISION,
+        local_dir=model_dir,
+        allow_patterns=[
+            "config.json", "configuration_moondream.py", "moondream.py",
+            "modeling_phi.py", "text_model.py", "vision_encoder.py",
+            "model.safetensors", "tokenizer.json", "tokenizer_config.json",
+            "special_tokens_map.json", "added_tokens.json", "merges.txt", "vocab.json",
+        ],
+    )
+    patch_moondream_model_code(model_dir)
+    tokenizer = Tokenizer.from_pretrained(model_dir)
+    moondream = AutoModelForCausalLM.from_pretrained(model_dir, trust_remote_code=True)
+    enable_moondream_generation(moondream)
+    moondream = moondream.to(device=device, dtype=dtype)
+    moondream.eval()
+    return [moondream, tokenizer]
 
 
+def enable_moondream_generation(moondream):
+    """Restore generation for Moondream1's legacy Phi model on Transformers 4.50+."""
+    text_model = moondream.text_model
+    if getattr(text_model, "_n_suite_generation_compat", False):
+        return
+
+    model_class = type(text_model)
+    original_prepare = model_class.prepare_inputs_for_generation
+
+    def prepare_inputs_for_generation(
+        self, input_ids=None, inputs_embeds=None, past_key_values=None,
+        attention_mask=None, **kwargs,
+    ):
+        prepared = original_prepare(
+            self, input_ids=input_ids, inputs_embeds=inputs_embeds,
+            past_key_values=past_key_values, attention_mask=attention_mask,
+            **kwargs,
+        )
+        # Moondream supplies image embeddings without padding. The newer
+        # generation API otherwise builds a mask one token too long.
+        prepared["attention_mask"] = None
+        return prepared
+
+    bases = (model_class,) if isinstance(text_model, GenerationMixin) else (model_class, GenerationMixin)
+    text_model.__class__ = type(
+        "GeneratingPhiForCausalLM", bases,
+        {"prepare_inputs_for_generation": prepare_inputs_for_generation},
+    )
+    text_model._n_suite_generation_compat = True
+    if text_model.generation_config is None:
+        text_model.generation_config = GenerationConfig.from_model_config(text_model.config)
 
 
+def patch_moondream_model_code(model_dir):
+    """Add GenerationMixin to the pinned Phi source before Transformers imports it."""
+    original_import = "from transformers import PretrainedConfig, PreTrainedModel"
+    parent_base = "class PhiPreTrainedModel(PreTrainedModel):"
+    previous_patch = "class PhiPreTrainedModel(PreTrainedModel, GenerationMixin):"
+    model_bases = (
+        ("class PhiModel(PhiPreTrainedModel):", "class PhiModel(PhiPreTrainedModel, GenerationMixin):"),
+        ("class PhiForCausalLM(PhiPreTrainedModel):", "class PhiForCausalLM(PhiPreTrainedModel, GenerationMixin):"),
+    )
+    needs_patch = not issubclass(PreTrainedModel, GenerationMixin)
 
-    return ([moondream, tokenizer])
+    def patch_source(path):
+        source = path.read_text()
+        if original_import not in source:
+            raise RuntimeError("Unexpected Moondream1 Phi source; cannot apply the generation compatibility patch")
+        if previous_patch in source:
+            source = source.replace(previous_patch, parent_base, 1)
+        for original, patched in model_bases:
+            if original not in source and patched not in source:
+                raise RuntimeError("Unexpected Moondream1 Phi source; cannot apply the generation compatibility patch")
+            if needs_patch:
+                source = source.replace(original, patched, 1)
+            else:
+                source = source.replace(patched, original, 1)
+        patched_import = original_import + ", GenerationMixin"
+        if needs_patch:
+            source = source.replace(original_import, patched_import, 1) if patched_import not in source else source
+        else:
+            source = source.replace(patched_import, original_import, 1)
+        if source != path.read_text():
+            path.write_text(source)
+
+    patch_source(Path(model_dir) / "modeling_phi.py")
+    cached_source = Path(HF_MODULES_CACHE) / "transformers_modules" / Path(model_dir).name / "modeling_phi.py"
+    if cached_source.is_file():
+        patch_source(cached_source)
     
 
 
@@ -249,15 +309,12 @@ def run_internlm(image, prompt, max_tags, model_funct):
      
 
 
-if not os.path.isdir(models_base_path):
-        os.mkdir(models_base_path)
+os.makedirs(models_base_path, exist_ok=True)
 
 #create folder if it doesn't exist
-if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","joytag")):
-        os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","joytag"))
+os.makedirs(os.path.join(models_base_path, "joytag"), exist_ok=True)
 
-if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","moondream")):
-        os.mkdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","moondream"))
+os.makedirs(os.path.join(models_base_path, "moondream"), exist_ok=True)
 
 """#internlm
 if not os.path.isdir(os.path.join(folder_paths.models_dir, "GPTcheckpoints","internlm")):
@@ -293,21 +350,33 @@ class GPTLoaderSimple:
               "gpu_layers": ("INT", {"default": 27, "min": 0, "max": 100, "step": 1}),
               "n_threads": ("INT", {"default": 8, "min": 1, "max": 100, "step": 1}),
               "max_ctx": ("INT", {"default": 2048, "min": 300, "max": 100000, "step": 64}),
-                             }}
+                             },
+                "hidden": {"unique_id": "UNIQUE_ID"}}
     
 
 
     RETURN_TYPES = ("CUSTOM", )
     RETURN_NAMES = ("model",)
     FUNCTION = "load_gpt_checkpoint"
-    DESCRIPTION = "Loads a Moondream or JoyTag image-captioning model. GGUF and LLaVA support was removed in version 1.2.0."
+    DESCRIPTION = "Loads Moondream (~3.72 GB) or JoyTag (~0.37 GB). The first use downloads the selected model; watch the ComfyUI console for progress."
 
     CATEGORY = "N-Suite/loaders"
  
-    def load_gpt_checkpoint(self, ckpt_name, gpu_layers, n_threads, max_ctx):
+    def load_gpt_checkpoint(self, ckpt_name, gpu_layers, n_threads, max_ctx, unique_id=None):
         ckpt_path = get_model_path(all_models,ckpt_name)
         if ckpt_name not in MODEL_LOAD_FUNCTIONS:
             raise ValueError(f"Unsupported model: {ckpt_name}")
+        model_dir = os.path.join(models_base_path, ckpt_name)
+        if not os.path.isfile(os.path.join(model_dir, "model.safetensors")):
+            model_name, size_gb = MODEL_DOWNLOADS[ckpt_name]
+            message = (f"{model_name}: downloading approximately {size_gb:.2f} GB on first use. "
+                       "This may take a while; watch the ComfyUI console for progress.")
+            print(f"[N-Suite] {message}", flush=True)
+            if PromptServer.instance is not None:
+                PromptServer.instance.send_sync(
+                    "n-suite-model-download",
+                    {"node_id": unique_id, "model": model_name, "size_gb": size_gb, "message": message},
+                )
         cpu = gpu_layers == 0
         llm = MODEL_LOAD_FUNCTIONS[ckpt_name](ckpt_path, cpu)
 

py/video_node_advanced.py

@@ -9,14 +9,14 @@ import cv2
 import os
 import imageio
 import shutil
-from moviepy.editor import VideoFileClip, AudioFileClip
+from moviepy import VideoFileClip, AudioFileClip
+from contextlib import ExitStack
 import random
 import math
 import json
 from comfy.cli_args import args
 import time
 import concurrent.futures
-import skbuild
 
 
 
@@ -417,13 +417,12 @@ class LoadVideoAdvanced:
         if file_extension in [".mp4", ".webm"]:
             list_files = extract_frames_from_video(file_path, full_temp_output_dir, fps, use_ram)
             
-            audio_clip = VideoFileClip(file_path).audio
             try:
-                # Save audio
-                audio_clip.write_audiofile(os.path.join(temp_output_dir, video.split(".")[0], "audio.mp3"))
-            except:
-                print("Could not save audio")
-                pass      
+                with VideoFileClip(file_path) as video_clip:
+                    if video_clip.audio is not None:
+                        video_clip.audio.write_audiofile(os.path.join(temp_output_dir, video.split(".")[0], "audio.mp3"))
+            except Exception as exc:
+                print(f"Could not save audio: {exc}")
         elif file_extension == ".gif":
             list_files = extract_frames_from_gif(file_path, output_dir)
         else:
@@ -603,32 +602,28 @@ class SaveVideo:
         if(file_name_number >= frame_number):
             create_video_from_frames(frames_output_dir, videos_output_temp_dir,frame_rate=fps)
         
-            video_clip = VideoFileClip(videos_output_temp_dir)
-            try:
-                audio_clip =  AudioFileClip(os.path.join(temp_output_dir,video_filename_original,"audio.mp3"))
-                video_clip = video_clip.set_audio(audio_clip)
-            except:
-                print("No audio found")
-                pass
-            
-            if SaveFrames == True:
-                #copy frames_output_dir to self.video_file_path/self.video_filename
-                frame_folder=os.path.join(videos_output_dir,self.video_filename.split(".")[0])
-                
-                shutil.copytree(frames_output_dir, frame_folder)
+            with ExitStack() as clips:
+                video_clip = clips.enter_context(VideoFileClip(videos_output_temp_dir))
+                audio_path = os.path.join(temp_output_dir, video_filename_original, "audio.mp3")
+                if os.path.isfile(audio_path):
+                    audio_clip = clips.enter_context(AudioFileClip(audio_path))
+                    video_clip = video_clip.with_audio(audio_clip)
 
-            if SaveVideo == True:
-                video_clip.write_videofile(self.video_file_path)
-                file_name = self.video_filename
-            else:
-                #delete all temporary files that start with video_preview
-                for file in os.listdir(video_preview_output_temp_dir):
-                    if file.startswith("video_preview"):
-                        os.remove(os.path.join(video_preview_output_temp_dir,file))
-                #random number
-                suffix = str(random.randint(1,100000))
-                file_name = f"video_preview_{suffix}.mp4"
-                video_clip.write_videofile(os.path.join(video_preview_output_temp_dir,file_name))
+                if SaveFrames == True:
+                    #copy frames_output_dir to self.video_file_path/self.video_filename
+                    frame_folder=os.path.join(videos_output_dir,self.video_filename.split(".")[0])
+                    shutil.copytree(frames_output_dir, frame_folder)
+
+                if SaveVideo == True:
+                    video_clip.write_videofile(self.video_file_path)
+                    file_name = self.video_filename
+                else:
+                    for file in os.listdir(video_preview_output_temp_dir):
+                        if file.startswith("video_preview"):
+                            os.remove(os.path.join(video_preview_output_temp_dir,file))
+                    suffix = str(random.randint(1,100000))
+                    file_name = f"video_preview_{suffix}.mp4"
+                    video_clip.write_videofile(os.path.join(video_preview_output_temp_dir,file_name))
                 
           
 
@@ -722,4 +717,4 @@ NODE_DISPLAY_NAME_MAPPINGS = {
     "SaveVideo [n-suite]": "SaveVideo [🅝-🅢🅤🅘🅣🅔]",
     "LoadFramesFromFolder [n-suite]": "LoadFramesFromFolder [🅝-🅢🅤🅘🅣🅔]",
     "SetMetadataForSaveVideo [n-suite]": "SetMetadataForSaveVideo [🅝-🅢🅤🅘🅣🅔]"
-}
+}

pyproject.toml

@@ -6,11 +6,11 @@ license = { file = "LICENSE" }
 dependencies = [
     "gitpython",
     "huggingface-hub",
-    "moviepy<2",
+    "moviepy>=2.2.1,<3",
     "opencv-python",
-    "scikit-build",
-    "timm==0.9.12",
-    "transformers==4.36.2",
+    "accelerate>=1.0,<2",
+    "timm>=1.0.22",
+    "transformers>=4.36.2,<5",
 ]
 
 [project.urls]

requirements.txt

@@ -1,7 +1,7 @@
 gitpython
 huggingface-hub
-moviepy<2
+moviepy>=2.2.1,<3
 opencv-python
-scikit-build
-timm==0.9.12
-transformers==4.36.2
+accelerate>=1.0,<2
+timm>=1.0.22
+transformers>=4.36.2,<5

tests/test_llama_cpp_removal.py

@@ -26,20 +26,22 @@ def test_readme_announces_breaking_change_and_model_downloads():
     readme = (ROOT / "README.md").read_text()
     assert "Breaking change in 1.2.0" in readme
     assert "was never downloaded automatically" in readme
-    assert "first time Moondream is loaded" in readme
-    assert "first time JoyTag is loaded" in readme
+    assert "Downloads happen on first model use" in readme
+    assert "3.72 GB" in readme
+    assert "0.37 GB" in readme
     assert "git checkout ae7cc84" in readme
-    assert "old LLaVA implementation was not independent" in readme
 
 
-def test_dependencies_are_manager_installable_and_moviepy_is_compatible():
+def test_dependencies_allow_current_comfyui_versions():
     requirements = (ROOT / "requirements.txt").read_text().splitlines()
     pyproject = (ROOT / "pyproject.toml").read_text()
-    assert "moviepy<2" in requirements
-    assert '"moviepy<2"' in pyproject
+    assert "moviepy>=2.2.1,<3" in requirements
+    assert '"moviepy>=2.2.1,<3"' in pyproject
     assert "huggingface-hub" in requirements
-    assert "transformers==4.36.2" in requirements
-    assert "timm==0.9.12" in requirements
+    assert "transformers>=4.36.2,<5" in requirements
+    assert "timm>=1.0.22" in requirements
+    assert "accelerate>=1.0,<2" in requirements
+    assert "scikit-build" not in requirements
 
 
 def test_extension_does_not_install_packages_during_import():
@@ -66,7 +68,8 @@ def test_dynamic_widgets_use_current_removal_api_and_node_id():
 def test_external_repositories_and_model_archive_are_pinned():
     bootstrap = (ROOT / "__init__.py").read_text()
     assert 'RIFE_REVISION = "a8a8035323b1c1a4a20753c751780e5b0a879455"' in bootstrap
-    assert 'MOONDREAM_REVISION = "38af98596e59f2a6c25c6b52b2bd5a672dab4144"' in bootstrap
+    captioning = (ROOT / "py" / "image_captioning_node.py").read_text()
+    assert 'MOONDREAM_REVISION = "f6e9da68e8f1b78b8f3ee10905d56826db7a5802"' in captioning
     assert 'RIFE_MODEL_REVISION = "572480112b87f9bfbff7579b8a38b483766e455f"' in bootstrap
     assert "/raw/main/RIFE_trained_model" not in bootstrap
     assert "repo.git.checkout(revision)" in bootstrap
@@ -78,5 +81,3 @@ def test_registry_publish_is_manual_after_nightly_validation():
     assert "push:" not in workflow
 
     readme = (ROOT / "README.md").read_text()
-    assert "Testing this release through Manager Nightly" in readme
-    assert "a pull request cannot be selected as nightly while it is still unmerged" in readme