chore: ✨ stash audio experiments

closes #120

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -2,7 +2,9 @@ name: 🐞 Bug Report
 title: "[bug] "
 description: Report a bug
 labels: ["type: 🐛 bug", "status: 🧹 needs triage"]
-
+assignees:
+  - melMass
+  
 body:
   - type: markdown
     attributes:
@@ -40,16 +42,30 @@ body:
       label: Expected behavior
       description: A clear description of what you expected to happen.
 
-  - type: textarea
-    id: info
+  - type: dropdown
+    id: os
     attributes:
-      label: Platform and versions
-      description: "informations about the environment you run Comfy in"
-      render: sh
-      placeholder: |
-        - OS: [e.g. Linux]
-        - Comfy Mode [e.g. custom env, standalone, google colab]
-
+      label: Operating System
+      description: What OS are you using?
+      options:
+        - Windows (Default)
+        - Linux
+        - Mac
+      default: 0
+    validations:
+      required: true
+      
+  - type: dropdown
+    id: comfy_mode
+    attributes:
+      label: Comfy Mode
+      description: What flavor of Comfy do you use?
+      options:
+        - Comfy Portable (embed) (Default)
+        - In a custom virtual env (venv, virtualenv, conda...)
+        - Google Colab
+        - Other (online services, containers etc..)
+      default: 0
     validations:
       required: true
 

INSTALL.md

@@ -4,7 +4,6 @@
     - [ComfyUI Manager](#comfyui-manager)
     - [Virtual Env](#virtual-env)
   - [Models Download](#models-download)
-    - [Web Extensions](#web-extensions)
   - [Old installation method (MANUAL)](#old-installation-method-manual)
     - [Dependencies](#dependencies)
 
@@ -35,11 +34,6 @@ then follow the prompt or just press enter to download every models.
   python scripts/download_models.py -y
   ```
 
-### Web Extensions
-
-On first run the script [tries to symlink](https://github.com/melMass/comfy_mtb/blob/d982b69a58c05ccead9c49370764beaa4549992a/__init__.py#L45-L61) the [web extensions](https://github.com/melMass/comfy_mtb/tree/main/web) to your comfy `web/extensions` folder. In case it fails you can manually copy the mtb folder to `ComfyUI/web/extensions` it only provides a color widget for now shared by a few nodes:  
-
-<img alt="color widget preview" src="https://github.com/melMass/comfy_mtb/assets/7041726/cff7e66a-4cc4-4866-b35b-10af0bb2d110" width=450>
 
 ## Old installation method (MANUAL)
 ### Dependencies

README.md

@@ -1,7 +1,8 @@
 # MTB Nodes
-
 [![embedded test](https://github.com/melMass/comfy_mtb/actions/workflows/test_embedded.yml/badge.svg)](https://github.com/melMass/comfy_mtb/actions/workflows/test_embedded.yml)
 
+![home](https://repository-images.githubusercontent.com/649047066/a3eef9a7-20dd-4ef9-b839-884502d4e873)
+
 <!-- omit in toc -->
 
 **Translated Readme (using DeepTranslate, PRs are welcome)**:  
@@ -18,20 +19,50 @@ Welcome to the MTB Nodes project! This codebase is open for you to explore and u
 
 Before proceeding, please be aware of the licenses associated with certain libraries used in this project. For example, the `deepbump` library is licensed under [GPLv3](https://github.com/HugoTini/DeepBump/blob/master/LICENSE).
 
+- [Web Extensions](#web-extensions)
 - [Node List](#node-list)
+  - [Animation](#animation)
   - [bbox](#bbox)
   - [colors](#colors)
-  - [face detection / swapping](#face-detection--swapping)
-  - [image interpolation (animation)](#image-interpolation-animation)
   - [image ops](#image-ops)
   - [latent utils](#latent-utils)
-  - [misc utils](#misc-utils)
   - [textures](#textures)
+  - [misc utils](#misc-utils)
+  - [Optional nodes](#optional-nodes)
+    - [face detection / swapping](#face-detection--swapping)
+    - [image interpolation (animation)](#image-interpolation-animation)
 - [Comfy Resources](#comfy-resources)
 
+# Web Extensions
+mtb add a few widgets like `COLOR`
+
+<img alt="color widget preview" src="https://github.com/melMass/comfy_mtb/assets/7041726/cff7e66a-4cc4-4866-b35b-10af0bb2d110" width=450>
+
+A few nodes have the concept of "dynamic" inputs:  
+<img alt="dynamic inputs" width=450  src="https://github.com/melMass/comfy_mtb/assets/7041726/10b3976e-b212-4968-91eb-f34c02bb80c3" />
+
 
 # Node List
 
+## Animation
+- `Animation Builder`: Convenient way to manage basic animation maths at the core of many of my workflows (both worflows for the following GIFs are in the [examples](https://github.com/melMass/comfy_mtb/wiki/Examples))  
+
+  **[Example lerping two conditions (blue car -> yellow car)](https://github.com/melMass/comfy_mtb/blob/main/examples/03-animation_builder-condition-lerp.json)**
+  
+  <img width=300 src="https://user-images.githubusercontent.com/7041726/260258970-d6d66d96-fb34-40d0-9038-cbabf0714c5d.gif"/>  
+  
+
+  **[Example using image transforms a feedback for a fake deforum effect](https://github.com/melMass/comfy_mtb/blob/main/examples/04-animation_builder-deforum.json)**
+  
+  <img width=300 src="https://user-images.githubusercontent.com/7041726/260261504-303a1037-60d3-4b31-a589-b15d549752f6.gif"/>
+  
+- `Batch Float`: Generates a batch of float values with interpolation.
+- `Batch Shape`: Generates a batch of 2D shapes with optional shading (experimental).
+- `Batch Transform`: Transform a batch of images using a batch of keyframes.  
+  <img width=400 src="https://github.com/melMass/comfy_mtb/assets/7041726/3f217de1-79aa-49b0-a66a-35cf29dd8f01"/>
+- `Export With Ffmpeg`: Export with FFmpeg, it used to be export to Proress and is still tailored for YUV
+- `Fit Number` : Fit the input float using a source and target range, you can also control the interpolation curve from a list of presets (default to linear)
+  
 ## bbox
 - `Bounding Box`: BBox constructor (custom type),
 - `BBox From Mask`: From a mask extract the bounding box
@@ -43,21 +74,7 @@ Before proceeding, please be aware of the licenses associated with certain libra
 - `RGB to HSV`: -,
 - `HSV to RGB`: -,
 - `Color Correct`: Basic color correction tools  
-  <img src="https://github.com/melMass/comfy_mtb/assets/7041726/7c20ac83-31ff-40ea-a1a0-06c2acefb2ef" width=345/>
-
-## face detection / swapping
-- `Face Swap`: Face swap using deepinsight/insightface models (this node used to be called `Roop` in early versions, it does the same, roop is *just* an app that uses those model)
-  > **Note**
-  > The face index allow you to choose which face to replace as you can see here:  
-  <img src="https://github.com/melMass/comfy_mtb/assets/7041726/2e9d6066-c466-4a01-bd6c-315f7f1e8b42" width=320/>
-- `Load Face Swap Model`: Load an insightface model for face swapping
-- `Restore Face`: Using [GFPGan](https://github.com/TencentARC/GFPGAN) to restore faces, works great in conjunction with `Face Swap` and supports Comfy native upscalers for the `bg_upscaler`
-  
-## image interpolation (animation)
-- `Load Film Model`: Loads a [FILM](https://github.com/google-research/frame-interpolation) model
-- `Film Interpolation`: Process input frames using [FILM](https://github.com/google-research/frame-interpolation)  
-  <img src="https://github.com/melMass/comfy_mtb/assets/7041726/3afd1647-6634-4b92-a34b-51432e6a9834" width=400/>
-- `Export to Prores (experimental)`: Exports the input frames to a ProRes 4444 mov file. This is using ffmpeg stdin to send raw numpy arrays, used with `Film Interpolation` and very simple for now but could be expanded upon.
+  <img src="https://github.com/melMass/comfy_mtb/assets/7041726/7c20ac83-31ff-40ea-a1a0-06c2acefb2ef" width=400/>
 
 ## image ops
 - `Blur`: Blur an image using a Gaussian filter.
@@ -74,8 +91,16 @@ Before proceeding, please be aware of the licenses associated with certain libra
 ## latent utils
 - `Latent Lerp`: Linear interpolation (blend) between two latent 
 
+## textures
+- `Model Patch Seamless`: Use the [seamless diffusion "hack"](https://gitlab.com/-/snippets/2395088) to patch any model to infere seamless images, check the [examples](https://github.com/melMass/comfy_mtb/wiki/Examples) to see how to use all those textures node together  
+  <img width=500 src="https://user-images.githubusercontent.com/7041726/272970506-9db516b5-45d2-4389-b904-b3a94660f24c.png"/>
+- `DeepBump`: Normal & height maps generation from single pictures  
+  <img width=500 src="https://user-images.githubusercontent.com/7041726/272970715-7e4477f6-8e18-4839-9864-83d07d6690a1.png"/>
+- `Image Tile Offset`: Mimics an old photoshop technique to check for seamless textures by offsetting tiles of the image.  
+  <img width=600 src="https://github.com/melMass/comfy_mtb/assets/7041726/cbcc51fb-922f-433f-acf1-c6c6c2a7ffc4" />
 
 ## misc utils
+- `Any To String`: Tries to take any input and convert it to a string.
 - `Concat Images`: Takes two image stream and merge them as a batch of images supported by other Comfy pipelines.
 - `Image Resize Factor`: **Deprecated**, I since discovered the builtin image resize.
 - `Text To Image`: Utils to convert text to image using a font
@@ -86,13 +111,57 @@ Before proceeding, please be aware of the licenses associated with certain libra
 - `Save Tensors`: Debug node that will probably be removed in the future
 - `Int to Number`: Supplement for WASSuite number nodes
 - `Smart Step`: A very basic tool to control the steps (start/stop) of the `KAdvancedSampler` using percentage
+- `Load Image From Url`: Load an image from the given URL
 
-## textures
 
-- `DeepBump`: Normal & height maps generation from single pictures
+## Optional nodes
+
+These nodes are still bundled in mtb, but moving forward (>0.2.0) they won't
+be setup by the install script and their dependencies won't install either.
+The reason is mostly that they all have a better alternatives available and tensorflow on windows was not a fun experience and since Python 3.11 not an experience at all.
+
+For linux and mac users though these nodes didn't cause any issue and I personally still use them, these are the extra requirements needed:
+
+```console
+.venv/python -m pip install tensorflow facexlib insightface basicsr
+```
+
+### face detection / swapping
+> **Warning**
+> Those nodes were among the first to be implemented they do work, but on windows the installation is still not properly handled for everyone  
+> As alternatives you can use [reactor](https://github.com/Gourieff/comfyui-reactor-node) for face swap and [facerestore](https://github.com/Haidra-Org/hordelib/tree/main/hordelib/nodes/facerestore) for restoration  
+> You can check [this video](https://www.youtube.com/watch?v=FShlpMxbU0E) for a tutorial by Ferniclestix using these alternatives  
+
+- `Face Swap`: Face swap using deepinsight/insightface models (this node used to be called `Roop` in early versions, it does the same, roop is *just* an app that uses those model)  
+  <img  width=320 src="https://user-images.githubusercontent.com/7041726/260261217-54e33446-183f-4dda-88b3-d38a1e6de980.gif"/>
+- `Load Face Swap Model`: Load an insightface model for face swapping
+- `Restore Face`: Using [GFPGan](https://github.com/TencentARC/GFPGAN) to restore faces, works great in conjunction with `Face Swap` and supports Comfy native upscalers for the `bg_upscaler`
+  
+### image interpolation (animation)
+> **Warning**
+> The FILM nodes will be deprecated at some point after 0.2.0, [Fannovel16](https://github.com/Fannovel16/ComfyUI-Frame-Interpolation)'s interpolation nodes implement it and they rely on a pytorch implementation of FILM
+> which solves the issues related to the ones included in mtb. They will probably remain available if your system meet the requirements and ignored otherwise.
+
+<details><summary>Why?</summary>
+  
+> **Windows only issue**: This requires tensorflow-gpu that is unfortunately not a thing anymore on Windows since 2.10.1 (unless you use a complex WSL passthrough setup but it's still not "Windows")  
+> Using this old version is quite clunky and require some patching that install.py does automatically, but the main issue is that no wheels are available for python > 3.10
+> Comfy-nightly is already using Python 11 so installing this old tf version won't work there.  
+> You can in any case install the normal up to date tensorflow but that will run on CPU and is much MUCH slower for FILM inference.
+</details>
+
+- `Load Film Model`: Loads a [FILM](https://github.com/google-research/frame-interpolation) model
+- `Film Interpolation`: Process input frames using [FILM](https://github.com/google-research/frame-interpolation)  
+  <img width=400 src="https://github.com/melMass/comfy_mtb/assets/7041726/3afd1647-6634-4b92-a34b-51432e6a9834"/>  
+  <img width=400 src="https://user-images.githubusercontent.com/7041726/260259079-c0f04a63-960c-43a7-ba78-a45cd5ac7514.gif"/>
+- `Export to Prores (experimental)`: Exports the input frames to a ProRes 4444 mov file. This is using ffmpeg stdin to send raw numpy arrays, used with `Film Interpolation` and very simple for now but could be expanded upon.
 
 # Comfy Resources
 
+**Misc**
+
+- [Slick ComfyUI by NoCrypt](https://colab.research.google.com/drive/1ZMvLWEiYITmBJngtqeIQToeNuiydwI0z#scrollTo=1fWMaexXS188): A colab notebook with batteries included!
+
 **Guides**:
 - [Official Examples (eng)](https://comfyanonymous.github.io/ComfyUI_examples/)
 - [ComfyUI Community Manual (eng)](https://blenderneko.github.io/ComfyUI-docs/) by @BlenderNeko

__init__.py

@@ -13,26 +13,37 @@ import os
 os.environ["TF_FORCE_GPU_ALLOW_GROWTH"] = "true"
 os.environ["TF_GPU_ALLOCATOR"] = "cuda_malloc_async"
 
-import traceback
-from .log import log, blue_text, cyan_text, get_summary, get_label
-from .utils import here
-from .utils import comfy_dir
-import importlib
-import os
 import ast
+import contextlib
+import importlib
 import json
+import logging
+import os
+import shutil
+import traceback
+from importlib import reload
+
+from aiohttp import web
+from server import PromptServer
+
+import nodes
+
+from .endpoint import endlog
+from .log import blue_text, cyan_text, get_label, get_summary, log
+from .utils import comfy_dir, here
 
 NODE_CLASS_MAPPINGS = {}
 NODE_DISPLAY_NAME_MAPPINGS = {}
 NODE_CLASS_MAPPINGS_DEBUG = {}
+WEB_DIRECTORY = "./web"
 
-__version__ = "0.1.4"
+__version__ = "0.2.0"
 
 
 def extract_nodes_from_source(filename):
     source_code = ""
 
-    with open(filename, "r") as file:
+    with open(filename, "r", encoding="utf8") as file:
         source_code = file.read()
 
     nodes = []
@@ -85,7 +96,7 @@ def load_nodes():
                 nodes_failed.extend(extract_nodes_from_source(filename))
 
     if errors:
-        log.info(
+        log.debug(
             f"Some nodes failed to load:\n\t"
             + "\n\t".join(errors)
             + "\n\n"
@@ -100,48 +111,17 @@ def load_nodes():
 web_extensions_root = comfy_dir / "web" / "extensions"
 web_mtb = web_extensions_root / "mtb"
 
-if web_mtb.exists():
-    log.debug(f"Web extensions folder found at {web_mtb}")
-    if not os.path.islink(web_mtb.as_posix()):
-        log.warn(
-            f"Web extensions folder at {web_mtb} is not a symlink, if updating please delete it before"
-        )
-
-
-elif web_extensions_root.exists():
-    web_tgt = here / "web"
-    src = web_tgt.as_posix()
-    dst = web_mtb.as_posix()
+if web_mtb.exists() and hasattr(nodes, "EXTENSION_WEB_DIRS"):
     try:
-        if os.name == "nt":
-            import _winapi
-
-            _winapi.CreateJunction(src, dst)
+        if web_mtb.is_symlink():
+            web_mtb.unlink()
         else:
-            os.symlink(web_tgt.as_posix(), web_mtb.as_posix())
-
-    except OSError:
-        log.warn(f"Failed to create symlink to {web_mtb}, trying to copy it")
-        try:
-            import shutil
-
-            shutil.copytree(web_tgt, web_mtb)
-            log.info(f"Successfully copied {web_tgt} to {web_mtb}")
-        except Exception as e:
-            log.warn(
-                f"Failed to symlink and copy {web_tgt} to {web_mtb}. Please copy the folder manually."
-            )
-            log.warn(e)
-
+            shutil.rmtree(web_mtb)
     except Exception as e:
-        log.warn(
-            f"Failed to create symlink to {web_mtb}. Please copy the folder manually."
+        log.warning(
+            f"Failed to remove web mtb directory: {e}\nPlease manually remove it from disk ({web_mtb}) and restart the server."
         )
-        log.warn(e)
-else:
-    log.warn(
-        f"Comfy root probably not found automatically, please copy the folder {web_mtb} manually in the web/extensions folder of ComfyUI"
-    )
+
 
 # - REGISTER NODES
 nodes, failed = load_nodes()
@@ -166,7 +146,7 @@ for node_class in nodes:
                 )
             )
 
-log.info(
+log.debug(
     f"Loaded the following nodes:\n\t"
     + "\n\t".join(
         f"{cyan_text(k)}: {blue_text(get_summary(doc)) if doc else '-'}"
@@ -174,24 +154,56 @@ log.info(
     )
 )
 
+log.info(f"loaded {cyan_text(len(nodes))} nodes successfuly")
+if failed:
+    with contextlib.suppress(Exception):
+        base_url, port = utils.get_server_info()
+        log.info(
+            f"Some nodes ({len(failed)}) could not be loaded. This can be ignored, but go to http://{base_url}:{port}/mtb if you want more information."
+        )
+
+
 # - ENDPOINT
-from server import PromptServer
-from .log import log
-from aiohttp import web
-from importlib import reload
-import logging
-from .endpoint import endlog
+
 
 if hasattr(PromptServer, "instance"):
     restore_deps = ["basicsr"]
-    swap_deps = ["insightface", "onnxruntime"]
-
+    onnx_deps = ["onnxruntime"]
+    swap_deps = ["insightface"] + onnx_deps
     node_dependency_mapping = {
+        "QrCode": ["qrcode"],
+        "DeepBump": onnx_deps,
         "FaceSwap": swap_deps,
         "LoadFaceSwapModel": swap_deps,
         "LoadFaceAnalysisModel": restore_deps,
     }
 
+    PromptServer.instance.app.router.add_static(
+        "/mtb-assets/", path=(here / "html").as_posix()
+    )
+
+    @PromptServer.instance.routes.get("/mtb/manage")
+    async def manage(request):
+        from . import endpoint
+
+        reload(endpoint)
+
+        endlog.debug("Initializing Manager")
+        if "text/html" in request.headers.get("Accept", ""):
+            csv_editor = endpoint.csv_editor()
+
+            tabview = endpoint.render_tab_view(Styles=csv_editor)
+            return web.Response(
+                text=endpoint.render_base_template("MTB", tabview),
+                content_type="text/html",
+            )
+
+        return web.json_response(
+            {
+                "message": "manage only has a POST api for now",
+            }
+        )
+
     @PromptServer.instance.routes.get("/mtb/status")
     async def get_full_library(request):
         from . import endpoint
@@ -255,6 +267,7 @@ if hasattr(PromptServer, "instance"):
             # # Return an HTML page
             html_response = """
             <div class="flex-container menu">
+                <a href="/mtb/manage">manage</a>
                 <a href="/mtb/debug">debug</a>
                 <a href="/mtb/status">status</a>
             </div>            
@@ -309,6 +322,14 @@ if hasattr(PromptServer, "instance"):
 
         return await endpoint.do_action(request)
 
+    @PromptServer.instance.routes.get("/mtb/audio")
+    async def get_audio(request):
+        from . import endpoint
+
+        reload(endpoint)
+
+        return await endpoint.get_audio(request)
+
 
 # - WAS Dictionary
 MANIFEST = {

endpoint.py

@@ -1,34 +1,78 @@
-from .utils import here, run_command, comfy_mode
+import csv
+
 from aiohttp import web
+
 from .log import mklog
-import sys
+from .utils import (
+    audioInputDir,
+    backup_file,
+    comfy_dir,
+    here,
+    import_install,
+    reqs_map,
+    run_command,
+    styles_dir,
+)
 
 endlog = mklog("mtb endpoint")
 
 # - ACTIONS
-import requirements
+import platform
+import sys
+from pathlib import Path
 
+import_install("requirements")
+
+
+def ACTIONS_loadAudio(args):
+    if not audioInputDir.exists():
+        audioInputDir.mkdir()
+
+    endlog.debug(f"Received Load Audio request for {args}")
+
+    if not args.file:
+        return web.Response(status=400)
+
+    filename = args.filename
+    if not filename:
+        return web.Response(status=400)
+
+    target = audioInputDir / filename
+    if target.exists():
+        target.unlink()
+
+    with target.open("wb") as f:
+        f.write(args.file.read())
+
+    return {"name": filename}
 
 
 def ACTIONS_installDependency(dependency_names=None):
     if dependency_names is None:
         return {"error": "No dependency name provided"}
     endlog.debug(f"Received Install Dependency request for {dependency_names}")
-    reqs = []
-    if comfy_mode == "embeded":
-        reqs = list(requirements.parse((here / "reqs_portable.txt").read_text()))
-    else:
-        reqs = list(requirements.parse((here / "reqs.txt").read_text()))
-    print([x.specs for x in reqs])
-    print(
-        "\n".join([f"{x.line} {''.join(x.specs[0] if x.specs else '')}" for x in reqs])
-    )
-    for dependency_name in dependency_names:
-        for req in reqs:
-            if req.name == dependency_name:
-                endlog.debug(f"Dependency {dependency_name} installed")
-                break
-    return {"success": True}
+    # reqs = []
+    resolved_names = [reqs_map.get(name, name) for name in dependency_names]
+    try:
+        run_command([Path(sys.executable), "-m", "pip", "install"] + resolved_names)
+        return {"success": True}
+
+    except Exception as e:
+        return {"error": f"Failed to install dependencies: {e}"}
+
+    # if platform.system() == "Windows":
+    #     reqs = list(requirements.parse((here / "reqs_windows.txt").read_text()))
+    # else:
+    #     reqs = list(requirements.parse((here / "reqs.txt").read_text()))
+    # print([x.specs for x in reqs])
+    # print(
+    #     "\n".join([f"{x.line} {''.join(x.specs[0] if x.specs else '')}" for x in reqs])
+    # )
+    # for dependency_name in dependency_names:
+    #     for req in reqs:
+    #         if req.name == dependency_name:
+    #             endlog.debug(f"Dependency {dependency_name} installed")
+    #             break
 
 
 def ACTIONS_getStyles(style_name=None):
@@ -48,9 +92,35 @@ def ACTIONS_getStyles(style_name=None):
     return {"error": "No styles found"}
 
 
+def ACTIONS_saveStyle(data):
+    # endlog.debug(f"Received Save Styles for {data.keys()}")
+    # endlog.debug(data)
+
+    styles = [f.name for f in styles_dir.iterdir() if f.suffix == ".csv"]
+    target = None
+    rows = []
+    for fp, content in data.items():
+        if fp in styles:
+            endlog.debug(f"Overwriting {fp}")
+            target = styles_dir / fp
+            rows = content
+            break
+
+    if not target:
+        endlog.warning(f"Could not determine the target file for {data.keys()}")
+        return {"error": "Could not determine the target file for the style"}
+
+    backup_file(target)
+
+    with target.open("w", newline="", encoding="utf-8") as file:
+        csv_writer = csv.writer(file, quoting=csv.QUOTE_ALL)
+        for row in rows:
+            csv_writer.writerow(row)
+
+
 async def do_action(request) -> web.Response:
     endlog.debug("Init action request")
-    request_data = await request.json()
+    request_data = await request.post()
     name = request_data.get("name")
     args = request_data.get("args")
 
@@ -62,14 +132,33 @@ async def do_action(request) -> web.Response:
     if callable(method):
         result = method(args) if args else method()
         endlog.debug(f"Action result: {result}")
-        return web.json_response({"result": result})
+        return web.json_response({"result": result}, status=200)
 
     available_methods = [
         attr[len("ACTIONS_") :] for attr in globals() if attr.startswith("ACTIONS_")
     ]
 
     return web.json_response(
-        {"error": "Invalid method name.", "available_methods": available_methods}
+        {"error": "Invalid method name.", "available_methods": available_methods},
+        status=400,
+    )
+
+
+async def get_audio(request):
+    name = request.rel_url.query.get("filename")
+    if not name:
+        return web.json_response(
+            {"error": "No filename provided as url query."}, status=400
+        )
+
+    target = audioInputDir / name
+    if not target.exists():
+        return web.json_response(
+            {"error": f"File {name} (in {audioInputDir}) not found..."}, status=404
+        )
+
+    return web.FileResponse(
+        target, headers={"Content-Disposition": f'filename="{name}"'}
     )
 
 
@@ -83,6 +172,129 @@ def dependencies_button(name, dependencies):
         """
 
 
+def csv_editor():
+    inputs = [f for f in styles_dir.iterdir() if f.suffix == ".csv"]
+    # rows = {f.stem: list(csv.reader(f.read_text("utf8"))) for f in styles}
+
+    style_files = {}
+    for file in inputs:
+        with open(file, "r", encoding="utf8") as f:
+            parsed = csv.reader(f)
+            style_files[file.name] = []
+            for row in parsed:
+                endlog.debug(f"Adding style {row[0]}")
+                style_files[file.name].append((row[0], row[1], row[2]))
+
+    html_out = """
+            <div id="style-editor">
+             <h1>Style Editor</h1>
+            
+            """
+    for current, styles in style_files.items():
+        current_out = f"<h3>{current}</h3>"
+        table_rows = []
+        for index, style in enumerate(styles):
+            table_rows += (
+                (["<tr>"] + [f"<th>{cell}</th>" for cell in style] + ["</tr>"])
+                if index == 0
+                else (
+                    ["<tr>"]
+                    + [
+                        f"<td><input type='text' value='{cell}'></td>"
+                        if i == 0
+                        else f"<td><textarea name='Text1' cols='40' rows='5'>{cell}</textarea></td>"
+                        for i, cell in enumerate(style)
+                    ]
+                    + ["</tr>"]
+                )
+            )
+        current_out += (
+            f"<table data-id='{current}' data-filename='{current}'>"
+            + "".join(table_rows)
+            + "</table>"
+        )
+        current_out += f"<button data-id='{current}' onclick='saveTableData(this.getAttribute(\"data-id\"))'>Save {current}</button>"
+
+        html_out += add_foldable_region(current, current_out)
+
+    html_out += "</div>"
+    html_out += """<script src='/mtb-assets/js/saveTableData.js'></script>"""
+
+    return html_out
+
+
+def render_tab_view(**kwargs):
+    tab_headers = []
+    tab_contents = []
+
+    for idx, (tab_name, content) in enumerate(kwargs.items()):
+        active_class = "active" if idx == 0 else ""
+        tab_headers.append(
+            f"<button class='tablinks {active_class}' onclick=\"openTab(event, '{tab_name}')\">{tab_name}</button>"
+        )
+        tab_contents.append(
+            f"<div id='{tab_name}' class='tabcontent {active_class}'>{content}</div>"
+        )
+
+    headers_str = "\n".join(tab_headers)
+    contents_str = "\n".join(tab_contents)
+
+    return f"""
+<div class='tab-container'>
+    <div class='tab'>
+        {headers_str}
+    </div>
+    {contents_str}
+    </div>
+    <script src='/mtb-assets/js/tabSwitch.js'></script>
+    """
+
+
+def add_foldable_region(title, content):
+    symbol_id = f"{title}-symbol"
+    return f"""
+    <div class='foldable'>
+        <div class='foldable-title' onclick="toggleFoldable('{title}', '{symbol_id}')">
+            <span id='{symbol_id}' class='foldable-symbol'>&#9655;</span>
+            {title}
+        </div>
+        <div id='{title}' class='foldable-content'>
+            {content}
+        </div>
+    </div>
+    <script src='/mtb-assets/js/foldable.js'></script>
+    """
+
+
+def add_split_pane(left_content, right_content, vertical=True):
+    orientation = "vertical" if vertical else "horizontal"
+    return f"""
+    <div class="split-pane {orientation}">
+        <div id="leftPane">
+            {left_content}
+        </div>
+        <div id="resizer"></div>
+        <div id="rightPane">
+            {right_content}
+        </div>
+    </div>
+    <script>
+        initSplitPane({str(vertical).lower()});
+    </script>
+    <script src='/mtb-assets/js/splitPane.js'></script>
+    """
+
+
+def add_dropdown(title, options):
+    option_str = "\n".join([f"<option value='{opt}'>{opt}</option>" for opt in options])
+    return f"""
+    <select>
+        <option disabled selected>{title}</option>
+        {option_str}
+    </select>
+    """
+
+
 def render_table(table_dict, sort=True, title=None):
     table_dict = sorted(
         table_dict.items(), key=lambda item: item[0]
@@ -122,21 +334,13 @@ def render_table(table_dict, sort=True, title=None):
 
 
 def render_base_template(title, content):
-    css_content = ""
-    css_path = here / "html" / "style.css"
-    if css_path:
-        with open(css_path, "r") as css_file:
-            css_content = css_file.read()
-
     github_icon_svg = """<svg xmlns="http://www.w3.org/2000/svg" fill="whitesmoke" height="3em" viewBox="0 0 496 512"><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"/></svg>"""
     return f"""
     <!DOCTYPE html>
     <html>
     <head>
         <title>{title}</title>
-        <style>
-            {css_content}
-        </style>
+        <link rel="stylesheet" href="/mtb-assets/style.css"/>
     </head>
     <script type="module">
         import {{ api }} from '/scripts/api.js'

errors.py

@@ -0,0 +1,7 @@
+class ModelNotFound(Exception):
+    def __init__(self, model_name, *args, **kwargs):
+        super().__init__(
+            f"The model {model_name} could not be found, make sure to download it using ComfyManager first.\nrepository: https://github.com/ltdrdata/ComfyUI-Manager",
+            *args,
+            **kwargs,
+        )

examples/05-seamless_texture.json

@@ -0,0 +1,905 @@
+{
+  "last_node_id": 97,
+  "last_link_id": 179,
+  "nodes": [
+    {
+      "id": 6,
+      "type": "CLIPTextEncode",
+      "pos": [
+        -1165.8749246009997,
+        30
+      ],
+      "size": [
+        422.84503173828125,
+        164.31304931640625
+      ],
+      "flags": {},
+      "order": 4,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "clip",
+          "type": "CLIP",
+          "link": 3
+        }
+      ],
+      "outputs": [
+        {
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            4,
+            158
+          ],
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        "Closeup texture of rocks"
+      ],
+      "color": "#432",
+      "bgcolor": "#653",
+      "shape": 1
+    },
+    {
+      "id": 7,
+      "type": "CLIPTextEncode",
+      "pos": [
+        -1175.8749246009997,
+        250
+      ],
+      "size": [
+        425.27801513671875,
+        180.6060791015625
+      ],
+      "flags": {},
+      "order": 5,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "clip",
+          "type": "CLIP",
+          "link": 5
+        }
+      ],
+      "outputs": [
+        {
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            6
+          ],
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        "((drawing, cartoon, painting, sketch, blur, depth of field, dof))"
+      ],
+      "color": "#432",
+      "bgcolor": "#653",
+      "shape": 1
+    },
+    {
+      "id": 89,
+      "type": "Reroute",
+      "pos": [
+        350,
+        803
+      ],
+      "size": [
+        75,
+        26
+      ],
+      "flags": {},
+      "order": 9,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "",
+          "type": "*",
+          "link": 176
+        }
+      ],
+      "outputs": [
+        {
+          "name": "",
+          "type": "IMAGE",
+          "links": [
+            167
+          ]
+        }
+      ],
+      "properties": {
+        "showOutputText": false,
+        "horizontal": false
+      }
+    },
+    {
+      "id": 4,
+      "type": "CheckpointLoaderSimple",
+      "pos": [
+        -1740,
+        236
+      ],
+      "size": [
+        315,
+        98
+      ],
+      "flags": {},
+      "order": 0,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "MODEL",
+          "type": "MODEL",
+          "links": [
+            170
+          ],
+          "slot_index": 0
+        },
+        {
+          "name": "CLIP",
+          "type": "CLIP",
+          "links": [
+            3,
+            5
+          ],
+          "slot_index": 1
+        },
+        {
+          "name": "VAE",
+          "type": "VAE",
+          "links": [],
+          "slot_index": 2
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CheckpointLoaderSimple"
+      },
+      "widgets_values": [
+        "revAnimated_v122.safetensors"
+      ],
+      "shape": 1
+    },
+    {
+      "id": 63,
+      "type": "SaveImage",
+      "pos": [
+        1315,
+        18
+      ],
+      "size": [
+        539.2050170898438,
+        617.2159423828125
+      ],
+      "flags": {},
+      "order": 13,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "images",
+          "type": "IMAGE",
+          "link": 115
+        }
+      ],
+      "title": "Normal",
+      "properties": {},
+      "widgets_values": [
+        "Normal"
+      ],
+      "shape": 1
+    },
+    {
+      "id": 67,
+      "type": "SaveImage",
+      "pos": [
+        2095,
+        22
+      ],
+      "size": [
+        539.2050170898438,
+        617.2159423828125
+      ],
+      "flags": {},
+      "order": 16,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "images",
+          "type": "IMAGE",
+          "link": 119
+        }
+      ],
+      "title": "Curvature",
+      "properties": {},
+      "widgets_values": [
+        "Curvature"
+      ],
+      "shape": 1
+    },
+    {
+      "id": 69,
+      "type": "SaveImage",
+      "pos": [
+        1560,
+        1290
+      ],
+      "size": [
+        539.2050170898438,
+        617.2159423828125
+      ],
+      "flags": {},
+      "order": 17,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "images",
+          "type": "IMAGE",
+          "link": 121
+        }
+      ],
+      "title": "Depth",
+      "properties": {},
+      "widgets_values": [
+        "Height"
+      ],
+      "shape": 1
+    },
+    {
+      "id": 91,
+      "type": "Model Patch Seamless (mtb)",
+      "pos": [
+        -1150,
+        -146
+      ],
+      "size": [
+        430.8000183105469,
+        78
+      ],
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 170
+        }
+      ],
+      "outputs": [
+        {
+          "name": "Original Model (passthrough)",
+          "type": "MODEL",
+          "links": null,
+          "shape": 3
+        },
+        {
+          "name": "Patched Model",
+          "type": "MODEL",
+          "links": [
+            169
+          ],
+          "shape": 3,
+          "slot_index": 1
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "Model Patch Seamless (mtb)"
+      },
+      "widgets_values": [
+        true
+      ],
+      "color": "#232",
+      "bgcolor": "#353"
+    },
+    {
+      "id": 93,
+      "type": "PreviewImage",
+      "pos": [
+        1115,
+        -597
+      ],
+      "size": [
+        451.3526306152344,
+        478.3444519042969
+      ],
+      "flags": {},
+      "order": 12,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "images",
+          "type": "IMAGE",
+          "link": 179
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "PreviewImage"
+      }
+    },
+    {
+      "id": 43,
+      "type": "VAELoader",
+      "pos": [
+        -598.2757622278747,
+        577.3595309932109
+      ],
+      "size": [
+        387.48089599609375,
+        70.60645294189453
+      ],
+      "flags": {},
+      "order": 1,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "VAE",
+          "type": "VAE",
+          "links": [
+            174
+          ],
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAELoader"
+      },
+      "widgets_values": [
+        "vae-ft-mse-840000-ema-pruned.safetensors"
+      ],
+      "shape": 1
+    },
+    {
+      "id": 97,
+      "type": "Image Tile Offset (mtb)",
+      "pos": [
+        617,
+        -598
+      ],
+      "size": [
+        315,
+        58
+      ],
+      "flags": {},
+      "order": 10,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "image",
+          "type": "IMAGE",
+          "link": 178
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            179
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "Image Tile Offset (mtb)"
+      },
+      "widgets_values": [
+        2
+      ],
+      "color": "#232",
+      "bgcolor": "#353"
+    },
+    {
+      "id": 96,
+      "type": "Vae Decode (mtb)",
+      "pos": [
+        -52,
+        40
+      ],
+      "size": [
+        315,
+        126
+      ],
+      "flags": {},
+      "order": 7,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "samples",
+          "type": "LATENT",
+          "link": 173
+        },
+        {
+          "name": "vae",
+          "type": "VAE",
+          "link": 174
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            175,
+            176,
+            178
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "Vae Decode (mtb)"
+      },
+      "widgets_values": [
+        true,
+        false,
+        512
+      ],
+      "color": "#232",
+      "bgcolor": "#353"
+    },
+    {
+      "id": 46,
+      "type": "SaveImage",
+      "pos": [
+        533,
+        25
+      ],
+      "size": [
+        539.2050170898438,
+        617.2159423828125
+      ],
+      "flags": {},
+      "order": 8,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "images",
+          "type": "IMAGE",
+          "link": 175
+        }
+      ],
+      "title": "Albedo",
+      "properties": {},
+      "widgets_values": [
+        "Albedo"
+      ],
+      "shape": 1
+    },
+    {
+      "id": 74,
+      "type": "EmptyLatentImage",
+      "pos": [
+        -1075.8749246009997,
+        480
+      ],
+      "size": [
+        315,
+        106
+      ],
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "outputs": [
+        {
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            132
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "EmptyLatentImage"
+      },
+      "widgets_values": [
+        768,
+        768,
+        1
+      ],
+      "color": "#323",
+      "bgcolor": "#535",
+      "shape": 1
+    },
+    {
+      "id": 62,
+      "type": "Deep Bump (mtb)",
+      "pos": [
+        727,
+        801
+      ],
+      "size": [
+        315,
+        130
+      ],
+      "flags": {},
+      "order": 11,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "image",
+          "type": "IMAGE",
+          "link": 167
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            115,
+            118,
+            122
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "Deep Bump (mtb)"
+      },
+      "widgets_values": [
+        "Color to Normals",
+        "SMALL",
+        "SMALLEST",
+        true
+      ],
+      "color": "#232",
+      "bgcolor": "#353",
+      "shape": 1
+    },
+    {
+      "id": 66,
+      "type": "Deep Bump (mtb)",
+      "pos": [
+        1626,
+        808
+      ],
+      "size": [
+        315,
+        130
+      ],
+      "flags": {},
+      "order": 14,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "image",
+          "type": "IMAGE",
+          "link": 118
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            119
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "Deep Bump (mtb)"
+      },
+      "widgets_values": [
+        "Normals to Curvature",
+        "SMALL",
+        "SMALLEST",
+        true
+      ],
+      "color": "#232",
+      "bgcolor": "#353",
+      "shape": 1
+    },
+    {
+      "id": 68,
+      "type": "Deep Bump (mtb)",
+      "pos": [
+        1185,
+        1288
+      ],
+      "size": [
+        315,
+        130
+      ],
+      "flags": {},
+      "order": 15,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "image",
+          "type": "IMAGE",
+          "link": 122
+        }
+      ],
+      "outputs": [
+        {
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            121
+          ],
+          "shape": 3,
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "Deep Bump (mtb)"
+      },
+      "widgets_values": [
+        "Normals to Height",
+        "SMALL",
+        "SMALLEST",
+        true
+      ],
+      "color": "#232",
+      "bgcolor": "#353",
+      "shape": 1
+    },
+    {
+      "id": 3,
+      "type": "KSampler",
+      "pos": [
+        -518.2757622278748,
+        47.359530993211024
+      ],
+      "size": [
+        315,
+        474
+      ],
+      "flags": {},
+      "order": 6,
+      "mode": 0,
+      "inputs": [
+        {
+          "name": "model",
+          "type": "MODEL",
+          "link": 169
+        },
+        {
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 4
+        },
+        {
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 6
+        },
+        {
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 132
+        }
+      ],
+      "outputs": [
+        {
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            173
+          ],
+          "slot_index": 0
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "KSampler"
+      },
+      "widgets_values": [
+        1001,
+        "fixed",
+        28,
+        8,
+        "dpmpp_2m",
+        "normal",
+        1
+      ],
+      "color": "#222",
+      "bgcolor": "#000",
+      "shape": 1
+    }
+  ],
+  "links": [
+    [
+      3,
+      4,
+      1,
+      6,
+      0,
+      "CLIP"
+    ],
+    [
+      4,
+      6,
+      0,
+      3,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      5,
+      4,
+      1,
+      7,
+      0,
+      "CLIP"
+    ],
+    [
+      6,
+      7,
+      0,
+      3,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      115,
+      62,
+      0,
+      63,
+      0,
+      "IMAGE"
+    ],
+    [
+      118,
+      62,
+      0,
+      66,
+      0,
+      "IMAGE"
+    ],
+    [
+      119,
+      66,
+      0,
+      67,
+      0,
+      "IMAGE"
+    ],
+    [
+      121,
+      68,
+      0,
+      69,
+      0,
+      "IMAGE"
+    ],
+    [
+      122,
+      62,
+      0,
+      68,
+      0,
+      "IMAGE"
+    ],
+    [
+      132,
+      74,
+      0,
+      3,
+      3,
+      "LATENT"
+    ],
+    [
+      158,
+      6,
+      0,
+      86,
+      0,
+      "*"
+    ],
+    [
+      167,
+      89,
+      0,
+      62,
+      0,
+      "IMAGE"
+    ],
+    [
+      169,
+      91,
+      1,
+      3,
+      0,
+      "MODEL"
+    ],
+    [
+      170,
+      4,
+      0,
+      91,
+      0,
+      "MODEL"
+    ],
+    [
+      173,
+      3,
+      0,
+      96,
+      0,
+      "LATENT"
+    ],
+    [
+      174,
+      43,
+      0,
+      96,
+      1,
+      "VAE"
+    ],
+    [
+      175,
+      96,
+      0,
+      46,
+      0,
+      "IMAGE"
+    ],
+    [
+      176,
+      96,
+      0,
+      89,
+      0,
+      "*"
+    ],
+    [
+      178,
+      96,
+      0,
+      97,
+      0,
+      "IMAGE"
+    ],
+    [
+      179,
+      97,
+      0,
+      93,
+      0,
+      "IMAGE"
+    ]
+  ],
+  "groups": [
+    {
+      "title": "Seamless Diffusion",
+      "bounding": [
+        -1752,
+        -392,
+        1658,
+        1102
+      ],
+      "color": "#3f789e",
+      "font_size": 76,
+      "locked": false
+    },
+    {
+      "title": "Seamless Check",
+      "bounding": [
+        421,
+        -795,
+        1374,
+        763
+      ],
+      "color": "#3f789e",
+      "font_size": 76,
+      "locked": false
+    }
+  ],
+  "config": {},
+  "extra": {},
+  "version": 0.4
+}

html/js/foldable.js

@@ -0,0 +1,20 @@
+/**
+ * File: foldable.js
+ * Project: comfy_mtb
+ * Author: Mel Massadian
+ *
+ * Copyright (c) 2023 Mel Massadian
+ *
+ */
+
+function toggleFoldable(elementId, symbolId) {
+  const content = document.getElementById(elementId)
+  const symbol = document.getElementById(symbolId)
+  if (content.style.display === 'none' || content.style.display === '') {
+    content.style.display = 'flex'
+    symbol.innerHTML = '▽' // Down arrow
+  } else {
+    content.style.display = 'none'
+    symbol.innerHTML = '▷' // Right arrow
+  }
+}

html/js/saveTableData.js

@@ -0,0 +1,54 @@
+/**
+ * File: saveTableData.js
+ * Project: comfy_mtb
+ * Author: Mel Massadian
+ *
+ * Copyright (c) 2023 Mel Massadian
+ *
+ */
+
+function saveTableData(identifier) {
+  const table = document.querySelector(
+    `#style-editor table[data-id='${identifier}']`
+  )
+
+  let currentData = []
+  const rows = table.querySelectorAll('tr')
+  const filename = table.getAttribute('data-id')
+
+  rows.forEach((row, rowIndex) => {
+    const rowData = []
+    const cells =
+      rowIndex === 0
+        ? row.querySelectorAll('th')
+        : row.querySelectorAll('td input, td textarea')
+
+    cells.forEach((cell) => {
+      rowData.push(rowIndex === 0 ? cell.textContent : cell.value)
+    })
+
+    currentData.push(rowData)
+  })
+
+  let tablesData = {}
+  tablesData[filename] = currentData
+
+  console.debug('Sending styles to manage endpoint:', tablesData)
+  fetch('/mtb/actions', {
+    method: 'POST',
+    headers: {
+      'Content-Type': 'application/json',
+    },
+    body: JSON.stringify({
+      name: 'saveStyle',
+      args: tablesData,
+    }),
+  })
+    .then((response) => response.json())
+    .then((data) => {
+      console.debug('Success:', data)
+    })
+    .catch((error) => {
+      console.error('Error:', error)
+    })
+}

html/js/splitPane.js

@@ -0,0 +1,34 @@
+/**
+ * File: splitPane.js
+ * Project: comfy_mtb
+ * Author: Mel Massadian
+ *
+ * Copyright (c) 2023 Mel Massadian
+ *
+ */
+
+function initSplitPane(vertical) {
+  let resizer = document.getElementById('resizer')
+  let left = document.getElementById('leftPane')
+  let right = document.getElementById('rightPane')
+  resizer.addEventListener('mousedown', function (e) {
+    document.addEventListener('mousemove', onMouseMove)
+    document.addEventListener('mouseup', function () {
+      document.removeEventListener('mousemove', onMouseMove)
+    })
+  })
+
+  const onMouseMove = (e) => {
+    if (vertical) {
+      let leftWidth = e.clientX
+      let rightWidth = window.innerWidth - e.clientX
+      left.style.width = leftWidth + 'px'
+      right.style.width = rightWidth + 'px'
+    } else {
+      let topHeight = e.clientY
+      let bottomHeight = window.innerHeight - e.clientY
+      left.style.height = topHeight + 'px'
+      right.style.height = bottomHeight + 'px'
+    }
+  }
+}

html/js/tabSwitch.js

@@ -0,0 +1,22 @@
+/**
+ * File: tabSwitch.js
+ * Project: comfy_mtb
+ * Author: Mel Massadian
+ *
+ * Copyright (c) 2023 Mel Massadian
+ *
+ */
+
+function openTab(evt, tabName) {
+  var i, tabcontent, tablinks
+  tabcontent = document.getElementsByClassName('tabcontent')
+  for (i = 0; i < tabcontent.length; i++) {
+    tabcontent[i].style.display = 'none'
+  }
+  tablinks = document.getElementsByClassName('tablinks')
+  for (i = 0; i < tablinks.length; i++) {
+    tablinks[i].className = tablinks[i].className.replace(' active', '')
+  }
+  document.getElementById(tabName).style.display = 'block'
+  evt.currentTarget.className += ' active'
+}

html/style.css

@@ -18,7 +18,7 @@ a {
 }
 
 table {
-
+    width: 100%;
     border-collapse: collapse;
 }
 
@@ -119,7 +119,7 @@ main {
     justify-content: center;
     padding: 1em;
     margin: 0;
-    height: 80%;
+    /* height: 80%; */
 }
 
 .flex-container {
@@ -130,4 +130,99 @@ main {
 .menu {
     font-size: 3em;
     text-align: center;
-}
+}
+
+input, button, textarea {
+    background-color: rgba(0,0,0,0.5);
+    color: white;
+    border: none;
+}
+
+button:hover {
+    background-color: rgba(0,0,0,0.3);
+
+}
+button {
+    padding: 14px 16px;
+
+}
+/* -STYLES EDITOR */
+
+#style-editor {
+    display: flex;
+    flex-direction: column;
+    width:100%;
+    
+}
+
+#style-editor > table {
+    /* background-color: red; */
+    width:100%;
+}
+#style-editor input, #style-editor textarea {
+    /* background-color: blue; */
+    width:100%;
+}
+
+#style-editor td{
+    width: 33.33%;
+}
+
+/* -TABS */
+
+.tab {
+    overflow: hidden;
+    width: 100%;
+    display: flex;
+    flex-direction: row;
+
+}
+
+.tab-container{
+    width: 100%;
+    display: flex;
+    flex-direction: column;
+}
+
+.tab button {
+    background-color: transparent;
+    color:white;
+    float: left;
+    border: none;
+    outline: none;
+    cursor: pointer;
+    padding: 14px 16px;
+    transition: 0.3s;
+    width:100%;
+    font-size: 1.5em;
+}
+
+.tab button.active {
+    background-color: #2e2e2e;
+}
+
+.tabcontent {
+    display: none;
+}
+
+.tabcontent.active {
+    display: block;
+}
+
+
+
+.foldable-title {
+    cursor: pointer;
+    font-weight: bold;
+    user-select: none;
+}
+
+.foldable-symbol {
+    margin-right: 10px;
+}
+
+.foldable-content {
+    display: none;
+    flex-direction: column;
+    margin-left: 20px;
+}

install.py

@@ -1,37 +1,34 @@
-import requests
-import os
-import ast
 import argparse
-import sys
-import subprocess
-from importlib import import_module
+import ast
+import os
 import platform
-from pathlib import Path
-import sys
+import shlex
 import stat
-import threading
-import signal
-from contextlib import suppress
-from queue import Queue, Empty
+import subprocess
+import sys
 from contextlib import contextmanager
+from importlib import import_module
+from pathlib import Path
 
+import requests
+
+# region constants
 here = Path(__file__).parent
-executable = sys.executable
+executable = Path(sys.executable)
 
 # - detect mode
 mode = None
 if os.environ.get("COLAB_GPU"):
     mode = "colab"
-elif "python_embeded" in executable:
+elif "python_embeded" in str(executable):
     mode = "embeded"
-elif ".venv" in executable:
+elif ".venv" in str(executable):
     mode = "venv"
 
 
 if mode is None:
     mode = "unknown"
 
-# - Constants
 repo_url = "https://github.com/melmass/comfy_mtb.git"
 repo_owner = "melmass"
 repo_name = "comfy_mtb"
@@ -40,6 +37,17 @@ short_platform = {
     "linux": "linux_x86_64",
 }
 current_platform = platform.system().lower()
+pip_map = {
+    "onnxruntime-gpu": "onnxruntime",
+    "opencv-contrib": "cv2",
+    "tb-nightly": "tensorboard",
+    "protobuf": "google.protobuf",
+    "qrcode[pil]": "qrcode",
+    "requirements-parser": "requirements"
+    # Add more mappings as needed
+}
+
+# endregion
 
 # region ansi
 # ANSI escape sequences for text styling
@@ -136,12 +144,6 @@ def print_formatted(text, *formats, color=None, background=None, **kwargs):
 
 
 # region utils
-def enqueue_output(out, queue):
-    for char in iter(lambda: out.read(1), b""):
-        queue.put(char)
-    out.close()
-
-
 def run_command(cmd, ignored_lines_start=None):
     if ignored_lines_start is None:
         ignored_lines_start = []
@@ -149,113 +151,49 @@ def run_command(cmd, ignored_lines_start=None):
     if isinstance(cmd, str):
         shell_cmd = cmd
     elif isinstance(cmd, list):
-        shell_cmd = ""
-        for arg in cmd:
-            if isinstance(arg, Path):
-                arg = arg.as_posix()
-            shell_cmd += f"{arg} "
+        shell_cmd = " ".join(
+            arg.as_posix() if isinstance(arg, Path) else shlex.quote(str(arg))
+            for arg in cmd
+        )
     else:
         raise ValueError(
             "Invalid 'cmd' argument. It must be a string or a list of arguments."
         )
 
-    process = subprocess.Popen(
+    try:
+        _run_command(shell_cmd, ignored_lines_start)
+    except subprocess.CalledProcessError as e:
+        print(f"Command failed with return code: {e.returncode}", file=sys.stderr)
+        print(e.stderr.strip(), file=sys.stderr)
+
+    except KeyboardInterrupt:
+        print("Command execution interrupted.")
+
+
+def _run_command(shell_cmd, ignored_lines_start):
+    print_formatted(f"Running {shell_cmd}", "bold")
+    result = subprocess.run(
         shell_cmd,
         stdout=subprocess.PIPE,
         stderr=subprocess.PIPE,
-        universal_newlines=True,
+        text=True,
         shell=True,
+        check=True,
     )
 
-    # Create separate threads to read standard output and standard error streams
-    stdout_queue = Queue()
-    stderr_queue = Queue()
-    stdout_thread = threading.Thread(
-        target=enqueue_output, args=(process.stdout, stdout_queue)
-    )
-    stderr_thread = threading.Thread(
-        target=enqueue_output, args=(process.stderr, stderr_queue)
-    )
-    stdout_thread.daemon = True
-    stderr_thread.daemon = True
-    stdout_thread.start()
-    stderr_thread.start()
+    stdout_lines = result.stdout.strip().split("\n")
+    stderr_lines = result.stderr.strip().split("\n")
 
-    interrupted = False
+    # Print stdout, skipping ignored lines
+    for line in stdout_lines:
+        if not any(line.startswith(ign) for ign in ignored_lines_start):
+            print(line)
 
-    def signal_handler(signum, frame):
-        nonlocal interrupted
-        interrupted = True
-        print("Command execution interrupted.")
+    # Print stderr
+    for line in stderr_lines:
+        print(line, file=sys.stderr)
 
-    # Register the signal handler for keyboard interrupts (SIGINT)
-    signal.signal(signal.SIGINT, signal_handler)
-
-    stdout_buffer = ""
-    stderr_buffer = ""
-
-    # Process output from both streams until the process completes or interrupted
-    while not interrupted and (
-        process.poll() is None or not stdout_queue.empty() or not stderr_queue.empty()
-    ):
-        with suppress(Empty):
-            stdout_char = stdout_queue.get_nowait()
-            stdout_buffer += stdout_char
-            if stdout_char == "\n":
-                if not any(
-                    stdout_buffer.startswith(ign) for ign in ignored_lines_start
-                ):
-                    print(stdout_buffer.strip())
-                stdout_buffer = ""
-        with suppress(Empty):
-            stderr_char = stderr_queue.get_nowait()
-            stderr_buffer += stderr_char
-            if stderr_char == "\n":
-                print(stderr_buffer.strip())
-                stderr_buffer = ""
-
-    # Print any remaining content in buffers
-    if stdout_buffer and not any(
-        stdout_buffer.startswith(ign) for ign in ignored_lines_start
-    ):
-        print(stdout_buffer.strip())
-    if stderr_buffer:
-        print(stderr_buffer.strip())
-
-    return_code = process.returncode
-
-    if return_code == 0 and not interrupted:
-        print("Command executed successfully!")
-    else:
-        if not interrupted:
-            print(f"Command failed with return code: {return_code}")
-
-
-# endregion
-
-try:
-    import requirements
-except ImportError:
-    print_formatted("Installing requirements-parser...", "italic", color="yellow")
-    run_command([sys.executable, "-m", "pip", "install", "requirements-parser"])
-    import requirements
-
-    print_formatted("Done.", "italic", color="green")
-
-try:
-    from tqdm import tqdm
-except ImportError:
-    print_formatted("Installing tqdm...", "italic", color="yellow")
-    run_command([sys.executable, "-m", "pip", "install", "--upgrade", "tqdm"])
-    from tqdm import tqdm
-
-pip_map = {
-    "onnxruntime-gpu": "onnxruntime",
-    "opencv-contrib": "cv2",
-    "tb-nightly": "tensorboard",
-    "protobuf": "google.protobuf",
-    # Add more mappings as needed
-}
+    print("Command executed successfully!")
 
 
 def is_pipe():
@@ -330,24 +268,6 @@ def download_file(url, file_name):
                 progress_bar.update(len(chunk))
 
 
-def get_requirements(path: Path):
-    with open(path.resolve(), "r") as requirements_file:
-        requirements_txt = requirements_file.read()
-
-    try:
-        parsed_requirements = requirements.parse(requirements_txt)
-    except AttributeError:
-        print_formatted(
-            f"Failed to parse {path}. Please make sure the file is correctly formatted.",
-            "bold",
-            color="red",
-        )
-
-        return
-
-    return parsed_requirements
-
-
 def try_import(requirement):
     dependency = requirement.name.strip()
     import_name = pip_map.get(dependency, dependency)
@@ -390,7 +310,7 @@ def import_or_install(requirement, dry=False):
             )
         else:
             try:
-                run_command([sys.executable, "-m", "pip", "install", pip_install_name])
+                run_command([executable, "-m", "pip", "install", pip_install_name])
                 print_formatted(
                     f"Package {pip_install_name} installed successfully using pip package name  (import name: '{import_name}')",
                     "bold",
@@ -427,29 +347,29 @@ def get_github_assets(tag=None):
     return tag_data, tag_name
 
 
-# Install dependencies from requirements.txt
-def install_dependencies(dry=False):
-    parsed_requirements = get_requirements(here / "reqs.txt")
-    if not parsed_requirements:
-        return
-    print_formatted(
-        "Installing dependencies from reqs.txt...", "italic", color="yellow"
-    )
-
-    for requirement in parsed_requirements:
-        import_or_install(requirement, dry=dry)
+# endregion
 
 
-if __name__ == "__main__":
-    full = False
+try:
+    from tqdm import tqdm
+except ImportError:
+    print_formatted("Installing tqdm...", "italic", color="yellow")
+    run_command([executable, "-m", "pip", "install", "--upgrade", "tqdm"])
+    from tqdm import tqdm
+
+
+def main():
     if len(sys.argv) == 1:
         print_formatted(
-            "No arguments provided, doing a full install/update...",
-            "italic",
-            color="yellow",
+            "mtb doesn't need an install script anymore.", "italic", color="yellow"
         )
-
-        full = True
+        return
+    if all(arg not in ("-p", "--path") for arg in sys.argv):
+        print(
+            "This script is only used for and edge case of remote installs on some cloud providers, unrecognized arguments:",
+            sys.argv[1:],
+        )
+        return
 
     # Parse command-line arguments
     parser = argparse.ArgumentParser(description="Comfy_mtb install script")
@@ -459,29 +379,11 @@ if __name__ == "__main__":
         type=str,
         help="Path to clone the repository to (i.e the absolute path to ComfyUI/custom_nodes)",
     )
-    parser.add_argument(
-        "--wheels", "-w", action="store_true", help="Install wheel dependencies"
-    )
-    parser.add_argument(
-        "--requirements", "-r", action="store_true", help="Install requirements.txt"
-    )
-    parser.add_argument(
-        "--dry",
-        action="store_true",
-        help="Print what will happen without doing it (still making requests to the GH Api)",
-    )
 
-    # - keep
-    # parser.add_argument(
-    #     "--version",
-    #     default=get_local_version(),
-    #     help="Version to check against the GitHub API",
-    # )
     print_formatted("mtb install", "bold", color="yellow")
 
     args = parser.parse_args()
 
-    # wheels_directory = here / "wheels"
     print_formatted(f"Detected environment: {apply_color(mode,'cyan')}")
 
     if args.path:
@@ -502,131 +404,18 @@ if __name__ == "__main__":
                     f"Directory {repo_dir} already exists, we will update it..."
                 )
                 run_command(["git", "pull", "-C", repo_dir])
-        # os.chdir(clone_dir)
         here = clone_dir
         full = True
 
-    # Install dependencies from requirements.txt
-    # if args.requirements or mode == "venv":
-
-    # if (not args.wheels and mode not in ["colab", "embeded"]) and not full:
-    #     print_formatted(
-    #         "Skipping wheel installation. Use --wheels to install wheel dependencies. (only needed for Comfy embed)",
-    #         "italic",
-    #         color="yellow",
-    #     )
-
-    #     install_dependencies(dry=args.dry)
-    #     sys.exit()
-
-    # if mode in ["colab", "embeded"]:
-    #     print_formatted(
-    #         f"Downloading and installing release wheels since we are in a Comfy {apply_color(mode,'cyan')} environment",
-    #         "italic",
-    #         color="yellow",
-    #     )
-    # if full:
-    #     print_formatted(
-    #         f"Downloading and installing release wheels since no arguments where provided",
-    #         "italic",
-    #         color="yellow",
-    #     )
-
     print_formatted("Checking environment...", "italic", color="yellow")
     missing_deps = []
-    if parsed_requirements := get_requirements(here / "reqs.txt"):
-        for requirement in parsed_requirements:
-            installed, pip_name, pip_spec, import_name = try_import(requirement)
-            if not installed:
-                missing_deps.append(pip_name.split("-")[0])
+    install_cmd = [executable, "-m", "pip", "install", "-r", "requirements.txt"]
+    run_command(install_cmd)
 
-    if not missing_deps:
-        print_formatted(
-            "All requirements are already installed. Enjoy 🚀",
-            "italic",
-            color="green",
-        )
-        sys.exit()
+    print_formatted(
+        "✅ Successfully installed all dependencies.", "italic", color="green"
+    )
 
-    # # - Get the tag version from the GitHub API
-    # tag_data, tag_name = get_github_assets(tag=None)
 
-    # # - keep
-    # version = args.version
-    # # Compare the local and tag versions
-    # if version and tag_name:
-    #     if re.match(r"v?(\d+(\.\d+)+)", version) and re.match(
-    #         r"v?(\d+(\.\d+)+)", tag_name
-    #     ):
-    #         version_parts = [int(part) for part in version.lstrip("v").split(".")]
-    #         tag_version_parts = [int(part) for part in tag_name.lstrip("v").split(".")]
-
-    #         if version_parts > tag_version_parts:
-    #             print_formatted(
-    #                 f"Local version ({version}) is greater than the release version ({tag_name}).",
-    #                 "bold",
-    #                 "yellow",
-    #             )
-    #             sys.exit()
-
-    # matching_assets = [
-    #     asset
-    #     for asset in tag_data["assets"]
-    #     if asset["name"].endswith(".whl")
-    #     and (
-    #         "any" in asset["name"] or short_platform[current_platform] in asset["name"]
-    #     )
-    # ]
-    # if not matching_assets:
-    #     print_formatted(
-    #         f"Unsupported operating system: {current_platform}", color="yellow"
-    #     )
-    # wheel_order_asset = next(
-    #     (asset for asset in tag_data["assets"] if asset["name"] == "wheel_order.txt"),
-    #     None,
-    # )
-    # if wheel_order_asset is not None:
-    #     print_formatted(
-    #         "⚙️ Sorting the release wheels using wheels order", "italic", color="yellow"
-    #     )
-    #     response = requests.get(wheel_order_asset["browser_download_url"])
-    #     if response.status_code == 200:
-    #         wheel_order = [line.strip() for line in response.text.splitlines()]
-
-    #         def get_order_index(val):
-    #             try:
-    #                 return wheel_order.index(val)
-    #             except ValueError:
-    #                 return len(wheel_order)
-
-    #         matching_assets = sorted(
-    #             matching_assets,
-    #             key=lambda x: get_order_index(x["name"].split("-")[0]),
-    #         )
-    #     else:
-    #         print("Failed to fetch wheel_order.txt. Status code:", response.status_code)
-
-    # missing_deps_urls = []
-    # for whl_file in matching_assets:
-    #     # check if installed
-    #     missing_deps_urls.append(whl_file["browser_download_url"])
-
-    install_cmd = [sys.executable, "-m", "pip", "install"]
-
-    # - Install all deps
-    if not args.dry:
-        if platform.system() == "Windows":
-            wheel_cmd = install_cmd + ["-r", (here / "reqs_windows.txt")]
-        else:
-            wheel_cmd = install_cmd + ["-r", (here / "reqs.txt")]
-
-        run_command(wheel_cmd)
-        print_formatted(
-            "✅ Successfully installed all dependencies.", "italic", color="green"
-        )
-    else:
-        print_formatted(
-            f"Would have run the following command:\n\t{apply_color(' '.join(install_cmd),'cyan')}",
-            "italic",
-            color="yellow",
-        )
+if __name__ == "__main__":
+    main()

log.py

@@ -1,6 +1,6 @@
 import logging
-import re
 import os
+import re
 
 base_log_level = logging.DEBUG if os.environ.get("MTB_DEBUG") else logging.INFO
 
@@ -75,5 +75,7 @@ def cyan_text(text):
 
 
 def get_label(label):
+    if label.startswith("MTB_"):
+        label = label[4:]
     words = re.findall(r"(?:^|[A-Z])[a-z]*", label)
     return " ".join(words).strip()

node_list.json

@@ -1,46 +1,61 @@
-{
-    "Animation Builder (mtb)": "Convenient way to manage basic animation maths at the core of many of my workflows",
-    "Any To String (mtb)": "Tries to take any input and convert it to a string",
-    "Bbox (mtb)": "The bounding box (BBOX) custom type used by other nodes",
-    "Bbox From Mask (mtb)": "From a mask extract the bounding box",
-    "Blur (mtb)": "Blur an image using a Gaussian filter.",
-    "Color Correct (mtb)": "Various color correction methods",
-    "Colored Image (mtb)": "Constant color image of given size",
-    "Concat Images (mtb)": "Add images to batch",
-    "Crop (mtb)": "Crops an image and an optional mask to a given bounding box\n\n    The bounding box can be given as a tuple of (x, y, width, height) or as a BBOX type\n    The BBOX input takes precedence over the tuple input\n    ",
-    "Debug (mtb)": "Experimental node to debug any Comfy values, support for more types and widgets is planned",
-    "Deep Bump (mtb)": "Normal & height maps generation from single pictures",
-    "Export With Ffmpeg (mtb)": "Export with FFmpeg (Experimental)",
-    "Face Swap (mtb)": "Face swap using deepinsight/insightface models",
-    "Film Interpolation (mtb)": "Google Research FILM frame interpolation for large motion",
-    "Fit Number (mtb)": "Fit the input float using a source and target range",
-    "Float To Number (mtb)": "Node addon for the WAS Suite. Converts a \"comfy\" FLOAT to a NUMBER.",
-    "Get Batch From History (mtb)": "Very experimental node to load images from the history of the server.\n\n    Queue items without output are ignored in the count.",
-    "Image Compare (mtb)": "Compare two images and return a difference image",
-    "Image Premultiply (mtb)": "Premultiply image with mask",
-    "Image Remove Background Rembg (mtb)": "Removes the background from the input using Rembg.",
-    "Image Resize Factor (mtb)": "Extracted mostly from WAS Node Suite, with a few edits (most notably multiple image support) and less features.",
-    "Int To Bool (mtb)": "Basic int to bool conversion",
-    "Int To Number (mtb)": "Node addon for the WAS Suite. Converts a \"comfy\" INT to a NUMBER.",
-    "Latent Lerp (mtb)": "Linear interpolation (blend) between two latent vectors",
-    "Load Face Analysis Model (mtb)": "Loads a face analysis model",
-    "Load Face Enhance Model (mtb)": "Loads a GFPGan or RestoreFormer model for face enhancement.",
-    "Load Face Swap Model (mtb)": "Loads a faceswap model",
-    "Load Film Model (mtb)": "Loads a FILM model",
-    "Load Image From Url (mtb)": "Load an image from the given URL",
-    "Load Image Sequence (mtb)": "Load an image sequence from a folder. The current frame is used to determine which image to load.\n\n    Usually used in conjunction with the `Primitive` node set to increment to load a sequence of images from a folder.\n    Use -1 to load all matching frames as a batch.\n    ",
-    "Mask To Image (mtb)": "Converts a mask (alpha) to an RGB image with a color and background",
-    "Qr Code (mtb)": "Basic QR Code generator",
-    "Restore Face (mtb)": "Uses GFPGan to restore faces",
-    "Save Gif (mtb)": "Save the images from the batch as a GIF",
-    "Save Image Grid (mtb)": "Save all the images in the input batch as a grid of images.",
-    "Save Image Sequence (mtb)": "Save an image sequence to a folder. The current frame is used to determine which image to save.\n\n    This is merely a wrapper around the `save_images` function with formatting for the output folder and filename.\n    ",
-    "Save Tensors (mtb)": "Save torch tensors (image, mask or latent) to disk, useful to debug things outside comfy",
-    "Smart Step (mtb)": "Utils to control the steps start/stop of the KAdvancedSampler in percentage",
-    "String Replace (mtb)": "Basic string replacement",
-    "Styles Loader (mtb)": "Load csv files and populate a dropdown from the rows (\u00e0 la A111)",
-    "Text To Image (mtb)": "Utils to convert text to image using a font\n\n\n    The tool looks for any .ttf file in the Comfy folder hierarchy.\n    ",
-    "Transform Image (mtb)": "Save torch tensors (image, mask or latent) to disk, useful to debug things outside comfy\n\n\n    it return a tensor representing the transformed images with the same shape as the input tensor\n    ",
-    "Uncrop (mtb)": "Uncrops an image to a given bounding box\n\n    The bounding box can be given as a tuple of (x, y, width, height) or as a BBOX type\n    The BBOX input takes precedence over the tuple input",
-    "Unsplash Image (mtb)": "Unsplash Image given a keyword and a size"
-}
+{
+    "Animation Builder (mtb)": "Convenient way to manage basic animation maths at the core of many of my workflows",
+    "Any To String (mtb)": "Tries to take any input and convert it to a string",
+    "Batch Float (mtb)": "Generates a batch of float values with interpolation",
+    "Batch Float Assemble (mtb)": "Assembles mutiple batches of floats into a single stream (batch)",
+    "Batch Float Fill (mtb)": "Fills a batch float with a single value until it reaches the target length",
+    "Batch Make (mtb)": "Simply duplicates the input frame as a batch",
+    "Batch Merge (mtb)": "Merges multiple image batches with different frame counts",
+    "Batch Shake (mtb)": "Applies a shaking effect to batches of images.",
+    "Batch Shape (mtb)": "Generates a batch of 2D shapes with optional shading (experimental)",
+    "Batch Transform (mtb)": "Transform a batch of images using a batch of keyframes",
+    "Bbox (mtb)": "The bounding box (BBOX) custom type used by other nodes",
+    "Bbox From Mask (mtb)": "From a mask extract the bounding box",
+    "Blur (mtb)": "Blur an image using a Gaussian filter.",
+    "Color Correct (mtb)": "Various color correction methods",
+    "Colored Image (mtb)": "Constant color image of given size",
+    "Concat Images (mtb)": "Add images to batch",
+    "Crop (mtb)": "Crops an image and an optional mask to a given bounding box\n\n    The bounding box can be given as a tuple of (x, y, width, height) or as a BBOX type\n    The BBOX input takes precedence over the tuple input\n    ",
+    "Debug (mtb)": "Experimental node to debug any Comfy values, support for more types and widgets is planned",
+    "Deep Bump (mtb)": "Normal & height maps generation from single pictures",
+    "Export With Ffmpeg (mtb)": "Export with FFmpeg (Experimental)",
+    "Face Swap (mtb)": "Face swap using deepinsight/insightface models",
+    "Film Interpolation (mtb)": "Google Research FILM frame interpolation for large motion",
+    "Fit Number (mtb)": "Fit the input float using a source and target range",
+    "Float To Number (mtb)": "Node addon for the WAS Suite. Converts a \"comfy\" FLOAT to a NUMBER.",
+    "Get Batch From History (mtb)": "Very experimental node to load images from the history of the server.\n\n    Queue items without output are ignored in the count.",
+    "Image Compare (mtb)": "Compare two images and return a difference image",
+    "Image Premultiply (mtb)": "Premultiply image with mask",
+    "Image Remove Background Rembg (mtb)": "Removes the background from the input using Rembg.",
+    "Image Resize Factor (mtb)": "Extracted mostly from WAS Node Suite, with a few edits (most notably multiple image support) and less features.",
+    "Image Tile Offset (mtb)": "Mimics an old photoshop technique to check for seamless textures",
+    "Int To Bool (mtb)": "Basic int to bool conversion",
+    "Int To Number (mtb)": "Node addon for the WAS Suite. Converts a \"comfy\" INT to a NUMBER.",
+    "Interpolate Clip Sequential (mtb)": null,
+    "Latent Lerp (mtb)": "Linear interpolation (blend) between two latent vectors",
+    "Load Face Analysis Model (mtb)": "Loads a face analysis model",
+    "Load Face Enhance Model (mtb)": "Loads a GFPGan or RestoreFormer model for face enhancement.",
+    "Load Face Swap Model (mtb)": "Loads a faceswap model",
+    "Load Film Model (mtb)": "Loads a FILM model",
+    "Load Image From Url (mtb)": "Load an image from the given URL",
+    "Load Image Sequence (mtb)": "Load an image sequence from a folder. The current frame is used to determine which image to load.\n\n    Usually used in conjunction with the `Primitive` node set to increment to load a sequence of images from a folder.\n    Use -1 to load all matching frames as a batch.\n    ",
+    "Mask To Image (mtb)": "Converts a mask (alpha) to an RGB image with a color and background",
+    "Math Expression (mtb)": "Node to evaluate a simple math expression string",
+    "Model Patch Seamless (mtb)": "Uses the stable diffusion 'hack' to infer seamless images by setting the model layers padding mode to circular (experimental)",
+    "Qr Code (mtb)": "Basic QR Code generator",
+    "Restore Face (mtb)": "Uses GFPGan to restore faces",
+    "Save Gif (mtb)": "Save the images from the batch as a GIF",
+    "Save Image Grid (mtb)": "Save all the images in the input batch as a grid of images.",
+    "Save Image Sequence (mtb)": "Save an image sequence to a folder. The current frame is used to determine which image to save.\n\n    This is merely a wrapper around the `save_images` function with formatting for the output folder and filename.\n    ",
+    "Save Tensors (mtb)": "Save torch tensors (image, mask or latent) to disk, useful to debug things outside comfy",
+    "Sharpen (mtb)": "Sharpens an image using a Gaussian kernel.",
+    "Smart Step (mtb)": "Utils to control the steps start/stop of the KAdvancedSampler in percentage",
+    "Stack Images (mtb)": "Stack the input images horizontally or vertically",
+    "String Replace (mtb)": "Basic string replacement",
+    "Styles Loader (mtb)": "Load csv files and populate a dropdown from the rows (\u00e0 la A111)",
+    "Text To Image (mtb)": "Utils to convert text to image using a font\n\n\n    The tool looks for any .ttf file in the Comfy folder hierarchy.\n    ",
+    "Transform Image (mtb)": "Save torch tensors (image, mask or latent) to disk, useful to debug things outside comfy\n\n\n    it return a tensor representing the transformed images with the same shape as the input tensor\n    ",
+    "Uncrop (mtb)": "Uncrops an image to a given bounding box\n\n    The bounding box can be given as a tuple of (x, y, width, height) or as a BBOX type\n    The BBOX input takes precedence over the tuple input",
+    "Unsplash Image (mtb)": "Unsplash Image given a keyword and a size",
+    "Vae Decode (mtb)": "Wrapper for the 2 core decoders but also adding the sd seamless hack, taken from: FlyingFireCo/tiled_ksampler"
+}

nodes/batch.py

@@ -0,0 +1,676 @@
+from io import BytesIO
+
+import cv2
+import torchaudio
+import numpy as np
+import torch
+from PIL import Image
+
+from ..log import log
+from ..utils import apply_easing, pil2tensor
+from .transform import TransformImage
+
+try:
+    import librosa
+except ImportError:
+    log.warning("librosa not installed. Batch Audio features will not be available.")
+
+
+def hex_to_rgb(hex_color, bgr=False):
+    hex_color = hex_color.lstrip("#")
+    if bgr:
+        return tuple(int(hex_color[i : i + 2], 16) for i in (4, 2, 0))
+
+    return tuple(int(hex_color[i : i + 2], 16) for i in (0, 2, 4))
+
+
+class BatchMake:
+    """Simply duplicates the input frame as a batch"""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "image": ("IMAGE",),
+                "count": ("INT", {"default": 1}),
+            }
+        }
+
+    RETURN_TYPES = ("IMAGE",)
+    FUNCTION = "generate_batch"
+    CATEGORY = "mtb/batch"
+
+    def generate_batch(self, image: torch.Tensor, count):
+        if len(image.shape) == 3:
+            image = image.unsqueeze(0)
+
+        return (image.repeat(count, 1, 1, 1),)
+
+
+class BatchShape:
+    """Generates a batch of 2D shapes with optional shading (experimental)"""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "count": ("INT", {"default": 1}),
+                "shape": (
+                    ["Box", "Circle", "Diamond"],
+                    {"default": "Box"},
+                ),
+                "image_width": ("INT", {"default": 512}),
+                "image_height": ("INT", {"default": 512}),
+                "shape_size": ("INT", {"default": 100}),
+                "color": ("COLOR", {"default": "#ffffff"}),
+                "bg_color": ("COLOR", {"default": "#000000"}),
+                "shade_color": ("COLOR", {"default": "#000000"}),
+                "shadex": ("FLOAT", {"default": 0.0}),
+                "shadey": ("FLOAT", {"default": 0.0}),
+            },
+        }
+
+    RETURN_TYPES = ("IMAGE",)
+    FUNCTION = "generate_shapes"
+    CATEGORY = "mtb/batch"
+
+    def generate_shapes(
+        self,
+        count,
+        shape,
+        image_width,
+        image_height,
+        shape_size,
+        color,
+        bg_color,
+        shade_color,
+        shadex,
+        shadey,
+    ):
+        print(f"COLOR: {color}")
+        print(f"BG_COLOR: {bg_color}")
+        print(f"SHADE_COLOR: {shade_color}")
+
+        # Parse color input to BGR tuple for OpenCV
+        color = hex_to_rgb(color)
+        bg_color = hex_to_rgb(bg_color)
+        shade_color = hex_to_rgb(shade_color)
+        res = []
+        for x in range(count):
+            # Initialize an image canvas
+            canvas = np.full((image_height, image_width, 3), bg_color, dtype=np.uint8)
+            mask = np.zeros((image_height, image_width), dtype=np.uint8)
+
+            # Compute the center point of the shape
+            center = (image_width // 2, image_height // 2)
+
+            if shape == "Box":
+                half_size = shape_size // 2
+                top_left = (center[0] - half_size, center[1] - half_size)
+                bottom_right = (center[0] + half_size, center[1] + half_size)
+                cv2.rectangle(mask, top_left, bottom_right, 255, -1)
+            elif shape == "Circle":
+                cv2.circle(mask, center, shape_size // 2, 255, -1)
+            elif shape == "Diamond":
+                pts = np.array(
+                    [
+                        [center[0], center[1] - shape_size // 2],
+                        [center[0] + shape_size // 2, center[1]],
+                        [center[0], center[1] + shape_size // 2],
+                        [center[0] - shape_size // 2, center[1]],
+                    ]
+                )
+                cv2.fillPoly(mask, [pts], 255)
+
+            # Color the shape
+            canvas[mask == 255] = color
+
+            # Apply shading effects to a separate shading canvas
+            shading = np.zeros_like(canvas, dtype=np.float32)
+            shading[:, :, 0] = shadex * np.linspace(0, 1, image_width)
+            shading[:, :, 1] = shadey * np.linspace(0, 1, image_height).reshape(-1, 1)
+            shading_canvas = cv2.addWeighted(
+                canvas.astype(np.float32), 1, shading, 1, 0
+            ).astype(np.uint8)
+
+            # Apply shading only to the shape area using the mask
+            canvas[mask == 255] = shading_canvas[mask == 255]
+            res.append(canvas)
+
+        return (pil2tensor(res),)
+
+
+class BatchFloatFill:
+    """Fills a batch float with a single value until it reaches the target length"""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "floats": ("FLOATS",),
+                "direction": (["head", "tail"], {"default": "tail"}),
+                "value": ("FLOAT", {"default": 0.0}),
+                "count": ("INT", {"default": 1}),
+            }
+        }
+
+    FUNCTION = "fill_floats"
+    RETURN_TYPES = ("FLOATS",)
+    CATEGORY = "mtb/batch"
+
+    def fill_floats(self, floats, direction, value, count):
+        size = len(floats)
+        if size > count:
+            raise ValueError(f"Size ({size}) is less then target count ({count})")
+
+        rem = count - size
+        if direction == "tail":
+            floats = floats + [value] * rem
+        else:
+            floats = [value] * rem + floats
+        return (floats,)
+
+
+class BatchFloatAssemble:
+    """Assembles mutiple batches of floats into a single stream (batch)"""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {"required": {"reverse": ("BOOLEAN", {"default": False})}}
+
+    FUNCTION = "assemble_floats"
+    RETURN_TYPES = ("FLOATS",)
+    CATEGORY = "mtb/batch"
+
+    def assemble_floats(self, reverse, **kwargs):
+        res = []
+        if reverse:
+            for x in reversed(kwargs.values()):
+                res += x
+        else:
+            for x in kwargs.values():
+                res += x
+
+        return (res,)
+
+
+class BatchFloat:
+    """Generates a batch of float values with interpolation"""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "mode": (
+                    ["Single", "Steps"],
+                    {"default": "Steps"},
+                ),
+                "count": ("INT", {"default": 1}),
+                "min": ("FLOAT", {"default": 0.0}),
+                "max": ("FLOAT", {"default": 1.0}),
+                "easing": (
+                    [
+                        "Linear",
+                        "Sine In",
+                        "Sine Out",
+                        "Sine In/Out",
+                        "Quart In",
+                        "Quart Out",
+                        "Quart In/Out",
+                        "Cubic In",
+                        "Cubic Out",
+                        "Cubic In/Out",
+                        "Circ In",
+                        "Circ Out",
+                        "Circ In/Out",
+                        "Back In",
+                        "Back Out",
+                        "Back In/Out",
+                        "Elastic In",
+                        "Elastic Out",
+                        "Elastic In/Out",
+                        "Bounce In",
+                        "Bounce Out",
+                        "Bounce In/Out",
+                    ],
+                    {"default": "Linear"},
+                ),
+            }
+        }
+
+    FUNCTION = "set_floats"
+    RETURN_TYPES = ("FLOATS",)
+    CATEGORY = "mtb/batch"
+
+    def set_floats(self, mode, count, min, max, easing):
+        keyframes = []
+        if mode == "Single":
+            keyframes = [min] * count
+            return (keyframes,)
+
+        for i in range(count):
+            normalized_step = i / (count - 1)
+            eased_step = apply_easing(normalized_step, easing)
+            eased_value = min + (max - min) * eased_step
+            keyframes.append(eased_value)
+
+        return (keyframes,)
+
+
+class BatchMerge:
+    """Merges multiple image batches with different frame counts"""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "fusion_mode": (["add", "multiply", "average"], {"default": "average"}),
+                "fill": (["head", "tail"], {"default": "tail"}),
+            }
+        }
+
+    RETURN_TYPES = ("IMAGE",)
+    FUNCTION = "merge_batches"
+    CATEGORY = "mtb/batch"
+
+    def merge_batches(self, fusion_mode, fill, **kwargs):
+        images = kwargs.values()
+        max_frames = max(img.shape[0] for img in images)
+
+        adjusted_images = []
+        for img in images:
+            frame_count = img.shape[0]
+            if frame_count < max_frames:
+                fill_frame = img[0] if fill == "head" else img[-1]
+                fill_frames = fill_frame.repeat(max_frames - frame_count, 1, 1, 1)
+                adjusted_batch = (
+                    torch.cat((fill_frames, img), dim=0)
+                    if fill == "head"
+                    else torch.cat((img, fill_frames), dim=0)
+                )
+            else:
+                adjusted_batch = img
+            adjusted_images.append(adjusted_batch)
+
+        # Merge the adjusted batches
+        merged_image = None
+        for img in adjusted_images:
+            if merged_image is None:
+                merged_image = img
+            else:
+                if fusion_mode == "add":
+                    merged_image += img
+                elif fusion_mode == "multiply":
+                    merged_image *= img
+                elif fusion_mode == "average":
+                    merged_image = (merged_image + img) / 2
+
+        return (merged_image,)
+
+
+class Batch2dTransform:
+    """Transform a batch of images using a batch of keyframes"""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "image": ("IMAGE",),
+                "border_handling": (
+                    ["edge", "constant", "reflect", "symmetric"],
+                    {"default": "edge"},
+                ),
+                "constant_color": ("COLOR", {"default": "#000000"}),
+            },
+            "optional": {
+                "x": ("FLOATS",),
+                "y": ("FLOATS",),
+                "zoom": ("FLOATS",),
+                "angle": ("FLOATS",),
+                "shear": ("FLOATS",),
+            },
+        }
+
+    RETURN_TYPES = ("IMAGE",)
+    FUNCTION = "transform_batch"
+    CATEGORY = "mtb/batch"
+
+    def get_num_elements(self, param) -> int:
+        if isinstance(param, torch.Tensor):
+            return torch.numel(param)
+        elif isinstance(param, list):
+            return len(param)
+
+        return 0
+
+    def transform_batch(
+        self,
+        image: torch.Tensor,
+        border_handling,
+        constant_color,
+        x=None,
+        y=None,
+        zoom=None,
+        angle=None,
+        shear=None,
+    ):
+        if all(
+            self.get_num_elements(param) <= 0 for param in [x, y, zoom, angle, shear]
+        ):
+            raise ValueError("At least one transform parameter must be provided")
+
+        keyframes = {"x": [], "y": [], "zoom": [], "angle": [], "shear": []}
+
+        default_vals = {"x": 0, "y": 0, "zoom": 1.0, "angle": 0, "shear": 0}
+
+        if self.get_num_elements(x) > 0:
+            keyframes["x"] = x
+        if self.get_num_elements(y) > 0:
+            keyframes["y"] = y
+        if self.get_num_elements(zoom) > 0:
+            keyframes["zoom"] = zoom
+        if self.get_num_elements(angle) > 0:
+            keyframes["angle"] = angle
+        if self.get_num_elements(shear) > 0:
+            keyframes["shear"] = shear
+
+        for name, values in keyframes.items():
+            count = len(values)
+            if count > 0 and count != image.shape[0]:
+                raise ValueError(
+                    f"Length of {name} values ({count}) must match number of images ({image.shape[0]})"
+                )
+            if count == 0:
+                keyframes[name] = [default_vals[name]] * image.shape[0]
+
+        transformer = TransformImage()
+        res = [
+            transformer.transform(
+                image[i].unsqueeze(0),
+                keyframes["x"][i],
+                keyframes["y"][i],
+                keyframes["zoom"][i],
+                keyframes["angle"][i],
+                keyframes["shear"][i],
+                border_handling,
+                constant_color,
+            )[0]
+            for i in range(image.shape[0])
+        ]
+        return (torch.cat(res, dim=0),)
+
+
+DEFAULT_INTERPOLANT = lambda t: t * t * t * (t * (t * 6 - 15) + 10)
+
+
+class BatchShake:
+    """Applies a shaking effect to batches of images."""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "images": ("IMAGE",),
+                "position_amount_x": ("FLOAT", {"default": 1.0}),
+                "position_amount_y": ("FLOAT", {"default": 1.0}),
+                "rotation_amount": ("FLOAT", {"default": 10.0}),
+                "frequency": ("FLOAT", {"default": 1.0, "min": 0.005}),
+                "frequency_divider": ("FLOAT", {"default": 1.0, "min": 0.005}),
+                "octaves": ("INT", {"default": 1, "min": 1}),
+                "seed": ("INT", {"default": 0}),
+            },
+        }
+
+    RETURN_TYPES = ("IMAGE", "FLOATS", "FLOATS", "FLOATS")
+    RETURN_NAMES = ("image", "pos_x", "pos_y", "rot")
+    FUNCTION = "apply_shake"
+    CATEGORY = "mtb/batch"
+
+    # def interpolant(self, t):
+    # return t * t * t * (t * (t * 6 - 15) + 10)
+
+    def generate_perlin_noise_2d(
+        self, shape, res, tileable=(False, False), interpolant=None
+    ):
+        """Generate a 2D numpy array of perlin noise.
+
+        Args:
+            shape: The shape of the generated array (tuple of two ints).
+                This must be a multple of res.
+            res: The number of periods of noise to generate along each
+                axis (tuple of two ints). Note shape must be a multiple of
+                res.
+            tileable: If the noise should be tileable along each axis
+                (tuple of two bools). Defaults to (False, False).
+            interpolant: The interpolation function, defaults to
+                t*t*t*(t*(t*6 - 15) + 10).
+
+        Returns:
+            A numpy array of shape shape with the generated noise.
+
+        Raises:
+            ValueError: If shape is not a multiple of res.
+        """
+        interpolant = interpolant or DEFAULT_INTERPOLANT
+        delta = (res[0] / shape[0], res[1] / shape[1])
+        d = (shape[0] // res[0], shape[1] // res[1])
+        grid = (
+            np.mgrid[0 : res[0] : delta[0], 0 : res[1] : delta[1]].transpose(1, 2, 0)
+            % 1
+        )
+        # Gradients
+        angles = 2 * np.pi * np.random.rand(res[0] + 1, res[1] + 1)
+        gradients = np.dstack((np.cos(angles), np.sin(angles)))
+        if tileable[0]:
+            gradients[-1, :] = gradients[0, :]
+        if tileable[1]:
+            gradients[:, -1] = gradients[:, 0]
+        gradients = gradients.repeat(d[0], 0).repeat(d[1], 1)
+        g00 = gradients[: -d[0], : -d[1]]
+        g10 = gradients[d[0] :, : -d[1]]
+        g01 = gradients[: -d[0], d[1] :]
+        g11 = gradients[d[0] :, d[1] :]
+        # Ramps
+        n00 = np.sum(np.dstack((grid[:, :, 0], grid[:, :, 1])) * g00, 2)
+        n10 = np.sum(np.dstack((grid[:, :, 0] - 1, grid[:, :, 1])) * g10, 2)
+        n01 = np.sum(np.dstack((grid[:, :, 0], grid[:, :, 1] - 1)) * g01, 2)
+        n11 = np.sum(np.dstack((grid[:, :, 0] - 1, grid[:, :, 1] - 1)) * g11, 2)
+        # Interpolation
+        t = interpolant(grid)
+        n0 = n00 * (1 - t[:, :, 0]) + t[:, :, 0] * n10
+        n1 = n01 * (1 - t[:, :, 0]) + t[:, :, 0] * n11
+        return np.sqrt(2) * ((1 - t[:, :, 1]) * n0 + t[:, :, 1] * n1)
+
+    def generate_fractal_noise_2d(
+        self,
+        shape,
+        res,
+        octaves=1,
+        persistence=0.5,
+        lacunarity=2,
+        tileable=(True, True),
+        interpolant=None,
+    ):
+        """Generate a 2D numpy array of fractal noise.
+
+        Args:
+            shape: The shape of the generated array (tuple of two ints).
+                This must be a multiple of lacunarity**(octaves-1)*res.
+            res: The number of periods of noise to generate along each
+                axis (tuple of two ints). Note shape must be a multiple of
+                (lacunarity**(octaves-1)*res).
+            octaves: The number of octaves in the noise. Defaults to 1.
+            persistence: The scaling factor between two octaves.
+            lacunarity: The frequency factor between two octaves.
+            tileable: If the noise should be tileable along each axis
+                (tuple of two bools). Defaults to (True,True).
+            interpolant: The, interpolation function, defaults to
+                t*t*t*(t*(t*6 - 15) + 10).
+
+        Returns:
+            A numpy array of fractal noise and of shape shape generated by
+            combining several octaves of perlin noise.
+
+        Raises:
+            ValueError: If shape is not a multiple of
+                (lacunarity**(octaves-1)*res).
+        """
+        interpolant = interpolant or DEFAULT_INTERPOLANT
+
+        noise = np.zeros(shape)
+        frequency = 1
+        amplitude = 1
+        for _ in range(octaves):
+            noise += amplitude * self.generate_perlin_noise_2d(
+                shape, (frequency * res[0], frequency * res[1]), tileable, interpolant
+            )
+            frequency *= lacunarity
+            amplitude *= persistence
+        return noise
+
+    def fbm(self, x, y, octaves):
+        # noise_2d = self.generate_fractal_noise_2d((256, 256), (8, 8), octaves)
+        # Now, extract a single noise value based on x and y, wrapping indices if necessary
+        x_idx = int(x) % 256
+        y_idx = int(y) % 256
+        return self.noise_pattern[x_idx, y_idx]
+
+    def apply_shake(
+        self,
+        images,
+        position_amount_x,
+        position_amount_y,
+        rotation_amount,
+        frequency,
+        frequency_divider,
+        octaves,
+        seed,
+    ):
+        # Rehash
+        np.random.seed(seed)
+        self.position_offset = np.random.uniform(-1e3, 1e3, 3)
+        self.rotation_offset = np.random.uniform(-1e3, 1e3, 3)
+        self.noise_pattern = self.generate_perlin_noise_2d(
+            (512, 512), (32, 32), (True, True)
+        )
+
+        # Assuming frame count is derived from the first dimension of images tensor
+        frame_count = images.shape[0]
+
+        frequency = frequency / frequency_divider
+
+        # Generate shaking parameters for each frame
+        x_translations = []
+        y_translations = []
+        rotations = []
+
+        for frame_num in range(frame_count):
+            time = frame_num * frequency
+            x_idx = (self.position_offset[0] + frame_num) % 256
+            y_idx = (self.position_offset[1] + frame_num) % 256
+
+            np_position = np.array(
+                [
+                    self.fbm(x_idx, time, octaves),
+                    self.fbm(y_idx, time, octaves),
+                ]
+            )
+
+            # np_position = np.array(
+            #     [
+            #         self.fbm(self.position_offset[0] + frame_num, time, octaves),
+            #         self.fbm(self.position_offset[1] + frame_num, time, octaves),
+            #     ]
+            # )
+            # np_rotation = self.fbm(self.rotation_offset[2] + frame_num, time, octaves)
+
+            rot_idx = (self.rotation_offset[2] + frame_num) % 256
+            np_rotation = self.fbm(rot_idx, time, octaves)
+
+            x_translations.append(np_position[0] * position_amount_x)
+            y_translations.append(np_position[1] * position_amount_y)
+            rotations.append(np_rotation * rotation_amount)
+
+        # Convert lists to tensors
+        # x_translations = torch.tensor(x_translations, dtype=torch.float32)
+        # y_translations = torch.tensor(y_translations, dtype=torch.float32)
+        # rotations = torch.tensor(rotations, dtype=torch.float32)
+
+        # Create an instance of Batch2dTransform
+        transform = Batch2dTransform()
+
+        log.debug(
+            f"Applying shaking with parameters: \nposition {position_amount_x}, {position_amount_y}\nrotation {rotation_amount}\nfrequency {frequency}\noctaves {octaves}"
+        )
+
+        # Apply shaking transformations to images
+        shaken_images = transform.transform_batch(
+            images,
+            border_handling="edge",  # Assuming edge handling as default
+            constant_color="#000000",  # Assuming black as default constant color
+            x=x_translations,
+            y=y_translations,
+            angle=rotations,
+        )[0]
+
+        return (shaken_images, x_translations, y_translations, rotations)
+
+
+class BatchFloatsFromSound:
+    """Extracts a list of floats based on audio frequency band peaks."""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "audio": ("AUDIO",),
+                "sensitivity": ("FLOAT", {"default": 1.0}),
+                "low_freq": ("FLOAT", {"default": 100.0}),
+                "high_freq": ("FLOAT", {"default": 2000.0}),
+                "hop_length": ("INT", {"default": 512}),
+            },
+        }
+
+    RETURN_TYPES = ("FLOATS",)
+    RETURN_NAMES = ("float_data",)
+    FUNCTION = "process_audio"
+    CATEGORY = "mtb/audio"
+
+    def process_audio(
+        self,
+        audio,
+        sensitivity=1.0,
+        low_freq=100,
+        high_freq=2000,
+        hop_length=512,
+    ):
+        # audio_data, _ = librosa.load(audio_file_path, sr=sample_rate)
+
+        # audio_data_tensor = audio.squeeze(1)  # Remove the channel dimension if present
+        # audio_tensor = audio_data_tensor.float()
+        audio_data = audio.to(device=torchaudio.transforms.Spectrogram().window.device)
+
+        hop_length = 512
+        stft = torchaudio.transforms.Spectrogram()(audio_data)
+        freqs = torchaudio.transforms.FrequencyMasking(low_freq, high_freq)(stft)
+        band_energy = torch.sum(freqs, dim=1)
+
+        min_val = torch.min(band_energy)
+        max_val = torch.max(band_energy)
+        normalized_peaks = (band_energy - min_val) / (max_val - min_val)
+        scaled_peaks = normalized_peaks * sensitivity
+
+        return (scaled_peaks.tolist(),)
+
+
+__nodes__ = [
+    BatchFloat,
+    Batch2dTransform,
+    BatchFloatsFromSound,
+    BatchShape,
+    BatchMake,
+    BatchFloatAssemble,
+    BatchFloatFill,
+    BatchMerge,
+    BatchShake,
+]

nodes/conditions.py

@@ -1,9 +1,93 @@
-from ..utils import here
-from ..log import log
-import folder_paths
-from pathlib import Path
-import shutil
 import csv
+import shutil
+from pathlib import Path
+
+import folder_paths
+
+from ..log import log
+from ..utils import here
+
+
+class InterpolateClipSequential:
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "base_text": ("STRING", {"multiline": True}),
+                "text_to_replace": ("STRING", {"default": ""}),
+                "clip": ("CLIP",),
+                "interpolation_strength": (
+                    "FLOAT",
+                    {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01},
+                ),
+            }
+        }
+
+    RETURN_TYPES = ("CONDITIONING",)
+    FUNCTION = "interpolate_encodings_sequential"
+
+    CATEGORY = "mtb/conditioning"
+
+    def interpolate_encodings_sequential(
+        self, base_text, text_to_replace, clip, interpolation_strength, **replacements
+    ):
+        log.debug(f"Received interpolation_strength: {interpolation_strength}")
+
+        # - Ensure interpolation strength is within [0, 1]
+        interpolation_strength = max(0.0, min(1.0, interpolation_strength))
+
+        # - Check if replacements were provided
+        if not replacements:
+            raise ValueError("At least one replacement should be provided.")
+
+        num_replacements = len(replacements)
+        log.debug(f"Number of replacements: {num_replacements}")
+
+        segment_length = 1.0 / num_replacements
+        log.debug(f"Calculated segment_length: {segment_length}")
+
+        # - Find the segment that the interpolation_strength falls into
+        segment_index = min(
+            int(interpolation_strength // segment_length), num_replacements - 1
+        )
+        log.debug(f"Segment index: {segment_index}")
+
+        # - Calculate the local strength within the segment
+        local_strength = (
+            interpolation_strength - (segment_index * segment_length)
+        ) / segment_length
+        log.debug(f"Local strength: {local_strength}")
+
+        # - If it's the first segment, interpolate between base_text and the first replacement
+        if segment_index == 0:
+            replacement_text = list(replacements.values())[0]
+            log.debug("Using the base text a the base blend")
+            # -  Start with the base_text condition
+            tokens = clip.tokenize(base_text)
+            cond_from, pooled_from = clip.encode_from_tokens(tokens, return_pooled=True)
+        else:
+            base_replace = list(replacements.values())[segment_index - 1]
+            log.debug(f"Using {base_replace} a the base blend")
+
+            # - Start with the base_text condition replaced by the closest replacement
+            tokens = clip.tokenize(base_text.replace(text_to_replace, base_replace))
+            cond_from, pooled_from = clip.encode_from_tokens(tokens, return_pooled=True)
+
+            replacement_text = list(replacements.values())[segment_index]
+
+        interpolated_text = base_text.replace(text_to_replace, replacement_text)
+        tokens = clip.tokenize(interpolated_text)
+        cond_to, pooled_to = clip.encode_from_tokens(tokens, return_pooled=True)
+
+        # - Linearly interpolate between the two conditions
+        interpolated_condition = (
+            1.0 - local_strength
+        ) * cond_from + local_strength * cond_to
+        interpolated_pooled = (
+            1.0 - local_strength
+        ) * pooled_from + local_strength * pooled_to
+
+        return ([[interpolated_condition, {"pooled_output": interpolated_pooled}]],)
 
 
 class SmartStep:
@@ -75,7 +159,13 @@ class StylesLoader:
                     parsed = csv.reader(f)
                     for row in parsed:
                         log.debug(f"Adding style {row[0]}")
-                        cls.options[row[0]] = (row[1], row[2])
+                        try:
+                            cls.options[row[0]] = (row[1], row[2])
+                        except Exception:
+                            log.warning(
+                                f"There was an error while parsing {file}, make sure it respects A1111 format, i.e 3 columns name, positive, negative"
+                            )
+                            continue
 
         else:
             log.debug(f"Using cached styles (count: {len(cls.options)})")
@@ -96,4 +186,4 @@ class StylesLoader:
         return (self.options[style_name][0], self.options[style_name][1])
 
 
-__nodes__ = [SmartStep, StylesLoader]
+__nodes__ = [SmartStep, StylesLoader, InterpolateClipSequential]

nodes/crop.py

@@ -1,9 +1,9 @@
-import torch
-from ..utils import tensor2pil, pil2tensor, tensor2np, np2tensor
-from PIL import Image, ImageFilter, ImageDraw, ImageChops
 import numpy as np
+import torch
+from PIL import Image, ImageChops, ImageDraw, ImageFilter
 
 from ..log import log
+from ..utils import np2tensor, pil2tensor, tensor2np, tensor2pil
 
 
 class Bbox:
@@ -32,7 +32,7 @@ class Bbox:
     CATEGORY = "mtb/crop"
 
     def do_crop(self, x, y, width, height):  # bbox
-        return (x, y, width, height)
+        return ((x, y, width, height),)
         # return bbox
 
 

nodes/debug.py

@@ -1,10 +1,71 @@
-from ..utils import tensor2pil
-from ..log import log
-import io, base64
-import torch
-import folder_paths
-from typing import Optional
+import base64
+import io
 from pathlib import Path
+from typing import Optional
+
+import folder_paths
+import torch
+
+from ..log import log
+from ..utils import tensor2pil
+
+
+# region processors
+def process_tensor(tensor):
+    log.debug(f"Tensor: {tensor.shape}")
+
+    image = tensor2pil(tensor)
+    b64_imgs = []
+    for im in image:
+        buffered = io.BytesIO()
+        im.save(buffered, format="PNG")
+        b64_imgs.append(
+            "data:image/png;base64,"
+            + base64.b64encode(buffered.getvalue()).decode("utf-8")
+        )
+
+    return {"b64_images": b64_imgs}
+
+
+def process_list(anything):
+    text = []
+    if not anything:
+        return {"text": []}
+
+    first_element = anything[0]
+    if (
+        isinstance(first_element, list)
+        and first_element
+        and isinstance(first_element[0], torch.Tensor)
+    ):
+        text.append(
+            f"List of List of Tensors: {first_element[0].shape} (x{len(anything)})"
+        )
+
+    elif isinstance(first_element, torch.Tensor):
+        text.append(f"List of Tensors: {first_element.shape} (x{len(anything)})")
+
+    return {"text": text}
+
+
+def process_dict(anything):
+    text = []
+    if "samples" in anything:
+        is_empty = "(empty)" if torch.count_nonzero(anything["samples"]) == 0 else ""
+        text.append(f"Latent Samples: {anything['samples'].shape} {is_empty}")
+
+    return {"text": text}
+
+
+def process_bool(anything):
+    return {"text": ["True" if anything else "False"]}
+
+
+def process_text(anything):
+    return {"text": [str(anything)]}
+
+
+# endregion
 
 
 class Debug:
@@ -13,46 +74,38 @@ class Debug:
     @classmethod
     def INPUT_TYPES(cls):
         return {
-            "required": {"anything_1": ("*")},
+            "required": {"output_to_console": ("BOOLEAN", {"default": False})},
         }
 
-    RETURN_TYPES = ("STRING",)
+    RETURN_TYPES = ()
     FUNCTION = "do_debug"
     CATEGORY = "mtb/debug"
     OUTPUT_NODE = True
 
-    def do_debug(self, **kwargs):
+    def do_debug(self, output_to_console, **kwargs):
         output = {
             "ui": {"b64_images": [], "text": []},
-            "result": ("A"),
+            # "result": ("A"),
         }
-        for k, v in kwargs.items():
-            anything = v
-            text = ""
-            if isinstance(anything, torch.Tensor):
-                log.debug(f"Tensor: {anything.shape}")
 
-                # write the images to temp
+        processors = {
+            torch.Tensor: process_tensor,
+            list: process_list,
+            dict: process_dict,
+            bool: process_bool,
+        }
+        if output_to_console:
+            print("bouh!")
 
-                image = tensor2pil(anything)
-                b64_imgs = []
-                for im in image:
-                    buffered = io.BytesIO()
-                    im.save(buffered, format="PNG")
-                    b64_imgs.append(
-                        "data:image/png;base64,"
-                        + base64.b64encode(buffered.getvalue()).decode("utf-8")
-                    )
+        for anything in kwargs.values():
+            processor = processors.get(type(anything), process_text)
+            processed_data = processor(anything)
 
-                output["ui"]["b64_images"] += b64_imgs
-                log.debug(f"Input {k} contains {len(b64_imgs)} images")
-            elif isinstance(anything, bool):
-                log.debug(f"Input {k} contains boolean: {anything}")
-                output["ui"]["text"] += ["True" if anything else "False"]
-            else:
-                text = str(anything)
-                log.debug(f"Input {k} contains text: {text}")
-                output["ui"]["text"] += [text]
+            for ui_key, ui_value in processed_data.items():
+                output["ui"][ui_key].extend(ui_value)
+            # log.debug(
+            #     f"Processed input {k}, found {len(processed_data.get('b64_images', []))} images and {len(processed_data.get('text', []))} text items."
+            # )
 
         return output
 

nodes/deep_bump.py

@@ -1,23 +1,38 @@
-import onnxruntime as ort
+import tempfile
+from pathlib import Path
+
 import numpy as np
-import pathlib
 import onnxruntime as ort
-import numpy as np
-from .. import utils as utils_inference
-from ..log import log
+import torch
+from PIL import Image
+
+from ..errors import ModelNotFound
+from ..log import mklog
+from ..utils import get_model_path, tensor2pil, tiles_infer, tiles_merge, tiles_split
 
 # Disable MS telemetry
 ort.disable_telemetry_events()
+log = mklog(__name__)
 
 
 # - COLOR to NORMALS
-def color_to_normals(color_img, overlap, progress_callback):
+def color_to_normals(color_img, overlap, progress_callback, save_temp=False):
     """Computes a normal map from the given color map. 'color_img' must be a numpy array
     in C,H,W format (with C as RGB). 'overlap' must be one of 'SMALL', 'MEDIUM', 'LARGE'.
     """
+    temp_dir = Path(tempfile.mkdtemp()) if save_temp else None
 
     # Remove alpha & convert to grayscale
-    img = np.mean(color_img[:3], axis=0, keepdimss=True)
+    img = np.mean(color_img[:3], axis=0, keepdims=True)
+
+    if temp_dir:
+        Image.fromarray((img[0] * 255).astype(np.uint8)).save(
+            temp_dir / "grayscale_img.png"
+        )
+
+    log.debug(
+        f"Converting color image to grayscale by taking the mean over color channels: {img.shape}"
+    )
 
     # Split image in tiles
     log.debug("DeepBump Color → Normals : tilling")
@@ -28,32 +43,56 @@ def color_to_normals(color_img, overlap, progress_callback):
         "LARGE": tile_size // 2,
     }
     stride_size = tile_size - overlaps[overlap]
-    tiles, paddings = utils_inference.tiles_split(
+    tiles, paddings = tiles_split(
         img, (tile_size, tile_size), (stride_size, stride_size)
     )
+    if temp_dir:
+        for i, tile in enumerate(tiles):
+            Image.fromarray((tile[0] * 255).astype(np.uint8)).save(
+                temp_dir / f"tile_{i}.png"
+            )
 
     # Load model
     log.debug("DeepBump Color → Normals : loading model")
-    addon_path = str(pathlib.Path(__file__).parent.absolute())
-    ort_session = ort.InferenceSession(f"{addon_path}/models/deepbump256.onnx")
+    model = get_model_path("deepbump", "deepbump256.onnx")
+    if not model or not model.exists():
+        raise ModelNotFound(f"deepbump ({model})")
+
+    ort_session = ort.InferenceSession(model)
 
     # Predict normal map for each tile
     log.debug("DeepBump Color → Normals : generating")
-    pred_tiles = utils_inference.tiles_infer(
-        tiles, ort_session, progress_callback=progress_callback
-    )
+    pred_tiles = tiles_infer(tiles, ort_session, progress_callback=progress_callback)
+
+    if temp_dir:
+        for i, pred_tile in enumerate(pred_tiles):
+            Image.fromarray((pred_tile.transpose(1, 2, 0) * 255).astype(np.uint8)).save(
+                temp_dir / f"pred_tile_{i}.png"
+            )
 
     # Merge tiles
     log.debug("DeepBump Color → Normals : merging")
-    pred_img = utils_inference.tiles_merge(
+    pred_img = tiles_merge(
         pred_tiles,
         (stride_size, stride_size),
         (3, img.shape[1], img.shape[2]),
         paddings,
     )
 
+    if temp_dir:
+        Image.fromarray((pred_img.transpose(1, 2, 0) * 255).astype(np.uint8)).save(
+            temp_dir / "merged_img.png"
+        )
+
     # Normalize each pixel to unit vector
-    pred_img = utils_inference.normalize(pred_img)
+    pred_img = normalize(pred_img)
+
+    if temp_dir:
+        Image.fromarray((pred_img.transpose(1, 2, 0) * 255).astype(np.uint8)).save(
+            temp_dir / "final_img.png"
+        )
+
+        log.debug(f"Debug images saved in {temp_dir}")
 
     return pred_img
 
@@ -261,7 +300,7 @@ class DeepBump:
                         "LARGEST",
                     ],
                 ),
-                "normals_to_height_seamless": ("BOOLEAN", {"default": False}),
+                "normals_to_height_seamless": ("BOOLEAN", {"default": True}),
             },
         }
 
@@ -278,25 +317,38 @@ class DeepBump:
         normals_to_curvature_blur_radius="SMALL",
         normals_to_height_seamless=True,
     ):
-        image = utils_inference.tensor2pil(image)
+        images = tensor2pil(image)
+        out_images = []
 
-        in_img = np.transpose(image, (2, 0, 1)) / 255
+        for image in images:
+            log.debug(f"Input image shape: {image}")
 
-        log.debug(f"Input image shape: {in_img.shape}")
+            in_img = np.transpose(image, (2, 0, 1)) / 255
+            log.debug(f"transposed for deep image shape: {in_img.shape}")
+            out_img = None
 
-        # Apply processing
-        if mode == "Color to Normals":
-            out_img = color_to_normals(in_img, color_to_normals_overlap, None)
-        if mode == "Normals to Curvature":
-            out_img = normals_to_curvature(
-                in_img, normals_to_curvature_blur_radius, None
-            )
-        if mode == "Normals to Height":
-            out_img = normals_to_height(in_img, normals_to_height_seamless, None)
+            # Apply processing
+            if mode == "Color to Normals":
+                out_img = color_to_normals(in_img, color_to_normals_overlap, None)
+            if mode == "Normals to Curvature":
+                out_img = normals_to_curvature(
+                    in_img, normals_to_curvature_blur_radius, None
+                )
+            if mode == "Normals to Height":
+                out_img = normals_to_height(in_img, normals_to_height_seamless, None)
 
-        out_img = (np.transpose(out_img, (1, 2, 0)) * 255).astype(np.uint8)
-
-        return (utils_inference.pil2tensor(out_img),)
+            if out_img is not None:
+                log.debug(f"Output image shape: {out_img.shape}")
+                out_images.append(
+                    torch.from_numpy(
+                        np.transpose(out_img, (1, 2, 0)).astype(np.float32)
+                    ).unsqueeze(0)
+                )
+            else:
+                log.error("No out img... This should not happen")
+        for outi in out_images:
+            log.debug(f"Shape fed to utils: {outi.shape}")
+        return (torch.cat(out_images, dim=0),)
 
 
 __nodes__ = [DeepBump]

nodes/faceenhance.py

@@ -1,21 +1,19 @@
-from gfpgan import GFPGANer
-import cv2
-import numpy as np
 import os
 from pathlib import Path
-import folder_paths
-from ..utils import pil2tensor, np2tensor, tensor2np
+from typing import Tuple
 
-from basicsr.utils import imwrite
-
-
-from PIL import Image
-import torch
-from ..log import NullWriter, log
-from comfy import model_management
 import comfy
 import comfy.utils
-from typing import Tuple
+import cv2
+import folder_paths
+import numpy as np
+import torch
+from comfy import model_management
+from gfpgan import GFPGANer
+from PIL import Image
+
+from ..log import NullWriter, log
+from ..utils import get_model_path, np2tensor, pil2tensor, tensor2np
 
 
 class LoadFaceEnhanceModel:
@@ -26,11 +24,12 @@ class LoadFaceEnhanceModel:
 
     @classmethod
     def get_models_root(cls):
-        fr = Path(folder_paths.models_dir) / "face_restore"
+        fr = get_model_path("face_restore")
+        # fr = Path(folder_paths.models_dir) / "face_restore"
         if fr.exists():
             return (fr, None)
 
-        um = Path(folder_paths.models_dir) / "upscale_models"
+        um = get_model_path("upscale_models")
         return (fr, um) if um.exists() else (None, None)
 
     @classmethod
@@ -247,16 +246,16 @@ class RestoreFace:
         ):
             face_id = idx + 1
             file = self.get_step_image_path("cropped_faces", face_id)
-            imwrite(cropped_face, file)
+            cv2.imwrite(file, cropped_face)
 
             file = self.get_step_image_path("cropped_faces_restored", face_id)
-            imwrite(restored_face, file)
+            cv2.imwrite(file, restored_face)
 
             file = self.get_step_image_path("cropped_faces_compare", face_id)
 
             # save comparison image
             cmp_img = np.concatenate((cropped_face, restored_face), axis=1)
-            imwrite(cmp_img, file)
+            cv2.imwrite(file, cmp_img)
 
 
 __nodes__ = [RestoreFace, LoadFaceEnhanceModel]

nodes/faceswap.py

@@ -1,21 +1,21 @@
+# Optional face enhance nodes
 # region imports
-import onnxruntime
+import sys
 from pathlib import Path
-from PIL import Image
-from typing import List, Set, Union, Optional
+from typing import List, Optional, Set, Union
+
+import comfy.model_management as model_management
 import cv2
-import folder_paths
-import glob
 import insightface
 import numpy as np
-import os
+import onnxruntime
 import torch
 from insightface.model_zoo.inswapper import INSwapper
-from ..utils import pil2tensor, tensor2pil, download_antelopev2
-from ..log import mklog, NullWriter
-import sys
-import comfy.model_management as model_management
+from PIL import Image
 
+from ..errors import ModelNotFound
+from ..log import NullWriter, mklog
+from ..utils import download_antelopev2, get_model_path, pil2tensor, tensor2pil
 
 # endregion
 
@@ -27,15 +27,6 @@ class LoadFaceAnalysisModel:
 
     models = []
 
-    @staticmethod
-    def get_models() -> List[str]:
-        models_path = os.path.join(folder_paths.models_dir, "insightface/*")
-        models = glob.glob(models_path)
-        models = [
-            Path(x).name for x in models if x.endswith(".onnx") or x.endswith(".pth")
-        ]
-        return models
-
     @classmethod
     def INPUT_TYPES(cls):
         return {
@@ -57,7 +48,7 @@ class LoadFaceAnalysisModel:
 
         face_analyser = insightface.app.FaceAnalysis(
             name=faceswap_model,
-            root=os.path.join(folder_paths.models_dir, "insightface"),
+            root=get_model_path("insightface"),
         )
         return (face_analyser,)
 
@@ -67,10 +58,8 @@ class LoadFaceSwapModel:
 
     @staticmethod
     def get_models() -> List[Path]:
-        models_path = os.path.join(folder_paths.models_dir, "insightface/*")
-        models = glob.glob(models_path)
-        models = [Path(x) for x in models if x.endswith(".onnx") or x.endswith(".pth")]
-        return models
+        models_path = get_model_path("insightface").iterdir()
+        return [x for x in models_path if x.suffix in [".onnx", ".pth"]]
 
     @classmethod
     def INPUT_TYPES(cls):
@@ -88,9 +77,10 @@ class LoadFaceSwapModel:
     CATEGORY = "mtb/facetools"
 
     def load_model(self, faceswap_model: str):
-        model_path = os.path.join(
-            folder_paths.models_dir, "insightface", faceswap_model
-        )
+        model_path = get_model_path("insightface", faceswap_model)
+        if not model_path or not model_path.exists():
+            raise ModelNotFound(f"{faceswap_model} ({model_path})")
+
         log.info(f"Loading model {model_path}")
         return (
             INSwapper(

nodes/graph_utils.py

@@ -1,22 +1,27 @@
-from ..log import log
-from PIL import Image
-import urllib.request
-import urllib.parse
-import torch
-import json
-from comfy.cli_args import args
-from ..utils import pil2tensor, apply_easing
 import io
+import json
+import urllib.parse
+import urllib.request
+
 import numpy as np
+import torch
+from PIL import Image
+
+from ..log import log
+from ..utils import apply_easing, get_server_info, pil2tensor
 
 
 def get_image(filename, subfolder, folder_type):
-    log.debug(f"Getting image {filename} from {subfolder} of {folder_type}")
+    log.debug(
+        f"Getting image {filename} from foldertype {folder_type} {f'in subfolder: {subfolder}' if subfolder else ''}"
+    )
     data = {"filename": filename, "subfolder": subfolder, "type": folder_type}
+    base_url, port = get_server_info()
+
     url_values = urllib.parse.urlencode(data)
-    with urllib.request.urlopen(
-        f"http://{args.listen}:{args.port}/view?{url_values}"
-    ) as response:
+    url = f"http://{base_url}:{port}/view?{url_values}"
+    log.debug(f"Fetching image from {url}")
+    with urllib.request.urlopen(url) as response:
         return io.BytesIO(response.read())
 
 
@@ -60,10 +65,18 @@ class GetBatchFromHistory:
             return (torch.zeros(0),)
         frames = []
 
-        with urllib.request.urlopen(
-            f"http://{args.listen}:{args.port}/history"
-        ) as response:
-            return self.load_batch_frames(response, offset, count, frames)
+        base_url, port = get_server_info()
+
+        history_url = f"http://{base_url}:{port}/history"
+        log.debug(f"Fetching history from {history_url}")
+        output = torch.zeros(0)
+        with urllib.request.urlopen(history_url) as response:
+            output = self.load_batch_frames(response, offset, count, frames)
+
+        if output.size(0) == 0:
+            log.warn("No output found in history")
+
+        return (output,)
 
     def load_batch_frames(self, response, offset, count, frames):
         history = json.loads(response.read())
@@ -80,7 +93,7 @@ class GetBatchFromHistory:
                         output_images.append(image_data)
 
         if not output_images:
-            return (torch.zeros(0),)
+            return torch.zeros(0)
 
         # Directly get desired range of images
         start_index = max(len(output_images) - offset - count, 0)
@@ -90,13 +103,11 @@ class GetBatchFromHistory:
         frames = [Image.open(image) for image in selected_images]
 
         if not frames:
-            return (torch.zeros(0),)
+            return torch.zeros(0)
         elif len(frames) != count:
             log.warning(f"Expected {count} images, got {len(frames)} instead")
 
-        output = pil2tensor(frames)
-
-        return (output,)
+        return pil2tensor(frames)
 
 
 class AnyToString:
@@ -159,6 +170,52 @@ class StringReplace:
         return (string,)
 
 
+class MTB_MathExpression:
+    """Node to evaluate a simple math expression string"""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "expression": ("STRING", {"default": "", "multiline": True}),
+            }
+        }
+
+    FUNCTION = "eval_expression"
+    RETURN_TYPES = ("FLOAT", "INT")
+    RETURN_NAMES = ("result (float)", "result (int)")
+    CATEGORY = "mtb/math"
+    DESCRIPTION = "evaluate a simple math expression string (!! Fallsback to eval)"
+
+    def eval_expression(self, expression, **kwargs):
+        import math
+        from ast import literal_eval
+
+        for key, value in kwargs.items():
+            print(f"Replacing placeholder <{key}> with value {value}")
+            expression = expression.replace(f"<{key}>", str(value))
+
+        result = -1
+        try:
+            result = literal_eval(expression)
+        except SyntaxError as e:
+            raise ValueError(
+                f"The expression syntax is wrong '{expression}': {e}"
+            ) from e
+
+        except ValueError:
+            try:
+                expression = expression.replace("^", "**")
+                result = eval(expression)
+            except Exception as e:
+                # Handle any other exceptions and provide a meaningful error message
+                raise ValueError(
+                    f"Error evaluating expression '{expression}': {e}"
+                ) from e
+
+        return (result, int(result))
+
+
 class FitNumber:
     """Fit the input float using a source and target range"""
 
@@ -168,10 +225,10 @@ class FitNumber:
             "required": {
                 "value": ("FLOAT", {"default": 0, "forceInput": True}),
                 "clamp": ("BOOLEAN", {"default": False}),
-                "source_min": ("FLOAT", {"default": 0.0}),
-                "source_max": ("FLOAT", {"default": 1.0}),
-                "target_min": ("FLOAT", {"default": 0.0}),
-                "target_max": ("FLOAT", {"default": 1.0}),
+                "source_min": ("FLOAT", {"default": 0.0, "step": 0.01}),
+                "source_max": ("FLOAT", {"default": 1.0, "step": 0.01}),
+                "target_min": ("FLOAT", {"default": 0.0, "step": 0.01}),
+                "target_max": ("FLOAT", {"default": 1.0, "step": 0.01}),
                 "easing": (
                     [
                         "Linear",
@@ -205,6 +262,7 @@ class FitNumber:
     FUNCTION = "set_range"
     RETURN_TYPES = ("FLOAT",)
     CATEGORY = "mtb/math"
+    DESCRIPTION = "Fit the input float using a source and target range"
 
     def set_range(
         self,
@@ -216,20 +274,53 @@ class FitNumber:
         target_max: float,
         easing: str,
     ):
-        normalized_value = (value - source_min) / (source_max - source_min)
+        if source_min == source_max:
+            normalized_value = 0
+        else:
+            normalized_value = (value - source_min) / (source_max - source_min)
+        if clamp:
+            normalized_value = max(min(normalized_value, 1), 0)
 
         eased_value = apply_easing(normalized_value, easing)
 
         # - Convert the eased value to the target range
         res = target_min + (target_max - target_min) * eased_value
 
-        if clamp:
-            if target_min > target_max:
-                res = max(min(res, target_min), target_max)
-            else:
-                res = max(min(res, target_max), target_min)
-
         return (res,)
 
 
-__nodes__ = [StringReplace, FitNumber, GetBatchFromHistory, AnyToString]
+class ConcatImages:
+    """Add images to batch"""
+
+    RETURN_TYPES = ("IMAGE",)
+    FUNCTION = "concatenate_tensors"
+    CATEGORY = "mtb/image"
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {"reverse": ("BOOLEAN", {"default": False})},
+        }
+
+    def concatenate_tensors(self, reverse, **kwargs):
+        tensors = tuple(kwargs.values())
+        batch_sizes = [tensor.size(0) for tensor in tensors]
+
+        concatenated = torch.cat(tensors, dim=0)
+
+        # Update the batch size in the concatenated tensor
+        concatenated_size = list(concatenated.size())
+        concatenated_size[0] = sum(batch_sizes)
+        concatenated = concatenated.view(*concatenated_size)
+
+        return (concatenated,)
+
+
+__nodes__ = [
+    StringReplace,
+    FitNumber,
+    GetBatchFromHistory,
+    AnyToString,
+    ConcatImages,
+    MTB_MathExpression,
+]

nodes/image_interpolation.py

@@ -1,16 +1,20 @@
-from typing import List
-from pathlib import Path
-import os
 import glob
-import folder_paths
-from ..log import log
-import torch
-from frame_interpolation.eval import util, interpolator
-import numpy as np
+import os
+from pathlib import Path
+from typing import List
+
 import comfy
-import comfy.utils
-import tensorflow as tf
 import comfy.model_management as model_management
+import comfy.utils
+import folder_paths
+import numpy as np
+import tensorflow as tf
+import torch
+from frame_interpolation.eval import interpolator, util
+
+from ..errors import ModelNotFound
+from ..log import log
+from ..utils import get_model_path
 
 
 class LoadFilmModel:
@@ -18,10 +22,9 @@ class LoadFilmModel:
 
     @staticmethod
     def get_models() -> List[Path]:
-        models_path = os.path.join(folder_paths.models_dir, "FILM/*")
-        models = glob.glob(models_path)
-        models = [Path(x) for x in models if x.endswith(".onnx") or x.endswith(".pth")]
-        return models
+        models_paths = get_model_path("FILM").iterdir()
+
+        return [x for x in models_paths if x.suffix in [".onnx", ".pth"]]
 
     @classmethod
     def INPUT_TYPES(cls):
@@ -39,7 +42,10 @@ class LoadFilmModel:
     CATEGORY = "mtb/frame iterpolation"
 
     def load_model(self, film_model: str):
-        model_path = Path(folder_paths.models_dir) / "FILM" / film_model
+        model_path = get_model_path("FILM", film_model)
+        if not model_path or not model_path.exists():
+            raise ModelNotFound(f"FILM ({model_path})")
+
         if not (model_path / "saved_model.pb").exists():
             model_path = model_path / "saved_model"
 
@@ -114,41 +120,4 @@ class FilmInterpolation:
         return (out_tensors,)
 
 
-class ConcatImages:
-    """Add images to batch"""
-
-    RETURN_TYPES = ("IMAGE",)
-    FUNCTION = "concat_images"
-    CATEGORY = "mtb/image"
-
-    @classmethod
-    def INPUT_TYPES(cls):
-        return {
-            "required": {
-                "imageA": ("IMAGE",),
-                "imageB": ("IMAGE",),
-            },
-        }
-
-    @classmethod
-    def concatenate_tensors(cls, A: torch.Tensor, B: torch.Tensor):
-        # Get the batch sizes of A and B
-        batch_size_A = A.size(0)
-        batch_size_B = B.size(0)
-
-        # Concatenate the tensors along the batch dimension
-        concatenated = torch.cat((A, B), dim=0)
-
-        # Update the batch size in the concatenated tensor
-        concatenated_size = list(concatenated.size())
-        concatenated_size[0] = batch_size_A + batch_size_B
-        concatenated = concatenated.view(*concatenated_size)
-
-        return concatenated
-
-    def concat_images(self, imageA: torch.Tensor, imageB: torch.Tensor):
-        log.debug(f"Concatenating A ({imageA.shape}) and B ({imageB.shape})")
-        return (self.concatenate_tensors(imageA, imageB),)
-
-
-__nodes__ = [LoadFilmModel, FilmInterpolation, ConcatImages]
+__nodes__ = [LoadFilmModel, FilmInterpolation]

nodes/image_processing.py

@@ -1,17 +1,20 @@
-import torch
-from skimage.filters import gaussian
-from skimage.util import compare_images
+import itertools
+import json
+import math
+import os
+
+import cv2
+import folder_paths
 import numpy as np
+import torch
 import torch.nn.functional as F
 from PIL import Image
-from ..utils import tensor2pil, pil2tensor, tensor2np
-import torch
-import folder_paths
 from PIL.PngImagePlugin import PngInfo
-import json
-import os
-import math
+from skimage.filters import gaussian
+from skimage.util import compare_images
 
+from ..log import log
+from ..utils import pil2tensor, tensor2np, tensor2pil
 
 # try:
 #     from cv2.ximgproc import guidedFilter
@@ -19,6 +22,18 @@ import math
 #     log.warning("cv2.ximgproc.guidedFilter not found, use opencv-contrib-python")
 
 
+def gaussian_kernel(kernel_size: int, sigma_x: float, sigma_y: float, device=None):
+    x, y = torch.meshgrid(
+        torch.linspace(-1, 1, kernel_size, device=device),
+        torch.linspace(-1, 1, kernel_size, device=device),
+        indexing="ij",
+    )
+    d_x = x * x / (2.0 * sigma_x * sigma_x)
+    d_y = y * y / (2.0 * sigma_y * sigma_y)
+    g = torch.exp(-(d_x + d_y))
+    return g / g.sum()
+
+
 class ColorCorrect:
     """Various color correction methods"""
 
@@ -177,7 +192,7 @@ class ColorCorrect:
         return (image,)
 
 
-class ImageCompare:
+class ImageCompare_:
     """Compare two images and return a difference image"""
 
     @classmethod
@@ -213,7 +228,7 @@ class ImageCompare:
 import requests
 
 
-class LoadImageFromUrl:
+class LoadImageFromUrl_:
     """Load an image from the given URL"""
 
     @classmethod
@@ -239,7 +254,7 @@ class LoadImageFromUrl:
         return (pil2tensor(image),)
 
 
-class Blur:
+class Blur_:
     """Blur an image using a Gaussian filter."""
 
     @classmethod
@@ -270,6 +285,78 @@ class Blur:
         return (torch.from_numpy(image),)
 
 
+class Sharpen_:
+    """Sharpens an image using a Gaussian kernel."""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "image": ("IMAGE",),
+                "sharpen_radius": (
+                    "INT",
+                    {"default": 1, "min": 1, "max": 31, "step": 1},
+                ),
+                "sigma_x": (
+                    "FLOAT",
+                    {"default": 1.0, "min": 0.1, "max": 10.0, "step": 0.1},
+                ),
+                "sigma_y": (
+                    "FLOAT",
+                    {"default": 1.0, "min": 0.1, "max": 10.0, "step": 0.1},
+                ),
+                "alpha": (
+                    "FLOAT",
+                    {"default": 1.0, "min": 0.0, "max": 5.0, "step": 0.1},
+                ),
+            },
+        }
+
+    RETURN_TYPES = ("IMAGE",)
+    FUNCTION = "do_sharp"
+    CATEGORY = "mtb/image processing"
+
+    def do_sharp(
+        self,
+        image: torch.Tensor,
+        sharpen_radius: int,
+        sigma_x: float,
+        sigma_y: float,
+        alpha: float,
+    ):
+        if sharpen_radius == 0:
+            return (image,)
+
+        channels = image.shape[3]
+
+        kernel_size = 2 * sharpen_radius + 1
+        kernel = gaussian_kernel(kernel_size, sigma_x, sigma_y) * -(alpha * 10)
+
+        # Modify center of kernel to make it a sharpening kernel
+        center = kernel_size // 2
+        kernel[center, center] = kernel[center, center] - kernel.sum() + 1.0
+
+        kernel = kernel.repeat(channels, 1, 1).unsqueeze(1)
+        tensor_image = image.permute(0, 3, 1, 2)
+
+        tensor_image = F.pad(
+            tensor_image,
+            (sharpen_radius, sharpen_radius, sharpen_radius, sharpen_radius),
+            "reflect",
+        )
+        sharpened = F.conv2d(tensor_image, kernel, padding=center, groups=channels)
+
+        # Remove padding
+        sharpened = sharpened[
+            :, :, sharpen_radius:-sharpen_radius, sharpen_radius:-sharpen_radius
+        ]
+
+        sharpened = sharpened.permute(0, 2, 3, 1)
+        result = torch.clamp(sharpened, 0, 1)
+
+        return (result,)
+
+
 # https://github.com/lllyasviel/AdverseCleaner/blob/main/clean.py
 # def deglaze_np_img(np_img):
 #     y = np_img.copy()
@@ -316,22 +403,26 @@ class MaskToImage:
     FUNCTION = "render_mask"
 
     def render_mask(self, mask, color, background):
-        mask = tensor2np(mask)
-        mask = Image.fromarray(mask).convert("L")
+        masks = tensor2np(mask)
+        images = []
+        for m in masks:
+            _mask = Image.fromarray(m).convert("L")
 
-        image = Image.new("RGBA", mask.size, color=color)
-        # apply the mask
-        image = Image.composite(
-            image, Image.new("RGBA", mask.size, color=background), mask
-        )
+            log.debug(f"Converted mask to PIL Image format, size: {_mask.size}")
 
-        # image = ImageChops.multiply(image, mask)
-        # apply over background
-        # image = Image.alpha_composite(Image.new("RGBA", image.size, color=background), image)
+            image = Image.new("RGBA", _mask.size, color=color)
+            # apply the mask
+            image = Image.composite(
+                image, Image.new("RGBA", _mask.size, color=background), _mask
+            )
 
-        image = pil2tensor(image.convert("RGB"))
+            # image = ImageChops.multiply(image, mask)
+            # apply over background
+            # image = Image.alpha_composite(Image.new("RGBA", image.size, color=background), image)
 
-        return (image,)
+            images.append(image.convert("RGB"))
+
+        return (pil2tensor(images),)
 
 
 class ColoredImage:
@@ -347,7 +438,11 @@ class ColoredImage:
                 "color": ("COLOR",),
                 "width": ("INT", {"default": 512, "min": 16, "max": 8160}),
                 "height": ("INT", {"default": 512, "min": 16, "max": 8160}),
-            }
+            },
+            "optional": {
+                "foreground_image": ("IMAGE",),
+                "foreground_mask": ("MASK",),
+            },
         }
 
     CATEGORY = "mtb/generate"
@@ -356,12 +451,46 @@ class ColoredImage:
 
     FUNCTION = "render_img"
 
-    def render_img(self, color, width, height):
-        image = Image.new("RGB", (width, height), color=color)
+    def render_img(
+        self, color, width, height, foreground_image=None, foreground_mask=None
+    ):
+        image = Image.new("RGBA", (width, height), color=color)
+        output = []
+        if foreground_image is not None:
+            if foreground_mask is None:
+                fg_images = tensor2pil(foreground_image)
+                for img in fg_images:
+                    if image.size != img.size:
+                        raise ValueError(
+                            f"Dimension mismatch: image {image.size}, img {img.size}"
+                        )
 
-        image = pil2tensor(image)
+                    if img.mode != "RGBA":
+                        raise ValueError(
+                            f"Foreground image must be in 'RGBA' mode when no mask is provided, got {img.mode}"
+                        )
 
-        return (image,)
+                    output.append(Image.alpha_composite(image, img).convert("RGB"))
+
+            elif foreground_image.size[0] != foreground_mask.size[0]:
+                raise ValueError("Foreground image and mask must have same batch size")
+            else:
+                fg_images = tensor2pil(foreground_image)
+                fg_masks = tensor2pil(foreground_mask)
+                output.extend(
+                    Image.composite(
+                        fg_image.convert("RGBA"),
+                        image,
+                        fg_mask,
+                    ).convert("RGB")
+                    for fg_image, fg_mask in zip(fg_images, fg_masks)
+                )
+        elif foreground_mask is not None:
+            log.warn("Mask ignored because no foreground image is given")
+
+        output = pil2tensor(output)
+
+        return (output,)
 
 
 class ImagePremultiply:
@@ -379,19 +508,13 @@ class ImagePremultiply:
 
     CATEGORY = "mtb/image"
     RETURN_TYPES = ("IMAGE",)
+    RETURN_NAMES = ("RGBA",)
     FUNCTION = "premultiply"
 
     def premultiply(self, image, mask, invert):
         images = tensor2pil(image)
-        if invert:
-            masks = tensor2pil(mask)  # .convert("L")
-        else:
-            masks = tensor2pil(1.0 - mask)
-
-        single = False
-        if len(mask) == 1:
-            single = True
-
+        masks = tensor2pil(mask) if invert else tensor2pil(1.0 - mask)
+        single = len(mask) == 1
         masks = [x.convert("L") for x in masks]
 
         out = []
@@ -501,7 +624,7 @@ class ImageResizeFactor:
         return (resized_image,)
 
 
-class SaveImageGrid:
+class SaveImageGrid_:
     """Save all the images in the input batch as a grid of images."""
 
     def __init__(self):
@@ -603,15 +726,65 @@ class SaveImageGrid:
         return {"ui": {"images": results}}
 
 
+class ImageTileOffset:
+    """Mimics an old photoshop technique to check for seamless textures"""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "image": ("IMAGE",),
+                "tiles": ("INT", {"default": 2}),
+            }
+        }
+
+    CATEGORY = "mtb/generate"
+
+    RETURN_TYPES = ("IMAGE",)
+
+    FUNCTION = "tile_image"
+
+    def tile_image(self, image: torch.Tensor, tiles: int = 2):
+        if tiles < 1:
+            raise ValueError("The number of tiles must be at least 1.")
+
+        batch_size, height, width, channels = image.shape
+        tile_height = height // tiles
+        tile_width = width // tiles
+
+        output_image = torch.zeros_like(image)
+
+        for i, j in itertools.product(range(tiles), range(tiles)):
+            start_h = i * tile_height
+            end_h = start_h + tile_height
+            start_w = j * tile_width
+            end_w = start_w + tile_width
+
+            tile = image[:, start_h:end_h, start_w:end_w, :]
+
+            output_start_h = (i + 1) % tiles * tile_height
+            output_start_w = (j + 1) % tiles * tile_width
+            output_end_h = output_start_h + tile_height
+            output_end_w = output_start_w + tile_width
+
+            output_image[
+                :, output_start_h:output_end_h, output_start_w:output_end_w, :
+            ] = tile
+
+        return (output_image,)
+
+
 __nodes__ = [
     ColorCorrect,
-    ImageCompare,
-    Blur,
+    ImageCompare_,
+    ImageTileOffset,
+    Blur_,
     # DeglazeImage,
     MaskToImage,
     ColoredImage,
     ImagePremultiply,
     ImageResizeFactor,
-    SaveImageGrid,
-    LoadImageFromUrl,
+    SaveImageGrid_,
+    LoadImageFromUrl_,
+    Sharpen_,
 ]

nodes/image_utils.py

@@ -0,0 +1,41 @@
+import torch
+
+from ..log import log
+
+
+class StackImages:
+    """Stack the input images horizontally or vertically"""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {"required": {"vertical": ("BOOLEAN", {"default": False})}}
+
+    RETURN_TYPES = ("IMAGE",)
+    FUNCTION = "stack"
+    CATEGORY = "mtb/image utils"
+
+    def stack(self, vertical, **kwargs):
+        if not kwargs:
+            raise ValueError("At least one tensor must be provided.")
+
+        tensors = list(kwargs.values())
+        log.debug(
+            f"Stacking {len(tensors)} tensors {'vertically' if vertical else 'horizontally'}"
+        )
+        log.debug(list(kwargs.keys()))
+
+        ref_shape = tensors[0].shape
+        for tensor in tensors[1:]:
+            if tensor.shape[1:] != ref_shape[1:]:
+                raise ValueError(
+                    "All tensors must have the same dimensions except for the stacking dimension."
+                )
+
+        dim = 1 if vertical else 2
+
+        stacked_tensor = torch.cat(tensors, dim=dim)
+
+        return (stacked_tensor,)
+
+
+__nodes__ = [StackImages]

nodes/io.py

@@ -1,14 +1,49 @@
-from ..utils import tensor2np, PIL_FILTER_MAP
-import uuid
-import folder_paths
-from ..log import log
-import comfy.model_management as model_management
+import json
 import subprocess
-import torch
+import uuid
 from pathlib import Path
+from typing import List, Optional
+
+import comfy.model_management as model_management
+import folder_paths
 import numpy as np
+import torch
+from comfy.model_management import get_torch_device
 from PIL import Image
-from typing import Optional, List
+
+from ..log import log
+from ..utils import PIL_FILTER_MAP, audioInputDir, tensor2np
+
+try:
+    import librosa
+except ImportError:
+    log.warning("librosa not installed. I/O Audio features will not be available.")
+
+
+class LoadAudio_:
+    """Load an audio file from the input folder (supports upload)"""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "audio": ("AUDIO_UPLOAD",),
+                "sample_rate": ("INT", {"default": 44100}),
+            }
+        }
+
+    RETURN_TYPES = ("AUDIO",)
+    RETURN_NAMES = ("audio",)
+    FUNCTION = "load_audio"
+    CATEGORY = "mtb/audio"
+
+    def load_audio(self, audio: str, sample_rate: int):
+        log.debug(f"Audio file: {audio}")
+        audio_file_path = audioInputDir / audio
+        log.debug(f"Loading audio file: {audio_file_path}")
+        audio_data, _ = librosa.load(audio_file_path.as_posix(), sr=sample_rate)
+        audio_tensor = torch.from_numpy(audio_data).to(get_torch_device())
+        return (audio_tensor.unsqueeze(0).float(),)
 
 
 class ExportWithFfmpeg:
@@ -27,7 +62,8 @@ class ExportWithFfmpeg:
                     ["prores_ks", "libx264", "libx265"],
                     {"default": "prores_ks"},
                 ),
-            }
+            },
+            "hidden": {"prompt": "PROMPT", "extra_pnginfo": "EXTRA_PNGINFO"},
         }
 
     RETURN_TYPES = ("VIDEO",)
@@ -42,9 +78,21 @@ class ExportWithFfmpeg:
         prefix: str,
         format: str,
         codec: str,
+        prompt=None,
+        extra_pnginfo=None,
     ):
+        metadata = {}
         if images.size(0) == 0:
             return ("",)
+
+        if extra_pnginfo is not None:
+            metadata["extra"] = {}
+            for x in extra_pnginfo:
+                metadata["extra"][x] = json.dumps(extra_pnginfo[x])
+
+        if prompt is not None:
+            metadata["prompt"] = json.dumps(prompt)
+
         output_dir = Path(folder_paths.get_output_directory())
         pix_fmt = "rgb48le" if codec == "prores_ks" else "yuv420p"
         file_ext = format
@@ -62,6 +110,15 @@ class ExportWithFfmpeg:
 
         out_path = (output_dir / file_id).as_posix()
 
+        metadata_cmd = []
+
+        if metadata:
+            for k, v in metadata.items():
+                metadata_cmd += [
+                    "-metadata:s:v",
+                    f"{k}='{v if isinstance(v,str) else json.dumps(v)}'",
+                ]
+
         # Prepare the FFmpeg command
         command = [
             "ffmpeg",
@@ -80,6 +137,7 @@ class ExportWithFfmpeg:
             "-",
             "-c:v",
             codec,
+            *metadata_cmd,
             "-r",
             str(fps),
             "-y",
@@ -192,4 +250,4 @@ class SaveGif:
         return {"ui": {"gif": results}}
 
 
-__nodes__ = [SaveGif, ExportWithFfmpeg]
+__nodes__ = [SaveGif, ExportWithFfmpeg, LoadAudio_]

nodes/mask.py

@@ -1,7 +1,8 @@
-from rembg import remove
-from ..utils import pil2tensor, tensor2pil
-from PIL import Image
 import comfy.utils
+from PIL import Image
+from rembg import remove
+
+from ..utils import pil2tensor, tensor2pil
 
 
 class ImageRemoveBackgroundRembg:
@@ -91,6 +92,8 @@ class ImageRemoveBackgroundRembg:
 
             image_on_bg.paste(img_rm, mask=mask)
 
+            image_on_bg = image_on_bg.convert("RGB")
+
             out_img.append(img_rm)
             out_mask.append(mask)
             out_img_on_bg.append(image_on_bg)

nodes/model.py

@@ -0,0 +1,99 @@
+import copy
+
+import torch
+
+from ..log import log
+
+
+class VaeDecode_:
+    """Wrapper for the 2 core decoders but also adding the sd seamless hack, taken from: FlyingFireCo/tiled_ksampler"""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "samples": ("LATENT",),
+                "vae": ("VAE",),
+                "seamless_model": ("BOOLEAN", {"default": False}),
+                "use_tiling_decoder": ("BOOLEAN", {"default": True}),
+                "tile_size": (
+                    "INT",
+                    {"default": 512, "min": 320, "max": 4096, "step": 64},
+                ),
+            }
+        }
+
+    RETURN_TYPES = ("IMAGE",)
+    FUNCTION = "decode"
+
+    CATEGORY = "mtb/decode"
+
+    def decode(
+        self, vae, samples, seamless_model, use_tiling_decoder=True, tile_size=512
+    ):
+        if seamless_model:
+            if use_tiling_decoder:
+                log.error(
+                    "You cannot use seamless mode with tiling decoder together, skipping tiling."
+                )
+                use_tiling_decoder = False
+            for layer in [
+                layer
+                for layer in vae.first_stage_model.modules()
+                if isinstance(layer, torch.nn.Conv2d)
+            ]:
+                layer.padding_mode = "circular"
+        if use_tiling_decoder:
+            return (
+                vae.decode_tiled(
+                    samples["samples"],
+                    tile_x=tile_size // 8,
+                    tile_y=tile_size // 8,
+                ),
+            )
+        else:
+            return (vae.decode(samples["samples"]),)
+
+
+class ModelPatchSeamless:
+    """Uses the stable diffusion 'hack' to infer seamless images by setting the model layers padding mode to circular (experimental)"""
+
+    @classmethod
+    def INPUT_TYPES(cls):
+        return {
+            "required": {
+                "model": ("MODEL",),
+                "tiling": (
+                    "BOOLEAN",
+                    {"default": True},
+                ),  # kept for testing not sure why it should be false
+            }
+        }
+
+    RETURN_TYPES = ("MODEL", "MODEL")
+    RETURN_NAMES = (
+        "Original Model (passthrough)",
+        "Patched Model",
+    )
+    FUNCTION = "hack"
+
+    CATEGORY = "mtb/textures"
+
+    def apply_circular(self, model, enable):
+        for layer in [
+            layer for layer in model.modules() if isinstance(layer, torch.nn.Conv2d)
+        ]:
+            layer.padding_mode = "circular" if enable else "zeros"
+        return model
+
+    def hack(
+        self,
+        model,
+        tiling,
+    ):
+        hacked_model = copy.deepcopy(model)
+        self.apply_circular(hacked_model.model, tiling)
+        return (model, hacked_model)
+
+
+__nodes__ = [ModelPatchSeamless, VaeDecode_]

reqs.txt

@@ -1,7 +0,0 @@
-onnxruntime-gpu==1.15.1
-qrcode[pil]
-rembg==2.0.50
-tensorflow
-facexlib==0.3.0
-insightface==0.7.3
-basicsr==1.4.2

reqs_windows.txt

@@ -1,18 +0,0 @@
-https://github.com/melMass/comfy_mtb/releases/download/v0.1.3/pycocotools-2.0.6-cp310-cp310-win_amd64.whl
-https://github.com/melMass/comfy_mtb/releases/download/v0.1.3/future-0.18.3-py3-none-any.whl
-https://github.com/melMass/comfy_mtb/releases/download/v0.1.3/filterpy-1.4.5-py3-none-any.whl
-https://github.com/melMass/comfy_mtb/releases/download/v0.1.3/easydict-1.10-py3-none-any.whl
-https://github.com/melMass/comfy_mtb/releases/download/v0.1.3/gdown-4.7.1-py3-none-any.whl
-https://github.com/melMass/comfy_mtb/releases/download/v0.1.3/basicsr-1.4.2-py3-none-any.whl
-https://github.com/melMass/comfy_mtb/releases/download/v0.1.3/mmcv-2.0.0-py2.py3-none-any.whl
-https://github.com/melMass/comfy_mtb/releases/download/v0.1.3/insightface-0.7.3-cp310-cp310-win_amd64.whl
-
-onnxruntime-gpu==1.15.1
-qrcode[pil]
-rembg==2.0.50
-# on windows non WSL 2.10 is the last version with GPU support
-tensorflow==2.10.1;
-tb-nightly==2.12.0a20230126; platform_system == "Windows"
-facexlib==0.3.0
-# the old tf version on windows comes with a breaking protobuf version
-protobuf==3.19.6

requirements.txt

@@ -0,0 +1,10 @@
+qrcode[pil]
+onnxruntime-gpu
+requirements-parser
+# opencv-contrib
+rembg
+imageio_ffmpeg
+rich
+rich_argparse
+librosa
+torchaudio

utils.py

@@ -1,16 +1,23 @@
-from PIL import Image
-import numpy as np
-import torch
-from pathlib import Path
-import sys
-from typing import List
-import signal
-from contextlib import suppress
-from queue import Queue, Empty
-import subprocess
-import threading
-import os
+import contextlib
+import functools
 import math
+import os
+import shlex
+import shutil
+import socket
+import subprocess
+import sys
+import uuid
+from pathlib import Path
+from typing import List, Optional, Union
+
+import folder_paths
+import numpy as np
+import requests
+import torch
+from PIL import Image
+
+from .install import pip_map
 
 try:
     from .log import log
@@ -26,7 +33,95 @@ except ImportError:
         log.warn("[comfy mtb] You probably called the file outside a module.")
 
 
+# region SANITY_CHECK Utilities
+
+
+def make_report():
+    pass
+
+
+# endregion
+
+
+# region SERVER Utilities
+class IPChecker:
+    def __init__(self):
+        self.ips = list(self.get_local_ips())
+        log.debug(f"Found {len(self.ips)} local ips")
+        self.checked_ips = set()
+
+    def get_working_ip(self, test_url_template):
+        for ip in self.ips:
+            if ip not in self.checked_ips:
+                self.checked_ips.add(ip)
+                test_url = test_url_template.format(ip)
+                if self._test_url(test_url):
+                    return ip
+        return None
+
+    @staticmethod
+    def get_local_ips(prefix="192.168."):
+        hostname = socket.gethostname()
+        log.debug(f"Getting local ips for {hostname}")
+        for info in socket.getaddrinfo(hostname, None):
+            # Filter out IPv6 addresses if you only want IPv4
+            log.debug(info)
+            # if info[1] == socket.SOCK_STREAM and
+            if info[0] == socket.AF_INET and info[4][0].startswith(prefix):
+                yield info[4][0]
+
+    def _test_url(self, url):
+        try:
+            response = requests.get(url)
+            return response.status_code == 200
+        except Exception:
+            return False
+
+
+@functools.lru_cache(maxsize=1)
+def get_server_info():
+    from comfy.cli_args import args
+
+    ip_checker = IPChecker()
+    base_url = args.listen
+    if base_url == "0.0.0.0":
+        log.debug("Server set to 0.0.0.0, we will try to resolve the host IP")
+        base_url = ip_checker.get_working_ip(f"http://{{}}:{args.port}/history")
+        log.debug(f"Setting ip to {base_url}")
+    return (base_url, args.port)
+
+
+# endregion
+
+
 # region MISC Utilities
+def backup_file(
+    fp: Path,
+    target: Optional[Path] = None,
+    backup_dir: str = ".bak",
+    suffix: Optional[str] = None,
+    prefix: Optional[str] = None,
+):
+    if not fp.exists():
+        raise FileNotFoundError(f"No file found at {fp}")
+
+    backup_directory = target or fp.parent / backup_dir
+    backup_directory.mkdir(parents=True, exist_ok=True)
+
+    stem = fp.stem
+
+    if suffix or prefix:
+        new_stem = f"{prefix or ''}{stem}{suffix or ''}"
+    else:
+        new_stem = f"{stem}_{uuid.uuid4()}"
+
+    backup_file_path = backup_directory / f"{new_stem}{fp.suffix}"
+
+    # Perform the backup
+    shutil.copy(fp, backup_file_path)
+    log.debug(f"File backed up to {backup_file_path}")
+
+
 def hex_to_rgb(hex_color):
     try:
         hex_color = hex_color.lstrip("#")
@@ -52,101 +147,76 @@ def add_path(path, prepend=False):
             sys.path.append(path)
 
 
-def enqueue_output(out, queue):
-    for line in iter(out.readline, b""):
-        queue.put(line)
-    out.close()
+def run_command(cmd, ignored_lines_start=None):
+    if ignored_lines_start is None:
+        ignored_lines_start = []
 
-
-def run_command(cmd):
     if isinstance(cmd, str):
         shell_cmd = cmd
     elif isinstance(cmd, list):
-        shell_cmd = ""
-        for arg in cmd:
-            if isinstance(arg, Path):
-                arg = arg.as_posix()
-            shell_cmd += f"{arg} "
+        shell_cmd = " ".join(
+            arg.as_posix() if isinstance(arg, Path) else shlex.quote(str(arg))
+            for arg in cmd
+        )
     else:
         raise ValueError(
             "Invalid 'cmd' argument. It must be a string or a list of arguments."
         )
 
-    process = subprocess.Popen(
+    try:
+        _run_command(shell_cmd, ignored_lines_start)
+    except subprocess.CalledProcessError as e:
+        print(f"Command failed with return code: {e.returncode}", file=sys.stderr)
+        print(e.stderr.strip(), file=sys.stderr)
+
+    except KeyboardInterrupt:
+        print("Command execution interrupted.")
+
+
+def _run_command(shell_cmd, ignored_lines_start):
+    log.debug(f"Running {shell_cmd}")
+
+    result = subprocess.run(
         shell_cmd,
         stdout=subprocess.PIPE,
         stderr=subprocess.PIPE,
-        universal_newlines=True,
+        text=True,
         shell=True,
+        check=True,
     )
 
-    # Create separate threads to read standard output and standard error streams
-    stdout_queue = Queue()
-    stderr_queue = Queue()
-    stdout_thread = threading.Thread(
-        target=enqueue_output, args=(process.stdout, stdout_queue)
-    )
-    stderr_thread = threading.Thread(
-        target=enqueue_output, args=(process.stderr, stderr_queue)
-    )
-    stdout_thread.daemon = True
-    stderr_thread.daemon = True
-    stdout_thread.start()
-    stderr_thread.start()
+    stdout_lines = result.stdout.strip().split("\n")
+    stderr_lines = result.stderr.strip().split("\n")
 
-    interrupted = False
+    # Print stdout, skipping ignored lines
+    for line in stdout_lines:
+        if not any(line.startswith(ign) for ign in ignored_lines_start):
+            print(line)
 
-    def signal_handler(signum, frame):
-        nonlocal interrupted
-        interrupted = True
-        print("Command execution interrupted.")
+    # Print stderr
+    for line in stderr_lines:
+        print(line, file=sys.stderr)
 
-    # Register the signal handler for keyboard interrupts (SIGINT)
-    signal.signal(signal.SIGINT, signal_handler)
-
-    # Process output from both streams until the process completes or interrupted
-    while not interrupted and (
-        process.poll() is None or not stdout_queue.empty() or not stderr_queue.empty()
-    ):
-        with suppress(Empty):
-            stdout_line = stdout_queue.get_nowait()
-            if stdout_line.strip() != "":
-                print(stdout_line.strip())
-        with suppress(Empty):
-            stderr_line = stderr_queue.get_nowait()
-            if stderr_line.strip() != "":
-                print(stderr_line.strip())
-    return_code = process.returncode
-
-    if return_code == 0 and not interrupted:
-        print("Command executed successfully!")
-    else:
-        if not interrupted:
-            print(f"Command failed with return code: {return_code}")
+    print("Command executed successfully!")
 
 
 # todo use the requirements library
-reqs_map = {
-    "onnxruntime": "onnxruntime-gpu==1.15.1",
-    "basicsr": "basicsr==1.4.2",
-    "rembg": "rembg==2.0.50",
-    "qrcode": "qrcode[pil]",
-}
+reqs_map = {value: key for key, value in pip_map.items()}
+
+import importlib
 
 
 def import_install(package_name):
-    from pip._internal import main as pip_main
+    package_spec = reqs_map.get(package_name, package_name)
 
     try:
-        __import__(package_name)
-    except ImportError:
-        package_spec = reqs_map.get(package_name)
-        if package_spec is None:
-            print(f"Installing {package_name}")
-            package_spec = package_name
+        importlib.import_module(package_name)
 
-        pip_main(["install", package_spec])
-        __import__(package_name)
+    except Exception:  # (ImportError, ModuleNotFoundError):
+        run_command(
+            [Path(sys.executable).as_posix(), "-m", "pip", "install", package_spec]
+        )
+        importlib.import_module(package_name)
 
 
 # endregion
@@ -163,10 +233,13 @@ elif ".venv" in sys.executable:
     comfy_mode = "venv"
 
 # - Get the absolute path of the parent directory of the current script
-here = Path(__file__).parent.resolve()
+here = Path(__file__).parent.absolute()
 
 # - Construct the absolute path to the ComfyUI directory
-comfy_dir = here.parent.parent
+comfy_dir = Path(folder_paths.base_path)
+models_dir = Path(folder_paths.models_dir)
+styles_dir = comfy_dir / "styles"
+audioInputDir = comfy_dir / "input" / "audio"
 
 # - Construct the path to the font file
 font_path = here / "font.ttf"
@@ -193,7 +266,7 @@ PIL_FILTER_MAP = {
 # endregion
 
 
-# region TENSOR UTILITIES
+# region TENSOR Utilities
 def tensor2pil(image: torch.Tensor) -> List[Image.Image]:
     batch_count = image.size(0) if len(image.shape) > 3 else 1
     if batch_count > 1:
@@ -209,14 +282,14 @@ def tensor2pil(image: torch.Tensor) -> List[Image.Image]:
     ]
 
 
-def pil2tensor(image: Image.Image | List[Image.Image]) -> torch.Tensor:
+def pil2tensor(image: Union[Image.Image, List[Image.Image]]) -> torch.Tensor:
     if isinstance(image, list):
         return torch.cat([pil2tensor(img) for img in image], dim=0)
 
     return torch.from_numpy(np.array(image).astype(np.float32) / 255.0).unsqueeze(0)
 
 
-def np2tensor(img_np: np.ndarray | List[np.ndarray]) -> torch.Tensor:
+def np2tensor(img_np: Union[np.ndarray, List[np.ndarray]]) -> torch.Tensor:
     if isinstance(img_np, list):
         return torch.cat([np2tensor(img) for img in img_np], dim=0)
 
@@ -234,6 +307,195 @@ def tensor2np(tensor: torch.Tensor) -> List[np.ndarray]:
     return [np.clip(255.0 * tensor.cpu().numpy().squeeze(), 0, 255).astype(np.uint8)]
 
 
+def pad(img, left, right, top, bottom):
+    pad_width = np.array(((0, 0), (top, bottom), (left, right)))
+    print(f"pad_width: {pad_width}, shape: {pad_width.shape}")  # Debugging line
+    return np.pad(img, pad_width, mode="wrap")
+
+
+def tiles_infer(tiles, ort_session, progress_callback=None):
+    """Infer each tile with the given model. progress_callback will be called with
+    arguments : current tile idx and total tiles amount (used to show progress on
+    cursor in Blender)."""
+
+    out_channels = 3  # normal map RGB channels
+    tiles_nb = tiles.shape[0]
+    pred_tiles = np.empty((tiles_nb, out_channels, tiles.shape[2], tiles.shape[3]))
+
+    for i in range(tiles_nb):
+        if progress_callback != None:
+            progress_callback(i + 1, tiles_nb)
+        pred_tiles[i] = ort_session.run(
+            None, {"input": tiles[i : i + 1].astype(np.float32)}
+        )[0]
+
+    return pred_tiles
+
+
+def generate_mask(tile_size, stride_size):
+    """Generates a pyramidal-like mask. Used for mixing overlapping predicted tiles."""
+
+    tile_h, tile_w = tile_size
+    stride_h, stride_w = stride_size
+    ramp_h = tile_h - stride_h
+    ramp_w = tile_w - stride_w
+
+    mask = np.ones((tile_h, tile_w))
+
+    # ramps in width direction
+    mask[ramp_h:-ramp_h, :ramp_w] = np.linspace(0, 1, num=ramp_w)
+    mask[ramp_h:-ramp_h, -ramp_w:] = np.linspace(1, 0, num=ramp_w)
+    # ramps in height direction
+    mask[:ramp_h, ramp_w:-ramp_w] = np.transpose(
+        np.linspace(0, 1, num=ramp_h)[None], (1, 0)
+    )
+    mask[-ramp_h:, ramp_w:-ramp_w] = np.transpose(
+        np.linspace(1, 0, num=ramp_h)[None], (1, 0)
+    )
+
+    # Assume tiles are squared
+    assert ramp_h == ramp_w
+    # top left corner
+    corner = np.rot90(corner_mask(ramp_h), 2)
+    mask[:ramp_h, :ramp_w] = corner
+    # top right corner
+    corner = np.flip(corner, 1)
+    mask[:ramp_h, -ramp_w:] = corner
+    # bottom right corner
+    corner = np.flip(corner, 0)
+    mask[-ramp_h:, -ramp_w:] = corner
+    # bottom right corner
+    corner = np.flip(corner, 1)
+    mask[-ramp_h:, :ramp_w] = corner
+
+    return mask
+
+
+def corner_mask(side_length):
+    """Generates the corner part of the pyramidal-like mask.
+    Currently, only for square shapes."""
+
+    corner = np.zeros([side_length, side_length])
+
+    for h in range(0, side_length):
+        for w in range(0, side_length):
+            if h >= w:
+                sh = h / (side_length - 1)
+                corner[h, w] = 1 - sh
+            if h <= w:
+                sw = w / (side_length - 1)
+                corner[h, w] = 1 - sw
+
+    return corner - 0.25 * scaling_mask(side_length)
+
+
+def scaling_mask(side_length):
+    scaling = np.zeros([side_length, side_length])
+
+    for h in range(0, side_length):
+        for w in range(0, side_length):
+            sh = h / (side_length - 1)
+            sw = w / (side_length - 1)
+            if h >= w and h <= side_length - w:
+                scaling[h, w] = sw
+            if h <= w and h <= side_length - w:
+                scaling[h, w] = sh
+            if h >= w and h >= side_length - w:
+                scaling[h, w] = 1 - sh
+            if h <= w and h >= side_length - w:
+                scaling[h, w] = 1 - sw
+
+    return 2 * scaling
+
+
+def tiles_merge(tiles, stride_size, img_size, paddings):
+    """Merges the list of tiles into one image. img_size is the original size, before
+    padding."""
+
+    _, tile_h, tile_w = tiles[0].shape
+    pad_left, pad_right, pad_top, pad_bottom = paddings
+    height = img_size[1] + pad_top + pad_bottom
+    width = img_size[2] + pad_left + pad_right
+    stride_h, stride_w = stride_size
+
+    # stride must be even
+    assert (stride_h % 2 == 0) and (stride_w % 2 == 0)
+    # stride must be greater or equal than half tile
+    assert (stride_h >= tile_h / 2) and (stride_w >= tile_w / 2)
+    # stride must be smaller or equal tile size
+    assert (stride_h <= tile_h) and (stride_w <= tile_w)
+
+    merged = np.zeros((img_size[0], height, width))
+    mask = generate_mask((tile_h, tile_w), stride_size)
+
+    h_range = ((height - tile_h) // stride_h) + 1
+    w_range = ((width - tile_w) // stride_w) + 1
+
+    idx = 0
+    for h in range(0, h_range):
+        for w in range(0, w_range):
+            h_from, h_to = h * stride_h, h * stride_h + tile_h
+            w_from, w_to = w * stride_w, w * stride_w + tile_w
+            merged[:, h_from:h_to, w_from:w_to] += tiles[idx] * mask
+            idx += 1
+
+    return merged[:, pad_top:-pad_bottom, pad_left:-pad_right]
+
+
+def tiles_split(img, tile_size, stride_size):
+    """Returns list of tiles from the given image and the padding used to fit the tiles
+    in it. Input image must have dimension C,H,W."""
+    log.debug(f"Splitting img: tile {tile_size}, stride {stride_size} ")
+    tile_h, tile_w = tile_size
+    stride_h, stride_w = stride_size
+    img_h, img_w = img.shape[0], img.shape[1]
+
+    # stride must be even
+    assert (stride_h % 2 == 0) and (stride_w % 2 == 0)
+    # stride must be greater or equal than half tile
+    assert (stride_h >= tile_h / 2) and (stride_w >= tile_w / 2)
+    # stride must be smaller or equal tile size
+    assert (stride_h <= tile_h) and (stride_w <= tile_w)
+
+    # find total height & width padding sizes
+    pad_h, pad_w = 0, 0
+    remainer_h = (img_h - tile_h) % stride_h
+    remainer_w = (img_w - tile_w) % stride_w
+    if remainer_h != 0:
+        pad_h = stride_h - remainer_h
+    if remainer_w != 0:
+        pad_w = stride_w - remainer_w
+
+    # if tile bigger than image, pad image to tile size
+    if tile_h > img_h:
+        pad_h = tile_h - img_h
+    if tile_w > img_w:
+        pad_w = tile_w - img_w
+
+    # pad image, add extra stride to padding to avoid pyramid
+    # weighting leaking onto the valid part of the picture
+    pad_left = pad_w // 2 + stride_w
+    pad_right = pad_left if pad_w % 2 == 0 else pad_left + 1
+    pad_top = pad_h // 2 + stride_h
+    pad_bottom = pad_top if pad_h % 2 == 0 else pad_top + 1
+    img = pad(img, pad_left, pad_right, pad_top, pad_bottom)
+    img_h, img_w = img.shape[1], img.shape[2]
+
+    # extract tiles
+    h_range = ((img_h - tile_h) // stride_h) + 1
+    w_range = ((img_w - tile_w) // stride_w) + 1
+    tiles = np.empty([h_range * w_range, img.shape[0], tile_h, tile_w])
+    idx = 0
+    for h in range(0, h_range):
+        for w in range(0, w_range):
+            h_from, h_to = h * stride_h, h * stride_h + tile_h
+            w_from, w_to = w * stride_w, w * stride_w + tile_w
+            tiles[idx] = img[:, h_from:h_to, w_from:w_to]
+            idx += 1
+
+    return tiles, (pad_left, pad_right, pad_top, pad_bottom)
+
+
 # endregion
 
 
@@ -244,11 +506,9 @@ def download_antelopev2():
     try:
         import gdown
 
-        import folder_paths
-
         log.debug("Loading antelopev2 model")
 
-        dest = Path(folder_paths.models_dir) / "insightface"
+        dest = get_model_path("insightface")
         archive = dest / "antelopev2.zip"
         final_path = dest / "models" / "antelopev2"
         if not final_path.exists():
@@ -275,6 +535,33 @@ def download_antelopev2():
         raise e
 
 
+def get_model_path(fam, model=None):
+    log.debug(f"Requesting {fam} with model {model}")
+    res = None
+    if model:
+        res = folder_paths.get_full_path(fam, model)
+    else:
+        # this one can raise errors...
+        with contextlib.suppress(KeyError):
+            res = folder_paths.get_folder_paths(fam)
+
+    if res:
+        if isinstance(res, list):
+            if len(res) > 1:
+                log.warning(
+                    f"Found multiple match, we will pick the first {res[0]}\n{res}"
+                )
+            res = res[0]
+        res = Path(res)
+        log.debug(f"Resolved model path from folder_paths: {res}")
+    else:
+        res = models_dir / fam
+        if model:
+            res /= model
+
+    return res
+
+
 # endregion
 
 
@@ -299,9 +586,6 @@ def create_uv_map_tensor(width=512, height=512):
 
 # region ANIMATION Utilities
 def apply_easing(value, easing_type):
-    if value < 0 or value > 1:
-        raise ValueError("The value should be between 0 and 1.")
-
     if easing_type == "Linear":
         return value
 

web/comfy_shared.js

@@ -7,7 +7,7 @@
  *
  */
 
-import { app } from '/scripts/app.js'
+import { app } from '../../scripts/app.js'
 
 export const log = (...args) => {
   if (window.MTB?.DEBUG) {
@@ -18,6 +18,30 @@ export const log = (...args) => {
 //- WIDGET UTILS
 export const CONVERTED_TYPE = 'converted-widget'
 
+export const hasWidgets = (node) => {
+  if (!node.widgets || !node.widgets?.[Symbol.iterator]) {
+    return false
+  }
+  return true
+}
+
+export const cleanupNode = (node) => {
+  if (!hasWidgets(node)) {
+    return
+  }
+
+  for (const w of node.widgets) {
+    if (w.canvas) {
+      w.canvas.remove()
+    }
+    if (w.inputEl) {
+      w.inputEl.remove()
+    }
+    // calls the widget remove callback
+    w.onRemoved?.()
+  }
+}
+
 export function offsetDOMWidget(
   widget,
   ctx,
@@ -68,13 +92,34 @@ export function getWidgetType(config) {
   }
   return { type, linkType }
 }
+export const setupDynamicConnections = (nodeType, prefix, inputType) => {
+  const onNodeCreated = nodeType.prototype.onNodeCreated
+  nodeType.prototype.onNodeCreated = function () {
+    const r = onNodeCreated ? onNodeCreated.apply(this, arguments) : undefined
+    this.addInput(`${prefix}_1`, inputType)
+    return r
+  }
 
+  const onConnectionsChange = nodeType.prototype.onConnectionsChange
+  nodeType.prototype.onConnectionsChange = function (
+    type,
+    index,
+    connected,
+    link_info
+  ) {
+    const r = onConnectionsChange
+      ? onConnectionsChange.apply(this, arguments)
+      : undefined
+    dynamic_connection(this, index, connected, `${prefix}_`, inputType)
+  }
+}
 export const dynamic_connection = (
   node,
   index,
   connected,
   connectionPrefix = 'input_',
-  connectionType = 'PSDLAYER'
+  connectionType = 'PSDLAYER',
+  nameArray = []
 ) => {
   // remove all non connected inputs
   if (!connected && node.inputs.length > 1) {
@@ -90,22 +135,24 @@ export const dynamic_connection = (
 
     // make inputs sequential again
     for (let i = 0; i < node.inputs.length; i++) {
-      node.inputs[i].label = `${connectionPrefix}${i + 1}`
+      const name =
+        i < nameArray.length ? nameArray[i] : `${connectionPrefix}${i + 1}`
+      node.inputs[i].label = name
+      node.inputs[i].name = name
     }
   }
 
   // add an extra input
   if (node.inputs[node.inputs.length - 1].link != undefined) {
-    log(
-      `Adding input ${node.inputs.length + 1} (${connectionPrefix}${
-        node.inputs.length + 1
-      })`
-    )
+    const nextIndex = node.inputs.length
+    const name =
+      nextIndex < nameArray.length
+        ? nameArray[nextIndex]
+        : `${connectionPrefix}${nextIndex + 1}`
 
-    node.addInput(
-      `${connectionPrefix}${node.inputs.length + 1}`,
-      connectionType
-    )
+    log(`Adding input ${nextIndex + 1} (${name})`)
+
+    node.addInput(name, connectionType)
   }
 }
 

web/debug.js

@@ -7,17 +7,35 @@
  *
  */
 
-import { app } from '/scripts/app.js'
-import * as shared from '/extensions/mtb/comfy_shared.js'
-import { log } from '/extensions/mtb/comfy_shared.js'
-import { MtbWidgets } from '/extensions/mtb/mtb_widgets.js'
+import { app } from '../../scripts/app.js'
+
+import * as shared from './comfy_shared.js'
+import { log } from './comfy_shared.js'
+import { MtbWidgets } from './mtb_widgets.js'
 
 // TODO: respect inputs order...
 
+function escapeHtml(unsafe) {
+  return unsafe
+    .replace(/&/g, '&')
+    .replace(/</g, '&lt;')
+    .replace(/>/g, '&gt;')
+    .replace(/"/g, '&quot;')
+    .replace(/'/g, '&#039;')
+}
 app.registerExtension({
   name: 'mtb.Debug',
   async beforeRegisterNodeDef(nodeType, nodeData, app) {
     if (nodeData.name === 'Debug (mtb)') {
+      const onNodeCreated = nodeType.prototype.onNodeCreated
+      nodeType.prototype.onNodeCreated = function () {
+        const r = onNodeCreated
+          ? onNodeCreated.apply(this, arguments)
+          : undefined
+        this.addInput(`anything_1`, '*')
+        return r
+      }
+
       const onConnectionsChange = nodeType.prototype.onConnectionsChange
       nodeType.prototype.onConnectionsChange = function (
         type,
@@ -57,15 +75,18 @@ app.registerExtension({
           // const pos = this.widgets.findIndex((w) => w.name === "anything_1");
           // if (pos !== -1) {
           for (let i = 0; i < this.widgets.length; i++) {
-            this.widgets[i].onRemoved?.()
+            if (this.widgets[i].name !== 'output_to_console') {
+              this.widgets[i].onRemoved?.()
+            }
           }
-          this.widgets.length = 0
+          this.widgets.length = 1
         }
         let widgetI = 1
+
         if (message.text) {
           for (const txt of message.text) {
             const w = this.addCustomWidget(
-              MtbWidgets.DEBUG_STRING(`${prefix}_${widgetI}`, txt)
+              MtbWidgets.DEBUG_STRING(`${prefix}_${widgetI}`, escapeHtml(txt))
             )
             w.parent = this
             widgetI++
@@ -81,15 +102,17 @@ app.registerExtension({
           }
           // this.onResize?.(this.size);
           // this.resize?.(this.size)
-          this.setSize(this.computeSize())
         }
 
+        this.setSize(this.computeSize())
+
         this.onRemoved = function () {
           // When removing this node we need to remove the input from the DOM
           for (let y in this.widgets) {
             if (this.widgets[y].canvas) {
               this.widgets[y].canvas.remove()
             }
+            shared.cleanupNode(this)
             this.widgets[y].onRemoved?.()
           }
         }

web/imageFeed.js

@@ -9,8 +9,8 @@
 
 // forked from pysssss's imageFeed.js
 
-import { api } from '/scripts/api.js'
-import { app } from '/scripts/app.js'
+import { api } from '../../scripts/api.js'
+import { app } from '../../scripts/app.js'
 
 const styles = {
   lighbox: {

web/mtb_widgets.js

@@ -7,13 +7,92 @@
  *
  */
 
-import { app } from '/scripts/app.js'
-import parseCss from '/extensions/mtb/extern/parse-css.js'
-import * as shared from '/extensions/mtb/comfy_shared.js'
-import { log } from '/extensions/mtb/comfy_shared.js'
-import { api } from '/scripts/api.js'
+import { app } from '../../scripts/app.js'
+import { api } from '../../scripts/api.js'
 
-const newTypes = [, /*'BOOL'*/ 'COLOR', 'BBOX']
+import parseCss from './extern/parse-css.js'
+import * as shared from './comfy_shared.js'
+import { log } from './comfy_shared.js'
+
+const newTypes = [, /*'BOOL'*/ 'COLOR', 'BBOX', 'AUDIO_UPLOAD']
+
+const withFont = (ctx, font, cb) => {
+  const oldFont = ctx.font
+  ctx.font = font
+  cb()
+  ctx.font = oldFont
+}
+
+const calculateTextDimensions = (ctx, value, width, fontSize = 16) => {
+  const words = value.split(' ')
+  const lines = []
+  let currentLine = ''
+  for (const word of words) {
+    const testLine = currentLine.length === 0 ? word : `${currentLine} ${word}`
+    const testWidth = ctx.measureText(testLine).width
+    if (testWidth > width) {
+      lines.push(currentLine)
+      currentLine = word
+    } else {
+      currentLine = testLine
+    }
+  }
+  if (lines.length === 0) lines.push(value)
+  const textHeight = (lines.length + 1) * fontSize
+  const maxLineWidth = lines.reduce(
+    (maxWidth, line) => Math.max(maxWidth, ctx.measureText(line).width),
+    0
+  )
+  return { textHeight, maxLineWidth }
+}
+function addPlaybackWidget(node, name, url) {
+  let isTick = true
+  const audio = new Audio(url)
+  const slider = node.addWidget(
+    'slider',
+    'loading',
+    0,
+    (v) => {
+      if (!isTick) {
+        audio.currentTime = v
+      }
+      isTick = false
+    },
+    {
+      min: 0,
+      max: 0,
+    }
+  )
+
+  const button = node.addWidget('button', `Play ${name}`, 'play', () => {
+    try {
+      if (audio.paused) {
+        audio.play()
+        button.name = `Pause ${name}`
+      } else {
+        audio.pause()
+        button.name = `Play ${name}`
+      }
+    } catch (error) {
+      alert(error)
+    }
+    app.canvas.setDirty(true)
+  })
+  audio.addEventListener('timeupdate', () => {
+    isTick = true
+    slider.value = audio.currentTime
+    app.canvas.setDirty(true)
+  })
+  audio.addEventListener('ended', () => {
+    button.name = `Play ${name}`
+    app.canvas.setDirty(true)
+  })
+  audio.addEventListener('loadedmetadata', () => {
+    slider.options.max = audio.duration
+    slider.name = `(${audio.duration})`
+    app.canvas.setDirty(true)
+  })
+}
 
 export const MtbWidgets = {
   BBOX: (key, val) => {
@@ -316,46 +395,22 @@ export const MtbWidgets = {
         // const [cw, ch] = this.computeSize(widgetWidth)
         shared.offsetDOMWidget(this, ctx, node, widgetWidth, widgetY, height)
       },
-      computeSize: function (width) {
-        const value = this.inputEl.innerHTML
-        if (!value) {
+      computeSize(width) {
+        if (!this.value) {
           return [32, 32]
         }
         if (!width) {
-          log(`No width ${this.parent.size}`)
+          console.debug(`No width ${this.parent.size}`)
         }
-
-        const oldFont = app.ctx.font
-        app.ctx.font = `${fontSize}px monospace`
-
-        const words = value.split(' ')
-        const lines = []
-        let currentLine = ''
-        for (const word of words) {
-          const testLine =
-            currentLine.length === 0 ? word : `${currentLine} ${word}`
-
-          const testWidth = app.ctx.measureText(testLine).width
-
-          if (testWidth > width) {
-            lines.push(currentLine)
-            currentLine = word
-          } else {
-            currentLine = testLine
-          }
-        }
-        app.ctx.font = oldFont
-        if (lines.length === 0) lines.push(currentLine)
-
-        const textHeight = (lines.length + 1) * fontSize
-
-        const maxLineWidth = lines.reduce(
-          (maxWidth, line) =>
-            Math.max(maxWidth, app.ctx.measureText(line).width),
-          0
+        let dimensions
+        withFont(app.ctx, `${fontSize}px monospace`, () => {
+          dimensions = calculateTextDimensions(app.ctx, this.value, width)
+        })
+        const widgetWidth = Math.max(
+          width || this.width || 32,
+          dimensions.maxLineWidth
         )
-        const widgetWidth = Math.max(width || this.width || 32, maxLineWidth)
-        const widgetHeight = textHeight * 1.5
+        const widgetHeight = dimensions.textHeight * 1.5
         return [widgetWidth, widgetHeight]
       },
       onRemoved: function () {
@@ -363,28 +418,139 @@ export const MtbWidgets = {
           this.inputEl.remove()
         }
       },
+      get value() {
+        return this.inputEl.innerHTML
+      },
+      set value(val) {
+        this.inputEl.innerHTML = val
+        this.parent?.setSize?.(this.parent?.computeSize())
+      },
+    }
+
+    w.inputEl = document.createElement('p')
+    w.inputEl.style = `
+      text-align: center;
+      font-size: ${fontSize}px;
+      color: var(--input-text);
+      line-height: 0;
+      font-family: monospace;
+    `
+    w.value = val
+    document.body.appendChild(w.inputEl)
+
+    return w
+  },
+
+  AUDIO_UPLOAD: function (name, val) {
+    const w = {
+      name,
+      type: 'audio_upload',
+      value: val,
+      draw: function (ctx, node, widgetWidth, widgetY, height) {
+        const [cw, ch] = this.computeSize(widgetWidth)
+        shared.offsetDOMWidget(this, ctx, node, widgetWidth, widgetY, ch)
+      },
+      computeSize: function (width) {
+        if (width) {
+          return [width, 64]
+        }
+        return [128, 128]
+      },
+      onRemoved: function () {
+        if (this.inputEl) {
+          this.inputEl.remove()
+        }
+      },
     }
 
-    Object.defineProperty(w, 'value', {
-      get() {
-        return this.inputEl.innerHTML
-      },
-      set(value) {
-        this.inputEl.innerHTML = value
-        this.parent?.setSize?.(this.parent?.computeSize())
+    const uploadFile = async (file, node) => {
+      try {
+        const body = new FormData()
+        body.append('name', 'loadAudio')
+        body.append('args', file)
+        const loadAudio = await api.fetchApi('/mtb/actions', {
+          method: 'POST',
+          body,
+        })
+
+        if (loadAudio.status === 200) {
+          const { result } = await loadAudio.json()
+          console.log('received from server', result)
+          console.log(
+            `Getting file /mtb/audio?filename=${encodeURIComponent(
+              result.name
+            )}`
+          )
+
+          w.value = result.name
+          addPlaybackWidget(
+            node,
+            result.name,
+            `/mtb/audio?filename=${encodeURIComponent(result.name)}`
+          )
+        } else {
+          alert(loadAudio.status + ' -' + loadAudio.statusText)
+        }
+        // if (resp.status === 200) {
+        //   const { name } = await resp.json()
+        //   pathWidget.value = name
+        //   addPlaybackWidget(
+        //     node,
+        //     name,
+        //     `/samplediffusion/audio?filename=${encodeURIComponent(name)}`
+        //   )
+        // } else {
+        //   alert(resp.status + ' - ' + resp.statusText)
+        // }
+      } catch (error) {
+        alert(error)
+        throw error
+      }
+    }
+
+    w.inputEl = document.createElement('div')
+    const hidden_input = document.createElement('input')
+    const label = document.createElement('label')
+
+    const uniqueId = 'input_' + Date.now()
+    Object.assign(hidden_input, {
+      type: 'file',
+      accept: 'audio/mpeg,audio/wav,audio/x-wav',
+      id: uniqueId,
+      style: `
+      width: 0.1px;
+      height: 0.1px;
+      opacity: 0;
+      overflow: hidden;
+      position: absolute;
+      z-index: -1;
+
+      `,
+      onchange: async () => {
+        if (hidden_input.files.length) {
+          console.log(hidden_input.files[0])
+          await uploadFile(hidden_input.files[0], this)
+        }
       },
     })
 
-    w.inputEl = document.createElement('p')
-    w.inputEl.style.textAlign = 'center'
-    w.inputEl.style.fontSize = `${fontSize}px`
-    w.inputEl.style.color = 'var(--input-text)'
-    w.inputEl.style.lineHeight = 0
-
-    w.inputEl.style.fontFamily = 'monospace'
-    w.value = val
+    Object.assign(label, {
+      htmlFor: uniqueId,
+    })
+    label.textContent = 'Upload Audio File'
+    label.style = `
+    font-size: 1.25em;
+    font-weight: 700;
+    font-family: monospace;
+    padding:0.5em;
+    border-radius: 5px;
+    color: white;
+    background-color: #1e1e1e;
+    display: inline-block;
+    `
     document.body.appendChild(w.inputEl)
-
+    w.inputEl.appendChild(hidden_input)
+    w.inputEl.appendChild(label)
     return w
   },
 }
@@ -470,6 +636,20 @@ const mtb_widgets = {
           minHeight: 30,
         }
       },
+      AUDIO_UPLOAD: (node, inputName, inputData, app) => {
+        console.debug('Registering audio')
+        return {
+          widget: node.addCustomWidget(
+            MtbWidgets.AUDIO_UPLOAD.bind(node)(
+              inputName,
+              inputData[1]?.default || ''
+            )
+          ),
+          minWidth: 150,
+          minHeight: 30,
+        }
+      },
+
       // BBOX: (node, inputName, inputData, app) => {
       //     console.debug("Registering bbox")
       //     return {
@@ -489,6 +669,10 @@ const mtb_widgets = {
   async beforeRegisterNodeDef(nodeType, nodeData, app) {
     // const rinputs = nodeData.input?.required
 
+    if (!nodeData.name.endsWith('(mtb)')) {
+      return
+    }
+
     let has_custom = false
     if (nodeData.input && nodeData.input.required) {
       for (const i of Object.keys(nodeData.input.required)) {
@@ -512,12 +696,7 @@ const mtb_widgets = {
 
         this.onRemoved = function () {
           // When removing this node we need to remove the input from the DOM
-          for (const w of this.widgets) {
-            if (w.canvas) {
-              w.canvas.remove()
-            }
-            w.onRemoved?.()
-          }
+          shared.cleanupNode(this)
         }
         return r
       }
@@ -650,22 +829,14 @@ const mtb_widgets = {
                 i++
               }
             }
-            this.setSize?.(this.computeSize())
-            return r
-          }
-
-          const onRemoved = nodeType.prototype.onRemoved
-          nodeType.prototype.onRemoved = function (message) {
-            const r = onRemoved ? onRemoved.apply(this, message) : undefined
-            if (!this.widgets) return r
-            for (const w of this.widgets) {
-              if (w.canvas) {
-                w.canvas.remove()
-              }
-              w.onRemoved?.()
+            const onRemoved = this.onRemoved
+            this.onRemoved = () => {
+              shared.cleanupNode(this)
+              return onRemoved?.()
             }
-            return r
           }
+          this.setSize?.(this.computeSize())
+          return r
         }
 
         break
@@ -733,12 +904,7 @@ const mtb_widgets = {
           })
 
           this.onRemoved = () => {
-            for (const w of this.widgets) {
-              if (w.canvas) {
-                w.canvas.remove()
-              }
-              w.onRemoved?.()
-            }
+            shared.cleanupNode(this)
             app.canvas.setDirty(true)
           }
 
@@ -890,6 +1056,66 @@ const mtb_widgets = {
 
         break
       }
+      case 'Stack Images (mtb)':
+      case 'Concat Images (mtb)': {
+        shared.setupDynamicConnections(nodeType, 'image', 'IMAGE')
+
+        break
+      }
+      case 'Batch Float Assemble (mtb)':
+      case 'Plot Batch Float (mtb)': {
+        shared.setupDynamicConnections(nodeType, 'floats', 'FLOATS')
+        break
+      }
+      case 'Batch Merge (mtb)': {
+        shared.setupDynamicConnections(nodeType, 'batches', 'IMAGE')
+
+        break
+      }
+      case 'Math Expression (mtb)': {
+        const onNodeCreated = nodeType.prototype.onNodeCreated
+        nodeType.prototype.onNodeCreated = function () {
+          const r = onNodeCreated
+            ? onNodeCreated.apply(this, arguments)
+            : undefined
+          this.addInput(`x`, '*')
+          return r
+        }
+
+        const onConnectionsChange = nodeType.prototype.onConnectionsChange
+        nodeType.prototype.onConnectionsChange = function (
+          type,
+          index,
+          connected,
+          link_info
+        ) {
+          const r = onConnectionsChange
+            ? onConnectionsChange.apply(this, arguments)
+            : undefined
+          shared.dynamic_connection(this, index, connected, 'var_', '*', [
+            'x',
+            'y',
+            'z',
+          ])
+
+          //- infer type
+          if (link_info) {
+            const fromNode = this.graph._nodes.find(
+              (otherNode) => otherNode.id == link_info.origin_id
+            )
+            const type = fromNode.outputs[link_info.origin_slot].type
+            this.inputs[index].type = type
+            // this.inputs[index].label = type.toLowerCase()
+          }
+          //- restore dynamic input
+          if (!connected) {
+            this.inputs[index].type = '*'
+            this.inputs[index].label = `number_${index + 1}`
+          }
+        }
+
+        break
+      }
       case 'Save Tensors (mtb)': {
         const onDrawBackground = nodeType.prototype.onDrawBackground
         nodeType.prototype.onDrawBackground = function (ctx, canvas) {

web/notify.js

@@ -7,7 +7,7 @@
  *
  */
 
-import { app } from '/scripts/app.js'
+import { app } from '../../scripts/app.js'
 
 const log = (...args) => {
   if (window.MTB?.TRACE) {