docs(readme): update project overview
This commit is contained in:
Artificial Sweetener
@@ -1,192 +1,198 @@
|
||||
# SimpleSyrup
|
||||
|
||||
[](LICENSE) [](https://registry.comfy.org/publishers/artificialsweetener/nodes/SimpleSyrup) [](https://registry.comfy.org/publishers/artificialsweetener/nodes/SimpleSyrup) [](https://www.python.org/downloads/)
|
||||
[](https://registry.comfy.org/publishers/artificialsweetener/nodes/SimpleSyrup) [](https://registry.comfy.org/publishers/artificialsweetener/nodes/SimpleSyrup) [](https://www.python.org/downloads/) [](LICENSE)
|
||||
|
||||
**SimpleSyrup** is a ComfyUI node pack that grew out of moving my A1111/WebUI image workflows into ComfyUI graphs.
|
||||
**SimpleSyrup** is a ComfyUI node pack that grew out of moving my A1111/WebUI image workflows into ComfyUI.
|
||||
|
||||
The WebUI side shows up in the things I kept reaching for: ADetailer-style inline `[SEP]` prompt batches, tiled diffusion, familiar checkpoint loader controls, CLIP skip where WebUI users expect it, and sampler/scheduler extras. The Comfy side matters just as much: the detailers are modeled heavily on ComfyUI Impact Pack's SEGS workflow, and the utility nodes are built for graph readability, ordered data, and explicit runtime behavior.
|
||||
The WebUI influence shows up all over the pack. I missed ADetailer's inline prompt batches, tiled diffusion, CLIP skip beside my checkpoint, and some of the sampler behavior I was used to. I also wanted the regional pieces to work with Impact Pack SEGS so I could use the same regions across detectors and detailers.
|
||||
|
||||
SimpleSyrup pulls from a few different places:
|
||||
The pack now covers model loading, regional prompting and segmentation, high-resolution sampling, image and mask utilities, tagging, and the smaller pieces I need to keep those workflows readable.
|
||||
|
||||
- Moving from **A1111/WebUI** to **ComfyUI** is the reason this pack brings tiled diffusion, familiar checkpoint loader control grouping, ADetailer-style prompt splitting, and A1111-flavored sampler behavior into node graphs.
|
||||
- **ComfyUI Impact Pack** is the main influence for the SEGS and detailer shape: detectors create SEGS, detailers sample cropped areas, masks are feathered, and results are composited back into the source image.
|
||||
- **ADetailer** is the influence for inline `[SEP]` per-segment prompt batches.
|
||||
- The remaining utility pieces cover practical graph needs: GPU Lanczos resizing through TorchLanc, latent provenance helpers, and smaller nodes for image, prompt, and conditioning workflows.
|
||||
[**SugarSubstitute**](https://github.com/Artificial-Sweetener/SugarSubstitute), my native desktop front-end for ComfyUI, can use Cubes built from any ComfyUI nodes available in its connected environment. Its first-party [**Base-Cubes**](https://github.com/Artificial-Sweetener/Base-Cubes) pack uses SimpleSyrup for model loading, regional prompts, segmentation, high-resolution sampling, and other graph work. You can also install SimpleSyrup on its own and use the nodes in normal ComfyUI workflows.
|
||||
|
||||
## Highlights
|
||||
|
||||
- Impact-compatible SEGS detection, sorting, combining, tiling, and detailing.
|
||||
- Detailer nodes modeled heavily on ComfyUI Impact Pack's SEGS workflow.
|
||||
- ADetailer-style inline `[SEP]` per-segment prompt batches.
|
||||
- MultiDiffusion, Mixture of Diffusers, and regional MultiDiffusion sampling paths.
|
||||
- A WebUI-familiar checkpoint loader with CLIP skip and VAE override controls.
|
||||
- A Simple Anima loader that keeps Anima's model, VAE, dtype, and device controls together.
|
||||
- Loaders for SAM, GroundingDINO, ViTMatte, Ultralytics, and WD14.
|
||||
- Tile, tag, and guide workflows that can generate conditioning from WD14 or a configured external vision LLM.
|
||||
- KSampler extras including A1111-style Euler ancestral behavior, AYS, GITS, automatic A1111 scheduling, and beta57.
|
||||
- GPU Lanczos resizing through [TorchLanc](https://github.com/Artificial-Sweetener/TorchLanc), with batch and mask handling.
|
||||
- Normal and tiled VAE encode/decode option nodes with spatial and temporal tiling controls.
|
||||
- Expandable SEGS and conditioning batch helpers for larger regional graphs.
|
||||
- Provenance-aware latent helpers for recovering the latent behind an unmodified decoded image.
|
||||
- Settings-backed model dropdowns that can show known downloadable models or only locally installed ones.
|
||||
- Loaders that keep checkpoints, Anima, FLUX.1, and FLUX.2 models together with the text encoders, VAE, precision, and device choices they need.
|
||||
- My original Contextual Diffusion method for coherent high-resolution edits, plus MultiDiffusion and Mixture of Diffusers tiled sampling.
|
||||
- Impact-compatible SEGS detection, segmentation, interactive preview, batching, and detailers.
|
||||
- ADetailer-style `[SEP]` prompt batches, masked conditioning, and regional samplers, with optional Prompt Control scheduling and LoRA hooks.
|
||||
- WD14 and external vision LLM tagging that stays aligned with the right regions.
|
||||
- Ordered image and mask loading, GPU Lanczos resizing, tiled VAE options, and provenance-aware latent tools.
|
||||
- WebUI-inspired sampler and scheduler extras including A1111 Euler ancestral behavior, AYS, GITS, `automatic_a1111`, and beta57.
|
||||
|
||||
## Installation
|
||||
## Contents
|
||||
|
||||
**Recommended: install through ComfyUI Manager**
|
||||
- [Install](#install)
|
||||
- [Model loading](#model-loading)
|
||||
- [Large images and high-resolution edits](#large-images-and-high-resolution-edits)
|
||||
- [Contextual Diffusion](#contextual-diffusion)
|
||||
- [Tiled Diffusion](#tiled-diffusion)
|
||||
- [SEGS, detailers, and regional prompts](#segs-detailers-and-regional-prompts)
|
||||
- [Tagging images and regions](#tagging-images-and-regions)
|
||||
- [Images, masks, latents, and sampler extras](#images-masks-latents-and-sampler-extras)
|
||||
- [Settings and optional integrations](#settings-and-optional-integrations)
|
||||
- [License, acknowledgements, and research](#license-acknowledgements-and-research)
|
||||
|
||||
Open **Manager** from the ComfyUI toolbar, click **Custom Nodes Manager**, search for **SimpleSyrup**, and click **Install**. Restart ComfyUI after installation.
|
||||
## Install
|
||||
|
||||
**Manual install**
|
||||
### ComfyUI Manager
|
||||
|
||||
If you would rather install it yourself, clone this repo into `ComfyUI/custom_nodes/`, activate your **ComfyUI venv**, and install this node pack's requirements.
|
||||
Open Manager and search the **Node Pack** list for **SimpleSyrup**, then select it and click **Install**. Restart ComfyUI when it finishes.
|
||||
|
||||
ComfyUI still has two Manager interfaces in circulation. In the legacy interface, the search is under **Custom Nodes Manager**.
|
||||
|
||||
### Manual install
|
||||
|
||||
Clone the repository into `ComfyUI/custom_nodes/` and install the requirements with the same Python environment that runs ComfyUI.
|
||||
|
||||
For a normal Windows virtual environment:
|
||||
|
||||
```powershell
|
||||
cd ComfyUI\custom_nodes
|
||||
Set-Location ComfyUI\custom_nodes
|
||||
git clone https://github.com/Artificial-Sweetener/SimpleSyrup.git
|
||||
cd SimpleSyrup
|
||||
pip install -r requirements.txt
|
||||
Set-Location SimpleSyrup
|
||||
..\..\venv\Scripts\python.exe -m pip install -r requirements.txt
|
||||
```
|
||||
|
||||
ComfyUI already provides the heavy shared runtime stack, including PyTorch. SimpleSyrup adds the packages it needs for specific features, including TorchLanc, Ultralytics, ONNX Runtime, Segment Anything, and Hugging Face download helpers.
|
||||
For ComfyUI Windows Portable, run this from the portable installation folder after cloning the repository:
|
||||
|
||||
## The Nodes
|
||||
```powershell
|
||||
.\python_embeded\python.exe -m pip install -r .\ComfyUI\custom_nodes\SimpleSyrup\requirements.txt
|
||||
```
|
||||
|
||||
SimpleSyrup is organized around workflow jobs, not socket types.
|
||||
Restart ComfyUI after installation. SimpleSyrup uses ComfyUI's v3 extension API, so you need a current version of ComfyUI for the nodes to appear.
|
||||
|
||||
### Impact-Style SEGS Workflows
|
||||
ComfyUI already supplies PyTorch and the rest of the shared runtime. SimpleSyrup installs the packages used by its own features, including [TorchLanc](https://github.com/Artificial-Sweetener/TorchLanc), Ultralytics, ONNX Runtime, Segment Anything, and the Hugging Face download helpers.
|
||||
|
||||
SimpleSyrup speaks the Impact Pack SEGS shape on purpose. It can read Impact-style SEGS, sort them, combine them, tile them, and emit SEGS payloads that Impact-style consumers can read.
|
||||
## Model loading
|
||||
|
||||
The detailer nodes are modeled heavily on **ComfyUI Impact Pack**. Detectors create SEGS, SEGS choose the crop areas, crops are sampled, masks are feathered, and the results are composited back into the source image.
|
||||
Loading a checkpoint used to feel like choosing one file. Newer model families can mean a diffusion model, several text encoders, a VAE, and then the precision and device choices for all of them. I made the SimpleSyrup loaders so I could deal with that setup once and get on with the workflow.
|
||||
|
||||
- **Prompt SEGS w/ SAM** uses GroundingDINO to find prompt-matched boxes, SAM to segment them, optional negative prompting to subtract unwanted areas, and optional ViTMatte refinement to clean up mask edges. It returns both SEGS and a combined mask.
|
||||
- **Detect SEGS w/ Ultralytics** runs bbox or segmentation detection, filters by threshold and size, supports label filtering, and returns Impact-compatible SEGS plus a combined mask.
|
||||
- **Batch SEGS** combines multiple SEGS inputs into one ordered SEGS payload.
|
||||
- **Detail SEGS by Scale Factor** upscales each SEG crop, samples it, downsizes it back, and composites it into the original image with feathering and optional denoise masks.
|
||||
- **Detail SEGS by Scale Factor w/ Tiled Diffusion** uses the same crop/detail idea, but samples large crops through SimpleSyrup's tiled diffusion path.
|
||||
- **Detail SEGS as Regions** runs one regional MultiDiffusion pass over the image and pairs every SEG with its matching `CONDITIONING_BATCH` entry.
|
||||
**Simple Load Checkpoint** is the normal checkpoint loader. It has an optional VAE override and keeps CLIP skip beside the model controls where I expect to find it.
|
||||
|
||||
The regional node is different from the per-crop detailers. It samples one full-image latent with regional conditioning, using the global prompt for full-image context while each SEG gets its own positive conditioning.
|
||||
**Simple Load Anima** loads Anima with its Qwen text encoder and Qwen image VAE. You can select every part yourself. If you don't want to, the automatic choices can find and download the known checksum-pinned support files.
|
||||
|
||||
### Per-Segment Prompt Batches
|
||||
**Simple Load FLUX** handles FLUX.1 with CLIP-L, T5-XXL, and its VAE. **Simple Load FLUX.2** inspects the selected diffusion model and chooses the matching text encoder family for FLUX.2 dev, Klein 4B, or Klein 9B/KV conditioning. Both loaders can find or download their known text encoders and VAEs with visible Comfy progress.
|
||||
|
||||
SimpleSyrup layers the ADetailer habit I missed from WebUI on top of the Impact-style detailer shape: writing per-segment prompt batches inline with `[SEP]`. Separators can carry an organizational name, such as `[SEP|Sky]`; the name is discarded during parsing, and prompt-to-SEG matching remains positional.
|
||||
The FLUX loaders only download those revision-locked, checksum-pinned support files. You still install and select the diffusion model. They also expose manual component selection, diffusion weight precision, and text-encoder device placement. Moving text encoding to the CPU can save VRAM, although it will take longer.
|
||||
|
||||
Those prompts become an ordered `CONDITIONING_BATCH`, so prompt 1 stays matched to SEG 1, prompt 2 stays matched to SEG 2, and so on. This keeps the graph readable when each detected item needs its own prompt.
|
||||
## Large images and high-resolution edits
|
||||
|
||||
- **Encode Prompt Batch** splits prompt text with `[SEP]` or `[SEP|name]` and encodes ordered positive and negative `CONDITIONING_BATCH` values.
|
||||
- **Conditioning Batch Start** and **Conditioning Batch Append** build ordered conditioning batches for per-segment and regional workflows.
|
||||
- **Batch Region Conditioning** combines normal `CONDITIONING` values and existing `CONDITIONING_BATCH` values into one ordered regional batch.
|
||||
- **Encode Prompt Batch w/ Prompt Control** is exported when Prompt Control is installed. It keeps the `[SEP]` and `[SEP|name]` batching workflow while letting Prompt Control handle prompt scheduling and encoding.
|
||||
- **Schedule & Encode Prompts** is also exported when Prompt Control is installed. It schedules Prompt Control LoRA tags, encodes positive and negative prompts, and returns normal conditioning or SimpleSyrup batches depending on whether `[SEP]` or `[SEP|name]` is used.
|
||||
Tiled diffusion handles the obvious large-image problem: sometimes the latent is too big to evaluate all at once. There is a worse version. The model has enough memory to run, but the canvas is so far outside its normal working resolution that it starts making terrible decisions anyway.
|
||||
|
||||
### Tile, Tag, and Guide
|
||||
Tiling keeps each evaluation small. It does not make the tiles understand the same complete image. That second problem is why I made Contextual Diffusion.
|
||||
|
||||
**Tile & Tag SEGS** is for workflows where tile regions should carry their own generated prompt guidance.
|
||||
### Contextual Diffusion
|
||||
|
||||
It splits an image into deterministic tile SEGS, crops each tile, runs WD14 tagging on each crop, prefixes your universal positive prompt text, and CLIP-encodes the resulting prompts into a `CONDITIONING_BATCH`. The order matters: the conditioning batch is aligned to the tile SEGS order so downstream per-SEG or regional nodes can pick the right prompt for the right area.
|
||||
**KSampler (Contextual Diffusion)** is an original sampling method I developed for editing and refining oversized latent canvases.
|
||||
|
||||
That is the kind of thing that is easy to do once by hand and annoying to keep correct in a real graph.
|
||||
The first real target was a 2160 × 3072 source image I wanted to edit with FLUX.2 Klein 4B. Downscaling made the edit coherent, but that defeated the point of starting with a high-resolution source. Ordinary tiled diffusion kept much more detail and looked promising at first. Then I looked at the whole image. One tile had found a figure, another had invented a second figure, and different parts of the cathedral had become different buildings. The overlaps were smooth! The scene was still nonsense.
|
||||
|
||||
- **Tag SEGS w/ WD14** tags existing SEGS crops with a connected WD14 tagger and returns conditioning aligned to the original SEGS order.
|
||||
- **Tag SEGS w/ External LLM** sends each SEG crop to a configured vision-capable external LLM, formats the returned tags, and returns aligned conditioning for detail or regional workflows.
|
||||
I needed Klein to see the complete composition and the full-resolution detail during the same denoising process. Contextual Diffusion does that by making overlapping local predictions on the original latent and a second prediction from a smaller, aspect-preserving view of the whole image during the early steps.
|
||||
|
||||
### External LLM Prompting
|
||||
That took some trial and error. Directly blending the whole-image prediction into the tiles made the result blurry. Leaving it active too long produced smears, repeated edges, and other low-resolution garbage in the final detail. What finally worked was subtracting the low-frequency interpretation already present in the tiled prediction and adding only the difference from the whole-image prediction:
|
||||
|
||||
**External LLM Prompt** sends system and user prompts to a configured OpenAI-compatible provider and returns the assistant response as text. It can also include the first image from an optional image input when the selected provider model supports vision.
|
||||
`prediction = local + scheduled_weight × (global_upsampled − local_low_frequency)`
|
||||
|
||||
The external LLM nodes use the endpoint and API key configured in SimpleSyrup settings.
|
||||
The whole-image correction is strongest at the beginning and can decay before the model starts settling fine texture. Distilled Klein models commonly finish in four steps, so even one corrected step is already a quarter of the denoising process.
|
||||
|
||||
### Tiled Sampling
|
||||
Contextual Diffusion is for edits the model already knows how to make at a normal resolution. I use it for clothing, material, color, jewelry, expression, local lighting, and other changes where I want to keep the source pose and composition. If I need a completely new pose, camera, and environment, I establish those at a normal working resolution first and refine the result afterward.
|
||||
|
||||
**KSampler (Tiled Diffusion)** is a KSampler-style node with selectable **MultiDiffusion** and **Mixture of Diffusers** modes.
|
||||
FLUX.2 reference latents stay complete and ordered in every local and whole-image evaluation. This lets one image retain the target composition while other images continue to provide complete subject or style references. The sampler also supports Anima's singleton-depth latent shape, which is useful when refining an illustration after a conventional resize.
|
||||
|
||||
It splits the latent into tiles, denoises tile predictions, and blends them back together during sampling. MultiDiffusion averages overlapping predictions. Mixture of Diffusers uses weighted blending. Both are there because large images and large upscale passes often need a different strategy than normal full-latent denoising.
|
||||
You can connect SEGS to replace the normal grid with a region-guided context plan. This gives you some control over where the local windows fall. Earlier versions ran regular tiles and a second bank of SAM views at the same time because I thought more views of the important objects would help. Instead, I got duplicated hats, extra limbs, repeated garment edges, and other semantic echoes. It was the wrong architecture, so I removed it. The current method uses one local plan at a time and returns its actual windows through `contexts_segs` so you can see what it evaluated.
|
||||
|
||||
Tiled diffusion is here because it was one of the high-resolution workflow tools I kept reaching for in my WebUI setup. SimpleSyrup brings MultiDiffusion and Mixture of Diffusers behavior into normal Comfy sampling nodes, so large latent jobs can be tiled without giving up Comfy's explicit conditioning and graph wiring.
|
||||
The cost is one tiled prediction pass per denoising step and one smaller whole-image evaluation for each step using the correction. UniPC, regional conditioning, ControlNet, and GLIGEN are currently rejected because I haven't validated their spatial behavior across both context sizes.
|
||||
|
||||
This also matters for Anima workflows. Anima can produce beautiful images, but pushing beyond its comfortable native size with untiled diffusion upscale can smear detail instead of improving it. The tiled path gives those workflows another route.
|
||||
I found the formula by comparing the failures and adjusting the method until the whole-image branch could fix composition without taking the detail away from the tiles. After I had implemented it, I learned about [Upsample Guidance](https://arxiv.org/abs/2404.01709). It uses a closely related separation between low-frequency guidance and a high-resolution residual.
|
||||
|
||||
### Diffusion Loaders
|
||||
Upsample Guidance wasn't part of how I developed Contextual Diffusion. There are also practical differences: my high-resolution prediction is assembled from bounded tiles, the complete image is fit into an aspect-preserving context, reference latents stay whole, and the node has its own early-step controls and optional SEGS planning. Still, the mathematical relationship is real. My experiments are qualitative, and I describe the method as independent development of a related multiscale idea instead of claiming priority over that paper.
|
||||
|
||||
**Simple Load Checkpoint** is meant to feel familiar if you come from WebUI, where the common generation controls live near the model selection.
|
||||
### Tiled Diffusion
|
||||
|
||||
- **Simple Load Checkpoint** loads a checkpoint, optionally replaces the checkpoint VAE, and keeps CLIP skip in the same place.
|
||||
- **Simple Load Anima** is for Anima workflows. It loads Anima with the Qwen text encoder and Qwen image VAE it expects. You can choose the files yourself or let SimpleSyrup resolve the known Anima assets automatically. It keeps model, VAE, dtype, and device decisions together so the rest of the graph can get on with the image.
|
||||
**KSampler (Tiled Diffusion)** is the more direct tiled sampler. It divides the latent into overlapping contexts, evaluates them in batches, and combines the predictions during every denoising step.
|
||||
|
||||
### Model and Detector Loaders
|
||||
MultiDiffusion averages the overlapping predictions. Mixture of Diffusers uses Gaussian weights that favor the center of each context. This works well when local evaluation and overlap blending are enough for the image. Contextual Diffusion adds the whole-image correction for edits where the separate contexts lose track of the complete scene.
|
||||
|
||||
These nodes load the models used by detection, segmentation, tagging, matting, and compatibility workflows.
|
||||
The same tiled sampling path is available in **Detail SEGS by Scale Factor w/ Tiled Diffusion** for large detailer crops and **KSampler (Prompt by Tiled Region)** for regional prompts on large canvases.
|
||||
|
||||
- **SAM Model Loader** loads SAM, SAM-HQ, and MobileSAM choices for segmentation workflows.
|
||||
- **GroundingDINO Model Loader** loads GroundingDINO with an explicit BERT text encoder.
|
||||
- **ViTMatte Model Loader** loads ViTMatte for mask edge refinement.
|
||||
- **Load Ultralytics Model** loads an Ultralytics detector and exposes both SimpleSyrup's native detector model and Impact-style compatibility outputs.
|
||||
- **Load WD14 Tagger** loads a SmilingWolf WD14 ONNX model and its tag CSV.
|
||||
- **LayerStyle SAM Models Adapter** splits a LayerStyle `LS_SAM_MODELS` bundle into separate `SAM_MODEL` and `DINO_MODEL` outputs.
|
||||
- **Grounded SAM Model Info** returns JSON metadata for selected SAM and GroundingDINO models.
|
||||
## SEGS, detailers, and regional prompts
|
||||
|
||||
The LayerStyle adapter exists because good ComfyUI workflows should not make you reload the same SAM or GroundingDINO model just because one node pack uses a different socket shape.
|
||||
Impact Pack already had a useful way to represent detected and masked regions: `SEGS`. I built SimpleSyrup around the same shape so regions can move between compatible detectors, these nodes, and Impact workflows without reloading models or rebuilding the masks.
|
||||
|
||||
### Sampler and Scheduler Extras
|
||||
**Prompt SEGS w/ SAM** uses GroundingDINO to find objects from text and SAM to segment them. It also supports negative prompting and optional ViTMatte edge refinement. **Detect SEGS w/ Ultralytics** creates regions from bounding-box or segmentation models with confidence, label, and size filtering. Existing masks can enter the same workflow through **Mask to SEGS**, while **SEGS from SAM Output** runs automatic unprompted segmentation from a connected SAM model.
|
||||
|
||||
**KSampler (Extras)** keeps the normal Comfy sampler shape, but adds sampler and scheduler behavior I wanted available without dragging in a separate sampler stack.
|
||||
**Simple Preview SEGS** shows the regions over the image, lets you select them from an interactive grid, and passes the original SEGS onward. **Batch SEGS** combines several ordered SEGS inputs.
|
||||
|
||||
It includes:
|
||||
The scale-factor detailers work on one crop at a time. They enlarge the crop, sample it, shrink it back, and composite it into the source image with feathering and optional denoise masks. **Detail SEGS as Regions** takes another route: it keeps the full image in one MultiDiffusion pass, uses the global conditioning across the image, and pairs each SEG with its own ordered regional conditioning.
|
||||
|
||||
- `euler_a_a1111`, an A1111/k-diffusion-style Euler ancestral sampler.
|
||||
- **AYS SD1** and **AYS SDXL** schedules.
|
||||
- **GITS**.
|
||||
- **automatic_a1111** scheduler behavior.
|
||||
- **beta57**, a local reimplementation of the RES4LYF beta57 scheduler preset.
|
||||
The prompt batching came directly from ADetailer. **Encode Prompt Batch** splits positive and negative text with `[SEP]`. You can write `[SEP|name]` to keep a long prompt readable; matching still follows the order of the prompts and regions. The first prompt is global, and each later prompt belongs to the corresponding mask or SEG.
|
||||
|
||||
The node still uses Comfy-style seed handling, partial denoise behavior, progress callbacks, and normal positive/negative conditioning inputs.
|
||||
**Conditioning Batch Start**, **Conditioning Batch Append**, and **Batch Region Conditioning** build the same ordered structure from existing conditioning. **Compose Regional Conditioning** converts a global-first prompt batch and ordered masks into normal masked Comfy conditioning. The dedicated **KSampler (Prompt by Region)** and tiled version apply the regional prompt batch during sampling.
|
||||
|
||||
### Image, Prompt, and Latent Utilities
|
||||
If [ComfyUI Prompt Control](https://github.com/asagi4/comfyui-prompt-control) is installed, SimpleSyrup also exports **Encode Prompt Batch w/ Prompt Control** and **Schedule & Encode Prompts**. They preserve Prompt Control scheduling and LoRA hooks across `[SEP]` regions. The rest of the pack loads normally when Prompt Control is absent.
|
||||
|
||||
These nodes handle the smaller jobs that show up all over image workflows.
|
||||
Model loading stays separate from detection. There are loaders for SAM, GroundingDINO, ViTMatte, and Ultralytics. **LayerStyle SAM Models Adapter** accepts a ComfyUI Layer Style Advance `LS_SAM_MODELS` bundle and exposes the loaded SAM and GroundingDINO models through the normal sockets used here.
|
||||
|
||||
- **Resize Image to Target** resizes image batches with stretch, keep-aspect, crop, and pad modes. It can round output dimensions to a divisibility target, anchor crop or pad placement, process batches in chunks, resize a mask with the image, and use GPU Lanczos through [TorchLanc](https://github.com/Artificial-Sweetener/TorchLanc).
|
||||
- **Simple VAE Encode** encodes an image to latent space, but reuses the source latent when the graph proves the image came from an unmodified `VAEDecode`.
|
||||
- **Upscale Latent From Image** finds the latent behind an unmodified decoded image and expands to Comfy's latent upscale behavior.
|
||||
- **VAE Encode (Options)** and **VAE Decode (Options)** wrap ComfyUI's normal and tiled VAE paths behind one explicit tiling toggle, with spatial and temporal tile controls available when tiling is enabled.
|
||||
- **Latent Diagnostics** passes a latent through unchanged while reporting shape, dtype, device, and tiling-fit details.
|
||||
- **Prompt Encode Style** creates Prompt Control style tags from an encode-style selection.
|
||||
- **Prompt Encode Style & Normalization** creates Prompt Control style and normalization tags together.
|
||||
- **Scale Factor** provides a bounded scale multiplier for nodes that expect one.
|
||||
- **Seed** provides a reusable seed value with ComfyUI seed controls.
|
||||
## Tagging images and regions
|
||||
|
||||
The provenance nodes trace the graph. They do not guess from tensor values. If an image has been loaded, edited, cropped, detailed, resized, or otherwise changed, the original latent provenance is broken and the node will not pretend otherwise.
|
||||
**Load WD14 Tagger** loads a SmilingWolf WD14 ONNX model and its tag CSV. **Tag SEGS w/ WD14** runs the tagger on existing SEG crops and keeps the resulting conditioning in the same order. **Tile & Tag SEGS** makes a deterministic set of tile regions, tags each crop, prefixes shared positive text, and returns the SEGS together with their matching conditioning batch.
|
||||
|
||||
## Settings
|
||||
The external LLM nodes use a configured OpenAI-compatible provider. **Tag SEGS w/ External LLM** sends each region crop to a vision-capable model and returns aligned conditioning. **External LLM Prompt** sends system and user prompts and returns the response as text, with an optional image for models that support vision.
|
||||
|
||||
SimpleSyrup adds ComfyUI settings for model visibility and external LLM access:
|
||||
## Images, masks, latents, and sampler extras
|
||||
|
||||
- **SimpleSyrup: Show downloadable models in loader dropdowns** controls whether known downloadable SAM, GroundingDINO, ViTMatte, and WD14 models appear before they are installed locally.
|
||||
- **SimpleSyrup: External LLM endpoint** stores the OpenAI-compatible base URL used by external LLM prompt nodes.
|
||||
- **SimpleSyrup: External LLM API key** stores the API key for that endpoint in OS credential storage.
|
||||
**Load Image List** loads files in selection order as separate image list items, so each image keeps its own dimensions. **Load Mask Batch** loads same-sized files as one `BHW` mask batch and applies the selected channel consistently to every file.
|
||||
|
||||
When downloadable models are shown, supported loaders list known model choices even if the files are not installed yet. When they are hidden, those dropdowns only show models SimpleSyrup can verify locally. Anima's automatic Qwen text encoder and VAE resolution is handled by the Anima loader itself.
|
||||
**Resize Image to Target** handles stretch, keep-aspect, crop, and pad modes. It supports divisibility rounding, anchored crop and pad placement, chunked batches, paired masks, and GPU Lanczos through TorchLanc.
|
||||
|
||||
## License & Acknowledgements
|
||||
**VAE Encode (Options)** and **VAE Decode (Options)** put the normal and tiled VAE paths behind one explicit tiling control, including spatial and temporal tile settings where Comfy supports them.
|
||||
|
||||
**SimpleSyrup** is licensed under the GNU Affero General Public License v3.0 or later (**AGPL-3.0-or-later**). Please read the full [LICENSE](LICENSE) included with this repo.
|
||||
**Simple VAE Encode** can reuse the source latent when the graph proves that its image came directly from an unmodified `VAEDecode`. **Upscale Latent From Image** uses the same provenance to find and resize the original latent. Loading, editing, cropping, detailing, or resizing the image breaks that provenance. These nodes follow the graph instead of trying to identify a latent from the finished tensor.
|
||||
|
||||
AGPL-3.0-or-later is a strong copyleft license. If you convey SimpleSyrup or a modified version, you must provide the corresponding source; and if you let users interact with a modified version over a network, you must offer those users the corresponding source for that modified version.
|
||||
**KSampler (Extras)** adds the A1111/k-diffusion-style `euler_a_a1111` sampler, AYS SD1 and SDXL schedules, GITS, the `automatic_a1111` scheduler, and a local implementation of the RES4LYF beta57 preset. It keeps Comfy's regular seed handling, partial denoise behavior, progress callbacks, and conditioning inputs.
|
||||
|
||||
The remaining utilities are **Latent Diagnostics**, **Scale Factor**, and **Seed**. Latent Diagnostics reports the latent shape, dtype, device, and tiled-sampling compatibility while passing it through unchanged.
|
||||
|
||||
## Settings and optional integrations
|
||||
|
||||
SimpleSyrup adds three ComfyUI settings:
|
||||
|
||||
- **SimpleSyrup: Show downloadable models in loader dropdowns** controls whether known downloadable SAM, GroundingDINO, ViTMatte, and WD14 choices appear before they are installed.
|
||||
- **SimpleSyrup: External LLM endpoint** stores the OpenAI-compatible base URL used to discover provider models and run the external prompt nodes.
|
||||
- **SimpleSyrup: External LLM API key** stores the provider key in OS credential storage.
|
||||
|
||||
With downloadable models enabled, selecting a known missing catalog entry lets its loader download the required files. With the setting disabled, the dropdowns contain models SimpleSyrup can verify locally. Anima, FLUX.1, and FLUX.2 support components are resolved by their own loaders and use checksum-pinned automatic choices.
|
||||
|
||||
Saving the external LLM endpoint and API key refreshes the provider models available in connected SimpleSyrup nodes. Image inputs require a provider model with vision support.
|
||||
|
||||
SimpleSyrup currently interoperates with:
|
||||
|
||||
- [ComfyUI Prompt Control](https://github.com/asagi4/comfyui-prompt-control) for scheduled prompts and regional LoRA hooks.
|
||||
- ComfyUI Impact Pack through compatible `SEGS` values.
|
||||
- ComfyUI Layer Style Advance through its `LS_SAM_MODELS` bundle.
|
||||
|
||||
## License, acknowledgements, and research
|
||||
|
||||
**SimpleSyrup** is licensed under the GNU Affero General Public License v3.0 or later (**AGPL-3.0-or-later**). Please read the full [LICENSE](LICENSE) included with this repository.
|
||||
|
||||
AGPL-3.0-or-later is a strong copyleft license. If you convey SimpleSyrup or a modified version, you must provide the corresponding source. If users interact with a modified version over a network, you must offer those users the corresponding source for that version.
|
||||
|
||||
SimpleSyrup owes a lot to other projects:
|
||||
|
||||
- [ComfyUI Impact Pack](https://github.com/ltdrdata/ComfyUI-Impact-Pack) for the SEGS workflow vocabulary and detailer shape this pack is heavily modeled around.
|
||||
- [ADetailer](https://github.com/Bing-su/adetailer) for the inline `[SEP]` per-segment prompt workflow I missed from WebUI.
|
||||
- [ComfyUI Layer Style Advance](https://github.com/chflame163/ComfyUI_LayerStyle_Advance) for the SAM workflow surface this pack interoperates with.
|
||||
- [Tiled Diffusion & VAE for AUTOMATIC1111](https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111) for practical tiled diffusion and Mixture of Diffusers behavior.
|
||||
- [RES4LYF](https://github.com/ClownsharkBatwing/RES4LYF) for the beta57 scheduler preset reimplemented here.
|
||||
- [ComfyUI](https://github.com/Comfy-Org/ComfyUI) provides the engine and graph ecosystem this pack runs on.
|
||||
- [ComfyUI Impact Pack](https://github.com/ltdrdata/ComfyUI-Impact-Pack) established the SEGS workflow vocabulary and detailer structure used here.
|
||||
- [ADetailer](https://github.com/Bing-su/adetailer) is where the inline `[SEP]` per-segment prompt workflow came from.
|
||||
- [ComfyUI Prompt Control](https://github.com/asagi4/comfyui-prompt-control) provides the scheduled prompt and LoRA-hook behavior used by the optional integration.
|
||||
- [ComfyUI Layer Style Advance](https://github.com/chflame163/ComfyUI_LayerStyle_Advance) provides the SAM model bundle SimpleSyrup can adapt.
|
||||
- [Tiled Diffusion & VAE for AUTOMATIC1111](https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111) informed the practical tiled diffusion and Mixture of Diffusers behavior reimplemented here.
|
||||
- [RES4LYF](https://github.com/ClownsharkBatwing/RES4LYF) is the source of the beta57 scheduler preset reimplemented here.
|
||||
|
||||
SimpleSyrup also vendors or reimplements selected third-party behavior for SAM-HQ, MobileSAM, GroundingDINO, AUTOMATIC1111 sampler behavior, k-diffusion, and tiled diffusion behavior. See [third_party/NOTICE.md](third_party/NOTICE.md) for the full third-party notices.
|
||||
SimpleSyrup also vendors or reimplements selected third-party behavior for SAM-HQ, MobileSAM, GroundingDINO, AUTOMATIC1111 sampler behavior, k-diffusion, and tiled diffusion. See [third_party/NOTICE.md](third_party/NOTICE.md) for the complete notices.
|
||||
|
||||
### Research Citations
|
||||
### Research citations
|
||||
|
||||
SimpleSyrup's tiled diffusion behavior is based on ideas from MultiDiffusion and Mixture of Diffusers.
|
||||
SimpleSyrup's tiled diffusion behavior builds on MultiDiffusion and Mixture of Diffusers. Contextual Diffusion was developed independently and was later found to share a related multiscale residual principle with Upsample Guidance.
|
||||
|
||||
```bibtex
|
||||
@article{bar2023multidiffusion,
|
||||
@@ -206,9 +212,17 @@ SimpleSyrup's tiled diffusion behavior is based on ideas from MultiDiffusion and
|
||||
}
|
||||
```
|
||||
|
||||
```bibtex
|
||||
@article{hwang2024upsample,
|
||||
title={Upsample Guidance: Scale Up Diffusion Models without Training},
|
||||
author={Hwang, Juno and Park, Yong-Hyun and Jo, Youngjung},
|
||||
journal={arXiv preprint arXiv:2404.01709},
|
||||
year={2024}
|
||||
}
|
||||
```
|
||||
|
||||
## From the Developer 💖
|
||||
|
||||
|
||||
- **Buy Me a Coffee**: You can help fuel more projects like this at my [Ko-fi page](https://ko-fi.com/artificial_sweetener).
|
||||
- **My Website & Socials**: See my art, poetry, and other dev updates at [artificialsweetener.ai](https://artificialsweetener.ai).
|
||||
- **My Website & Socials**: See my art, poetry, research notes, and other development updates at [artificialsweetener.ai](https://artificialsweetener.ai).
|
||||
- **If you like this project**, it would mean a lot to me if you gave me a star here on GitHub!! ⭐
|
||||
|
||||
Reference in New Issue
Block a user