Compare commits

..
15 Commits
Author SHA1 Message Date
Daisy 9948cb3433 chore(release): 1.9.3 [skip ci]
## [1.9.3](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.9.2...v1.9.3) (2026-09-20)

### Bug Fixes

* **attention-coupling:** restore regional LoRA sampling ([0255a0f](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/0255a0f5044278f14452b6b2582ec6646083f756))
2026-09-20 21:17:02 +00:00
Artificial Sweetener 6cb9bbe868 fix(attention-coupling): restore regional LoRA sampling 2026-09-20 17:07:51 -04:00
Daisy 561b73630c chore(release): 1.9.2 [skip ci]
## [1.9.2](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.9.1...v1.9.2) (2026-09-20)

### Bug Fixes

* **registry:** remove flagged package content ([f3a53b5](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/f3a53b5ec6c080d98f7e9599cf55e849cf338021))
2026-09-20 03:15:34 +00:00
Artificial Sweetener 51efa670e0 fix(registry): remove flagged package content 2026-09-19 23:08:05 -04:00
Daisy d188a3764b chore(release): 1.9.1 [skip ci]
## [1.9.1](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.9.0...v1.9.1) (2026-09-20)

### Bug Fixes

* **contextual-diffusion:** project reference latents into views ([4cd780a](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/4cd780a2451aa472ce826834e4426b65693c46e8))
2026-09-20 02:14:41 +00:00
Artificial Sweetener f1d0630729 fix(contextual-diffusion): project reference latents into views 2026-09-19 22:06:33 -04:00
Daisy 0cd1032073 chore(release): 1.9.0 [skip ci]
# [1.9.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.8.0...v1.9.0) (2026-09-19)

### Features

* **prompts:** add automatic NegPiP support ([6d052e9](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/6d052e9800985696972bf431fa7dae4972a56313))
2026-09-19 20:44:48 +00:00
Artificial Sweetener d01b085082 feat(prompts): add automatic NegPiP support 2026-09-19 16:39:12 -04:00
Daisy 0583ba2675 chore(release): 1.8.0 [skip ci]
# [1.8.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.7.1...v1.8.0) (2026-09-19)

### Bug Fixes

* **downloads:** keep unknown sizes indeterminate ([a31467c](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/a31467cd3a4299e9e3929d44281018dba0322b3e))
* **models:** hide installed catalog choices ([d887e87](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/d887e87e03eb6d0d9d5325fe43b67fbd9e47cf71))

### Features

* **models:** add curated ultralytics downloads ([b907fa2](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/b907fa2a17afa30170bc22cf1134a750241e55c2))
* **models:** prioritize installed ultralytics choices ([d30e04f](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/d30e04f229366d3e1d2388bd4d713b2b47a12a1f))
2026-09-19 16:39:28 +00:00
Artificial Sweetener dcc37d7ab6 fix(downloads): keep unknown sizes indeterminate 2026-09-19 10:48:43 -04:00
Artificial Sweetener 583b22a4bf feat(models): prioritize installed ultralytics choices 2026-09-19 10:32:49 -04:00
Artificial Sweetener ebe01efc49 fix(models): hide installed catalog choices 2026-09-19 00:37:32 -04:00
Artificial Sweetener 41e8a2b61c feat(models): add curated ultralytics downloads 2026-09-19 00:25:58 -04:00
Daisy 22e4a5d202 chore(release): 1.7.1 [skip ci]
## [1.7.1](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.7.0...v1.7.1) (2026-09-11)

### Bug Fixes

* **regional:** preserve shared model patch ancestry ([6059a3f](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/6059a3f913a9502671666e83faeb8686a7a8da27))
2026-09-11 15:12:46 +00:00
Artificial Sweetener 017e3fc7fe fix(regional): preserve shared model patch ancestry
Keep compatible parallel regional paths on one inherited model lineage, including NegPip interoperability, while retaining bounded fused and optional Triton execution paths.

Expand graph-shape, lifecycle, memory-safety, and runtime regressions across the supported attention families.
2026-09-10 23:11:24 -04:00
149 changed files with 9863 additions and 1344 deletions
+13
View File
@@ -0,0 +1,13 @@
.github/
tests/
tools/
scripts/
web/src/
web/tests/
AGENTS.md
.releaserc.cjs
eslint.config.js
package-lock.json
package.json
tsconfig.json
vitest.config.ts
+49
View File
@@ -1,3 +1,52 @@
## [1.9.3](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.9.2...v1.9.3) (2026-09-20)
### Bug Fixes
* **attention-coupling:** restore regional LoRA sampling ([0255a0f](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/0255a0f5044278f14452b6b2582ec6646083f756))
## [1.9.2](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.9.1...v1.9.2) (2026-09-20)
### Bug Fixes
* **registry:** remove flagged package content ([f3a53b5](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/f3a53b5ec6c080d98f7e9599cf55e849cf338021))
## [1.9.1](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.9.0...v1.9.1) (2026-09-20)
### Bug Fixes
* **contextual-diffusion:** project reference latents into views ([4cd780a](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/4cd780a2451aa472ce826834e4426b65693c46e8))
# [1.9.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.8.0...v1.9.0) (2026-09-19)
### Features
* **prompts:** add automatic NegPiP support ([6d052e9](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/6d052e9800985696972bf431fa7dae4972a56313))
# [1.8.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.7.1...v1.8.0) (2026-09-19)
### Bug Fixes
* **downloads:** keep unknown sizes indeterminate ([a31467c](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/a31467cd3a4299e9e3929d44281018dba0322b3e))
* **models:** hide installed catalog choices ([d887e87](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/d887e87e03eb6d0d9d5325fe43b67fbd9e47cf71))
### Features
* **models:** add curated ultralytics downloads ([b907fa2](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/b907fa2a17afa30170bc22cf1134a750241e55c2))
* **models:** prioritize installed ultralytics choices ([d30e04f](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/d30e04f229366d3e1d2388bd4d713b2b47a12a1f))
## [1.7.1](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.7.0...v1.7.1) (2026-09-11)
### Bug Fixes
* **regional:** preserve shared model patch ancestry ([6059a3f](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/6059a3f913a9502671666e83faeb8686a7a8da27))
# [1.7.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.6.0...v1.7.0) (2026-09-05)
+1
View File
@@ -189,6 +189,7 @@ SimpleSyrup owes a lot to other projects:
- [ComfyUI Layer Style Advance](https://github.com/chflame163/ComfyUI_LayerStyle_Advance) provides the SAM model bundle SimpleSyrup can adapt.
- [Tiled Diffusion & VAE for AUTOMATIC1111](https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111) informed the practical tiled diffusion and Mixture of Diffusers behavior reimplemented here.
- [RES4LYF](https://github.com/ClownsharkBatwing/RES4LYF) is the source of the beta57 scheduler preset reimplemented here.
- [ComfyUI-ppm](https://github.com/pamparamm/ComfyUI-ppm) by pamparamm provides the ModelPatcher-based NegPiP behavior adapted here and builds on the [ComfyUI port](https://github.com/laksjdjf/cd-tuner_negpip-ComfyUI) by laksjdjf and the [original WebUI implementation](https://github.com/hako-mikan/sd-webui-negpip) by hako-mikan.
SimpleSyrup also vendors or reimplements selected third-party behavior for SAM-HQ, MobileSAM, GroundingDINO, AUTOMATIC1111 sampler behavior, k-diffusion, and tiled diffusion. See [third_party/NOTICE.md](third_party/NOTICE.md) for the complete notices.
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "simple-syrup-comfyui",
"version": "1.7.0",
"version": "1.9.3",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "simple-syrup-comfyui",
"version": "1.7.0",
"version": "1.9.3",
"license": "AGPL-3.0-or-later",
"devDependencies": {
"@eslint/js": "^9.39.1",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "simple-syrup-comfyui",
"version": "1.7.0",
"version": "1.9.3",
"private": true,
"license": "AGPL-3.0-or-later",
"type": "module",
+1 -1
View File
@@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "SimpleSyrup"
description = "Workflow-focused ComfyUI extensions for image generation."
version = "1.7.0"
version = "1.9.3"
license = "AGPL-3.0-or-later"
license-files = ["LICENSE"]
requires-python = ">=3.11"
+1 -1
View File
@@ -6,6 +6,6 @@
from __future__ import annotations
__version__ = "1.7.0"
__version__ = "1.9.3"
__all__: list[str] = ["__version__"]
@@ -0,0 +1,89 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Detect effective negative weights in Comfy-style prompt emphasis."""
from __future__ import annotations
from dataclasses import dataclass
@dataclass(frozen=True, slots=True)
class _WeightedPromptSegment:
"""Retain one parsed prompt fragment and its effective scalar weight."""
text: str
weight: float
def contains_negative_prompt_weight(text: str) -> bool:
"""Return whether valid nested emphasis gives any prompt text a negative weight."""
if not isinstance(text, str):
raise TypeError("Negative prompt-weight detection requires text.")
escaped = text.replace(r"\)", "\0\1").replace(r"\(", "\0\2")
return any(
segment.text and segment.weight < 0.0
for segment in _weighted_segments(escaped, 1.0)
)
def _weighted_segments(
text: str,
current_weight: float,
) -> tuple[_WeightedPromptSegment, ...]:
"""Parse emphasis with the same nesting and final-colon rules as ComfyUI."""
parsed: list[_WeightedPromptSegment] = []
for item in _parenthesized_items(text):
weight = current_weight
if len(item) >= 2 and item[0] == "(" and item[-1] == ")":
inner = item[1:-1]
delimiter = inner.rfind(":")
weight *= 1.1
if delimiter > 0:
try:
weight = float(inner[delimiter + 1 :])
except ValueError:
pass
else:
inner = inner[:delimiter]
parsed.extend(_weighted_segments(inner, weight))
continue
parsed.append(
_WeightedPromptSegment(
item.replace("\0\1", ")").replace("\0\2", "("),
current_weight,
)
)
return tuple(parsed)
def _parenthesized_items(text: str) -> tuple[str, ...]:
"""Split top-level parenthesized regions while preserving malformed input."""
result: list[str] = []
current = ""
nesting = 0
for character in text:
if character == "(":
if nesting == 0:
if current:
result.append(current)
current = "("
else:
current += character
nesting += 1
elif character == ")":
nesting -= 1
if nesting == 0:
result.append(f"{current})")
current = ""
else:
current += character
else:
current += character
if current:
result.append(current)
return tuple(result)
@@ -30,6 +30,7 @@ class ProcessedRegionalAttentionEntry:
schedule: ConditioningScheduleRange
cross_attention: torch.Tensor
strength: float
cross_attention_value_multiplier: torch.Tensor | None = None
def __post_init__(self) -> None:
"""Validate entry order, model context, and finite scalar strength."""
@@ -62,6 +63,21 @@ class ProcessedRegionalAttentionEntry:
if not math.isfinite(float(self.strength)):
raise ValueError("Processed conditioning strength must be finite.")
object.__setattr__(self, "strength", float(self.strength))
multiplier = self.cross_attention_value_multiplier
if multiplier is None:
return
if (
not isinstance(multiplier, torch.Tensor)
or multiplier.shape != (*self.cross_attention.shape[:2], 1)
or not multiplier.is_floating_point()
or multiplier.device != self.cross_attention.device
or multiplier.dtype != self.cross_attention.dtype
or not bool(torch.isfinite(multiplier).all().item())
):
raise ValueError(
"Processed attention value multiplier must be a finite floating "
"BxSx1 tensor aligned with cross_attention."
)
@dataclass(frozen=True, slots=True)
@@ -48,6 +48,7 @@ class BatchedRegionalAttentionEntry:
entry_index: int
context: torch.Tensor
strengths: tuple[float, ...]
cross_attention_value_multiplier: torch.Tensor | None = None
def __post_init__(self) -> None:
"""Validate entry order, aligned context, and finite sample strengths."""
@@ -70,6 +71,11 @@ class BatchedRegionalAttentionEntry:
)
if not math.isfinite(float(strength)):
raise ValueError("Regional attention entry strength must be finite.")
_validate_value_multiplier(
self.cross_attention_value_multiplier,
self.context,
name="entry",
)
@dataclass(frozen=True, slots=True)
@@ -109,6 +115,7 @@ class BatchedRegionalAttentionContexts:
chunks: tuple[RegionalAttentionChunkBatch, ...]
base_context: torch.Tensor
regions: tuple[BatchedRegionalAttentionRegion, ...]
base_value_multiplier: torch.Tensor | None = None
def __post_init__(self) -> None:
"""Validate complete chunk and tensor alignment."""
@@ -141,6 +148,11 @@ class BatchedRegionalAttentionContexts:
expected_batch=expected_start,
name="base",
)
_validate_value_multiplier(
self.base_value_multiplier,
self.base_context,
name="base",
)
if not isinstance(self.regions, tuple):
raise TypeError("Regional attention regions must be a tuple.")
if tuple(region.region_index for region in self.regions) != tuple(
@@ -189,3 +201,27 @@ def _validate_aligned_context(
raise ValueError(
f"Regional attention {name} context must contain finite floating values."
)
def _validate_value_multiplier(
multiplier: object,
context: torch.Tensor,
*,
name: str,
) -> None:
"""Validate one optional value multiplier against its aligned context."""
if multiplier is None:
return
if (
not isinstance(multiplier, torch.Tensor)
or multiplier.shape != (*context.shape[:2], 1)
or not multiplier.is_floating_point()
or multiplier.device != context.device
or multiplier.dtype != context.dtype
or not bool(torch.isfinite(multiplier).all().item())
):
raise ValueError(
f"Regional attention {name} value multiplier must be a finite "
"floating BxSx1 tensor aligned with its context."
)
+16 -5
View File
@@ -8,7 +8,7 @@ from __future__ import annotations
from typing import Any
from ..runtime.model_catalog import grounding_dino_choices, sam_choices
from ..runtime.model_choices import ModelChoiceService, default_choice
from ..runtime.model_metadata import GroundedSAMModelMetadata
from . import tooltips
@@ -17,6 +17,7 @@ class GroundedSAMModelInfo:
"""Expose selected grounded SAM source and local path metadata."""
_metadata = GroundedSAMModelMetadata()
_choices = ModelChoiceService()
RETURN_TYPES = ("STRING",)
RETURN_NAMES = ("model_info",)
@@ -31,19 +32,27 @@ class GroundedSAMModelInfo:
def INPUT_TYPES(cls) -> dict[str, dict[str, tuple[Any, ...]]]:
"""Declare deterministic model metadata inputs."""
sam_model_choices = cls._choices.sam_choices()
grounding_dino_model_choices = cls._choices.grounding_dino_choices()
return {
"required": {
"sam_model": (
sam_choices(),
sam_model_choices,
{
"default": "sam_hq_vit_b (379MB)",
"default": default_choice(
sam_model_choices,
"sam_hq_vit_b (379MB)",
),
"tooltip": tooltips.SAM_MODEL_INPUT,
},
),
"grounding_dino_model": (
grounding_dino_choices(),
grounding_dino_model_choices,
{
"default": "GroundingDINO_SwinT_OGC (694MB)",
"default": default_choice(
grounding_dino_model_choices,
"GroundingDINO_SwinT_OGC (694MB)",
),
"tooltip": tooltips.GROUNDING_DINO_MODEL_INPUT,
},
),
@@ -53,4 +62,6 @@ class GroundedSAMModelInfo:
def describe(self, sam_model: str, grounding_dino_model: str) -> tuple[str]:
"""Return JSON metadata for selected model entries."""
self._choices.reject_sentinel(sam_model)
self._choices.reject_sentinel(grounding_dino_model)
return (self._metadata.describe_selection(sam_model, grounding_dino_model),)
+7 -2
View File
@@ -8,6 +8,7 @@ from __future__ import annotations
from typing import Any, ClassVar
from ..runtime.model_downloads import ComfyProgressReporter
from ..runtime.ultralytics_loader import UltralyticsLoaderService
@@ -40,7 +41,8 @@ class LoadUltralyticsModel:
{
"default": choices[0],
"tooltip": (
"Ultralytics model file in the ComfyUI models folder."
"A local Ultralytics model or a curated model that "
"downloads to ComfyUI's Impact Pack-compatible folders."
),
},
)
@@ -50,5 +52,8 @@ class LoadUltralyticsModel:
def load(self, model_name: str) -> tuple[object, object, object]:
"""Load the selected detector and paired compatibility facades."""
loaded = self.service_class().load(model_name)
loaded = self.service_class().load(
model_name,
progress=ComfyProgressReporter(),
)
return loaded.detector_model, loaded.bbox_detector, loaded.segm_detector
+2
View File
@@ -135,6 +135,7 @@ def get_nodes() -> list[type[object]]:
if not prompt_control_is_available():
return nodes
from .apply_automatic_negpip import ApplyAutomaticNegpipV3
from .attach_regional_global_conditioning import (
AttachRegionalGlobalConditioningV3,
)
@@ -149,6 +150,7 @@ def get_nodes() -> list[type[object]]:
return [
*nodes,
ApplyAutomaticNegpipV3,
AttachRegionalGlobalConditioningV3,
EncodePromptBatchWithPromptControl,
LabelRegionalLoraHooksV3,
@@ -0,0 +1,69 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Internal Comfy v3 node for model-family automatic NegPiP preparation."""
from __future__ import annotations
from importlib import import_module
from typing import TYPE_CHECKING, Any
from ..services.negpip_model_service import NEGPIP_MODEL_SERVICE
if TYPE_CHECKING:
class _ComfyNodeBase:
"""Type-checking base for Comfy v3 nodes."""
pass
else:
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
class ApplyAutomaticNegpipV3(_ComfyNodeBase):
"""Patch supported MODEL/CLIP pairs after a negative prompt-weight trigger."""
@classmethod
def define_schema(cls) -> Any:
"""Declare the internal runtime patch boundary."""
return _comfy_io.Schema(
node_id="SimpleSyrup.ApplyAutomaticNegpip",
display_name="Apply Automatic NegPiP (Internal)",
category="SimpleSyrup/Internal",
description=(
"Internal model-family NegPiP preparation injected by Schedule & "
"Encode Prompts after detecting a negative prompt weight."
),
is_dev_only=True,
inputs=[
_comfy_io.Model.Input(
"model",
tooltip="MODEL inspected and cloned only when NegPiP is supported.",
),
_comfy_io.Clip.Input(
"clip",
tooltip="CLIP cloned with the matching NegPiP encoder behavior.",
),
],
outputs=[
_comfy_io.Model.Output(
"model",
tooltip="MODEL carrying one supported NegPiP attention patch set.",
),
_comfy_io.Clip.Output(
"clip",
tooltip="CLIP carrying matching negative-weight encoding behavior.",
),
],
)
@classmethod
def execute(cls, model: object, clip: object) -> tuple[object, object]:
"""Return the supported patched pair or the original unsupported pair."""
return NEGPIP_MODEL_SERVICE.prepare(model, clip)
@@ -53,12 +53,14 @@ class KSamplerAttentionCouplingV3(_ComfyNodeBase):
"With conditioning batches and masks, denoises supported Anima "
"and standard SD/SDXL models through one "
"shared trajectory while coupling global and masked regional "
"cross-attention. The input MODEL may carry a global LoRA. Anima "
"regions may also carry ordered, independently scheduled Prompt "
"Control model LoRAs whose overlapping deltas compose in declared "
"order. Runtime scales with active adapters, ranks, and targets. "
"Standard SD/SDXL regional model-side hooks and unsupported Anima "
"adapter targets fail before sampling."
"cross-attention. LoRAs on the input MODEL and Prompt Control model "
"LoRAs on global conditioning entry 0 apply across the image. Regions "
"may also carry ordered, independently scheduled model LoRAs "
"whose overlapping deltas compose in declared order. Runtime scales "
"with active adapters, ranks, and targets. "
"Global LoRA and regional LoRA retain independent schedules; "
"regional model-side hooks are supported on admitted model families. "
"Unsupported adapter targets fail before sampling."
),
search_aliases=[
"attention coupling",
@@ -50,12 +50,14 @@ class KSamplerContextualAttentionCouplingV3(_ComfyNodeBase):
description=(
"Preserves large-image composition through Contextual Diffusion "
"while coupling regional attention in every local and reduced-global "
"Anima or standard SD/SDXL view. Global LoRAs remain on the input "
"model. Anima regional LoRA stacks are prepared once, retain "
"Anima or standard SD/SDXL view. LoRAs on the input MODEL and Prompt "
"Control model LoRAs on global conditioning entry 0 apply in every "
"view. Regional LoRA stacks are prepared once, retain "
"independent schedules and full quality, and skip inactive work. "
"Optional SEGS guide the shared local tile plan. Standard SD/SDXL "
"regional model-side hooks and unsupported Anima targets fail before "
"sampling."
"Global LoRA and regional LoRA stacks remain independently scheduled; "
"regional model-side hooks are supported on admitted model families. "
"Optional SEGS guide the shared local tile plan. Unsupported adapter "
"targets fail before sampling."
),
search_aliases=[
"contextual attention coupling",
+10 -8
View File
@@ -248,7 +248,8 @@ def attention_coupling_ksampler_inputs(
"model",
tooltip=(
"Supported Anima or standard SD/SDXL model used for one shared "
"denoiser trajectory; apply global model LoRAs before connecting it."
"denoiser trajectory. LoRAs patched on this model and Prompt Control "
"model LoRAs on conditioning entry 0 apply globally."
),
),
*base[1:6],
@@ -258,8 +259,8 @@ def attention_coupling_ksampler_inputs(
tooltip=(
"Global-first positive conditioning: entry 0 is global and later "
"entries pair with masks. Regional Prompt Control WeightHooks may "
"contain ordered full-rank Anima LoRA stacks with independent "
"schedules; standard SD/SDXL rejects regional model-side hooks."
"contain ordered regional LoRA stacks with independent schedules. "
"Model LoRA hooks on entry 0 apply across the image."
),
),
comfy_io.MultiType.Input(
@@ -267,8 +268,9 @@ def attention_coupling_ksampler_inputs(
[comfy_io.Conditioning, conditioning_batch],
tooltip=(
"Global-first negative conditioning aligned to the same masks; "
"Anima regional LoRA hooks retain their negative-branch ownership "
"and independent schedules."
"its global model hooks must match the positive global entry. "
"Regional LoRA hooks retain their negative-branch ownership and "
"independent schedules."
),
),
comfy_io.Mask.Input(
@@ -278,7 +280,7 @@ def attention_coupling_ksampler_inputs(
"Optional ordered masks paired with conditioning entries 1 onward. "
"Leave disconnected with ordinary conditioning to bypass Attention "
"Coupling. In overlaps, prompt contributions are normalized while "
"Anima regional LoRA deltas add in declared adapter and region order."
"regional LoRA deltas add in declared adapter and region order."
),
),
comfy_io.Float.Input(
@@ -291,7 +293,7 @@ def attention_coupling_ksampler_inputs(
tooltip=(
"Balances regional cross-attention against the global prompt from "
"0 (global only) to 1 (regional only inside solid masks); regional "
"Anima LoRA strength remains controlled by each hook."
"LoRA strength remains controlled by each hook."
),
),
comfy_io.Int.Input(
@@ -301,7 +303,7 @@ def attention_coupling_ksampler_inputs(
max=512,
step=1,
tooltip=(
"Softens Attention Coupling and Anima regional LoRA boundaries by "
"Softens Attention Coupling and regional LoRA boundaries by "
"this many image pixels; 0 preserves authored mask values."
),
),
@@ -54,12 +54,15 @@ class KSamplerTiledAttentionCouplingV3(_ComfyNodeBase):
"batches and masks, denoises large Anima and standard SD/SDXL "
"latents in tiles through "
"one shared model trajectory per tile batch while coupling global "
"and masked regional cross-attention. The input MODEL may carry "
"global LoRAs. Anima regions may carry independently scheduled "
"and masked regional cross-attention. LoRAs on the input MODEL and "
"Prompt Control model LoRAs on global conditioning entry 0 apply "
"across every tile. Regions may carry independently scheduled "
"regional LoRA stacks; inactive attention and LoRA work is pruned "
"without changing quality. MultiDiffusion or Mixture of Diffusers "
"fuses restored tile predictions. Standard SD/SDXL regional "
"model-side hooks and unsupported Anima targets fail before sampling."
"fuses restored tile predictions. Global LoRA and regional LoRA "
"stacks retain independent schedules; regional model-side hooks are "
"supported on admitted model families. Unsupported adapter targets "
"fail before sampling."
),
search_aliases=[
"attention coupling tiled",
@@ -60,8 +60,6 @@ _HIDDEN_INPUTS = {
"DYNPROMPT": "dynprompt",
"EXTRA_PNGINFO": "extra_pnginfo",
"UNIQUE_ID": "unique_id",
"AUTH_TOKEN_COMFY_ORG": "auth_token_comfy_org",
"API_KEY_COMFY_ORG": "api_key_comfy_org",
}
+56 -41
View File
@@ -8,16 +8,19 @@ from __future__ import annotations
from dataclasses import dataclass
import comfy.model_patcher
from comfy.patcher_extension import CallbacksMP
from ..model_attention_patch_mutations import ModelAttn2PatchesMutation
from ..model_patcher_mutations import ModelKeyedCallbackMutation
from ..patcher_lifecycle import PATCHER_LIFECYCLE, ModelMutation
from ..regional_lora.standard_unet_native_admission import (
StandardUnetNativeLoraAdmission,
from ..ppm_negpip_interop import PpmNegpipInterop
from ..regional_lora.operation_assembly import REGIONAL_OPERATION_ASSEMBLER
from ..regional_lora.standard_unet_operation_preparation import (
StandardUnetOperationAdmission,
)
from ..regional_lora.standard_unet_variant_runtime import (
StandardUnetVariantRuntimeMutation,
)
from ..regional_lora.standard_unet_variant_template import (
STANDARD_UNET_VARIANT_TEMPLATE_CACHE,
from ..regional_lora.standard_unet_operation_session import (
StandardUnetRegionalOperationSession,
)
from .unet_attention_context_wrapper import unet_attention_context_wrapper_mutation
from .unet_attention_phase_session import StandardUnetAttentionPhaseSession
@@ -42,60 +45,72 @@ class StandardUnetAttentionBackend:
*,
model: object,
state: StandardUnetAttentionState,
admission: StandardUnetNativeLoraAdmission,
admission: StandardUnetOperationAdmission,
negpip: PpmNegpipInterop | None = None,
) -> StandardUnetAttentionModel:
"""Return a direct MODEL child containing only the paired UNet patches."""
if not isinstance(state, StandardUnetAttentionState):
raise TypeError("Standard UNet backend requires attention state.")
if not isinstance(admission, StandardUnetNativeLoraAdmission):
raise TypeError("Standard UNet backend requires native admission.")
if not isinstance(admission, StandardUnetOperationAdmission):
raise TypeError("Standard UNet backend requires operation admission.")
if admission.adaptation.plan != state.plan.lora_plan:
raise ValueError(
"Standard UNet admission and processed conditioning must share "
"the same regional LoRA plan."
)
if negpip is not None and not isinstance(negpip, PpmNegpipInterop):
raise TypeError("Standard UNet backend NegPiP state has an invalid type.")
attention_phase = StandardUnetAttentionPhaseSession()
template = (
STANDARD_UNET_VARIANT_TEMPLATE_CACHE.resolve(model, admission)
if admission.adaptation.plan.adapters
else None
)
variant_mutations = (
(
StandardUnetVariantRuntimeMutation(
state,
admission,
attention_phase,
template,
operation_session: StandardUnetRegionalOperationSession | None = None
operation_mutations: tuple[ModelMutation, ...] = ()
if admission.adaptation.plan.adapters:
if (
not isinstance(model, comfy.model_patcher.ModelPatcher)
or admission.binding is None
or admission.cache is None
):
raise TypeError(
"Standard UNet regional operations require complete MODEL "
"admission."
)
assembly = REGIONAL_OPERATION_ASSEMBLER.assemble(
admission.binding,
model=model,
cache=admission.cache,
)
operation_session = StandardUnetRegionalOperationSession(
admission.adaptation.plan,
state.plan.mask_bank,
admission.module_roles,
assembly.call_scope,
)
operation_mutations = (
assembly.cache_lifecycle.mutation(),
ModelKeyedCallbackMutation(
CallbacksMP.ON_DETACH,
"simple_syrup.standard_unet_regional_operation_schedule",
operation_session.clear,
),
)
if template is not None
else ()
patches = UnetAttn2PatchPair(
StandardUnetAttn2ExecutionResolver(state),
operation_scope=operation_session,
)
derivation_source = (
template.bind_request(model) if template is not None else model
)
attention_mutations: tuple[ModelMutation, ...] = ()
if template is None:
patches = UnetAttn2PatchPair(
StandardUnetAttn2ExecutionResolver(state),
)
attention_mutations = (
ModelAttn2PatchesMutation(
patches.input_patch,
patches.output_patch,
),
)
derived = PATCHER_LIFECYCLE.derive_model(
derivation_source,
model,
(
unet_attention_context_wrapper_mutation(
state,
attention_phase,
operation_session,
),
*attention_mutations,
*variant_mutations,
ModelAttn2PatchesMutation(
patches.input_patch,
patches.output_patch,
(() if negpip is None else (negpip.attention_patch,)),
),
*operation_mutations,
),
operation="standard UNet Attention Coupling",
)
@@ -6,12 +6,17 @@
from __future__ import annotations
from contextlib import ExitStack
import torch
from ..diffusion_wrapper_executor import DiffusionWrapperExecutor
from ..diffusion_wrapper_invocation import DIFFUSION_WRAPPER_INVOCATION_VALIDATOR
from ..model_patcher_mutations import ModelDiffusionWrapperMutation
from ..regional_attention_model_call import RegionalAttentionModelCallResolver
from ..regional_lora.standard_unet_operation_session import (
StandardUnetRegionalOperationSession,
)
from .standard_unet_model_output_validation import (
STANDARD_UNET_MODEL_OUTPUT_VALIDATOR,
StandardUnetModelOutputValidator,
@@ -32,6 +37,7 @@ class StandardUnetAttentionContextDiffusionWrapper:
self,
state: StandardUnetAttentionState,
attention_phase: StandardUnetAttentionPhaseSession,
operation_session: StandardUnetRegionalOperationSession | None = None,
*,
model_call_resolver: RegionalAttentionModelCallResolver = (
STANDARD_UNET_MODEL_CALL_RESOLVER
@@ -46,12 +52,18 @@ class StandardUnetAttentionContextDiffusionWrapper:
raise TypeError("Standard UNet context wrapper requires attention state.")
if not isinstance(attention_phase, StandardUnetAttentionPhaseSession):
raise TypeError("Standard UNet context wrapper requires phase state.")
if operation_session is not None and not isinstance(
operation_session,
StandardUnetRegionalOperationSession,
):
raise TypeError("Standard UNet operation session has an invalid type.")
if not isinstance(model_call_resolver, RegionalAttentionModelCallResolver):
raise TypeError(
"Standard UNet context wrapper requires a model-call resolver."
)
self._state = state
self._attention_phase = attention_phase
self._operation_session = operation_session
self._model_call_resolver = model_call_resolver
if not isinstance(output_validator, StandardUnetModelOutputValidator):
raise TypeError("Standard UNet output validator has an invalid type.")
@@ -90,11 +102,14 @@ class StandardUnetAttentionContextDiffusionWrapper:
transformer_options=args[5],
)
forwarded_args = (*args[:2], contexts.base_context, *args[3:])
with (
self._attention_phase.activate(args[5]),
self._state.execution_context.activate(contexts),
self._state.resolution_cache.activate(),
):
with ExitStack() as scopes:
scopes.enter_context(self._attention_phase.activate(args[5]))
scopes.enter_context(self._state.execution_context.activate(contexts))
scopes.enter_context(self._state.resolution_cache.activate())
if self._operation_session is not None:
scopes.enter_context(
self._operation_session.activate(contexts, args[5])
)
output = executor(*forwarded_args, **kwargs)
return self._output_validator.validate(output, model_input=args[0])
@@ -102,6 +117,7 @@ class StandardUnetAttentionContextDiffusionWrapper:
def unet_attention_context_wrapper_mutation(
state: StandardUnetAttentionState,
attention_phase: StandardUnetAttentionPhaseSession,
operation_session: StandardUnetRegionalOperationSession | None = None,
) -> ModelDiffusionWrapperMutation:
"""Return the clone-local standard-UNet context wrapper mutation."""
@@ -110,5 +126,6 @@ def unet_attention_context_wrapper_mutation(
StandardUnetAttentionContextDiffusionWrapper(
state,
attention_phase,
operation_session,
),
)
@@ -6,6 +6,7 @@
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
from typing import Any, cast
@@ -50,6 +51,79 @@ class ClipHookScheduleMutation:
register_hooks(self.hooks, self.target)
@dataclass(frozen=True)
class ClipCallableObjectPatchMutation:
"""Patch one callable text-encoder object on a derived CLIP patcher."""
path: str
replacement: Callable[..., object]
def apply(self, clip: object) -> None:
"""Validate the path and collision state before installing the callback."""
if (
not isinstance(self.path, str)
or not self.path
or any(not segment for segment in self.path.split("."))
):
raise ValueError("CLIP callable patch path must be a dotted path.")
if not callable(self.replacement):
raise TypeError("CLIP callable object replacement must be callable.")
patcher = _required_attribute(clip, "patcher", value_name="CLIP")
getter = getattr(patcher, "get_model_object", None)
adder = getattr(patcher, "add_object_patch", None)
object_patches = getattr(patcher, "object_patches", None)
if (
not callable(getter)
or not callable(adder)
or not isinstance(object_patches, dict)
):
raise TypeError("CLIP patcher does not expose callable object patches.")
if self.path in object_patches:
raise ValueError(f"CLIP object path '{self.path}' already has a patch.")
if not callable(getter(self.path)):
raise TypeError(f"CLIP object path '{self.path}' must be callable.")
adder(self.path, self.replacement)
@dataclass(frozen=True)
class ClipTokenizerMutation:
"""Replace the tokenizer on a derived CLIP after exact source validation."""
expected_source: object
replacement: object
def apply(self, clip: object) -> None:
"""Install one tokenizer proxy only on the expected cloned source value."""
if getattr(clip, "tokenizer", None) is not self.expected_source:
raise ValueError("Derived CLIP tokenizer does not match its source.")
cast(Any, clip).tokenizer = self.replacement
@dataclass(frozen=True)
class ClipBooleanOptionMutation:
"""Publish one collision-safe boolean option on a derived CLIP patcher."""
key: str
value: bool
def apply(self, clip: object) -> None:
"""Set an approved ownership marker after validating the option mapping."""
if self.key not in {"ppm_negpip", "simple_syrup_negpip"}:
raise ValueError("Unsupported CLIP boolean option marker.")
if not isinstance(self.value, bool):
raise TypeError("CLIP option marker value must be boolean.")
patcher = _required_attribute(clip, "patcher", value_name="CLIP")
options = getattr(patcher, "model_options", None)
if not isinstance(options, dict):
raise TypeError("CLIP patcher model_options must be a dictionary.")
if self.key in options:
raise ValueError(f"CLIP option '{self.key}' is already present.")
options[self.key] = self.value
def _required_attribute(value: object, name: str, *, value_name: str) -> object:
"""Return a required dynamic ComfyUI boundary attribute."""
@@ -27,6 +27,7 @@ from ..services.attention_coupling_preparation_service import (
AttentionCouplingPreparation,
)
from .attention_coupling.context_validation import RegionalContextValidator
from .ppm_negpip_interop import PpmNegpipInterop
class ComfyRegionalConditioningProcessor:
@@ -40,6 +41,7 @@ class ComfyRegionalConditioningProcessor:
noise: torch.Tensor,
device: torch.device,
context_validator: RegionalContextValidator,
negpip: PpmNegpipInterop | None = None,
) -> ProcessedRegionalAttentionPlan:
"""Return model-ready positive and negative context banks."""
@@ -57,6 +59,8 @@ class ComfyRegionalConditioningProcessor:
raise TypeError("Regional context processing device must be torch.device.")
if not isinstance(context_validator, RegionalContextValidator):
raise TypeError("Regional context validator has an invalid type.")
if negpip is not None and not isinstance(negpip, PpmNegpipInterop):
raise TypeError("Regional conditioning NegPiP state has an invalid type.")
base_model = getattr(model, "model", None)
extra_conds = getattr(base_model, "extra_conds", None)
if not callable(extra_conds):
@@ -75,6 +79,7 @@ class ComfyRegionalConditioningProcessor:
noise=noise,
device=device,
context_validator=context_validator,
negpip=negpip,
)
negative = self._process_branch(
preparation.plan.negative,
@@ -84,6 +89,7 @@ class ComfyRegionalConditioningProcessor:
noise=noise,
device=device,
context_validator=context_validator,
negpip=negpip,
)
return ProcessedRegionalAttentionPlan(
positive=positive,
@@ -102,6 +108,7 @@ class ComfyRegionalConditioningProcessor:
noise: torch.Tensor,
device: torch.device,
context_validator: RegionalContextValidator,
negpip: PpmNegpipInterop | None,
) -> ProcessedRegionalAttentionBranch:
"""Process one base plus its ordered regional context bank."""
@@ -115,6 +122,7 @@ class ComfyRegionalConditioningProcessor:
noise=noise,
device=device,
context_validator=context_validator,
negpip=negpip,
)
regional = tuple(
self._process_context(
@@ -127,6 +135,7 @@ class ComfyRegionalConditioningProcessor:
noise=noise,
device=device,
context_validator=context_validator,
negpip=negpip,
)
for context in branch.regional_contexts
)
@@ -144,6 +153,7 @@ class ComfyRegionalConditioningProcessor:
noise: torch.Tensor,
device: torch.device,
context_validator: RegionalContextValidator,
negpip: PpmNegpipInterop | None,
) -> ProcessedRegionalAttentionContext:
"""Convert and extract one exact post-adapter Anima context tensor."""
@@ -168,6 +178,7 @@ class ComfyRegionalConditioningProcessor:
conditioning_index=conditioning_index,
prompt_type=prompt_type,
context_validator=context_validator,
negpip=negpip,
)
for entry_index, encoded_item in enumerate(encoded)
)
@@ -185,6 +196,7 @@ class ComfyRegionalConditioningProcessor:
conditioning_index: int,
prompt_type: str,
context_validator: RegionalContextValidator,
negpip: PpmNegpipInterop | None,
) -> ProcessedRegionalAttentionEntry:
"""Extract one exact post-adapter Anima context and Comfy strength."""
@@ -233,6 +245,11 @@ class ComfyRegionalConditioningProcessor:
),
cross_attention=context,
strength=float(strength),
cross_attention_value_multiplier=(
None
if negpip is None
else negpip.extract_value_multiplier(model_conds, context)
),
)
@staticmethod
@@ -49,6 +49,7 @@ class ContextualDiffusionModelWrapper:
self._tile_predictions = TilePredictionAccumulator(
plan.tile_plan,
diffusion_mode=diffusion_mode,
project_canvas_reference_latents=True,
)
@property
@@ -110,6 +111,7 @@ class ContextualDiffusionModelWrapper:
global_args = make_spatial_view_model_args(
args=args,
layout=global_layout,
project_canvas_reference_latents=True,
)
global_prediction = self._call_original(apply_model, global_args)
global_view = self._plan.global_view
@@ -0,0 +1,74 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Compose global-first conditioning hooks for conventional regional sampling."""
from __future__ import annotations
from typing import Any, TypeAlias
from comfy.hooks import HookGroup
from .regional_lora_conditioning_sources import conditioning_hook_groups
Conditioning: TypeAlias = list[list[Any]]
class GlobalFirstConditioningHookComposer:
"""Apply one global HookGroup to every regional conditioning model state."""
def global_hooks(
self,
conditioning: Conditioning,
*,
source_label: str,
) -> HookGroup | None:
"""Return the single uniform HookGroup carried by global conditioning."""
groups = conditioning_hook_groups(conditioning)
if not groups:
return None
authority = groups[0]
if any(group is not authority for group in groups[1:]):
raise ValueError(
f"{source_label} uses different HookGroups across conditioning "
"entries. Keep one shared Prompt Control hook schedule on the "
"global segment."
)
return authority
def compose(
self,
conditioning: Conditioning,
global_hooks: HookGroup | None,
*,
source_label: str,
cache: dict[tuple[HookGroup, HookGroup], HookGroup],
) -> Conditioning:
"""Prepend global hooks to every local HookGroup without mutating inputs."""
if global_hooks is None:
return [[item[0], dict(item[1])] for item in conditioning]
composed: Conditioning = []
for item_index, item in enumerate(conditioning):
metadata = dict(item[1])
local_hooks = metadata.get("hooks")
if local_hooks is None:
metadata["hooks"] = global_hooks
elif not isinstance(local_hooks, HookGroup):
raise TypeError(
f"{source_label} item {item_index} hooks must be a Comfy HookGroup."
)
else:
key = (global_hooks, local_hooks)
combined = cache.get(key)
if combined is None:
combined = global_hooks.clone_and_combine(local_hooks)
cache[key] = combined
metadata["hooks"] = combined
composed.append([item[0], metadata])
return composed
GLOBAL_FIRST_CONDITIONING_HOOK_COMPOSER = GlobalFirstConditioningHookComposer()
@@ -0,0 +1,63 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Resolve the effective Comfy MODEL before global conditioning hooks execute."""
from __future__ import annotations
from typing import cast
from ..domain.conditioning_batch import select_conditioning
from .regional_lora_conditioning_sources import conditioning_hook_groups
class GlobalHookModelResolver:
"""Mirror Comfy's dynamic-to-static handoff before regional derivation."""
def resolve(
self,
model: object,
*,
positive: object,
negative: object,
) -> object:
"""Return the model that Comfy will use for global hooked conditioning."""
if not self._has_global_hooks(positive) and not self._has_global_hooks(
negative
):
return model
is_dynamic = getattr(model, "is_dynamic", None)
if not callable(is_dynamic):
raise TypeError(
"Global conditioning hooks require MODEL dynamic-mode state."
)
dynamic = is_dynamic()
if not isinstance(dynamic, bool):
raise TypeError("MODEL is_dynamic() must return a bool.")
if not dynamic:
return model
delegate_factory = getattr(model, "get_non_dynamic_delegate", None)
if not callable(delegate_factory):
raise TypeError(
"Dynamic MODEL global conditioning hooks require Comfy's "
"get_non_dynamic_delegate()."
)
resolved = delegate_factory()
if resolved is model:
raise RuntimeError("Dynamic MODEL returned itself as its static delegate.")
resolved_is_dynamic = getattr(resolved, "is_dynamic", None)
if not callable(resolved_is_dynamic) or resolved_is_dynamic() is not False:
raise RuntimeError("Global conditioning hook delegate must be static.")
return cast(object, resolved)
@staticmethod
def _has_global_hooks(conditioning: object) -> bool:
"""Report hooks only on consumer-defined global entry zero."""
global_conditioning = select_conditioning(conditioning, 0)
return bool(conditioning_hook_groups(global_conditioning))
GLOBAL_HOOK_MODEL_RESOLVER = GlobalHookModelResolver()
@@ -22,6 +22,7 @@ def _apply_paired_attention_patches(
attention_name: str,
input_patch: Callable[..., object],
output_patch: Callable[..., object],
trailing_input_patches: tuple[Callable[..., object], ...],
) -> None:
"""Validate and atomically install one paired attention callback surface."""
@@ -29,6 +30,12 @@ def _apply_paired_attention_patches(
raise TypeError(f"MODEL {attention_name} input patch must be callable.")
if not callable(output_patch):
raise TypeError(f"MODEL {attention_name} output patch must be callable.")
if not isinstance(trailing_input_patches, tuple) or any(
not callable(patch) for patch in trailing_input_patches
):
raise TypeError(
f"MODEL {attention_name} preserved input patches must be callables."
)
input_setter = _require_bound_method(
model,
f"set_model_{attention_name}_patch",
@@ -54,11 +61,21 @@ def _apply_paired_attention_patches(
output_name = f"{attention_name}_output_patch"
input_exists = _require_callable_patch_list(patches, input_name)
output_exists = _require_callable_patch_list(patches, output_name)
if input_exists:
existing_input = patches.get(input_name, [])
if input_exists and (
not trailing_input_patches or existing_input != list(trailing_input_patches)
):
raise ValueError(f"MODEL {attention_name} input patch is already installed.")
if not input_exists and trailing_input_patches:
raise ValueError(
f"MODEL {attention_name} preserved input patch is not installed."
)
if output_exists:
raise ValueError(f"MODEL {attention_name} output patch is already installed.")
input_setter(input_patch)
if trailing_input_patches:
patches[input_name] = [input_patch, *trailing_input_patches]
else:
input_setter(input_patch)
output_setter(output_patch)
@@ -68,6 +85,7 @@ class ModelAttn2PatchesMutation:
input_patch: Callable[..., object]
output_patch: Callable[..., object]
trailing_input_patches: tuple[Callable[..., object], ...] = ()
def apply(self, model: object) -> None:
"""Validate both attn2 surfaces before either mutation."""
@@ -77,4 +95,5 @@ class ModelAttn2PatchesMutation:
attention_name="attn2",
input_patch=self.input_patch,
output_patch=self.output_patch,
trailing_input_patches=self.trailing_input_patches,
)
+308 -2
View File
@@ -2,7 +2,7 @@
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Known model metadata for grounded SAM masking."""
"""Known model metadata for downloadable SimpleSyrup model loaders."""
from __future__ import annotations
@@ -11,13 +11,14 @@ from enum import StrEnum
class ModelFamily(StrEnum):
"""Catalog families used by grounded SAM model selection."""
"""Catalog families used by SimpleSyrup model selection."""
SAM = "sam"
GROUNDING_DINO = "grounding_dino"
TEXT_ENCODER = "text_encoder"
VITMATTE = "vitmatte"
WD14_TAGGER = "wd14_tagger"
ULTRALYTICS = "ultralytics"
@dataclass(frozen=True)
@@ -29,6 +30,7 @@ class ModelArtifact:
folder_name: str
source_url: str
description: str
sha256: str | None = None
@dataclass(frozen=True)
@@ -386,6 +388,298 @@ WD14_TAGGER_ENTRIES: tuple[ModelEntry, ...] = (
)
_ANZHCS_YOLOS_REVISION = "f5a2306d7fed4f3cfc26c25ff1ab2e3f3cfce855"
_ANZHCS_YOLOS_REPOSITORY = "Anzhc/Anzhcs_YOLOs"
def _huggingface_yolo_entry(
*,
entry_id: str,
display_name: str,
filename: str,
folder_name: str,
model_type: str,
source_repo: str,
revision: str,
license_note: str,
description: str,
sha256: str,
) -> ModelEntry:
"""Build one revision-pinned Hugging Face Ultralytics catalog entry."""
encoded_filename = filename.replace(" ", "%20")
return ModelEntry(
entry_id=entry_id,
display_name=display_name,
family=ModelFamily.ULTRALYTICS,
model_type=model_type,
source_repo=source_repo,
license_note=license_note,
artifacts=(
ModelArtifact(
artifact_id=f"{entry_id}_checkpoint",
filename=filename,
folder_name=folder_name,
source_url=(
f"https://huggingface.co/{source_repo}/resolve/{revision}/"
f"{encoded_filename}"
),
description=description,
sha256=sha256,
),
),
)
def _anzhc_yolo_entry(
*,
entry_id: str,
display_name: str,
filename: str,
folder_name: str,
model_type: str,
description: str,
sha256: str,
) -> ModelEntry:
"""Build one revision-pinned Anzhc Ultralytics catalog entry."""
return _huggingface_yolo_entry(
entry_id=entry_id,
display_name=display_name,
filename=filename,
folder_name=folder_name,
model_type=model_type,
source_repo=_ANZHCS_YOLOS_REPOSITORY,
revision=_ANZHCS_YOLOS_REVISION,
license_note="AGPL-3.0",
description=description,
sha256=sha256,
)
ULTRALYTICS_ENTRIES: tuple[ModelEntry, ...] = (
_anzhc_yolo_entry(
entry_id="anzhc_face_seg",
display_name="Anzhc Face -seg (6.52MB)",
filename="Anzhc Face -seg.pt",
folder_name="ultralytics_segm",
model_type="segment",
description="Anzhc face segmentation model",
sha256="dbf083201298a495e332113de0612d1be1ae8307628628eb7972a31979cdbbb3",
),
_anzhc_yolo_entry(
entry_id="anzhc_face_seg_640_v2_y8n",
display_name="Anzhc Face seg 640 v2 y8n (6.56MB)",
filename="Anzhc Face seg 640 v2 y8n.pt",
folder_name="ultralytics_segm",
model_type="segment",
description="Anzhc face segmentation model",
sha256="d473e8bccc4c833d8eb36c95e566ce6460ffdc8b2899c859910e380c85def276",
),
_anzhc_yolo_entry(
entry_id="anzhc_face_seg_768_v2_y8n",
display_name="Anzhc Face seg 768 v2 y8n (6.58MB)",
filename="Anzhc Face seg 768 v2 y8n.pt",
folder_name="ultralytics_segm",
model_type="segment",
description="Anzhc face segmentation model",
sha256="9a1e5b154c1d190812447431bda6b8f260f132877812b4a2f163981f54558355",
),
_anzhc_yolo_entry(
entry_id="anzhc_face_seg_768ms_v2_y8n",
display_name="Anzhc Face seg 768MS v2 y8n (6.60MB)",
filename="Anzhc Face seg 768MS v2 y8n.pt",
folder_name="ultralytics_segm",
model_type="segment",
description="Anzhc multi-scale face segmentation model",
sha256="429e88d9aecb9fa4167ffd41a6ebc42c97b7fa785aa5468a7eb302ceb9837aae",
),
_anzhc_yolo_entry(
entry_id="anzhc_face_seg_1024_v2_y8n",
display_name="Anzhc Face seg 1024 v2 y8n (6.63MB)",
filename="Anzhc Face seg 1024 v2 y8n.pt",
folder_name="ultralytics_segm",
model_type="segment",
description="Anzhc face segmentation model",
sha256="1bbcfd7a9f407c6f6e4389a371dbcc392f9444421cf7f824152e92bf563dc6a3",
),
_anzhc_yolo_entry(
entry_id="anzhc_face_seg_640_v3_y11n",
display_name="Anzhc Face seg 640 v3 y11n (5.80MB)",
filename="Anzhc Face seg 640 v3 y11n.pt",
folder_name="ultralytics_segm",
model_type="segment",
description="Anzhc YOLO11 face segmentation model",
sha256="96437afc773bacd118e275e6cddc1fb7263c78dc11299989c7a00a26506c45bf",
),
_anzhc_yolo_entry(
entry_id="anzhc_face_seg_640_v4_y11n",
display_name="Anzhc Face seg 640 v4 y11n (5.74MB)",
filename="Anzhc Face seg 640 v4 y11n.pt",
folder_name="ultralytics_segm",
model_type="segment",
description="Anzhc YOLO11 face segmentation model",
sha256="1e77ad7bd349babd8a4a90478bfc965348642b63a8d95d3b43ee13db42fd0a64",
),
_anzhc_yolo_entry(
entry_id="anzhcs_manface_v02_1024_y8n",
display_name="Anzhcs ManFace v02 1024 y8n (6.06MB)",
filename="Anzhcs ManFace v02 1024 y8n.pt",
folder_name="ultralytics_segm",
model_type="segment",
description="Anzhc male face segmentation model",
sha256="184b9a680afb3c4a559e46e2fe692338fe7bdd6267979fa4ef10526fa96c1b31",
),
_anzhc_yolo_entry(
entry_id="anzhcs_womanface_v05_1024_y8n",
display_name="Anzhcs WomanFace v05 1024 y8n (6.07MB)",
filename="Anzhcs WomanFace v05 1024 y8n.pt",
folder_name="ultralytics_segm",
model_type="segment",
description="Anzhc female face segmentation model",
sha256="84db37616e1ca975c4e23fa5a300acf0edd9144ec287bbbdbd1ad0f4a3afa9c1",
),
_anzhc_yolo_entry(
entry_id="anzhc_eyes_seg_hd",
display_name="Anzhc Eyes -seg-hd (6.59MB)",
filename="Anzhc Eyes -seg-hd.pt",
folder_name="ultralytics_segm",
model_type="segment",
description="Anzhc eye segmentation model",
sha256="6be1c13ca7a51c2425e278e07e7ae3d4c94ee125b874a0104a142f4f5a35a308",
),
_anzhc_yolo_entry(
entry_id="anzhc_headhair_seg_y8n",
display_name="Anzhc HeadHair seg y8n (6.50MB)",
filename="Anzhc HeadHair seg y8n.pt",
folder_name="ultralytics_segm",
model_type="segment",
description="Anzhc head and hair segmentation model",
sha256="a6e99b1305f600c35e7f6400741c2322b198ae03755f91dc1c59d7a78d77f13c",
),
_anzhc_yolo_entry(
entry_id="anzhc_headhair_seg_y8m",
display_name="Anzhc HeadHair seg y8m (52.34MB)",
filename="Anzhc HeadHair seg y8m.pt",
folder_name="ultralytics_segm",
model_type="segment",
description="Anzhc head and hair segmentation model",
sha256="f63aa1cdb63a26c0025a4a984588248241a5838aff4edfeea93d9c155efe0b5e",
),
_anzhc_yolo_entry(
entry_id="anzhc_breasts_seg_v1_1024n",
display_name="Anzhc Breasts Seg v1 1024n (6.58MB)",
filename="Anzhc Breasts Seg v1 1024n.pt",
folder_name="ultralytics_segm",
model_type="segment",
description="Anzhc breast segmentation model",
sha256="d469bd7abdcbe32a946e0e342bc1fe96aa021987787d51245f97a29e114cb31b",
),
_anzhc_yolo_entry(
entry_id="anzhc_breasts_seg_v1_1024s",
display_name="Anzhc Breasts Seg v1 1024s (22.86MB)",
filename="Anzhc Breasts Seg v1 1024s.pt",
folder_name="ultralytics_segm",
model_type="segment",
description="Anzhc breast segmentation model",
sha256="413a9b948a40f96a83769a882816ef0dd2b91b49673c91bff75463660077b395",
),
_anzhc_yolo_entry(
entry_id="anzhc_breasts_seg_v1_1024m",
display_name="Anzhc Breasts Seg v1 1024m (52.39MB)",
filename="Anzhc Breasts Seg v1 1024m.pt",
folder_name="ultralytics_segm",
model_type="segment",
description="Anzhc breast segmentation model",
sha256="53d15e82a8308f8056f4929838e00e42c8da576b661e0c2b4fef5837d8b5b2b4",
),
_huggingface_yolo_entry(
entry_id="bingsu_face_yolov8n_v2",
display_name="Bingsu Face YOLOv8n v2 (6.23MB)",
filename="face_yolov8n_v2.pt",
folder_name="ultralytics_bbox",
model_type="detect",
source_repo="Bingsu/adetailer",
revision="53cc19de382014514d9d4038601d261a7faa9b7b",
license_note="Apache-2.0",
description="Bingsu ADetailer face detection model",
sha256="8f5f2110f83c4e00712993fab48c771d26036e2e80ec62bd5b9cb37c29e36b36",
),
_huggingface_yolo_entry(
entry_id="bingsu_face_yolov8s",
display_name="Bingsu Face YOLOv8s (22.5MB)",
filename="face_yolov8s.pt",
folder_name="ultralytics_bbox",
model_type="detect",
source_repo="Bingsu/adetailer",
revision="53cc19de382014514d9d4038601d261a7faa9b7b",
license_note="Apache-2.0",
description="Bingsu ADetailer face detection model",
sha256="c7237eff25787377de196961140ceaed324d859ee8de5a775d93d33a0e3fab78",
),
_huggingface_yolo_entry(
entry_id="bingsu_hand_yolov8n",
display_name="Bingsu Hand YOLOv8n (6.23MB)",
filename="hand_yolov8n.pt",
folder_name="ultralytics_bbox",
model_type="detect",
source_repo="Bingsu/adetailer",
revision="53cc19de382014514d9d4038601d261a7faa9b7b",
license_note="Apache-2.0",
description="Bingsu ADetailer hand detection model",
sha256="3991202eb69e9ddcb3b9ba80cdeb41e734ffaf844403d6c9f47d515cd88c6f29",
),
_huggingface_yolo_entry(
entry_id="bingsu_hand_yolov8s",
display_name="Bingsu Hand YOLOv8s (22.5MB)",
filename="hand_yolov8s.pt",
folder_name="ultralytics_bbox",
model_type="detect",
source_repo="Bingsu/adetailer",
revision="53cc19de382014514d9d4038601d261a7faa9b7b",
license_note="Apache-2.0",
description="Bingsu ADetailer hand detection model",
sha256="70b540063fbc385736d8258970744a4afbc4cbf7932134bae3b24cdadeadec06",
),
_huggingface_yolo_entry(
entry_id="bingsu_person_yolov8n_seg",
display_name="Bingsu Person YOLOv8n-seg (6.78MB)",
filename="person_yolov8n-seg.pt",
folder_name="ultralytics_segm",
model_type="segment",
source_repo="Bingsu/adetailer",
revision="53cc19de382014514d9d4038601d261a7faa9b7b",
license_note="Apache-2.0",
description="Bingsu ADetailer person segmentation model",
sha256="38fc8aaae97cb6e70be4ec44770005b26ed473471362afcda62a0037d7ccf432",
),
_huggingface_yolo_entry(
entry_id="bingsu_person_yolov8s_seg",
display_name="Bingsu Person YOLOv8s-seg (23.9MB)",
filename="person_yolov8s-seg.pt",
folder_name="ultralytics_segm",
model_type="segment",
source_repo="Bingsu/adetailer",
revision="53cc19de382014514d9d4038601d261a7faa9b7b",
license_note="Apache-2.0",
description="Bingsu ADetailer person segmentation model",
sha256="53c54aec2239355faffc6c5b70d0f3d05042f386f956cbec39cec46ad456f050",
),
_huggingface_yolo_entry(
entry_id="fuyucchi_yolov8x6_animeface",
display_name="Fuyucchi YOLOv8x6 Anime Face (195MB)",
filename="yolov8x6_animeface.pt",
folder_name="ultralytics_bbox",
model_type="detect",
source_repo="Fuyucchi/yolov8_animeface",
revision="b0841ce930453c0f23ceb8086d6554c17de5fe4a",
license_note="AGPL-3.0",
description="Fuyucchi high-resolution anime face detection model",
sha256="f3cdc1a6266347322439fd9b3c8f5a1222668eb10c8adf00e17b28c48b95213c",
),
)
def sam_choices() -> list[str]:
"""Return deterministic SAM dropdown choices."""
@@ -410,6 +704,12 @@ def wd14_tagger_choices() -> list[str]:
return [entry.display_name for entry in WD14_TAGGER_ENTRIES]
def ultralytics_choices() -> list[str]:
"""Return deterministic Ultralytics dropdown choices."""
return [entry.display_name for entry in ULTRALYTICS_ENTRIES]
def get_sam_entry(selection: str) -> ModelEntry:
"""Return the SAM catalog entry matching an id or display name."""
@@ -434,6 +734,12 @@ def get_wd14_tagger_entry(selection: str) -> ModelEntry:
return _get_entry(selection, WD14_TAGGER_ENTRIES, "WD14 tagger")
def get_ultralytics_entry(selection: str) -> ModelEntry:
"""Return the Ultralytics catalog entry matching an id or display name."""
return _get_entry(selection, ULTRALYTICS_ENTRIES, "Ultralytics")
def _get_entry(
selection: str,
entries: tuple[ModelEntry, ...],
+18 -64
View File
@@ -6,24 +6,17 @@
from __future__ import annotations
from types import ModuleType
from typing import Protocol
from .model_catalog import (
GROUNDING_DINO_ENTRIES,
SAM_ENTRIES,
VITMATTE_ENTRIES,
WD14_TAGGER_ENTRIES,
ModelEntry,
grounding_dino_choices,
sam_choices,
ultralytics_choices,
vitmatte_choices,
wd14_tagger_choices,
)
from .model_folders import resolve_model_file
from .settings import SimpleSyrupSettings
from .settings_repository import SimpleSyrupSettingsRepository
from .vitmatte_loader import ViTMatteLoaderService
NO_LOCAL_SAM_MODELS = "No local SAM models found"
NO_LOCAL_GROUNDING_DINO_MODELS = "No local GroundingDINO models found"
@@ -44,69 +37,52 @@ class ModelChoiceService:
def __init__(
self,
settings_repository: SettingsProvider | None = None,
folder_paths_module: ModuleType | None = None,
) -> None:
"""Create the choice service with injectable external boundaries."""
"""Create the choice service with an injectable settings boundary."""
self._settings_repository = (
settings_repository or SimpleSyrupSettingsRepository()
)
self._folder_paths_module = folder_paths_module
self._vitmatte_loader = ViTMatteLoaderService(
folder_paths_module=folder_paths_module
)
def sam_choices(self) -> list[str]:
"""Return settings-aware SAM dropdown choices."""
"""Return curated SAM choices when catalog models are visible."""
if self._show_downloadable_models():
return sam_choices()
choices = [
entry.display_name
for entry in SAM_ENTRIES
if self._entry_artifacts_are_local(entry)
]
return choices or [NO_LOCAL_SAM_MODELS]
return [NO_LOCAL_SAM_MODELS]
def grounding_dino_choices(self) -> list[str]:
"""Return settings-aware GroundingDINO dropdown choices."""
"""Return curated GroundingDINO choices when catalog models are visible."""
if self._show_downloadable_models():
return grounding_dino_choices()
choices = [
entry.display_name
for entry in GROUNDING_DINO_ENTRIES
if self._entry_artifacts_are_local(entry)
]
return choices or [NO_LOCAL_GROUNDING_DINO_MODELS]
return [NO_LOCAL_GROUNDING_DINO_MODELS]
def vitmatte_choices(self) -> list[str]:
"""Return settings-aware ViTMatte dropdown choices."""
"""Return curated ViTMatte choices when catalog models are visible."""
if self._show_downloadable_models():
return vitmatte_choices()
choices = [
entry.display_name
for entry in VITMATTE_ENTRIES
if self._vitmatte_entry_is_local(entry)
]
return choices or [NO_LOCAL_VITMATTE_MODELS]
return [NO_LOCAL_VITMATTE_MODELS]
def wd14_tagger_choices(self) -> list[str]:
"""Return settings-aware WD14 tagger dropdown choices."""
"""Return curated WD14 tagger choices when catalog models are visible."""
if self._show_downloadable_models():
return wd14_tagger_choices()
choices = [
entry.display_name
for entry in WD14_TAGGER_ENTRIES
if self._entry_artifacts_are_local(entry)
]
return choices or [NO_LOCAL_WD14_TAGGER_MODELS]
return [NO_LOCAL_WD14_TAGGER_MODELS]
def ultralytics_choices(self) -> list[str]:
"""Return curated Ultralytics choices when catalog models are visible."""
if self._show_downloadable_models():
return ultralytics_choices()
return []
def reject_sentinel(self, selection: str) -> None:
"""Reject placeholder dropdown selections before loader work begins."""
@@ -143,28 +119,6 @@ class ModelChoiceService:
return self._settings_repository.load().show_downloadable_models
def _entry_artifacts_are_local(self, entry: ModelEntry) -> bool:
"""Return whether every catalog artifact exists locally."""
return all(
resolve_model_file(
artifact.folder_name,
artifact.filename,
self._folder_paths_module,
)
is not None
for artifact in entry.artifacts
)
def _vitmatte_entry_is_local(self, entry: ModelEntry) -> bool:
"""Return whether a valid ViTMatte directory exists locally."""
try:
self._vitmatte_loader.resolve_model_directory(entry, auto_download=False)
except FileNotFoundError:
return False
return True
def default_choice(choices: list[str], preferred: str) -> str:
"""Return the preferred default when visible, otherwise the first choice."""
+11 -10
View File
@@ -59,30 +59,30 @@ class ComfyProgressReporter:
"""Initialize an empty ComfyUI progress reporter."""
self._progress_bar: object | None = None
self._total = 1
self._total: int | None = None
def start(self, label: str, total: int | None) -> None:
"""Create a ComfyUI progress bar for one artifact."""
comfy_utils = importlib.import_module("comfy.utils")
progress_bar_class = comfy_utils.ProgressBar
self._total = total if total and total > 0 else 1
self._progress_bar = progress_bar_class(self._total)
self._total = total if total and total > 0 else None
progress_total = self._total or 1
self._progress_bar = progress_bar_class(progress_total)
self.advance(0, total)
LOGGER.info("download progress started", extra={"label": label, "total": total})
def advance(self, current: int, total: int | None) -> None:
"""Update the ComfyUI progress bar."""
"""Update known-size downloads without falsely completing unknown ones."""
if self._progress_bar is None:
return
if total and total > 0 and total != self._total:
if total is None or total <= 0:
return
if total != self._total:
self._total = total
value = (
current if total and total > 0 else min(current // CHUNK_SIZE, self._total)
)
progress_bar = cast(_ComfyProgressBar, self._progress_bar)
progress_bar.update_absolute(value, self._total)
progress_bar.update_absolute(current, self._total)
def finish(self) -> None:
"""Mark the current ComfyUI progress bar complete."""
@@ -90,7 +90,8 @@ class ComfyProgressReporter:
if self._progress_bar is None:
return
progress_bar = cast(_ComfyProgressBar, self._progress_bar)
progress_bar.update_absolute(self._total, self._total)
total = self._total or 1
progress_bar.update_absolute(total, total)
@dataclass(frozen=True)
+1
View File
@@ -78,6 +78,7 @@ class GroundedSAMModelMetadata:
"artifact_id": artifact.artifact_id,
"filename": artifact.filename,
"source_url": artifact.source_url,
"sha256": artifact.sha256,
"expected_path": str(expected),
"local_path": str(local_path) if local_path else None,
"installed": local_path is not None,
@@ -237,6 +237,102 @@ class ModelDiffusionWrapperMutation:
).apply(model)
@dataclass(frozen=True)
class ModelInteropDiffusionWrapperMutation:
"""Install the exact legacy key required for PPM Anima interoperability."""
key: str
wrapper: Callable[..., object]
def apply(self, model: object) -> None:
"""Install only the documented PPM Anima wrapper surface."""
if self.key != "ppm_negpip_anima":
raise ValueError("NegPiP interop wrapper must use PPM's Anima key.")
getter = _require_bound_method(model, "get_wrappers", ("wrapper_type", "key"))
adder = _require_bound_method(
model,
"add_wrapper_with_key",
("wrapper_type", "key", "wrapper"),
)
existing = getter(WrappersMP.DIFFUSION_MODEL, self.key)
if not isinstance(existing, list) or any(
not callable(callback) for callback in existing
):
raise TypeError("Existing NegPiP wrappers must be a callable list.")
if existing:
raise ValueError("PPM's Anima NegPiP wrapper key is already installed.")
adder(WrappersMP.DIFFUSION_MODEL, self.key, self.wrapper)
@dataclass(frozen=True)
class ModelAttentionPatchMutation:
"""Append one validated Comfy attention patch to a derived MODEL."""
patch_name: str
callback: Callable[..., object]
def apply(self, model: object) -> None:
"""Install an attn1 or attn2 callback through the public patcher setter."""
if self.patch_name not in {"attn1", "attn2"}:
raise ValueError("MODEL attention patch name must be 'attn1' or 'attn2'.")
if not callable(self.callback):
raise TypeError("MODEL attention patch callback must be callable.")
setter = getattr(model, f"set_model_{self.patch_name}_patch", None)
if not callable(setter):
raise TypeError(f"MODEL does not support {self.patch_name} patches.")
setter(self.callback)
@dataclass(frozen=True)
class ModelBooleanOptionMutation:
"""Publish one collision-safe boolean MODEL option marker."""
key: str
value: bool
def apply(self, model: object) -> None:
"""Set one supported marker only when no value already owns the key."""
if self.key != "ppm_negpip":
raise ValueError("Unsupported MODEL boolean option marker.")
if not isinstance(self.value, bool):
raise TypeError("MODEL option marker value must be boolean.")
options = _require_dictionary_attribute(model, "model_options")
if self.key in options:
raise ValueError(f"MODEL option '{self.key}' is already present.")
options[self.key] = self.value
@dataclass(frozen=True)
class ModelCallableObjectPatchMutation:
"""Replace one callable model object after collision validation."""
path: str
replacement: Callable[..., object]
def apply(self, model: object) -> None:
"""Patch one callable path without relying on bound-method identity."""
if (
not isinstance(self.path, str)
or not self.path
or any(not segment for segment in self.path.split("."))
):
raise ValueError("MODEL callable patch path must be a dotted path.")
if not callable(self.replacement):
raise TypeError("MODEL callable object replacement must be callable.")
getter = _require_bound_method(model, "get_model_object", ("name",))
adder = _require_bound_method(model, "add_object_patch", ("name", "obj"))
object_patches = _require_dictionary_attribute(model, "object_patches")
if self.path in object_patches:
raise ValueError(f"MODEL object path '{self.path}' already has a patch.")
if not callable(getter(self.path)):
raise TypeError(f"MODEL object path '{self.path}' must be callable.")
adder(self.path, self.replacement)
@dataclass(frozen=True)
class ModelExactObjectPatchMutation:
"""Replace one exact model object after collision and identity validation."""
+5
View File
@@ -0,0 +1,5 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Provide model-family NegPiP runtime adapters."""
+108
View File
@@ -0,0 +1,108 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Apply PPM-compatible value-mask NegPiP behavior to Anima."""
# NegPiP behavior is adapted from ComfyUI-ppm and its credited predecessors.
# See third_party/manifest.toml and third_party/NOTICE.md.
from __future__ import annotations
from collections.abc import Callable
from typing import Any
import torch
from comfy import conds
WRAPPER_KEY = "ppm_negpip_anima"
CONDITION_MASK_KEY = "c_ppm_negpip_mask"
TRANSFORMER_MASK_KEY = "ppm_negpip_mask"
def anima_extra_conds_negpip_wrapper(
previous_extra_conds: Callable[..., dict[str, object]],
) -> Callable[..., dict[str, object]]:
"""Convert signed T5 weights into a model condition while preserving magnitude."""
def wrapped_extra_conds(**kwargs: object) -> dict[str, object]:
"""Publish a sequence-aligned value multiplier for one conditioning."""
weights = kwargs.get("t5xxl_weights")
multiplier: torch.Tensor | None = None
if weights is not None:
if not isinstance(weights, torch.Tensor):
raise TypeError("Anima NegPiP T5 weights must be a tensor.")
magnitude = weights.abs()
multiplier = (
torch.where(
weights < 0.0,
weights.new_tensor(-1.0),
weights.new_tensor(1.0),
)
.unsqueeze(0)
.unsqueeze(-1)
)
if multiplier.shape[1] < 512:
multiplier = torch.nn.functional.pad(
multiplier,
(0, 0, 0, 512 - multiplier.shape[1]),
value=1.0,
)
kwargs["t5xxl_weights"] = magnitude
output = previous_extra_conds(**kwargs)
if not isinstance(output, dict):
raise TypeError("Anima extra conditions must be a dictionary.")
if multiplier is not None:
output[CONDITION_MASK_KEY] = conds.CONDRegular(multiplier)
return output
return wrapped_extra_conds
def anima_diffusion_negpip_wrapper(
executor: Callable[..., object],
*args: object,
**kwargs: object,
) -> object:
"""Move the processed Anima multiplier into isolated transformer options."""
if len(args) < 3 or not isinstance(args[2], torch.Tensor):
raise TypeError("Anima NegPiP wrapper requires tensor conditioning context.")
context = args[2]
transformer_options = kwargs.get("transformer_options", {})
if not isinstance(transformer_options, dict):
raise TypeError("Anima transformer options must be a dictionary.")
prepared = transformer_options.copy()
multiplier = kwargs.get(CONDITION_MASK_KEY)
if multiplier is not None:
if not isinstance(multiplier, torch.Tensor):
raise TypeError("Anima NegPiP multiplier must be a tensor.")
prepared[TRANSFORMER_MASK_KEY] = multiplier.to(context)
kwargs["transformer_options"] = prepared
return executor(*args, **kwargs)
def anima_attn2_negpip(
query: torch.Tensor,
key: torch.Tensor,
value: torch.Tensor,
pe: torch.Tensor | None = None,
attn_mask: torch.Tensor | None = None,
extra_options: dict[str, Any] | None = None,
) -> dict[str, torch.Tensor | None]:
"""Apply the signed multiplier only to Anima cross-attention values."""
multiplier = (
None if extra_options is None else extra_options.get(TRANSFORMER_MASK_KEY)
)
if multiplier is not None and not isinstance(multiplier, torch.Tensor):
raise TypeError("Anima NegPiP attention multiplier must be a tensor.")
return {
"q": query,
"k": key,
"v": value if multiplier is None else value * multiplier,
"pe": pe,
"attn_mask": attn_mask,
}
+291
View File
@@ -0,0 +1,291 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Adapt NegPiP value masking to Krea 2's layered Qwen conditioning."""
# NegPiP behavior is adapted from ComfyUI-ppm and its credited predecessors.
# See third_party/manifest.toml and third_party/NOTICE.md.
from __future__ import annotations
from collections.abc import Callable, Sequence
from typing import Any
import torch
from comfy import conds
CLIP_MARKER = "simple_syrup_negpip"
WRAPPER_KEY = "simple_syrup.negpip.krea2"
ENCODER_MASK_KEY = "simple_syrup_negpip_mask"
CONDITION_MASK_KEY = "c_simple_syrup_negpip_mask"
TRANSFORMER_MASK_KEY = "simple_syrup_negpip_mask"
KREA_TOKEN_KEY = "qwen3vl_4b"
IM_START_TOKEN = 151644
USER_TOKEN = 872
NEWLINE_TOKEN = 198
IMAGE_PAD_TOKEN = 151655
class Krea2NegpipTokenizer:
"""Preserve Krea templates while enabling Comfy prompt-weight tokenization."""
def __init__(self, source: object) -> None:
"""Retain one cloned CLIP's shared source tokenizer without mutating it."""
self._source = source
def __getattr__(self, name: str) -> object:
"""Delegate tokenizer metadata and helpers to the installed Krea tokenizer."""
return getattr(self._source, name)
def tokenize_with_weights(
self,
text: str,
return_word_ids: bool = False,
llama_template: str | None = None,
images: Sequence[torch.Tensor] = (),
prevent_empty_text: bool = False,
thinking: bool = True,
**kwargs: object,
) -> dict[str, list[list[tuple[object, ...]]]]:
"""Tokenize the normal Krea template while retaining parsed scalar weights."""
image = kwargs.pop("image", None)
if image is not None and not images:
if not isinstance(image, torch.Tensor):
raise TypeError("Krea tokenizer image input must be a tensor.")
images = tuple(image[index : index + 1] for index in range(image.shape[0]))
skip_template = bool(kwargs.pop("skip_template", False)) or text.startswith(
"<|im_start|>"
)
kwargs.pop("disable_weights", None)
if prevent_empty_text and text == "":
text = " "
if skip_template:
prepared_text = text
else:
template = llama_template
if template is None:
template_name = (
"llama_template" if not images else "llama_template_images"
)
template = getattr(self._source, template_name)
if not isinstance(template, str):
raise TypeError("Krea tokenizer template must be text.")
if len(images) > 1:
vision_block = "<|vision_start|><|image_pad|><|vision_end|>"
template = template.replace(
vision_block,
vision_block * len(images),
1,
)
prepared_text = template.format(text)
if not thinking:
prepared_text += "<think>\n\n</think>\n\n"
inner = getattr(self._source, KREA_TOKEN_KEY)
tokens = inner.tokenize_with_weights(
prepared_text,
return_word_ids=return_word_ids,
disable_weights=False,
**kwargs,
)
embedded_count = 0
for section in tokens:
for index, pair in enumerate(section):
token = pair[0]
if (
isinstance(token, (int, float))
and token == IMAGE_PAD_TOKEN
and embedded_count < len(images)
):
section[index] = (
{
"type": "image",
"data": images[embedded_count],
"original_type": "image",
},
*pair[1:],
)
embedded_count += 1
return {KREA_TOKEN_KEY: tokens}
def encode_krea2_token_weights_negpip(
original: Callable[..., tuple[object, ...]],
token_weight_pairs: dict[str, list[list[tuple[object, ...]]]],
template_end: int = -1,
) -> tuple[object, ...]:
"""Encode absolute Krea magnitudes and publish a post-template sign mask."""
sections = token_weight_pairs.get(KREA_TOKEN_KEY)
if not isinstance(sections, list) or len(sections) != 1:
raise ValueError("Krea NegPiP requires exactly one Qwen token section.")
source_section = sections[0]
absolute_section = [
(pair[0], abs(_token_weight(pair)), *pair[2:]) for pair in source_section
]
absolute_tokens = dict(token_weight_pairs)
absolute_tokens[KREA_TOKEN_KEY] = [absolute_section]
encoded = original(absolute_tokens, template_end=template_end)
if len(encoded) < 3 or not isinstance(encoded[0], torch.Tensor):
raise TypeError("Krea NegPiP encoder must return tensor conditioning metadata.")
extra = encoded[2]
if not isinstance(extra, dict):
raise TypeError("Krea NegPiP encoder metadata must be a dictionary.")
cut = _template_end(source_section) if template_end == -1 else template_end
signs = [
-1.0 if _token_weight(pair) < 0.0 else 1.0 for pair in source_section[cut:]
]
sequence_length = int(encoded[0].shape[1])
if len(signs) != sequence_length:
raise ValueError(
"Krea NegPiP sign mask does not match post-template conditioning: "
f"{len(signs)} signs for {sequence_length} tokens."
)
prepared_extra = dict(extra)
prepared_extra[ENCODER_MASK_KEY] = torch.tensor(signs).reshape(1, -1, 1)
return encoded[0], encoded[1], prepared_extra
def krea2_extra_conds_negpip_wrapper(
previous_extra_conds: Callable[..., dict[str, object]],
) -> Callable[..., dict[str, object]]:
"""Publish the Krea token-sign mask as a processed model condition."""
def wrapped_extra_conds(**kwargs: object) -> dict[str, object]:
"""Attach a validated sequence multiplier without altering other conditions."""
output = previous_extra_conds(**kwargs)
if not isinstance(output, dict):
raise TypeError("Krea extra conditions must be a dictionary.")
multiplier = kwargs.get(ENCODER_MASK_KEY)
if multiplier is not None:
if not isinstance(multiplier, torch.Tensor):
raise TypeError("Krea NegPiP sign mask must be a tensor.")
if (
multiplier.ndim != 3
or multiplier.shape[0] != 1
or multiplier.shape[2] != 1
):
raise ValueError(
"Krea NegPiP sign mask must have shape (1, sequence, 1)."
)
output[CONDITION_MASK_KEY] = conds.CONDRegular(multiplier)
return output
return wrapped_extra_conds
def krea2_diffusion_negpip_wrapper(
executor: Callable[..., object],
*args: object,
**kwargs: object,
) -> object:
"""Move a processed Krea sign mask into call-local transformer options."""
positional_options = args[5] if len(args) > 5 else None
transformer_options = (
positional_options
if positional_options is not None
else kwargs.get("transformer_options", {})
)
if not isinstance(transformer_options, dict):
raise TypeError("Krea transformer options must be a dictionary.")
prepared = transformer_options.copy()
multiplier = kwargs.get(CONDITION_MASK_KEY)
if multiplier is not None:
if not isinstance(multiplier, torch.Tensor):
raise TypeError("Krea NegPiP processed mask must be a tensor.")
prepared[TRANSFORMER_MASK_KEY] = multiplier
if len(args) > 5:
prepared_args = list(args)
prepared_args[5] = prepared
return executor(*prepared_args, **kwargs)
kwargs["transformer_options"] = prepared
return executor(*args, **kwargs)
def krea2_attn1_negpip(
query: torch.Tensor,
key: torch.Tensor,
value: torch.Tensor,
pe: torch.Tensor | None = None,
attn_mask: torch.Tensor | None = None,
extra_options: dict[str, Any] | None = None,
) -> dict[str, torch.Tensor | None]:
"""Apply negative signs only to Krea text values in the joint token stream."""
options = {} if extra_options is None else extra_options
multiplier = options.get(TRANSFORMER_MASK_KEY)
if multiplier is None:
return {"q": query, "k": key, "v": value, "pe": pe, "attn_mask": attn_mask}
if not isinstance(multiplier, torch.Tensor):
raise TypeError("Krea NegPiP attention mask must be a tensor.")
image_slice = options.get("img_slice")
if (
not isinstance(image_slice, (list, tuple))
or len(image_slice) != 2
or any(
isinstance(item, bool) or not isinstance(item, int) for item in image_slice
)
):
raise ValueError("Krea NegPiP requires the model's text/image token boundary.")
text_length = image_slice[0]
if text_length != multiplier.shape[1] or value.shape[2] < text_length:
raise ValueError(
"Krea NegPiP mask does not match the joint attention sequence."
)
if multiplier.shape[0] not in {1, value.shape[0]} or multiplier.shape[2] != 1:
raise ValueError("Krea NegPiP mask has an incompatible batch or channel shape.")
text_multiplier = multiplier.to(device=value.device, dtype=value.dtype).unsqueeze(1)
prepared_value = value.to(copy=True)
prepared_value[:, :, :text_length, :] *= text_multiplier
return {
"q": query,
"k": key,
"v": prepared_value,
"pe": pe,
"attn_mask": attn_mask,
}
def _token_weight(pair: tuple[object, ...]) -> float:
"""Return one finite scalar token weight from a tokenizer tuple."""
if (
len(pair) < 2
or isinstance(pair[1], bool)
or not isinstance(pair[1], (int, float))
):
raise TypeError("Krea token weights must be numeric.")
weight = float(pair[1])
if not torch.isfinite(torch.tensor(weight)):
raise ValueError("Krea token weights must be finite.")
return weight
def _template_end(section: list[tuple[object, ...]]) -> int:
"""Resolve the exact Krea system and user-opening prefix boundary."""
count = 0
template_end = -1
for index, pair in enumerate(section):
token = pair[0]
if (
not isinstance(token, torch.Tensor)
and token == IM_START_TOKEN
and count < 2
):
template_end = index
count += 1
if (
len(section) > template_end + 3
and section[template_end + 1][0] == USER_TOKEN
and section[template_end + 2][0] == NEWLINE_TOKEN
):
template_end += 3
return template_end
+123
View File
@@ -0,0 +1,123 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Apply PPM-compatible NegPiP encoding for standard cross-attention models."""
# NegPiP behavior is adapted from ComfyUI-ppm and its credited predecessors.
# See third_party/manifest.toml and third_party/NOTICE.md.
from __future__ import annotations
from typing import Any
import torch
from comfy import model_management
from comfy.sd1_clip import SDClipModel, gen_empty_tokens
def standard_attn2_negpip(
query: torch.Tensor,
key: torch.Tensor,
value: torch.Tensor,
extra_options: dict[str, Any],
) -> tuple[torch.Tensor, torch.Tensor, torch.Tensor]:
"""Select magnitude embeddings for keys and signed embeddings for values."""
del extra_options
return query, key[:, 0::2], value[:, 1::2]
def encode_token_weights_negpip(
encoder: SDClipModel,
token_weight_pairs: list[list[tuple[object, float]]],
) -> tuple[object, ...]:
"""Encode absolute prompt magnitude and interleave signed value embeddings."""
tokens_to_encode: list[list[object]] = []
maximum_length = 0
has_weights = False
for section in token_weight_pairs:
tokens = [pair[0] for pair in section]
maximum_length = max(len(tokens), maximum_length)
has_weights = has_weights or any(pair[1] != 1.0 for pair in section)
tokens_to_encode.append(tokens)
section_count = len(tokens_to_encode)
if has_weights or section_count == 0:
if hasattr(encoder, "gen_empty_tokens"):
empty_tokens = encoder.gen_empty_tokens(
encoder.special_tokens,
maximum_length,
)
else:
empty_tokens = gen_empty_tokens(encoder.special_tokens, maximum_length)
tokens_to_encode.append(empty_tokens)
encoded = encoder.encode(tokens_to_encode)
output_tensor, pooled = encoded[:2]
if not isinstance(output_tensor, torch.Tensor):
raise TypeError("NegPiP text encoder output must be a tensor.")
first_pooled = (
pooled[0:1].to(device=model_management.intermediate_device())
if isinstance(pooled, torch.Tensor)
else pooled
)
outputs: list[torch.Tensor] = []
for section_index in range(section_count):
key_embedding = output_tensor[section_index : section_index + 1].to(copy=True)
value_embedding = key_embedding.to(copy=True)
if has_weights:
empty_embedding = output_tensor[-1]
for batch_index in range(len(key_embedding)):
for token_index in range(len(key_embedding[batch_index])):
weight = token_weight_pairs[section_index][token_index][1]
if weight == 1.0:
continue
magnitude = abs(weight)
key_embedding[batch_index][token_index] = (
key_embedding[batch_index][token_index]
- empty_embedding[token_index]
) * magnitude + empty_embedding[token_index]
value_embedding[batch_index][token_index] = (
value_embedding[batch_index][token_index]
- empty_embedding[token_index]
) * magnitude + empty_embedding[token_index]
if weight < 0.0:
value_embedding[batch_index][token_index].neg_()
interleaved = torch.zeros_like(key_embedding).repeat(1, 2, 1)
interleaved[:, 0::2, :] = key_embedding
interleaved[:, 1::2, :] = value_embedding
outputs.append(interleaved)
if outputs:
result: tuple[object, ...] = (
torch.cat(outputs, dim=-2).to(
device=model_management.intermediate_device()
),
first_pooled,
)
else:
result = (
output_tensor[-1:].to(device=model_management.intermediate_device()),
first_pooled,
)
if len(encoded) <= 2:
return result
source_extra = encoded[2]
if not isinstance(source_extra, dict):
raise TypeError("NegPiP text encoder metadata must be a dictionary.")
extra: dict[str, object] = {}
for key, value in source_extra.items():
if key == "attention_mask" and isinstance(value, torch.Tensor):
value = (
value[:section_count]
.flatten()
.unsqueeze(dim=0)
.to(device=model_management.intermediate_device())
)
extra[str(key)] = value
return (*result, extra)
+54 -2
View File
@@ -7,7 +7,8 @@
from __future__ import annotations
from collections.abc import Iterable
from typing import Protocol, TypeVar, cast
from dataclasses import dataclass
from typing import Any, Protocol, TypeVar, cast
from .clip_patcher_model_alignment import align_clip_text_encoder_with_patcher
@@ -27,6 +28,14 @@ class ClipMutation(Protocol):
PatcherValue = TypeVar("PatcherValue")
_MODEL_FALLBACK_BOUNDARY_ATTACHMENT = "simple_syrup.model_fallback_boundary"
@dataclass(frozen=True, slots=True)
class _ModelFallbackBoundary:
"""Retain the durable Comfy patcher beneath consecutive Syrup derivations."""
patcher: object
class ComfyPatcherLifecycle:
@@ -48,6 +57,7 @@ class ComfyPatcherLifecycle:
disable_dynamic=disable_dynamic,
)
self._require_direct_parent(source, derived, operation=operation)
self._stabilize_model_fallback(source, derived)
for mutation in mutations:
mutation.apply(derived)
return derived
@@ -66,13 +76,19 @@ class ComfyPatcherLifecycle:
getter = getattr(model_override_source, "get_clone_model_override", None)
if not callable(getter):
raise TypeError(f"{operation} requires a model-override source.")
model_override = getter()
derived = self._clone(
source,
operation=operation,
disable_dynamic=disable_dynamic,
model_override=getter(),
model_override=model_override,
)
self._require_direct_parent(source, derived, operation=operation)
self._stabilize_model_fallback(
source,
derived,
explicit_boundary=model_override_source,
)
for mutation in mutations:
mutation.apply(derived)
return derived
@@ -107,6 +123,7 @@ class ComfyPatcherLifecycle:
derived_patcher,
operation=operation,
)
self._stabilize_model_fallback(source_patcher, derived_patcher)
align_clip_text_encoder_with_patcher(derived)
for mutation in mutations:
mutation.apply(derived)
@@ -181,6 +198,41 @@ class ComfyPatcherLifecycle:
f"{operation} produced a derived patcher without its source as parent."
)
@staticmethod
def _stabilize_model_fallback(
source: object,
derived: object,
*,
explicit_boundary: object | None = None,
) -> None:
"""Collapse Syrup-only lineage onto the durable same-model boundary."""
derived_attachments = getattr(derived, "attachments", None)
if not isinstance(derived_attachments, dict):
return
boundary = explicit_boundary
if boundary is None:
source_attachments = getattr(source, "attachments", None)
inherited = (
source_attachments.get(_MODEL_FALLBACK_BOUNDARY_ATTACHMENT)
if isinstance(source_attachments, dict)
else None
)
if inherited is not None and not isinstance(
inherited,
_ModelFallbackBoundary,
):
raise TypeError("SimpleSyrup MODEL fallback boundary is invalid.")
boundary = inherited.patcher if inherited is not None else source
if getattr(boundary, "model", None) is not getattr(derived, "model", None):
derived_attachments.pop(_MODEL_FALLBACK_BOUNDARY_ATTACHMENT, None)
return
cast(Any, derived).parent = boundary
derived_attachments[_MODEL_FALLBACK_BOUNDARY_ATTACHMENT] = (
_ModelFallbackBoundary(boundary)
)
@staticmethod
def _required_clip_patcher(value: object, *, value_name: str) -> object:
"""Return the CLIP patcher required for lineage validation."""
+253
View File
@@ -0,0 +1,253 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Adapt the complete installed PPM NegPiP patch family to regional execution."""
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
from enum import StrEnum
from typing import cast
import torch
from comfy.patcher_extension import WrappersMP
from ..domain.regional_model_capabilities import RegionalModelFamily
_MODEL_MARKER = "ppm_negpip"
_ANIMA_WRAPPER_KEY = "ppm_negpip_anima"
_ANIMA_CONDITION_KEY = "c_ppm_negpip_mask"
_ANIMA_TRANSFORMER_KEY = "ppm_negpip_mask"
_EXTRA_CONDS_PATH = "extra_conds"
_ATTN2_PATCH_NAME = "attn2_patch"
_UNET_CALLBACKS = (
("src.negpip.unet_negpip", "sdxl_attn2_negpip"),
("simple_syrup.runtime.negpip.standard", "standard_attn2_negpip"),
)
_ANIMA_CALLBACKS = (
("src.negpip.anima_negpip", "cosmos_attn2_negpip"),
("simple_syrup.runtime.negpip.anima", "anima_attn2_negpip"),
)
_ANIMA_WRAPPERS = (
("src.negpip.anima_negpip", "cosmos_diffusion_negpip_wrapper"),
(
"simple_syrup.runtime.negpip.anima",
"anima_diffusion_negpip_wrapper",
),
)
_ANIMA_EXTRA_CONDS_CALLBACKS = (
(
"src.negpip.anima_negpip",
"anima_extra_conds_negpip_wrapper.<locals>._anima_extra_conds_negpip_wrapper",
),
(
"simple_syrup.runtime.negpip.anima",
"anima_extra_conds_negpip_wrapper.<locals>.wrapped_extra_conds",
),
)
class PpmNegpipSemantics(StrEnum):
"""Identify the family-specific NegPiP conditioning representation."""
STANDARD_UNET_SPLIT_KEY_VALUE = "standard-unet-split-key-value"
ANIMA_VALUE_MASK = "anima-value-mask"
@dataclass(frozen=True, slots=True)
class PpmNegpipInterop:
"""Retain identity-validated PPM objects needed by regional execution."""
semantics: PpmNegpipSemantics
attention_patch: Callable[..., object]
def __post_init__(self) -> None:
"""Require one typed semantic mode and callable preserved callback."""
if not isinstance(self.semantics, PpmNegpipSemantics):
raise TypeError("NegPiP semantics have an invalid type.")
if not callable(self.attention_patch):
raise TypeError("NegPiP attention patch must be callable.")
def extract_value_multiplier(
self,
model_conditions: dict[object, object],
context: torch.Tensor,
) -> torch.Tensor | None:
"""Return one validated Anima value multiplier or no UNet multiplier."""
if self.semantics is PpmNegpipSemantics.STANDARD_UNET_SPLIT_KEY_VALUE:
return None
condition = model_conditions.get(_ANIMA_CONDITION_KEY)
if condition is None:
return context.new_ones((*context.shape[:2], 1))
multiplier = getattr(condition, "cond", None)
if not isinstance(multiplier, torch.Tensor):
raise TypeError("Anima NegPiP value mask condition must contain a tensor.")
if (
multiplier.ndim != 3
or int(multiplier.shape[0]) != int(context.shape[0])
or int(multiplier.shape[1]) != int(context.shape[1])
or int(multiplier.shape[2]) != 1
):
raise ValueError(
"Anima NegPiP value mask must match the conditioning batch and "
"sequence with one multiplier channel."
)
multiplier = multiplier.to(device=context.device, dtype=context.dtype)
if not bool(((multiplier == 1) | (multiplier == -1)).all().item()):
raise ValueError("Anima NegPiP value mask must contain only -1 and 1.")
return multiplier
def prepare_anima_transformer_options(
self,
source: dict[str, object],
packed_multiplier: torch.Tensor,
) -> dict[str, object]:
"""Publish a packed mask on an isolated Anima cross-attention call."""
if self.semantics is not PpmNegpipSemantics.ANIMA_VALUE_MASK:
raise ValueError("Only Anima NegPiP semantics can publish a value mask.")
if not isinstance(packed_multiplier, torch.Tensor):
raise TypeError("Packed Anima NegPiP multiplier must be a tensor.")
prepared = source.copy()
prepared[_ANIMA_TRANSFORMER_KEY] = packed_multiplier
return prepared
class PpmNegpipInteropValidator:
"""Admit only one complete identity-validated PPM NegPiP family."""
def validate(
self,
family: RegionalModelFamily,
*,
model_options: dict[object, object],
wrappers: dict[str, dict[object, list[object]]],
object_patches: dict[object, object],
transformer_patches: dict[str, list[object]],
) -> PpmNegpipInterop | None:
"""Return preserved NegPiP state or reject every partial/conflicting form."""
if not isinstance(family, RegionalModelFamily):
raise TypeError("NegPiP interop requires a model family.")
marker = model_options.get(_MODEL_MARKER, False)
if not isinstance(marker, bool):
raise TypeError("MODEL ppm_negpip marker must be boolean.")
attention = transformer_patches.get(_ATTN2_PATCH_NAME, [])
anima_wrappers = wrappers.get(WrappersMP.DIFFUSION_MODEL, {}).get(
_ANIMA_WRAPPER_KEY,
[],
)
extra_conds = object_patches.get(_EXTRA_CONDS_PATH)
recognized_surface = any(
(
any(_matches_any_identity(item, _UNET_CALLBACKS) for item in attention),
any(
_matches_any_identity(item, _ANIMA_CALLBACKS) for item in attention
),
bool(anima_wrappers),
_matches_any_identity(extra_conds, _ANIMA_EXTRA_CONDS_CALLBACKS),
)
)
if not marker:
if recognized_surface:
raise ValueError(
"MODEL contains an incomplete NegPiP patch family without its "
"marker. Reapply CLIP NegPip to a clean MODEL."
)
return None
if family is RegionalModelFamily.STANDARD_UNET:
return self._validate_standard_unet(
attention,
anima_wrappers=anima_wrappers,
extra_conds=extra_conds,
)
if family is RegionalModelFamily.ANIMA:
return self._validate_anima(
attention,
anima_wrappers=anima_wrappers,
extra_conds=extra_conds,
)
raise ValueError(f"NegPiP does not support model family {family.value!r}.")
@staticmethod
def _validate_standard_unet(
attention: list[object],
*,
anima_wrappers: list[object],
extra_conds: object,
) -> PpmNegpipInterop:
"""Require exactly PPM's single UNet split-K/V callback surface."""
if (
len(attention) != 1
or not _matches_any_identity(attention[0], _UNET_CALLBACKS)
or anima_wrappers
or _matches_any_identity(extra_conds, _ANIMA_EXTRA_CONDS_CALLBACKS)
):
raise ValueError(
"Standard UNet NegPiP requires exactly its PPM split-K/V attention "
"patch and no Anima NegPiP surfaces."
)
return PpmNegpipInterop(
PpmNegpipSemantics.STANDARD_UNET_SPLIT_KEY_VALUE,
cast(Callable[..., object], attention[0]),
)
@staticmethod
def _validate_anima(
attention: list[object],
*,
anima_wrappers: list[object],
extra_conds: object,
) -> PpmNegpipInterop:
"""Require PPM's exact callback, wrapper, and extra-condition surfaces."""
if (
len(attention) != 1
or not _matches_any_identity(attention[0], _ANIMA_CALLBACKS)
or len(anima_wrappers) != 1
or not _matches_any_identity(anima_wrappers[0], _ANIMA_WRAPPERS)
or not _matches_any_identity(extra_conds, _ANIMA_EXTRA_CONDS_CALLBACKS)
):
raise ValueError(
"Anima NegPiP requires exactly its PPM attention patch, keyed "
"diffusion wrapper, and extra_conds object patch."
)
return PpmNegpipInterop(
PpmNegpipSemantics.ANIMA_VALUE_MASK,
cast(Callable[..., object], attention[0]),
)
def _is_identity(
value: object,
module_suffix: str,
qualified_name: str,
) -> bool:
"""Match one callable by its stable defining module suffix and qualified name."""
if not callable(value):
return False
module = getattr(value, "__module__", None)
qualname = getattr(value, "__qualname__", None)
return (
isinstance(module, str)
and (module == module_suffix or module.endswith(f".{module_suffix}"))
and qualname == qualified_name
)
def _matches_any_identity(
value: object,
identities: tuple[tuple[str, str], ...],
) -> bool:
"""Match a callable against either the installed PPM or owned equivalent."""
return any(_is_identity(value, *identity) for identity in identities)
PPM_NEGPIP_INTEROP_VALIDATOR = PpmNegpipInteropValidator()
@@ -86,6 +86,28 @@ class PromptControlGraphAdapter:
self.merge_expand(expand, negative.expand, "negative LoRA scheduling")
return negative.args[0], negative.args[1]
def apply_automatic_negpip(
self,
*,
model: Any,
clip: Any,
expand: dict[str, dict[str, Any]],
) -> tuple[Any, Any]:
"""Insert the runtime family check after a negative prompt-weight trigger."""
graph = self._graph_utils.GraphBuilder()
prepared = graph.node(
"SimpleSyrup.ApplyAutomaticNegpip",
model=model,
clip=clip,
)
self.merge_expand(
expand,
cast(dict[str, dict[str, Any]], graph.finalize()),
"automatic NegPiP preparation",
)
return prepared.out(0), prepared.out(1)
def encode_segment(
self,
*,
@@ -8,6 +8,7 @@ from __future__ import annotations
from typing import Any
from ..domain.negative_prompt_weights import contains_negative_prompt_weight
from ..domain.prompt_batch_parser import DEFAULT_PROMPT_BATCH_SEPARATOR
from ..domain.prompt_control_prompt import PreparedPromptSide, apply_encode_style
from ..services.prompt_control_segment_planning_service import (
@@ -48,6 +49,12 @@ class PromptControlScheduleEncodeGraphBuilder:
)
adapter = self.graph_adapter_class.load(PROMPT_CONTROL_MISSING_MESSAGE)
expand: dict[str, dict[str, Any]] = {}
if self._requires_negpip(plan):
model, clip = adapter.apply_automatic_negpip(
model=model,
clip=clip,
expand=expand,
)
scheduled_model, encoding_clip = self._sampling_inputs(
model=model,
clip=clip,
@@ -84,6 +91,16 @@ class PromptControlScheduleEncodeGraphBuilder:
expand=expand,
)
@staticmethod
def _requires_negpip(plan: PromptControlSegmentPlan) -> bool:
"""Return whether any cleaned positive or negative segment needs NegPiP."""
return any(
contains_negative_prompt_weight(chunk.text)
for side in (plan.positive, plan.negative)
for chunk in side.chunks
)
def _sampling_inputs(
self,
*,
@@ -7,6 +7,7 @@
from __future__ import annotations
import torch
import torch.nn.functional as functional
from comfy.utils import repeat_to_batch_size
from ..domain.processed_regional_attention import (
@@ -143,6 +144,13 @@ class RegionalAttentionBatchingService:
)
for region_index in range(plan.mask_bank.region_count)
),
base_value_multiplier=self._align_value_multiplier(
tuple(chunk.base_entry for chunk in selected),
latent_batch_size=latent_batch_size,
device=aligned_base_context.device,
dtype=aligned_base_context.dtype,
target_sequence_length=target_sequence_length,
),
)
def _align_region(
@@ -163,6 +171,7 @@ class RegionalAttentionBatchingService:
for entry_index in range(entry_count):
context_parts: list[torch.Tensor] = []
strengths: list[float] = []
multiplier_entries: list[ProcessedRegionalAttentionEntry] = []
for chunk, source in zip(chunks, sources, strict=True):
if source is None:
entry = chunk.base_entry
@@ -186,12 +195,20 @@ class RegionalAttentionBatchingService:
target_length=target_sequence_length,
)
context_parts.append(repeated)
multiplier_entries.append(entry)
strengths.extend((strength,) * latent_batch_size)
entries.append(
BatchedRegionalAttentionEntry(
entry_index,
torch.cat(context_parts, dim=0),
tuple(strengths),
self._align_value_multiplier(
tuple(multiplier_entries),
latent_batch_size=latent_batch_size,
device=device,
dtype=dtype,
target_sequence_length=target_sequence_length,
),
)
)
return BatchedRegionalAttentionRegion(region_index, tuple(entries))
@@ -201,6 +218,7 @@ class RegionalAttentionBatchingService:
entries = self._entries(plan)
authority = entries[0].cross_attention
has_value_multiplier = entries[0].cross_attention_value_multiplier is not None
for entry in entries[1:]:
tensor = entry.cross_attention
shape_mismatch = (
@@ -216,6 +234,48 @@ class RegionalAttentionBatchingService:
raise ValueError("Regional attention context devices must match.")
if tensor.dtype != authority.dtype:
raise ValueError("Regional attention context dtypes must match.")
if (
entry.cross_attention_value_multiplier is not None
) is not has_value_multiplier:
raise ValueError(
"Regional attention value multiplier presence must be uniform."
)
@staticmethod
def _align_value_multiplier(
entries: tuple[ProcessedRegionalAttentionEntry, ...],
*,
latent_batch_size: int,
device: torch.device,
dtype: torch.dtype,
target_sequence_length: int,
) -> torch.Tensor | None:
"""Repeat and sequence-align one chunk-major value-multiplier bank."""
if not entries or entries[0].cross_attention_value_multiplier is None:
return None
parts: list[torch.Tensor] = []
for entry in entries:
multiplier = entry.cross_attention_value_multiplier
if multiplier is None:
raise ValueError(
"Regional attention value multiplier presence must be uniform."
)
repeated = repeat_to_batch_size(multiplier, latent_batch_size).to(
device=device,
dtype=dtype,
)
sequence_length = int(repeated.shape[1])
if sequence_length < target_sequence_length:
repeated = functional.pad(
repeated,
(0, 0, 0, target_sequence_length - sequence_length),
value=1.0,
)
elif sequence_length > target_sequence_length:
repeated = repeated[:, :target_sequence_length]
parts.append(repeated)
return torch.cat(parts, dim=0)
@staticmethod
def _entries(
@@ -7,6 +7,7 @@
from __future__ import annotations
from ..patcher_lifecycle import ModelMutation
from ..ppm_negpip_interop import PpmNegpipInterop
from .anima_activation_context import (
ANIMA_ACTIVATION_CONTEXT,
AnimaActivationContext,
@@ -75,6 +76,7 @@ def anima_attention_coupling_mutations(
),
phase_context: AnimaCompositionPhaseContext = (ANIMA_COMPOSITION_PHASE_CONTEXT),
query_mask_context: AnimaQueryMaskContext = ANIMA_QUERY_MASK_CONTEXT,
negpip: PpmNegpipInterop | None = None,
) -> tuple[ModelMutation, ...]:
"""Return attention-only or complete regional-LoRA mutation composition."""
@@ -106,6 +108,7 @@ def anima_attention_coupling_mutations(
invocation_context=cross_attention_context,
phase_context=phase_context,
query_activity=query_activity,
negpip=negpip,
)
phase_wrapper = anima_composition_phase_wrapper_mutation(
surface,
@@ -18,6 +18,7 @@ from ...domain.regional_conditioning_output import (
RegionalConditioningOutputCombiner,
)
from ..model_patcher_mutations import ModelExactObjectPatchMutation
from ..ppm_negpip_interop import PpmNegpipInterop
from .anima_activation_context import (
ANIMA_ACTIVATION_CONTEXT,
AnimaActivationContext,
@@ -26,6 +27,7 @@ from .anima_activation_context import (
from .anima_attention_execution import AnimaRegionalAttentionExecution
from .anima_branch_batch import (
ANIMA_BASE_BRANCH_KEY,
AnimaRegionalBranchBatch,
AnimaRegionalBranchKey,
)
from .anima_composition_phase_context import AnimaCompositionPhaseContext
@@ -71,6 +73,7 @@ class AnimaRegionalCrossAttentionPatch(nn.Module):
),
weighting: RegionalAttentionWeightingPolicy | None = None,
entry_combiner: RegionalConditioningOutputCombiner | None = None,
negpip: PpmNegpipInterop | None = None,
) -> None:
"""Retain the exact original attention owner and focused collaborators."""
@@ -92,6 +95,9 @@ class AnimaRegionalCrossAttentionPatch(nn.Module):
self._query_activity = query_activity
self._weighting = weighting or ANIMA_CROSS_ATTENTION_WEIGHTING_POLICY
self._entry_combiner = entry_combiner or REGIONAL_CONDITIONING_OUTPUT_COMBINER
if negpip is not None and not isinstance(negpip, PpmNegpipInterop):
raise TypeError("Anima cross-attention NegPiP state has an invalid type.")
self._negpip = negpip
def __setattr__(self, name: str, value: Any) -> None:
"""Keep later exact child patches synchronized with installed attention."""
@@ -143,14 +149,18 @@ class AnimaRegionalCrossAttentionPatch(nn.Module):
branch_batch = activity.attention_branches
branch_x = branch_batch.pack_source(x)
branch_context = branch_batch.pack_branch_values(branch_values)
original_options = {} if transformer_options is None else transformer_options
forwarded_options = self._prepare_transformer_options(
original_options,
branch_batch,
execution_contexts,
)
with self._invocation_context.activate(branch_batch.invocation):
branch_output = self._backing.module(
branch_x,
branch_context,
rope_emb=rope_emb,
transformer_options=(
{} if transformer_options is None else transformer_options
),
transformer_options=forwarded_options,
)
if not isinstance(branch_output, torch.Tensor):
raise TypeError("Original Anima cross-attention must return a tensor.")
@@ -186,6 +196,41 @@ class AnimaRegionalCrossAttentionPatch(nn.Module):
regional_outputs=torch.stack(regional_outputs),
)
def _prepare_transformer_options(
self,
source: dict[str, Any],
branch_batch: AnimaRegionalBranchBatch,
contexts: BatchedRegionalAttentionContexts,
) -> dict[str, object]:
"""Pack NegPiP multipliers or preserve ordinary option identity."""
if self._negpip is None:
if contexts.base_value_multiplier is not None:
raise ValueError(
"Anima value multipliers require admitted NegPiP semantics."
)
return source
if not isinstance(branch_batch, AnimaRegionalBranchBatch):
raise TypeError("Anima NegPiP branch batch has an invalid type.")
base_multiplier = contexts.base_value_multiplier
if base_multiplier is None:
raise ValueError("Anima NegPiP requires an aligned base value multiplier.")
values = {ANIMA_BASE_BRANCH_KEY: base_multiplier}
for region in contexts.regions:
for entry in region.entries:
multiplier = entry.cross_attention_value_multiplier
if multiplier is None:
raise ValueError(
"Anima NegPiP requires every regional value multiplier."
)
values[
AnimaRegionalBranchKey(region.region_index, entry.entry_index)
] = multiplier
return self._negpip.prepare_anima_transformer_options(
source,
branch_batch.pack_branch_values(values),
)
def _validate_inputs(
self,
*,
@@ -246,6 +291,7 @@ def anima_cross_attention_mutations(
query_activity: AnimaRegionalQueryActivityContext = (
ANIMA_REGIONAL_QUERY_ACTIVITY_CONTEXT
),
negpip: PpmNegpipInterop | None = None,
) -> tuple[ModelExactObjectPatchMutation, ...]:
"""Build one exact clone-local cross-attention replacement per Anima block."""
@@ -262,6 +308,7 @@ def anima_cross_attention_mutations(
invocation_context=invocation_context,
phase_context=phase_context,
query_activity=query_activity,
negpip=negpip,
),
)
for block in surface.blocks
@@ -10,6 +10,7 @@ from dataclasses import dataclass
from ...domain.processed_regional_attention import ProcessedRegionalAttentionPlan
from ..patcher_lifecycle import PATCHER_LIFECYCLE
from ..ppm_negpip_interop import PpmNegpipInterop
from ..regional_attention_template import build_regional_attention_template
from ..regional_lora_plan_adapter import RegionalLoraPlanAdaptation
from .anima_attention_context_wrapper import (
@@ -19,7 +20,6 @@ from .anima_attention_coupling import anima_attention_coupling_mutations
from .anima_attention_execution import AnimaRegionalAttentionExecution
from .anima_composition import AnimaRegionalLoraComposition
from .anima_execution_scope import AnimaRegionalLoraAdapterExecution
from .anima_global_lora_overlap import ANIMA_GLOBAL_REGIONAL_LORA_OVERLAP_VALIDATOR
from .anima_model_patcher_surface import ANIMA_MODEL_PATCHER_SURFACE_RESOLVER
from .anima_plan_admission import ANIMA_REGIONAL_LORA_PLAN_ADMISSION_SERVICE
from .execution_cache import ModelCloneLineage, RegionalLoraExecutionCache
@@ -44,6 +44,7 @@ class FullContextAnimaAttentionBackend:
adaptation: RegionalLoraPlanAdaptation,
region_strengths: tuple[float, ...],
latent_batch_size: int,
negpip: PpmNegpipInterop | None = None,
) -> FullContextAnimaAttentionModel:
"""Return one collision-safe clone prepared for dynamic sampler calls."""
@@ -51,8 +52,9 @@ class FullContextAnimaAttentionBackend:
raise TypeError("Anima backend requires a processed attention plan.")
if not isinstance(adaptation, RegionalLoraPlanAdaptation):
raise TypeError("Anima backend requires a regional LoRA adaptation.")
if negpip is not None and not isinstance(negpip, PpmNegpipInterop):
raise TypeError("Anima backend NegPiP state has an invalid type.")
admitted = ANIMA_REGIONAL_LORA_PLAN_ADMISSION_SERVICE.admit(adaptation)
ANIMA_GLOBAL_REGIONAL_LORA_OVERLAP_VALIDATOR.validate(model, admitted)
template = build_regional_attention_template(
processed_plan,
latent_batch_size=latent_batch_size,
@@ -89,6 +91,7 @@ class FullContextAnimaAttentionBackend:
surface,
attention,
composition=composition,
negpip=negpip,
),
)
derived = PATCHER_LIFECYCLE.derive_model(
@@ -1,134 +0,0 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Reject exact static-global and admitted-regional Anima LoRA overlap."""
from __future__ import annotations
import math
from collections.abc import Mapping
import torch
from comfy.weight_adapter.lora import LoRAAdapter
from .anima_plan_admission import (
AnimaRegionalLoraAdapterAdmission,
AnimaRegionalLoraPlanAdmission,
)
from .standard_adapter import StandardLoraTarget
class AnimaGlobalRegionalLoraOverlapError(ValueError):
"""Report regional adapters already present in the static global MODEL."""
class AnimaGlobalRegionalLoraOverlapValidator:
"""Compare exact admitted regional A/B tensors with static global patches."""
def validate(
self,
model: object,
admission: AnimaRegionalLoraPlanAdmission,
) -> None:
"""Reject every regional adapter whose complete content is global."""
if not isinstance(admission, AnimaRegionalLoraPlanAdmission):
raise TypeError("Anima global LoRA overlap requires an admitted plan.")
patches = getattr(model, "patches", None)
if not isinstance(patches, Mapping):
raise TypeError("Anima global LoRA overlap requires MODEL patches.")
duplicates = tuple(
adapter.adapter_plan.adapter_identity.value
for adapter in admission.adapters
if self._duplicates_global_content(patches, adapter)
)
unique_duplicates = tuple(dict.fromkeys(duplicates))
if unique_duplicates:
identities = ", ".join(repr(value) for value in unique_duplicates)
raise AnimaGlobalRegionalLoraOverlapError(
"Regional Anima LoRA content is already applied globally to the "
f"input MODEL: {identities}. Remove either the global or regional "
"application before sampling."
)
def _duplicates_global_content(
self,
patches: Mapping[object, object],
adapter: AnimaRegionalLoraAdapterAdmission,
) -> bool:
"""Return whether every admitted regional target has an exact global pair."""
targets = adapter.admission.targets
return bool(targets) and all(
self._target_matches(patches, target.adapter) for target in targets
)
def _target_matches(
self,
patches: Mapping[object, object],
regional: StandardLoraTarget,
) -> bool:
"""Match one regional target against nonzero comparable static patches."""
key = f"{regional.target}.weight"
entries = patches.get(key, ())
if entries == ():
return False
if not isinstance(entries, list):
raise TypeError(f"MODEL patches[{key!r}] must be a list.")
for index, entry in enumerate(entries):
if not isinstance(entry, tuple) or len(entry) < 3:
raise TypeError(
f"MODEL patches[{key!r}][{index}] must be a Comfy patch tuple."
)
if _nonzero_strength(entry[0], key=key, index=index) and _matches_pair(
entry[1],
regional,
):
return True
return False
def _nonzero_strength(value: object, *, key: str, index: int) -> bool:
"""Validate one installed static patch strength and report its activity."""
if isinstance(value, bool) or not isinstance(value, int | float):
raise TypeError(f"MODEL patches[{key!r}][{index}] strength must be numeric.")
strength = float(value)
if not math.isfinite(strength):
raise ValueError(f"MODEL patches[{key!r}][{index}] strength must be finite.")
return strength != 0.0
def _matches_pair(value: object, regional: StandardLoraTarget) -> bool:
"""Compare one installed standard LoRA patch without copies or transfers."""
if not isinstance(value, LoRAAdapter):
return False
weights = value.weights
if not isinstance(weights, tuple) or len(weights) != 6:
return False
up, down, alpha, mid, dora_scale, reshape = weights
if any(item is not None for item in (alpha, mid, dora_scale, reshape)):
return False
if not isinstance(down, torch.Tensor) or not isinstance(up, torch.Tensor):
return False
return _same_tensor(down, regional.down) and _same_tensor(up, regional.up)
def _same_tensor(left: torch.Tensor, right: torch.Tensor) -> bool:
"""Use an identity fast path before exact same-residency tensor equality."""
if left is right:
return True
if (
left.shape != right.shape
or left.dtype != right.dtype
or left.device != right.device
):
return False
return bool(torch.equal(left, right))
ANIMA_GLOBAL_REGIONAL_LORA_OVERLAP_VALIDATOR = AnimaGlobalRegionalLoraOverlapValidator()
@@ -5,22 +5,62 @@
from __future__ import annotations
from types import ModuleType
from typing import Protocol, cast, runtime_checkable
import torch
from .fused_active_accumulation_kernel import (
MAX_FUSED_ADAPTERS_PER_LAUNCH,
REGIONAL_LORA_FUSED_ACCUMULATION_KERNEL,
from .fused_active_accumulation_contract import MAX_FUSED_ADAPTERS_PER_LAUNCH
from .triton_runtime import TRITON_RUNTIME_RESOLVER, TritonRuntimeResolver
_TRITON_BACKEND_MODULE = (
"simple_syrup.runtime.regional_lora.fused_active_accumulation_kernel"
)
@runtime_checkable
class _FusedAccumulationKernel(Protocol):
"""Describe the lazily resolved fused CUDA launch surface."""
def launch(
self,
output: torch.Tensor,
*,
rank_values: torch.Tensor,
up: torch.Tensor,
multipliers: tuple[torch.Tensor, ...],
indices: torch.Tensor | None,
adapter_start: int,
target_indices: torch.Tensor | None = None,
) -> None:
"""Launch one validated fused accumulation chunk."""
...
class _FusedAccumulationBackend(Protocol):
"""Describe the exported lazy backend module surface."""
REGIONAL_LORA_FUSED_ACCUMULATION_KERNEL: _FusedAccumulationKernel
class RegionalLoraFusedActiveAccumulator:
"""Own verified CUDA fusion for B projection and ordered output updates."""
@staticmethod
def supports(output: torch.Tensor, up: torch.Tensor) -> bool:
def __init__(
self,
resolver: TritonRuntimeResolver = TRITON_RUNTIME_RESOLVER,
) -> None:
"""Retain the process-level optional acceleration authority."""
if not isinstance(resolver, TritonRuntimeResolver):
raise TypeError("Fused accumulation requires a Triton resolver.")
self._resolver = resolver
def supports(self, output: torch.Tensor, up: torch.Tensor) -> bool:
"""Admit only verified contiguous CUDA bf16/fp16 projection shapes."""
return (
eligible = (
isinstance(output, torch.Tensor)
and isinstance(up, torch.Tensor)
and output.device.type == "cuda"
@@ -32,6 +72,7 @@ class RegionalLoraFusedActiveAccumulator:
and output.is_contiguous()
and up.is_contiguous()
)
return eligible and self._backend() is not None
def add(
self,
@@ -59,7 +100,7 @@ class RegionalLoraFusedActiveAccumulator:
return output
for start in range(0, adapter_count, MAX_FUSED_ADAPTERS_PER_LAUNCH):
stop = min(start + MAX_FUSED_ADAPTERS_PER_LAUNCH, adapter_count)
REGIONAL_LORA_FUSED_ACCUMULATION_KERNEL.launch(
self._require_backend().launch(
output,
rank_values=rank_values,
up=up,
@@ -97,7 +138,7 @@ class RegionalLoraFusedActiveAccumulator:
)
if active_row_count == 0:
return output
REGIONAL_LORA_FUSED_ACCUMULATION_KERNEL.launch(
self._require_backend().launch(
output,
rank_values=rank_values,
up=up,
@@ -137,7 +178,7 @@ class RegionalLoraFusedActiveAccumulator:
return output
for start in range(0, len(multipliers), MAX_FUSED_ADAPTERS_PER_LAUNCH):
stop = min(start + MAX_FUSED_ADAPTERS_PER_LAUNCH, len(multipliers))
REGIONAL_LORA_FUSED_ACCUMULATION_KERNEL.launch(
self._require_backend().launch(
output,
rank_values=rank_values,
up=up,
@@ -148,9 +189,28 @@ class RegionalLoraFusedActiveAccumulator:
)
return output
@classmethod
def _backend(self) -> _FusedAccumulationKernel | None:
"""Return the cached optional fused kernel without hiding failures."""
backend = self._resolver.resolve(_TRITON_BACKEND_MODULE)
if backend is None:
return None
module = cast(_FusedAccumulationBackend, cast(ModuleType, backend))
kernel = module.REGIONAL_LORA_FUSED_ACCUMULATION_KERNEL
if not isinstance(kernel, _FusedAccumulationKernel):
raise TypeError("Triton fused backend has an invalid kernel surface.")
return kernel
def _require_backend(self) -> _FusedAccumulationKernel:
"""Return the admitted kernel or reject an invalid direct fused call."""
backend = self._backend()
if backend is None:
raise RuntimeError("Triton fused accumulation is unavailable.")
return backend
def _validate_mapped_projection(
cls,
self,
output: torch.Tensor,
rank_values: torch.Tensor,
up: torch.Tensor,
@@ -159,7 +219,7 @@ class RegionalLoraFusedActiveAccumulator:
) -> None:
"""Require one unique-target batch and valid declared group mapping."""
if not cls.supports(output, up):
if not self.supports(output, up):
raise ValueError("Mapped fused accumulation received an unsupported path.")
if (
rank_values.device != output.device
@@ -191,9 +251,8 @@ class RegionalLoraFusedActiveAccumulator:
):
raise ValueError("Mapped fused output indices are misaligned.")
@classmethod
def _validate_projection(
cls,
self,
output: torch.Tensor,
rank_values: torch.Tensor,
up: torch.Tensor,
@@ -201,7 +260,7 @@ class RegionalLoraFusedActiveAccumulator:
) -> None:
"""Require one complete aligned CUDA projection contract."""
if not cls.supports(output, up):
if not self.supports(output, up):
raise ValueError("Fused active accumulation received an unsupported path.")
if (
rank_values.device != output.device
@@ -0,0 +1,7 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Define the shared launch-width contract for fused LoRA accumulation."""
MAX_FUSED_ADAPTERS_PER_LAUNCH = 8
@@ -14,7 +14,8 @@ import torch
import triton # type: ignore[import-untyped]
import triton.language as tl # type: ignore[import-untyped]
MAX_FUSED_ADAPTERS_PER_LAUNCH = 8
from .fused_active_accumulation_contract import MAX_FUSED_ADAPTERS_PER_LAUNCH
_BLOCK_ROWS = 16
_BLOCK_OUTPUT_FEATURES = 64
@@ -1,67 +1,48 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
# mypy: disable-error-code="no-untyped-def"
# ruff: noqa: ANN001, ANN202
"""Accumulate ordered adapter outputs with exact execution-dtype rounding."""
from __future__ import annotations
from typing import Any, cast
from types import ModuleType
from typing import Protocol, cast
import torch
import triton # type: ignore[import-untyped]
import triton.language as tl # type: ignore[import-untyped]
_MAX_DELTAS_PER_LAUNCH = 8
_BLOCK_SIZE = 256
from .triton_runtime import TRITON_RUNTIME_RESOLVER, TritonRuntimeResolver
_TRITON_BACKEND_MODULE = (
"simple_syrup.runtime.regional_lora.ordered_accumulation_triton"
)
@triton.jit # type: ignore[untyped-decorator]
def _ordered_accumulation_kernel(
output,
base,
delta_0,
delta_1,
delta_2,
delta_3,
delta_4,
delta_5,
delta_6,
delta_7,
element_count,
delta_count: tl.constexpr,
execution_dtype: tl.constexpr,
block_size: tl.constexpr,
):
"""Add one ordered chunk and round after every declared adapter."""
class _OrderedAccumulationBackend(Protocol):
"""Describe the lazy CUDA backend surface consumed by this owner."""
offsets = tl.program_id(0) * block_size + tl.arange(0, block_size)
active = offsets < element_count
value = tl.load(base + offsets, mask=active)
if delta_count > 0:
value = (value + tl.load(delta_0 + offsets, mask=active)).to(execution_dtype)
if delta_count > 1:
value = (value + tl.load(delta_1 + offsets, mask=active)).to(execution_dtype)
if delta_count > 2:
value = (value + tl.load(delta_2 + offsets, mask=active)).to(execution_dtype)
if delta_count > 3:
value = (value + tl.load(delta_3 + offsets, mask=active)).to(execution_dtype)
if delta_count > 4:
value = (value + tl.load(delta_4 + offsets, mask=active)).to(execution_dtype)
if delta_count > 5:
value = (value + tl.load(delta_5 + offsets, mask=active)).to(execution_dtype)
if delta_count > 6:
value = (value + tl.load(delta_6 + offsets, mask=active)).to(execution_dtype)
if delta_count > 7:
value = (value + tl.load(delta_7 + offsets, mask=active)).to(execution_dtype)
tl.store(output + offsets, value, mask=active)
def accumulate(
self,
base: torch.Tensor,
deltas: tuple[torch.Tensor, ...],
) -> torch.Tensor:
"""Accumulate validated CUDA tensors in declared order."""
...
class OrderedTensorAccumulator:
"""Own exact ordered accumulation and its CUDA launch policy."""
def __init__(
self,
resolver: TritonRuntimeResolver = TRITON_RUNTIME_RESOLVER,
) -> None:
"""Retain the process-level optional acceleration authority."""
if not isinstance(resolver, TritonRuntimeResolver):
raise TypeError("Ordered accumulation requires a Triton resolver.")
self._resolver = resolver
def accumulate(
self,
base: torch.Tensor,
@@ -74,25 +55,13 @@ class OrderedTensorAccumulator:
return base
if base.device.type != "cuda":
return self._torch_accumulate(base, deltas)
execution_dtype = self._triton_dtype(base.dtype)
remaining = deltas
result = base
while remaining:
chunk = remaining[:_MAX_DELTAS_PER_LAUNCH]
remaining = remaining[_MAX_DELTAS_PER_LAUNCH:]
padded = (*chunk, *((result,) * (_MAX_DELTAS_PER_LAUNCH - len(chunk))))
grid = (triton.cdiv(result.numel(), _BLOCK_SIZE),)
kernel = cast(Any, _ordered_accumulation_kernel)
kernel[grid](
result,
result,
*padded,
result.numel(),
delta_count=len(chunk),
execution_dtype=execution_dtype,
block_size=_BLOCK_SIZE,
)
return result
backend = self._resolver.resolve(_TRITON_BACKEND_MODULE)
if backend is None:
return self._torch_accumulate(base, deltas)
return cast(_OrderedAccumulationBackend, cast(ModuleType, backend)).accumulate(
base,
deltas,
)
@staticmethod
def _torch_accumulate(
@@ -106,18 +75,6 @@ class OrderedTensorAccumulator:
result.add_(delta)
return result
@staticmethod
def _triton_dtype(dtype: torch.dtype) -> Any:
"""Map the admitted floating execution dtype to a Triton scalar dtype."""
if dtype is torch.bfloat16:
return tl.bfloat16
if dtype is torch.float16:
return tl.float16
if dtype is torch.float32:
return tl.float32
raise TypeError(f"Ordered CUDA accumulation does not support {dtype}.")
@staticmethod
def _validate(
base: torch.Tensor,
@@ -0,0 +1,95 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
# mypy: disable-error-code="no-untyped-def"
# ruff: noqa: ANN001, ANN202
"""Provide the lazily imported Triton ordered-accumulation backend."""
from __future__ import annotations
from typing import Any, cast
import torch
import triton # type: ignore[import-untyped]
import triton.language as tl # type: ignore[import-untyped]
_MAX_DELTAS_PER_LAUNCH = 8
_BLOCK_SIZE = 256
@triton.jit # type: ignore[untyped-decorator]
def _ordered_accumulation_kernel(
output,
base,
delta_0,
delta_1,
delta_2,
delta_3,
delta_4,
delta_5,
delta_6,
delta_7,
element_count,
delta_count: tl.constexpr,
execution_dtype: tl.constexpr,
block_size: tl.constexpr,
):
"""Add one ordered chunk and round after every declared adapter."""
offsets = tl.program_id(0) * block_size + tl.arange(0, block_size)
active = offsets < element_count
value = tl.load(base + offsets, mask=active)
if delta_count > 0:
value = (value + tl.load(delta_0 + offsets, mask=active)).to(execution_dtype)
if delta_count > 1:
value = (value + tl.load(delta_1 + offsets, mask=active)).to(execution_dtype)
if delta_count > 2:
value = (value + tl.load(delta_2 + offsets, mask=active)).to(execution_dtype)
if delta_count > 3:
value = (value + tl.load(delta_3 + offsets, mask=active)).to(execution_dtype)
if delta_count > 4:
value = (value + tl.load(delta_4 + offsets, mask=active)).to(execution_dtype)
if delta_count > 5:
value = (value + tl.load(delta_5 + offsets, mask=active)).to(execution_dtype)
if delta_count > 6:
value = (value + tl.load(delta_6 + offsets, mask=active)).to(execution_dtype)
if delta_count > 7:
value = (value + tl.load(delta_7 + offsets, mask=active)).to(execution_dtype)
tl.store(output + offsets, value, mask=active)
def accumulate(base: torch.Tensor, deltas: tuple[torch.Tensor, ...]) -> torch.Tensor:
"""Accumulate CUDA deltas with the established launch and rounding policy."""
execution_dtype = _triton_dtype(base.dtype)
remaining = deltas
result = base
while remaining:
chunk = remaining[:_MAX_DELTAS_PER_LAUNCH]
remaining = remaining[_MAX_DELTAS_PER_LAUNCH:]
padded = (*chunk, *((result,) * (_MAX_DELTAS_PER_LAUNCH - len(chunk))))
grid = (triton.cdiv(result.numel(), _BLOCK_SIZE),)
kernel = cast(Any, _ordered_accumulation_kernel)
kernel[grid](
result,
result,
*padded,
result.numel(),
delta_count=len(chunk),
execution_dtype=execution_dtype,
block_size=_BLOCK_SIZE,
)
return result
def _triton_dtype(dtype: torch.dtype) -> Any:
"""Map the admitted floating execution dtype to a Triton scalar dtype."""
if dtype is torch.bfloat16:
return tl.bfloat16
if dtype is torch.float16:
return tl.float16
if dtype is torch.float32:
return tl.float32
raise TypeError(f"Ordered CUDA accumulation does not support {dtype}.")
@@ -0,0 +1,126 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Admit complete standard-UNet regional LoRA operation surfaces."""
from __future__ import annotations
from dataclasses import dataclass
import comfy.model_patcher
from torch import nn
from ..attention_coupling.family_admission import AttentionCouplingFamilyAdmission
from ..regional_lora_plan_adapter import RegionalLoraPlanAdaptation
from .comfy_adapter_resolver import COMFY_REGIONAL_ADAPTER_RESOLVER
from .execution_cache import RegionalLoraExecutionCache
from .resolved_operation_translator import COMFY_RESOLVED_OPERATION_TRANSLATOR
from .standard_unet_target_capabilities import (
STANDARD_UNET_TARGET_CAPABILITY_CLASSIFIER,
)
from .target_binder import REGIONAL_LORA_TARGET_BINDER
from .target_binding import (
BoundRegionalLoraSpatialCapability,
RegionalLoraBindingResult,
)
@dataclass(frozen=True, slots=True)
class StandardUnetOperationAdmission(AttentionCouplingFamilyAdmission):
"""Retain target bindings and exact runtime consumer-role evidence."""
binding: RegionalLoraBindingResult | None
module_roles: dict[str, BoundRegionalLoraSpatialCapability]
cache: RegionalLoraExecutionCache | None
def __post_init__(self) -> None:
"""Require either an empty admission or a complete executable surface."""
AttentionCouplingFamilyAdmission.__post_init__(self)
if not self.adaptation.plan.adapters:
if self.binding is not None or self.module_roles or self.cache is not None:
raise ValueError("Empty standard admission cannot retain operations.")
return
if (
not isinstance(self.binding, RegionalLoraBindingResult)
or not self.binding.admissible
or not self.binding.entries
):
raise ValueError("Standard admission requires complete target binding.")
if not self.module_roles or not isinstance(
self.cache,
RegionalLoraExecutionCache,
):
raise ValueError("Standard admission requires roles and execution cache.")
class StandardUnetOperationPreparation:
"""Resolve, translate, bind, and classify regional LoRA operations."""
def admit(
self,
model: object,
adaptation: RegionalLoraPlanAdaptation,
) -> StandardUnetOperationAdmission:
"""Return complete immutable evidence before installing call-scoped work."""
if not isinstance(adaptation, RegionalLoraPlanAdaptation):
raise TypeError("Standard UNet operation admission requires adaptation.")
if not adaptation.plan.adapters:
return StandardUnetOperationAdmission(adaptation, None, {}, None)
if not isinstance(model, comfy.model_patcher.ModelPatcher):
raise TypeError("Standard UNet operation admission requires a MODEL.")
graph_root = model.model
if not isinstance(graph_root, nn.Module):
raise TypeError("Standard UNet MODEL graph must be an nn.Module.")
capabilities = STANDARD_UNET_TARGET_CAPABILITY_CLASSIFIER.classify(graph_root)
resolution = COMFY_REGIONAL_ADAPTER_RESOLVER.resolve(
adaptation,
model=model,
)
operations = COMFY_RESOLVED_OPERATION_TRANSLATOR.translate(resolution)
binding = REGIONAL_LORA_TARGET_BINDER.bind(
source=model,
candidate=model,
resolution=resolution,
operations=operations,
linear_spatial_capabilities=capabilities.linear_roles,
)
if not binding.admissible:
messages = tuple(issue.message for issue in binding.issues)
raise ValueError(
f"Standard UNet regional LoRA target admission failed: {messages!r}."
)
unavailable = tuple(
entry.descriptor.target.parameter_path
for entry in binding.entries
if entry.spatial_capability
in (
BoundRegionalLoraSpatialCapability.GLOBAL_ONLY,
BoundRegionalLoraSpatialCapability.UNSUPPORTED,
)
)
if unavailable:
raise ValueError(
"Standard UNet regional LoRA targets lack executable consumer roles: "
f"{unavailable!r}."
)
module_roles: dict[str, BoundRegionalLoraSpatialCapability] = {}
for entry in binding.entries:
path = entry.descriptor.target.model_target
role = entry.spatial_capability
previous = module_roles.setdefault(path, role)
if previous is not role:
raise ValueError(
f"Standard UNet operation {path!r} has conflicting roles."
)
return StandardUnetOperationAdmission(
adaptation,
binding,
module_roles,
RegionalLoraExecutionCache(),
)
STANDARD_UNET_OPERATION_PREPARATION = StandardUnetOperationPreparation()
@@ -0,0 +1,333 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Resolve spatial regional LoRA operations within one standard-UNet call."""
from __future__ import annotations
from collections.abc import Iterator, Mapping
from contextlib import contextmanager
from contextvars import ContextVar
from dataclasses import dataclass
import torch
from ...domain.regional_activation_geometry import (
RegionalActivationGeometry,
RegionalActivationLayout,
RegionalTemporalOwnership,
)
from ...domain.regional_attention_batch import BatchedRegionalAttentionContexts
from ...domain.regional_lora_plan import RegionalLoraPlan
from ...domain.regional_mask_bank import RegionalMaskBank
from ...domain.spatial_views import SpatialBatchLayout
from ...masking.regional_activation_mask_projection import (
REGIONAL_ACTIVATION_MASK_PROJECTOR,
)
from ...masking.regional_mask_projection import (
RegionalMaskForm,
RegionalMaskProjectionMode,
)
from ..attention_coupling.unet_attn2_execution import UnetAttn2Execution
from ..spatial_model_arguments import (
SIMPLE_SYRUP_TRANSFORMER_NAMESPACE,
SPATIAL_BATCH_LAYOUT_KEY,
)
from .activation_batch_alignment import (
REGIONAL_ACTIVATION_BATCH_ALIGNMENT_RESOLVER,
)
from .convolution_execution_plan import RegionalConvolutionExecutionPlan
from .convolution_rank_geometry import REGIONAL_CONVOLUTION_RANK_GEOMETRY_RESOLVER
from .linear_execution_plan import RegionalLinearExecutionPlan
from .operation_call_scope import RegionalOperationCallScope
from .operation_invocation import (
REGIONAL_OPERATION_INVOCATION_CONTEXT,
RegionalOperationExecutionPlan,
RegionalOperationInvocation,
RegionalOperationInvocationContext,
)
from .operation_mask_resolution import REGIONAL_OPERATION_MASK_RESOLVER
from .standard_unet_lora_schedule import StandardUnetLoraSchedule
from .standard_unet_packed_operation_masks import (
STANDARD_UNET_PACKED_OPERATION_MASK_RESOLVER,
)
from .target_binding import BoundRegionalLoraSpatialCapability
@dataclass(slots=True)
class _ActiveStandardUnetOperationCall:
"""Retain call authorities and optional compact attn2 execution state."""
contexts: BatchedRegionalAttentionContexts
transformer_options: dict[str, object]
schedule_strengths: tuple[float, ...]
packed_execution: UnetAttn2Execution | None = None
class StandardUnetRegionalOperationSession:
"""Own one shared UNet trajectory with spatial regional LoRA deltas."""
def __init__(
self,
plan: RegionalLoraPlan,
mask_bank: RegionalMaskBank,
module_roles: Mapping[str, BoundRegionalLoraSpatialCapability],
call_scope: RegionalOperationCallScope,
*,
invocation_context: RegionalOperationInvocationContext = (
REGIONAL_OPERATION_INVOCATION_CONTEXT
),
) -> None:
"""Retain immutable composition, mask, role, and operation authorities."""
if not isinstance(plan, RegionalLoraPlan) or not plan.adapters:
raise ValueError("Standard UNet operation session requires adapters.")
if not isinstance(mask_bank, RegionalMaskBank):
raise TypeError("Standard UNet operation session requires a mask bank.")
if not isinstance(module_roles, Mapping) or not module_roles:
raise ValueError("Standard UNet operation session requires module roles.")
roles = dict(module_roles)
if any(not isinstance(path, str) or not path for path in roles):
raise ValueError("Standard UNet operation paths must be nonempty.")
supported = (
BoundRegionalLoraSpatialCapability.SPATIAL_TOKENS,
BoundRegionalLoraSpatialCapability.PACKED_IMAGE_TOKENS,
BoundRegionalLoraSpatialCapability.PACKED_CONTEXT_TOKENS,
BoundRegionalLoraSpatialCapability.DIRECT,
)
if any(role not in supported for role in roles.values()):
raise ValueError("Standard UNet operation role is unsupported.")
if not isinstance(call_scope, RegionalOperationCallScope):
raise TypeError("Standard UNet operation session requires a call scope.")
if not isinstance(invocation_context, RegionalOperationInvocationContext):
raise TypeError(
"Standard UNet operation session requires invocation context."
)
self._mask_bank = mask_bank
self._module_roles = roles
self._call_scope = call_scope
self._invocation_context = invocation_context
self._schedule = StandardUnetLoraSchedule(plan)
self._active: ContextVar[_ActiveStandardUnetOperationCall | None] = ContextVar(
"simple_syrup_standard_unet_regional_operation_call",
default=None,
)
@contextmanager
def activate(
self,
contexts: BatchedRegionalAttentionContexts,
transformer_options: dict[str, object],
) -> Iterator[None]:
"""Publish operation masks and install wrappers for one model call."""
if not isinstance(contexts, BatchedRegionalAttentionContexts):
raise TypeError("Standard UNet operation call requires contexts.")
if not isinstance(transformer_options, dict):
raise TypeError("Standard UNet operation call requires options.")
active = _ActiveStandardUnetOperationCall(
contexts,
transformer_options,
self._schedule.resolve(transformer_options),
)
token = self._active.set(active)
try:
with (
self._invocation_context.activate(self),
self._call_scope.activate(),
):
yield
finally:
active.packed_execution = None
self._active.reset(token)
def begin_packed(self, execution: UnetAttn2Execution) -> None:
"""Publish compact attn2 execution until its paired output callback."""
active = self._require_active()
if active.packed_execution is not None:
raise ValueError("Standard UNet operation call already has packed state.")
if not isinstance(execution, UnetAttn2Execution):
raise TypeError("Standard UNet packed state requires attn2 execution.")
active.packed_execution = execution
def end_packed(self, execution: UnetAttn2Execution) -> None:
"""Clear only the compact execution opened by the input callback."""
active = self._require_active()
if active.packed_execution is not execution:
raise ValueError("Standard UNet packed output does not match input state.")
active.packed_execution = None
def resolve(
self,
module_path: str,
plan: RegionalOperationExecutionPlan,
inputs: torch.Tensor,
) -> RegionalOperationInvocation | None:
"""Resolve one installed operation from its declared consumer role."""
active = self._require_active()
role = self._module_roles.get(module_path)
if role is None:
return None
strengths = tuple(
active.schedule_strengths[use.composition_index] for use in plan.uses
)
if role is BoundRegionalLoraSpatialCapability.PACKED_IMAGE_TOKENS:
execution = self._require_packed(active)
if not isinstance(plan, RegionalLinearExecutionPlan):
raise TypeError("Packed image role requires a Linear plan.")
masks = STANDARD_UNET_PACKED_OPERATION_MASK_RESOLVER.resolve_image_tokens(
execution,
uses=plan.uses,
inputs=inputs,
)
elif role is BoundRegionalLoraSpatialCapability.PACKED_CONTEXT_TOKENS:
execution = self._require_packed(active)
if not isinstance(plan, RegionalLinearExecutionPlan):
raise TypeError("Packed context role requires a Linear plan.")
masks = STANDARD_UNET_PACKED_OPERATION_MASK_RESOLVER.resolve_context_tokens(
execution,
uses=plan.uses,
inputs=inputs,
)
else:
if active.packed_execution is not None:
raise ValueError("Ordinary regional operation ran inside packed attn2.")
geometry = self._ordinary_geometry(
role,
plan=plan,
inputs=inputs,
active=active,
)
spatial = REGIONAL_ACTIVATION_MASK_PROJECTOR.project(
bank=self._mask_bank,
geometry=geometry,
form=RegionalMaskForm.CONDITIONING,
mode=RegionalMaskProjectionMode.CONTINUOUS_COVERAGE,
device=inputs.device,
dtype=inputs.dtype,
)
masks = REGIONAL_OPERATION_MASK_RESOLVER.resolve(
spatial,
contexts=active.contexts,
uses=plan.uses,
)
return RegionalOperationInvocation(masks, strengths)
def clear(self, model: object, unpatch_all: bool) -> None:
"""Release retained sampling schedule state on model detach."""
del model, unpatch_all
self._schedule.clear()
def _ordinary_geometry(
self,
role: BoundRegionalLoraSpatialCapability,
*,
plan: RegionalOperationExecutionPlan,
inputs: torch.Tensor,
active: _ActiveStandardUnetOperationCall,
) -> RegionalActivationGeometry:
"""Resolve exact ordinary token or convolution activation geometry."""
layout = _spatial_layout(active.transformer_options)
alignment = REGIONAL_ACTIVATION_BATCH_ALIGNMENT_RESOLVER.resolve(
active.contexts,
spatial_layout=layout,
)
if role is BoundRegionalLoraSpatialCapability.SPATIAL_TOKENS:
if not isinstance(plan, RegionalLinearExecutionPlan) or inputs.ndim != 3:
raise ValueError("Spatial-token role requires B/S/C Linear inputs.")
activation_shape = _activation_shape(active.transformer_options)
if int(inputs.shape[0]) != activation_shape[0] or int(inputs.shape[1]) != (
activation_shape[2] * activation_shape[3]
):
raise ValueError("Spatial-token inputs must match live activation H/W.")
return RegionalActivationGeometry(
RegionalActivationLayout.CONSUMER_SPATIALIZED,
tuple(inputs.shape),
2,
activation_shape[2],
activation_shape[3],
alignment,
)
if role is not BoundRegionalLoraSpatialCapability.DIRECT or not isinstance(
plan,
RegionalConvolutionExecutionPlan,
):
raise ValueError("Standard UNet ordinary operation role is inconsistent.")
use = plan.uses[0]
spatial = REGIONAL_CONVOLUTION_RANK_GEOMETRY_RESOLVER.resolve(
tuple(int(value) for value in inputs.shape[2:]),
use,
)
rank_channels = int(use.preparation.down.shape[0]) * use.parameters.groups
layouts = {
1: RegionalActivationLayout.DIRECT_CONVOLUTION_1D,
2: RegionalActivationLayout.DIRECT_CONVOLUTION_2D,
3: RegionalActivationLayout.DIRECT_CONVOLUTION_3D,
}
return RegionalActivationGeometry(
layouts[use.parameters.dimension],
(int(inputs.shape[0]), rank_channels, *spatial),
1,
1 if use.parameters.dimension == 1 else spatial[-2],
spatial[-1],
alignment,
temporal_axis=2 if use.parameters.dimension == 3 else None,
temporal_ownership=(
RegionalTemporalOwnership.REPEAT_SPATIAL_MASK
if use.parameters.dimension == 3
else RegionalTemporalOwnership.NONE
),
)
def _require_active(self) -> _ActiveStandardUnetOperationCall:
"""Return the current call or reject execution outside its owner."""
active = self._active.get()
if active is None:
raise RuntimeError("Standard UNet regional operation ran outside a call.")
return active
@staticmethod
def _require_packed(
active: _ActiveStandardUnetOperationCall,
) -> UnetAttn2Execution:
"""Return the compact attn2 authority for the current projection."""
if active.packed_execution is None:
raise RuntimeError("Packed regional operation ran outside attn2 scope.")
return active.packed_execution
def _activation_shape(options: dict[str, object]) -> tuple[int, int, int, int]:
"""Narrow Comfy's live spatial-transformer BCHW metadata."""
value = options.get("activations_shape")
if not isinstance(value, list | tuple) or len(value) != 4:
raise TypeError("Standard UNet activations_shape must be a BCHW sequence.")
shape = tuple(value)
if any(
isinstance(item, bool) or not isinstance(item, int) or item < 1
for item in shape
):
raise ValueError("Standard UNet activation dimensions must be positive.")
return shape[0], shape[1], shape[2], shape[3]
def _spatial_layout(options: dict[str, object]) -> SpatialBatchLayout | None:
"""Return the optional authoritative full, tiled, or Contextual layout."""
namespace = options.get(SIMPLE_SYRUP_TRANSFORMER_NAMESPACE)
if namespace is None:
return None
if not isinstance(namespace, dict):
raise TypeError("Standard UNet SimpleSyrup namespace must be a dictionary.")
layout = namespace.get(SPATIAL_BATCH_LAYOUT_KEY)
if layout is not None and not isinstance(layout, SpatialBatchLayout):
raise TypeError("Standard UNet spatial layout has an invalid type.")
return layout
@@ -0,0 +1,147 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Resolve regional operation masks for compact standard-UNet attn2 rows."""
from __future__ import annotations
from collections.abc import Sequence
import torch
from ...domain.regional_activation_geometry import (
RegionalActivationBatchAlignment,
RegionalActivationGeometry,
RegionalActivationLayout,
)
from ..attention_coupling.unet_attn2_execution import UnetAttn2Execution
from .operation_mask_resolution import (
REGIONAL_OPERATION_BRANCH_GATE_RESOLVER,
RegionalOperationMaskBatch,
RegionalOperationMaskUse,
)
class StandardUnetPackedOperationMaskResolver:
"""Map regional uses onto exact compact attn2 image or context rows."""
def resolve_image_tokens(
self,
execution: UnetAttn2Execution,
*,
uses: Sequence[RegionalOperationMaskUse],
inputs: torch.Tensor,
) -> RegionalOperationMaskBatch:
"""Return query-grid masks for packed query and output projections."""
self._validate(execution, uses=uses, inputs=inputs)
if int(inputs.shape[1]) != execution.query_height * execution.query_width:
raise ValueError("Packed image tokens must match attn2 query H/W.")
use_masks = tuple(
self._packed_use_mask(execution, use=use, inputs=inputs, spatial=True)
for use in uses
)
geometry = RegionalActivationGeometry(
RegionalActivationLayout.CONSUMER_SPATIALIZED,
tuple(inputs.shape),
2,
execution.query_height,
execution.query_width,
RegionalActivationBatchAlignment(int(inputs.shape[0]), 1),
)
return RegionalOperationMaskBatch(
torch.stack(use_masks),
geometry,
tuple(use.composition_index for use in uses),
)
def resolve_context_tokens(
self,
execution: UnetAttn2Execution,
*,
uses: Sequence[RegionalOperationMaskUse],
inputs: torch.Tensor,
) -> RegionalOperationMaskBatch:
"""Return branch gates broadcast over untouched context tokens."""
self._validate(execution, uses=uses, inputs=inputs)
use_masks = tuple(
self._packed_use_mask(execution, use=use, inputs=inputs, spatial=False)
for use in uses
)
geometry = RegionalActivationGeometry(
RegionalActivationLayout.BRANCH_TOKENS,
tuple(inputs.shape),
2,
1,
int(inputs.shape[1]),
RegionalActivationBatchAlignment(int(inputs.shape[0]), 1),
)
return RegionalOperationMaskBatch(
torch.stack(use_masks),
geometry,
tuple(use.composition_index for use in uses),
)
@staticmethod
def _validate(
execution: object,
*,
uses: Sequence[RegionalOperationMaskUse],
inputs: object,
) -> None:
"""Require one exact packed B/S/C activation and ordered use sequence."""
if not isinstance(execution, UnetAttn2Execution):
raise TypeError("Packed operation masks require an attn2 execution.")
if not isinstance(inputs, torch.Tensor) or inputs.ndim != 3:
raise ValueError("Packed operation inputs must use B/S/C layout.")
if int(inputs.shape[0]) != execution.branches.packed_batch_size:
raise ValueError("Packed operation batch must match attn2 branches.")
if not isinstance(uses, Sequence) or not uses:
raise ValueError("Packed operation masks require target uses.")
composition = tuple(use.composition_index for use in uses)
if composition != tuple(sorted(composition)):
raise ValueError("Packed operation uses must follow composition order.")
@staticmethod
def _packed_use_mask(
execution: UnetAttn2Execution,
*,
use: RegionalOperationMaskUse,
inputs: torch.Tensor,
spatial: bool,
) -> torch.Tensor:
"""Return one use mask in exact compact branch-segment order."""
if use.region_index >= int(execution.query_masks.shape[0]):
raise ValueError("Packed operation use references an unavailable region.")
source_gate = REGIONAL_OPERATION_BRANCH_GATE_RESOLVER.resolve(
execution.contexts,
branch=use.branch,
authority=inputs,
)
segments: list[torch.Tensor] = []
for segment in execution.branches.segments:
count = int(segment.source_indices.shape[0])
if segment.key.region_index != use.region_index:
segments.append(inputs.new_zeros((count, int(inputs.shape[1]), 1)))
continue
gate = source_gate.index_select(0, segment.source_indices).reshape(
count,
1,
1,
)
if spatial:
mask = execution.query_masks[use.region_index].index_select(
0,
segment.source_indices,
)
segments.append(mask.unsqueeze(-1) * gate)
else:
segments.append(gate.expand(-1, int(inputs.shape[1]), -1))
return torch.cat(tuple(segments))
STANDARD_UNET_PACKED_OPERATION_MASK_RESOLVER = StandardUnetPackedOperationMaskResolver()
@@ -10,6 +10,7 @@ from ..attention_coupling.unet_attn2_execution_resolver import (
UnetAttn2ExecutionResolver,
)
from ..attention_coupling.unet_attn2_patch import UnetAttn2PatchPair
from ..ppm_negpip_interop import PpmNegpipInterop
_INPUT_PATCH_KEY = "attn2_patch"
_OUTPUT_PATCH_KEY = "attn2_output_patch"
@@ -18,12 +19,20 @@ _OUTPUT_PATCH_KEY = "attn2_output_patch"
class StandardUnetVariantBaseAttention:
"""Install Attention Couple only on graph-local unpatched base execution."""
def __init__(self, resolver: UnetAttn2ExecutionResolver) -> None:
def __init__(
self,
resolver: UnetAttn2ExecutionResolver,
*,
negpip: PpmNegpipInterop | None = None,
) -> None:
"""Retain one paired callback authority for the active request state."""
if not isinstance(resolver, UnetAttn2ExecutionResolver):
raise TypeError("Standard UNet base attention requires a resolver.")
self._patches = UnetAttn2PatchPair(resolver)
if negpip is not None and not isinstance(negpip, PpmNegpipInterop):
raise TypeError("Standard UNet base attention NegPiP state is invalid.")
self._negpip = negpip
def prepare(self, source: dict[str, object]) -> dict[str, object]:
"""Return isolated transformer options with one collision-free pair."""
@@ -40,10 +49,22 @@ class StandardUnetVariantBaseAttention:
patches = source_patches.copy()
else:
raise TypeError("Standard UNet transformer patches must be a dictionary.")
for key in (_INPUT_PATCH_KEY, _OUTPUT_PATCH_KEY):
if key in patches:
raise ValueError(f"Standard UNet base graph already contains {key!r}.")
patches[_INPUT_PATCH_KEY] = [self._patches.input_patch]
expected_input = [] if self._negpip is None else [self._negpip.attention_patch]
if (
self._negpip is None
and _INPUT_PATCH_KEY in patches
or self._negpip is not None
and patches.get(_INPUT_PATCH_KEY) != expected_input
):
raise ValueError(
"Standard UNet base graph already contains an unadmitted attn2 "
"input patch."
)
if _OUTPUT_PATCH_KEY in patches:
raise ValueError(
"Standard UNet base graph already contains 'attn2_output_patch'."
)
patches[_INPUT_PATCH_KEY] = [self._patches.input_patch, *expected_input]
patches[_OUTPUT_PATCH_KEY] = [self._patches.output_patch]
prepared["patches"] = patches
return prepared
@@ -22,6 +22,7 @@ from ..model_patcher_mutations import (
ModelKeyedCallbackMutation,
ModelKeyedWrapperMutation,
)
from ..ppm_negpip_interop import PpmNegpipInterop
from .standard_unet_cold_sampling import (
StandardUnetColdSamplingDiagnosticsMutation,
)
@@ -48,6 +49,7 @@ class StandardUnetVariantRuntimeMutation:
admission: StandardUnetNativeLoraAdmission
attention_phase: StandardUnetAttentionPhaseSession
template: StandardUnetVariantTemplate
negpip: PpmNegpipInterop | None = None
def apply(self, model: object) -> None:
"""Build persistent variants before installing the private root clone."""
@@ -69,7 +71,8 @@ class StandardUnetVariantRuntimeMutation:
),
attention_phase=self.attention_phase,
base_attention=StandardUnetVariantBaseAttention(
StandardUnetAttn2ExecutionResolver(self.state)
StandardUnetAttn2ExecutionResolver(self.state),
negpip=self.negpip,
),
)
execution.prime()
@@ -148,8 +148,10 @@ class StandardUnetVariantTemplate:
)
if not isinstance(request, ModelPatcher) or request.is_dynamic():
raise TypeError("Comfy did not bind a static standard-UNet request.")
if request.parent is not source:
raise RuntimeError("Static standard-UNet request lost source lineage.")
if request.parent is not self.model:
raise RuntimeError(
"Static standard-UNet request lost its template fallback boundary."
)
if "diffusion_model" in request.object_patches_backup:
ModelSharedObjectPatchMutation(
"diffusion_model",
@@ -0,0 +1,70 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Resolve optional Triton backends without importing them on Torch paths."""
from __future__ import annotations
import logging
from collections.abc import Callable
from importlib import import_module
from threading import Lock
LOGGER = logging.getLogger(__name__)
class TritonRuntimeResolver:
"""Own thread-safe lazy backend imports and optional-package fallback."""
def __init__(
self,
*,
import_module: Callable[[str], object] = import_module,
) -> None:
"""Retain an injectable importer and empty process-lifetime cache."""
if not callable(import_module):
raise TypeError("Triton runtime importer must be callable.")
self._import_module = import_module
self._lock = Lock()
self._backends: dict[str, object | None] = {}
self._missing_warning_emitted = False
def resolve(self, backend_module: str) -> object | None:
"""Return one cached backend or None only when Triton is absent."""
if not isinstance(backend_module, str) or not backend_module:
raise ValueError("Triton backend module must be a non-empty string.")
with self._lock:
if backend_module in self._backends:
return self._backends[backend_module]
try:
backend = self._import_module(backend_module)
except ModuleNotFoundError as error:
if error.name != "triton" and not (
isinstance(error.name, str) and error.name.startswith("triton.")
):
raise RuntimeError(
f"Triton backend {backend_module!r} failed to initialize."
) from error
backend = None
if not self._missing_warning_emitted:
LOGGER.warning(
"Triton acceleration is unavailable; using the Torch "
"execution path",
extra={
"backend_module": backend_module,
"missing_dependency": error.name,
},
)
self._missing_warning_emitted = True
except Exception as error:
raise RuntimeError(
f"Triton backend {backend_module!r} failed to initialize."
) from error
self._backends[backend_module] = backend
return backend
TRITON_RUNTIME_RESOLVER = TritonRuntimeResolver()
@@ -68,8 +68,7 @@ class RegionalLoraConditioningSourceCollector:
if not isinstance(plan, RawRegionalAttentionPlan):
raise TypeError("Regional LoRA source collection requires a plan.")
self._require_unhooked_base("positive", plan.positive)
self._require_unhooked_base("negative", plan.negative)
self._require_compatible_base_hooks(plan)
return (
*self._branch_sources(plan.positive, branch=RegionalLoraBranch.POSITIVE),
*self._branch_sources(plan.negative, branch=RegionalLoraBranch.NEGATIVE),
@@ -112,31 +111,49 @@ class RegionalLoraConditioningSourceCollector:
)
return tuple(sources)
def _require_unhooked_base(
def _require_compatible_base_hooks(
self,
plan: RawRegionalAttentionPlan,
) -> None:
"""Require one shared global model-hook schedule across CFG branches."""
positive = self._base_hook_signature("positive", plan.positive)
negative = self._base_hook_signature("negative", plan.negative)
if positive != negative:
raise ValueError(
"Attention Coupling global model hooks must match across positive "
"and negative conditioning. Encode both branches through the same "
"Prompt Control global segment."
)
def _base_hook_signature(
self,
branch_name: str,
branch: RawRegionalAttentionBranch,
) -> None:
"""Require only model-active global LoRAs to arrive on the input MODEL."""
) -> tuple[tuple[object, ...], ...]:
"""Return one uniform global model-hook signature for a CFG branch."""
groups = conditioning_hook_groups(branch.base_conditioning)
model_hook_count = sum(
len(
signatures = tuple(
self._group_signature(
self._model_hook_selection(
group,
source_label=(
f"Attention Coupling {branch_name} global conditioning"
),
).model_hooks
)
)
for group in groups
for group in conditioning_hook_groups(branch.base_conditioning)
)
if model_hook_count:
if not signatures:
return ()
authority = signatures[0]
if any(signature != authority for signature in signatures[1:]):
raise ValueError(
f"Attention Coupling {branch_name} global conditioning contains "
"model hooks. Apply global LoRAs to the input MODEL; reserve "
"conditioning hooks for masked regional entries."
f"Attention Coupling {branch_name} global conditioning uses "
"different model HookGroups across text schedule entries. Keep "
"model LoRA scheduling on one shared WeightHook schedule."
)
return authority
def _uniform_hooks(
self,
@@ -20,11 +20,13 @@ from ..domain.regional_model_capabilities import (
RegionalModelFamily,
RegionalPatchConflict,
)
from .ppm_negpip_interop import (
PPM_NEGPIP_INTEROP_VALIDATOR,
PpmNegpipInterop,
)
LOGGER = logging.getLogger(__name__)
_NEGPIP_MODEL_OPTION = "ppm_negpip"
_NEGPIP_ANIMA_WRAPPER_KEY = "ppm_negpip_anima"
_EASYCACHE_OPTION = "easycache"
_ATTN2_PATCH_CONFLICTS = {
RegionalPatchConflict.ATTN2_INPUT_PATCH: "attn2_patch",
@@ -49,6 +51,7 @@ class RegionalModelPatchInteropReport:
model_family: RegionalModelFamily
preserved_modifiers: tuple[RegionalPreservedModelModifier, ...]
negpip: PpmNegpipInterop | None = None
def __post_init__(self) -> None:
"""Require a typed family and canonical unique modifier order."""
@@ -62,6 +65,8 @@ class RegionalModelPatchInteropReport:
raise TypeError("Regional interop report modifiers have invalid types.")
if len(set(self.preserved_modifiers)) != len(self.preserved_modifiers):
raise ValueError("Regional interop report modifiers must be unique.")
if self.negpip is not None and not isinstance(self.negpip, PpmNegpipInterop):
raise TypeError("Regional interop report NegPiP state has an invalid type.")
@property
def cache_modifier(self) -> RegionalPreservedModelModifier | None:
@@ -99,12 +104,22 @@ class RegionalModelPatchInteropValidator:
model_weight_patches = _require_dictionary_attribute(model, "patches")
patches = _require_optional_patch_state(transformer_options)
self._reject_negpip(model_options, wrappers)
negpip = PPM_NEGPIP_INTEROP_VALIDATOR.validate(
capabilities.model_family,
model_options=model_options,
wrappers=wrappers,
object_patches=object_patches,
transformer_patches=patches,
)
cache_modifier = self._validate_cache_state(
transformer_options,
wrappers,
)
self._reject_attention_collisions(patches, capabilities)
self._reject_attention_collisions(
patches,
capabilities,
admitted_negpip=negpip,
)
modifiers: list[RegionalPreservedModelModifier] = []
model_wrapper = model_options.get("model_function_wrapper")
@@ -135,6 +150,7 @@ class RegionalModelPatchInteropValidator:
report = RegionalModelPatchInteropReport(
capabilities.model_family,
tuple(modifiers),
negpip,
)
LOGGER.info(
"Regional MODEL patch interoperability admitted",
@@ -188,30 +204,6 @@ class RegionalModelPatchInteropValidator:
},
)
@staticmethod
def _reject_negpip(
model_options: dict[object, object],
wrappers: dict[str, dict[object, list[object]]],
) -> None:
"""Reject installed NegPiP before its mask can enter branch packing."""
marker = model_options.get(_NEGPIP_MODEL_OPTION, False)
if not isinstance(marker, bool):
raise TypeError("MODEL ppm_negpip marker must be boolean.")
negpip_wrapper = bool(
wrappers.get(WrappersMP.DIFFUSION_MODEL, {}).get(
_NEGPIP_ANIMA_WRAPPER_KEY,
(),
)
)
if marker or negpip_wrapper:
raise ValueError(
"Attention Coupling does not support NegPiP because its attention "
"mask is aligned to the ordinary conditioning batch rather than "
"SimpleSyrup's regional branch batch. Remove CLIP NegPip before "
"the Attention Coupling sampler."
)
@staticmethod
def _validate_cache_state(
transformer_options: dict[object, object],
@@ -260,6 +252,8 @@ class RegionalModelPatchInteropValidator:
def _reject_attention_collisions(
patches: dict[str, list[object]],
capabilities: RegionalModelCapabilities,
*,
admitted_negpip: PpmNegpipInterop | None,
) -> None:
"""Reject every populated attention surface owned by the backend."""
@@ -268,6 +262,11 @@ class RegionalModelPatchInteropValidator:
for conflict, patch_name in _ATTN2_PATCH_CONFLICTS.items()
if conflict in capabilities.known_patch_conflicts
and patches.get(patch_name)
and not (
patch_name == "attn2_patch"
and admitted_negpip is not None
and patches[patch_name] == [admitted_negpip.attention_patch]
)
)
if conflicts:
raise ValueError(
@@ -34,8 +34,9 @@ def make_tiled_model_args(
input_batch_size: int,
latent_height: int,
latent_width: int,
project_canvas_reference_latents: bool = False,
) -> dict[str, Any]:
"""Create apply-model args for one spatial tile batch."""
"""Create tile arguments with optional canvas-reference projection."""
layout = tiled_batch_layout(
tiles=tiles,
@@ -63,6 +64,7 @@ def make_tiled_model_args(
conditioning=conditioning,
layout=layout,
view_timestep=tiled_timestep,
project_canvas_reference_latents=project_canvas_reference_latents,
)
tiled_args = args.copy()
tiled_args["input"] = tiled_x
@@ -114,8 +116,9 @@ def make_spatial_view_model_args(
*,
args: dict[str, Any],
layout: SpatialBatchLayout,
project_canvas_reference_latents: bool = False,
) -> dict[str, Any]:
"""Create apply-model arguments for equally shaped spatial views."""
"""Create equal-view arguments with optional canvas-reference projection."""
target_shape = (layout.views[0].model_height, layout.views[0].model_width)
if any(
@@ -149,6 +152,7 @@ def make_spatial_view_model_args(
conditioning=conditioning,
layout=layout,
view_timestep=view_timestep,
project_canvas_reference_latents=project_canvas_reference_latents,
)
view_args = args.copy()
view_args["input"] = view_x
@@ -172,14 +176,18 @@ def spatial_view_conditioning(
conditioning: dict[str, Any],
layout: SpatialBatchLayout,
view_timestep: torch.Tensor,
project_canvas_reference_latents: bool = False,
) -> dict[str, Any]:
"""Resize spatial conditioning alongside arbitrary latent views."""
"""Project spatial conditioning and optionally canvas-aligned references."""
transformed: dict[str, Any] = {}
for key, value in conditioning.items():
if key == "transformer_options":
continue
if key in SPATIAL_INVARIANT_CONDITIONING_KEYS:
if (
key in SPATIAL_INVARIANT_CONDITIONING_KEYS
and not project_canvas_reference_latents
):
transformed[key] = repeat_spatial_invariant_value(
value,
view_count=layout.view_count,
@@ -158,11 +158,18 @@ class TileBlendWeightCache:
class TilePredictionAccumulator:
"""Evaluate tiled model views and combine them with one selected policy."""
def __init__(self, plan: TiledDiffusionPlan, *, diffusion_mode: str) -> None:
"""Bind an immutable plan to its overlap weighting policy."""
def __init__(
self,
plan: TiledDiffusionPlan,
*,
diffusion_mode: str,
project_canvas_reference_latents: bool = False,
) -> None:
"""Bind a plan to its weighting and reference-projection policies."""
self._plan = plan
self._blend_weights = TileBlendWeightCache(plan, diffusion_mode)
self._project_canvas_reference_latents = project_canvas_reference_latents
def predict(
self,
@@ -183,6 +190,9 @@ class TilePredictionAccumulator:
input_batch_size=input_batch_size,
latent_height=self._plan.latent_height,
latent_width=self._plan.latent_width,
project_canvas_reference_latents=(
self._project_canvas_reference_latents
),
)
tile_output = evaluate(tiled_args)
for index, tile in enumerate(batch):
+132 -14
View File
@@ -14,7 +14,14 @@ from types import ModuleType
from typing import Any, TypeAlias, cast
from ..shared.logging import get_logger
from .model_folders import SUPPORTED_MODEL_EXTENSIONS
from .model_catalog import ULTRALYTICS_ENTRIES, ModelEntry
from .model_choices import ModelChoiceService
from .model_downloads import DownloadRequest, ModelDownloader, ProgressReporter
from .model_folders import (
SUPPORTED_MODEL_EXTENSIONS,
expected_model_file,
resolve_model_file,
)
from .model_instance_cache import ModelInstanceCache
LOGGER = get_logger(__name__)
@@ -52,7 +59,6 @@ class LoadedUltralyticsDetector:
class UltralyticsModelCacheKey:
"""Identify a loaded Ultralytics detector for process-level reuse."""
model_name: str
model_path: Path
@@ -68,6 +74,8 @@ class UltralyticsLoaderService:
self,
folder_paths_module: ModuleType | None = None,
ultralytics_module: ModuleType | None = None,
downloader: ModelDownloader | None = None,
choice_service: ModelChoiceService | None = None,
cache: (
MutableMapping[UltralyticsModelCacheKey, LoadedUltralyticsDetector] | None
) = None,
@@ -76,6 +84,8 @@ class UltralyticsLoaderService:
self._folder_paths_module = folder_paths_module
self._ultralytics_module = ultralytics_module
self._downloader = downloader or ModelDownloader()
self._choice_service = choice_service or ModelChoiceService()
self._cache: ModelInstanceCache[
UltralyticsModelCacheKey, LoadedUltralyticsDetector
] = ModelInstanceCache(
@@ -83,9 +93,39 @@ class UltralyticsLoaderService:
)
def model_choices(self) -> list[str]:
"""Return local Ultralytics model choices for ComfyUI dropdowns."""
"""Return installed choices first, followed by downloadable catalog choices."""
choices = self.available_models()
self._register_model_folders()
curated_choices = self._choice_service.ultralytics_choices()
catalog_choice_labels = {
_catalog_selection(entry): entry.display_name
for entry in ULTRALYTICS_ENTRIES
}
available_choices = self.available_models()
visible_catalog_choices = set(curated_choices)
installed_catalog_choices = [
entry.display_name
for entry in ULTRALYTICS_ENTRIES
if (
entry.display_name in visible_catalog_choices
and _catalog_selection(entry) in available_choices
)
]
installed_non_catalog_choices = [
choice
for choice in available_choices
if choice not in catalog_choice_labels
]
downloadable_choices = [
choice
for choice in curated_choices
if choice not in installed_catalog_choices
]
choices = (
installed_non_catalog_choices
+ installed_catalog_choices
+ downloadable_choices
)
return choices or [NO_LOCAL_ULTRALYTICS_MODELS]
def available_models(self) -> list[str]:
@@ -133,16 +173,23 @@ class UltralyticsLoaderService:
return sorted(choices)
def load(self, model_name: str) -> LoadedUltralyticsDetector:
def load(
self,
model_name: str,
progress: ProgressReporter | None = None,
) -> LoadedUltralyticsDetector:
"""Load one Ultralytics model and create compatibility facades."""
self.reject_sentinel(model_name)
model_path = self.resolve_model_path(model_name)
normalized_name = _normalized_model_name(model_name)
key = UltralyticsModelCacheKey(
model_name=normalized_name,
model_path=model_path.resolve(),
)
entry = _catalog_entry_or_none(model_name)
if entry is None:
model_path = self.resolve_model_path(model_name)
normalized_name = _normalized_model_name(model_name)
else:
model_path = self._resolve_catalog_entry(entry, progress)
normalized_name = _catalog_selection(entry)
key = UltralyticsModelCacheKey(model_path=model_path.resolve())
already_loaded = key in self._cache.entries
loaded = self._cache.get_or_load(
key,
@@ -160,6 +207,40 @@ class UltralyticsLoaderService:
)
return loaded
def _resolve_catalog_entry(
self,
entry: ModelEntry,
progress: ProgressReporter | None,
) -> Path:
"""Resolve or securely download one curated Ultralytics checkpoint."""
if len(entry.artifacts) != 1:
raise RuntimeError(
f"Ultralytics catalog entry '{entry.entry_id}' must have one artifact."
)
self._register_model_folders()
artifact = entry.artifacts[0]
existing = resolve_model_file(
artifact.folder_name,
artifact.filename,
self._folder_paths_module,
)
destination = existing or expected_model_file(
artifact.folder_name, artifact.filename, self._folder_paths_module
)
result = self._downloader.download(
DownloadRequest(
source_url=artifact.source_url,
destination_path=destination,
expected_folder=destination.parent,
description=artifact.description,
expected_sha256=artifact.sha256,
),
progress,
)
return result.path
def _load_uncached_detector(
self,
model_name: str,
@@ -227,9 +308,10 @@ class UltralyticsLoaderService:
if model_name == NO_LOCAL_ULTRALYTICS_MODELS:
raise ValueError(
"No local Ultralytics models are available. Install a model in "
"models\\ultralytics, models\\ultralytics\\bbox, or "
"models\\ultralytics\\segm."
"No local Ultralytics models are available. Enable 'Show "
"downloadable models in loader dropdowns' in SimpleSyrup settings "
"or install a model in models\\ultralytics, "
"models\\ultralytics\\bbox, or models\\ultralytics\\segm."
)
def resolve_model_path(self, model_name: str) -> Path:
@@ -387,6 +469,42 @@ def _normalized_model_name(model_name: str) -> str:
return model_name.replace("\\", "/")
def _catalog_entry_or_none(selection: str) -> ModelEntry | None:
"""Return a curated Ultralytics entry when a dropdown label matches it."""
return next(
(
entry
for entry in ULTRALYTICS_ENTRIES
if selection in (entry.entry_id, entry.display_name)
),
None,
)
def _catalog_selection(entry: ModelEntry) -> str:
"""Return the local conventional selection path for one catalog entry."""
if len(entry.artifacts) != 1:
raise ValueError(
f"Ultralytics catalog entry '{entry.entry_id}' must have one artifact."
)
artifact = entry.artifacts[0]
prefix_by_folder = {
ULTRALYTICS_BBOX_FOLDER: "bbox",
ULTRALYTICS_SEGM_FOLDER: "segm",
}
try:
prefix = prefix_by_folder[artifact.folder_name]
except KeyError as error:
raise ValueError(
f"Ultralytics catalog entry '{entry.entry_id}' has unsupported folder "
f"'{artifact.folder_name}'."
) from error
return f"{prefix}/{artifact.filename}"
def _model_task(model_name: str, raw_model: object) -> str:
"""Infer detector task from choice prefix or model metadata."""
@@ -107,5 +107,6 @@ class AnimaAttentionCouplingModelFamily:
adaptation=admission.adaptation,
region_strengths=region_strengths,
latent_batch_size=latent_batch_size,
negpip=interop_report.negpip,
)
return built.model
@@ -26,6 +26,7 @@ from ..runtime.comfy_conditioning_processing import (
ComfyRegionalConditioningProcessor,
)
from ..runtime.comfy_latent_normalization import ComfyLatentNormalizer
from ..runtime.global_hook_model_resolver import GlobalHookModelResolver
from ..runtime.regional_lora_conditioning_adapter import (
RegionalLoraConditioningAdapter,
)
@@ -82,6 +83,9 @@ class AttentionCouplingModelPreparationService:
latent_normalizer_class: ClassVar[type[ComfyLatentNormalizer]] = (
ComfyLatentNormalizer
)
global_hook_model_resolver_class: ClassVar[type[GlobalHookModelResolver]] = (
GlobalHookModelResolver
)
model_family_selector_class: ClassVar[
type[AttentionCouplingModelFamilySelector]
] = AttentionCouplingModelFamilySelector
@@ -116,6 +120,11 @@ class AttentionCouplingModelPreparationService:
)
interop_validator = self.interop_validator_class()
interop_report = interop_validator.validate(model, capabilities)
model = self.global_hook_model_resolver_class().resolve(
model,
positive=positive,
negative=negative,
)
model_family = self.model_family_selector_class().select(capabilities)
samples = self.latent_normalizer_class().normalize(
model=model,
@@ -218,6 +227,7 @@ class AttentionCouplingModelPreparationService:
noise=samples.to(device),
device=device,
context_validator=model_family.context_validator,
negpip=interop_report.negpip,
)
interop_validator.validate_execution(
interop_report,
@@ -0,0 +1,213 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Clone and patch supported MODEL/CLIP pairs for automatic NegPiP."""
# NegPiP behavior is adapted from ComfyUI-ppm and its credited predecessors.
# See third_party/manifest.toml and third_party/NOTICE.md.
from __future__ import annotations
import logging
from functools import partial
from comfy.model_base import SDXL, Anima, BaseModel, Krea2, SDXLRefiner
from comfy.model_patcher import ModelPatcher
from comfy.sd import CLIP
from ..runtime.clip_patcher_mutations import (
ClipBooleanOptionMutation,
ClipCallableObjectPatchMutation,
ClipTokenizerMutation,
)
from ..runtime.model_patcher_mutations import (
ModelAttentionPatchMutation,
ModelBooleanOptionMutation,
ModelCallableObjectPatchMutation,
ModelDiffusionWrapperMutation,
ModelInteropDiffusionWrapperMutation,
)
from ..runtime.negpip.anima import (
WRAPPER_KEY as ANIMA_WRAPPER_KEY,
)
from ..runtime.negpip.anima import (
anima_attn2_negpip,
anima_diffusion_negpip_wrapper,
anima_extra_conds_negpip_wrapper,
)
from ..runtime.negpip.krea2 import (
CLIP_MARKER,
KREA_TOKEN_KEY,
Krea2NegpipTokenizer,
encode_krea2_token_weights_negpip,
krea2_attn1_negpip,
krea2_diffusion_negpip_wrapper,
krea2_extra_conds_negpip_wrapper,
)
from ..runtime.negpip.krea2 import (
WRAPPER_KEY as KREA_WRAPPER_KEY,
)
from ..runtime.negpip.standard import (
encode_token_weights_negpip,
standard_attn2_negpip,
)
from ..runtime.patcher_lifecycle import PATCHER_LIFECYCLE
LOGGER = logging.getLogger(__name__)
MODEL_MARKER = "ppm_negpip"
SUPPORTED_STANDARD_ENCODERS = ("clip_g", "clip_l", "t5xxl", "llama", "qwen3_06b")
class NegpipModelService:
"""Apply exactly one family-specific NegPiP patch set when supported."""
def prepare(self, model: object, clip: object) -> tuple[object, object]:
"""Return a patched clone pair or the original unsupported pair unchanged."""
if not isinstance(model, ModelPatcher) or not isinstance(clip, CLIP):
raise TypeError("Automatic NegPiP requires Comfy MODEL and CLIP objects.")
marker = model.model_options.get(MODEL_MARKER, False)
if not isinstance(marker, bool):
raise TypeError("MODEL ppm_negpip marker must be boolean.")
if marker:
LOGGER.debug("Automatic NegPiP reused an already-patched MODEL")
return model, clip
model_type = type(model.model)
if model_type is Krea2:
return self._prepare_krea2(model, clip)
if model_type is Anima:
return self._prepare_anima(model, clip)
if model_type is BaseModel or issubclass(model_type, (SDXL, SDXLRefiner)):
return self._prepare_standard(model, clip)
LOGGER.debug(
"Automatic NegPiP skipped unsupported model family",
extra={"model_type": model_type.__qualname__},
)
return model, clip
def _prepare_standard(
self,
model: ModelPatcher,
clip: CLIP,
) -> tuple[ModelPatcher, CLIP]:
"""Install PPM-compatible interleaved key/value encoding on SD1 or SDXL."""
encoders = [
name
for name in SUPPORTED_STANDARD_ENCODERS
if hasattr(clip.patcher.model, name)
]
if not encoders:
LOGGER.warning("Automatic NegPiP found no supported standard text encoder")
return model, clip
prepared_clip = PATCHER_LIFECYCLE.derive_clip(
clip,
(
*(
ClipCallableObjectPatchMutation(
f"{encoder_name}.encode_token_weights",
partial(
encode_token_weights_negpip,
getattr(clip.patcher.model, encoder_name),
),
)
for encoder_name in encoders
),
ClipBooleanOptionMutation(MODEL_MARKER, True),
),
operation="automatic standard NegPiP CLIP preparation",
)
prepared_model = PATCHER_LIFECYCLE.derive_model(
model,
(
ModelAttentionPatchMutation("attn2", standard_attn2_negpip),
ModelBooleanOptionMutation(MODEL_MARKER, True),
),
operation="automatic standard NegPiP MODEL preparation",
)
return prepared_model, prepared_clip
def _prepare_anima(
self,
model: ModelPatcher,
clip: CLIP,
) -> tuple[ModelPatcher, CLIP]:
"""Install PPM-compatible Anima weight-mask conditions and attention."""
previous = model.get_model_object("extra_conds")
prepared_model = PATCHER_LIFECYCLE.derive_model(
model,
(
ModelCallableObjectPatchMutation(
"extra_conds",
anima_extra_conds_negpip_wrapper(previous),
),
ModelInteropDiffusionWrapperMutation(
ANIMA_WRAPPER_KEY,
anima_diffusion_negpip_wrapper,
),
ModelAttentionPatchMutation("attn2", anima_attn2_negpip),
ModelBooleanOptionMutation(MODEL_MARKER, True),
),
operation="automatic Anima NegPiP MODEL preparation",
)
prepared_clip = PATCHER_LIFECYCLE.derive_clip(
clip,
(ClipBooleanOptionMutation(MODEL_MARKER, True),),
operation="automatic Anima NegPiP CLIP preparation",
)
return prepared_model, prepared_clip
def _prepare_krea2(
self,
model: ModelPatcher,
clip: CLIP,
) -> tuple[ModelPatcher, CLIP]:
"""Install Krea's shape-preserving sign-mask encoder and attention patch."""
if not hasattr(clip.patcher.model, KREA_TOKEN_KEY):
LOGGER.warning("Automatic NegPiP found no Krea Qwen3-VL text encoder")
return model, clip
outer_encoder = clip.patcher.get_model_object("encode_token_weights")
prepared_clip = PATCHER_LIFECYCLE.derive_clip(
clip,
(
ClipTokenizerMutation(
clip.tokenizer,
Krea2NegpipTokenizer(clip.tokenizer),
),
ClipCallableObjectPatchMutation(
"encode_token_weights",
partial(encode_krea2_token_weights_negpip, outer_encoder),
),
ClipBooleanOptionMutation(CLIP_MARKER, True),
),
operation="automatic Krea 2 NegPiP CLIP preparation",
)
previous = model.get_model_object("extra_conds")
prepared_model = PATCHER_LIFECYCLE.derive_model(
model,
(
ModelCallableObjectPatchMutation(
"extra_conds",
krea2_extra_conds_negpip_wrapper(previous),
),
ModelDiffusionWrapperMutation(
KREA_WRAPPER_KEY,
krea2_diffusion_negpip_wrapper,
),
ModelAttentionPatchMutation("attn1", krea2_attn1_negpip),
ModelBooleanOptionMutation(MODEL_MARKER, True),
),
operation="automatic Krea 2 NegPiP MODEL preparation",
)
LOGGER.info(
"Automatic NegPiP enabled",
extra={"model_family": "krea2", "encoder": KREA_TOKEN_KEY},
)
return prepared_model, prepared_clip
NEGPIP_MODEL_SERVICE = NegpipModelService()
@@ -9,6 +9,7 @@ from __future__ import annotations
from typing import Any, TypeAlias
import torch
from comfy.hooks import HookGroup
from ..domain.conditioning_batch import ConditioningBatch
from ..domain.regional_prompting import (
@@ -19,6 +20,9 @@ from ..masking.regional_prompt_masks import (
prepare_regional_mask_batch,
regional_mask,
)
from ..runtime.global_first_conditioning_hooks import (
GLOBAL_FIRST_CONDITIONING_HOOK_COMPOSER,
)
from ..runtime.regional_conditioning_companion import detach_global_companion
from ..shared.logging import get_logger
@@ -114,6 +118,11 @@ class RegionalConditioningService:
if not plan.pairs:
return self._copy_conditioning(global_conditioning)
global_hooks = GLOBAL_FIRST_CONDITIONING_HOOK_COMPOSER.global_hooks(
global_conditioning,
source_label=f"{input_name} global conditioning",
)
hook_cache: dict[tuple[HookGroup, HookGroup], HookGroup] = {}
assembled = self._as_default(global_conditioning)
for pair in plan.pairs:
conditioning = self._validate_conditioning(
@@ -121,6 +130,22 @@ class RegionalConditioningService:
input_name=(f"{input_name} regional entry {pair.conditioning_index}"),
)
conditioning, global_companion = detach_global_companion(conditioning)
conditioning = GLOBAL_FIRST_CONDITIONING_HOOK_COMPOSER.compose(
conditioning,
global_hooks,
source_label=(f"{input_name} regional entry {pair.conditioning_index}"),
cache=hook_cache,
)
if global_companion is not None:
global_companion = GLOBAL_FIRST_CONDITIONING_HOOK_COMPOSER.compose(
global_companion,
global_hooks,
source_label=(
f"{input_name} regional entry {pair.conditioning_index} "
"global companion"
),
cache=hook_cache,
)
mask = regional_mask(mask_batch, pair.mask_index)
if global_companion is not None and regional_prompt_weight < 1.0:
assembled.extend(
@@ -27,9 +27,9 @@ from ..runtime.attention_coupling.unet_context import (
from ..runtime.regional_attention_diagnostics import (
RegionalAttentionDiagnosticsBuilder,
)
from ..runtime.regional_lora.standard_unet_native_admission import (
StandardUnetNativeLoraAdmission,
StandardUnetNativeLoraAdmissionService,
from ..runtime.regional_lora.standard_unet_operation_preparation import (
StandardUnetOperationAdmission,
StandardUnetOperationPreparation,
)
from ..runtime.regional_lora_plan_adapter import RegionalLoraPlanAdaptation
from ..runtime.regional_model_patch_interop import RegionalModelPatchInteropReport
@@ -47,8 +47,8 @@ class StandardUnetAttentionCouplingModelFamily:
backend_class: ClassVar[type[StandardUnetAttentionBackend]] = (
StandardUnetAttentionBackend
)
native_admission_class: ClassVar[type[StandardUnetNativeLoraAdmissionService]] = (
StandardUnetNativeLoraAdmissionService
operation_preparation_class: ClassVar[type[StandardUnetOperationPreparation]] = (
StandardUnetOperationPreparation
)
@property
@@ -84,7 +84,7 @@ class StandardUnetAttentionCouplingModelFamily:
raise TypeError(
"Standard UNet Attention Coupling requires regional adaptation."
)
return self.native_admission_class().admit(model, adaptation)
return self.operation_preparation_class().admit(model, adaptation)
def prepare_sampler_conditioning(
self,
@@ -113,8 +113,8 @@ class StandardUnetAttentionCouplingModelFamily:
) -> object:
"""Build shared diagnostics state and derive the paired attn2 backend."""
if not isinstance(admission, StandardUnetNativeLoraAdmission):
raise TypeError("Standard UNet derivation requires native admission.")
if not isinstance(admission, StandardUnetOperationAdmission):
raise TypeError("Standard UNet derivation requires operation admission.")
if not isinstance(interop_report, RegionalModelPatchInteropReport):
raise TypeError("Standard UNet derivation requires interop evidence.")
if admission.adaptation.plan != processed_plan.lora_plan:
@@ -142,6 +142,7 @@ class StandardUnetAttentionCouplingModelFamily:
model=model,
state=state,
admission=admission,
negpip=interop_report.negpip,
)
.model
)
@@ -76,6 +76,7 @@ def test_anima_family_retains_single_frame_context_and_backend_policy() -> None:
"adaptation": adaptation,
"region_strengths": (0.75,),
"latent_batch_size": 2,
"negpip": None,
}
]
+77
View File
@@ -31,6 +31,10 @@ from simple_syrup.domain.spatial_views import (
SpatialViewKind,
)
from simple_syrup.runtime.patcher_lifecycle import PATCHER_LIFECYCLE
from simple_syrup.runtime.ppm_negpip_interop import (
PpmNegpipInterop,
PpmNegpipSemantics,
)
from simple_syrup.runtime.regional_lora.anima_activation_context import (
AnimaActivationContext,
AnimaActivationGeometry,
@@ -258,6 +262,79 @@ def test_patch_batches_complete_branches_and_blends_expected_outputs(
assert invocation_context.current_or_none() is None
def test_patch_packs_anima_negpip_masks_with_the_same_branch_segments() -> None:
"""Align each compact regional context with its own NegPiP value mask."""
activation_context = AnimaActivationContext()
invocation_context = AnimaCrossAttentionInvocationContext()
original = _DeterministicCrossAttention(invocation_context)
base = torch.zeros((1, 2, 1))
region = torch.ones_like(base)
base_multiplier = torch.tensor([[[1.0], [-1.0]]])
region_multiplier = torch.tensor([[[-1.0], [1.0]]])
contexts = BatchedRegionalAttentionContexts(
latent_batch_size=1,
chunks=(
RegionalAttentionChunkBatch(
0,
RegionalAttentionBranch.POSITIVE,
0,
1,
),
),
base_context=base,
regions=(
BatchedRegionalAttentionRegion(
0,
(
BatchedRegionalAttentionEntry(
0,
region,
(1.0,),
region_multiplier,
),
),
),
),
base_value_multiplier=base_multiplier,
)
execution = AnimaRegionalAttentionExecution(
contexts,
_bank(torch.full((1, 1, 1), 0.5)),
(1.0,),
)
negpip = PpmNegpipInterop(
PpmNegpipSemantics.ANIMA_VALUE_MASK,
lambda *args, **kwargs: (args, kwargs),
)
patch = AnimaRegionalCrossAttentionPatch(
original,
execution,
activation_context=activation_context,
invocation_context=invocation_context,
phase_context=_FullRegionalPhaseContext(),
negpip=negpip,
)
old_mask = torch.ones_like(base_multiplier)
options: dict[str, object] = {"ppm_negpip_mask": old_mask}
with activation_context.activate(_geometry(batch=1, height=1, width=1)):
patch(
torch.zeros((1, 1, 1)),
base,
transformer_options=options,
)
observed_options = original.calls[0][3]
assert isinstance(observed_options, dict)
assert observed_options is not options
assert torch.equal(
observed_options["ppm_negpip_mask"],
torch.cat((base_multiplier, region_multiplier)),
)
assert options["ppm_negpip_mask"] is old_mask
def test_cross_attention_backing_module_does_not_leak_a_host_weight_namespace() -> None:
"""Keep the retained installed attention outside PyTorch child discovery."""
-219
View File
@@ -1,219 +0,0 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Verify exact static-global and regional Anima LoRA overlap rejection."""
from __future__ import annotations
from types import SimpleNamespace
import pytest
import torch
from comfy.weight_adapter.lora import LoRAAdapter
from simple_syrup.domain.regional_lora_plan import (
RegionalLoraAdapterIdentity,
RegionalLoraAdapterPlan,
RegionalLoraBranch,
RegionalLoraPlan,
RegionalLoraScheduleBoundary,
)
from simple_syrup.runtime.regional_lora.anima_global_lora_overlap import (
AnimaGlobalRegionalLoraOverlapError,
AnimaGlobalRegionalLoraOverlapValidator,
)
from simple_syrup.runtime.regional_lora.anima_plan_admission import (
AnimaRegionalLoraAdapterAdmission,
AnimaRegionalLoraPlanAdmission,
)
from simple_syrup.runtime.regional_lora.anima_targets import (
AnimaLoraAdmission,
AnimaLoraTarget,
AnimaLoraTargetFamily,
anima_lora_target_name,
expected_anima_lora_features,
)
from simple_syrup.runtime.regional_lora.standard_adapter import StandardLoraTarget
def test_overlap_validator_accepts_empty_distinct_and_partial_global_state() -> None:
"""Preserve unpatched and content-distinct global MODEL LoRAs."""
admission, targets = _admission(target_count=2)
validator = AnimaGlobalRegionalLoraOverlapValidator()
validator.validate(SimpleNamespace(patches={}), admission)
validator.validate(
SimpleNamespace(patches={_key(targets[0]): [_patch(targets[0])]}),
admission,
)
changed = targets[0].up.clone()
changed[0, 0] += 1.0
validator.validate(
SimpleNamespace(
patches={
_key(targets[0]): [_patch(targets[0], up=changed)],
_key(targets[1]): [_patch(targets[1])],
}
),
admission,
)
@pytest.mark.parametrize("clone_tensors", [False, True], ids=("identity", "content"))
def test_overlap_validator_rejects_complete_exact_global_content(
clone_tensors: bool,
) -> None:
"""Reject exact adapter content with identity and cloned-tensor paths."""
admission, targets = _admission(target_count=2)
patches = {
_key(target): [
_patch(
target,
down=target.down.clone() if clone_tensors else target.down,
up=target.up.clone() if clone_tensors else target.up,
)
]
for target in targets
}
with pytest.raises(
AnimaGlobalRegionalLoraOverlapError,
match="already applied globally.*regional.safetensors",
):
AnimaGlobalRegionalLoraOverlapValidator().validate(
SimpleNamespace(patches=patches),
admission,
)
@pytest.mark.parametrize(
"patch_factory",
(
lambda target: _patch(target, strength=0.0),
lambda target: (1.0, ("diff", (target.up,)), 1.0, None, None),
lambda target: _patch(target, alpha=1.0),
),
ids=("zero-strength", "non-lora", "alpha-form"),
)
def test_overlap_validator_preserves_noncomparable_global_patches(
patch_factory: object,
) -> None:
"""Keep inactive and other global patch formats unchanged."""
admission, targets = _admission(target_count=1)
factory = patch_factory
assert callable(factory)
AnimaGlobalRegionalLoraOverlapValidator().validate(
SimpleNamespace(patches={_key(targets[0]): [factory(targets[0])]}),
admission,
)
@pytest.mark.parametrize(
("patches", "message"),
(
(None, "requires MODEL patches"),
({"diffusion_model.blocks.0.self_attn.q_proj.weight": object()}, "list"),
({"diffusion_model.blocks.0.self_attn.q_proj.weight": [object()]}, "tuple"),
(
{
"diffusion_model.blocks.0.self_attn.q_proj.weight": [
(float("nan"), object(), 1.0)
]
},
"finite",
),
),
)
def test_overlap_validator_fails_closed_on_malformed_model_patch_state(
patches: object,
message: str,
) -> None:
"""Reject installed-host patch drift before regional execution setup."""
admission, _ = _admission(target_count=1)
with pytest.raises((TypeError, ValueError), match=message):
AnimaGlobalRegionalLoraOverlapValidator().validate(
SimpleNamespace(patches=patches),
admission,
)
def _admission(
*, target_count: int
) -> tuple[AnimaRegionalLoraPlanAdmission, tuple[StandardLoraTarget, ...]]:
"""Build one admitted regional adapter with small valid Anima targets."""
families = tuple(AnimaLoraTargetFamily)[:target_count]
targets = tuple(_target(family) for family in families)
plan_entry = RegionalLoraAdapterPlan(
adapter_identity=RegionalLoraAdapterIdentity("regional.safetensors"),
composition_index=0,
region_index=0,
branch=RegionalLoraBranch.POSITIVE,
model_strength=0.8,
schedule=(RegionalLoraScheduleBoundary(0.0, 1.0, 1.0, 0),),
)
plan = RegionalLoraPlan((plan_entry,))
admission = AnimaLoraAdmission(
tuple(
AnimaLoraTarget(0, family, target)
for family, target in zip(families, targets, strict=True)
)
)
return (
AnimaRegionalLoraPlanAdmission(
plan,
(AnimaRegionalLoraAdapterAdmission(plan_entry, admission),),
),
targets,
)
def _target(family: AnimaLoraTargetFamily) -> StandardLoraTarget:
"""Return one rank-one target with installed Anima feature dimensions."""
input_features, output_features = expected_anima_lora_features(family)
return StandardLoraTarget(
target=anima_lora_target_name(0, family),
down=torch.arange(input_features, dtype=torch.float32).reshape(1, -1),
up=torch.arange(output_features, dtype=torch.float32).reshape(-1, 1),
rank=1,
input_features=input_features,
output_features=output_features,
)
def _key(target: StandardLoraTarget) -> str:
"""Return the installed Comfy MODEL patch key for one target."""
return f"{target.target}.weight"
def _patch(
target: StandardLoraTarget,
*,
strength: float = 1.0,
down: torch.Tensor | None = None,
up: torch.Tensor | None = None,
alpha: float | None = None,
) -> tuple[object, ...]:
"""Return one installed-Comfy standard static LoRA patch entry."""
adapter = LoRAAdapter(
set(),
(
target.up if up is None else up,
target.down if down is None else down,
alpha,
None,
None,
None,
),
)
return (strength, adapter, 1.0, None, None)
@@ -25,10 +25,14 @@ from simple_syrup.domain.regional_attention_execution import (
RegionalAttentionExecutionMode,
)
from simple_syrup.domain.regional_lora_plan import EMPTY_REGIONAL_LORA_PLAN
from simple_syrup.domain.regional_model_capabilities import RegionalModelFamily
from simple_syrup.runtime.attention_coupling.family_admission import (
AttentionCouplingFamilyAdmission,
)
from simple_syrup.runtime.regional_lora_plan_adapter import RegionalLoraPlanAdaptation
from simple_syrup.runtime.regional_model_patch_interop import (
RegionalModelPatchInteropReport,
)
from simple_syrup.services.attention_coupling_model_family import (
AttentionCouplingPreparedModelReuse,
AttentionCouplingSamplerConditioning,
@@ -59,10 +63,16 @@ class _CapabilityService:
class _InteropValidator:
"""Record centralized modifier admission without requiring a real patcher."""
report: ClassVar[object] = object()
report: ClassVar[RegionalModelPatchInteropReport] = RegionalModelPatchInteropReport(
RegionalModelFamily.ANIMA, ()
)
calls: ClassVar[list[tuple[object, ...]]] = []
def validate(self, model: object, capabilities: object) -> object:
def validate(
self,
model: object,
capabilities: object,
) -> RegionalModelPatchInteropReport:
"""Record exact orchestration inputs without changing them."""
type(self).calls.append((model, capabilities))
+44 -1
View File
@@ -14,7 +14,10 @@ from typing import Any, cast
import pytest
import torch
from simple_syrup.runtime.patcher_lifecycle import ComfyPatcherLifecycle
from simple_syrup.runtime.patcher_lifecycle import (
PATCHER_LIFECYCLE,
ComfyPatcherLifecycle,
)
from simple_syrup.runtime.regional_lora.execution_cache import ModelCloneLineage
@@ -64,6 +67,46 @@ def test_real_comfy_anima_lifecycle_regression(
_assert_supported_model_mutations_share_one_clone()
@pytest.mark.parametrize("derivation_count", (2, 3, 5))
def test_stacked_model_derivations_survive_simultaneous_cyclic_release(
caplog: pytest.LogCaptureFixture,
monkeypatch: pytest.MonkeyPatch,
derivation_count: int,
) -> None:
"""Keep Comfy on a foreign boundary when any Syrup stack dies together."""
import comfy.model_management
from comfy.model_management import LoadedModel
encoder = AnimaTEModel_()
loader = _patcher(encoder)
foreign_boundary = loader.clone()
derived_models: list[object] = []
current = foreign_boundary
for stage in range(derivation_count):
current = PATCHER_LIFECYCLE.derive_model(
current,
(),
operation=f"stacked lifecycle regression stage {stage}",
)
derived_models.append(current)
loaded = LoadedModel(current)
loaded.real_model = weakref.ref(encoder)
monkeypatch.setattr(comfy.model_management, "current_loaded_models", [loaded])
execution_cycle: list[object] = [*derived_models]
execution_cycle.append(execution_cycle)
del current, derived_models, execution_cycle
gc.collect()
with caplog.at_level(logging.INFO):
comfy.model_management.cleanup_models_gc()
assert loaded.model is foreign_boundary
assert loaded.is_dead() is False
assert "Potential memory leak detected" not in caplog.text
assert "WARNING, memory leak" not in caplog.text
def test_clip_alignment_precedes_mutations_after_dynamic_to_static_clone() -> None:
"""Mutate the same independently reloaded encoder the returned CLIP executes."""
@@ -8,7 +8,7 @@ from __future__ import annotations
from pathlib import Path
from types import SimpleNamespace
from typing import Any
from typing import Any, cast
from uuid import UUID
import comfy.conds
@@ -31,6 +31,10 @@ from simple_syrup.runtime.attention_coupling.unet_context import (
from simple_syrup.runtime.comfy_conditioning_processing import (
COMFY_REGIONAL_CONDITIONING_PROCESSOR,
)
from simple_syrup.runtime.ppm_negpip_interop import (
PpmNegpipInterop,
PpmNegpipSemantics,
)
from simple_syrup.services.attention_coupling_preparation_service import (
ATTENTION_COUPLING_PREPARATION_SERVICE,
AttentionCouplingPreparation,
@@ -89,6 +93,26 @@ class _LinearModelSampling:
return 100.0 * (1.0 - float(percent))
class _NegpipRecordingAnimaModel(_RecordingAnimaModel):
"""Return a distinct PPM-style value mask for every processed context."""
def extra_conds(self, **kwargs: Any) -> dict[str, object]:
"""Add a binary value mask beside the ordinary cross-attention output."""
result = super().extra_conds(**kwargs)
output = self.outputs[-1]
value = int(kwargs["negpip_value"])
result["c_ppm_negpip_mask"] = comfy.conds.CONDRegular(
torch.full(
(*output.shape[:2], 1),
value,
dtype=torch.int32,
device=output.device,
)
)
return result
def test_processor_uses_comfy_conversion_and_model_post_adapter_contexts() -> None:
"""Retain weighted padded model outputs in exact positive/negative order."""
@@ -146,6 +170,47 @@ def test_processor_uses_comfy_conversion_and_model_post_adapter_contexts() -> No
assert source_positive_context.shape == (1, 3, 1024)
def test_processor_retains_each_anima_negpip_mask_with_its_scheduled_entry() -> None:
"""Keep value semantics attached through Comfy conversion and UUID ownership."""
model = _NegpipRecordingAnimaModel()
preparation = _preparation(
positive=(
_conditioning(1.0, negpip_value=1),
_conditioning(2.0, negpip_value=-1),
),
negative=(
_conditioning(-1.0, negpip_value=-1),
_conditioning(-2.0, negpip_value=1),
),
)
negpip = PpmNegpipInterop(
PpmNegpipSemantics.ANIMA_VALUE_MASK,
lambda *args, **kwargs: (args, kwargs),
)
processed = COMFY_REGIONAL_CONDITIONING_PROCESSOR.process(
preparation,
model=SimpleNamespace(model=model),
noise=torch.zeros((1, 16, 8, 8)),
device=torch.device("cpu"),
context_validator=ANIMA_REGIONAL_CONTEXT_VALIDATOR,
negpip=negpip,
)
entries = (
processed.positive.base_context.entries[0],
processed.positive.regional_contexts[0].entries[0],
processed.negative.base_context.entries[0],
processed.negative.regional_contexts[0].entries[0],
)
assert all(entry.cross_attention_value_multiplier is not None for entry in entries)
assert [
int(cast(torch.Tensor, entry.cross_attention_value_multiplier)[0, 0, 0].item())
for entry in entries
] == [1, -1, -1, 1]
@pytest.mark.parametrize(
("sequence_length", "feature_width", "message"),
[
@@ -347,6 +412,7 @@ def _conditioning(
strength: float | None = None,
start_percent: float | None = None,
end_percent: float | None = None,
negpip_value: int | None = None,
) -> list[list[object]]:
"""Build one small standard conditioning with model-consumed metadata."""
@@ -357,6 +423,8 @@ def _conditioning(
metadata["start_percent"] = start_percent
if end_percent is not None:
metadata["end_percent"] = end_percent
if negpip_value is not None:
metadata["negpip_value"] = negpip_value
return [
[
torch.full((1, 3, ANIMA_CONTEXT_FEATURE_WIDTH), value),
+18 -5
View File
@@ -182,11 +182,16 @@ def test_global_call_uses_one_full_source_reduced_model_layout(
*,
args: dict[str, Any],
layout: SpatialBatchLayout,
project_canvas_reference_latents: bool = False,
) -> dict[str, Any]:
"""Capture and apply the global model-argument layout."""
layouts.append(layout)
return transform(args=args, layout=layout)
return transform(
args=args,
layout=layout,
project_canvas_reference_latents=project_canvas_reference_latents,
)
monkeypatch.setattr(
wrapper_module,
@@ -389,8 +394,8 @@ def test_weighted_correction_formula_is_exact_for_both_local_fusion_modes(
assert torch.allclose(output[:, :, 1::2], torch.full((1, 1, 8, 32), 0.5))
def test_local_and_global_calls_receive_complete_reference_latents() -> None:
"""Keep independent reference images intact through both spatial views."""
def test_local_and_global_calls_project_canvas_reference_latents() -> None:
"""Give every Contextual Diffusion view its spatially aligned reference."""
reference = torch.arange(1 * 4 * 16 * 32, dtype=torch.float32).reshape(
(1, 4, 16, 32)
@@ -419,8 +424,16 @@ def test_local_and_global_calls_receive_complete_reference_latents() -> None:
)
assert len(received) == 2
assert torch.equal(received[0], torch.cat((reference, reference), dim=0))
assert torch.equal(received[1], reference)
assert torch.equal(
received[0],
torch.cat((reference[..., :16], reference[..., 16:]), dim=0),
)
expected_global = torch.nn.functional.interpolate(
reference.reshape(-1, 1, 16, 32),
size=(8, 16),
mode="nearest-exact",
).reshape(1, 4, 8, 16)
assert torch.equal(received[1], expected_global)
def test_one_tile_plan_delegates_to_one_original_evaluation() -> None:
@@ -140,6 +140,40 @@ def test_positive_conditioning_batch_selects_by_segment_index() -> None:
assert [call.negative for call in sampler.sample_calls] == [negative, negative]
def test_detailer_keeps_prompt_control_hooks_peer_scoped_by_segment() -> None:
"""Preserve each scheduled LoRA hook on only its selected face conditioning."""
sampler = _FakeSampler()
first = _segment(CropRegion(0, 0, 4, 4), BoundingBox(1, 1, 3, 3))
second = _segment(CropRegion(4, 4, 8, 8), BoundingBox(5, 5, 7, 7))
first_hooks = object()
second_hooks = object()
first_conditioning = [["first", {"hooks": first_hooks}]]
second_conditioning = [["second", {"hooks": second_hooks}]]
service = _service(sampler)
service.detail(
_image(),
_segs(first, second),
object(),
object(),
ConditioningBatch((first_conditioning, second_conditioning)),
[],
**_settings(),
)
assert sampler.sample_calls[0].positive is first_conditioning
assert sampler.sample_calls[1].positive is second_conditioning
selected_first = cast(list[list[object]], sampler.sample_calls[0].positive)
selected_second = cast(list[list[object]], sampler.sample_calls[1].positive)
first_metadata = selected_first[0][1]
second_metadata = selected_second[0][1]
assert isinstance(first_metadata, dict)
assert isinstance(second_metadata, dict)
assert first_metadata["hooks"] is first_hooks
assert second_metadata["hooks"] is second_hooks
def test_negative_conditioning_batch_selects_by_segment_index() -> None:
"""A negative batch varies by SEG while normal positive broadcasts."""
+94
View File
@@ -0,0 +1,94 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Verify pre-derivation Comfy MODEL selection for global prompt hooks."""
from __future__ import annotations
from comfy.hooks import HookGroup
from simple_syrup.domain.conditioning_batch import ConditioningBatch
from simple_syrup.runtime.global_hook_model_resolver import GlobalHookModelResolver
class _Model:
"""Expose the dynamic-model boundary used by Comfy's CFG guider."""
def __init__(self, *, dynamic: bool, delegate: object | None = None) -> None:
"""Retain configured dynamic state and delegate result."""
self.dynamic = dynamic
self.delegate = delegate
self.delegate_calls = 0
def is_dynamic(self) -> bool:
"""Return the configured Comfy model mode."""
return self.dynamic
def get_non_dynamic_delegate(self) -> object:
"""Return and record the configured static delegate."""
self.delegate_calls += 1
return self.delegate
def test_unhooked_global_entry_preserves_dynamic_model() -> None:
"""Leave regional-only hooks to the existing custom regional runtime."""
model = _Model(dynamic=True)
conditioning = ConditioningBatch((_conditioning(), _conditioning(HookGroup())))
resolved = GlobalHookModelResolver().resolve(
model,
positive=conditioning,
negative=conditioning,
)
assert resolved is model
assert model.delegate_calls == 0
def test_global_hook_preserves_already_static_model() -> None:
"""Avoid unnecessary model replacement when hook execution is already static."""
model = _Model(dynamic=False)
resolved = GlobalHookModelResolver().resolve(
model,
positive=_conditioning(HookGroup()),
negative=_conditioning(HookGroup()),
)
assert resolved is model
assert model.delegate_calls == 0
def test_global_hook_selects_static_delegate_before_regional_derivation() -> None:
"""Bind regional wrappers to the same static graph Comfy samples with hooks."""
delegate = _Model(dynamic=False)
model = _Model(dynamic=True, delegate=delegate)
resolved = GlobalHookModelResolver().resolve(
model,
positive=ConditioningBatch(
(_conditioning(HookGroup()), _conditioning(HookGroup()))
),
negative=ConditioningBatch(
(_conditioning(HookGroup()), _conditioning(HookGroup()))
),
)
assert resolved is delegate
assert model.delegate_calls == 1
def _conditioning(hooks: HookGroup | None = None) -> list[list[object]]:
"""Return one standard conditioning with optional model hooks."""
metadata: dict[str, object] = {}
if hooks is not None:
metadata["hooks"] = hooks
return [["embedding", metadata]]
-7
View File
@@ -29,13 +29,6 @@ def test_matrix_covers_global_strength_regional_and_duplicate_placement() -> Non
1.0,
1.0,
]
assert [case.expect_overlap_rejection for case in definitions] == [
False,
False,
False,
False,
True,
]
def test_prompt_renderer_places_primary_adapter_only_in_declared_segments() -> None:
+143
View File
@@ -0,0 +1,143 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Verify exact user-prompt global LoRA proof graph mutations."""
from __future__ import annotations
from tools.run_global_prompt_lora_proof import (
ProofCase,
build_case_graph,
cases,
render_global_first_prompt,
)
def test_prompt_renderer_preserves_named_separators_and_places_both_scopes() -> None:
"""Keep global-first layout while preserving the user's named regions."""
rendered = render_global_first_prompt(
"global text[SEP|Taffy]left text[SEP|Anise]right text",
global_tag="<lora:shared:0.5>",
regional_tag="<lora:shared:0.8>",
)
assert rendered == (
"<lora:shared:0.5>\nglobal text"
"[SEP|Taffy]<lora:shared:0.8>\nleft text"
"[SEP|Anise]right text"
)
def test_matrix_covers_same_lora_turbo_tiled_and_contextual() -> None:
"""Keep all requested managed proof variants explicit and ordered."""
definitions = cases()
assert [case.case_id for case in definitions] == [
"sdxl-global-and-regional-same-lora",
"anima-global-and-regional-same-lora",
"anima-turbo-global-arcane-regional-tiled",
"anima-turbo-global-arcane-regional-contextual",
]
assert definitions[0].global_tag == definitions[0].regional_tag
assert "ArcaneViolet" in definitions[1].global_tag
assert "ArcaneViolet" in definitions[1].regional_tag
assert [case.turbo for case in definitions] == [False, False, True, True]
assert [case.contextual for case in definitions] == [False, False, False, True]
assert definitions[0].region_mask_feather == 10
assert {case.region_mask_feather for case in definitions[1:]} == {64}
def test_turbo_contextual_graph_uses_appropriate_sampling_contract() -> None:
"""Apply Turbo's low-step CFG-one contract to full and contextual stages."""
template = _anima_template()
case = ProofCase(
"fixture",
"anima",
"<lora:turbo:0.7>",
"<lora:regional:0.8>",
turbo=True,
contextual=True,
)
graph, save_ids = build_case_graph(template, case, run_id="run")
assert save_ids == ("proof:source", "proof:refinement")
for node_id in (
"anima-prompt-region:ksampler",
"anima-diffusion-upscale:ksampler",
):
inputs = graph[node_id]["inputs"]
assert isinstance(inputs, dict)
assert inputs["steps"] == 10
assert inputs["cfg"] == 1.0
assert inputs["sampler_name"] == "euler"
assert inputs["scheduler"] == "simple"
assert inputs["region_mask_feather"] == 64
refinement = graph["anima-diffusion-upscale:ksampler"]
assert refinement["class_type"] == "SimpleSyrup.KSamplerAttentionCouplingContextual"
refinement_inputs = refinement["inputs"]
assert isinstance(refinement_inputs, dict)
assert "latent_context_size" in refinement_inputs
assert "latent_tile_width" not in refinement_inputs
def test_replayed_multiselect_mask_values_are_literal_wrapped() -> None:
"""Keep executed multiselect lists from being reinterpreted as graph links."""
template = _anima_template()
template["mask-loader"] = {
"class_type": "SimpleSyrup.LoadMaskBatch",
"inputs": {"image": ["left.png", "right.png"], "channel": "red"},
}
case = ProofCase("fixture", "anima", "<lora:g:1>", "<lora:r:1>")
graph, _save_ids = build_case_graph(template, case, run_id="run")
assert graph["mask-loader"]["inputs"] == {
"image": {"__value__": ["left.png", "right.png"]},
"channel": "red",
}
def _anima_template() -> dict[str, dict[str, object]]:
"""Return one minimal expanded Anima graph accepted by the mutator."""
return {
"anima-prompt-region:positive_prompt": {
"class_type": "PrimitiveStringMultiline",
"inputs": {"value": "global[SEP|Taffy]left[SEP|Anise]right"},
},
"anima-diffusion-upscale:positive_prompt": {
"class_type": "PrimitiveStringMultiline",
"inputs": {"value": "global[SEP|Taffy]left[SEP|Anise]right"},
},
"anima-prompt-region:ksampler": {
"class_type": "SimpleSyrup.KSamplerAttentionCoupling",
"inputs": {},
},
"anima-diffusion-upscale:ksampler": {
"class_type": "SimpleSyrup.KSamplerAttentionCouplingTiled",
"inputs": {
"latent_tile_width": 128,
"latent_tile_height": 128,
"latent_tile_overlap": 16,
"latent_tile_batch_size": 4,
},
},
"anima-prompt-region:vae_decode": {
"class_type": "VAEDecode",
"inputs": {},
},
"anima-diffusion-upscale:vae_decode": {
"class_type": "VAEDecode",
"inputs": {},
},
"__sugarcubes_cube_output__:fixture": {
"class_type": "SugarCubes.CubeOutput",
"inputs": {},
},
}
+4 -29
View File
@@ -49,19 +49,12 @@ def test_p9_3_matrix_covers_global_regional_distinct_and_duplicate_cases() -> No
0,
1,
]
assert [case.expect_overlap_error for case in definitions] == [
False,
False,
False,
False,
True,
]
def test_p9_3_result_requires_success_outputs_and_exact_overlap_rejection(
def test_p9_3_result_requires_success_outputs_for_every_placement(
tmp_path: Path,
) -> None:
"""Persist four images and one pre-sampling rejection before completion."""
"""Persist all five global, regional, and additive placement images."""
recorder = GlobalRegionalLoraResultRecorder(tmp_path)
workflow = BuiltAnimaAttentionCouplingWorkflow(
@@ -77,7 +70,7 @@ def test_p9_3_result_requires_success_outputs_and_exact_overlap_rejection(
"status": {"status_str": "success", "completed": True},
"outputs": {"metrics": {"benchmark_metrics": [{"model_call_count": STEPS}]}},
}
for case in definitions[:-1]:
for case in definitions:
color = (
(240, 10, 10)
if "global-global_adapter-regional" in case.case_id
@@ -93,24 +86,6 @@ def test_p9_3_result_requires_success_outputs_and_exact_overlap_rejection(
reference=ImageReference("image.png", "", "output"),
image_bytes=_png(color),
)
rejection_history: JsonObject = {
"status": {
"status_str": "error",
"completed": False,
"messages": [
"Regional Anima LoRA content is already applied globally to the "
"input MODEL: 'adapter-a.safetensors'."
],
},
"outputs": {},
}
recorder.record_overlap_rejection(
definitions[-1],
workflow,
prompt_id="prompt-duplicate",
history=rejection_history,
)
result_path = recorder.finalize(
definitions,
system_stats={"devices": []},
@@ -125,7 +100,7 @@ def test_p9_3_result_requires_success_outputs_and_exact_overlap_rejection(
"success",
"success",
"success",
"rejected_before_sampling",
"success",
]
assert result["transition"] == {
"distinct_before_after": {
+63 -2
View File
@@ -6,7 +6,9 @@
from __future__ import annotations
from typing import Any
from typing import Any, cast
import pytest
from simple_syrup.nodes.grounded_sam_model_info import GroundedSAMModelInfo
@@ -20,9 +22,25 @@ def test_model_info_node_contract_constants() -> None:
assert GroundedSAMModelInfo.CATEGORY == "SimpleSyrup/Masking"
def test_model_info_node_declares_expected_inputs() -> None:
def test_model_info_node_declares_expected_inputs(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Model info node exposes model selectors."""
class FakeChoices:
"""Return the known downloadable selections for declaration tests."""
def sam_choices(self) -> list[str]:
"""Return the expected SAM choice."""
return ["sam_hq_vit_b (379MB)"]
def grounding_dino_choices(self) -> list[str]:
"""Return the expected GroundingDINO choice."""
return ["GroundingDINO_SwinT_OGC (694MB)"]
monkeypatch.setattr(GroundedSAMModelInfo, "_choices", cast(Any, FakeChoices()))
input_types: dict[str, dict[str, tuple[Any, ...]]] = (
GroundedSAMModelInfo.INPUT_TYPES()
)
@@ -33,6 +51,38 @@ def test_model_info_node_declares_expected_inputs() -> None:
assert "GroundingDINO_SwinT_OGC (694MB)" in required["grounding_dino_model"][0]
def test_model_info_node_uses_settings_aware_choices() -> None:
"""Model metadata selectors follow the downloadable-models preference."""
class FakeChoices:
"""Return the local-only choices supplied by settings policy."""
def sam_choices(self) -> list[str]:
"""Return the available SAM choices."""
return ["local-sam"]
def grounding_dino_choices(self) -> list[str]:
"""Return the available GroundingDINO choices."""
return ["local-dino"]
def reject_sentinel(self, selection: str) -> None:
"""Accept the deterministic test selections."""
del selection
original = GroundedSAMModelInfo._choices
GroundedSAMModelInfo._choices = cast(Any, FakeChoices())
try:
required = GroundedSAMModelInfo.INPUT_TYPES()["required"]
finally:
GroundedSAMModelInfo._choices = original
assert required["sam_model"][0] == ["local-sam"]
assert required["grounding_dino_model"][0] == ["local-dino"]
def test_model_info_node_delegates_to_metadata_provider() -> None:
"""Node execution delegates metadata creation to its metadata provider."""
@@ -44,12 +94,23 @@ def test_model_info_node_delegates_to_metadata_provider() -> None:
return f"{sam_model}|{grounding_dino_model}"
class FakeChoices:
"""Accept all model selections while exercising metadata delegation."""
def reject_sentinel(self, selection: str) -> None:
"""Accept the deterministic test selections."""
del selection
node = GroundedSAMModelInfo()
original = GroundedSAMModelInfo._metadata
original_choices = GroundedSAMModelInfo._choices
GroundedSAMModelInfo._metadata = FakeMetadata() # type: ignore[assignment]
GroundedSAMModelInfo._choices = cast(Any, FakeChoices())
try:
result = node.describe("sam", "dino")
finally:
GroundedSAMModelInfo._metadata = original
GroundedSAMModelInfo._choices = original_choices
assert result == ("sam|dino",)
+13 -1
View File
@@ -28,9 +28,21 @@ def test_grounding_dino_model_loader_contract() -> None:
assert GroundingDINOModelLoader.CATEGORY == "SimpleSyrup/Masking"
def test_grounding_dino_model_loader_declares_expected_inputs() -> None:
def test_grounding_dino_model_loader_declares_expected_inputs(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""GroundingDINO loader makes text encoder selection explicit."""
def catalog_choices() -> list[str]:
"""Return the catalog choice expected by this declaration test."""
return ["GroundingDINO_SwinT_OGC (694MB)"]
monkeypatch.setattr(
GroundingDINOModelLoader._choices,
"grounding_dino_choices",
catalog_choices,
)
input_types: dict[str, dict[str, tuple[Any, ...]]] = (
GroundingDINOModelLoader.INPUT_TYPES()
)
+5 -1
View File
@@ -41,7 +41,10 @@ class _FakeLegacyNode:
{"default": "hello", "tooltip": "Text to delegate."},
),
},
"hidden": {"prompt": "PROMPT"},
"hidden": {
"prompt": "PROMPT",
"ignored": "UNSUPPORTED_SECRET_SENTINEL",
},
}
def run(self, text: str, prompt: object | None = None) -> tuple[str]:
@@ -70,6 +73,7 @@ def test_legacy_node_v3_adapter_builds_schema() -> None:
assert schema.category == "SimpleSyrup/Test"
assert schema.inputs[0].id == "text"
assert schema.inputs[0].tooltip == "Text to delegate."
assert len(schema.hidden) == 1
assert schema.hidden[0].value == "PROMPT"
assert schema.outputs[0].id == "result"
assert schema.outputs[0].tooltip == "Delegated result."
+7 -1
View File
@@ -11,6 +11,7 @@ from typing import Any, cast
import pytest
from simple_syrup.nodes.load_ultralytics_model import LoadUltralyticsModel
from simple_syrup.runtime.model_downloads import ProgressReporter
from simple_syrup.runtime.ultralytics_loader import LoadedUltralyticsDetector
@@ -50,8 +51,13 @@ class _FakeLoaderService:
return ["model.pt"]
def load(self, model_name: str) -> LoadedUltralyticsDetector:
def load(
self,
model_name: str,
progress: ProgressReporter | None = None,
) -> LoadedUltralyticsDetector:
"""Return deterministic loaded outputs."""
del progress
assert model_name == "model.pt"
return LoadedUltralyticsDetector(cast(Any, "native"), "bbox", "segm")
+74
View File
@@ -16,10 +16,13 @@ from simple_syrup.runtime.model_catalog import (
BERT_ENTRY,
GROUNDING_DINO_ENTRIES,
SAM_ENTRIES,
ULTRALYTICS_ENTRIES,
get_grounding_dino_entry,
get_sam_entry,
get_ultralytics_entry,
grounding_dino_choices,
sam_choices,
ultralytics_choices,
)
@@ -54,6 +57,77 @@ def test_catalog_choices_are_deterministic() -> None:
assert grounding_dino_choices() == [
entry.display_name for entry in GROUNDING_DINO_ENTRIES
]
assert ultralytics_choices() == [
entry.display_name for entry in ULTRALYTICS_ENTRIES
]
def test_ultralytics_catalog_has_pinned_verified_anzhc_checkpoints() -> None:
"""Curated Anzhc models are revision-pinned, verified, and task-foldered."""
anzhc_entries = tuple(
entry
for entry in ULTRALYTICS_ENTRIES
if entry.source_repo == "Anzhc/Anzhcs_YOLOs"
)
assert len(anzhc_entries) == 15
assert all(entry.source_repo == "Anzhc/Anzhcs_YOLOs" for entry in anzhc_entries)
assert all(len(entry.artifacts) == 1 for entry in anzhc_entries)
assert all(
artifact.source_url.startswith(
"https://huggingface.co/Anzhc/Anzhcs_YOLOs/resolve/"
"f5a2306d7fed4f3cfc26c25ff1ab2e3f3cfce855/"
)
for entry in anzhc_entries
for artifact in entry.artifacts
)
assert all(
artifact.folder_name == "ultralytics_segm"
and artifact.sha256 is not None
and len(artifact.sha256) == 64
for entry in anzhc_entries
for artifact in entry.artifacts
)
assert all(
"Drones" not in artifact.filename
and "Score" not in artifact.filename
and "Breast size" not in artifact.filename
for entry in anzhc_entries
for artifact in entry.artifacts
)
def test_ultralytics_catalog_has_verified_adetailer_and_anime_models() -> None:
"""ADetailer and anime face checkpoints have compatible curated metadata."""
assert len(ULTRALYTICS_ENTRIES) == 22
face = get_ultralytics_entry("bingsu_face_yolov8n_v2")
hand = get_ultralytics_entry("bingsu_hand_yolov8s")
person = get_ultralytics_entry("bingsu_person_yolov8s_seg")
anime_face = get_ultralytics_entry("fuyucchi_yolov8x6_animeface")
assert face.artifacts[0].folder_name == "ultralytics_bbox"
assert hand.artifacts[0].folder_name == "ultralytics_bbox"
assert person.artifacts[0].folder_name == "ultralytics_segm"
assert anime_face.artifacts[0].folder_name == "ultralytics_bbox"
assert face.source_repo == "Bingsu/adetailer"
assert face.license_note == "Apache-2.0"
assert (
"/resolve/53cc19de382014514d9d4038601d261a7faa9b7b/"
in face.artifacts[0].source_url
)
assert anime_face.source_repo == "Fuyucchi/yolov8_animeface"
assert anime_face.license_note == "AGPL-3.0"
assert "/resolve/b0841ce930453c0f23ceb8086d6554c17de5fe4a/" in (
anime_face.artifacts[0].source_url
)
assert all(
artifact.sha256 is not None and len(artifact.sha256) == 64
for entry in (face, hand, person, anime_face)
for artifact in entry.artifacts
)
def test_catalog_lookup_rejects_unknown_selection() -> None:
+23 -37
View File
@@ -7,7 +7,6 @@
from __future__ import annotations
from pathlib import Path
from types import ModuleType
import pytest
@@ -40,13 +39,13 @@ def test_downloadable_mode_includes_catalog_entries(tmp_path: Path) -> None:
service = ModelChoiceService(
FakeSettingsRepository(show_downloadable_models=True),
fake_folder_paths(tmp_path),
)
assert "sam_vit_b (375MB)" in service.sam_choices()
assert "GroundingDINO_SwinT_OGC (694MB)" in service.grounding_dino_choices()
assert "vitmatte-small-composition-1k" in service.vitmatte_choices()
assert "wd-eva02-large-tagger-v3" in service.wd14_tagger_choices()
assert "Bingsu Hand YOLOv8n (6.23MB)" in service.ultralytics_choices()
def test_local_only_mode_returns_sentinels_when_no_models_exist(
@@ -60,23 +59,22 @@ def test_local_only_mode_returns_sentinels_when_no_models_exist(
assert service.grounding_dino_choices() == [NO_LOCAL_GROUNDING_DINO_MODELS]
assert service.vitmatte_choices() == [NO_LOCAL_VITMATTE_MODELS]
assert service.wd14_tagger_choices() == [NO_LOCAL_WD14_TAGGER_MODELS]
assert service.ultralytics_choices() == []
def test_sam_local_only_lists_installed_catalog_artifacts(tmp_path: Path) -> None:
"""SAM local-only mode lists installed known checkpoint files."""
def test_hidden_catalog_mode_excludes_installed_sam_artifacts(tmp_path: Path) -> None:
"""Catalog mode hides installed SAM entries when disabled."""
(tmp_path / "models" / "sams").mkdir(parents=True)
(tmp_path / "models" / "sams" / "sam_vit_b_01ec64.pth").write_bytes(b"sam")
choices = local_only_service(tmp_path).sam_choices()
assert choices == ["sam_vit_b (375MB)"]
assert local_only_service(tmp_path).sam_choices() == [NO_LOCAL_SAM_MODELS]
def test_grounding_dino_local_only_requires_complete_artifacts(
def test_hidden_catalog_mode_excludes_installed_grounding_dino_artifacts(
tmp_path: Path,
) -> None:
"""GroundingDINO local-only mode excludes partial config/checkpoint pairs."""
"""Catalog mode hides installed GroundingDINO entries when disabled."""
model_dir = tmp_path / "models" / "grounding-dino"
model_dir.mkdir(parents=True)
@@ -84,37 +82,35 @@ def test_grounding_dino_local_only_requires_complete_artifacts(
(model_dir / "groundingdino_swint_ogc.pth").write_bytes(b"dino")
(model_dir / "GroundingDINO_SwinB.cfg.py").write_text("", encoding="utf-8")
choices = local_only_service(tmp_path).grounding_dino_choices()
assert choices == ["GroundingDINO_SwinT_OGC (694MB)"]
assert local_only_service(tmp_path).grounding_dino_choices() == [
NO_LOCAL_GROUNDING_DINO_MODELS
]
def test_vitmatte_local_only_lists_valid_canonical_directory(
def test_hidden_catalog_mode_excludes_installed_vitmatte_directory(
tmp_path: Path,
) -> None:
"""ViTMatte local-only mode accepts canonical SimpleSyrup directories."""
"""Catalog mode hides installed ViTMatte entries when disabled."""
create_vitmatte_snapshot(
tmp_path / "models" / "vitmatte" / "vitmatte-small-composition-1k"
)
choices = local_only_service(tmp_path).vitmatte_choices()
assert choices == ["vitmatte-small-composition-1k"]
assert local_only_service(tmp_path).vitmatte_choices() == [NO_LOCAL_VITMATTE_MODELS]
def test_vitmatte_local_only_lists_layerstyle_directory(tmp_path: Path) -> None:
"""ViTMatte local-only mode accepts LayerStyle-compatible directories."""
def test_hidden_catalog_mode_excludes_layerstyle_vitmatte_directory(
tmp_path: Path,
) -> None:
"""Catalog mode hides LayerStyle-compatible entries when disabled."""
create_vitmatte_snapshot(tmp_path / "models" / "vitmatte-base-composition-1k")
choices = local_only_service(tmp_path).vitmatte_choices()
assert choices == ["vitmatte-base-composition-1k"]
assert local_only_service(tmp_path).vitmatte_choices() == [NO_LOCAL_VITMATTE_MODELS]
def test_wd14_local_only_requires_complete_artifacts(tmp_path: Path) -> None:
"""WD14 local-only mode excludes partial ONNX/CSV pairs."""
def test_hidden_catalog_mode_excludes_installed_wd14_artifacts(tmp_path: Path) -> None:
"""Catalog mode hides installed WD14 entries when disabled."""
model_dir = tmp_path / "models" / "wd14_tagger"
model_dir.mkdir(parents=True)
@@ -125,9 +121,9 @@ def test_wd14_local_only_requires_complete_artifacts(tmp_path: Path) -> None:
)
(model_dir / "wd-vit-tagger-v3.onnx").write_bytes(b"onnx")
choices = local_only_service(tmp_path).wd14_tagger_choices()
assert choices == ["wd-eva02-large-tagger-v3"]
assert local_only_service(tmp_path).wd14_tagger_choices() == [
NO_LOCAL_WD14_TAGGER_MODELS
]
@pytest.mark.parametrize(
@@ -157,19 +153,9 @@ def local_only_service(tmp_path: Path) -> ModelChoiceService:
return ModelChoiceService(
FakeSettingsRepository(show_downloadable_models=False),
fake_folder_paths(tmp_path),
)
def fake_folder_paths(tmp_path: Path) -> ModuleType:
"""Create a minimal fake Comfy folder_paths module."""
module = ModuleType("folder_paths")
module.models_dir = str(tmp_path / "models") # type: ignore[attr-defined]
module.folder_names_and_paths = {} # type: ignore[attr-defined]
return module
def create_vitmatte_snapshot(path: Path) -> None:
"""Create the minimal file set required for a valid ViTMatte directory."""
+35
View File
@@ -113,6 +113,41 @@ def test_comfy_progress_reporter_updates_the_active_node_progress(
assert updates == [(-1, 6), (0, 6), (3, 6), (6, 6)]
def test_comfy_progress_reporter_does_not_falsely_complete_unknown_downloads(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Unknown response lengths emit no misleading progress until completion."""
updates: list[tuple[int, int | None]] = []
class FakeProgressBar:
"""Record ComfyUI absolute progress updates."""
def __init__(self, total: int) -> None:
"""Record the total selected for the progress bar."""
updates.append((-1, total))
def update_absolute(self, value: int, total: int | None = None) -> None:
"""Record one absolute progress update."""
updates.append((value, total))
comfy_module = ModuleType("comfy")
comfy_utils = ModuleType("comfy.utils")
comfy_utils.ProgressBar = FakeProgressBar # type: ignore[attr-defined]
comfy_module.utils = comfy_utils # type: ignore[attr-defined]
monkeypatch.setitem(sys.modules, "comfy", comfy_module)
monkeypatch.setitem(sys.modules, "comfy.utils", comfy_utils)
reporter = ComfyProgressReporter()
reporter.start("Downloading unknown-size model", None)
reporter.advance(1024 * 1024, None)
reporter.finish()
assert updates == [(-1, 1), (1, 1)]
def test_downloader_streams_file_and_reports_progress(
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
+29
View File
@@ -235,6 +235,35 @@ def test_collision_safe_mutations_integrate_through_one_real_comfy_clone() -> No
assert ModelPatcher.set_model_attn2_patch is original_attn2_setter
def test_attn2_mutation_prepends_before_exact_preserved_input_patch() -> None:
"""Compose regional packing before an identity-admitted input transformer."""
model = _patcher(torch.nn.Linear(1, 1))
def preserved(*args: object) -> tuple[object, ...]:
"""Return preserved callback arguments."""
return args
def regional(*args: object) -> tuple[object, ...]:
"""Return regional callback arguments."""
return args
def output(*args: object) -> tuple[object, ...]:
"""Return output callback arguments."""
return args
model.set_model_attn2_patch(preserved)
ModelAttn2PatchesMutation(regional, output, (preserved,)).apply(model)
patches = model.model_options["transformer_options"]["patches"]
assert patches["attn2_patch"] == [regional, preserved]
assert patches["attn2_output_patch"] == [output]
@pytest.mark.parametrize(
("wrapper_type", "key", "wrapper", "message"),
[
@@ -152,8 +152,10 @@ def test_native_sampler_activates_hooks_from_masked_regional_conditioning(
) -> None:
"""Comfy activates each preserved hook group through direct and tiled paths."""
global_hooks = comfy.hooks.HookGroup()
regional_hooks = comfy.hooks.HookGroup()
global_hooks = comfy.hooks.create_hook_lora({}, 0.7, 0.0)
regional_hooks = comfy.hooks.create_hook_lora({}, 0.9, 0.0)
global_hooks.get_type(comfy.hooks.EnumHookType.Weight)[0].hook_ref = "global"
regional_hooks.get_type(comfy.hooks.EnumHookType.Weight)[0].hook_ref = "regional"
assembled, _ = RegionalConditioningService().assemble(
positive=ConditioningBatch(
(
@@ -166,6 +168,12 @@ def test_native_sampler_activates_hooks_from_masked_regional_conditioning(
regional_prompt_weight=0.5,
region_mask_feather=0,
)
combined_hooks = assembled[1][1]["hooks"]
assert isinstance(combined_hooks, comfy.hooks.HookGroup)
assert [
hook.hook_ref
for hook in combined_hooks.get_type(comfy.hooks.EnumHookType.Weight)
] == ["global", "regional"]
converted = comfy.sampler_helpers.convert_cond(assembled)
for conditioning in converted:
cross_attn = conditioning.pop("cross_attn")
@@ -187,9 +195,9 @@ def test_native_sampler_activates_hooks_from_masked_regional_conditioning(
)
assert len(outputs) == 1
assert set(model.current_patcher.prepared) == {global_hooks, regional_hooks}
assert set(model.current_patcher.applied) == {global_hooks, regional_hooks}
assert set(model.model_calls) == {global_hooks, regional_hooks}
assert set(model.current_patcher.prepared) == {global_hooks, combined_hooks}
assert set(model.current_patcher.applied) == {global_hooks, combined_hooks}
assert set(model.model_calls) == {global_hooks, combined_hooks}
def _conditioning(
+61
View File
@@ -0,0 +1,61 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Verify automatic NegPiP trigger detection."""
from __future__ import annotations
import pytest
from simple_syrup.domain.negative_prompt_weights import (
contains_negative_prompt_weight,
)
@pytest.mark.parametrize(
"text",
[
"(1girl:-2.00)",
"portrait, (red jacket: -1.5)",
"((nested):-0.25)",
"[plain:(scheduled concept:-1.0):0.5]",
"outer ((inner):-3.0)",
],
)
def test_detector_admits_effective_negative_prompt_weights(text: str) -> None:
"""Recognize valid negative emphasis wherever Prompt Control may schedule it."""
assert contains_negative_prompt_weight(text) is True
@pytest.mark.parametrize(
"text",
[
"1girl:-2.00",
"(1girl:2.00)",
"(1girl)",
"(1girl:not-a-number)",
r"escaped \(1girl:-2.0\)",
"unfinished (1girl:-2.0",
"STYLE(A1111, length)",
"",
],
)
def test_detector_rejects_non_negative_weight_syntax(text: str) -> None:
"""Do not activate for plain text, positive weights, escapes, or malformed input."""
assert contains_negative_prompt_weight(text) is False
def test_detector_resolves_nested_effective_weight() -> None:
"""An inner explicit positive weight overrides a negative outer emphasis."""
assert contains_negative_prompt_weight("((kept positive:2.0):-3.0)") is False
def test_detector_requires_text() -> None:
"""Reject dynamic non-text values before prompt planning."""
with pytest.raises(TypeError, match="requires text"):
contains_negative_prompt_weight(object()) # type: ignore[arg-type]
+188
View File
@@ -0,0 +1,188 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Verify isolated automatic NegPiP proof workflow construction."""
from __future__ import annotations
from typing import cast
import pytest
from tools.negpip_integration.workflow import (
REFINER_BASE_PROMPT,
REFINER_SWITCH_STEP,
NegpipFixtureSelections,
NegpipLiveFamily,
NegpipLiveWorkflowBuilder,
)
@pytest.fixture
def builder() -> NegpipLiveWorkflowBuilder:
"""Return a builder with deterministic nested Comfy selections."""
return NegpipLiveWorkflowBuilder(
NegpipFixtureSelections(
sd1_checkpoint=r"proof\sd1.safetensors",
sdxl_checkpoint=r"proof\sdxl.safetensors",
sdxl_refiner_checkpoint=r"proof\sdxl-refiner.safetensors",
anima_diffusion=r"proof\anima.safetensors",
anima_text_encoder=r"proof\anima-te.safetensors",
krea2_diffusion=r"proof\krea2.safetensors",
krea2_text_encoder=r"proof\krea2-te.safetensors",
qwen_image_vae=r"proof\qwen-image-vae.safetensors",
)
)
@pytest.mark.parametrize("family", tuple(NegpipLiveFamily))
def test_triggered_workflow_samples_and_requires_runtime_evidence(
builder: NegpipLiveWorkflowBuilder,
family: NegpipLiveFamily,
) -> None:
"""Every family uses the public node and synchronized callback observer."""
built = builder.build(family, run_id=f"proof:{family.value}", trigger=True)
class_types = [str(node["class_type"]) for node in built.prompt.values()]
schedule = next(
node
for node in built.prompt.values()
if node["class_type"] == "SimpleSyrup.ScheduleAndEncodePromptsWithPromptControl"
)
inputs = cast(dict[str, object], schedule["inputs"])
assert "bright (red:-1.0) jacket" in str(inputs["positive_prompt"])
expected_sampler = (
"KSamplerAdvanced" if family is NegpipLiveFamily.SDXL_REFINER else "KSampler"
)
assert expected_sampler in class_types
assert "VAEDecode" in class_types
assert "SaveImage" in class_types
assert "SimpleSyrupBenchmark.InstrumentNegpipModel" in class_types
assert "SimpleSyrupBenchmark.ReadNegpipRuntime" in class_types
assert built.runtime_node_id is not None
assert built.conditioning_node_id is not None
@pytest.mark.parametrize("family", tuple(NegpipLiveFamily))
def test_control_workflow_proves_automatic_gate_stays_off(
builder: NegpipLiveWorkflowBuilder,
family: NegpipLiveFamily,
) -> None:
"""A no-negative-weight control saves a sampled unmodified image."""
built = builder.build(family, run_id=f"control:{family.value}", trigger=False)
class_types = [str(node["class_type"]) for node in built.prompt.values()]
assert "SimpleSyrupBenchmark.SnapshotModelModifier" in class_types
expected_sampler = (
"KSamplerAdvanced" if family is NegpipLiveFamily.SDXL_REFINER else "KSampler"
)
assert expected_sampler in class_types
assert "VAEDecode" in class_types
assert "SaveImage" in class_types
assert "SimpleSyrupBenchmark.InstrumentNegpipModel" not in class_types
assert built.runtime_node_id is None
assert built.conditioning_node_id is None
def test_family_workflows_select_native_loaders_and_latents(
builder: NegpipLiveWorkflowBuilder,
) -> None:
"""Use real family-specific loader and latent contracts."""
expected = {
NegpipLiveFamily.SD1: {"CheckpointLoaderSimple", "EmptyLatentImage"},
NegpipLiveFamily.SDXL: {"CheckpointLoaderSimple", "EmptyLatentImage"},
NegpipLiveFamily.SDXL_REFINER: {
"CheckpointLoaderSimple",
"EmptyLatentImage",
},
NegpipLiveFamily.ANIMA: {
"SimpleSyrup.SimpleLoadAnima",
"EmptyCosmosLatentVideo",
},
NegpipLiveFamily.KREA2: {
"UNETLoader",
"CLIPLoader",
"VAELoader",
"EmptySD3LatentImage",
},
}
for family, required in expected.items():
built = builder.build(family, run_id=family.value, trigger=True)
class_types = {str(node["class_type"]) for node in built.prompt.values()}
assert required.issubset(class_types)
def test_refiner_workflow_runs_base_then_refiner_sampling(
builder: NegpipLiveWorkflowBuilder,
) -> None:
"""Use SDXL base for high noise and the probed refiner for low noise."""
built = builder.build(
NegpipLiveFamily.SDXL_REFINER,
run_id="refiner",
trigger=True,
)
samplers = [
node
for node in built.prompt.values()
if node["class_type"] == "KSamplerAdvanced"
]
assert len(samplers) == 2
base_inputs = cast(dict[str, object], samplers[0]["inputs"])
refiner_inputs = cast(dict[str, object], samplers[1]["inputs"])
assert base_inputs["add_noise"] == "enable"
assert base_inputs["end_at_step"] == REFINER_SWITCH_STEP
assert base_inputs["return_with_leftover_noise"] == "enable"
assert refiner_inputs["add_noise"] == "disable"
assert refiner_inputs["start_at_step"] == REFINER_SWITCH_STEP
assert refiner_inputs["end_at_step"] == 24
base_positive = next(
node
for node in built.prompt.values()
if node["class_type"] == "CLIPTextEncode"
and cast(dict[str, object], node["inputs"])["text"] == REFINER_BASE_PROMPT
)
base_text = cast(dict[str, object], base_positive["inputs"])["text"]
assert isinstance(base_text, str)
assert "red" not in base_text
def test_ppm_baseline_prepatches_the_same_schedule_path(
builder: NegpipLiveWorkflowBuilder,
) -> None:
"""Put pinned PPM before Schedule & Encode as the behavioral oracle."""
built = builder.build(
NegpipLiveFamily.SD1,
run_id="ppm-baseline",
trigger=True,
baseline_ppm=True,
)
class_types = [str(node["class_type"]) for node in built.prompt.values()]
assert built.mode == "ppm_baseline"
assert class_types.count("CLIPNegPip") == 1
assert (
class_types.count("SimpleSyrup.ScheduleAndEncodePromptsWithPromptControl") == 1
)
def test_ppm_baseline_rejects_an_untriggered_workflow(
builder: NegpipLiveWorkflowBuilder,
) -> None:
"""Keep ordinary controls free from every NegPiP patch."""
with pytest.raises(ValueError, match="requires a negative weight"):
builder.build(
NegpipLiveFamily.SD1,
run_id="invalid",
trigger=False,
baseline_ppm=True,
)
+192
View File
@@ -0,0 +1,192 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Verify automatic NegPiP dispatch on real Comfy patcher objects."""
from __future__ import annotations
from collections.abc import Callable
from typing import Any, cast
import pytest
import torch
from comfy.model_base import SDXL, Anima, BaseModel, Krea2, SDXLRefiner
from comfy.model_patcher import ModelPatcher
from comfy.patcher_extension import WrappersMP
from comfy.sd import CLIP
from simple_syrup.runtime.negpip.anima import (
WRAPPER_KEY as ANIMA_WRAPPER_KEY,
)
from simple_syrup.runtime.negpip.anima import (
anima_attn2_negpip,
)
from simple_syrup.runtime.negpip.krea2 import (
CLIP_MARKER,
KREA_TOKEN_KEY,
Krea2NegpipTokenizer,
krea2_attn1_negpip,
)
from simple_syrup.runtime.negpip.krea2 import (
WRAPPER_KEY as KREA_WRAPPER_KEY,
)
from simple_syrup.runtime.negpip.standard import standard_attn2_negpip
from simple_syrup.services.negpip_model_service import (
MODEL_MARKER,
NegpipModelService,
)
class _Encoder(torch.nn.Module):
"""Expose the encoder method patched by the standard NegPiP path."""
def encode_token_weights(self, pairs: object) -> object:
"""Return the supplied placeholder pairs."""
return pairs
class _ClipRoot(torch.nn.Module):
"""Provide the model object structure used by supported CLIP families."""
def __init__(self, *, krea: bool = False) -> None:
"""Install either the standard or Krea encoder surface."""
super().__init__()
if krea:
setattr(self, KREA_TOKEN_KEY, _Encoder())
else:
self.clip_l = _Encoder()
def encode_token_weights(
self,
pairs: object,
*,
template_end: int = -1,
) -> tuple[object, None, dict[str, object]]:
"""Stand in for Krea's root shape-preserving encoder."""
del template_end
return pairs, None, {}
class _Tokenizer:
"""Represent the installed tokenizer retained by a Krea proxy."""
@pytest.mark.parametrize("model_class", (BaseModel, SDXL, SDXLRefiner))
def test_service_patches_every_standard_ppm_family(
model_class: type[BaseModel],
) -> None:
"""SD1, SDXL, and SDXL Refiner receive one cloned PPM-equivalent path."""
model = _model_patcher(model_class)
clip = _clip(krea=False)
prepared_model, prepared_clip = NegpipModelService().prepare(model, clip)
assert isinstance(prepared_model, ModelPatcher)
assert isinstance(prepared_clip, CLIP)
assert prepared_model is not model
assert prepared_clip is not clip
assert MODEL_MARKER not in model.model_options
assert MODEL_MARKER not in clip.patcher.model_options
assert prepared_model.model_options[MODEL_MARKER] is True
assert prepared_clip.patcher.model_options[MODEL_MARKER] is True
assert _attention_patch(prepared_model, "attn2_patch") is standard_attn2_negpip
assert "clip_l.encode_token_weights" in prepared_clip.patcher.object_patches
def test_service_patches_anima_with_mask_wrapper_and_attention() -> None:
"""Anima receives its extra condition, diffusion wrapper, and V patch."""
prepared_model, prepared_clip = NegpipModelService().prepare(
_model_patcher(Anima),
_clip(krea=False),
)
model = cast(ModelPatcher, prepared_model)
clip = cast(CLIP, prepared_clip)
assert model.model_options[MODEL_MARKER] is True
assert clip.patcher.model_options[MODEL_MARKER] is True
assert "extra_conds" in model.object_patches
assert _attention_patch(model, "attn2_patch") is anima_attn2_negpip
assert ANIMA_WRAPPER_KEY in model.wrappers[WrappersMP.DIFFUSION_MODEL]
def test_service_patches_krea_without_claiming_ppm_clip_encoding() -> None:
"""Krea uses its layered encoder proxy and joint attn1 value patch."""
source_clip = _clip(krea=True)
prepared_model, prepared_clip = NegpipModelService().prepare(
_model_patcher(Krea2),
source_clip,
)
model = cast(ModelPatcher, prepared_model)
clip = cast(CLIP, prepared_clip)
assert model.model_options[MODEL_MARKER] is True
assert MODEL_MARKER not in clip.patcher.model_options
assert clip.patcher.model_options[CLIP_MARKER] is True
assert isinstance(clip.tokenizer, Krea2NegpipTokenizer)
assert clip.tokenizer is not source_clip.tokenizer
assert "encode_token_weights" in clip.patcher.object_patches
assert "extra_conds" in model.object_patches
assert _attention_patch(model, "attn1_patch") is krea2_attn1_negpip
assert KREA_WRAPPER_KEY in model.wrappers[WrappersMP.DIFFUSION_MODEL]
def test_service_reuses_already_patched_pair_without_double_patching() -> None:
"""An existing PPM model marker makes automatic preparation idempotent."""
model = _model_patcher(BaseModel)
clip = _clip(krea=False)
model.model_options[MODEL_MARKER] = True
prepared_model, prepared_clip = NegpipModelService().prepare(model, clip)
assert prepared_model is model
assert prepared_clip is clip
prepared_model_typed = cast(ModelPatcher, prepared_model)
transformer_options = cast(
dict[str, object],
prepared_model_typed.model_options["transformer_options"],
)
assert "patches" not in transformer_options
def _model_patcher(model_class: type[BaseModel]) -> ModelPatcher:
"""Construct an unloaded family instance behind Comfy's real patcher."""
model = object.__new__(model_class)
torch.nn.Module.__init__(model)
device = torch.device("cpu")
return ModelPatcher(model, load_device=device, offload_device=device)
def _clip(*, krea: bool) -> CLIP:
"""Construct a cloneable unloaded CLIP around a real model patcher."""
clip = CLIP(no_init=True)
root = _ClipRoot(krea=krea)
device = torch.device("cpu")
clip.patcher = ModelPatcher(root, load_device=device, offload_device=device)
clip.cond_stage_model = root
clip.tokenizer = _Tokenizer()
clip.layer_idx = None
clip.tokenizer_options = {}
clip.use_clip_schedule = False
clip.apply_hooks_to_conds = None
return clip
def _attention_patch(model: ModelPatcher, key: str) -> Callable[..., Any]:
"""Return the single installed attention patch from model options."""
transformer_options = cast(
dict[str, object], model.model_options["transformer_options"]
)
patches = cast(dict[str, list[Callable[..., Any]]], transformer_options["patches"])
assert len(patches[key]) == 1
return patches[key][0]
+291
View File
@@ -0,0 +1,291 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Prove family-specific NegPiP tensor and wrapper invariants."""
from __future__ import annotations
from typing import Any, cast
import pytest
import torch
from simple_syrup.runtime.negpip.anima import (
CONDITION_MASK_KEY as ANIMA_CONDITION_MASK_KEY,
)
from simple_syrup.runtime.negpip.anima import (
TRANSFORMER_MASK_KEY as ANIMA_TRANSFORMER_MASK_KEY,
)
from simple_syrup.runtime.negpip.anima import (
anima_attn2_negpip,
anima_diffusion_negpip_wrapper,
anima_extra_conds_negpip_wrapper,
)
from simple_syrup.runtime.negpip.krea2 import (
CONDITION_MASK_KEY as KREA_CONDITION_MASK_KEY,
)
from simple_syrup.runtime.negpip.krea2 import (
ENCODER_MASK_KEY,
encode_krea2_token_weights_negpip,
krea2_attn1_negpip,
krea2_diffusion_negpip_wrapper,
krea2_extra_conds_negpip_wrapper,
)
from simple_syrup.runtime.negpip.krea2 import (
TRANSFORMER_MASK_KEY as KREA_TRANSFORMER_MASK_KEY,
)
from simple_syrup.runtime.negpip.standard import (
encode_token_weights_negpip,
standard_attn2_negpip,
)
class _StandardEncoder:
"""Produce deterministic token and empty-prompt embeddings."""
special_tokens: dict[str, int] = {}
def gen_empty_tokens(
self,
special_tokens: dict[str, int],
length: int,
) -> list[int]:
"""Return a fixed empty token row matching the requested length."""
del special_tokens
return [0] * length
def encode(self, sections: list[list[object]]) -> tuple[torch.Tensor, None]:
"""Map source tokens to scalar embeddings and empty tokens to one."""
rows = [
[[1.0 if token == 0 else float(cast(int, token))] for token in section]
for section in sections
]
return torch.tensor(rows), None
def test_standard_negpip_interleaves_magnitude_keys_and_signed_values() -> None:
"""Standard encoding doubles tokens and signs only the value positions."""
encoded, pooled = encode_token_weights_negpip(
cast(Any, _StandardEncoder()),
[[(3, -2.0), (5, 0.5)]],
)
assert pooled is None
assert isinstance(encoded, torch.Tensor)
assert encoded.flatten().tolist() == [5.0, -5.0, 3.0, 3.0]
query = torch.tensor([[[9.0], [8.0]]])
key = encoded.clone()
value = encoded.clone()
prepared_query, prepared_key, prepared_value = standard_attn2_negpip(
query,
key,
value,
{},
)
assert prepared_query is query
assert prepared_key.flatten().tolist() == [5.0, 3.0]
assert prepared_value.flatten().tolist() == [-5.0, 3.0]
def test_anima_negpip_preserves_magnitude_and_propagates_value_mask() -> None:
"""Anima moves signs through conditions and changes only attention values."""
observed_weights: list[torch.Tensor] = []
def base_extra_conds(**kwargs: object) -> dict[str, object]:
weights = kwargs["t5xxl_weights"]
assert isinstance(weights, torch.Tensor)
observed_weights.append(weights)
return {"base": "condition"}
wrapped = anima_extra_conds_negpip_wrapper(base_extra_conds)
output = wrapped(t5xxl_weights=torch.tensor([-2.0, 0.5, 1.0]))
assert torch.equal(observed_weights[0], torch.tensor([2.0, 0.5, 1.0]))
condition = output[ANIMA_CONDITION_MASK_KEY]
multiplier = cast(Any, condition).cond
assert multiplier.shape == (1, 512, 1)
assert multiplier[0, :3, 0].tolist() == [-1.0, 1.0, 1.0]
assert torch.all(multiplier[0, 3:, 0] == 1.0)
captured: dict[str, object] = {}
def executor(*args: object, **kwargs: object) -> str:
del args
captured.update(kwargs)
return "executed"
context = torch.zeros((1, 512, 4))
result = anima_diffusion_negpip_wrapper(
executor,
object(),
object(),
context,
transformer_options={"existing": True},
**{ANIMA_CONDITION_MASK_KEY: multiplier},
)
assert result == "executed"
options = cast(dict[str, object], captured["transformer_options"])
assert options["existing"] is True
assert torch.equal(
cast(torch.Tensor, options[ANIMA_TRANSFORMER_MASK_KEY]), multiplier
)
query = torch.ones((1, 1, 3, 1))
key = torch.full_like(query, 2.0)
value = torch.tensor([[[[3.0], [4.0], [5.0]]]])
attention = anima_attn2_negpip(
query,
key,
value,
extra_options={ANIMA_TRANSFORMER_MASK_KEY: multiplier[:, :3]},
)
assert attention["q"] is query
assert attention["k"] is key
assert cast(torch.Tensor, attention["v"]).flatten().tolist() == [
-3.0,
4.0,
5.0,
]
def test_krea2_negpip_preserves_shape_and_signs_only_text_values() -> None:
"""Krea retains layered encoding and leaves Q, K, and image V untouched."""
tokens: dict[str, list[list[tuple[object, ...]]]] = {
"qwen3vl_4b": [
[
(151644, 1.0),
(0, 1.0),
(198, 1.0),
(151644, 1.0),
(872, 1.0),
(198, 1.0),
(10, -2.0),
(11, 0.5),
]
]
}
observed: dict[str, object] = {}
def original(
prepared: dict[str, list[list[tuple[object, ...]]]],
*,
template_end: int,
) -> tuple[torch.Tensor, None, dict[str, object]]:
observed["tokens"] = prepared
observed["template_end"] = template_end
return torch.ones((1, 2, 30_720)), None, {"source": True}
conditioning, pooled, extra = encode_krea2_token_weights_negpip(
original,
tokens,
)
conditioning_tensor = cast(torch.Tensor, conditioning)
assert conditioning_tensor.shape == (1, 2, 30_720)
assert pooled is None
absolute = cast(
dict[str, list[list[tuple[object, ...]]]],
observed["tokens"],
)
assert [pair[1] for pair in absolute["qwen3vl_4b"][0][-2:]] == [2.0, 0.5]
metadata = cast(dict[str, object], extra)
multiplier = cast(torch.Tensor, metadata[ENCODER_MASK_KEY])
assert multiplier.flatten().tolist() == [-1.0, 1.0]
wrapped_extra = krea2_extra_conds_negpip_wrapper(lambda **kwargs: {})
processed = wrapped_extra(**{ENCODER_MASK_KEY: multiplier})
condition = processed[KREA_CONDITION_MASK_KEY]
processed_multiplier = cast(Any, condition).cond
captured: dict[str, object] = {}
def executor(*args: object, **kwargs: object) -> str:
del args
captured.update(kwargs)
return "executed"
assert (
krea2_diffusion_negpip_wrapper(
executor,
transformer_options={"img_slice": [2, 4]},
**{KREA_CONDITION_MASK_KEY: processed_multiplier},
)
== "executed"
)
options = cast(dict[str, Any], captured["transformer_options"])
assert options["img_slice"] == [2, 4]
positional_capture: dict[str, object] = {}
def positional_executor(*args: object, **kwargs: object) -> str:
positional_capture["args"] = args
positional_capture["kwargs"] = kwargs
return "positional"
positional_options = {"img_slice": [2, 4]}
assert (
krea2_diffusion_negpip_wrapper(
positional_executor,
object(),
object(),
object(),
None,
None,
positional_options,
**{KREA_CONDITION_MASK_KEY: processed_multiplier},
)
== "positional"
)
positional_args = cast(tuple[object, ...], positional_capture["args"])
prepared_positional = cast(dict[str, object], positional_args[5])
assert prepared_positional is not positional_options
assert prepared_positional[KREA_TRANSFORMER_MASK_KEY] is processed_multiplier
assert "transformer_options" not in cast(
dict[str, object], positional_capture["kwargs"]
)
query = torch.arange(8.0).reshape(1, 1, 4, 2)
key = query + 10.0
value = query + 20.0
attention = krea2_attn1_negpip(
query,
key,
value,
extra_options=options,
)
assert attention["q"] is query
assert attention["k"] is key
prepared_value = cast(torch.Tensor, attention["v"])
assert prepared_value[0, 0, 0].tolist() == [-20.0, -21.0]
assert prepared_value[0, 0, 1].tolist() == [22.0, 23.0]
assert torch.equal(prepared_value[:, :, 2:], value[:, :, 2:])
assert torch.equal(value, query + 20.0)
@pytest.mark.parametrize(
"image_slice",
(None, [3, 4]),
)
def test_krea2_negpip_rejects_unprovable_text_boundaries(
image_slice: object,
) -> None:
"""Krea fails closed when the model boundary cannot align to its sign mask."""
options: dict[str, object] = {KREA_TRANSFORMER_MASK_KEY: torch.ones((1, 2, 1))}
if image_slice is not None:
options["img_slice"] = image_slice
with pytest.raises(ValueError, match="boundary|does not match"):
krea2_attn1_negpip(
torch.ones((1, 1, 4, 1)),
torch.ones((1, 1, 4, 1)),
torch.ones((1, 1, 4, 1)),
extra_options=options,
)
+133
View File
@@ -0,0 +1,133 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Verify benchmark-only live NegPiP callback introspection."""
from __future__ import annotations
import torch
from simple_syrup.runtime.negpip.krea2 import TRANSFORMER_MASK_KEY
from tools.attention_coupling_benchmark.comfy_probe.negpip_runtime import (
InstrumentNegpipModelV3,
ReadNegpipRuntimeV3,
_NegpipProbeState,
_observe_masked,
_observe_standard,
_owned_negpip_callback,
)
def test_negpip_runtime_probe_schemas_are_stable() -> None:
"""Expose distinct instrument and synchronized evidence node IDs."""
instrument = InstrumentNegpipModelV3.define_schema()
reader = ReadNegpipRuntimeV3.define_schema()
assert instrument.node_id == "SimpleSyrupBenchmark.InstrumentNegpipModel"
assert reader.node_id == "SimpleSyrupBenchmark.ReadNegpipRuntime"
def test_standard_probe_validates_live_interleaved_selection() -> None:
"""Record one standard callback only after exact even/odd selection."""
state = _NegpipProbeState("standard", "attn2_patch", "callback")
query = torch.ones((1, 2, 1))
key = torch.tensor([[[1.0], [1.0], [2.0], [2.0]]])
value = torch.tensor([[[1.0], [-1.0], [2.0], [2.0]]])
_observe_standard(
state,
query,
key,
value,
(query, key[:, 0::2], value[:, 1::2]),
)
assert state.attention_calls == 1
assert state.negative_mask_calls == 1
assert state.input_value_shape == [1, 4, 1]
assert state.output_value_shape == [1, 2, 1]
assert state.negative_token_count == 1
assert state.negative_token_positions == [0]
assert state.negative_token_locations == [[0, 0]]
def test_standard_probe_finds_signed_tokens_outside_cfg_batch_zero() -> None:
"""Inspect every CFG row rather than assuming the positive prompt is first."""
state = _NegpipProbeState("standard", "attn2_patch", "callback")
query = torch.ones((2, 2, 1))
key = torch.tensor(
[
[[1.0], [1.0], [2.0], [2.0]],
[[1.0], [1.0], [2.0], [2.0]],
]
)
value = torch.tensor(
[
[[1.0], [1.0], [2.0], [2.0]],
[[1.0], [1.0], [2.0], [-2.0]],
]
)
_observe_standard(
state,
query,
key,
value,
(query, key[:, 0::2], value[:, 1::2]),
)
assert state.negative_mask_calls == 1
assert state.negative_token_count == 1
assert state.negative_token_positions == [1]
assert state.negative_token_locations == [[1, 1]]
def test_probe_admits_the_pinned_ppm_standard_callback_identity() -> None:
"""Recognize PPM even when Comfy prefixes its module with a Windows path."""
def sdxl_attn2_negpip() -> None:
"""Stand in for the identity-checked pinned PPM callback."""
sdxl_attn2_negpip.__module__ = "managed_comfyui_ppm.src.negpip.unet_negpip"
sdxl_attn2_negpip.__qualname__ = "sdxl_attn2_negpip"
patch_name, family, callback = _owned_negpip_callback(
{"attn2_patch": [sdxl_attn2_negpip]}
)
assert patch_name == "attn2_patch"
assert family == "standard"
assert callback is sdxl_attn2_negpip
def test_krea_probe_validates_text_only_value_signing() -> None:
"""Record Krea only when its image suffix remains exact."""
state = _NegpipProbeState("krea2", "attn1_patch", "callback")
query = torch.ones((1, 1, 4, 1))
key = torch.ones((1, 1, 4, 1)) * 2
value = torch.tensor([[[[3.0], [4.0], [5.0], [6.0]]]])
multiplier = torch.tensor([[[-1.0], [1.0]]])
output_value = value.clone()
output_value[:, :, :2] *= multiplier.unsqueeze(1)
_observe_masked(
state,
query,
key,
value,
{"q": query, "k": key, "v": output_value},
{TRANSFORMER_MASK_KEY: multiplier, "img_slice": [2, 4]},
)
assert state.attention_calls == 1
assert state.negative_mask_calls == 1
assert state.text_length == 2
assert state.mask_shape == [1, 2, 1]
assert state.negative_token_count == 1
assert state.negative_token_positions == [0]
assert state.negative_token_locations == [[0, 0]]
+93
View File
@@ -0,0 +1,93 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Verify decoded NegPiP proof artifacts and contact-sheet evidence."""
from __future__ import annotations
import io
from pathlib import Path
from PIL import Image
from tools.comfy_api import JsonObject
from tools.negpip_integration.visual_proof import NegpipVisualProofRecorder
from tools.negpip_integration.workflow import NegpipLiveFamily
def test_visual_recorder_persists_pairs_and_contact_sheet(tmp_path: Path) -> None:
"""Every family receives originals, pixel deltas, and visible labels."""
recorder = NegpipVisualProofRecorder(tmp_path)
families: JsonObject = {}
for index, family in enumerate(NegpipLiveFamily):
control = recorder.record(
family,
"control",
_png_bytes((20 + index, 40, 60)),
)
negative = recorder.record(
family,
"negative",
_png_bytes((120 + index, 40, 60)),
)
families[family.value] = {
"control": {"image": control},
"negative": {
"image": negative,
"runtime": {
"family": family.value,
"attention_calls": 3,
"negative_mask_calls": 3,
},
},
}
sheet = recorder.finalize(families)
assert (tmp_path / str(sheet["file"])).is_file()
assert sheet["width"] == 1346
assert sheet["height"] == 2528
for family in NegpipLiveFamily:
result = families[family.value]
assert isinstance(result, dict)
comparison = result["image_comparison"]
assert isinstance(comparison, dict)
assert comparison["changed_pixels"] == 512 * 512
assert comparison["mean_absolute_rgb_delta"] > 0
def test_visual_recorder_rejects_identical_pair(tmp_path: Path) -> None:
"""A decoded image must visibly change in every supported family."""
recorder = NegpipVisualProofRecorder(tmp_path)
image = _png_bytes((20, 40, 60))
families: JsonObject = {}
for family in NegpipLiveFamily:
families[family.value] = {
"control": {"image": recorder.record(family, "control", image)},
"negative": {
"image": recorder.record(family, "negative", image),
"runtime": {
"family": family.value,
"attention_calls": 1,
"negative_mask_calls": 1,
},
},
}
try:
recorder.finalize(families)
except ValueError as error:
assert "images are equal" in str(error)
else:
raise AssertionError("Identical NegPiP proof images must be rejected.")
def _png_bytes(color: tuple[int, int, int]) -> bytes:
"""Return one deterministic 512-square PNG."""
stream = io.BytesIO()
Image.new("RGB", (512, 512), color).save(stream, format="PNG")
return stream.getvalue()
+140
View File
@@ -0,0 +1,140 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Prove Triton remains an optional CUDA acceleration dependency."""
from __future__ import annotations
import subprocess
import sys
from pathlib import Path
import pytest
from simple_syrup.runtime.regional_lora.triton_runtime import (
TritonRuntimeResolver,
)
def test_node_registration_and_cpu_accumulation_do_not_import_triton() -> None:
"""Load public nodes and execute CPU accumulation with Triton blocked."""
script = """
import importlib.abc
import sys
from pathlib import Path
sys.path.insert(0, str(Path.cwd().parents[1]))
sys.argv = [sys.argv[0], "--cpu"]
import comfy.options
comfy.options.enable_args_parsing()
class BlockTriton(importlib.abc.MetaPathFinder):
def find_spec(self, fullname, path, target=None):
if fullname == "triton" or fullname.startswith("triton."):
raise ModuleNotFoundError("blocked optional Triton", name=fullname)
return None
sys.meta_path.insert(0, BlockTriton())
import torch
from simple_syrup.nodes_v3 import get_nodes
from simple_syrup.runtime.regional_lora import ordered_accumulation
assert get_nodes()
base = torch.tensor([1.0, 2.0])
result = ordered_accumulation.OrderedTensorAccumulator().accumulate(
base,
(torch.tensor([3.0, 4.0]), torch.tensor([5.0, 6.0])),
)
assert result.tolist() == [9.0, 12.0]
assert not any(name == "triton" or name.startswith("triton.") for name in sys.modules)
"""
completed = subprocess.run(
[sys.executable, "-c", script],
cwd=Path(__file__).resolve().parents[1],
capture_output=True,
text=True,
timeout=60,
check=False,
)
assert completed.returncode == 0, completed.stderr
def test_resolver_caches_one_missing_result_and_warns_once(
caplog: pytest.LogCaptureFixture,
) -> None:
"""Treat an absent Triton package as one observable optional miss."""
calls: list[str] = []
def missing_import(name: str) -> object:
calls.append(name)
raise ModuleNotFoundError("missing", name="triton")
resolver = TritonRuntimeResolver(import_module=missing_import)
with caplog.at_level("WARNING"):
assert resolver.resolve("fake.backend") is None
assert resolver.resolve("fake.backend") is None
assert calls == ["fake.backend"]
assert [record.message for record in caplog.records] == [
"Triton acceleration is unavailable; using the Torch execution path"
]
def test_resolver_exposes_broken_backend_import_with_original_cause() -> None:
"""Fail visibly when a present backend cannot initialize correctly."""
failure = RuntimeError("JIT initialization failed")
def broken_import(_name: str) -> object:
raise failure
resolver = TritonRuntimeResolver(import_module=broken_import)
with pytest.raises(RuntimeError, match="failed to initialize") as raised:
resolver.resolve("fake.backend")
assert raised.value.__cause__ is failure
def test_resolver_is_thread_safe_and_returns_one_cached_backend() -> None:
"""Publish exactly one imported backend across concurrent callers."""
from concurrent.futures import ThreadPoolExecutor
backend = object()
calls: list[str] = []
def import_backend(name: str) -> object:
calls.append(name)
return backend
resolver = TritonRuntimeResolver(import_module=import_backend)
with ThreadPoolExecutor(max_workers=8) as executor:
results = tuple(executor.map(resolver.resolve, ("fake.backend",) * 32))
assert all(result is backend for result in results)
assert calls == ["fake.backend"]
@pytest.mark.parametrize(
"missing_name",
["fake.backend", "unrelated_dependency"],
)
def test_resolver_does_not_hide_non_triton_module_failures(missing_name: str) -> None:
"""Reserve optional fallback exclusively for the Triton package family."""
def missing_import(_name: str) -> object:
raise ModuleNotFoundError("missing", name=missing_name)
resolver = TritonRuntimeResolver(
import_module=missing_import,
)
with pytest.raises(RuntimeError, match="failed to initialize"):
resolver.resolve("fake.backend")
+94
View File
@@ -0,0 +1,94 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Verify typed PPM NegPiP conditioning and call-local option adaptation."""
from __future__ import annotations
from types import SimpleNamespace
import comfy.conds
import pytest
import torch
from simple_syrup.runtime.ppm_negpip_interop import (
PpmNegpipInterop,
PpmNegpipSemantics,
)
def test_anima_adapter_extracts_exact_typed_value_multiplier() -> None:
"""Convert PPM's model condition to context-aligned execution state."""
interop = _anima_interop()
context = torch.zeros((1, 3, 4), dtype=torch.float16)
source = torch.tensor([[[1], [-1], [1]]], dtype=torch.int32)
multiplier = interop.extract_value_multiplier(
{"c_ppm_negpip_mask": comfy.conds.CONDRegular(source)},
context,
)
assert multiplier is not None
assert multiplier.dtype is context.dtype
assert multiplier.device == context.device
assert multiplier.tolist() == [[[1.0], [-1.0], [1.0]]]
def test_anima_adapter_uses_neutral_multiplier_when_condition_is_absent() -> None:
"""Represent an all-positive prompt without leaving branch state partial."""
context = torch.zeros((2, 3, 4))
multiplier = _anima_interop().extract_value_multiplier({}, context)
assert multiplier is not None
assert torch.equal(multiplier, torch.ones((2, 3, 1)))
@pytest.mark.parametrize(
"source",
[
torch.ones((1, 2, 1)),
torch.zeros((1, 3, 1)),
torch.ones((1, 3, 2)),
],
)
def test_anima_adapter_rejects_misaligned_or_nonbinary_masks(
source: torch.Tensor,
) -> None:
"""Fail closed before malformed PPM state reaches regional packing."""
with pytest.raises(ValueError):
_anima_interop().extract_value_multiplier(
{"c_ppm_negpip_mask": SimpleNamespace(cond=source)},
torch.zeros((1, 3, 4)),
)
def test_anima_adapter_publishes_mask_on_an_isolated_option_copy() -> None:
"""Keep the ordinary call options unchanged outside original cross-attention."""
old_mask = torch.ones((1, 2, 1))
packed = torch.tensor([[[1.0], [-1.0]], [[-1.0], [1.0]]])
source: dict[str, object] = {
"ppm_negpip_mask": old_mask,
"preserved": object(),
}
prepared = _anima_interop().prepare_anima_transformer_options(source, packed)
assert prepared is not source
assert prepared["preserved"] is source["preserved"]
assert prepared["ppm_negpip_mask"] is packed
assert source["ppm_negpip_mask"] is old_mask
def _anima_interop() -> PpmNegpipInterop:
"""Return one focused admitted Anima semantic adapter."""
return PpmNegpipInterop(
PpmNegpipSemantics.ANIMA_VALUE_MASK,
lambda *args, **kwargs: (args, kwargs),
)
@@ -26,6 +26,7 @@ from simple_syrup.runtime.comfy_conditioning_model_loader import (
from simple_syrup.runtime.comfy_conditioning_processing import (
ComfyRegionalConditioningProcessor,
)
from simple_syrup.runtime.ppm_negpip_interop import PpmNegpipInterop
from simple_syrup.runtime.regional_lora_conditioning_adapter import (
RegionalLoraConditioningAdapter,
)
@@ -135,6 +136,7 @@ def test_profiled_collaborators_preserve_arguments_results_and_stage_order(
noise: torch.Tensor,
device: torch.device,
context_validator: RegionalContextValidator,
negpip: PpmNegpipInterop | None = None,
) -> ProcessedRegionalAttentionPlan:
calls.append(
(
@@ -145,6 +147,7 @@ def test_profiled_collaborators_preserve_arguments_results_and_stage_order(
"noise": noise,
"device": device,
"context_validator": context_validator,
"negpip": negpip,
},
)
)
@@ -190,6 +193,7 @@ def test_profiled_collaborators_preserve_arguments_results_and_stage_order(
"noise": noise,
"device": device,
"context_validator": validator,
"negpip": None,
}
assert _stages(caplog) == [
"source_model_load",
@@ -60,6 +60,63 @@ def test_schedule_encode_graph_builds_single_conditioning_outputs(
]
@pytest.mark.parametrize(
("positive_prompt", "negative_prompt"),
[
("portrait of (1girl:-2.0)", "blur"),
("portrait", "(blur:-0.5)"),
("portrait [SEP] (hands:-1.2)", "blur"),
("portrait [0:(eyes:-1.5):0.5]", "blur"),
],
)
def test_schedule_encode_graph_injects_negpip_for_negative_weights(
monkeypatch: pytest.MonkeyPatch,
positive_prompt: str,
negative_prompt: str,
) -> None:
"""Any effective negative segment weight prepares MODEL and CLIP first."""
calls = _install_fake_prompt_control(monkeypatch)
output = PromptControlScheduleEncodeGraphBuilder().build(
model=["model", 0],
clip=["clip", 0],
positive_prompt=positive_prompt,
negative_prompt=negative_prompt,
)
assert output.expand is not None
preparation_nodes = [
node
for node in output.expand.values()
if node["class_type"] == "SimpleSyrup.ApplyAutomaticNegpip"
]
assert len(preparation_nodes) == 1
assert calls["encode"]
assert all(call["clip"] != ["clip", 0] for call in calls["encode"])
def test_schedule_encode_graph_does_not_inject_negpip_for_nonnegative_weights(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Ordinary and positive-weight prompts retain the existing graph path."""
_install_fake_prompt_control(monkeypatch)
output = PromptControlScheduleEncodeGraphBuilder().build(
model=["model", 0],
clip=["clip", 0],
positive_prompt="portrait of (1girl:2.0)",
negative_prompt="blur",
)
assert output.expand is not None
assert not any(
node["class_type"] == "SimpleSyrup.ApplyAutomaticNegpip"
for node in output.expand.values()
)
def test_schedule_encode_graph_packs_both_sides_to_matched_segment_counts(
monkeypatch: pytest.MonkeyPatch,
) -> None:
+78
View File
@@ -108,6 +108,56 @@ def test_batching_builds_canonical_regions_with_base_fallback() -> None:
]
def test_batching_aligns_value_multipliers_with_cfg_regions_and_fallbacks() -> None:
"""Keep each scheduled branch's value semantics in identical chunk order."""
source = _plan()
plan = ProcessedRegionalAttentionPlan(
ProcessedRegionalAttentionBranch(
_context_with_multiplier(source.positive.base_context, 1.0),
(
_context_with_multiplier(
source.positive.regional_contexts[0],
-1.0,
),
_context_with_multiplier(
source.positive.regional_contexts[1],
1.0,
),
),
),
ProcessedRegionalAttentionBranch(
_context_with_multiplier(source.negative.base_context, -1.0),
(
_context_with_multiplier(
source.negative.regional_contexts[0],
1.0,
),
),
),
source.mask_bank,
source.lora_plan,
)
aligned = REGIONAL_ATTENTION_BATCHING_SERVICE.align(
plan,
base_context=_runtime_base(plan, [1, 0], 2),
cond_or_uncond=[1, 0],
conditioning_uuids=_uuids_for_selectors(plan, [1, 0]),
sigma=0.5,
latent_batch_size=2,
)
assert aligned.base_value_multiplier is not None
assert aligned.base_value_multiplier[:, 0, 0].tolist() == [-1.0, -1.0, 1.0, 1.0]
region_zero = aligned.regions[0].entries[0].cross_attention_value_multiplier
region_one = aligned.regions[1].entries[0].cross_attention_value_multiplier
assert region_zero is not None
assert region_one is not None
assert region_zero[:, 0, 0].tolist() == [1.0, 1.0, -1.0, -1.0]
assert region_one[:, 0, 0].tolist() == [-1.0, -1.0, 1.0, 1.0]
def test_batching_preserves_all_regional_entries_and_per_sample_strengths() -> None:
"""Align simultaneous regional entries without collapsing their order."""
@@ -409,6 +459,34 @@ def _multi_entry_context(
)
def _context_with_multiplier(
context: ProcessedRegionalAttentionContext,
value: float,
) -> ProcessedRegionalAttentionContext:
"""Copy one context with a uniform sequence-aligned value multiplier."""
return ProcessedRegionalAttentionContext(
context.conditioning_index,
context.region_index,
tuple(
ProcessedRegionalAttentionEntry(
entry.entry_index,
entry.uuid,
entry.schedule,
entry.cross_attention,
entry.strength,
torch.full(
(*entry.cross_attention.shape[:2], 1),
value,
dtype=entry.cross_attention.dtype,
device=entry.cross_attention.device,
),
)
for entry in context.entries
),
)
def _entry(
entry_index: int,
tensor: torch.Tensor,
+78 -4
View File
@@ -8,6 +8,7 @@ from __future__ import annotations
import pytest
import torch
from comfy.hooks import EnumHookType, HookGroup, create_hook_lora
from simple_syrup.domain.conditioning_batch import ConditioningBatch
from simple_syrup.runtime.regional_conditioning_companion import (
@@ -141,10 +142,10 @@ def test_feathering_preserves_inputs_and_softens_regional_copy() -> None:
def test_mask_composition_preserves_segment_lora_hook_metadata() -> None:
"""Global and regional hook groups survive standard mask composition."""
"""Global hooks compose into every conventional regional model state."""
global_hooks = object()
regional_hooks = object()
global_hooks = _hooks("global")
regional_hooks = _hooks("regional")
positive = ConditioningBatch(
(
[["global", {"hooks": global_hooks, "other": "global metadata"}]],
@@ -162,10 +163,68 @@ def test_mask_composition_preserves_segment_lora_hook_metadata() -> None:
assert assembled[0][1]["hooks"] is global_hooks
assert assembled[0][1]["other"] == "global metadata"
assert assembled[1][1]["hooks"] is regional_hooks
combined = assembled[1][1]["hooks"]
assert isinstance(combined, HookGroup)
assert _hook_refs(combined) == ["global", "regional"]
assert assembled[1][1]["other"] == "region metadata"
def test_global_hooks_compose_with_regional_companion_and_both_cfg_sides() -> None:
"""Keep one global patch under every local and fallback regional prompt share."""
positive_global = _hooks("positive global")
positive_regional = _hooks("positive regional")
negative_global = _hooks("negative global")
negative_regional = _hooks("negative regional")
positive_region = attach_global_companion(
[["positive region", {"hooks": positive_regional}]],
[["positive fallback", {"hooks": positive_regional}]],
)
negative_region = attach_global_companion(
[["negative region", {"hooks": negative_regional}]],
[["negative fallback", {"hooks": negative_regional}]],
)
positive, negative = RegionalConditioningService().assemble(
positive=ConditioningBatch(
(
[["positive global", {"hooks": positive_global}]],
positive_region,
)
),
negative=ConditioningBatch(
(
[["negative global", {"hooks": negative_global}]],
negative_region,
)
),
masks=torch.ones((1, 2, 2)),
regional_prompt_weight=0.5,
region_mask_feather=0,
)
assert _hook_refs(positive[0][1]["hooks"]) == ["positive global"]
assert _hook_refs(positive[1][1]["hooks"]) == [
"positive global",
"positive regional",
]
assert _hook_refs(positive[2][1]["hooks"]) == [
"positive global",
"positive regional",
]
assert positive[1][1]["hooks"] is positive[2][1]["hooks"]
assert _hook_refs(negative[0][1]["hooks"]) == ["negative global"]
assert _hook_refs(negative[1][1]["hooks"]) == [
"negative global",
"negative regional",
]
assert _hook_refs(negative[2][1]["hooks"]) == [
"negative global",
"negative regional",
]
assert negative[1][1]["hooks"] is negative[2][1]["hooks"]
@pytest.mark.parametrize(
("regional_prompt_weight", "expected_sources", "expected_strengths"),
[
@@ -281,3 +340,18 @@ def test_invalid_regional_prompt_weight_fails_before_mask_processing(
regional_prompt_weight=weight,
region_mask_feather=0,
)
def _hooks(identity: str) -> HookGroup:
"""Return one recognizable model-active Prompt Control-style HookGroup."""
hooks = create_hook_lora({}, strength_model=1.0, strength_clip=0.0)
hooks.get_type(EnumHookType.Weight)[0].hook_ref = identity
return hooks
def _hook_refs(value: object) -> list[object]:
"""Return ordered WeightHook references from one asserted HookGroup."""
assert isinstance(value, HookGroup)
return [hook.hook_ref for hook in value.get_type(EnumHookType.Weight)]

Some files were not shown because too many files have changed in this diff Show More