Compare commits

..
15 Commits
Author SHA1 Message Date
Daisy 3c1030fa47 chore(release): 1.14.1 [skip ci]
## [1.14.1](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.14.0...v1.14.1) (2026-10-11)

### Bug Fixes

* **sampling:** prepare captured noise before native validation ([889962d](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/889962d3545f6c6e45038ab9f8ee401308f919e9))
2026-10-11 21:24:18 +00:00
Artificial Sweetener 889962d354 fix(sampling): prepare captured noise before native validation 2026-10-11 17:17:24 -04:00
Daisy a073d587bc chore(release): 1.14.0 [skip ci]
# [1.14.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.13.0...v1.14.0) (2026-10-11)

### Bug Fixes

* **sampling:** prepare captured noise for direct cube queues ([d4771e1](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/d4771e1f4767593aa2e679815d403c9366e63aa0))

### Features

* **latent:** add learned upscaling and captured noise reuse ([027efb8](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/027efb8eef05aba45fcab8d2bce5d4261689f926))
* **sampling:** preserve native trajectories for CD noise reuse ([f89b436](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/f89b436a09199ae443f6847c996d04938e3348ce))
2026-10-11 19:26:00 +00:00
Artificial Sweetener 3cb70074dc ci(release): install Triton for CPU host verification 2026-10-11 15:19:22 -04:00
Artificial Sweetener aaf8ce6d32 ci(release): use CPU wheels for CPU-only host checks 2026-10-11 15:12:46 -04:00
Artificial Sweetener 58e80b0c83 test(sampling): supply complete native CFG hook inputs 2026-10-11 15:12:46 -04:00
Artificial Sweetener ce0dfc133b test(typing): classify the host easycache import boundary 2026-10-11 14:48:24 -04:00
Artificial Sweetener d4771e1f47 fix(sampling): prepare captured noise for direct cube queues 2026-10-11 14:48:24 -04:00
Artificial Sweetener f89b436a09 feat(sampling): preserve native trajectories for CD noise reuse
Capture the nearest existing source model evaluation without inserting sampling steps, retain its actual noise coordinate, and recover residuals with stable native scaling.

Observe source and receiver states at ComfyUI's shared boundaries, discover compatible named/custom source nodes, and report graph node identities and selected samplers in failures. Admit the complete sampler catalog through CD and tiled routes with regression coverage for future additions.

Verify the isolated CD-only tree: 4322 Python tests passed, 48 skipped; 115 frontend tests passed; format, lint, strict typing, architecture, and test governance passed. Contextual Attention remains outside this commit.
2026-10-06 01:25:06 -04:00
Artificial Sweetener 027efb8eef feat(latent): add learned upscaling and captured noise reuse
Add provenance recovery, coordinated native sampler-state capture, frequency-completed noise reuse, and optional full-resolution inversion finishing. Support automatic SesquiLSR selection and native bislerp fallback for unknown latents.

Verify exported Anima and Krea CD workflows with zero VAE encodes. Finishing outputs reproduce prior images exactly; reuse-only outputs match independent references at the agreed capture coordinates. All 3813 Python tests and 115 frontend tests pass alongside required quality gates.
2026-10-04 00:14:54 -04:00
Daisy 05667f6517 chore(release): 1.13.0 [skip ci]
# [1.13.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.12.0...v1.13.0) (2026-10-02)

### Bug Fixes

* **negpip:** support Krea attention on ComfyUI 0.28 ([e4eabfe](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/e4eabfefd540f3a6066c29761cefb8f8b3c80f68))

### Features

* **sampling:** add noise inversion and composable sampler options ([be4bd9b](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/be4bd9bb162f6ed4252e0c4d4ef0f3efebe8c674))
* **sampling:** refine sampler options and inversion controls ([ab7cebc](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/ab7cebcebc4d5dfa95aa8d834456af1778fe56a6))
2026-10-02 17:16:58 +00:00
Artificial Sweetener ab7cebcebc feat(sampling): refine sampler options and inversion controls 2026-10-02 12:51:27 -04:00
Artificial Sweetener e4eabfefd5 fix(negpip): support Krea attention on ComfyUI 0.28 2026-09-30 21:48:07 -04:00
Artificial Sweetener bbe86060c6 chore(licensing): complete inversion source and test headers 2026-09-30 21:48:07 -04:00
Artificial Sweetener be4bd9bb16 feat(sampling): add noise inversion and composable sampler options 2026-09-30 20:50:48 -04:00
152 changed files with 10676 additions and 356 deletions
+2 -1
View File
@@ -92,10 +92,11 @@ jobs:
set -euo pipefail
git clone --depth 1 https://github.com/comfyanonymous/ComfyUI.git "$RUNNER_TEMP/ComfyUI"
rsync -a --exclude=".git" "$RUNNER_TEMP/ComfyUI/" "$GITHUB_WORKSPACE/../.."/
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cpu
pip install -r "$GITHUB_WORKSPACE/../../requirements.txt"
- name: Install Python dependencies
run: pip install -e . pytest pytest-xdist ruff mypy
run: pip install -e . pytest pytest-xdist ruff mypy triton
- name: Verify Python formatting
run: ruff format --check .
+33
View File
@@ -1,3 +1,36 @@
## [1.14.1](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.14.0...v1.14.1) (2026-10-11)
### Bug Fixes
* **sampling:** prepare captured noise before native validation ([889962d](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/889962d3545f6c6e45038ab9f8ee401308f919e9))
# [1.14.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.13.0...v1.14.0) (2026-10-11)
### Bug Fixes
* **sampling:** prepare captured noise for direct cube queues ([d4771e1](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/d4771e1f4767593aa2e679815d403c9366e63aa0))
### Features
* **latent:** add learned upscaling and captured noise reuse ([027efb8](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/027efb8eef05aba45fcab8d2bce5d4261689f926))
* **sampling:** preserve native trajectories for CD noise reuse ([f89b436](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/f89b436a09199ae443f6847c996d04938e3348ce))
# [1.13.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.12.0...v1.13.0) (2026-10-02)
### Bug Fixes
* **negpip:** support Krea attention on ComfyUI 0.28 ([e4eabfe](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/e4eabfefd540f3a6066c29761cefb8f8b3c80f68))
### Features
* **sampling:** add noise inversion and composable sampler options ([be4bd9b](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/be4bd9bb162f6ed4252e0c4d4ef0f3efebe8c674))
* **sampling:** refine sampler options and inversion controls ([ab7cebc](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/ab7cebcebc4d5dfa95aa8d834456af1778fe56a6))
# [1.12.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.11.1...v1.12.0) (2026-09-29)
+4
View File
@@ -30,6 +30,9 @@ from .simple_syrup.runtime.attention_region_prompt_handler import ( # noqa: E40
from .simple_syrup.runtime.comfy_safetensors_dtypes import ( # noqa: E402
register_comfy_safetensors_dtypes,
)
from .simple_syrup.runtime.noise_reuse_validation import ( # noqa: E402
register_noise_reuse_validation,
)
WEB_DIRECTORY = "./web/dist"
@@ -58,6 +61,7 @@ register_quant_cache_routes()
register_external_llm_routes()
register_mask_batch_preview_routes()
register_attention_region_prompt_handler()
register_noise_reuse_validation()
__all__ = [
"WEB_DIRECTORY",
+1 -2
View File
@@ -1,11 +1,10 @@
schema_version = 1
review_by = 2027-03-31
fingerprint = "sha256:15ead72c26b1b5911aac191d07f36e5d8fd22d583447c2fcb62353be4ad1cd47"
fingerprint = "sha256:840e212b0027c20ca1984ad69ef3ee024a46edfdefc36a53a3774677d3c7be77"
cohesive_paths = [
"simple_syrup/masking/prompt_segs_with_sam_service.py",
"simple_syrup/nodes/prompt_segs_with_sam.py",
"simple_syrup/nodes_v3/legacy_node_wrappers.py",
"simple_syrup/runtime/attention_region_affinity.py",
"simple_syrup/runtime/attention_region_capture.py",
"simple_syrup/runtime/attention_sampler_lineage.py",
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "simple-syrup-comfyui",
"version": "1.12.0",
"version": "1.14.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "simple-syrup-comfyui",
"version": "1.12.0",
"version": "1.14.1",
"license": "AGPL-3.0-or-later",
"devDependencies": {
"@eslint/js": "^9.39.1",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "simple-syrup-comfyui",
"version": "1.12.0",
"version": "1.14.1",
"private": true,
"license": "AGPL-3.0-or-later",
"type": "module",
+4 -2
View File
@@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"
[project]
name = "SimpleSyrup"
description = "Workflow-focused ComfyUI extensions for image generation."
version = "1.12.0"
version = "1.14.1"
license = "AGPL-3.0-or-later"
license-files = ["LICENSE"]
requires-python = ">=3.11"
@@ -38,6 +38,7 @@ extend-exclude = [
"simple_syrup/third_party/groundingdino_runtime",
"simple_syrup/third_party/res4lyf_runtime",
"simple_syrup/third_party/sam_hq_runtime",
"simple_syrup/third_party/sesqui_lsr",
]
[tool.ruff.lint]
@@ -62,10 +63,11 @@ exclude = [
"simple_syrup/third_party/groundingdino_runtime",
"simple_syrup/third_party/res4lyf_runtime",
"simple_syrup/third_party/sam_hq_runtime",
"simple_syrup/third_party/sesqui_lsr",
]
[[tool.mypy.overrides]]
module = ["comfy.*"]
module = ["comfy.*", "comfy_extras.nodes_easycache"]
ignore_missing_imports = true
[[tool.mypy.overrides]]
+1 -1
View File
@@ -6,6 +6,6 @@
from __future__ import annotations
__version__ = "1.12.0"
__version__ = "1.14.1"
__all__: list[str] = ["__version__"]
+32 -8
View File
@@ -27,16 +27,37 @@ class ContextualDiffusionControls:
global_weight: float
global_steps: int
global_decay: float
latent_tile_width: int | None = None
latent_tile_height: int | None = None
@property
def tile_width(self) -> int:
"""Use explicit local geometry or the convenience node's context size."""
return self.latent_tile_width or self.latent_context_size
@property
def tile_height(self) -> int:
"""Keep the global context independent of a rectangular local tile."""
return self.latent_tile_height or self.latent_context_size
def validate(self) -> None:
"""Reject controls that cannot produce a stable bounded context plan."""
if self.latent_context_size < 16:
raise ValueError("latent_context_size must be at least 16 latent pixels.")
if not 0 <= self.latent_context_overlap < self.latent_context_size:
for value in (self.latent_tile_width, self.latent_tile_height):
if value is not None and (type(value) is not int or value < 16):
raise ValueError(
"Local tile dimensions must be at least 16 latent pixels."
)
if (
not 0
<= self.latent_context_overlap
< min(self.tile_width, self.tile_height)
):
raise ValueError(
"latent_context_overlap must be non-negative and smaller than "
"latent_context_size."
"both local tile dimensions."
)
if self.latent_context_batch_size < 1:
raise ValueError("latent_context_batch_size must be at least 1.")
@@ -65,6 +86,7 @@ def build_contextual_diffusion_plan(
controls: ContextualDiffusionControls,
segs: NativeSegs | None,
region_masks: torch.Tensor | None = None,
segs_canvas: tuple[int, int] | None = None,
) -> ContextualDiffusionPlan:
"""Return a global context plus the regular or SEGS-guided context plan."""
@@ -89,27 +111,29 @@ def build_contextual_diffusion_plan(
segs=segs,
latent_width=latent_width,
latent_height=latent_height,
tile_width=controls.latent_context_size,
tile_height=controls.latent_context_size,
tile_width=controls.tile_width,
tile_height=controls.tile_height,
overlap=controls.latent_context_overlap,
tile_batch_size=controls.latent_context_batch_size,
segs_canvas=segs_canvas,
)
elif segs is not None:
tile_plan = build_segs_guided_tiled_diffusion_plan(
segs=segs,
latent_width=latent_width,
latent_height=latent_height,
tile_width=controls.latent_context_size,
tile_height=controls.latent_context_size,
tile_width=controls.tile_width,
tile_height=controls.tile_height,
overlap=controls.latent_context_overlap,
tile_batch_size=controls.latent_context_batch_size,
segs_canvas=segs_canvas,
)
else:
tile_plan = build_tiled_diffusion_plan(
latent_width=latent_width,
latent_height=latent_height,
tile_width=controls.latent_context_size,
tile_height=controls.latent_context_size,
tile_width=controls.tile_width,
tile_height=controls.tile_height,
overlap=controls.latent_context_overlap,
tile_batch_size=controls.latent_context_batch_size,
)
+89
View File
@@ -0,0 +1,89 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Integrate source-derived inversion states without ComfyUI dependencies."""
from __future__ import annotations
from collections.abc import Callable
from dataclasses import dataclass
import torch
from .noise_inversion import INVERSION_METHODS, InversionMethod
InversionVelocity = Callable[[torch.Tensor, torch.Tensor, int], torch.Tensor]
SpatialResize = Callable[[torch.Tensor, int, int], torch.Tensor]
@dataclass(slots=True)
class InversionSolverEvidence:
"""Count the actual denoiser evaluations performed by an integration stage."""
evaluations: int = 0
def integrate_inversion(
source: torch.Tensor,
sigmas: torch.Tensor,
evaluate: InversionVelocity,
*,
method: InversionMethod,
evidence: InversionSolverEvidence | None = None,
) -> torch.Tensor:
"""Advance a finite state over strictly increasing positive inversion sigmas."""
if method not in INVERSION_METHODS:
raise ValueError("Inversion method must be euler or heun.")
if sigmas.ndim != 1 or len(sigmas) < 2 or not bool(torch.isfinite(sigmas).all()):
raise ValueError("A finite one-dimensional inversion schedule is required.")
if not bool(torch.all(sigmas > 0)) or not bool(torch.all(sigmas[1:] > sigmas[:-1])):
raise ValueError("Inversion sigmas must be positive and increasing.")
if not source.is_floating_point() or not bool(torch.isfinite(source).all()):
raise ValueError("Inversion source must contain finite floating-point values.")
record = evidence if evidence is not None else InversionSolverEvidence()
state = source.clone()
def velocity(x: torch.Tensor, sigma: torch.Tensor, index: int) -> torch.Tensor:
"""Count every denoiser evaluation and reject corrupted predictions."""
record.evaluations += 1
value = evaluate(x, sigma, index)
if value.shape != x.shape or not bool(torch.isfinite(value).all()):
raise FloatingPointError("Invalid inversion velocity shape or values.")
return value
for index in range(len(sigmas) - 1):
current, following = sigmas[index], sigmas[index + 1]
delta = following - current
estimate = velocity(state, current, index)
if method == "heun":
corrected = velocity(state + delta * estimate, following, index)
estimate = (estimate + corrected) / 2
state = state + delta * estimate
if not bool(torch.isfinite(state).all()):
raise FloatingPointError(f"Non-finite inversion state at step {index}.")
return state
def lift_inversion_displacement(
full_source: torch.Tensor,
coarse_source: torch.Tensor,
coarse_endpoint: torch.Tensor,
*,
resize: SpatialResize,
) -> torch.Tensor:
"""Lift only the inferred change so existing full-size detail survives transfer."""
if coarse_source.shape != coarse_endpoint.shape:
raise ValueError("Coarse inversion source and endpoint shapes must match.")
if full_source.shape[:-2] != coarse_source.shape[:-2]:
raise ValueError(
"Inversion transfer must preserve batch and channel dimensions."
)
height, width = full_source.shape[-2:]
lifted = resize(coarse_endpoint - coarse_source, height, width)
if lifted.shape != full_source.shape:
raise ValueError("Inversion displacement resize produced an invalid shape.")
endpoint = full_source + lifted
if not bool(torch.isfinite(endpoint).all()):
raise FloatingPointError("Inversion transfer produced non-finite values.")
return endpoint
+58
View File
@@ -0,0 +1,58 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Define bounded learned scaling stages and verified format recommendations."""
from __future__ import annotations
import math
NATIVE_LATENT_METHODS = ("nearest-exact", "bilinear", "area", "bicubic", "bislerp")
FORMAT_FAMILIES = {
"SDXL": "sdxl",
"Flux": "flux",
"Flux2": "flux2",
"Wan21": "wan",
"Cosmos1CV8x8x8": "wan",
}
def upscale_target(shape: tuple[int, int], factor: float) -> tuple[int, int]:
"""Match Comfy's rounded latent dimensions while rejecting empty outputs."""
if not math.isfinite(factor) or not 0.01 <= factor <= 8:
raise ValueError("Latent scale factor must be between 0.01 and 8.")
target = (round(shape[0] * factor), round(shape[1] * factor))
if min(target) < 1:
raise ValueError("Latent scale factor produces an empty spatial dimension.")
return target
def learned_upscale_stages(
source: tuple[int, int], target: tuple[int, int]
) -> tuple[tuple[int, int], ...]:
"""Reach exact dimensions through stages within the learned model's 1–2x range."""
if min(*source, *target) < 2 or any(
new < old for old, new in zip(source, target, strict=True)
):
raise ValueError(
"Learned latent upscaling requires enlargement and "
"dimensions of at least 2 latent pixels."
)
current = source
result: list[tuple[int, int]] = []
while current != target:
current = (min(current[0] * 2, target[0]), min(current[1] * 2, target[1]))
result.append(current)
return tuple(result)
def recommended_family(latent_format: str) -> str:
"""Select by verified format identity instead of ambiguous channel count."""
family = FORMAT_FAMILIES.get(latent_format)
if family is None:
raise ValueError(
f"Auto latent upscaling has no compatible model for "
f"{latent_format}; select a native method or a compatible "
f"checkpoint."
)
return family
+68
View File
@@ -0,0 +1,68 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Define validated source-preserving noise inversion configuration."""
from __future__ import annotations
import math
from dataclasses import dataclass
from typing import Literal
InversionMethod = Literal["euler", "heun"]
INVERSION_METHODS: tuple[InversionMethod, ...] = ("euler", "heun")
@dataclass(frozen=True, slots=True)
class NoiseInversionOptions:
"""Configure reduced-resolution inversion and an optional full-size finish.
The transition is a fraction of the target inversion sigma, not the forward
denoise steps. A full-size inversion uses ``steps`` and needs no transfer.
"""
method: InversionMethod = "euler"
resolution_scale: float = 0.5
steps: int = 2
switch_fraction: float = 0.75
finishing_steps: int = 1
def __post_init__(self) -> None:
"""Reject invalid or internally incomplete inversion recipes."""
if self.method not in INVERSION_METHODS:
raise ValueError("Inversion method must be euler or heun.")
if (
not math.isfinite(self.resolution_scale)
or not 0 < self.resolution_scale <= 1
):
raise ValueError("Inversion resolution scale must be in (0, 1].")
if type(self.steps) is not int or not 1 <= self.steps <= 64:
raise ValueError("Inversion steps must be an integer between 1 and 64.")
if type(self.finishing_steps) is not int or not 0 <= self.finishing_steps <= 64:
raise ValueError("Inversion finishing steps must be between 0 and 64.")
if not math.isfinite(self.switch_fraction) or not 0 < self.switch_fraction <= 1:
raise ValueError("Inversion transition must be in (0, 1].")
if self.resolution_scale < 1 and self.finishing_steps > 0:
if self.switch_fraction == 1:
raise ValueError("A full-size finish requires a transition below 100%.")
@property
def coarse_target_fraction(self) -> float:
"""Reach the full target unless an enabled full-size stage follows transfer."""
return (
self.switch_fraction
if self.resolution_scale < 1 and self.finishing_steps
else 1.0
)
def coarse_shape(self, height: int, width: int) -> tuple[int, int]:
"""Preserve full dimensions or align reduced transformer grids to even sizes."""
if height < 1 or width < 1:
raise ValueError("Inversion source dimensions must be positive.")
if self.resolution_scale == 1:
return height, width
return (
max(2, round(height * self.resolution_scale / 2) * 2),
max(2, round(width * self.resolution_scale / 2) * 2),
)
+107
View File
@@ -0,0 +1,107 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Describe native trajectory captures and independent finishing controls."""
from __future__ import annotations
import math
from dataclasses import dataclass
import torch
from .noise_inversion import InversionMethod
PROVENANCE_KEY = "simple_syrup_sampling_provenance"
@dataclass(frozen=True, slots=True)
class NoiseReuseOptions:
"""Request source noise at the receiving schedule's starting coordinate."""
finishing_steps: int = 0
finishing_method: InversionMethod = "euler"
finishing_start_fraction: float = 0.75
def __post_init__(self) -> None:
"""Reject singular coordinates and unbounded integration requests."""
if (
type(self.finishing_steps) is not int
or not 0 <= self.finishing_steps <= 100
):
raise ValueError("Finishing steps must be between 0 and 100.")
if self.finishing_method not in ("euler", "heun"):
raise ValueError("Finishing method must be Euler or Heun.")
if (
not math.isfinite(self.finishing_start_fraction)
or not 0 < self.finishing_start_fraction <= 1
):
raise ValueError(
"Finishing start fraction must be greater than 0 and at most 1."
)
if self.finishing_steps and self.finishing_start_fraction == 1:
raise ValueError(
"Finishing start fraction must be below 1 when finishing is enabled."
)
def capture_coordinate(self, target: float) -> float:
"""Round the requested coordinate exactly as the native float32 schedule."""
fraction = self.finishing_start_fraction if self.finishing_steps else 1.0
return float(torch.tensor(target * fraction, dtype=torch.float32))
@dataclass(frozen=True, slots=True)
class CapturedSamplingState:
"""Associate a requested level with a real observed state and recovered noise."""
requested_coordinate: float
actual_coordinate: float
state: torch.Tensor
residual: torch.Tensor
def __post_init__(self) -> None:
"""Require finite positive coordinates and matching real tensor layouts."""
if not all(
math.isfinite(value) and value > 0
for value in (self.requested_coordinate, self.actual_coordinate)
):
raise ValueError("Saved noise levels must be finite and greater than zero.")
if self.state.shape != self.residual.shape:
raise ValueError(
"Saved noisy state and recovered noise must have one shape."
)
if not bool(torch.isfinite(self.state).all()) or not bool(
torch.isfinite(self.residual).all()
):
raise ValueError("The original sampler saved invalid noise values.")
@dataclass(frozen=True, slots=True)
class SamplingProvenance:
"""Carry the source identity and real model-space states beside external latents."""
latent_format: str
clean: torch.Tensor
captures: tuple[CapturedSamplingState, ...] = ()
schedule: torch.Tensor | None = None
graph_context: str = ""
def capture_at(self, coordinate: float) -> CapturedSamplingState:
"""Require an observation explicitly associated with this receiving request."""
for capture in self.captures:
if capture.requested_coordinate == coordinate:
return capture
raise ValueError(
f"The original sampling run did not save noise for the requested level "
f"{coordinate:.9g}. Queue the connected workflow again with Noise Reuse "
"Options enabled."
)
def state_at(self, coordinate: float) -> torch.Tensor:
"""Return the real noisy state associated with the requested level."""
return self.capture_at(coordinate).state
def residual_at(self, coordinate: float) -> torch.Tensor:
"""Return noise separated with the originating model's own parameterization."""
return self.capture_at(coordinate).residual
@@ -0,0 +1,57 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Project semantic regional-detailing ownership into each inversion resolution."""
from __future__ import annotations
import math
import torch.nn.functional as functional
from .regional_detailing import LatentBox, LatentRegion
def project_inversion_regions(
regions: tuple[LatentRegion, ...],
*,
source_width: int,
source_height: int,
target_width: int,
target_height: int,
) -> tuple[LatentRegion, ...]:
"""Preserve region identity and conditioning while scaling masks and bounds."""
if min(source_width, source_height, target_width, target_height) < 1:
raise ValueError("Regional inversion canvases must have positive dimensions.")
if (source_width, source_height) == (target_width, target_height):
return regions
projected: list[LatentRegion] = []
for region in regions:
box = region.latent_box
if tuple(region.latent_mask.shape) != (source_height, source_width):
raise ValueError("Regional inversion mask must match its canonical canvas.")
if not (
0 <= box.x < box.x + box.width <= source_width
and 0 <= box.y < box.y + box.height <= source_height
):
raise ValueError("Regional inversion bounds must remain inside the canvas.")
left = math.floor(box.x * target_width / source_width)
top = math.floor(box.y * target_height / source_height)
right = math.ceil((box.x + box.width) * target_width / source_width)
bottom = math.ceil((box.y + box.height) * target_height / source_height)
mask = functional.interpolate(
region.latent_mask[None, None].float(),
size=(target_height, target_width),
mode="nearest",
)[0, 0].to(region.latent_mask)
projected.append(
LatentRegion(
region.index,
region.label,
LatentBox(left, top, right - left, bottom - top),
mask,
region.positive,
)
)
return tuple(projected)
@@ -26,6 +26,7 @@ def build_region_constrained_tiled_diffusion_plan(
tile_height: int,
overlap: int,
tile_batch_size: int,
segs_canvas: tuple[int, int] | None = None,
) -> TiledDiffusionPlan:
"""Build tiles split wherever regional composition or optional SEGS change."""
@@ -37,7 +38,8 @@ def build_region_constrained_tiled_diffusion_plan(
ownership_masks = region_ownership
if segs is not None:
native_segs = coerce_segs(segs)
validate_segs_aspect_ratio(native_segs, latent_height, latent_width)
canvas_height, canvas_width = segs_canvas or (latent_height, latent_width)
validate_segs_aspect_ratio(native_segs, canvas_height, canvas_width)
semantic_ownership = segs_ownership_masks(
native_segs,
latent_height=latent_height,
+174
View File
@@ -0,0 +1,174 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Assemble immutable, order-independent sampler capability configuration."""
from __future__ import annotations
import math
from dataclasses import dataclass, replace
from typing import TypeAlias
from .noise_inversion import NoiseInversionOptions
from .noise_reuse import NoiseReuseOptions
from .regional_prompting import validate_regional_prompt_weight
from .tiled_diffusion import validate_tiled_diffusion_mode
@dataclass(frozen=True, slots=True)
class TilingOptions:
"""Own the single local tile layout and blending configuration."""
diffusion_mode: str = "multidiffusion"
width: int = 128
height: int = 128
overlap: int = 32
batch_size: int = 4
differential_diffusion: bool = False
def __post_init__(self) -> None:
"""Reject tile settings that cannot form a bounded prediction plan."""
validate_tiled_diffusion_mode(self.diffusion_mode)
for name, value in (("width", self.width), ("height", self.height)):
if type(value) is not int or not 16 <= value <= 512:
raise ValueError(
f"Tile {name} must be between 16 and 512 latent pixels."
)
if type(self.overlap) is not int or not 0 <= self.overlap < min(
self.width, self.height
):
raise ValueError(
"Tile overlap must be nonnegative and smaller than both dimensions."
)
if type(self.batch_size) is not int or self.batch_size < 1:
raise ValueError("Tile batch size must be a positive integer.")
if type(self.differential_diffusion) is not bool:
raise TypeError("Differential diffusion must be a boolean.")
@dataclass(frozen=True, slots=True)
class ContextualDiffusionOptions:
"""Own global context and square local sampling geometry independently of tiling."""
context_size: int = 96
global_weight: float = 1.0
global_steps: int = 1
global_decay: float = 0.5
diffusion_mode: str = "multidiffusion"
overlap: int = 32
batch_size: int = 4
differential_diffusion: bool = False
def __post_init__(self) -> None:
"""Reject invalid global schedules and square local sampling settings."""
if type(self.context_size) is not int or not 16 <= self.context_size <= 512:
raise ValueError("Context size must be between 16 and 512 latent pixels.")
if not math.isfinite(self.global_weight) or not 0 <= self.global_weight <= 2:
raise ValueError("Global context weight must be between 0 and 2.")
if type(self.global_steps) is not int or self.global_steps < 0:
raise ValueError("Global context steps must be a nonnegative integer.")
if not math.isfinite(self.global_decay) or not 0 <= self.global_decay <= 1:
raise ValueError("Global context decay must be between 0 and 1.")
self.local_tiling()
def local_tiling(self) -> TilingOptions:
"""Use context size for both dimensions of the sole local sampling plan."""
return TilingOptions(
diffusion_mode=self.diffusion_mode,
width=self.context_size,
height=self.context_size,
overlap=self.overlap,
batch_size=self.batch_size,
differential_diffusion=self.differential_diffusion,
)
@dataclass(frozen=True, slots=True)
class AttentionCouplingOptions:
"""Configure regional attention strength without binding masks or a model."""
regional_prompt_weight: float = 1.0
region_mask_feather: int = 0
def __post_init__(self) -> None:
"""Require valid regional attention strengths and feathering controls."""
validate_regional_prompt_weight(self.regional_prompt_weight)
if type(self.region_mask_feather) is not int or self.region_mask_feather < 0:
raise ValueError("Region mask feather must be a nonnegative integer.")
SamplerCapability: TypeAlias = (
TilingOptions
| ContextualDiffusionOptions
| NoiseInversionOptions
| AttentionCouplingOptions
| NoiseReuseOptions
)
@dataclass(frozen=True, slots=True)
class SamplerOptions:
"""Own one immutable setting per capability, independent of graph order."""
tiling: TilingOptions | None = None
contextual_diffusion: ContextualDiffusionOptions | None = None
noise_inversion: NoiseInversionOptions | None = None
attention_coupling: AttentionCouplingOptions | None = None
noise_reuse: NoiseReuseOptions | None = None
def __post_init__(self) -> None:
"""Reject malformed connection payloads at the typed configuration boundary."""
for name, expected in (
("tiling", TilingOptions),
("contextual_diffusion", ContextualDiffusionOptions),
("noise_inversion", NoiseInversionOptions),
("attention_coupling", AttentionCouplingOptions),
("noise_reuse", NoiseReuseOptions),
):
value = getattr(self, name)
if value is not None and not isinstance(value, expected):
raise TypeError(f"Sampler option {name} must be {expected.__name__}.")
if self.noise_reuse is not None and self.noise_inversion is not None:
raise ValueError(
"Noise Reuse Options cannot be combined with Noise "
"Inversion Options; use the reuse finishing controls."
)
def with_capability(self, capability: SamplerCapability) -> SamplerOptions:
"""Return a fresh configuration or reject an ambiguous duplicate feature."""
names: dict[type[object], str] = {
TilingOptions: "tiling",
ContextualDiffusionOptions: "contextual_diffusion",
NoiseInversionOptions: "noise_inversion",
AttentionCouplingOptions: "attention_coupling",
NoiseReuseOptions: "noise_reuse",
}
name = names.get(type(capability))
if name is None:
raise TypeError("Unsupported sampler capability configuration.")
if getattr(self, name) is not None:
raise ValueError(
f"Duplicate sampler capability: {name}. Bypass or remove one node."
)
if isinstance(capability, TilingOptions):
return replace(self, tiling=capability)
if isinstance(capability, ContextualDiffusionOptions):
return replace(self, contextual_diffusion=capability)
if isinstance(capability, NoiseInversionOptions):
return replace(self, noise_inversion=capability)
if isinstance(capability, NoiseReuseOptions):
return replace(self, noise_reuse=capability)
return replace(self, attention_coupling=capability)
def append_sampler_capability(
options: SamplerOptions | None, capability: SamplerCapability | None
) -> SamplerOptions:
"""Append a capability or pass through a disabled contribution after validation."""
if options is not None and not isinstance(options, SamplerOptions):
raise TypeError(
"Options input must be a SimpleSyrup sampler options connection."
)
current = options if options is not None else SamplerOptions()
return current if capability is None else current.with_capability(capability)
+5 -1
View File
@@ -22,16 +22,20 @@ def build_segs_guided_tiled_diffusion_plan(
tile_height: int,
overlap: int,
tile_batch_size: int,
segs_canvas: tuple[int, int] | None = None,
) -> TiledDiffusionPlan:
"""Build bounded sampling windows whose irregular cores follow supplied SEGS.
Every latent pixel receives exactly one ownership core. Each core is sampled
through a rectangular window, while its local blend mask retains the irregular
boundary and shares a feathered overlap with neighboring cores.
A reduced inversion stage validates proportions against its original canvas
because rounding the reduced dimensions can change their aspect ratio.
"""
native_segs = coerce_segs(segs)
validate_segs_aspect_ratio(native_segs, latent_height, latent_width)
canvas_height, canvas_width = segs_canvas or (latent_height, latent_width)
validate_segs_aspect_ratio(native_segs, canvas_height, canvas_width)
ownership_masks = segs_ownership_masks(
native_segs,
latent_height=latent_height,
+64
View File
@@ -0,0 +1,64 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Lift separated residuals in orthonormal spatial frequency coordinates."""
from __future__ import annotations
from functools import lru_cache
from typing import cast
import torch
@lru_cache(maxsize=16)
def dct_basis(length: int) -> torch.Tensor:
"""Build a CPU DCT-II basis without retaining accelerator allocations."""
if length < 1:
raise ValueError("DCT dimensions must be positive.")
frequency = torch.arange(length, dtype=torch.float64)[:, None]
position = torch.arange(length, dtype=torch.float64)[None, :] + 0.5
matrix = torch.cos(torch.pi * frequency * position / length) * (2 / length) ** 0.5
matrix[0] /= 2**0.5
return cast(torch.Tensor, matrix.float())
def spatial_dct(value: torch.Tensor, *, inverse: bool = False) -> torch.Tensor:
"""Transform spatial axes while preserving batch, channel and frame axes."""
result = value.float()
for axis in (-1, -2):
matrix = dct_basis(result.shape[axis]).to(result.device)
result = result.movedim(axis, -1)
result = result @ (matrix if inverse else matrix.T)
result = result.movedim(-1, axis)
return result
def lift_residual(residual: torch.Tensor, fresh: torch.Tensor) -> torch.Tensor:
"""Retain source coefficients and fill only newly introduced frequency bands."""
if (
residual.ndim not in (4, 5)
or residual.ndim != fresh.ndim
or residual.shape[:-2] != fresh.shape[:-2]
):
raise ValueError(
"Noise reuse requires matching batch, channel and frame dimensions."
)
if any(a > b for a, b in zip(residual.shape[-2:], fresh.shape[-2:], strict=True)):
raise ValueError(
"Noise reuse requires spatial enlargement or unchanged dimensions."
)
if not bool(torch.isfinite(residual).all()) or not bool(
torch.isfinite(fresh).all()
):
raise ValueError(
"Noise reuse requires finite residual and fresh noise tensors."
)
height, width = residual.shape[-2:]
old_spectrum = torch.zeros_like(fresh, dtype=torch.float32)
old_spectrum[..., :height, :width] = spatial_dct(residual)
new_spectrum = spatial_dct(fresh).clone()
new_spectrum[..., :height, :width] = 0
return spatial_dct(old_spectrum, inverse=True) + spatial_dct(
new_spectrum, inverse=True
)
@@ -10,6 +10,7 @@ from typing import Any, ClassVar
import torch
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.segs import coerce_segs_group
from ..nodes import tooltips
from ..nodes.detailer_input_adapters import (
@@ -215,6 +216,7 @@ class DetailSEGSAsRegions:
noise_mask_feather: object = 20,
tiled_encode: object = False,
tiled_decode: object = False,
noise_inversion: NoiseInversionOptions | None = None,
) -> tuple[object]:
"""Run regional detailing and return the detailed image."""
@@ -237,6 +239,7 @@ class DetailSEGSAsRegions:
strict=True,
):
result = service.detail(
noise_inversion=noise_inversion,
image=single_image,
segs=single_segs,
model=single_input(model, "model", list_mode, OPERATION),
@@ -10,6 +10,7 @@ from typing import Any, ClassVar
import torch
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.segs import coerce_segs_group
from ..domain.tiled_diffusion import TILED_DIFFUSION_MODES
from ..nodes import tooltips
@@ -263,6 +264,7 @@ class DetailSEGSByScaleFactorTiledDiffusion:
latent_tile_height: object = 128,
latent_tile_overlap: object = 16,
latent_tile_batch_size: object = 4,
noise_inversion: NoiseInversionOptions | None = None,
) -> tuple[object]:
"""Run tiled diffusion scale-factor detailing and return the image."""
@@ -275,6 +277,7 @@ class DetailSEGSByScaleFactorTiledDiffusion:
outputs: list[torch.Tensor] = []
for single_image, single_segs in zip(images, segs_group, strict=True):
result = service.detail(
noise_inversion=noise_inversion,
image=single_image,
segs=single_segs,
model=single_input(model, "model", list_mode, OPERATION),
+26
View File
@@ -14,13 +14,16 @@ def get_nodes() -> list[type[object]]:
from .all_prompt_attention_segs import AllPromptAttentionSEGSV3
from .attention_capture_model import AttentionCaptureModelV3
from .attention_coupling_options import AttentionCouplingOptionsV3
from .attention_masked_conditioning import AttentionMaskedConditioningV3
from .attention_region_mask import AttentionRegionMaskV3
from .batch_region_conditioning import BatchRegionConditioningV3
from .batch_segs import BatchSEGSV3
from .compose_regional_conditioning import ComposeRegionalConditioningV3
from .concept_attention_segs import ConceptAttentionSEGSV3
from .contextual_diffusion_options import ContextualDiffusionOptionsV3
from .external_llm_prompt import ExternalLLMPromptV3
from .ksampler import KSamplerV3
from .ksampler_attention_coupling import KSamplerAttentionCouplingV3
from .ksampler_contextual_attention_coupling import (
KSamplerContextualAttentionCouplingV3,
@@ -32,6 +35,7 @@ def get_nodes() -> list[type[object]]:
KSamplerTiledAttentionCouplingV3,
)
from .ksampler_tiled_diffusion import KSamplerTiledDiffusionV3
from .latent_from_image import LatentFromImageV3
from .legacy_node_wrappers import (
ConditioningBatchAppendV3,
ConditioningBatchStartV3,
@@ -62,21 +66,43 @@ def get_nodes() -> list[type[object]]:
from .load_image_list import LoadImageListV3
from .load_mask_batch import LoadMaskBatchV3
from .mask_to_segs import MaskToSEGSV3
from .noise_inversion_options import NoiseInversionOptionsV3
from .noise_reuse_options import NoiseReuseOptionsV3
from .scale_factor import ScaleFactorV3
from .seed_variation import SeedVariationV3
from .simple_load_checkpoint import SimpleLoadCheckpointV3
from .simple_load_flux import SimpleLoadFluxV3
from .simple_load_flux2 import SimpleLoadFlux2V3
from .simple_load_krea2 import SimpleLoadKrea2V3
from .simple_upscale_latent import SimpleUpscaleLatentV3
from .tag_segs_with_external_llm import TagSEGSWithExternalLLMV3
from .tag_segs_with_wd14 import TagSEGSWithWD14V3
from .tile_and_tag_segs import TileAndTagSEGSV3
from .tiling_options import TilingOptionsV3
from .trajectory_capture import (
AttachSamplingProvenanceV3,
NoiseCaptureRequestV3,
NoiseReuseErrorV3,
TrajectoryCaptureModelV3,
)
from .vae_decode_options import VAEDecodeOptionsV3
from .vae_encode_options import VAEEncodeOptionsV3
from .wd14_tagger_loader import WD14TaggerLoaderV3
nodes: list[type[object]] = [
AllPromptAttentionSEGSV3,
AttentionCouplingOptionsV3,
ContextualDiffusionOptionsV3,
NoiseInversionOptionsV3,
TilingOptionsV3,
KSamplerV3,
LatentFromImageV3,
SimpleUpscaleLatentV3,
NoiseReuseOptionsV3,
NoiseCaptureRequestV3,
TrajectoryCaptureModelV3,
AttachSamplingProvenanceV3,
NoiseReuseErrorV3,
AttentionCaptureModelV3,
AttentionMaskedConditioningV3,
AttentionRegionMaskV3,
@@ -0,0 +1,67 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Configure regional attention without applying MODEL patches in the options graph."""
from __future__ import annotations
from typing import Any
from ..domain.sampler_options import (
AttentionCouplingOptions,
SamplerOptions,
append_sampler_capability,
)
from .ksampler_schema import attention_coupling_ksampler_inputs
from .sampler_options_schema import (
COMFY_IO,
OptionsNodeBase,
options_input,
options_output,
)
class AttentionCouplingOptionsV3(OptionsNodeBase):
"""Configure the sampler's regional attention strength and mask feathering."""
@classmethod
def define_schema(cls) -> Any:
"""Expose strength and feathering without binding region payloads."""
controls = attention_coupling_ksampler_inputs(COMFY_IO)
return COMFY_IO.Schema(
node_id="SimpleSyrup.AttentionCouplingOptions",
display_name="Attention Coupling Options",
category="SimpleSyrup/Sampling/Options",
description=(
"Routes global-first sampler conditioning to ordered image "
"regions through regional attention and LoRA hooks."
),
inputs=[
options_input(COMFY_IO),
*[
control
for control in controls
if control.id in {"regional_prompt_weight", "region_mask_feather"}
],
],
outputs=[options_output(COMFY_IO)],
)
@classmethod
def execute(
cls,
regional_prompt_weight: float = 1.0,
region_mask_feather: int = 0,
options: SamplerOptions | None = None,
) -> tuple[SamplerOptions]:
"""Append regional attention controls while deferring model preparation."""
return (
append_sampler_capability(
options,
AttentionCouplingOptions(
regional_prompt_weight=regional_prompt_weight,
region_mask_feather=region_mask_feather,
),
),
)
@@ -0,0 +1,91 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Configure complete contextual sampling with one square local context plan."""
from __future__ import annotations
from typing import Any
from ..domain.sampler_options import (
ContextualDiffusionOptions,
SamplerOptions,
append_sampler_capability,
)
from .ksampler_schema import contextual_diffusion_inputs
from .sampler_options_schema import (
COMFY_IO,
OptionsNodeBase,
options_input,
options_output,
)
class ContextualDiffusionOptionsV3(OptionsNodeBase):
"""Schedule global scene authority over one local tile prediction."""
@classmethod
def define_schema(cls) -> Any:
"""Append local controls after existing widgets to preserve saved values."""
controls = {
control.id: control for control in contextual_diffusion_inputs(COMFY_IO)
}
return COMFY_IO.Schema(
node_id="SimpleSyrup.ContextualDiffusionOptions",
display_name="Contextual Diffusion Options",
category="SimpleSyrup/Sampling/Options",
description=(
"Samples local contexts with global scene guidance; "
"takes precedence over connected Tiling Options."
),
inputs=[
options_input(COMFY_IO),
controls["latent_context_size"],
controls["global_weight"],
controls["global_steps"],
controls["global_decay"],
controls["diffusion_mode"],
controls["latent_context_overlap"],
controls["latent_context_batch_size"],
COMFY_IO.Boolean.Input(
"differential_diffusion",
default=False,
tooltip=(
"Uses the noise mask to vary denoising strength spatially; "
"preserves existing model mask behavior."
),
),
],
outputs=[options_output(COMFY_IO)],
)
@classmethod
def execute(
cls,
latent_context_size: int = 96,
global_weight: float = 1.0,
global_steps: int = 1,
global_decay: float = 0.5,
options: SamplerOptions | None = None,
diffusion_mode: str = "multidiffusion",
latent_context_overlap: int = 32,
latent_context_batch_size: int = 4,
differential_diffusion: bool = False,
) -> tuple[SamplerOptions]:
"""Append complete context settings without inheriting a Tiling contribution."""
return (
append_sampler_capability(
options,
ContextualDiffusionOptions(
context_size=latent_context_size,
global_weight=global_weight,
global_steps=global_steps,
global_decay=global_decay,
diffusion_mode=diffusion_mode,
overlap=latent_context_overlap,
batch_size=latent_context_batch_size,
differential_diffusion=differential_diffusion,
),
),
)
+109
View File
@@ -0,0 +1,109 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Expose one native KSampler consuming a composable capability configuration."""
from __future__ import annotations
from typing import Any, ClassVar
from ..domain.sampler_options import SamplerOptions
from ..nodes import tooltips
from ..services.noise_reuse_graph_preparation_service import (
NoiseReuseGraphPreparationService,
)
from ..services.sampler_options_sampling_service import SamplerOptionsSamplingService
from .ksampler_schema import ksampler_inputs
from .sampler_options_schema import COMFY_IO, OptionsNodeBase, options_input
class KSamplerV3(OptionsNodeBase):
"""Execute tiling, context, inversion and attention through shared authorities."""
service_class: ClassVar[type[SamplerOptionsSamplingService]] = (
SamplerOptionsSamplingService
)
SUGARCUBES_PROMPT_PREPARATION_API_VERSION: ClassVar[int] = 1
@classmethod
def prepare_sugarcubes_prompt(cls, prompt: dict[str, dict[str, object]]) -> None:
"""Install source capture dependencies before Cube queue validation."""
NoiseReuseGraphPreparationService().prepare(prompt)
@classmethod
def define_schema(cls) -> Any:
"""Declare sampling controls, capabilities and optional spatial regions."""
return COMFY_IO.Schema(
node_id="SimpleSyrup.KSampler",
display_name="KSampler (SimpleSyrup)",
category="SimpleSyrup/Sampling",
description=(
"Samples latents with connected sampler options for tiling, "
"Contextual Diffusion, noise inversion and Attention Coupling."
),
inputs=[
*ksampler_inputs(COMFY_IO, steps_default=20, cfg_default=8.0),
options_input(COMFY_IO),
COMFY_IO.SEGS.Input(
"segs",
optional=True,
tooltip=(
"Guides local sampling regions when Tiling or Contextual "
"Diffusion options are connected; ignored otherwise."
),
),
COMFY_IO.Mask.Input(
"region_masks",
optional=True,
tooltip=(
"Ordered masks paired with global-first conditioning batches "
"when Attention Coupling options are connected; "
"ignored otherwise."
),
),
],
outputs=[
COMFY_IO.Latent.Output(
"latent", tooltip=tooltips.DENOISED_LATENT_OUTPUT
)
],
)
@classmethod
def execute(
cls,
model: Any,
seed: int,
steps: int,
cfg: float,
sampler_name: str,
scheduler: str,
positive: object,
negative: object | None = None,
latent_image: dict[str, Any] | None = None,
denoise: float = 1.0,
options: SamplerOptions | None = None,
segs: object | None = None,
region_masks: object | None = None,
) -> tuple[dict[str, Any]]:
"""Delegate sampling without mutating capability configuration."""
if latent_image is None:
raise TypeError("KSampler requires latent_image.")
return (
cls.service_class().sample(
model=model,
seed=seed,
steps=steps,
cfg=cfg,
sampler_name=sampler_name,
scheduler=scheduler,
positive=positive,
negative=negative,
latent_image=latent_image,
denoise=denoise,
options=options,
segs=segs,
region_masks=region_masks,
),
)
@@ -17,6 +17,7 @@ from .ksampler_schema import (
ATTENTION_COUPLING_REGIONAL_PROMPT_WEIGHT_DEFAULT,
attention_coupling_ksampler_inputs,
)
from .sampler_options_schema import inversion_from_controls, noise_inversion_inputs
if TYPE_CHECKING:
@@ -68,10 +69,13 @@ class KSamplerAttentionCouplingV3(_ComfyNodeBase):
"anima regional prompt",
"sdxl regional prompt",
],
inputs=attention_coupling_ksampler_inputs(
_comfy_io,
region_masks_optional=True,
),
inputs=[
*attention_coupling_ksampler_inputs(
_comfy_io,
region_masks_optional=True,
),
*noise_inversion_inputs(_comfy_io, convenience=True),
],
outputs=[
_comfy_io.Latent.Output(
"latent",
@@ -98,12 +102,24 @@ class KSamplerAttentionCouplingV3(_ComfyNodeBase):
ATTENTION_COUPLING_REGIONAL_PROMPT_WEIGHT_DEFAULT
),
region_mask_feather: int = 0,
inversion_method: str = "euler",
inversion_resolution_scale: float = 0.5,
inversion_steps: int = 2,
inversion_switch_fraction: float = 0.75,
inversion_finishing_steps: int = 1,
) -> tuple[dict[str, Any]]:
"""Delegate ordinary or regional sampling to the routing service."""
if latent_image is None:
raise TypeError("KSampler Attention Coupling requires latent_image.")
output = cls.sampling_service_class().sample(
noise_inversion=inversion_from_controls(
inversion_method=inversion_method,
inversion_resolution_scale=inversion_resolution_scale,
inversion_steps=inversion_steps,
inversion_switch_fraction=inversion_switch_fraction,
inversion_finishing_steps=inversion_finishing_steps,
),
model=model,
seed=seed,
steps=steps,
@@ -17,6 +17,7 @@ from .ksampler_schema import (
attention_coupling_ksampler_inputs,
contextual_diffusion_inputs,
)
from .sampler_options_schema import inversion_from_controls, noise_inversion_inputs
if TYPE_CHECKING:
@@ -75,6 +76,7 @@ class KSamplerContextualAttentionCouplingV3(_ComfyNodeBase):
optional=True,
tooltip=tooltips.CONTEXTUAL_DIFFUSION_SEGS,
),
*noise_inversion_inputs(_comfy_io, convenience=True),
],
outputs=[
_comfy_io.Latent.Output(
@@ -112,6 +114,11 @@ class KSamplerContextualAttentionCouplingV3(_ComfyNodeBase):
global_steps: int = 1,
global_decay: float = 0.5,
segs: object | None = None,
inversion_method: str = "euler",
inversion_resolution_scale: float = 0.5,
inversion_steps: int = 2,
inversion_switch_fraction: float = 0.75,
inversion_finishing_steps: int = 1,
) -> tuple[dict[str, Any], object]:
"""Delegate the complete request to the combined application service."""
@@ -145,5 +152,12 @@ class KSamplerContextualAttentionCouplingV3(_ComfyNodeBase):
global_steps=global_steps,
global_decay=global_decay,
segs=segs,
noise_inversion=inversion_from_controls(
inversion_method=inversion_method,
inversion_resolution_scale=inversion_resolution_scale,
inversion_steps=inversion_steps,
inversion_switch_fraction=inversion_switch_fraction,
inversion_finishing_steps=inversion_finishing_steps,
),
)
return result.latent, result.contexts
@@ -18,6 +18,7 @@ from .ksampler_schema import (
ksampler_inputs,
optional_regional_sampling_inputs,
)
from .sampler_options_schema import inversion_from_controls, noise_inversion_inputs
if TYPE_CHECKING:
@@ -67,6 +68,7 @@ class KSamplerContextualDiffusionV3(_ComfyNodeBase):
_comfy_io,
segs_tooltip=tooltips.CONTEXTUAL_DIFFUSION_SEGS,
),
*noise_inversion_inputs(_comfy_io, convenience=True),
],
outputs=[
_comfy_io.Latent.Output(
@@ -104,6 +106,11 @@ class KSamplerContextualDiffusionV3(_ComfyNodeBase):
region_masks: object | None = None,
regional_prompt_weight: float = 0.5,
region_mask_feather: int = 0,
inversion_method: str = "euler",
inversion_resolution_scale: float = 0.5,
inversion_steps: int = 2,
inversion_switch_fraction: float = 0.75,
inversion_finishing_steps: int = 1,
) -> tuple[dict[str, Any], object]:
"""Delegate Contextual Diffusion sampling to its application service."""
@@ -131,5 +138,12 @@ class KSamplerContextualDiffusionV3(_ComfyNodeBase):
region_masks=region_masks,
regional_prompt_weight=regional_prompt_weight,
region_mask_feather=region_mask_feather,
noise_inversion=inversion_from_controls(
inversion_method=inversion_method,
inversion_resolution_scale=inversion_resolution_scale,
inversion_steps=inversion_steps,
inversion_switch_fraction=inversion_switch_fraction,
inversion_finishing_steps=inversion_finishing_steps,
),
)
return result.latent, result.contexts
@@ -13,6 +13,7 @@ from ..nodes import tooltips
from ..services.ksampler_sampling_service import KSamplerSamplingService
from ..services.regional_conditioning_service import RegionalConditioningService
from .ksampler_schema import regional_ksampler_inputs
from .sampler_options_schema import inversion_from_controls, noise_inversion_inputs
if TYPE_CHECKING:
@@ -51,7 +52,10 @@ class KSamplerPromptByRegionV3(_ComfyNodeBase):
"mask-bound regional prompts."
),
search_aliases=["ksampler", "regional prompt", "masked prompt"],
inputs=regional_ksampler_inputs(_comfy_io),
inputs=[
*regional_ksampler_inputs(_comfy_io),
*noise_inversion_inputs(_comfy_io, convenience=True),
],
outputs=[
_comfy_io.Latent.Output(
"latent",
@@ -76,6 +80,11 @@ class KSamplerPromptByRegionV3(_ComfyNodeBase):
region_mask_feather: int = 0,
latent_image: dict[str, Any] | None = None,
denoise: float = 1.0,
inversion_method: str = "euler",
inversion_resolution_scale: float = 0.5,
inversion_steps: int = 2,
inversion_switch_fraction: float = 0.75,
inversion_finishing_steps: int = 1,
) -> tuple[dict[str, Any]]:
"""Assemble regional conditioning and sample the full latent."""
@@ -93,6 +102,13 @@ class KSamplerPromptByRegionV3(_ComfyNodeBase):
)
)
output = cls.sampling_service_class().sample(
noise_inversion=inversion_from_controls(
inversion_method=inversion_method,
inversion_resolution_scale=inversion_resolution_scale,
inversion_steps=inversion_steps,
inversion_switch_fraction=inversion_switch_fraction,
inversion_finishing_steps=inversion_finishing_steps,
),
model=model,
seed=seed,
steps=steps,
@@ -14,6 +14,7 @@ from ..nodes import tooltips
from ..services.regional_conditioning_service import RegionalConditioningService
from ..services.tiled_diffusion_sampling_service import TiledDiffusionSamplingService
from .ksampler_schema import regional_ksampler_inputs, tiled_diffusion_inputs
from .sampler_options_schema import inversion_from_controls, noise_inversion_inputs
if TYPE_CHECKING:
@@ -60,6 +61,7 @@ class KSamplerPromptByTiledRegionV3(_ComfyNodeBase):
inputs=[
*regional_ksampler_inputs(_comfy_io),
*tiled_diffusion_inputs(_comfy_io),
*noise_inversion_inputs(_comfy_io, convenience=True),
],
outputs=[
_comfy_io.Latent.Output(
@@ -90,6 +92,11 @@ class KSamplerPromptByTiledRegionV3(_ComfyNodeBase):
latent_tile_height: int = 128,
latent_tile_overlap: int = 16,
latent_tile_batch_size: int = 4,
inversion_method: str = "euler",
inversion_resolution_scale: float = 0.5,
inversion_steps: int = 2,
inversion_switch_fraction: float = 0.75,
inversion_finishing_steps: int = 1,
) -> tuple[dict[str, Any]]:
"""Assemble regional conditioning and sample overlapping latent tiles."""
@@ -107,6 +114,13 @@ class KSamplerPromptByTiledRegionV3(_ComfyNodeBase):
)
)
output = cls.sampling_service_class().sample(
noise_inversion=inversion_from_controls(
inversion_method=inversion_method,
inversion_resolution_scale=inversion_resolution_scale,
inversion_steps=inversion_steps,
inversion_switch_fraction=inversion_switch_fraction,
inversion_finishing_steps=inversion_finishing_steps,
),
diffusion_mode=diffusion_mode,
model=model,
seed=seed,
@@ -18,6 +18,7 @@ from .ksampler_schema import (
attention_coupling_ksampler_inputs,
tiled_diffusion_inputs,
)
from .sampler_options_schema import inversion_from_controls, noise_inversion_inputs
if TYPE_CHECKING:
@@ -78,6 +79,7 @@ class KSamplerTiledAttentionCouplingV3(_ComfyNodeBase):
region_masks_optional=True,
),
*tiled_diffusion_inputs(_comfy_io),
*noise_inversion_inputs(_comfy_io, convenience=True),
],
outputs=[
_comfy_io.Latent.Output(
@@ -105,6 +107,11 @@ class KSamplerTiledAttentionCouplingV3(_ComfyNodeBase):
latent_tile_height: int = 128,
latent_tile_overlap: int = 16,
latent_tile_batch_size: int = 4,
inversion_method: str = "euler",
inversion_resolution_scale: float = 0.5,
inversion_steps: int = 2,
inversion_switch_fraction: float = 0.75,
inversion_finishing_steps: int = 1,
region_masks: object | None = None,
regional_prompt_weight: float = (
ATTENTION_COUPLING_REGIONAL_PROMPT_WEIGHT_DEFAULT
@@ -116,6 +123,13 @@ class KSamplerTiledAttentionCouplingV3(_ComfyNodeBase):
if latent_image is None:
raise TypeError("KSampler Tiled Attention Coupling requires latent_image.")
output = cls.sampling_service_class().sample(
noise_inversion=inversion_from_controls(
inversion_method=inversion_method,
inversion_resolution_scale=inversion_resolution_scale,
inversion_steps=inversion_steps,
inversion_switch_fraction=inversion_switch_fraction,
inversion_finishing_steps=inversion_finishing_steps,
),
diffusion_mode=diffusion_mode,
model=model,
seed=seed,
@@ -16,6 +16,7 @@ from .ksampler_schema import (
optional_regional_sampling_inputs,
tiled_diffusion_inputs,
)
from .sampler_options_schema import inversion_from_controls, noise_inversion_inputs
if TYPE_CHECKING:
@@ -65,6 +66,7 @@ class KSamplerTiledDiffusionV3(_ComfyNodeBase):
"boundaries while preserving the configured overlap."
),
),
*noise_inversion_inputs(_comfy_io, convenience=True),
],
outputs=[
_comfy_io.Latent.Output(
@@ -96,6 +98,11 @@ class KSamplerTiledDiffusionV3(_ComfyNodeBase):
region_masks: object | None = None,
regional_prompt_weight: float = 0.5,
region_mask_feather: int = 0,
inversion_method: str = "euler",
inversion_resolution_scale: float = 0.5,
inversion_steps: int = 2,
inversion_switch_fraction: float = 0.75,
inversion_finishing_steps: int = 1,
) -> tuple[dict[str, Any]]:
"""Delegate tiled diffusion sampling to its application service."""
@@ -122,5 +129,12 @@ class KSamplerTiledDiffusionV3(_ComfyNodeBase):
region_masks=region_masks,
regional_prompt_weight=regional_prompt_weight,
region_mask_feather=region_mask_feather,
noise_inversion=inversion_from_controls(
inversion_method=inversion_method,
inversion_resolution_scale=inversion_resolution_scale,
inversion_steps=inversion_steps,
inversion_switch_fraction=inversion_switch_fraction,
inversion_finishing_steps=inversion_finishing_steps,
),
)
return (output,)
@@ -0,0 +1,57 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Expose original decoded-image latent recovery as a strict provenance bridge."""
from __future__ import annotations
from importlib import import_module
from typing import Any, ClassVar
from ..services.latent_provenance_service import LatentProvenanceService
from .sampler_options_schema import COMFY_IO, OptionsNodeBase
class LatentFromImageV3(OptionsNodeBase):
"""Return the original graph latent while keeping image pixels outside execution."""
hidden: ClassVar[Any]
@classmethod
def define_schema(cls) -> Any:
"""Declare the widget-free bridge with graph metadata supplied by Comfy."""
return COMFY_IO.Schema(
node_id="SimpleSyrup.LatentFromImage",
display_name="Latent From Image",
category="SimpleSyrup/Latent",
enable_expand=True,
description=(
"Recovers the original latent behind an unmodified "
"decoded image using graph provenance."
),
inputs=[
COMFY_IO.Image.Input(
"image",
raw_link=True,
tooltip=(
"Connect an unmodified VAE Decode image; the original "
"latent is recovered through graph provenance."
),
)
],
hidden=[COMFY_IO.Hidden.prompt],
outputs=[
COMFY_IO.Latent.Output(
"latent",
tooltip="Original sampler latent with its sampling provenance.",
)
],
)
@classmethod
def execute(cls, image: object) -> Any:
"""Expand to the original latent link without invoking a VAE encoder."""
link = LatentProvenanceService().recover_link(
cls.hidden.prompt, image, import_module("nodes").NODE_CLASS_MAPPINGS
)
return COMFY_IO.NodeOutput([link[0], link[1]], expand={})
@@ -0,0 +1,73 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Expose shared inversion controls on maintained implementation-backed samplers."""
from __future__ import annotations
from typing import Any
from ..nodes.detailer_input_adapters import (
float_input,
int_input,
str_input,
)
from .legacy_node_adapter import LegacyNodeV3Adapter
from .sampler_options_schema import (
COMFY_IO,
inversion_from_controls,
noise_inversion_inputs,
)
class LegacyInversionNodeV3Adapter(LegacyNodeV3Adapter):
"""Normalize direct or list-mode widgets into the shared inversion domain value."""
@classmethod
def define_schema(cls) -> Any:
"""Append the shared five inversion controls after the sampler inputs."""
schema = super().define_schema()
schema.inputs.extend(noise_inversion_inputs(COMFY_IO, convenience=True))
return schema
@classmethod
def execute(cls, **kwargs: object) -> Any:
"""Narrow inversion widgets before delegating normal implementation inputs."""
values = dict(kwargs)
list_mode = bool(getattr(cls.LEGACY_NODE_CLASS, "INPUT_IS_LIST", False))
operation = cls.DISPLAY_NAME
inversion = inversion_from_controls(
inversion_method=str_input(
values.pop("inversion_method", "euler"),
"inversion_method",
list_mode,
operation,
),
inversion_resolution_scale=float_input(
values.pop("inversion_resolution_scale", 0.5),
"inversion_resolution_scale",
list_mode,
operation,
),
inversion_steps=int_input(
values.pop("inversion_steps", 2),
"inversion_steps",
list_mode,
operation,
),
inversion_switch_fraction=float_input(
values.pop("inversion_switch_fraction", 0.75),
"inversion_switch_fraction",
list_mode,
operation,
),
inversion_finishing_steps=int_input(
values.pop("inversion_finishing_steps", 1),
"inversion_finishing_steps",
list_mode,
operation,
),
)
values["noise_inversion"] = inversion
return super().execute(**values)
@@ -0,0 +1,275 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Translate maintained implementation contracts into native Comfy v3 schemas."""
from __future__ import annotations
from collections.abc import Mapping
from importlib import import_module
from typing import TYPE_CHECKING, Any, ClassVar
if TYPE_CHECKING:
class _ComfyNodeBase:
"""Type-checking base for Comfy v3 nodes."""
hidden: ClassVar[Any]
RETURN_TYPES: ClassVar[list[str]]
RETURN_NAMES: ClassVar[list[str]]
else:
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
_HIDDEN_INPUTS = {
"PROMPT": "prompt",
"DYNPROMPT": "dynprompt",
"EXTRA_PNGINFO": "extra_pnginfo",
"UNIQUE_ID": "unique_id",
}
class LegacyNodeV3Adapter(_ComfyNodeBase):
"""Build a v3 schema and execution bridge for a legacy implementation class."""
LEGACY_NODE_CLASS: ClassVar[type[Any]]
NODE_ID: ClassVar[str]
DISPLAY_NAME: ClassVar[str]
ENABLE_EXPAND: ClassVar[bool] = False
WORKFLOW_INPUT_ORDER: ClassVar[tuple[str, ...] | None] = None
@classmethod
def define_schema(cls) -> Any:
"""Declare a v3 schema from the implementation class contract."""
legacy = cls.LEGACY_NODE_CLASS
return _comfy_io.Schema(
node_id=cls.NODE_ID,
display_name=cls.DISPLAY_NAME,
category=str(getattr(legacy, "CATEGORY", "SimpleSyrup")),
description=str(getattr(legacy, "DESCRIPTION", "")),
search_aliases=list(getattr(legacy, "SEARCH_ALIASES", [])),
inputs=_v3_inputs(
legacy.INPUT_TYPES(),
workflow_order=cls.WORKFLOW_INPUT_ORDER,
),
outputs=_v3_outputs(legacy),
hidden=_v3_hidden_inputs(legacy.INPUT_TYPES()),
is_input_list=bool(getattr(legacy, "INPUT_IS_LIST", False)),
is_output_node=bool(getattr(legacy, "OUTPUT_NODE", False)),
enable_expand=cls.ENABLE_EXPAND,
)
@classmethod
def execute(cls, **kwargs: object) -> Any:
"""Run the wrapped implementation with v3-provided inputs."""
values = dict(kwargs)
for name, hidden_attr in _legacy_hidden_inputs(
cls.LEGACY_NODE_CLASS.INPUT_TYPES()
).items():
if name not in values:
values[name] = getattr(cls.hidden, hidden_attr)
function_name = str(cls.LEGACY_NODE_CLASS.FUNCTION)
implementation = cls.LEGACY_NODE_CLASS()
function = getattr(implementation, function_name)
return function(**values)
def _v3_inputs(
input_types: Mapping[str, Mapping[str, object]],
*,
workflow_order: tuple[str, ...] | None = None,
) -> list[Any]:
"""Return v3 inputs while preserving any explicit persisted socket order."""
declarations: dict[str, tuple[object, bool]] = {}
for section_name, optional in (("required", False), ("optional", True)):
section = input_types.get(section_name, {})
for name, declaration in section.items():
if name in declarations:
raise ValueError(f"legacy input {name} is declared more than once.")
declarations[name] = (declaration, optional)
order = tuple(declarations) if workflow_order is None else workflow_order
if len(order) != len(set(order)) or set(order) != set(declarations):
raise ValueError("legacy workflow input order must name every input once.")
return [
_v3_input(name, declarations[name][0], optional=declarations[name][1])
for name in order
]
def _v3_input(name: str, declaration: object, *, optional: bool) -> Any:
"""Return one v3 input declaration from a legacy field declaration."""
if not isinstance(declaration, tuple) or not declaration:
raise TypeError(f"legacy input {name} declaration must be a tuple.")
io_declaration = declaration[0]
options = _input_options(declaration)
tooltip = _string_option(options, "tooltip")
advanced = _bool_option(options, "advanced")
raw_link = _bool_option(options, "rawLink") or _bool_option(options, "raw_link")
force_input = _bool_option(options, "forceInput") or _bool_option(
options, "force_input"
)
if isinstance(io_declaration, (list, tuple)):
return _comfy_io.Combo.Input(
name,
options=list(io_declaration),
optional=optional,
default=options.get("default"),
control_after_generate=options.get("control_after_generate"),
tooltip=tooltip,
raw_link=raw_link,
advanced=advanced,
)
if not isinstance(io_declaration, str):
raise TypeError(f"legacy input {name} type must be a string or options list.")
input_type = io_declaration
input_class = _io_class(input_type)
common_options = {
"optional": optional,
"tooltip": tooltip,
"raw_link": raw_link,
"advanced": advanced,
}
if input_type == "INT":
return input_class.Input(
name,
default=options.get("default"),
min=options.get("min"),
max=options.get("max"),
step=options.get("step"),
control_after_generate=options.get("control_after_generate"),
**common_options,
)
if input_type == "FLOAT":
return input_class.Input(
name,
default=options.get("default"),
min=options.get("min"),
max=options.get("max"),
step=options.get("step"),
round=options.get("round"),
**common_options,
)
if input_type == "STRING":
return input_class.Input(
name,
default=options.get("default"),
multiline=bool(options.get("multiline", False)),
force_input=force_input,
**common_options,
)
if input_type == "BOOLEAN":
return input_class.Input(
name,
default=options.get("default"),
label_on=options.get("label_on"),
label_off=options.get("label_off"),
**common_options,
)
return input_class.Input(name, **common_options)
def _v3_outputs(legacy: type[Any]) -> list[Any]:
"""Return v3 output declarations from legacy return metadata."""
return_types = tuple(getattr(legacy, "RETURN_TYPES", ()))
return_names = getattr(legacy, "RETURN_NAMES", None)
output_tooltips = tuple(getattr(legacy, "OUTPUT_TOOLTIPS", ()))
output_is_list = tuple(
getattr(legacy, "OUTPUT_IS_LIST", (False,) * len(return_types))
)
outputs: list[Any] = []
for index, io_type in enumerate(return_types):
output_name = None
if isinstance(return_names, tuple) and index < len(return_names):
output_name = str(return_names[index])
tooltip = None
if index < len(output_tooltips):
tooltip = str(output_tooltips[index])
is_output_list = index < len(output_is_list) and bool(output_is_list[index])
outputs.append(
_io_class(str(io_type)).Output(
output_name,
tooltip=tooltip,
is_output_list=is_output_list,
)
)
return outputs
def _v3_hidden_inputs(input_types: Mapping[str, Mapping[str, object]]) -> list[Any]:
"""Return v3 hidden declarations requested by legacy hidden inputs."""
hidden_values = set(_legacy_hidden_inputs(input_types).values())
return [getattr(_comfy_io.Hidden, value) for value in sorted(hidden_values)]
def _legacy_hidden_inputs(
input_types: Mapping[str, Mapping[str, object]],
) -> dict[str, str]:
"""Return legacy hidden input names mapped to v3 hidden holder attributes."""
hidden_inputs: dict[str, str] = {}
for name, sentinel in input_types.get("hidden", {}).items():
if isinstance(sentinel, str) and sentinel in _HIDDEN_INPUTS:
hidden_inputs[name] = _HIDDEN_INPUTS[sentinel]
return hidden_inputs
def _io_class(io_type: str) -> Any:
"""Return the v3 IO class for a legacy Comfy type string."""
known_types = {
"BOOLEAN": _comfy_io.Boolean,
"INT": _comfy_io.Int,
"FLOAT": _comfy_io.Float,
"STRING": _comfy_io.String,
"IMAGE": _comfy_io.Image,
"MASK": _comfy_io.Mask,
"LATENT": _comfy_io.Latent,
"MODEL": _comfy_io.Model,
"CLIP": _comfy_io.Clip,
"VAE": _comfy_io.Vae,
"CONDITIONING": _comfy_io.Conditioning,
"SEGS": _comfy_io.SEGS,
}
return known_types.get(io_type, _comfy_io.Custom(io_type))
def _input_options(declaration: tuple[object, ...]) -> dict[str, object]:
"""Return an input options dictionary from a legacy declaration."""
if len(declaration) < 2 or not isinstance(declaration[1], dict):
return {}
return dict(declaration[1])
def _string_option(options: Mapping[str, object], name: str) -> str | None:
"""Return a string option when present."""
value = options.get(name)
if isinstance(value, str):
return value
return None
def _bool_option(options: Mapping[str, object], name: str) -> bool | None:
"""Return a boolean option when present."""
value = options.get(name)
if isinstance(value, bool):
return value
return None
+4 -270
View File
@@ -6,10 +6,6 @@
from __future__ import annotations
from collections.abc import Mapping
from importlib import import_module
from typing import TYPE_CHECKING, Any, ClassVar
from ..nodes.conditioning_batch_pack import (
ConditioningBatchAppend,
ConditioningBatchStart,
@@ -40,6 +36,8 @@ from ..nodes.segs_from_sam_output import SEGSFromSAMOutput
from ..nodes.simple_load_anima import SimpleLoadAnima
from ..nodes.simple_preview_segs import SimplePreviewSEGS
from ..nodes.vitmatte_model_loader import ViTMatteModelLoader
from .legacy_inversion_node_adapter import LegacyInversionNodeV3Adapter
from .legacy_node_adapter import LegacyNodeV3Adapter
from .legacy_workflow_input_order import (
DETAIL_SEGS_AS_REGIONS_INPUT_ORDER,
DETAIL_SEGS_BY_SCALE_FACTOR_INPUT_ORDER,
@@ -47,75 +45,6 @@ from .legacy_workflow_input_order import (
KSAMPLER_EXTRAS_INPUT_ORDER,
)
if TYPE_CHECKING:
class _ComfyNodeBase:
"""Type-checking base for Comfy v3 nodes."""
hidden: ClassVar[Any]
RETURN_TYPES: ClassVar[list[str]]
RETURN_NAMES: ClassVar[list[str]]
else:
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
_HIDDEN_INPUTS = {
"PROMPT": "prompt",
"DYNPROMPT": "dynprompt",
"EXTRA_PNGINFO": "extra_pnginfo",
"UNIQUE_ID": "unique_id",
}
class LegacyNodeV3Adapter(_ComfyNodeBase):
"""Build a v3 schema and execution bridge for a legacy implementation class."""
LEGACY_NODE_CLASS: ClassVar[type[Any]]
NODE_ID: ClassVar[str]
DISPLAY_NAME: ClassVar[str]
ENABLE_EXPAND: ClassVar[bool] = False
WORKFLOW_INPUT_ORDER: ClassVar[tuple[str, ...] | None] = None
@classmethod
def define_schema(cls) -> Any:
"""Declare a v3 schema from the implementation class contract."""
legacy = cls.LEGACY_NODE_CLASS
return _comfy_io.Schema(
node_id=cls.NODE_ID,
display_name=cls.DISPLAY_NAME,
category=str(getattr(legacy, "CATEGORY", "SimpleSyrup")),
description=str(getattr(legacy, "DESCRIPTION", "")),
search_aliases=list(getattr(legacy, "SEARCH_ALIASES", [])),
inputs=_v3_inputs(
legacy.INPUT_TYPES(),
workflow_order=cls.WORKFLOW_INPUT_ORDER,
),
outputs=_v3_outputs(legacy),
hidden=_v3_hidden_inputs(legacy.INPUT_TYPES()),
is_input_list=bool(getattr(legacy, "INPUT_IS_LIST", False)),
is_output_node=bool(getattr(legacy, "OUTPUT_NODE", False)),
enable_expand=cls.ENABLE_EXPAND,
)
@classmethod
def execute(cls, **kwargs: object) -> Any:
"""Run the wrapped implementation with v3-provided inputs."""
values = dict(kwargs)
for name, hidden_attr in _legacy_hidden_inputs(
cls.LEGACY_NODE_CLASS.INPUT_TYPES()
).items():
if name not in values:
values[name] = getattr(cls.hidden, hidden_attr)
function_name = str(cls.LEGACY_NODE_CLASS.FUNCTION)
implementation = cls.LEGACY_NODE_CLASS()
function = getattr(implementation, function_name)
return function(**values)
class ConditioningBatchStartV3(LegacyNodeV3Adapter):
"""Expose Conditioning Batch Start through Comfy v3 only."""
@@ -224,7 +153,7 @@ class ResizeImageToTargetV3(LegacyNodeV3Adapter):
DISPLAY_NAME = "Resize Image to Target"
class DetailSEGSAsRegionsV3(LegacyNodeV3Adapter):
class DetailSEGSAsRegionsV3(LegacyInversionNodeV3Adapter):
"""Expose Detail SEGS as Regions through Comfy v3 only."""
LEGACY_NODE_CLASS = DetailSEGSAsRegions
@@ -242,7 +171,7 @@ class DetailSEGSByScaleFactorV3(LegacyNodeV3Adapter):
WORKFLOW_INPUT_ORDER = DETAIL_SEGS_BY_SCALE_FACTOR_INPUT_ORDER
class DetailSEGSByScaleFactorTiledDiffusionV3(LegacyNodeV3Adapter):
class DetailSEGSByScaleFactorTiledDiffusionV3(LegacyInversionNodeV3Adapter):
"""Expose Detail SEGS by Scale Factor with Tiled Diffusion through Comfy v3."""
LEGACY_NODE_CLASS = DetailSEGSByScaleFactorTiledDiffusion
@@ -323,201 +252,6 @@ class ViTMatteModelLoaderV3(LegacyNodeV3Adapter):
DISPLAY_NAME = "ViTMatte Model Loader"
def _v3_inputs(
input_types: Mapping[str, Mapping[str, object]],
*,
workflow_order: tuple[str, ...] | None = None,
) -> list[Any]:
"""Return v3 inputs while preserving any explicit persisted socket order."""
declarations: dict[str, tuple[object, bool]] = {}
for section_name, optional in (("required", False), ("optional", True)):
section = input_types.get(section_name, {})
for name, declaration in section.items():
if name in declarations:
raise ValueError(f"legacy input {name} is declared more than once.")
declarations[name] = (declaration, optional)
order = tuple(declarations) if workflow_order is None else workflow_order
if len(order) != len(set(order)) or set(order) != set(declarations):
raise ValueError("legacy workflow input order must name every input once.")
return [
_v3_input(name, declarations[name][0], optional=declarations[name][1])
for name in order
]
def _v3_input(name: str, declaration: object, *, optional: bool) -> Any:
"""Return one v3 input declaration from a legacy field declaration."""
if not isinstance(declaration, tuple) or not declaration:
raise TypeError(f"legacy input {name} declaration must be a tuple.")
io_declaration = declaration[0]
options = _input_options(declaration)
tooltip = _string_option(options, "tooltip")
advanced = _bool_option(options, "advanced")
raw_link = _bool_option(options, "rawLink") or _bool_option(options, "raw_link")
force_input = _bool_option(options, "forceInput") or _bool_option(
options, "force_input"
)
if isinstance(io_declaration, (list, tuple)):
return _comfy_io.Combo.Input(
name,
options=list(io_declaration),
optional=optional,
default=options.get("default"),
control_after_generate=options.get("control_after_generate"),
tooltip=tooltip,
raw_link=raw_link,
advanced=advanced,
)
if not isinstance(io_declaration, str):
raise TypeError(f"legacy input {name} type must be a string or options list.")
input_type = io_declaration
input_class = _io_class(input_type)
common_options = {
"optional": optional,
"tooltip": tooltip,
"raw_link": raw_link,
"advanced": advanced,
}
if input_type == "INT":
return input_class.Input(
name,
default=options.get("default"),
min=options.get("min"),
max=options.get("max"),
step=options.get("step"),
control_after_generate=options.get("control_after_generate"),
**common_options,
)
if input_type == "FLOAT":
return input_class.Input(
name,
default=options.get("default"),
min=options.get("min"),
max=options.get("max"),
step=options.get("step"),
round=options.get("round"),
**common_options,
)
if input_type == "STRING":
return input_class.Input(
name,
default=options.get("default"),
multiline=bool(options.get("multiline", False)),
force_input=force_input,
**common_options,
)
if input_type == "BOOLEAN":
return input_class.Input(
name,
default=options.get("default"),
label_on=options.get("label_on"),
label_off=options.get("label_off"),
**common_options,
)
return input_class.Input(name, **common_options)
def _v3_outputs(legacy: type[Any]) -> list[Any]:
"""Return v3 output declarations from legacy return metadata."""
return_types = tuple(getattr(legacy, "RETURN_TYPES", ()))
return_names = getattr(legacy, "RETURN_NAMES", None)
output_tooltips = tuple(getattr(legacy, "OUTPUT_TOOLTIPS", ()))
output_is_list = tuple(
getattr(legacy, "OUTPUT_IS_LIST", (False,) * len(return_types))
)
outputs: list[Any] = []
for index, io_type in enumerate(return_types):
output_name = None
if isinstance(return_names, tuple) and index < len(return_names):
output_name = str(return_names[index])
tooltip = None
if index < len(output_tooltips):
tooltip = str(output_tooltips[index])
is_output_list = index < len(output_is_list) and bool(output_is_list[index])
outputs.append(
_io_class(str(io_type)).Output(
output_name,
tooltip=tooltip,
is_output_list=is_output_list,
)
)
return outputs
def _v3_hidden_inputs(input_types: Mapping[str, Mapping[str, object]]) -> list[Any]:
"""Return v3 hidden declarations requested by legacy hidden inputs."""
hidden_values = set(_legacy_hidden_inputs(input_types).values())
return [getattr(_comfy_io.Hidden, value) for value in sorted(hidden_values)]
def _legacy_hidden_inputs(
input_types: Mapping[str, Mapping[str, object]],
) -> dict[str, str]:
"""Return legacy hidden input names mapped to v3 hidden holder attributes."""
hidden_inputs: dict[str, str] = {}
for name, sentinel in input_types.get("hidden", {}).items():
if isinstance(sentinel, str) and sentinel in _HIDDEN_INPUTS:
hidden_inputs[name] = _HIDDEN_INPUTS[sentinel]
return hidden_inputs
def _io_class(io_type: str) -> Any:
"""Return the v3 IO class for a legacy Comfy type string."""
known_types = {
"BOOLEAN": _comfy_io.Boolean,
"INT": _comfy_io.Int,
"FLOAT": _comfy_io.Float,
"STRING": _comfy_io.String,
"IMAGE": _comfy_io.Image,
"MASK": _comfy_io.Mask,
"LATENT": _comfy_io.Latent,
"MODEL": _comfy_io.Model,
"CLIP": _comfy_io.Clip,
"VAE": _comfy_io.Vae,
"CONDITIONING": _comfy_io.Conditioning,
"SEGS": _comfy_io.SEGS,
}
return known_types.get(io_type, _comfy_io.Custom(io_type))
def _input_options(declaration: tuple[object, ...]) -> dict[str, object]:
"""Return an input options dictionary from a legacy declaration."""
if len(declaration) < 2 or not isinstance(declaration[1], dict):
return {}
return dict(declaration[1])
def _string_option(options: Mapping[str, object], name: str) -> str | None:
"""Return a string option when present."""
value = options.get(name)
if isinstance(value, str):
return value
return None
def _bool_option(options: Mapping[str, object], name: str) -> bool | None:
"""Return a boolean option when present."""
value = options.get(name)
if isinstance(value, bool):
return value
return None
__all__ = [
"ConditioningBatchAppendV3",
"ConditioningBatchStartV3",
@@ -0,0 +1,58 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Expose inversion resolution and integration controls as sampler options."""
from __future__ import annotations
from typing import Any
from ..domain.sampler_options import SamplerOptions, append_sampler_capability
from .sampler_options_schema import (
COMFY_IO,
OptionsNodeBase,
inversion_from_controls,
noise_inversion_inputs,
options_input,
options_output,
)
class NoiseInversionOptionsV3(OptionsNodeBase):
"""Add source-derived starting noise to an immutable sampler options chain."""
@classmethod
def define_schema(cls) -> Any:
"""Declare the accepted recipe with independently editable controls."""
return COMFY_IO.Schema(
node_id="SimpleSyrup.NoiseInversionOptions",
display_name="Noise Inversion Options",
category="SimpleSyrup/Sampling/Options",
description=(
"Derives starting noise from an input image before sampling; "
"control inversion quality and cost independently."
),
inputs=[options_input(COMFY_IO), *noise_inversion_inputs(COMFY_IO)],
outputs=[options_output(COMFY_IO)],
)
@classmethod
def execute(
cls,
inversion_method: str = "euler",
inversion_resolution_scale: float = 0.5,
inversion_steps: int = 2,
inversion_switch_fraction: float = 0.75,
inversion_finishing_steps: int = 1,
options: SamplerOptions | None = None,
) -> tuple[SamplerOptions]:
"""Append inversion or pass through at zero steps without preparing a model."""
inversion = inversion_from_controls(
inversion_method=inversion_method,
inversion_resolution_scale=inversion_resolution_scale,
inversion_steps=inversion_steps,
inversion_switch_fraction=inversion_switch_fraction,
inversion_finishing_steps=inversion_finishing_steps,
)
return (append_sampler_capability(options, inversion),)
@@ -0,0 +1,102 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Expose captured noise reuse and optional full-resolution inversion finishing."""
from __future__ import annotations
from typing import Any, cast
from ..domain.noise_inversion import InversionMethod
from ..domain.noise_reuse import NoiseReuseOptions
from ..domain.sampler_options import SamplerOptions, append_sampler_capability
from .sampler_options_schema import (
COMFY_IO,
OptionsNodeBase,
options_input,
options_output,
)
class NoiseReuseOptionsV3(OptionsNodeBase):
"""Contribute one enabled request without changing preceding capabilities."""
@classmethod
def define_schema(cls) -> Any:
"""Place the explicit enable switch above independent finishing controls."""
return COMFY_IO.Schema(
node_id="SimpleSyrup.NoiseReuseOptions",
display_name="Noise Reuse Options",
category="SimpleSyrup/Sampling/Options",
description=(
"Reuses noise from the source sampling run, with optional "
"full-resolution inversion finishing before sampling."
),
inputs=[
options_input(COMFY_IO),
COMFY_IO.Boolean.Input(
"enabled",
default=True,
tooltip=(
"Requests captured source noise when enabled; disabled "
"passes preceding options through unchanged."
),
),
COMFY_IO.Int.Input(
"finishing_steps",
default=0,
min=0,
max=100,
tooltip=(
"Full-resolution inversion steps before sampling; 0 "
"reuses captured noise without finishing."
),
),
COMFY_IO.Combo.Input(
"finishing_method",
options=["euler", "heun"],
default="euler",
tooltip=(
"Euler uses one evaluation per finishing step; Heun uses "
"two for greater accuracy."
),
),
COMFY_IO.Float.Input(
"finishing_start_fraction",
default=0.75,
min=0.01,
max=0.99,
step=0.01,
tooltip=(
"With finishing enabled, captures at this fraction of the "
"receiving sampler's starting noise coordinate."
),
),
],
outputs=[options_output(COMFY_IO)],
)
@classmethod
def execute(
cls,
enabled: bool = True,
finishing_steps: int = 0,
finishing_method: str = "euler",
finishing_start_fraction: float = 0.75,
options: SamplerOptions | None = None,
) -> tuple[SamplerOptions]:
"""Validate enabled controls and preserve the exact disabled input object."""
if type(enabled) is not bool:
raise TypeError("Noise reuse enabled must be a boolean.")
if finishing_method not in ("euler", "heun"):
raise ValueError("Finishing method must be Euler or Heun.")
reuse = (
NoiseReuseOptions(
finishing_steps,
cast(InversionMethod, finishing_method),
finishing_start_fraction,
)
if enabled
else None
)
return (append_sampler_capability(options, reuse),)
@@ -0,0 +1,134 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Declare bypass-compatible sampler options sockets and inversion controls."""
from __future__ import annotations
from importlib import import_module
from typing import TYPE_CHECKING, Any, ClassVar, cast
from ..domain.noise_inversion import (
INVERSION_METHODS,
InversionMethod,
NoiseInversionOptions,
)
if TYPE_CHECKING:
class OptionsNodeBase:
"""Describe Comfy's host-facing node metadata for strict type checking."""
RETURN_TYPES: ClassVar[list[str]]
RETURN_NAMES: ClassVar[list[str]]
else:
OptionsNodeBase = import_module("comfy_api.latest").io.ComfyNode
COMFY_IO: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
OPTIONS_TYPE = "SIMPLE_SYRUP_SAMPLER_OPTIONS"
def options_input(comfy_io: Any) -> Any:
"""Allow any capability to start a chain or consume a preceding capability."""
return comfy_io.Custom(OPTIONS_TYPE).Input(
"options",
optional=True,
tooltip=(
"Optional preceding sampler options; bypass this node "
"to omit its contribution."
),
)
def options_output(comfy_io: Any) -> Any:
"""Match the input type so Comfy can bypass capability nodes natively."""
return comfy_io.Custom(OPTIONS_TYPE).Output(
"options",
tooltip="Combined sampler options; connect another options node or KSampler.",
)
def noise_inversion_inputs(comfy_io: Any, *, convenience: bool = False) -> list[Any]:
"""Default to the accepted recipe and use zero steps to disable inversion."""
return [
comfy_io.Combo.Input(
"inversion_method",
options=list(INVERSION_METHODS),
default="euler",
optional=convenience,
tooltip=(
"Applies to both inversion stages; Euler uses one evaluation per step, "
"Heun uses two for greater accuracy."
),
),
comfy_io.Float.Input(
"inversion_resolution_scale",
default=0.5,
min=0.01,
max=1.0,
step=0.05,
optional=convenience,
tooltip=(
"Scales inversion width and height; "
"0.5 uses half-sized dimensions for lower cost."
),
),
comfy_io.Int.Input(
"inversion_steps",
default=2,
min=0,
max=64,
optional=convenience,
tooltip=(
"Steps at the selected inversion resolution; 0 disables all inversion, "
"including finishing. More steps cost more model evaluations."
),
),
comfy_io.Float.Input(
"inversion_switch_fraction",
default=0.75,
min=0.01,
max=1.0,
step=0.05,
optional=convenience,
tooltip=(
"Noise-level fraction reached before the full-resolution finish; "
"0.75 means 75%."
),
),
comfy_io.Int.Input(
"inversion_finishing_steps",
default=1,
min=0,
max=64,
optional=convenience,
tooltip=(
"Full-resolution inversion steps after a reduced stage; "
"0 finishes entirely at reduced size."
),
),
]
def inversion_from_controls(
*,
inversion_method: str = "euler",
inversion_resolution_scale: float = 0.5,
inversion_steps: int = 2,
inversion_switch_fraction: float = 0.75,
inversion_finishing_steps: int = 1,
) -> NoiseInversionOptions | None:
"""Disable all stages at zero steps or construct a shared-method recipe."""
if type(inversion_steps) is not int or not 0 <= inversion_steps <= 64:
raise ValueError("Inversion steps must be an integer between 0 and 64.")
if inversion_steps == 0:
return None
return NoiseInversionOptions(
method=cast(InversionMethod, inversion_method),
resolution_scale=inversion_resolution_scale,
steps=inversion_steps,
switch_fraction=inversion_switch_fraction,
finishing_steps=inversion_finishing_steps,
)
@@ -0,0 +1,76 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Expose automatic learned and native latent scaling through one method control."""
from __future__ import annotations
from typing import Any
from ..runtime.latent_upscale_catalog import latent_model_choices
from ..services.latent_upscale_service import LatentUpscaleService
from .sampler_options_schema import COMFY_IO, OptionsNodeBase
class SimpleUpscaleLatentV3(OptionsNodeBase):
"""Resize the clean latent while retaining its source noise provenance."""
@classmethod
def define_schema(cls) -> Any:
"""Declare one method dropdown and a scale multiplier without loading
weights.
"""
return COMFY_IO.Schema(
node_id="SimpleSyrup.SimpleUpscaleLatent",
display_name="Simple Upscale Latent",
category="SimpleSyrup/Latent",
description=(
"Upscales clean latents with a compatible learned model "
"or native interpolation, preserving sampling provenance."
),
inputs=[
COMFY_IO.Latent.Input(
"latent",
tooltip=(
"Clean latent to enlarge, carrying source sampling "
"provenance when noise reuse is needed."
),
),
COMFY_IO.Combo.Input(
"method",
options=latent_model_choices(),
default="Auto",
tooltip=(
"Auto selects a compatible learned model from sampling "
"provenance, or Comfy's bislerp for unknown latent formats."
),
),
COMFY_IO.Float.Input(
"scale_factor",
default=2.0,
min=0.01,
max=8.0,
step=0.01,
tooltip=(
"Width and height multiplier; learned models enlarge "
"through internal stages of at most 2x."
),
),
],
outputs=[
COMFY_IO.Latent.Output(
"latent",
tooltip=(
"Resized clean latent retaining source sampling "
"provenance for downstream noise reuse."
),
)
],
)
@classmethod
def execute(
cls, latent: dict[str, Any], method: str = "Auto", scale_factor: float = 2.0
) -> tuple[dict[str, Any]]:
"""Delegate method selection and inference to the latent upscale service."""
return (LatentUpscaleService().upscale(latent, method, scale_factor),)
+78
View File
@@ -0,0 +1,78 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Configure the single local tiling authority for a sampler options chain."""
from __future__ import annotations
from typing import Any
from ..domain.sampler_options import (
SamplerOptions,
TilingOptions,
append_sampler_capability,
)
from .ksampler_schema import tiled_diffusion_inputs
from .sampler_options_schema import (
COMFY_IO,
OptionsNodeBase,
options_input,
options_output,
)
class TilingOptionsV3(OptionsNodeBase):
"""Add bounded local tiles, blend policy and mask-dependent denoising."""
@classmethod
def define_schema(cls) -> Any:
"""Share tiled controls and expose mask-dependent denoising."""
return COMFY_IO.Schema(
node_id="SimpleSyrup.TilingOptions",
display_name="Tiling Options",
category="SimpleSyrup/Sampling/Options",
description=(
"Samples bounded local tiles; ignored when "
"Contextual Diffusion Options is connected."
),
inputs=[
options_input(COMFY_IO),
*tiled_diffusion_inputs(COMFY_IO),
COMFY_IO.Boolean.Input(
"differential_diffusion",
default=False,
tooltip=(
"Uses the noise mask to vary denoising strength spatially; "
"preserves existing model mask behavior."
),
),
],
outputs=[options_output(COMFY_IO)],
)
@classmethod
def execute(
cls,
diffusion_mode: str = "multidiffusion",
latent_tile_width: int = 128,
latent_tile_height: int = 128,
latent_tile_overlap: int = 16,
latent_tile_batch_size: int = 4,
differential_diffusion: bool = False,
options: SamplerOptions | None = None,
) -> tuple[SamplerOptions]:
"""Append validated tiling without changing the incoming chain."""
return (
append_sampler_capability(
options,
TilingOptions(
diffusion_mode=diffusion_mode,
width=latent_tile_width,
height=latent_tile_height,
overlap=latent_tile_overlap,
batch_size=latent_tile_batch_size,
differential_diffusion=differential_diffusion,
),
),
)
+238
View File
@@ -0,0 +1,238 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Expose internal cache-keyed capture planning and handoff nodes."""
from __future__ import annotations
from typing import Any
from ..domain.sampler_options import SamplerOptions
from ..runtime.trajectory_capture import TrajectoryCapture
from ..services.noise_capture_request_service import NoiseCaptureRequestService
from .sampler_options_schema import COMFY_IO, OptionsNodeBase, options_input
REQUEST_TYPE = "SIMPLE_SYRUP_CAPTURE_REQUEST"
CAPTURE_TYPE = "SIMPLE_SYRUP_TRAJECTORY_CAPTURE"
class NoiseCaptureRequestV3(OptionsNodeBase):
"""Resolve the receiver's live schedule before source generation."""
@classmethod
def define_schema(cls) -> Any:
"""Include every schedule-affecting dependency in capture caching."""
return COMFY_IO.Schema(
node_id="SimpleSyrup.NoiseCaptureRequest",
display_name="Noise Capture Request (Internal)",
category="SimpleSyrup/Internal",
is_dev_only=True,
description=(
"Coordinates upstream state capture using receiving sampler settings."
),
inputs=[
COMFY_IO.Model.Input(
"model",
tooltip="Receiving sampler model and schedule parameterization.",
),
options_input(COMFY_IO),
COMFY_IO.Latent.Input(
"source_latent",
tooltip="Original source input used only for schedule geometry.",
),
COMFY_IO.Int.Input("steps", tooltip="Receiving sampling step count."),
COMFY_IO.String.Input(
"sampler_name", tooltip="Receiving sampler identifier."
),
COMFY_IO.String.Input(
"scheduler", tooltip="Receiving scheduler identifier."
),
COMFY_IO.Float.Input(
"denoise", tooltip="Receiving sampler denoise strength."
),
COMFY_IO.String.Input(
"geometry_json",
tooltip="Validated resizing operations for schedule planning.",
),
COMFY_IO.Custom(REQUEST_TYPE).Input(
"previous",
optional=True,
tooltip="Other receiving requests sharing this source sampler.",
),
],
outputs=[
COMFY_IO.Custom(REQUEST_TYPE).Output(
"requests", tooltip="Exact source capture coordinates."
)
],
)
@classmethod
def execute(
cls,
model: Any,
source_latent: dict[str, Any],
steps: int,
sampler_name: str,
scheduler: str,
denoise: float,
geometry_json: str,
options: SamplerOptions | None = None,
previous: tuple[float, ...] = (),
) -> tuple[tuple[float, ...]]:
"""Delegate to the shared receiver schedule authority."""
return (
NoiseCaptureRequestService().coordinates(
model=model,
options=options or SamplerOptions(),
steps=steps,
sampler_name=sampler_name,
scheduler=scheduler,
denoise=denoise,
source_latent=source_latent,
geometry_json=geometry_json,
previous=previous,
),
)
class TrajectoryCaptureModelV3(OptionsNodeBase):
"""Install capture on a source MODEL without modifying shared model instances."""
@classmethod
def define_schema(cls) -> Any:
"""Expose only internal derivation dependencies."""
return COMFY_IO.Schema(
node_id="SimpleSyrup.TrajectoryCaptureModel",
display_name="Trajectory Capture Model (Internal)",
category="SimpleSyrup/Internal",
is_dev_only=True,
description=(
"Captures real sampler states for downstream latent noise reuse."
),
inputs=[
COMFY_IO.Model.Input(
"model", tooltip="Source MODEL cloned for capture."
),
COMFY_IO.String.Input(
"sampler_name", tooltip="Source solver identifier."
),
COMFY_IO.Sampler.Input(
"sampler",
optional=True,
tooltip="Connected source algorithm used by a custom sampler.",
),
COMFY_IO.String.Input(
"graph_context",
optional=True,
default="",
tooltip="Source and receiving node names for error messages.",
),
COMFY_IO.Custom(REQUEST_TYPE).Input(
"requests",
optional=True,
tooltip=(
"Requested real-state coordinates; omitted records format "
"provenance only."
),
),
],
outputs=[
COMFY_IO.Model.Output(
"model", tooltip="Source MODEL carrying its capture observer."
),
COMFY_IO.Custom(CAPTURE_TYPE).Output(
"capture", tooltip="Source-local capture lifecycle."
),
],
)
@classmethod
def execute(
cls,
model: Any,
sampler_name: str,
requests: tuple[float, ...] = (),
sampler: Any = None,
graph_context: str = "",
) -> tuple[Any, TrajectoryCapture]:
"""Keep observer state scoped to this source and cache identity."""
if not sampler_name and sampler is not None:
function = getattr(sampler, "sampler_function", None)
sampler_name = getattr(function, "__name__", type(sampler).__name__)
capture = TrajectoryCapture(
model, sampler_name, requests, graph_context=graph_context
)
return capture.prepare_model(), capture
class AttachSamplingProvenanceV3(OptionsNodeBase):
"""Attach a verified source capture before latents enter resizing or decode."""
@classmethod
def define_schema(cls) -> Any:
"""Require both completed source output and its precise capture owner."""
return COMFY_IO.Schema(
node_id="SimpleSyrup.AttachSamplingProvenance",
display_name="Attach Sampling Provenance (Internal)",
category="SimpleSyrup/Internal",
is_dev_only=True,
description=(
"Associates original sampler output with matching captured states."
),
inputs=[
COMFY_IO.Latent.Input(
"latent", tooltip="Completed source sampler output."
),
COMFY_IO.Custom(CAPTURE_TYPE).Input(
"capture",
tooltip="Capture belonging to this exact source sampler execution.",
),
],
outputs=[
COMFY_IO.Latent.Output(
"latent",
tooltip="Original latent with verified sampling provenance.",
)
],
)
@classmethod
def execute(
cls, latent: dict[str, Any], capture: TrajectoryCapture
) -> tuple[dict[str, Any]]:
"""Validate source identity before publishing capture metadata."""
return (capture.attach(latent),)
class NoiseReuseErrorV3(OptionsNodeBase):
"""Surface graph planning failures through normal Comfy execution errors."""
@classmethod
def define_schema(cls) -> Any:
"""Avoid prompt-handler exception swallowing by making failure explicit."""
return COMFY_IO.Schema(
node_id="SimpleSyrup.NoiseReuseError",
display_name="Noise Reuse Validation (Internal)",
category="SimpleSyrup/Internal",
is_dev_only=True,
description=(
"Reports invalid sampling provenance before downstream sampling."
),
inputs=[
COMFY_IO.String.Input(
"message", tooltip="Actionable source graph validation error."
)
],
outputs=[
COMFY_IO.Latent.Output(
"latent", tooltip="Validation must succeed before sampling."
)
],
)
@classmethod
def execute(cls, message: str) -> tuple[dict[str, Any]]:
"""Fail closed instead of substituting fresh noise or encoded pixels."""
raise ValueError(message)
@@ -16,16 +16,25 @@ from ..domain.contextual_diffusion import (
ContextualDiffusionControls,
ContextualDiffusionPlan,
)
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.regional_features import (
EMPTY_REGIONAL_CAPABILITY_ADMISSION,
RegionalCapabilityAdmission,
)
from ..domain.sampler_options import TilingOptions
from ..domain.segs import NativeSegs
from ..shared.logging import get_logger
from . import sampling_noise, sampling_samplers, sampling_schedulers
from .contextual_model_wrapper import ContextualDiffusionModelWrapper
from .differential_diffusion import (
differential_diffusion_mutation,
has_denoise_mask_function,
)
from .guided_sampling import sample_with_optional_negative
from .inversion_model_factory import InversionModelFactory
from .model_patcher_mutations import ModelUnetWrapperMutation
from .patcher_lifecycle import PATCHER_LIFECYCLE
from .noise_reuse_scope import needs_finishing_model
from .patcher_lifecycle import PATCHER_LIFECYCLE, ModelMutation
from .sampling_model_types import ModelFunctionWrapper
from .tiled_sampling_validation import (
Latent,
@@ -37,7 +46,6 @@ from .tiled_sampling_validation import (
LOGGER = get_logger(__name__)
SAMPLER_LABEL = "Contextual Diffusion"
UNIPC_SAMPLERS = frozenset({"uni_pc", "uni_pc_bh2"})
def sample_contextual_diffusion(
@@ -58,19 +66,21 @@ def sample_contextual_diffusion(
capability_admission: RegionalCapabilityAdmission = (
EMPTY_REGIONAL_CAPABILITY_ADMISSION
),
noise_inversion: NoiseInversionOptions | None = None,
inversion_segs: NativeSegs | None = None,
inversion_region_masks: torch.Tensor | None = None,
differential_diffusion: bool = False,
) -> Latent:
"""Sample one latent through global context and one tiled prediction plan."""
validate_sampling_controls(
steps=steps,
denoise=denoise,
latent_tile_width=controls.latent_context_size,
latent_tile_height=controls.latent_context_size,
latent_tile_width=controls.tile_width,
latent_tile_height=controls.tile_height,
latent_tile_batch_size=controls.latent_context_batch_size,
)
controls.validate()
if sampler_name in UNIPC_SAMPLERS:
raise ValueError("Contextual Diffusion is not compatible with UniPC samplers.")
reject_unsupported_conditioning(
positive,
sampler_label=SAMPLER_LABEL,
@@ -90,8 +100,8 @@ def sample_contextual_diffusion(
steps=steps,
denoise=denoise,
view=sampling_schedulers.SchedulerView(
latent_width=controls.latent_context_size,
latent_height=controls.latent_context_size,
latent_width=controls.tile_width,
latent_height=controls.tile_height,
),
).to(model.load_device)
latent_samples = validate_latent_samples(latent_image, sampler_label=SAMPLER_LABEL)
@@ -114,8 +124,30 @@ def sample_contextual_diffusion(
controls=controls,
sigmas=sigmas,
diffusion_mode=diffusion_mode,
differential_diffusion=differential_diffusion,
)
batch_inds = latent_image.get("batch_index")
inversion_factory = (
InversionModelFactory(
model=model,
canvas_width=plan.latent_width,
canvas_height=plan.latent_height,
tiling=TilingOptions(
diffusion_mode=diffusion_mode,
width=controls.tile_width,
height=controls.tile_height,
overlap=controls.latent_context_overlap,
batch_size=controls.latent_context_batch_size,
differential_diffusion=differential_diffusion,
),
context=controls,
forward_sigmas=sigmas,
segs=inversion_segs,
region_masks=inversion_region_masks,
)
if noise_inversion is not None or needs_finishing_model()
else None
)
noise = sampling_noise.prepare_sampling_noise(
comfy_sample=comfy_sample,
sampler_name=sampler_name,
@@ -131,6 +163,7 @@ def sample_contextual_diffusion(
noise=noise,
cfg=cfg,
sampler=sampler,
sampler_name=sampler_name,
sigmas=sigmas,
positive=positive,
negative=negative,
@@ -139,6 +172,8 @@ def sample_contextual_diffusion(
callback=callback,
disable_pbar=not comfy_utils.PROGRESS_BAR_ENABLED,
seed=seed,
noise_inversion=noise_inversion,
inversion_model_factory=inversion_factory,
)
LOGGER.info(
@@ -178,10 +213,12 @@ def clone_model_with_contextual_diffusion(
controls: ContextualDiffusionControls,
sigmas: torch.Tensor,
diffusion_mode: str,
differential_diffusion: bool = False,
existing_wrapper: ModelFunctionWrapper | None = None,
) -> Any:
"""Derive a model with one pre-CFG contextual prediction wrapper."""
old_wrapper = model.model_options.get("model_function_wrapper")
old_wrapper = existing_wrapper or model.model_options.get("model_function_wrapper")
if old_wrapper is not None and not callable(old_wrapper):
raise ValueError("Existing model_function_wrapper is not callable.")
wrapper = ContextualDiffusionModelWrapper(
@@ -191,9 +228,13 @@ def clone_model_with_contextual_diffusion(
diffusion_mode=diffusion_mode,
existing_wrapper=cast(ModelFunctionWrapper | None, old_wrapper),
)
mutations: list[ModelMutation] = []
if differential_diffusion and not has_denoise_mask_function(model):
mutations.append(differential_diffusion_mutation())
mutations.append(ModelUnetWrapperMutation(wrapper))
return PATCHER_LIFECYCLE.derive_model(
model,
(ModelUnetWrapperMutation(wrapper),),
mutations,
operation="SimpleSyrup contextual diffusion",
)
+1
View File
@@ -100,6 +100,7 @@ class DetailSampler:
noise=noise,
cfg=cfg,
sampler=sampler,
sampler_name=sampler_name,
sigmas=sigmas,
positive=positive,
negative=negative,
+59 -2
View File
@@ -11,7 +11,12 @@ from typing import Any, cast
import torch
from ..domain.noise_inversion import NoiseInversionOptions
from ..shared.logging import get_logger
from .noise_inversion import InversionModelFactory, invert_sampling_noise
from .noise_reuse_sampling import prepare_reused_noise
from .noise_reuse_scope import current_noise_reuse
from .noise_reuse_start import NoiseReuseStartVerification
LOGGER = get_logger(__name__)
@@ -23,6 +28,7 @@ def sample_with_optional_negative(
noise: torch.Tensor,
cfg: float,
sampler: Any,
sampler_name: str,
sigmas: torch.Tensor,
positive: Any,
negative: Any | None,
@@ -31,11 +37,57 @@ def sample_with_optional_negative(
callback: Any = None,
disable_pbar: bool = False,
seed: int | None = None,
noise_inversion: NoiseInversionOptions | None = None,
inversion_model_factory: InversionModelFactory | None = None,
) -> torch.Tensor:
"""Sample with CFG when negative exists or Comfy's positive-only path otherwise."""
"""Prepare optional source-derived noise and select the actual Comfy guider."""
reuse = current_noise_reuse()
start_verification: NoiseReuseStartVerification | None = None
if reuse is not None:
if noise_inversion is not None:
raise ValueError(
"Noise reuse and full noise inversion cannot own the same "
"sampling pass."
)
noise, expected = prepare_reused_noise(
execution=reuse,
model=model,
fresh=noise,
latent=latent_image,
sigmas=sigmas,
positive=positive,
negative=negative,
cfg=cfg,
seed=seed,
noise_mask=noise_mask,
model_factory=inversion_model_factory,
)
start_verification = NoiseReuseStartVerification(
model.get_model_object("model_sampling.noise_scaling"),
expected,
graph_context=reuse.provenance.graph_context,
sampler_name=sampler_name,
)
model = start_verification.derive_model(model)
if noise_inversion is not None:
inversion = invert_sampling_noise(
model=model,
latent=latent_image,
forward_sigmas=sigmas,
positive=positive,
negative=negative,
cfg=cfg,
seed=seed,
options=noise_inversion,
model_factory=inversion_model_factory,
noise_mask=noise_mask,
)
noise = inversion.noise.to(noise)
if negative is not None:
return cast(
samples = cast(
torch.Tensor,
comfy_sample.sample_custom(
model,
@@ -52,6 +104,9 @@ def sample_with_optional_negative(
seed=seed,
),
)
if start_verification is not None:
start_verification.require_verified()
return samples
comfy_samplers = import_module("comfy.samplers")
model_management = import_module("comfy.model_management")
@@ -67,6 +122,8 @@ def sample_with_optional_negative(
disable_pbar=disable_pbar,
seed=seed,
)
if start_verification is not None:
start_verification.require_verified()
LOGGER.debug(
"Positive-only ComfyUI guider selected",
extra={
@@ -0,0 +1,164 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Prepare each inversion resolution from the original, spatially unwrapped MODEL."""
from __future__ import annotations
from typing import Any, cast
import torch
import torch.nn.functional as functional
from ..domain.contextual_diffusion import (
ContextualDiffusionControls,
build_contextual_diffusion_plan,
)
from ..domain.regional_tiled_diffusion import (
build_region_constrained_tiled_diffusion_plan,
)
from ..domain.sampler_options import TilingOptions
from ..domain.segs import NativeSegs
from ..domain.segs_tiled_diffusion import build_segs_guided_tiled_diffusion_plan
from ..domain.tiled_diffusion import TiledDiffusionPlan, build_tiled_diffusion_plan
from .inversion_spatial_context import InversionSpatialContextWrapper
from .model_patcher_mutations import ModelUnetWrapperMutation
from .patcher_lifecycle import PATCHER_LIFECYCLE
from .sampling_model_types import ModelFunctionWrapper
class InversionModelFactory:
"""Own stage planning while sharing the forward spatial wrapper authorities."""
def __init__(
self,
*,
model: Any,
canvas_width: int,
canvas_height: int,
tiling: TilingOptions | None = None,
context: ContextualDiffusionControls | None = None,
forward_sigmas: torch.Tensor | None = None,
segs: NativeSegs | None = None,
region_masks: torch.Tensor | None = None,
) -> None:
"""Retain canonical inputs, not an already wrapped full-resolution model."""
if context is not None and (tiling is None or forward_sigmas is None):
raise ValueError(
"Contextual inversion requires tile controls and forward sigmas."
)
self._model = model
self._width = canvas_width
self._height = canvas_height
self._tiling = tiling
self._context = context
self._sigmas = forward_sigmas
self._segs = segs
self._masks = region_masks
def __call__(self, latent: torch.Tensor) -> Any:
"""Replan one resolution with canonical regional-mask coordinates."""
from .contextual_diffusion_sampling import clone_model_with_contextual_diffusion
from .mixture_of_diffusers_sampling import clone_model_with_mixture_of_diffusers
from .multidiffusion_sampling import clone_model_with_multidiffusion
width, height = int(latent.shape[-1]), int(latent.shape[-2])
old_wrapper = self._model.model_options.get("model_function_wrapper")
if old_wrapper is not None and not callable(old_wrapper):
raise TypeError("Existing model_function_wrapper must be callable.")
wrapper = cast(ModelFunctionWrapper | None, old_wrapper)
if wrapper is not None and (width, height) != (self._width, self._height):
wrapper = InversionSpatialContextWrapper(
wrapper,
canvas_width=self._width,
canvas_height=self._height,
stage_width=width,
stage_height=height,
)
masks = self._stage_masks(height, width)
if self._context is not None:
assert self._tiling is not None and self._sigmas is not None
plan = build_contextual_diffusion_plan(
latent_width=width,
latent_height=height,
controls=self._context,
segs=self._segs,
region_masks=masks,
segs_canvas=(self._height, self._width),
)
return clone_model_with_contextual_diffusion(
self._model,
plan=plan,
controls=self._context,
sigmas=self._sigmas,
diffusion_mode=self._tiling.diffusion_mode,
differential_diffusion=self._tiling.differential_diffusion,
existing_wrapper=wrapper,
)
if self._tiling is not None:
tile_plan = self._tile_plan(width, height, masks)
clone = (
clone_model_with_multidiffusion
if self._tiling.diffusion_mode == "multidiffusion"
else clone_model_with_mixture_of_diffusers
)
derived, _ = clone(
self._model,
latent_width=width,
latent_height=height,
tile_width=self._tiling.width,
tile_height=self._tiling.height,
overlap=self._tiling.overlap,
tile_batch_size=self._tiling.batch_size,
differential_diffusion=self._tiling.differential_diffusion,
tiled_plan=tile_plan,
existing_wrapper=wrapper,
)
return derived
if wrapper is old_wrapper:
return self._model
assert wrapper is not None
return PATCHER_LIFECYCLE.derive_model(
self._model,
(ModelUnetWrapperMutation(wrapper),),
operation="SimpleSyrup inversion spatial context",
)
def _stage_masks(self, height: int, width: int) -> torch.Tensor | None:
"""Resize planning masks once; attention masks keep their canonical bank."""
if self._masks is None:
return None
if tuple(self._masks.shape[-2:]) == (height, width):
return self._masks
return functional.interpolate(
self._masks.unsqueeze(1).float(), size=(height, width), mode="nearest"
).squeeze(1)
def _tile_plan(
self, width: int, height: int, masks: torch.Tensor | None
) -> TiledDiffusionPlan:
"""Share regular, SEGS and regional ownership with forward sampling."""
assert self._tiling is not None
geometry = {
"latent_width": width,
"latent_height": height,
"tile_width": self._tiling.width,
"tile_height": self._tiling.height,
"overlap": self._tiling.overlap,
"tile_batch_size": self._tiling.batch_size,
}
if masks is not None:
return build_region_constrained_tiled_diffusion_plan(
region_masks=masks,
segs=self._segs,
segs_canvas=(self._height, self._width),
**geometry,
)
if self._segs is not None:
return build_segs_guided_tiled_diffusion_plan(
segs=self._segs,
segs_canvas=(self._height, self._width),
**geometry,
)
return build_tiled_diffusion_plan(**geometry)
@@ -0,0 +1,128 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Project reduced inversion views into the original regional-attention canvas."""
from __future__ import annotations
from typing import Any
import torch
from ..domain.spatial_views import SpatialBatchLayout, SpatialView, SpatialViewKind
from .sampling_model_types import ApplyModel, ModelFunctionWrapper
from .spatial_model_arguments import (
SIMPLE_SYRUP_TRANSFORMER_NAMESPACE,
SPATIAL_BATCH_LAYOUT_KEY,
)
def rebase_inversion_layout(
layout: SpatialBatchLayout, *, canvas_width: int, canvas_height: int
) -> SpatialBatchLayout:
"""Keep actual model dimensions while mapping source rectangles to full masks."""
scale_x = canvas_width / layout.canvas_width
scale_y = canvas_height / layout.canvas_height
views: list[SpatialView] = []
for view in layout.views:
left = round(view.source_x * scale_x)
top = round(view.source_y * scale_y)
right = round(view.source_right * scale_x)
bottom = round(view.source_bottom * scale_y)
views.append(
SpatialView(
kind=(
SpatialViewKind.CONTEXTUAL_GLOBAL
if view.kind is SpatialViewKind.FULL
else view.kind
),
source_x=left,
source_y=top,
source_width=right - left,
source_height=bottom - top,
model_width=view.model_width,
model_height=view.model_height,
)
)
return SpatialBatchLayout(
canvas_width, canvas_height, tuple(views), layout.input_batch_size
)
class InversionSpatialContextWrapper:
"""Preserve regional mask coordinates without resizing conditioning twice."""
def __init__(
self,
existing_wrapper: ModelFunctionWrapper,
*,
canvas_width: int,
canvas_height: int,
stage_width: int,
stage_height: int,
) -> None:
"""Bind one stage's geometry to the original attention-mask canvas."""
self._existing_wrapper = existing_wrapper
self._canvas_width = canvas_width
self._canvas_height = canvas_height
self._stage_width = stage_width
self._stage_height = stage_height
def __call__(self, apply_model: ApplyModel, args: dict[str, Any]) -> torch.Tensor:
"""Replace only layout metadata, preserving expanded CFG batch metadata."""
x = args.get("input")
c = args.get("c", {})
if not isinstance(x, torch.Tensor) or not isinstance(c, dict):
raise TypeError(
"Inversion spatial context requires tensor input and conditioning."
)
transformer_options = c.get("transformer_options", {})
if not isinstance(transformer_options, dict):
raise TypeError("Inversion transformer_options must be a dictionary.")
namespace = transformer_options.get(SIMPLE_SYRUP_TRANSFORMER_NAMESPACE, {})
if not isinstance(namespace, dict):
raise TypeError(
"Inversion SimpleSyrup transformer namespace must be a dictionary."
)
layout = namespace.get(SPATIAL_BATCH_LAYOUT_KEY)
if layout is None:
if tuple(x.shape[-2:]) != (self._stage_height, self._stage_width):
raise ValueError(
"Reduced inversion calls require explicit spatial layout."
)
layout = SpatialBatchLayout(
self._stage_width,
self._stage_height,
(
SpatialView(
SpatialViewKind.FULL,
0,
0,
self._stage_width,
self._stage_height,
self._stage_width,
self._stage_height,
),
),
int(x.shape[0]),
)
if not isinstance(layout, SpatialBatchLayout):
raise TypeError("Inversion spatial layout has an invalid type.")
if (layout.canvas_width, layout.canvas_height) != (
self._stage_width,
self._stage_height,
) or layout.expanded_batch_size != int(x.shape[0]):
raise ValueError(
"Inversion layout must describe the current stage and model batch."
)
rebased = rebase_inversion_layout(
layout, canvas_width=self._canvas_width, canvas_height=self._canvas_height
)
projected_namespace = {**namespace, SPATIAL_BATCH_LAYOUT_KEY: rebased}
projected_options = {
**transformer_options,
SIMPLE_SYRUP_TRANSFORMER_NAMESPACE: projected_namespace,
}
projected_args = {**args, "c": {**c, "transformer_options": projected_options}}
return self._existing_wrapper(apply_model, projected_args)
@@ -0,0 +1,75 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""List checksum-pinned SesquiLSR artifacts in Comfy's standard latent model folder."""
from __future__ import annotations
from importlib import import_module
from .auto_model_artifact import AutoModelArtifact
SESQUI_REVISION = "befae004248c403f38b76b9f65fd43b901ea3eaa"
SESQUI_REPOSITORY = "https://github.com/LoganBooker/SesquiLSR"
_CHECKPOINTS = (
(
"wan",
"Wan21",
12373048,
"d1b4c2c1d80b2af81e6bbfd38501ffb5adffb9f7c0ed2599555b837d0d2d015a",
),
(
"sdxl",
"SDXL",
12293696,
"392c0e36fc0389a170cf6c1682556972d91d750c31aa392cdad1b13cad4e18e4",
),
(
"flux",
"Flux",
12374768,
"dbc6362c910a179a22eca26f656fdacde17eded876ee8f05b8f5b73630e4ddbe",
),
(
"flux2",
"Flux2",
12525400,
"f385ff2250b3f2c72ad422768558414d617b9a006cc0b1ce37a296cc99d17177",
),
)
LATENT_UPSCALE_ARTIFACTS = {
family: AutoModelArtifact(
cache_id=f"sesqui_lsr_{family}_{SESQUI_REVISION}",
filename=f"upscaler_{name}.safetensors",
folder_name="latent_upscale_models",
canonical_subfolder=family,
source_url=(
f"https://raw.githubusercontent.com/LoganBooker/SesquiLSR/"
f"{SESQUI_REVISION}/models/upscaler_{name}.safetensors"
),
source_repo=SESQUI_REPOSITORY,
description=f"SesquiLSR {name} latent upscaler",
sha256=checksum,
file_size_bytes=size,
)
for family, name, size, checksum in _CHECKPOINTS
}
def latent_model_choices() -> list[str]:
"""Advertise pinned downloads and installed checkpoints without hashing or
IO loading.
"""
from ..domain.latent_upscale import NATIVE_LATENT_METHODS
known = [
f"{artifact.canonical_subfolder}/{artifact.filename}"
for artifact in LATENT_UPSCALE_ARTIFACTS.values()
]
installed = [
name.replace("\\", "/")
for name in import_module("folder_paths").get_filename_list(
"latent_upscale_models"
)
]
return list(dict.fromkeys(["Auto", *known, *installed, *NATIVE_LATENT_METHODS]))
@@ -0,0 +1,96 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Resolve trusted learned checkpoints through registered Comfy model paths."""
from __future__ import annotations
from importlib import import_module
from pathlib import Path
from typing import Any
import torch
from ..domain.latent_upscale import recommended_family
from ..domain.noise_reuse import PROVENANCE_KEY, SamplingProvenance
from .auto_model_resolver import AutoModelResolver
from .latent_upscale_catalog import LATENT_UPSCALE_ARTIFACTS
from .model_downloads import ComfyProgressReporter
def resolve_latent_upscale_model(
latent: dict[str, Any], method: str
) -> tuple[str, Path]:
"""Resolve verified Auto identity or a checkpoint with a recognized architecture."""
provenance = latent.get(PROVENANCE_KEY)
if method == "Auto":
if not isinstance(provenance, SamplingProvenance):
raise ValueError(
"Auto latent upscaling requires sampling provenance; "
"connect a source sampler or select a compatible "
"checkpoint explicitly."
)
family = recommended_family(provenance.latent_format)
return family, AutoModelResolver().resolve(
LATENT_UPSCALE_ARTIFACTS[family], ComfyProgressReporter()
).path
for family, artifact in LATENT_UPSCALE_ARTIFACTS.items():
if method.replace("\\", "/") == f"{family}/{artifact.filename}":
if (
isinstance(provenance, SamplingProvenance)
and recommended_family(provenance.latent_format) != family
):
raise ValueError(
"Selected latent upscale checkpoint does not match the "
"source latent format."
)
return family, AutoModelResolver().resolve(
artifact, ComfyProgressReporter()
).path
folders = import_module("folder_paths")
path = Path(folders.get_full_path_or_raise("latent_upscale_models", method))
state = import_module("comfy.utils").load_torch_file(str(path), safe_load=True)
head = state.get("head_conv.0.weight")
if (
not isinstance(head, torch.Tensor)
or head.ndim != 4
or "upsample.base_taps" not in state
):
raise ValueError(
"Selected checkpoint is not a supported SesquiLSR latent upscaler."
)
named_family = next(
(
family
for family, artifact in LATENT_UPSCALE_ARTIFACTS.items()
if path.name.casefold() == artifact.filename.casefold()
),
None,
)
if named_family is not None:
expected_channels = {"sdxl": 4, "flux": 16, "wan": 16, "flux2": 32}[
named_family
]
if head.shape[1] != expected_channels:
raise ValueError(
"Selected checkpoint filename and latent channels disagree."
)
detected_family = named_family or {4: "sdxl", 32: "flux2"}.get(int(head.shape[1]))
if detected_family is None and head.shape[1] == 16:
if not isinstance(provenance, SamplingProvenance):
raise ValueError(
"A 16-channel checkpoint requires verified Flux or Wan "
"sampling provenance."
)
detected_family = recommended_family(provenance.latent_format)
if detected_family not in LATENT_UPSCALE_ARTIFACTS:
raise ValueError("Selected checkpoint has unsupported latent channels.")
if (
isinstance(provenance, SamplingProvenance)
and recommended_family(provenance.latent_format) != detected_family
):
raise ValueError(
"Selected latent upscale checkpoint does not match "
"the source latent format."
)
return detected_family, path
@@ -16,10 +16,13 @@ from typing import Any, cast
import torch
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.regional_features import (
EMPTY_REGIONAL_CAPABILITY_ADMISSION,
RegionalCapabilityAdmission,
)
from ..domain.sampler_options import TilingOptions
from ..domain.segs import NativeSegs
from ..domain.tiled_diffusion import (
TiledDiffusionPlan,
build_tiled_diffusion_plan,
@@ -32,7 +35,9 @@ from .differential_diffusion import (
has_denoise_mask_function,
)
from .guided_sampling import sample_with_optional_negative
from .inversion_model_factory import InversionModelFactory
from .model_patcher_mutations import ModelUnetWrapperMutation
from .noise_reuse_scope import needs_finishing_model
from .patcher_lifecycle import PATCHER_LIFECYCLE, ModelMutation
from .sampling_model_types import (
ApplyModel,
@@ -73,6 +78,9 @@ def sample_mixture_of_diffusers(
EMPTY_REGIONAL_CAPABILITY_ADMISSION
),
tiled_plan: TiledDiffusionPlan | None = None,
noise_inversion: NoiseInversionOptions | None = None,
inversion_segs: NativeSegs | None = None,
inversion_region_masks: torch.Tensor | None = None,
) -> Latent:
"""Sample a latent with a cloned model patched for Mixture of Diffusers."""
@@ -151,6 +159,7 @@ def sample_mixture_of_diffusers(
noise=noise,
cfg=cfg,
sampler=sampler,
sampler_name=sampler_name,
sigmas=sigmas,
positive=positive,
negative=negative,
@@ -159,6 +168,26 @@ def sample_mixture_of_diffusers(
callback=callback,
disable_pbar=not comfy_utils.PROGRESS_BAR_ENABLED,
seed=seed,
noise_inversion=noise_inversion,
inversion_model_factory=(
InversionModelFactory(
model=model,
canvas_width=latent_width,
canvas_height=latent_height,
tiling=TilingOptions(
diffusion_mode="mixture_of_diffusers",
width=latent_tile_width,
height=latent_tile_height,
overlap=latent_tile_overlap,
batch_size=latent_tile_batch_size,
differential_diffusion=differential_diffusion,
),
segs=inversion_segs,
region_masks=inversion_region_masks,
)
if noise_inversion is not None or needs_finishing_model()
else None
),
)
LOGGER.info(
@@ -197,6 +226,7 @@ def clone_model_with_mixture_of_diffusers(
tile_batch_size: int,
differential_diffusion: bool = False,
tiled_plan: TiledDiffusionPlan | None = None,
existing_wrapper: ModelFunctionWrapper | None = None,
) -> tuple[Any, TiledDiffusionPlan]:
"""Return a derived model patched with a pre-CFG Mixture wrapper."""
@@ -209,7 +239,7 @@ def clone_model_with_mixture_of_diffusers(
tile_batch_size=tile_batch_size,
)
_validate_supplied_plan(plan, latent_width, latent_height)
old_wrapper = model.model_options.get("model_function_wrapper")
old_wrapper = existing_wrapper or model.model_options.get("model_function_wrapper")
if old_wrapper is not None and not callable(old_wrapper):
raise ValueError("Existing model_function_wrapper is not callable.")
+30 -10
View File
@@ -16,10 +16,13 @@ from typing import Any, cast
import torch
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.regional_features import (
EMPTY_REGIONAL_CAPABILITY_ADMISSION,
RegionalCapabilityAdmission,
)
from ..domain.sampler_options import TilingOptions
from ..domain.segs import NativeSegs
from ..domain.tiled_diffusion import (
TiledDiffusionPlan,
build_tiled_diffusion_plan,
@@ -32,7 +35,9 @@ from .differential_diffusion import (
has_denoise_mask_function,
)
from .guided_sampling import sample_with_optional_negative
from .inversion_model_factory import InversionModelFactory
from .model_patcher_mutations import ModelUnetWrapperMutation
from .noise_reuse_scope import needs_finishing_model
from .patcher_lifecycle import PATCHER_LIFECYCLE, ModelMutation
from .sampling_model_types import (
ApplyModel,
@@ -49,7 +54,6 @@ from .tiled_sampling_validation import (
LOGGER = get_logger(__name__)
SAMPLER_LABEL = "MultiDiffusion"
UNIPC_SAMPLERS = frozenset({"uni_pc", "uni_pc_bh2"})
def sample_multidiffusion(
@@ -74,6 +78,9 @@ def sample_multidiffusion(
EMPTY_REGIONAL_CAPABILITY_ADMISSION
),
tiled_plan: TiledDiffusionPlan | None = None,
noise_inversion: NoiseInversionOptions | None = None,
inversion_segs: NativeSegs | None = None,
inversion_region_masks: torch.Tensor | None = None,
) -> Latent:
"""Sample a latent with a cloned model patched for MultiDiffusion."""
@@ -84,7 +91,6 @@ def sample_multidiffusion(
latent_tile_height=latent_tile_height,
latent_tile_batch_size=latent_tile_batch_size,
)
_reject_unipc_sampler(sampler_name)
reject_unsupported_conditioning(
positive,
sampler_label=SAMPLER_LABEL,
@@ -153,6 +159,7 @@ def sample_multidiffusion(
noise=noise,
cfg=cfg,
sampler=sampler,
sampler_name=sampler_name,
sigmas=sigmas,
positive=positive,
negative=negative,
@@ -161,6 +168,25 @@ def sample_multidiffusion(
callback=callback,
disable_pbar=not comfy_utils.PROGRESS_BAR_ENABLED,
seed=seed,
noise_inversion=noise_inversion,
inversion_model_factory=(
InversionModelFactory(
model=model,
canvas_width=latent_width,
canvas_height=latent_height,
tiling=TilingOptions(
width=latent_tile_width,
height=latent_tile_height,
overlap=latent_tile_overlap,
batch_size=latent_tile_batch_size,
differential_diffusion=differential_diffusion,
),
segs=inversion_segs,
region_masks=inversion_region_masks,
)
if noise_inversion is not None or needs_finishing_model()
else None
),
)
LOGGER.info(
@@ -200,6 +226,7 @@ def clone_model_with_multidiffusion(
tile_batch_size: int,
differential_diffusion: bool = False,
tiled_plan: TiledDiffusionPlan | None = None,
existing_wrapper: ModelFunctionWrapper | None = None,
) -> tuple[Any, TiledDiffusionPlan]:
"""Return a derived model patched with a pre-CFG MultiDiffusion wrapper."""
@@ -212,7 +239,7 @@ def clone_model_with_multidiffusion(
tile_batch_size=tile_batch_size,
)
_validate_supplied_plan(plan, latent_width, latent_height)
old_wrapper = model.model_options.get("model_function_wrapper")
old_wrapper = existing_wrapper or model.model_options.get("model_function_wrapper")
if old_wrapper is not None and not callable(old_wrapper):
raise ValueError("Existing model_function_wrapper is not callable.")
@@ -299,13 +326,6 @@ class MultiDiffusionModelWrapper:
return apply_model(args["input"], args["timestep"], **conditioning)
def _reject_unipc_sampler(sampler_name: str) -> None:
"""Reject UniPC samplers because MultiDiffusion is incompatible with them."""
if sampler_name in UNIPC_SAMPLERS:
raise ValueError("MultiDiffusion is not compatible with UniPC samplers.")
def _validate_supplied_plan(
plan: TiledDiffusionPlan,
latent_width: int,
@@ -0,0 +1,51 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Separate clean references and residuals with the model's actual parameterization."""
from __future__ import annotations
from typing import Any
import torch
def noise_coefficients(
scaling: Any, clean: torch.Tensor, coordinate: float
) -> tuple[torch.Tensor, torch.Tensor]:
"""Validate affine noise mixing instead of assuming DDPM alpha/sigma units."""
sigma = clean.new_tensor(coordinate)
base = scaling.noise_scaling(sigma, torch.zeros_like(clean), clean, False)
zero = torch.zeros_like(clean)
amplitude = scaling.noise_scaling(sigma, torch.ones_like(clean), zero, False)
amplitude = amplitude - scaling.noise_scaling(sigma, zero, zero, False)
if not isinstance(base, torch.Tensor) or not isinstance(amplitude, torch.Tensor):
raise TypeError("Native noise scaling must produce tensor coefficients.")
if base.shape != clean.shape or amplitude.shape != clean.shape:
raise ValueError("Native noise scaling must preserve the clean latent shape.")
if not bool(torch.isfinite(amplitude).all()) or bool((amplitude == 0).any()):
raise ValueError(
"Capture coordinate must have finite invertible noise amplitude."
)
probe = scaling.noise_scaling(sigma, torch.full_like(clean, 0.37), clean, False)
if not isinstance(probe, torch.Tensor) or not torch.allclose(
probe, base + amplitude * 0.37, atol=2e-5, rtol=2e-5
):
raise ValueError("Native noise scaling must be affine in its noise argument.")
return base, amplitude
def recover_residual(
scaling: Any, clean: torch.Tensor, state: torch.Tensor, coordinate: float
) -> torch.Tensor:
"""Require the recovered residual to reconstruct its actual captured state."""
if clean.shape != state.shape:
raise ValueError("Captured state and original clean reference shapes differ.")
base, amplitude = noise_coefficients(scaling, clean, coordinate)
residual = (state - base) / amplitude
if not bool(torch.isfinite(residual).all()) or not torch.allclose(
base + amplitude * residual, state, atol=2e-5, rtol=2e-5
):
raise ValueError("Recovered residual cannot reconstruct the captured state.")
return residual
@@ -0,0 +1,48 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Validate shared model scaling and partial-denoise receiver coordinates."""
from __future__ import annotations
import math
from importlib import import_module
from typing import Any
import torch
MINIMUM_PARTIAL_SIGMA = 0.0001
def validate_partial_sampling_target(
model: Any, sigmas: torch.Tensor, *, operation: str
) -> float:
"""Reject unsupported scaling and singular targets before model execution."""
sampling_types = import_module("comfy.model_sampling")
sampling = model.get_model_object("model_sampling")
if not isinstance(
sampling, (sampling_types.CONST, sampling_types.EPS)
) or isinstance(
sampling, (sampling_types.IMG_TO_IMG, sampling_types.IMG_TO_IMG_FLOW)
):
raise ValueError(f"{operation} requires a flow or EPS-compatible image model.")
if sigmas.ndim != 1 or len(sigmas) < 2 or not bool(torch.isfinite(sigmas).all()):
raise ValueError(f"{operation} requires a finite forward sampling schedule.")
target = float(sigmas[0])
if target <= MINIMUM_PARTIAL_SIGMA:
raise ValueError(
f"{operation} requires a positive partial-denoise start sigma."
)
if isinstance(sampling, sampling_types.CONST) and target >= 0.9999:
raise ValueError(
f"Flow {operation.lower()} requires denoise below the full-noise endpoint."
)
if isinstance(sampling, sampling_types.EPS):
maximum = float(sampling.sigma_max)
if target > maximum or math.isclose(target, maximum, rel_tol=1e-5):
raise ValueError(
f"{operation} requires partial denoise, below the model's "
"maximum sigma."
)
return target
+11 -4
View File
@@ -185,9 +185,16 @@ def krea2_diffusion_negpip_wrapper(
*args: object,
**kwargs: object,
) -> object:
"""Move a processed Krea sign mask into call-local transformer options."""
"""Inject call-local signs at either supported Comfy Krea argument boundary."""
positional_options = args[5] if len(args) > 5 else None
options_index = (
5
if len(args) > 5
else 4
if len(args) == 5 and isinstance(args[4], dict)
else None
)
positional_options = args[options_index] if options_index is not None else None
transformer_options = (
positional_options
if positional_options is not None
@@ -201,9 +208,9 @@ def krea2_diffusion_negpip_wrapper(
if not isinstance(multiplier, torch.Tensor):
raise TypeError("Krea NegPiP processed mask must be a tensor.")
prepared[TRANSFORMER_MASK_KEY] = multiplier
if len(args) > 5:
if options_index is not None:
prepared_args = list(args)
prepared_args[5] = prepared
prepared_args[options_index] = prepared
return executor(*prepared_args, **kwargs)
kwargs["transformer_options"] = prepared
return executor(*args, **kwargs)
+144
View File
@@ -0,0 +1,144 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Supply model-local NegPiP attention hooks for Comfy's earlier Krea boundary."""
from __future__ import annotations
import logging
from dataclasses import dataclass
from inspect import signature
from typing import Any, cast
import torch
from comfy.ldm.flux.math import apply_rope
from comfy.ldm.krea2.model import Attention
from comfy.ldm.modules.attention import optimized_attention_masked
from comfy.model_patcher import ModelPatcher
from einops import rearrange
from ..model_patcher_mutations import ModelCallableObjectPatchMutation
from .krea2 import TRANSFORMER_MASK_KEY
LOGGER = logging.getLogger(__name__)
def krea2_host_mutations(
model: ModelPatcher,
) -> tuple[ModelCallableObjectPatchMutation, ...]:
"""Adapt only the known five-argument host; retain native reference-capable Krea."""
diffusion = model.get_model_object("diffusion_model")
parameters = tuple(signature(diffusion._forward).parameters)
if "ref_latents" in parameters:
return ()
if parameters != (
"x",
"timesteps",
"context",
"attention_mask",
"transformer_options",
"kwargs",
):
raise ValueError(
f"Krea NegPiP does not support model signature {parameters!r}."
)
mutations: list[ModelCallableObjectPatchMutation] = []
for index, block in enumerate(diffusion.blocks):
attention = block.attn
if not isinstance(attention, Attention):
raise TypeError(f"Krea NegPiP requires host Attention at block {index}.")
mutations.append(
ModelCallableObjectPatchMutation(
f"diffusion_model.blocks.{index}.attn.forward",
Krea2HostAttention(attention, index, len(diffusion.blocks)),
)
)
if not mutations:
raise ValueError("Krea NegPiP requires at least one joint attention block.")
LOGGER.info(
"Krea NegPiP installed model-local host attention hooks",
extra={"blocks": len(mutations)},
)
return tuple(mutations)
@dataclass(frozen=True)
class Krea2HostAttention:
"""Preserve host attention math while exposing its missing pre-RoPE patch point.
Comfy 0.28's public model boundary lacks attention callbacks. Object patches
scope this adapter to the derived MODEL and Comfy restores them on unload.
Text-fusion attention stays untouched; only joint text/image blocks use it.
"""
attention: Any # Comfy Attention exposes dynamically constructed linear modules.
block_index: int
total_blocks: int
def __call__(
self,
x: torch.Tensor,
freqs: torch.Tensor | None = None,
mask: torch.Tensor | None = None,
transformer_options: dict[str, Any] | None = None,
) -> torch.Tensor:
"""Run host QKV projections and attention with local patch metadata."""
options = {} if transformer_options is None else transformer_options.copy()
multiplier = options.get(TRANSFORMER_MASK_KEY)
if multiplier is None:
return cast(
torch.Tensor,
type(self.attention).forward(
self.attention,
x,
freqs,
mask,
transformer_options=options,
),
)
if not isinstance(multiplier, torch.Tensor) or multiplier.ndim != 3:
raise ValueError(
"Krea NegPiP host attention requires a processed token sign mask."
)
options.update(
block_index=self.block_index,
total_blocks=self.total_blocks,
block_type="single",
img_slice=[multiplier.shape[1], x.shape[1]],
)
attention = self.attention
q, k, v, gate = (
attention.wq(x),
attention.wk(x),
attention.wv(x),
attention.gate(x),
)
q = rearrange(q, "B L (H D) -> B H L D", H=attention.heads)
k = rearrange(k, "B L (H D) -> B H L D", H=attention.kvheads)
v = rearrange(v, "B L (H D) -> B H L D", H=attention.kvheads)
q, k = attention.qknorm(q, k)
for patch in options.get("patches", {}).get("attn1_patch", []):
result = patch(
q, k, v, pe=freqs, attn_mask=mask, extra_options=options.copy()
)
q, k, v = result.get("q", q), result.get("k", k), result.get("v", v)
freqs, mask = result.get("pe", freqs), result.get("attn_mask", mask)
if freqs is not None:
q, k = apply_rope(q, k, freqs)
if attention.kvheads != attention.heads:
repeats = attention.heads // attention.kvheads
k = k.repeat_interleave(repeats, dim=1)
v = v.repeat_interleave(repeats, dim=1)
out = optimized_attention_masked(
q,
k,
v,
attention.heads,
mask=mask,
skip_reshape=True,
transformer_options=options,
)
for patch in options.get("patches", {}).get("attn1_output_patch", []):
out = patch(out, options.copy())
return cast(torch.Tensor, attention.wo(out * torch.nn.functional.sigmoid(gate)))
+295
View File
@@ -0,0 +1,295 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Derive source-dependent sampling noise with measured, uncached inversion stages."""
from __future__ import annotations
import time
from collections.abc import Callable
from dataclasses import dataclass
from importlib import import_module
from typing import Any, TypeAlias
import torch
from ..domain.inversion_solver import (
InversionSolverEvidence,
integrate_inversion,
lift_inversion_displacement,
)
from ..domain.noise_inversion import InversionMethod, NoiseInversionOptions
from ..shared.logging import get_logger
from .native_sampling_target import validate_partial_sampling_target
from .spatial_tensor_projection import resize_spatial_tensor
from .tiled_sampling_validation import validate_tensor_shape
LOGGER = get_logger(__name__)
INVERSION_START_SIGMA = 0.0001
InversionModelFactory: TypeAlias = Callable[[torch.Tensor], Any]
@dataclass(frozen=True, slots=True)
class InversionStageMeasurement:
"""Describe actual work and elapsed time for one inversion stage."""
name: str
seconds: float
latent_shape: tuple[int, ...]
steps: int
evaluations: int
@dataclass(frozen=True, slots=True)
class NoiseInversionResult:
"""Return inferred noise with paid inversion cost and reconstruction evidence."""
noise: torch.Tensor
seconds: float
stages: tuple[InversionStageMeasurement, ...]
reconstruction_max_error: float
def _clock(device: torch.device) -> float:
"""Measure completed CUDA work rather than asynchronous kernel submission."""
if device.type == "cuda":
torch.cuda.synchronize(device)
return time.perf_counter()
def _source_in_model_space(model: Any, latent: torch.Tensor) -> torch.Tensor:
"""Narrow host latent processing before arithmetic or model execution."""
source = model.model.process_latent_in(latent)
if not isinstance(source, torch.Tensor) or source.shape != latent.shape:
raise ValueError(
"Model latent processing must preserve the inversion source shape."
)
if not source.is_floating_point() or not bool(torch.isfinite(source).all()):
raise ValueError(
"Model latent processing must produce finite floating-point values."
)
return source.detach().float().cpu()
def invert_sampling_noise(
*,
model: Any,
latent: torch.Tensor,
forward_sigmas: torch.Tensor,
positive: Any,
negative: Any,
cfg: float,
seed: int | None,
options: NoiseInversionOptions,
model_factory: InversionModelFactory | None = None,
noise_mask: Any = None,
) -> NoiseInversionResult:
"""Invert a source latent through Comfy's actual CFG or positive-only guider.
A spatial sampler supplies a factory that replans each resolution from its
unwrapped prepared model. The final noise recreates the inferred endpoint
under the model's own affine noise scaling; no inference cache is used.
"""
from .guided_sampling import sample_with_optional_negative
if not isinstance(options, NoiseInversionOptions):
raise TypeError("Noise inversion requires validated NoiseInversionOptions.")
validate_tensor_shape(latent, sampler_label="Noise Inversion")
if not latent.is_floating_point() or not bool(torch.isfinite(latent).all()):
raise ValueError(
"Noise inversion source must contain finite floating-point values."
)
target = validate_partial_sampling_target(
model, forward_sigmas, operation="Noise inversion"
)
coarse_target = target * options.coarse_target_fraction
if coarse_target <= INVERSION_START_SIGMA:
raise ValueError(
"Noise inversion transition must exceed the initial inversion sigma."
)
device = torch.device(model.load_device)
started = _clock(device)
sampling = model.get_model_object("model_sampling")
source = _source_in_model_space(model, latent)
phases: list[InversionStageMeasurement] = []
comfy_sample = import_module("comfy.sample")
comfy_samplers = import_module("comfy.samplers")
def stage(
stage_latent: torch.Tensor,
begin: float,
end: float,
count: int,
initial: torch.Tensor | None,
name: str,
method: InversionMethod,
) -> torch.Tensor:
"""Capture the model-space endpoint before Comfy converts output latents."""
stage_started = _clock(device)
stage_model = (
model_factory(stage_latent) if model_factory is not None else model
)
schedule = torch.linspace(begin, end, count + 1, device=device)
evidence = InversionSolverEvidence()
endpoints: list[torch.Tensor] = []
def invert(
model_fn: Any,
state: torch.Tensor,
sigmas: torch.Tensor,
extra_args: dict[str, Any],
callback: Any,
disable: bool,
) -> torch.Tensor:
"""Use Comfy's denoised predictions as the inversion velocity field."""
if initial is not None:
state = initial.to(state)
def evaluate(
x: torch.Tensor, sigma: torch.Tensor, index: int
) -> torch.Tensor:
"""Narrow the dynamic Comfy model result before numeric integration."""
prediction = model_fn(
x, sigma * x.new_ones((x.shape[0],)), **extra_args
)
if not isinstance(prediction, torch.Tensor):
raise TypeError(
"Noise inversion model must return tensor predictions."
)
return (x - prediction) / sigma
endpoint = integrate_inversion(
state, sigmas, evaluate, method=method, evidence=evidence
)
endpoints.append(endpoint.detach().float().cpu())
return endpoint
sample_with_optional_negative(
comfy_sample=comfy_sample,
model=stage_model,
noise=torch.zeros_like(stage_latent),
cfg=cfg,
sampler=comfy_samplers.KSAMPLER(invert),
sampler_name=options.method,
sigmas=schedule,
positive=positive,
negative=negative,
latent_image=stage_latent,
noise_mask=noise_mask,
seed=seed,
disable_pbar=True,
)
if len(endpoints) != 1:
raise RuntimeError(
"Noise inversion must produce exactly one endpoint per stage."
)
phases.append(
InversionStageMeasurement(
name,
_clock(device) - stage_started,
tuple(stage_latent.shape),
count,
evidence.evaluations,
)
)
return endpoints[0]
if options.resolution_scale == 1:
endpoint = stage(
latent,
INVERSION_START_SIGMA,
target,
options.steps,
None,
"full",
options.method,
)
else:
height, width = options.coarse_shape(
int(latent.shape[-2]), int(latent.shape[-1])
)
coarse = resize_spatial_tensor(latent, height=height, width=width, mode="area")
coarse_source = _source_in_model_space(model, coarse)
coarse_endpoint = stage(
coarse,
INVERSION_START_SIGMA,
coarse_target,
options.steps,
None,
"coarse",
options.method,
)
endpoint = lift_inversion_displacement(
source,
coarse_source,
coarse_endpoint,
resize=lambda x, h, w: resize_spatial_tensor(
x, height=h, width=w, mode="bilinear"
),
)
if options.finishing_steps:
endpoint = stage(
latent,
coarse_target,
target,
options.finishing_steps,
endpoint,
"full_finish",
options.method,
)
sigma = torch.tensor(target)
zero = torch.zeros_like(source)
base = sampling.noise_scaling(sigma, zero.clone(), source, max_denoise=False)
amplitude = sampling.noise_scaling(
sigma, torch.ones_like(source), zero, max_denoise=False
)
if not isinstance(base, torch.Tensor) or not isinstance(amplitude, torch.Tensor):
raise TypeError("Model noise scaling must return tensors.")
if not bool(torch.isfinite(amplitude).all()) or bool(torch.any(amplitude == 0)):
raise ValueError("Model noise scaling is not invertible at the target sigma.")
noise = (endpoint - base) / amplitude
if not bool(torch.isfinite(noise).all()):
raise FloatingPointError("Noise inversion produced non-finite sampling noise.")
reconstructed = sampling.noise_scaling(
sigma, noise.clone(), source, max_denoise=False
)
if (
not isinstance(reconstructed, torch.Tensor)
or reconstructed.shape != endpoint.shape
):
raise ValueError(
"Model noise scaling must preserve the inversion endpoint shape."
)
if not torch.allclose(reconstructed, endpoint, atol=1e-5, rtol=1e-5):
raise ValueError(
"Model noise scaling cannot reconstruct the inversion endpoint."
)
error = float((reconstructed - endpoint).abs().max())
elapsed = _clock(device) - started
LOGGER.info(
"Noise inversion completed in %.3f seconds",
elapsed,
extra={
"operation": "noise_inversion",
"method": options.method,
"resolution_scale": options.resolution_scale,
"steps": options.steps,
"finishing_steps": options.finishing_steps,
"inversion_seconds": elapsed,
"inversion_stages": [
{
"name": phase.name,
"seconds": phase.seconds,
"latent_shape": phase.latent_shape,
"steps": phase.steps,
"evaluations": phase.evaluations,
}
for phase in phases
],
"evaluations": sum(phase.evaluations for phase in phases),
"endpoint_reconstruction_max_error": error,
},
)
return NoiseInversionResult(noise, elapsed, tuple(phases), error)
@@ -0,0 +1,111 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Advance an already reconstructed state through full-resolution inversion
finishing.
"""
from __future__ import annotations
from importlib import import_module
from typing import Any
import torch
from ..domain.inversion_solver import InversionSolverEvidence, integrate_inversion
from ..domain.noise_reuse import NoiseReuseOptions
from ..shared.logging import get_logger
from .noise_inversion import InversionModelFactory
from .noise_reuse_scope import noise_reuse_scope
LOGGER = get_logger(__name__)
def finish_reused_state(
*,
model: Any,
latent: torch.Tensor,
initial: torch.Tensor,
begin: float,
target: float,
options: NoiseReuseOptions,
positive: Any,
negative: Any,
cfg: float,
seed: int | None,
noise_mask: Any,
model_factory: InversionModelFactory | None,
) -> torch.Tensor:
"""Run only the requested finishing evaluations, using native CFG and
spatial policy.
"""
from .guided_sampling import sample_with_optional_negative
if options.finishing_steps < 1 or begin >= target:
raise ValueError(
"Finishing requires positive steps and a start below the "
"sampler coordinate."
)
device = torch.device(model.load_device)
schedule = torch.linspace(begin, target, options.finishing_steps + 1, device=device)
evidence = InversionSolverEvidence()
endpoints: list[torch.Tensor] = []
def integrate(
model_fn: Any,
state: torch.Tensor,
sigmas: torch.Tensor,
extra_args: dict[str, Any],
callback: Any,
disable: bool,
) -> torch.Tensor:
"""Consume the captured initial state without adding a second layer of noise."""
state = initial.to(state)
def evaluate(x: torch.Tensor, sigma: torch.Tensor, index: int) -> torch.Tensor:
"""Use the production inversion velocity from native denoised
predictions.
"""
prediction = model_fn(x, sigma * x.new_ones((x.shape[0],)), **extra_args)
if not isinstance(prediction, torch.Tensor):
raise TypeError("Finishing model must return tensor predictions.")
return (x - prediction) / sigma
endpoint = integrate_inversion(
state, sigmas, evaluate, method=options.finishing_method, evidence=evidence
)
endpoints.append(endpoint.detach().float().cpu())
return endpoint
stage_model = model_factory(latent) if model_factory is not None else model
with noise_reuse_scope(None):
sample_with_optional_negative(
comfy_sample=import_module("comfy.sample"),
model=stage_model,
noise=torch.zeros_like(latent),
cfg=cfg,
sampler=import_module("comfy.samplers").KSAMPLER(integrate),
sampler_name=options.finishing_method,
sigmas=schedule,
positive=positive,
negative=negative,
latent_image=latent,
noise_mask=noise_mask,
seed=seed,
disable_pbar=True,
)
if len(endpoints) != 1:
raise RuntimeError("Finishing must produce exactly one native endpoint.")
LOGGER.info(
"Captured noise finishing completed",
extra={
"operation": "noise_reuse_finish",
"steps": options.finishing_steps,
"evaluations": evidence.evaluations,
"begin_sigma": begin,
"target_sigma": target,
"coarse_inversion_evaluations": 0,
},
)
return endpoints[0]
+268
View File
@@ -0,0 +1,268 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Resolve strict latent lineage and inject cache-keyed upstream capture
dependencies.
"""
from __future__ import annotations
import json
from collections.abc import Mapping
from dataclasses import dataclass
from typing import Any
from ..domain.graph_provenance import BrokenProvenance, GraphLink
from .comfy_graph_provenance import (
NodeRegistry,
parse_graph_link,
trace_vae_decode_provenance,
)
from .noise_reuse_source import (
capture_graph_context,
describe_graph_node,
resolve_source_sampling_graph,
)
@dataclass(frozen=True, slots=True)
class CaptureConsumer:
"""Describe source capture requirements without storing runtime model objects."""
source: str
receiver: str | None
geometry_json: str
def source_lineage(
prompt: Mapping[str, Any], start: object, registry: NodeRegistry
) -> tuple[str, str]:
"""Trace only latent-preserving resizing and unmodified decoded-image bridges."""
current = parse_graph_link(start)
visited: set[GraphLink] = set()
geometry: list[dict[str, object]] = []
while current is not None and len(visited) < 128:
if current in visited:
raise ValueError("Noise reuse latent provenance contains a cycle.")
visited.add(current)
node = prompt.get(current[0])
if not isinstance(node, Mapping) or current[1] != 0:
raise ValueError(
"Noise reuse source must be a supported sampler's latent output."
)
kind, inputs = node.get("class_type"), node.get("inputs")
if not isinstance(inputs, Mapping):
raise ValueError("Noise reuse source node has missing inputs.")
if resolve_source_sampling_graph(prompt, current[0]) is not None:
return current[0], json.dumps(list(reversed(geometry)))
if kind == "SimpleSyrup.LatentFromImage":
decoded = trace_vae_decode_provenance(prompt, inputs.get("image"), registry)
if isinstance(decoded, BrokenProvenance):
raise ValueError(
f"Noise reuse image provenance is broken: {decoded.reason}."
)
current = decoded.samples_link
continue
if kind in {"SimpleSyrup.SimpleUpscaleLatent", "LatentUpscaleBy"}:
key = (
"scale_factor"
if kind == "SimpleSyrup.SimpleUpscaleLatent"
else "scale_by"
)
scale = inputs.get(key, 2.0 if key == "scale_factor" else 1.5)
geometry.append({"scale": scale})
current = parse_graph_link(
inputs.get("latent" if key == "scale_factor" else "samples")
)
continue
if kind == "LatentUpscale":
if inputs.get("crop", "disabled") != "disabled":
raise ValueError(
"Noise reuse requires latent upscaling with crop disabled."
)
geometry.append({"pixels": [inputs.get("height"), inputs.get("width")]})
current = parse_graph_link(inputs.get("samples"))
continue
class_def = registry.get(str(kind))
rules = getattr(class_def, "GRAPH_PASSTHROUGH_OUTPUTS", None)
if isinstance(rules, Mapping) and isinstance(rules.get(0), str):
current = parse_graph_link(inputs.get(rules[0]))
continue
raise ValueError(
f"Noise reuse cannot resolve latent provenance through "
f"{describe_graph_node(prompt, current[0])}. "
"Use a direct sampler latent, Simple Upscale "
f"Latent, or Latent From Image."
)
raise ValueError("Noise reuse requires a graph-connected source sampler latent.")
def has_noise_reuse(prompt: Mapping[str, Any], start: object) -> bool:
"""Detect potentially enabled reuse without evaluating options or loading models."""
current = parse_graph_link(start)
visited: set[GraphLink] = set()
while current is not None and current not in visited and len(visited) < 128:
visited.add(current)
node = prompt.get(current[0])
if not isinstance(node, Mapping):
return False
inputs = node.get("inputs", {})
if not isinstance(inputs, Mapping):
return False
if (
node.get("class_type") == "SimpleSyrup.NoiseReuseOptions"
and inputs.get("enabled", True) is not False
):
return True
current = parse_graph_link(inputs.get("options"))
return False
def rewrite_noise_reuse_graph(prompt: dict[str, Any], registry: NodeRegistry) -> None:
"""Group receiving requests and redirect every source latent consumer once."""
if any(
isinstance(node, Mapping)
and node.get("class_type")
in {"SimpleSyrup.TrajectoryCaptureModel", "SimpleSyrup.NoiseReuseError"}
for node in prompt.values()
):
return
consumers: list[CaptureConsumer] = []
failures: dict[str, str] = {}
for node_id, node in tuple(prompt.items()):
if not isinstance(node, Mapping):
continue
kind, inputs = node.get("class_type"), node.get("inputs", {})
if not isinstance(inputs, Mapping):
continue
receiver = kind == "SimpleSyrup.KSampler" and has_noise_reuse(
prompt, inputs.get("options")
)
auto = (
kind == "SimpleSyrup.SimpleUpscaleLatent"
and inputs.get("method", "Auto") == "Auto"
)
if not receiver and not auto:
continue
try:
source, geometry = source_lineage(
prompt, inputs.get("latent_image" if receiver else "latent"), registry
)
consumers.append(
CaptureConsumer(source, node_id if receiver else None, geometry)
)
except ValueError as error:
if receiver:
failures[node_id] = (
f"Noise Reuse Options on receiving sampler "
f"{describe_graph_node(prompt, node_id)} cannot reach a usable "
f"source sampler. {error}"
)
for source in sorted({consumer.source for consumer in consumers}):
_inject_source(
prompt,
source,
[consumer for consumer in consumers if consumer.source == source],
)
for node_id, message in failures.items():
error_id = _unique_id(prompt, f"{node_id}.noise_reuse_error")
prompt[error_id] = {
"class_type": "SimpleSyrup.NoiseReuseError",
"inputs": {"message": message},
}
inputs = prompt[node_id]["inputs"]
inputs[
"latent_image"
if prompt[node_id]["class_type"] == "SimpleSyrup.KSampler"
else "latent"
] = [error_id, 0]
def _inject_source(
prompt: dict[str, Any], source: str, consumers: list[CaptureConsumer]
) -> None:
"""Put receiver settings in the source MODEL dependency chain, then attach
its result.
"""
source_inputs = prompt[source]["inputs"]
source_graph = resolve_source_sampling_graph(prompt, source)
if source_graph is None:
raise ValueError(
f"Source {describe_graph_node(prompt, source)} is not a sampler."
)
request: list[object] | None = None
for consumer in consumers:
if consumer.receiver is None:
continue
receiver = prompt[consumer.receiver]["inputs"]
request_id = _unique_id(prompt, f"{consumer.receiver}.noise_capture_request")
values = {
key: receiver[key]
for key in ("model", "steps", "sampler_name", "scheduler")
}
values.update(
options=receiver["options"],
denoise=receiver.get("denoise", 1.0),
source_latent=source_graph.latent_input,
geometry_json=consumer.geometry_json,
)
if request is not None:
values["previous"] = request
prompt[request_id] = {
"class_type": "SimpleSyrup.NoiseCaptureRequest",
"inputs": values,
}
request = [request_id, 0]
capture_id = _unique_id(prompt, f"{source}.trajectory_capture")
model_inputs = prompt[source_graph.model_owner]["inputs"]
if source_graph.model_owner != source:
guider_id = _unique_id(prompt, f"{source}.trajectory_guider")
guider_node = prompt[source_graph.model_owner].copy()
model_inputs = model_inputs.copy()
guider_node["inputs"] = model_inputs
prompt[guider_id] = guider_node
guider_link = parse_graph_link(source_inputs["guider"])
if guider_link is None:
raise ValueError("Source sampler has an invalid guider connection.")
source_inputs["guider"] = [guider_id, guider_link[1]]
values = {
"model": model_inputs["model"],
"sampler_name": source_graph.sampler_name,
"graph_context": capture_graph_context(
prompt,
source,
[
consumer.receiver
for consumer in consumers
if consumer.receiver is not None
],
),
}
if source_graph.sampler is not None:
values["sampler"] = source_graph.sampler
if request is not None:
values["requests"] = request
prompt[capture_id] = {
"class_type": "SimpleSyrup.TrajectoryCaptureModel",
"inputs": values,
}
model_inputs["model"] = [capture_id, 0]
attach_id = _unique_id(prompt, f"{source}.sampling_provenance")
for node in prompt.values():
inputs = node.get("inputs", {})
for key, value in tuple(inputs.items()):
if parse_graph_link(value) == (source, 0):
inputs[key] = [attach_id, 0]
prompt[attach_id] = {
"class_type": "SimpleSyrup.AttachSamplingProvenance",
"inputs": {"latent": [source, 0], "capture": [capture_id, 1]},
}
def _unique_id(prompt: Mapping[str, Any], base: str) -> str:
"""Avoid collisions with user-authored or earlier injected graph identifiers."""
result = base
while result in prompt:
result += "_"
return result
@@ -0,0 +1,91 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Prepare captured residuals for the shared native sampler starting-noise input."""
from __future__ import annotations
from typing import Any
import torch
from ..domain.spectral_noise import lift_residual
from ..shared.logging import get_logger
from .native_noise_scaling import noise_coefficients, recover_residual
from .native_sampling_target import validate_partial_sampling_target
from .noise_inversion import InversionModelFactory
from .noise_reuse_finishing import finish_reused_state
from .noise_reuse_scope import NoiseReuseExecution
LOGGER = get_logger(__name__)
def prepare_reused_noise(
*,
execution: NoiseReuseExecution,
model: Any,
fresh: torch.Tensor,
latent: torch.Tensor,
sigmas: torch.Tensor,
positive: Any,
negative: Any,
cfg: float,
seed: int | None,
noise_mask: Any,
model_factory: InversionModelFactory | None,
) -> tuple[torch.Tensor, torch.Tensor]:
"""Lift source coefficients and return the expected actual model-space
starting state.
"""
if execution.consumed:
raise ValueError(
"Noise reuse supports one native sampling pass per "
"receiver; use separate samplers for conditioning "
"batches."
)
execution.consumed = True
target = validate_partial_sampling_target(model, sigmas, operation="Noise reuse")
coordinate = execution.options.capture_coordinate(target)
provenance = execution.provenance
if type(model.model.latent_format).__name__ != provenance.latent_format:
raise ValueError(
"Noise reuse source and receiving model use different latent formats."
)
scaling = model.get_model_object("model_sampling")
capture = provenance.capture_at(coordinate)
residual = capture.residual
lifted = lift_residual(residual, fresh.detach().float().cpu())
clean = model.model.process_latent_in(latent).detach().float().cpu()
if execution.options.finishing_steps:
base, amplitude = noise_coefficients(scaling, clean, coordinate)
endpoint = finish_reused_state(
model=model,
latent=latent,
initial=base + amplitude * lifted,
begin=coordinate,
target=target,
options=execution.options,
positive=positive,
negative=negative,
cfg=cfg,
seed=seed,
noise_mask=noise_mask,
model_factory=model_factory,
)
lifted = recover_residual(scaling, clean, endpoint, target)
base, amplitude = noise_coefficients(scaling, clean, target)
expected = base + amplitude * lifted
LOGGER.info(
"Captured sampling noise prepared",
extra={
"operation": "noise_reuse",
"capture_sigma": coordinate,
"actual_capture_sigma": capture.actual_coordinate,
"target_sigma": target,
"source_shape": tuple(residual.shape),
"target_shape": tuple(lifted.shape),
"finishing_steps": execution.options.finishing_steps,
},
)
return lifted.to(fresh), expected
+48
View File
@@ -0,0 +1,48 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Own one request-scoped noise handoff across existing sampler routing."""
from __future__ import annotations
from collections.abc import Iterator
from contextlib import contextmanager
from contextvars import ContextVar
from dataclasses import dataclass
from ..domain.noise_reuse import NoiseReuseOptions, SamplingProvenance
@dataclass(slots=True)
class NoiseReuseExecution:
"""Keep a single receiver's immutable capture and consumption lifecycle."""
options: NoiseReuseOptions
provenance: SamplingProvenance
consumed: bool = False
_CURRENT: ContextVar[NoiseReuseExecution | None] = ContextVar(
"simple_syrup_noise_reuse", default=None
)
def current_noise_reuse() -> NoiseReuseExecution | None:
"""Resolve only the receiving service's current execution, never cached globals."""
return _CURRENT.get()
def needs_finishing_model() -> bool:
"""Request a spatial inversion factory only for enabled finishing work."""
current = _CURRENT.get()
return current is not None and current.options.finishing_steps > 0
@contextmanager
def noise_reuse_scope(execution: NoiseReuseExecution | None) -> Iterator[None]:
"""Restore the caller's context even when validation or native inference fails."""
token = _CURRENT.set(execution)
try:
yield
finally:
_CURRENT.reset(token)
+100
View File
@@ -0,0 +1,100 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Adapt source sampling graphs through their shared model and latent contracts."""
from __future__ import annotations
from collections.abc import Mapping
from dataclasses import dataclass
from typing import Any
from .comfy_graph_provenance import parse_graph_link
@dataclass(frozen=True, slots=True)
class SourceSamplingGraph:
"""Describe the exact graph dependencies needed to observe one source sampler."""
model_owner: str
latent_input: object
sampler_name: object
sampler: object | None = None
def resolve_source_sampling_graph(
prompt: Mapping[str, Any], node_id: str
) -> SourceSamplingGraph | None:
"""Recognize named and custom samplers by their host-facing sampling inputs."""
node = prompt.get(node_id)
if not isinstance(node, Mapping):
return None
inputs = node.get("inputs")
if not isinstance(inputs, Mapping) or "latent_image" not in inputs:
return None
if "model" in inputs and "sampler_name" in inputs:
return SourceSamplingGraph(
node_id, inputs["latent_image"], inputs["sampler_name"]
)
if "sampler" not in inputs or "sigmas" not in inputs:
return None
owner = node_id
if "model" not in inputs:
guider = parse_graph_link(inputs.get("guider"))
producer = prompt.get(guider[0]) if guider is not None else None
if not isinstance(producer, Mapping):
return None
assert guider is not None
guider_inputs = producer.get("inputs")
if not isinstance(guider_inputs, Mapping) or "model" not in guider_inputs:
raise ValueError(
"The source custom sampler's guider must expose its connected model "
"so Noise Reuse Options can observe that source run."
)
owner = guider[0]
link = parse_graph_link(inputs["sampler"])
selector = prompt.get(link[0]) if link is not None else None
selector_inputs = (
selector.get("inputs", {}) if isinstance(selector, Mapping) else {}
)
name = (
selector_inputs.get("sampler_name", "")
if isinstance(selector_inputs, Mapping)
else ""
)
return SourceSamplingGraph(owner, inputs["latent_image"], name, inputs["sampler"])
def describe_graph_node(prompt: Mapping[str, Any], node_id: str) -> str:
"""Name user titles, node types and graph IDs together in actionable errors."""
node = prompt.get(node_id, {})
if not isinstance(node, Mapping):
return f"missing node {node_id}"
kind = str(node.get("class_type", "unknown node"))
metadata = node.get("_meta", {})
title = metadata.get("title") if isinstance(metadata, Mapping) else None
return (
f"'{title}' ({kind}, node {node_id})"
if isinstance(title, str) and title
else f"{kind} (node {node_id})"
)
def capture_graph_context(
prompt: Mapping[str, Any], source: str, receivers: list[str]
) -> str:
"""Explain which downstream option reaches back into the source graph."""
if not receivers:
return (
"Latent format discovery is observing source "
f"{describe_graph_node(prompt, source)}."
)
downstream = "; ".join(
describe_graph_node(prompt, node_id) for node_id in receivers
)
return (
f"Noise Reuse Options on receiving sampler {downstream} reaches back into "
f"source sampler {describe_graph_node(prompt, source)} "
"to save its original noise."
)
+94
View File
@@ -0,0 +1,94 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Verify reused starting states at native scaling before any sampler algorithm runs."""
from __future__ import annotations
from collections.abc import Callable
from typing import Any
import torch
from ..shared.logging import get_logger
from .model_patcher_mutations import ModelCallableObjectPatchMutation
from .patcher_lifecycle import PATCHER_LIFECYCLE
LOGGER = get_logger(__name__)
class NoiseReuseStartVerification:
"""Check the first native scaling result without changing callbacks or tensors."""
def __init__(
self,
original: Callable[..., torch.Tensor],
expected: torch.Tensor,
*,
graph_context: str,
sampler_name: str,
) -> None:
"""Retain one receiver's expected state and actionable graph identity."""
self.original = original
self.expected = expected
self.graph_context = graph_context
self.sampler_name = sampler_name
self.verified = False
def __call__(
self,
sigma: torch.Tensor,
noise: torch.Tensor,
latent_image: torch.Tensor,
max_denoise: bool = False,
) -> torch.Tensor:
"""Return native scaling unchanged after validating its first result."""
result = self.original(sigma, noise, latent_image, max_denoise)
if not self.verified:
if result.shape != self.expected.shape:
raise ValueError(self._message("produced a different latent shape"))
error = float((result.detach().float().cpu() - self.expected).abs().max())
if not bool(torch.isfinite(result).all()) or error > 2e-5:
raise ValueError(
self._message(
f"changed the reused starting latent (error {error:.6g})"
)
)
self.verified = True
LOGGER.info(
"Noise reuse starting state verified",
extra={
"operation": "noise_reuse_start",
"sampler_name": self.sampler_name,
"graph_context": self.graph_context,
"max_error": error,
},
)
return result
def require_verified(self) -> None:
"""Reject receiver execution paths that bypass native starting-noise scaling."""
if not self.verified:
raise ValueError(
self._message(
"is not supported by noise reuse on this execution path: "
"it did not expose native starting-noise scaling"
)
)
def _message(self, reason: str) -> str:
"""Identify the graph traversal and selected receiving algorithm plainly."""
return (
f"{self.graph_context} The receiving sampler '{self.sampler_name}' "
f"{reason}. Choose a sampler node that uses "
"ComfyUI's native sampling boundary."
)
def derive_model(self, model: Any) -> Any:
"""Install verification through the authoritative MODEL lifecycle."""
return PATCHER_LIFECYCLE.derive_model(
model,
(ModelCallableObjectPatchMutation("model_sampling.noise_scaling", self),),
operation="noise_reuse_start",
)
@@ -0,0 +1,87 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Prepare capture at Comfy's native validation boundary for every queue caller."""
from __future__ import annotations
from collections.abc import Awaitable, Callable
from functools import wraps
from importlib import import_module
from typing import Any, Protocol, cast
from ..shared.logging import get_logger
from .noise_reuse_graph import rewrite_noise_reuse_graph
LOGGER = get_logger(__name__)
_MARKER = "_simple_syrup_noise_reuse_validation"
NativePromptValidator = Callable[
[str, dict[str, Any], list[str] | None],
Awaitable[tuple[bool, object, object, object]],
]
class NativeValidationModule(Protocol):
"""Describe the host's replaceable async prompt validation boundary."""
validate_prompt: NativePromptValidator
def prepare_noise_reuse_prompt(prompt: dict[str, Any]) -> None:
"""Install cache-keyed capture dependencies before source sampling executes."""
rewrite_noise_reuse_graph(prompt, import_module("nodes").NODE_CLASS_MAPPINGS)
class NoiseReuseValidationAdapter:
"""Own capture preparation while preserving the host validator's contract."""
def __init__(self, validator: NativePromptValidator) -> None:
"""Retain the current validator, including other installed extensions."""
self._validator = validator
def wrap(self) -> NativePromptValidator:
"""Preserve the host's async function identity and introspection metadata."""
@wraps(self._validator)
async def validate_prompt(
prompt_id: str,
prompt: dict[str, Any],
partial_execution_list: list[str] | None,
) -> tuple[bool, object, object, object]:
"""Delegate through this adapter without changing the native signature."""
return await self(prompt_id, prompt, partial_execution_list)
setattr(validate_prompt, _MARKER, True)
return validate_prompt
async def __call__(
self,
prompt_id: str,
prompt: dict[str, Any],
partial_execution_list: list[str] | None,
) -> tuple[bool, object, object, object]:
"""Prepare once before native validation and propagate its exact result."""
try:
prepare_noise_reuse_prompt(prompt)
except Exception:
LOGGER.exception(
"Noise reuse capture preparation failed before Comfy validation",
extra={"prompt_id": prompt_id, "operation": "prepare_capture"},
)
raise
return await self._validator(prompt_id, prompt, partial_execution_list)
def register_noise_reuse_validation() -> None:
"""Install once on the shared boundary used by HTTP and direct graph queues."""
try:
execution = cast(NativeValidationModule, import_module("execution"))
except ModuleNotFoundError as error:
if error.name != "execution":
raise
return
validator = execution.validate_prompt
if not getattr(validator, _MARKER, False):
execution.validate_prompt = NoiseReuseValidationAdapter(validator).wrap()
LOGGER.info("Installed noise reuse capture preparation before Comfy validation")
@@ -10,6 +10,7 @@ from typing import Any
import torch
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.regional_detailing import LatentRegion
from . import regional_multidiffusion_sampling
from .detail_previews import DetailPreviewContext
@@ -51,6 +52,7 @@ class RegionalDetailSampler:
global_prompt_weight: float,
preview_context: DetailPreviewContext | None = None,
differential_diffusion: bool = False,
noise_inversion: NoiseInversionOptions | None = None,
) -> Latent:
"""Sample one full latent with regional MultiDiffusion."""
@@ -69,4 +71,5 @@ class RegionalDetailSampler:
global_prompt_weight=global_prompt_weight,
preview_context=preview_context,
differential_diffusion=differential_diffusion,
noise_inversion=noise_inversion,
)
@@ -0,0 +1,66 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Rebuild regional prediction ownership from canonical inputs for inversion stages."""
from __future__ import annotations
from typing import Any
import torch
from ..domain.regional_detailing import LatentRegion
from ..domain.regional_inversion_geometry import project_inversion_regions
from .inversion_model_factory import InversionModelFactory
class RegionalInversionModelFactory:
"""Retain one original MODEL and the full-resolution region bank."""
def __init__(
self,
*,
model: Any,
canvas_width: int,
canvas_height: int,
regions: tuple[LatentRegion, ...],
global_prompt_weight: float,
differential_diffusion: bool,
) -> None:
"""Keep regional conditioning unchanged across source-sized inversion views."""
self._base = InversionModelFactory(
model=model,
canvas_width=canvas_width,
canvas_height=canvas_height,
)
self._width = canvas_width
self._height = canvas_height
self._regions = regions
self._weight = global_prompt_weight
self._differential = differential_diffusion
def __call__(self, latent: torch.Tensor) -> Any:
"""Use the authoritative regional calc-cond-batch wrapper at each stage size."""
from .regional_multidiffusion_sampling import (
clone_model_with_regional_multidiffusion,
)
height, width = int(latent.shape[-2]), int(latent.shape[-1])
regions = project_inversion_regions(
self._regions,
source_width=self._width,
source_height=self._height,
target_width=width,
target_height=height,
)
derived, _ = clone_model_with_regional_multidiffusion(
self._base(latent),
latent_width=width,
latent_height=height,
latent_ndim=latent.ndim,
regions=regions,
global_prompt_weight=self._weight,
differential_diffusion=self._differential,
)
return derived
@@ -15,6 +15,7 @@ from importlib import import_module
from types import ModuleType
from typing import Any, cast
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.regional_detailing import LatentRegion
from ..domain.regional_features import EMPTY_REGIONAL_CAPABILITY_ADMISSION
from ..shared.logging import get_logger
@@ -26,7 +27,9 @@ from .differential_diffusion import (
)
from .guided_sampling import sample_with_optional_negative
from .model_patcher_mutations import ModelCalcCondBatchMutation
from .noise_reuse_scope import needs_finishing_model
from .patcher_lifecycle import PATCHER_LIFECYCLE, ModelMutation
from .regional_inversion_model_factory import RegionalInversionModelFactory
from .regional_multidiffusion_prediction import (
CalcCondBatchFunction,
RegionalMultiDiffusionCalcCondBatch,
@@ -41,7 +44,6 @@ from .tiled_sampling_validation import (
LOGGER = get_logger(__name__)
SAMPLER_LABEL = "Regional MultiDiffusion"
UNIPC_SAMPLERS = frozenset({"uni_pc", "uni_pc_bh2"})
@dataclass(frozen=True)
@@ -72,6 +74,7 @@ def sample_regional_multidiffusion(
global_prompt_weight: float,
preview_context: DetailPreviewContext | None = None,
differential_diffusion: bool = False,
noise_inversion: NoiseInversionOptions | None = None,
) -> Latent:
"""Sample a latent with regional MultiDiffusion prompt blending."""
@@ -80,7 +83,6 @@ def sample_regional_multidiffusion(
denoise=denoise,
global_prompt_weight=global_prompt_weight,
)
_reject_unipc_sampler(sampler_name)
if not regions:
raise ValueError("Regional MultiDiffusion requires at least one region.")
reject_unsupported_conditioning(
@@ -151,6 +153,7 @@ def sample_regional_multidiffusion(
noise=noise,
cfg=cfg,
sampler=sampler,
sampler_name=sampler_name,
sigmas=sigmas,
positive=positive,
negative=negative,
@@ -159,6 +162,19 @@ def sample_regional_multidiffusion(
callback=callback,
disable_pbar=not comfy_utils.PROGRESS_BAR_ENABLED,
seed=seed,
noise_inversion=noise_inversion,
inversion_model_factory=(
RegionalInversionModelFactory(
model=model,
canvas_width=latent_width,
canvas_height=latent_height,
regions=regions,
global_prompt_weight=global_prompt_weight,
differential_diffusion=differential_diffusion,
)
if noise_inversion is not None or needs_finishing_model()
else None
),
)
LOGGER.info(
@@ -266,15 +282,6 @@ def _validate_global_prompt_weight(global_prompt_weight: float) -> None:
raise ValueError("global_prompt_weight must be between 0.0 and 1.0.")
def _reject_unipc_sampler(sampler_name: str) -> None:
"""Reject UniPC samplers because MultiDiffusion is incompatible with them."""
if sampler_name in UNIPC_SAMPLERS:
raise ValueError(
"Regional MultiDiffusion is not compatible with UniPC samplers."
)
def _sampling_callback(
model: Any,
steps: int,
@@ -0,0 +1,96 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Adapt SesquiLSR inference to Comfy external latent layouts and device management."""
from __future__ import annotations
from importlib import import_module
from pathlib import Path
from typing import Any
import torch
from ..domain.latent_upscale import learned_upscale_stages
from ..shared.logging import get_logger
LOGGER = get_logger(__name__)
_CHANNELS = {"sdxl": 4, "flux": 16, "wan": 16, "flux2": 32}
def upscale_with_sesqui(
samples: torch.Tensor, *, path: Path, family: str, target: tuple[int, int]
) -> torch.Tensor:
"""Run float32 learned stages in raw VAE latent units, without image decoding."""
module: Any = import_module("simple_syrup.third_party.sesqui_lsr")
management = import_module("comfy.model_management")
utils = import_module("comfy.utils")
channels = _CHANNELS[family]
external_channels = (
128 if family == "flux2" and samples.shape[1] == 128 else channels
)
if samples.ndim not in (4, 5) or samples.shape[1] != external_channels:
raise ValueError(
f"SesquiLSR {family} requires BCHW or BCTHW latents with "
f"{external_channels} channels."
)
if not samples.is_floating_point() or not bool(torch.isfinite(samples).all()):
raise ValueError("Latent upscaling requires finite floating-point samples.")
network = module.LatentUpscaler(in_channels=channels)
network.load_state_dict(
utils.load_torch_file(str(path), safe_load=True), strict=True
)
device = management.get_torch_device()
management.free_memory(
management.module_size(network) + samples.numel() * 64, device
)
network.eval().requires_grad_(False).to(device=device, dtype=torch.float32)
batch, frames = (
int(samples.shape[0]),
int(samples.shape[2]) if samples.ndim == 5 else 1,
)
flat = (
samples.permute(0, 2, 1, 3, 4).reshape(
batch * frames, external_channels, *samples.shape[-2:]
)
if samples.ndim == 5
else samples
)
adaptor = (
module.make_flux2()
if external_channels == 128
else module.make_identity(channels)
)
current: torch.Tensor = adaptor.to_vae_latent(
flat.to(device=device, dtype=torch.float32)
)
vae_target = adaptor.vae_target_size(target)
stages = learned_upscale_stages(
(int(current.shape[-2]), int(current.shape[-1])), vae_target
)
try:
with torch.inference_mode():
for size in stages:
current = network(current, size)
output: torch.Tensor = adaptor.from_vae_latent(current.float()).to(samples)
finally:
network.to(management.unet_offload_device())
if output.shape[-2:] != target:
raise RuntimeError(
"Learned latent upscaler did not produce the requested dimensions."
)
if samples.ndim == 5:
output = output.reshape(batch, frames, external_channels, *target).permute(
0, 2, 1, 3, 4
)
LOGGER.info(
"Learned latent upscale completed",
extra={
"operation": "latent_upscale",
"family": family,
"model": path.name,
"stages": stages,
"output_shape": tuple(output.shape),
},
)
return output
+252
View File
@@ -0,0 +1,252 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Observe native source sampling through solver-independent Comfy boundaries."""
from __future__ import annotations
import math
from dataclasses import dataclass, field
from importlib import import_module
from typing import Any
import torch
from ..domain.noise_reuse import (
PROVENANCE_KEY,
CapturedSamplingState,
SamplingProvenance,
)
from ..shared.logging import get_logger
from .model_patcher_mutations import ModelKeyedWrapperMutation
from .native_noise_scaling import recover_residual
from .patcher_lifecycle import PATCHER_LIFECYCLE
LOGGER = get_logger(__name__)
@dataclass(frozen=True, slots=True)
class ObservedSamplingState:
"""Retain one real model-space observation without manufacturing a timestep."""
coordinate: float
state: torch.Tensor
@dataclass(slots=True)
class TrajectoryCapture:
"""Own bounded source observations without changing its solver or schedule."""
model: Any
sampler_name: str
coordinates: tuple[float, ...]
graph_context: str = ""
states: dict[float, ObservedSamplingState] = field(default_factory=dict)
clean: torch.Tensor | None = None
schedule: torch.Tensor | None = None
_active: bool = False
def __post_init__(self) -> None:
"""Normalize requests to the receiving native float32 coordinate precision."""
if any(not math.isfinite(value) or value <= 0 for value in self.coordinates):
raise ValueError("Requested noise levels must be finite and above zero.")
self.coordinates = tuple(
sorted(
{float(torch.tensor(value)) for value in self.coordinates}, reverse=True
)
)
def prepare_model(self) -> Any:
"""Install one observer for every solver using Comfy's sampling contract."""
try:
return self._prepare_model()
except (TypeError, ValueError) as error:
raise type(error)(self._failure_message(error)) from error
def _prepare_model(self) -> Any:
"""Derive the capture model through the authoritative patcher lifecycle."""
if not self.coordinates:
return self.model
wrappers = import_module("comfy.patcher_extension").WrappersMP
return PATCHER_LIFECYCLE.derive_model(
self.model,
(
ModelKeyedWrapperMutation(
wrappers.SAMPLER_SAMPLE,
"simple_syrup.trajectory_capture",
self.observe,
),
ModelKeyedWrapperMutation(
wrappers.PREDICT_NOISE,
"simple_syrup.trajectory_state",
self.observe_prediction,
),
),
operation="SimpleSyrup trajectory capture",
)
def observe(
self, executor: Any, *arguments: Any, **keyword_arguments: Any
) -> torch.Tensor:
"""Scope observations to one unchanged native sampler execution and endpoint."""
try:
return self._observe(executor, *arguments, **keyword_arguments)
except (TypeError, ValueError) as error:
raise type(error)(self._failure_message(error)) from error
def _observe(
self, executor: Any, *arguments: Any, **keyword_arguments: Any
) -> torch.Tensor:
"""Retain actual states and release observation scope on every exit."""
schedule = (
arguments[1] if len(arguments) > 1 else keyword_arguments.get("sigmas")
)
if not isinstance(schedule, torch.Tensor) or schedule.ndim != 1:
raise ValueError("The original sampler did not provide its noise schedule.")
if not bool(torch.isfinite(schedule).all()) or bool((schedule < 0).any()):
raise ValueError("The original sampler provided an invalid noise schedule.")
self.states.clear()
self.clean = None
self.schedule = schedule.detach().float().cpu().clone()
self._active = True
try:
result = executor(*arguments, **keyword_arguments)
if not isinstance(result, torch.Tensor):
raise TypeError("The original sampler did not return a latent tensor.")
self.clean = result.detach().float().cpu().clone()
return result
finally:
self._active = False
def observe_prediction(
self, executor: Any, *arguments: Any, **keyword_arguments: Any
) -> torch.Tensor:
"""Save real input states at actual solver coordinates, including substeps."""
if self._active:
state = arguments[0] if arguments else keyword_arguments.get("x")
sigma = (
arguments[1]
if len(arguments) > 1
else keyword_arguments.get("timestep")
)
if not isinstance(state, torch.Tensor) or not isinstance(
sigma, torch.Tensor
):
raise TypeError("The original sampler did not expose its noisy latent.")
values = sigma.detach().float().flatten()
if values.numel() == 0 or not bool(torch.isfinite(values).all()):
raise ValueError(
"The original sampler reported an invalid noise level."
)
if not bool((values == values[0]).all()):
raise ValueError(
"Noise reuse requires one source noise level per batch."
)
coordinate = float(values[0])
if coordinate > 0:
self._record(state, coordinate)
result = executor(*arguments, **keyword_arguments)
if not isinstance(result, torch.Tensor):
raise TypeError(
"The original sampler's model returned an invalid prediction."
)
return result
def _record(self, state: torch.Tensor, coordinate: float) -> None:
"""Prefer exact matches, then the closest real level with deterministic ties."""
selected: torch.Tensor | None = None
for requested in self.coordinates:
previous = self.states.get(requested)
rank = (abs(coordinate - requested), -coordinate)
if previous is not None and rank >= (
abs(previous.coordinate - requested),
-previous.coordinate,
):
continue
if selected is None:
selected = state.detach().float().cpu().clone()
if not bool(torch.isfinite(selected).all()):
raise ValueError(
"The original sampler produced an invalid noisy latent."
)
self.states[requested] = ObservedSamplingState(coordinate, selected)
def attach(self, latent: dict[str, Any]) -> dict[str, Any]:
"""Surface capture failures with the original graph identities and algorithm."""
try:
return self._attach(latent)
except (TypeError, ValueError) as error:
raise type(error)(self._failure_message(error)) from error
def _failure_message(self, error: Exception) -> str:
"""Identify the graph reach-through and selected algorithm in plain English."""
LOGGER.error(
"Source noise handoff failed",
extra={
"sampler": self.sampler_name,
"graph_context": self.graph_context,
"operation": "trajectory_capture",
"reason": str(error),
},
)
return (
f"{self.graph_context} Chosen source sampler: '{self.sampler_name}'. "
f"{error}"
).strip()
def _attach(self, latent: dict[str, Any]) -> dict[str, Any]:
"""Associate observations with their source output and parameterization."""
samples = latent.get("samples")
if not isinstance(samples, torch.Tensor):
raise TypeError("The original sampler output must contain a latent tensor.")
actual = self.model.model.process_latent_in(samples).detach().float().cpu()
if self.coordinates:
if (
self.clean is None
or self.schedule is None
or set(self.states) != set(self.coordinates)
):
raise ValueError(
f"The original sampler '{self.sampler_name}' "
"is not supported by noise reuse on this execution path: "
"it finished without exposing the original noisy latent. "
"Run the complete connected workflow again; if this repeats, "
"that source node must expose ComfyUI's sampling boundaries."
)
if not torch.allclose(actual, self.clean, atol=1e-5, rtol=1e-5):
raise ValueError(
"Saved noise belongs to a different original sampler output."
)
actual = self.clean
scaling = (
self.model.get_model_object("model_sampling") if self.coordinates else None
)
captures = tuple(
CapturedSamplingState(
requested,
observed.coordinate,
observed.state,
recover_residual(scaling, actual, observed.state, observed.coordinate),
)
for requested, observed in self.states.items()
)
result = latent.copy()
result[PROVENANCE_KEY] = SamplingProvenance(
type(self.model.model.latent_format).__name__,
actual,
captures,
self.schedule,
self.graph_context,
)
LOGGER.info(
"Sampling provenance attached",
extra={
"operation": "trajectory_capture",
"sampler": self.sampler_name,
"requested_coordinates": self.coordinates,
"actual_coordinates": tuple(c.actual_coordinate for c in captures),
"source_shape": tuple(actual.shape),
},
)
return result
@@ -12,6 +12,7 @@ from ..domain.attention_coupling_request import (
AttentionCouplingRequestMode,
classify_attention_coupling_request,
)
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.regional_attention_execution import RegionalAttentionExecutionMode
from .attention_coupling_model_preparation_service import (
AttentionCouplingModelPreparationService,
@@ -45,6 +46,7 @@ class AttentionCouplingSamplingService:
region_mask_feather: int,
latent_image: dict[str, Any],
denoise: float,
noise_inversion: NoiseInversionOptions | None = None,
) -> dict[str, Any]:
"""Bypass ordinary requests or prepare one complete regional request."""
@@ -65,6 +67,7 @@ class AttentionCouplingSamplingService:
negative=negative,
latent_image=latent_image,
denoise=denoise,
noise_inversion=noise_inversion,
)
prepared = self.model_preparation_service_class().prepare(
@@ -88,6 +91,7 @@ class AttentionCouplingSamplingService:
negative=prepared.negative,
latent_image=latent_image,
denoise=denoise,
noise_inversion=noise_inversion,
)
@@ -8,8 +8,10 @@ from __future__ import annotations
from typing import Any, ClassVar
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.regional_attention_execution import RegionalAttentionExecutionMode
from ..domain.regional_features import RegionalFeature, RegionalFeatureRequest
from ..domain.sampler_options import TilingOptions
from .attention_coupling_model_preparation_service import (
AttentionCouplingModelPreparationService,
)
@@ -57,6 +59,8 @@ class ContextualAttentionCouplingSamplingService:
global_steps: int,
global_decay: float,
segs: object | None = None,
noise_inversion: NoiseInversionOptions | None = None,
tiling: TilingOptions | None = None,
) -> ContextualDiffusionSamplingResult:
"""Prepare once and invoke established Contextual local-view execution."""
@@ -94,6 +98,8 @@ class ContextualAttentionCouplingSamplingService:
region_mask_feather=0,
feature_request=_CONTEXTUAL_ATTENTION_REQUEST,
planning_region_masks=prepared.mask_bank.planning_masks,
noise_inversion=noise_inversion,
tiling=tiling,
)
@@ -21,6 +21,7 @@ from ..domain.contextual_diffusion import (
ContextualDiffusionControls,
build_contextual_diffusion_plan,
)
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.regional_features import (
CONTEXTUAL_DIFFUSION_REGIONAL_SAMPLER_CAPABILITIES,
EMPTY_REGIONAL_FEATURE_REQUEST,
@@ -28,6 +29,7 @@ from ..domain.regional_features import (
RegionalFeature,
RegionalFeatureRequest,
)
from ..domain.sampler_options import TilingOptions
from ..domain.segs import NativeSegs, coerce_segs_group
from ..domain.tiled_diffusion import validate_tiled_diffusion_mode
from ..runtime.contextual_diffusion_sampling import sample_contextual_diffusion
@@ -91,10 +93,16 @@ class ContextualDiffusionSamplingService:
region_mask_feather: int = 0,
feature_request: RegionalFeatureRequest = EMPTY_REGIONAL_FEATURE_REQUEST,
planning_region_masks: torch.Tensor | None = None,
noise_inversion: NoiseInversionOptions | None = None,
tiling: TilingOptions | None = None,
) -> ContextualDiffusionSamplingResult:
"""Sample a latent with global and bounded detail contexts."""
"""Sample one local tile authority with global context and inversion."""
validate_tiled_diffusion_mode(diffusion_mode)
if tiling is not None:
diffusion_mode = tiling.diffusion_mode
latent_context_overlap = tiling.overlap
latent_context_batch_size = tiling.batch_size
controls = ContextualDiffusionControls(
latent_context_size=latent_context_size,
latent_context_overlap=latent_context_overlap,
@@ -102,6 +110,8 @@ class ContextualDiffusionSamplingService:
global_weight=global_weight,
global_steps=global_steps,
global_decay=global_decay,
latent_tile_width=tiling.width if tiling is not None else None,
latent_tile_height=tiling.height if tiling is not None else None,
)
controls.validate()
regional = self.regional_preparation_service_class().prepare(
@@ -172,6 +182,10 @@ class ContextualDiffusionSamplingService:
image_width=image_width,
region_masks=planning_masks,
capability_admission=capability_admission,
noise_inversion=noise_inversion,
differential_diffusion=tiling.differential_diffusion
if tiling is not None
else False,
)
outputs: list[torch.Tensor] = []
@@ -207,6 +221,10 @@ class ContextualDiffusionSamplingService:
image_width=image_width,
region_masks=planning_masks,
capability_admission=capability_admission,
noise_inversion=noise_inversion,
differential_diffusion=tiling.differential_diffusion
if tiling is not None
else False,
)
samples = item_result.latent.get("samples")
if not isinstance(samples, torch.Tensor):
@@ -240,8 +258,10 @@ class ContextualDiffusionSamplingService:
image_width: int,
region_masks: torch.Tensor | None,
capability_admission: RegionalCapabilityAdmission,
noise_inversion: NoiseInversionOptions | None,
differential_diffusion: bool,
) -> ContextualDiffusionSamplingResult:
"""Build one canvas plan and execute it through the runtime adapter."""
"""Build the forward plan and retain canonical ownership for inversion."""
samples = latent_image.get("samples")
if not isinstance(samples, torch.Tensor):
@@ -270,6 +290,10 @@ class ContextualDiffusionSamplingService:
controls=controls,
plan=plan,
capability_admission=capability_admission,
noise_inversion=noise_inversion,
inversion_segs=segs,
inversion_region_masks=region_masks,
differential_diffusion=differential_diffusion,
)
return ContextualDiffusionSamplingResult(
latent=latent,
@@ -16,6 +16,7 @@ from typing import Any, Protocol
import torch
from ..domain.conditioning_batch import ConditioningBatch
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.regional_detailing import (
LatentRegion,
pair_segments_with_conditioning,
@@ -65,6 +66,7 @@ class RegionalDetailSamplingBoundary(Protocol):
global_prompt_weight: float,
preview_context: DetailPreviewContext | None = None,
differential_diffusion: bool = False,
noise_inversion: NoiseInversionOptions | None = None,
) -> Latent:
"""Sample one full latent with paired regional conditioning."""
@@ -133,6 +135,7 @@ class DetailSEGSAsRegionsService:
tiled_encode: bool,
tiled_decode: bool,
global_prompt_weight: float,
noise_inversion: NoiseInversionOptions | None = None,
) -> DetailSEGSAsRegionsResult:
"""Run regional MultiDiffusion detailing for provided SEGS."""
@@ -234,6 +237,7 @@ class DetailSEGSAsRegionsService:
sampled_region=CropRegion(0, 0, image_width, image_height),
),
differential_diffusion=differential_diffusion,
noise_inversion=noise_inversion,
)
decoded = self._sampler.decode(vae, sampled, tiled_decode)
if decoded.shape[1:3] != image_tensor.shape[1:3]:
@@ -13,6 +13,7 @@ import torch
from ..domain.conditioning_batch import select_conditioning
from ..domain.detail_geometry import DetailScalePlan, build_detail_scale_plan
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.segs import Segment, coerce_segs
from ..domain.segs_mask_ops import (
crop_image,
@@ -59,6 +60,7 @@ class TiledDetailSamplingBoundary(Protocol):
latent_tile_batch_size: int,
preview_context: DetailPreviewContext | None = None,
differential_diffusion: bool = False,
noise_inversion: NoiseInversionOptions | None = None,
) -> Latent:
"""Sample one latent crop with the requested tiled diffusion mode."""
@@ -131,6 +133,7 @@ class DetailSEGSByScaleFactorTiledDiffusionService:
latent_tile_height: int,
latent_tile_overlap: int,
latent_tile_batch_size: int,
noise_inversion: NoiseInversionOptions | None = None,
) -> TiledDetailerResult:
"""Run crop sampling and composite-back detailing with tiled diffusion."""
@@ -198,6 +201,7 @@ class DetailSEGSByScaleFactorTiledDiffusionService:
latent_tile_overlap=latent_tile_overlap,
latent_tile_batch_size=latent_tile_batch_size,
differential_diffusion=differential_diffusion,
noise_inversion=noise_inversion,
)
LOGGER.info(
@@ -246,6 +250,7 @@ class DetailSEGSByScaleFactorTiledDiffusionService:
latent_tile_overlap: int,
latent_tile_batch_size: int,
differential_diffusion: bool,
noise_inversion: NoiseInversionOptions | None,
) -> torch.Tensor:
"""Detail one segment with tiled diffusion and return the updated image."""
@@ -281,6 +286,7 @@ class DetailSEGSByScaleFactorTiledDiffusionService:
latent_tile_overlap=latent_tile_overlap,
latent_tile_batch_size=latent_tile_batch_size,
differential_diffusion=differential_diffusion,
noise_inversion=noise_inversion,
preview_context=DetailPreviewContext(
image=working_image,
work_region=segment.crop_region,
@@ -12,9 +12,12 @@ from typing import Any, TypeAlias
import torch
from ..domain.conditioning_batch import ConditioningBatch, select_conditioning
from ..domain.noise_inversion import NoiseInversionOptions
from ..runtime import sampling_noise, sampling_samplers, sampling_schedulers
from ..runtime.comfy_latent_normalization import COMFY_LATENT_NORMALIZER
from ..runtime.guided_sampling import sample_with_optional_negative
from ..runtime.inversion_model_factory import InversionModelFactory
from ..runtime.noise_reuse_scope import needs_finishing_model
from ..shared.logging import get_logger
Latent: TypeAlias = dict[str, Any]
@@ -37,8 +40,9 @@ class KSamplerSamplingService:
negative: Any,
latent_image: Latent,
denoise: float,
noise_inversion: NoiseInversionOptions | None = None,
) -> Latent:
"""Sample a latent with configured SimpleSyrup sampler extensions."""
"""Sample full latents with optional inversion and per-item conditioning."""
sampler = sampling_samplers.resolve_sampler(sampler_name)
latent_samples = latent_image["samples"]
@@ -85,6 +89,7 @@ class KSamplerSamplingService:
noise=noise,
cfg=cfg,
sampler=sampler,
sampler_name=sampler_name,
sigmas=sigmas,
positive=positive,
negative=negative,
@@ -93,6 +98,7 @@ class KSamplerSamplingService:
callback=callback,
disable_pbar=disable_pbar,
seed=seed,
noise_inversion=noise_inversion,
)
else:
samples = sample_with_optional_negative(
@@ -101,6 +107,7 @@ class KSamplerSamplingService:
noise=noise,
cfg=cfg,
sampler=sampler,
sampler_name=sampler_name,
sigmas=sigmas,
positive=positive,
negative=negative,
@@ -109,6 +116,14 @@ class KSamplerSamplingService:
callback=callback,
disable_pbar=disable_pbar,
seed=seed,
noise_inversion=noise_inversion,
inversion_model_factory=InversionModelFactory(
model=model,
canvas_width=int(latent_samples.shape[-1]),
canvas_height=int(latent_samples.shape[-2]),
)
if noise_inversion is not None or needs_finishing_model()
else None,
)
if not isinstance(samples, torch.Tensor):
raise TypeError("KSampler output samples must be a torch.Tensor.")
@@ -139,6 +154,7 @@ class KSamplerSamplingService:
noise: torch.Tensor,
cfg: float,
sampler: Any,
sampler_name: str,
sigmas: torch.Tensor,
positive: Any,
negative: Any,
@@ -147,6 +163,7 @@ class KSamplerSamplingService:
callback: Any,
disable_pbar: bool,
seed: int,
noise_inversion: NoiseInversionOptions | None,
) -> torch.Tensor:
"""Sample latent items with existing per-item batch selection."""
@@ -159,6 +176,7 @@ class KSamplerSamplingService:
noise=noise[index : index + 1],
cfg=cfg,
sampler=sampler,
sampler_name=sampler_name,
sigmas=sigmas,
positive=select_conditioning(positive, index),
negative=select_conditioning(negative, index),
@@ -171,6 +189,14 @@ class KSamplerSamplingService:
callback=callback,
disable_pbar=disable_pbar,
seed=seed,
noise_inversion=noise_inversion,
inversion_model_factory=InversionModelFactory(
model=model,
canvas_width=int(latent_samples.shape[-1]),
canvas_height=int(latent_samples.shape[-2]),
)
if noise_inversion is not None or needs_finishing_model()
else None,
)
)
return torch.cat(sampled, dim=0)
@@ -0,0 +1,29 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Recover original latents from image graph provenance without pixel encoding."""
from __future__ import annotations
from collections.abc import Mapping
from typing import Any
from ..domain.graph_provenance import BrokenProvenance, GraphLink
from ..runtime.comfy_graph_provenance import NodeRegistry, trace_vae_decode_provenance
class LatentProvenanceService:
"""Admit only unmodified image paths backed by a VAE decode source."""
def recover_link(
self, prompt: Mapping[str, Any], image: object, registry: NodeRegistry
) -> GraphLink:
"""Return the original samples link or a specific actionable graph error."""
provenance = trace_vae_decode_provenance(prompt, image, registry)
if isinstance(provenance, BrokenProvenance):
raise ValueError(
f"Latent From Image requires an unmodified VAE Decode "
f"source: {provenance.reason}. Connect the original "
f"decoded image or a direct latent connection."
)
return provenance.samples_link
@@ -0,0 +1,81 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Own latent upscale method selection, provenance preservation and mask geometry."""
from __future__ import annotations
from importlib import import_module
from typing import Any
import torch
from ..domain.latent_upscale import (
FORMAT_FAMILIES,
NATIVE_LATENT_METHODS,
upscale_target,
)
from ..domain.noise_reuse import PROVENANCE_KEY, SamplingProvenance
from ..runtime.latent_upscale_models import resolve_latent_upscale_model
from ..runtime.sesqui_latent_upscale import upscale_with_sesqui
from ..shared.logging import get_logger
LOGGER = get_logger(__name__)
class LatentUpscaleService:
"""Resolve learned checkpoints or delegate native scaling to Comfy's node."""
def upscale(
self, latent: dict[str, Any], method: str, scale_factor: float
) -> dict[str, Any]:
"""Use native bislerp for unknown Auto formats and preserve provenance."""
samples = latent.get("samples")
if not isinstance(samples, torch.Tensor) or samples.ndim not in (4, 5):
raise ValueError("Simple Upscale Latent requires BCHW or BCTHW samples.")
if not samples.is_floating_point() or not bool(torch.isfinite(samples).all()):
raise ValueError(
"Simple Upscale Latent requires finite floating-point samples."
)
mask = latent.get("noise_mask")
if mask is not None and (not isinstance(mask, torch.Tensor) or mask.ndim < 2):
raise TypeError("Latent noise mask must contain spatial tensor dimensions.")
target = upscale_target(
(int(samples.shape[-2]), int(samples.shape[-1])), scale_factor
)
if method == "Auto":
provenance = latent.get(PROVENANCE_KEY)
latent_format = (
provenance.latent_format
if isinstance(provenance, SamplingProvenance)
else None
)
if latent_format not in FORMAT_FAMILIES:
method = "bislerp"
LOGGER.info(
"Simple Upscale Latent selected native bislerp: "
"latent_format=%s scale_factor=%s",
latent_format or "unknown",
scale_factor,
)
if method in NATIVE_LATENT_METHODS:
result: dict[str, Any] = (
import_module("nodes")
.LatentUpscaleBy()
.upscale(latent, method, scale_factor)[0]
)
else:
family, path = resolve_latent_upscale_model(latent, method)
result = latent.copy()
result["samples"] = upscale_with_sesqui(
samples, path=path, family=family, target=target
)
if mask is not None:
assert isinstance(mask, torch.Tensor)
resized = torch.nn.functional.interpolate(
mask.reshape(-1, 1, *mask.shape[-2:]).float(),
size=target,
mode="nearest",
)
result["noise_mask"] = resized.reshape(*mask.shape[:-2], *target).to(mask)
return result
@@ -48,6 +48,7 @@ from ..runtime.negpip.krea2 import (
from ..runtime.negpip.krea2 import (
WRAPPER_KEY as KREA_WRAPPER_KEY,
)
from ..runtime.negpip.krea2_host import krea2_host_mutations
from ..runtime.negpip.standard import (
encode_token_weights_negpip,
standard_attn2_negpip,
@@ -190,6 +191,7 @@ class NegpipModelService:
prepared_model = PATCHER_LIFECYCLE.derive_model(
model,
(
*krea2_host_mutations(model),
ModelCallableObjectPatchMutation(
"extra_conds",
krea2_extra_conds_negpip_wrapper(previous),
@@ -0,0 +1,117 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Resolve upstream capture requests from the receiving sampler's live settings."""
from __future__ import annotations
import json
from typing import Any
import torch
from ..domain.sampler_options import SamplerOptions
from ..runtime.comfy_latent_normalization import COMFY_LATENT_NORMALIZER
from ..runtime.native_sampling_target import validate_partial_sampling_target
from ..runtime.sampling_schedulers import SchedulerView, calculate_sigmas
class NoiseCaptureRequestService:
"""Evaluate schedule planning before the originating sampler can execute."""
def coordinates(
self,
*,
model: Any,
options: SamplerOptions,
steps: int,
sampler_name: str,
scheduler: str,
denoise: float,
source_latent: dict[str, Any],
geometry_json: str,
previous: tuple[float, ...] = (),
) -> tuple[float, ...]:
"""Calculate the same model-view schedule as the receiving sampling route."""
if not isinstance(options, SamplerOptions):
raise TypeError("Capture planning requires SimpleSyrup sampler options.")
if options.noise_reuse is None:
return previous
samples = source_latent.get("samples")
if not isinstance(samples, torch.Tensor):
raise TypeError("Capture planning requires source latent geometry.")
context, tiling = options.contextual_diffusion, options.tiling
if context is not None:
view = SchedulerView(context.context_size, context.context_size)
elif tiling is not None:
view = SchedulerView(tiling.width, tiling.height)
elif scheduler == "Flux2":
samples = COMFY_LATENT_NORMALIZER.normalize(
model=model,
samples=samples,
spatial_downscale_ratio=source_latent.get("downscale_ratio_spacial"),
)
view = self._view(samples, geometry_json)
else:
view = SchedulerView.from_tensor(samples)
sigmas = calculate_sigmas(
model, scheduler, sampler_name, steps, denoise, view=view
)
target = validate_partial_sampling_target(
model, sigmas, operation="Noise reuse"
)
coordinate = options.noise_reuse.capture_coordinate(target)
return tuple(sorted({*previous, coordinate}, reverse=True))
def _view(self, samples: torch.Tensor, geometry_json: str) -> SchedulerView:
"""Resolve only validated native sizing operations for resolution-
dependent schedules.
"""
operations = json.loads(geometry_json)
if not isinstance(operations, list):
raise ValueError("Noise capture geometry must be an operation list.")
height, width = int(samples.shape[-2]), int(samples.shape[-1])
for operation in operations:
if isinstance(operation, dict) and set(operation) == {"pixels"}:
pixels = operation["pixels"]
if (
not isinstance(pixels, list)
or len(pixels) != 2
or not all(type(value) is int and value >= 0 for value in pixels)
):
raise ValueError(
"Resolution-dependent noise capture requires literal "
"non-negative pixel dimensions."
)
pixel_height, pixel_width = pixels
if pixel_height == 0 and pixel_width == 0:
continue
if pixel_width == 0:
pixel_height = max(64, pixel_height)
pixel_width = max(64, round(width * pixel_height / height))
elif pixel_height == 0:
pixel_width = max(64, pixel_width)
pixel_height = max(64, round(height * pixel_width / width))
else:
pixel_width, pixel_height = (
max(64, pixel_width),
max(64, pixel_height),
)
height, width = pixel_height // 8, pixel_width // 8
continue
if not isinstance(operation, dict) or set(operation) != {"scale"}:
raise ValueError(
"Noise capture geometry contains an unsupported operation."
)
scale = operation["scale"]
if (
not isinstance(scale, (int, float))
or isinstance(scale, bool)
or not 0.01 <= scale <= 8
):
raise ValueError(
"Noise capture geometry scale must be between 0.01 and 8."
)
height, width = round(height * scale), round(width * scale)
return SchedulerView(width, height)
@@ -0,0 +1,16 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Expose capture preparation through the existing public Cube capability."""
from __future__ import annotations
from ..runtime.noise_reuse_validation import prepare_noise_reuse_prompt
class NoiseReuseGraphPreparationService:
"""Delegate public host requests to the authoritative capture planner."""
def prepare(self, prompt: dict[str, dict[str, object]]) -> None:
"""Install cache-keyed source capture before any sampler executes."""
prepare_noise_reuse_prompt(prompt)
@@ -10,6 +10,7 @@ from typing import Any, TypeAlias
import torch
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.regional_features import RegionalCapabilityAdmission
from ..domain.regional_tiled_diffusion import (
build_region_constrained_tiled_diffusion_plan,
@@ -49,6 +50,7 @@ class RegionalTiledDiffusionSamplingService:
preview_context: DetailPreviewContext | None,
differential_diffusion: bool,
capability_admission: RegionalCapabilityAdmission,
noise_inversion: NoiseInversionOptions | None = None,
) -> Latent:
"""Sample every latent item with one shared regional composition."""
@@ -106,6 +108,13 @@ class RegionalTiledDiffusionSamplingService:
differential_diffusion=differential_diffusion,
capability_admission=capability_admission,
tiled_plan=plan,
noise_inversion=noise_inversion,
inversion_segs=(
segs_group[0 if len(segs_group) == 1 else index]
if segs_group
else None
),
inversion_region_masks=region_masks,
)
output_samples = output.get("samples")
if not isinstance(output_samples, torch.Tensor):
@@ -0,0 +1,284 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Compile capability configuration into one authoritative sampling execution."""
from __future__ import annotations
from typing import Any, TypedDict
import torch
from ..domain.attention_coupling_request import (
AttentionCouplingRequestMode,
classify_attention_coupling_request,
)
from ..domain.noise_reuse import PROVENANCE_KEY, SamplingProvenance
from ..domain.sampler_options import SamplerOptions, TilingOptions
from ..runtime import sampling_schedulers
from ..runtime.native_sampling_target import validate_partial_sampling_target
from ..runtime.noise_reuse_scope import NoiseReuseExecution, noise_reuse_scope
from ..runtime.tiled_sampling_validation import validate_sampling_controls
from ..shared.logging import get_logger
from .attention_coupling_sampling_service import AttentionCouplingSamplingService
from .contextual_attention_coupling_sampling_service import (
ContextualAttentionCouplingSamplingService,
)
from .contextual_diffusion_sampling_service import ContextualDiffusionSamplingService
from .ksampler_sampling_service import KSamplerSamplingService
from .tiled_attention_coupling_sampling_service import (
TiledAttentionCouplingSamplingService,
)
from .tiled_diffusion_sampling_service import TiledDiffusionSamplingService
LOGGER = get_logger(__name__)
class SamplingArguments(TypedDict):
"""Narrow controls while retaining dynamic host MODEL and tensor payloads."""
model: Any
seed: int
steps: int
cfg: float
sampler_name: str
scheduler: str
positive: object
negative: object
latent_image: dict[str, Any]
denoise: float
class ContextualArguments(TypedDict):
"""Describe global context and the sole local tile plan's execution controls."""
diffusion_mode: str
latent_context_size: int
latent_context_overlap: int
latent_context_batch_size: int
global_weight: float
global_steps: int
global_decay: float
class TiledArguments(TypedDict):
"""Describe local geometry and mask-dependent denoising policy."""
diffusion_mode: str
latent_tile_width: int
latent_tile_height: int
latent_tile_overlap: int
latent_tile_batch_size: int
differential_diffusion: bool
class SamplerOptionsSamplingService:
"""Route configuration independently of connection order or node placement."""
def sample(
self,
*,
model: Any,
seed: int,
steps: int,
cfg: float,
sampler_name: str,
scheduler: str,
positive: object,
negative: object,
latent_image: dict[str, Any],
denoise: float,
options: SamplerOptions | None = None,
segs: object | None = None,
region_masks: object | None = None,
) -> dict[str, Any]:
"""Own the receiving request lifecycle across every capability route."""
reuse = options.noise_reuse if isinstance(options, SamplerOptions) else None
execution = None
if reuse is not None:
provenance = latent_image.get(PROVENANCE_KEY)
if not isinstance(provenance, SamplingProvenance):
raise ValueError(
"Noise Reuse Options requires captured sampling provenance. "
"Connect a supported sampler's latent directly or through "
"Latent From Image."
)
execution = NoiseReuseExecution(reuse, provenance)
with noise_reuse_scope(execution):
result = self._sample_configured(
model=model,
seed=seed,
steps=steps,
cfg=cfg,
sampler_name=sampler_name,
scheduler=scheduler,
positive=positive,
negative=negative,
latent_image=latent_image,
denoise=denoise,
options=options,
segs=segs,
region_masks=region_masks,
)
if execution is not None and not execution.consumed:
raise RuntimeError(
"Receiving sampler did not consume the captured noise request."
)
return result
def _sample_configured(
self,
*,
model: Any,
seed: int,
steps: int,
cfg: float,
sampler_name: str,
scheduler: str,
positive: object,
negative: object,
latent_image: dict[str, Any],
denoise: float,
options: SamplerOptions | None = None,
segs: object | None = None,
region_masks: object | None = None,
) -> dict[str, Any]:
"""Admit connected region data only through its enabled sampling capability."""
if options is not None and not isinstance(options, SamplerOptions):
raise TypeError(
"KSampler options must come from SimpleSyrup options nodes."
)
configured = options if options is not None else SamplerOptions()
context = configured.contextual_diffusion
tiling = configured.tiling
if context is not None:
if tiling is not None:
LOGGER.warning(
"Contextual Diffusion takes precedence; Tiling Options ignored.",
extra={"node_id": "SimpleSyrup.KSampler"},
)
tiling = context.local_tiling()
arguments: SamplingArguments = {
"model": model,
"seed": seed,
"steps": steps,
"cfg": cfg,
"sampler_name": sampler_name,
"scheduler": scheduler,
"positive": positive,
"negative": negative,
"latent_image": latent_image,
"denoise": denoise,
}
self._preflight(arguments, configured, tiling)
attention = configured.attention_coupling
if (
attention is not None
and classify_attention_coupling_request(
positive=positive, negative=negative, region_masks=region_masks
)
is AttentionCouplingRequestMode.BYPASS
):
attention = None
inversion = configured.noise_inversion
if context is not None:
assert tiling is not None
contextual_arguments: ContextualArguments = {
"diffusion_mode": tiling.diffusion_mode,
"latent_context_size": context.context_size,
"latent_context_overlap": tiling.overlap,
"latent_context_batch_size": tiling.batch_size,
"global_weight": context.global_weight,
"global_steps": context.global_steps,
"global_decay": context.global_decay,
}
if attention is not None:
result = ContextualAttentionCouplingSamplingService().sample(
**arguments,
**contextual_arguments,
region_masks=region_masks,
regional_prompt_weight=attention.regional_prompt_weight,
region_mask_feather=attention.region_mask_feather,
segs=segs,
tiling=tiling,
noise_inversion=inversion,
)
else:
result = ContextualDiffusionSamplingService().sample(
**arguments,
**contextual_arguments,
segs=segs,
tiling=tiling,
noise_inversion=inversion,
)
return result.latent
if tiling is not None:
tiled_arguments: TiledArguments = {
"diffusion_mode": tiling.diffusion_mode,
"latent_tile_width": tiling.width,
"latent_tile_height": tiling.height,
"latent_tile_overlap": tiling.overlap,
"latent_tile_batch_size": tiling.batch_size,
"differential_diffusion": tiling.differential_diffusion,
}
if attention is not None:
return TiledAttentionCouplingSamplingService().sample(
**arguments,
**tiled_arguments,
region_masks=region_masks,
regional_prompt_weight=attention.regional_prompt_weight,
region_mask_feather=attention.region_mask_feather,
segs=segs,
noise_inversion=inversion,
)
return TiledDiffusionSamplingService().sample(
**arguments,
**tiled_arguments,
segs=segs,
noise_inversion=inversion,
)
if attention is not None:
return AttentionCouplingSamplingService().sample(
**arguments,
region_masks=region_masks,
regional_prompt_weight=attention.regional_prompt_weight,
region_mask_feather=attention.region_mask_feather,
noise_inversion=inversion,
)
return KSamplerSamplingService().sample(**arguments, noise_inversion=inversion)
def _preflight(
self,
arguments: SamplingArguments,
options: SamplerOptions,
tiling: TilingOptions | None,
) -> None:
"""Reject unsupported schedules and endpoints before preparing models."""
samples = arguments["latent_image"].get("samples")
if not isinstance(samples, torch.Tensor):
raise TypeError("KSampler latent samples must be a torch.Tensor.")
validate_sampling_controls(
steps=arguments["steps"],
denoise=arguments["denoise"],
latent_tile_width=tiling.width if tiling is not None else 16,
latent_tile_height=tiling.height if tiling is not None else 16,
latent_tile_batch_size=tiling.batch_size if tiling is not None else 1,
)
if options.noise_inversion is not None:
view = (
sampling_schedulers.SchedulerView(tiling.width, tiling.height)
if tiling is not None
else sampling_schedulers.SchedulerView.from_tensor(samples)
)
sigmas = sampling_schedulers.calculate_sigmas(
model=arguments["model"],
scheduler_name=arguments["scheduler"],
sampler_name=arguments["sampler_name"],
steps=arguments["steps"],
denoise=arguments["denoise"],
view=view,
)
validate_partial_sampling_target(
arguments["model"], sigmas, operation="Noise inversion"
)
@@ -11,6 +11,7 @@ from typing import Any, TypeAlias
import torch
from ..domain.conditioning_batch import ConditioningBatch, select_conditioning
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.regional_features import RegionalCapabilityAdmission
from ..domain.segs import coerce_segs_group
from ..domain.segs_tiled_diffusion import build_segs_guided_tiled_diffusion_plan
@@ -47,6 +48,7 @@ class SEGSGuidedTiledDiffusionSamplingService:
preview_context: DetailPreviewContext | None,
differential_diffusion: bool,
capability_admission: RegionalCapabilityAdmission,
noise_inversion: NoiseInversionOptions | None = None,
) -> Latent:
"""Sample every latent batch item using its connected SEGS guide."""
@@ -108,6 +110,8 @@ class SEGSGuidedTiledDiffusionSamplingService:
differential_diffusion=differential_diffusion,
capability_admission=capability_admission,
tiled_plan=plan,
noise_inversion=noise_inversion,
inversion_segs=segs_for_item,
)
output_samples = output["samples"]
if not isinstance(output_samples, torch.Tensor):
@@ -12,6 +12,7 @@ from ..domain.attention_coupling_request import (
AttentionCouplingRequestMode,
classify_attention_coupling_request,
)
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.regional_attention_execution import RegionalAttentionExecutionMode
from ..domain.regional_features import (
EMPTY_REGIONAL_FEATURE_REQUEST,
@@ -62,6 +63,8 @@ class TiledAttentionCouplingSamplingService:
latent_tile_batch_size: int,
preview_context: DetailPreviewContext | None = None,
differential_diffusion: bool = False,
segs: object | None = None,
noise_inversion: NoiseInversionOptions | None = None,
) -> dict[str, Any]:
"""Bypass ordinary requests or prepare one complete regional request."""
@@ -90,6 +93,8 @@ class TiledAttentionCouplingSamplingService:
preview_context=preview_context,
differential_diffusion=differential_diffusion,
feature_request=EMPTY_REGIONAL_FEATURE_REQUEST,
segs=segs,
noise_inversion=noise_inversion,
)
prepared = self.model_preparation_service_class().prepare(
@@ -121,6 +126,8 @@ class TiledAttentionCouplingSamplingService:
preview_context=preview_context,
differential_diffusion=differential_diffusion,
feature_request=_TILED_ATTENTION_REQUEST,
segs=segs,
noise_inversion=noise_inversion,
)
def _sample_tiled(
@@ -144,6 +151,8 @@ class TiledAttentionCouplingSamplingService:
preview_context: DetailPreviewContext | None,
differential_diffusion: bool,
feature_request: RegionalFeatureRequest,
segs: object | None,
noise_inversion: NoiseInversionOptions | None,
) -> dict[str, Any]:
"""Delegate one ordinary or Attention Coupling tiled request."""
@@ -166,10 +175,11 @@ class TiledAttentionCouplingSamplingService:
preview_context=preview_context,
differential_diffusion=differential_diffusion,
feature_request=feature_request,
segs=None,
segs=segs,
region_masks=None,
regional_prompt_weight=0.5,
region_mask_feather=0,
noise_inversion=noise_inversion,
)
@@ -10,6 +10,7 @@ from typing import Any, Protocol
import torch
from ..domain.noise_inversion import NoiseInversionOptions
from ..runtime.detail_previews import DetailPreviewContext
from ..runtime.detail_sampling import DetailSampler, Latent
from .tiled_diffusion_sampling_service import TiledDiffusionSamplingService
@@ -38,6 +39,7 @@ class TiledDiffusionLatentSamplingBoundary(Protocol):
latent_tile_batch_size: int,
preview_context: DetailPreviewContext | None = None,
differential_diffusion: bool = False,
noise_inversion: NoiseInversionOptions | None = None,
) -> Latent:
"""Sample a latent using the selected tiled diffusion mode."""
@@ -87,6 +89,7 @@ class TiledDetailSampler:
latent_tile_batch_size: int,
preview_context: DetailPreviewContext | None = None,
differential_diffusion: bool = False,
noise_inversion: NoiseInversionOptions | None = None,
) -> Latent:
"""Sample one latent crop with the selected tiled diffusion runtime."""
@@ -108,4 +111,5 @@ class TiledDetailSampler:
latent_tile_batch_size=latent_tile_batch_size,
preview_context=preview_context,
differential_diffusion=differential_diffusion,
noise_inversion=noise_inversion,
)
@@ -11,6 +11,7 @@ from typing import Any, TypeAlias
import torch
from ..domain.conditioning_batch import select_conditioning
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.regional_features import RegionalCapabilityAdmission
from ..runtime.detail_previews import DetailPreviewContext
from .sampling_batch import combine_latent_outputs, single_item_latent
@@ -44,6 +45,7 @@ class TiledDiffusionConditioningBatchService:
preview_context: DetailPreviewContext | None,
differential_diffusion: bool,
capability_admission: RegionalCapabilityAdmission,
noise_inversion: NoiseInversionOptions | None = None,
) -> Latent:
"""Sample each latent item with its selected conditioning values."""
@@ -72,6 +74,7 @@ class TiledDiffusionConditioningBatchService:
preview_context=preview_context,
differential_diffusion=differential_diffusion,
capability_admission=capability_admission,
noise_inversion=noise_inversion,
)
output_samples = output.get("samples")
if not isinstance(output_samples, torch.Tensor):
@@ -8,7 +8,11 @@ from __future__ import annotations
from typing import Any, Protocol, TypeAlias
import torch
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.regional_features import RegionalCapabilityAdmission
from ..domain.segs import NativeSegs
from ..domain.tiled_diffusion import TiledDiffusionPlan
from ..runtime import mixture_of_diffusers_sampling, multidiffusion_sampling
from ..runtime.detail_previews import DetailPreviewContext
@@ -41,6 +45,9 @@ class TiledDiffusionItemSampler(Protocol):
differential_diffusion: bool,
capability_admission: RegionalCapabilityAdmission,
tiled_plan: TiledDiffusionPlan | None = None,
noise_inversion: NoiseInversionOptions | None = None,
inversion_segs: NativeSegs | None = None,
inversion_region_masks: torch.Tensor | None = None,
) -> Latent:
"""Return one sampled latent item."""
@@ -70,6 +77,9 @@ class TiledDiffusionItemSamplingService:
differential_diffusion: bool,
capability_admission: RegionalCapabilityAdmission,
tiled_plan: TiledDiffusionPlan | None = None,
noise_inversion: NoiseInversionOptions | None = None,
inversion_segs: NativeSegs | None = None,
inversion_region_masks: torch.Tensor | None = None,
) -> Latent:
"""Invoke exactly one runtime with the unchanged sampling request."""
@@ -97,4 +107,7 @@ class TiledDiffusionItemSamplingService:
differential_diffusion=differential_diffusion,
capability_admission=capability_admission,
tiled_plan=tiled_plan,
noise_inversion=noise_inversion,
inversion_segs=inversion_segs,
inversion_region_masks=inversion_region_masks,
)
@@ -9,6 +9,7 @@ from __future__ import annotations
from typing import Any, ClassVar
from ..domain.conditioning_batch import ConditioningBatch
from ..domain.noise_inversion import NoiseInversionOptions
from ..domain.regional_features import (
EMPTY_REGIONAL_FEATURE_REQUEST,
TILED_DIFFUSION_REGIONAL_SAMPLER_CAPABILITIES,
@@ -84,6 +85,7 @@ class TiledDiffusionSamplingService:
region_masks: object | None = None,
regional_prompt_weight: float = 0.5,
region_mask_feather: int = 0,
noise_inversion: NoiseInversionOptions | None = None,
) -> Latent:
"""Sample a latent with the selected tiled diffusion method."""
@@ -129,6 +131,7 @@ class TiledDiffusionSamplingService:
preview_context=preview_context,
differential_diffusion=differential_diffusion,
capability_admission=capability_admission,
noise_inversion=noise_inversion,
)
if segs is not None:
return self.segs_sampling_service_class().sample(
@@ -151,6 +154,7 @@ class TiledDiffusionSamplingService:
preview_context=preview_context,
differential_diffusion=differential_diffusion,
capability_admission=capability_admission,
noise_inversion=noise_inversion,
segs=segs,
)
if isinstance(positive, ConditioningBatch) or isinstance(
@@ -177,6 +181,7 @@ class TiledDiffusionSamplingService:
preview_context=preview_context,
differential_diffusion=differential_diffusion,
capability_admission=capability_admission,
noise_inversion=noise_inversion,
)
return self.item_sampling_service_class().sample(
diffusion_mode=diffusion_mode,
@@ -197,4 +202,5 @@ class TiledDiffusionSamplingService:
preview_context=preview_context,
differential_diffusion=differential_diffusion,
capability_admission=capability_admission,
noise_inversion=noise_inversion,
)
+27
View File
@@ -0,0 +1,27 @@
"""
SesquiLSR - Latent Super-Resolution Upscaler.
Operates on VAE latents at arbitrary scales in [1x, 2x].
"""
from .inference_adaptors import (
LatentFormatAdaptor,
make_flux,
make_flux2,
make_ideogram4,
make_identity,
make_sdxl,
make_wan21,
)
from .upscaler import LatentUpscaler
__all__ = [
"LatentUpscaler",
"LatentFormatAdaptor",
"make_identity",
"make_sdxl",
"make_flux",
"make_flux2",
"make_ideogram4",
"make_wan21",
]
@@ -0,0 +1,355 @@
"""Latent-format adaptors for Sesqui inference.
Each adaptor converts between a pipeline's external latent space (what the
diffusion model passes around) and the canonical VAE latent space that
Sesqui models are trained on (as defined by the VAE adaptor).
Adaptors are only needed when the pipeline modifies the VAE output.
"""
from __future__ import annotations
import torch
import torch.nn.functional as F
from torch import Tensor
# ── Patchify helpers (shared by Flux2-family formats) ─────────────────────
def _patchify(z: Tensor) -> Tensor:
"""32ch -> 128ch via 2x2 spatial packing."""
B, C, H, W = z.shape
return (
z.float()
.reshape(B, C, H // 2, 2, W // 2, 2)
.permute(0, 1, 3, 5, 2, 4)
.reshape(B, C * 4, H // 2, W // 2)
)
def _unpatchify(z: Tensor) -> Tensor:
"""128ch -> 32ch via 2x2 spatial unpacking."""
B, C, H, W = z.shape
return (
z.float()
.reshape(B, C // 4, 2, 2, H, W)
.permute(0, 1, 4, 2, 5, 3)
.reshape(B, C // 4, H * 2, W * 2)
)
# ── Base class ────────────────────────────────────────────────────────────
class LatentFormatAdaptor:
"""Convert between an external pipeline's latent and VAE latent.
Subclass or instantiate via the ``make_*`` constructors below.
Parameters
----------
external_channels : int
Channel count of the external/pipeline latent.
spatial_scale : int, default 1
Spatial multiplier between external and VAE-latent coords.
Flux2-family formats use ``2`` (VAE latent is 2x spatially larger
because the pipeline packs 2x2 into channels). All others use ``1``.
"""
def __init__(self, external_channels: int, spatial_scale: int = 1):
self.external_channels = external_channels
self.spatial_scale = spatial_scale
def to_vae_latent(self, z: Tensor) -> Tensor:
"""External/pipeline latent -> VAE latent."""
return z.float()
def from_vae_latent(self, z: Tensor) -> Tensor:
"""VAE latent -> external/pipeline latent."""
return z
def vae_target_size(self, target_size: tuple[int, int]) -> tuple[int, int]:
"""Target size in external coords -> VAE-latent coords."""
h, w = target_size
s = self.spatial_scale
return (h * s, w * s) if s != 1 else target_size
# ── Constructors ──────────────────────────────────────────────────────────
def make_identity(external_channels: int) -> LatentFormatAdaptor:
"""Adaptor for formats where the external latent IS the model latent.
Wan 2.1, Anima, Qwen Image, Krea 2, and any other model where ComfyUI's
latent_format does not apply a scaling transform (process_in/out are
identity or cancel out before reaching custom LATENT nodes).
"""
return LatentFormatAdaptor(external_channels=external_channels)
def make_sdxl() -> LatentFormatAdaptor:
"""SDXL - 4ch VAE latent with 0.13025 scaling.
ComfyUI stores raw VAE latents between nodes; KSampler applies
process_in (x0.13025) / process_out (/0.13025) around the model.
Sesqui was trained on the scaled latents.
"""
return _AffineAdaptor(external_channels=4, scale=0.13025, shift=0.0)
def make_flux() -> LatentFormatAdaptor:
"""Flux / Z-Image Turbo / Lumina - 16ch VAE latent with shift/scale.
ComfyUI stores raw VAE latents between nodes; KSampler applies
process_in ((z - 0.1159) x 0.3611) / process_out (z / 0.3611 + 0.1159).
Sesqui was trained on the scaled latents.
"""
return _AffineAdaptor(external_channels=16, scale=0.3611, shift=0.1159)
def make_flux2(
running_mean: Tensor | None = None,
running_var: Tensor | None = None,
eps: float = 1e-4,
) -> LatentFormatAdaptor:
"""Flux2 - 32ch raw VAE ↔ 128ch patched + BN pipeline latent.
Uses BN statistics from the Flux2 VAE's ``bn`` module.
Defaults are extracted from the official Flux2 VAE checkpoint.
"""
if running_mean is None:
running_mean = torch.tensor([
-0.06762, -0.07152, -0.07534, -0.07449, 0.02228, 0.01800, 0.01420, 0.01836,
-0.00006, -0.00625, -0.00021, -0.00314, -0.02720, -0.02811, -0.02765, -0.02903,
-0.07689, -0.06717, -0.09019, -0.08921, 0.01684, 0.01521, 0.00790, 0.00858,
0.00835, 0.00154, 0.00026, -0.00428, -0.04388, -0.04190, -0.04378, -0.04315,
-0.01025, -0.01319, -0.00662, -0.00477, -0.03106, -0.03055, -0.02790, -0.01795,
0.00302, 0.00150, 0.01259, 0.01447, 0.03472, 0.03377, 0.03366, 0.02830,
0.00198, 0.00473, 0.00465, 0.00496, 0.01227, 0.00810, 0.00806, 0.01458,
0.06811, 0.06790, 0.07665, 0.07319, -0.04621, -0.04739, -0.03919, -0.05109,
-0.05278, -0.04774, -0.04700, -0.05172, -0.03171, -0.03163, -0.03447, -0.02826,
0.05097, 0.04450, 0.05781, 0.04580, -0.04116, -0.04583, -0.04874, -0.04674,
-0.00884, -0.01063, -0.00881, -0.00461, -0.03758, -0.04322, -0.04357, -0.04989,
0.01185, 0.01664, 0.02028, 0.02790, 0.01127, 0.01290, 0.00156, 0.00716,
-0.01180, -0.00184, -0.01414, -0.00537, -0.00910, -0.01380, -0.01447, -0.01870,
0.03225, 0.03050, 0.02587, 0.02996, 0.05400, 0.06144, 0.04954, 0.05899,
-0.05108, -0.06033, -0.04778, -0.05240, -0.02268, -0.02742, -0.01537, -0.02546,
-0.05721, -0.05648, -0.05176, -0.04956, 0.01159, 0.00542, 0.01630, 0.01038,
])
if running_var is None:
running_var = torch.tensor([
3.25021, 3.16341, 3.19243, 3.18137, 3.13891, 3.09414, 3.10118, 3.05509,
3.00518, 3.01795, 3.00676, 3.00764, 3.46902, 3.43252, 3.47023, 3.45539,
3.09498, 3.07138, 3.08192, 3.09134, 3.01471, 3.02746, 3.01199, 3.02529,
3.00746, 2.97413, 3.02488, 2.99405, 3.08042, 3.06691, 3.08315, 3.05815,
3.40362, 3.40553, 3.44087, 3.43550, 3.32671, 3.17300, 3.18745, 3.22017,
3.25698, 3.19532, 3.13096, 3.12421, 3.16203, 3.12096, 3.21296, 3.18538,
3.09027, 3.03003, 3.05658, 3.01625, 3.22585, 3.23913, 3.21108, 3.21309,
3.16103, 3.14950, 3.14238, 3.15017, 3.07164, 3.04400, 3.11775, 3.06079,
3.15937, 3.13995, 3.17299, 3.17302, 3.29846, 3.24451, 3.24831, 3.25173,
3.07203, 3.00360, 3.08447, 3.05619, 3.10095, 3.06496, 3.12614, 3.10201,
3.12051, 3.07826, 3.17810, 3.14189, 3.20242, 3.23967, 3.19097, 3.15400,
3.10219, 3.10638, 3.08341, 3.08930, 3.16211, 3.12266, 3.17199, 3.16812,
2.95874, 2.91292, 2.98084, 2.92094, 3.16569, 3.08972, 3.06321, 3.04655,
3.09284, 3.06227, 3.07098, 3.01419, 3.10315, 3.08778, 3.04287, 3.03801,
3.06550, 3.10085, 3.10954, 3.10174, 2.97687, 2.93585, 2.99999, 2.96735,
3.12007, 3.10587, 3.13934, 3.12008, 3.04748, 3.04194, 3.08653, 3.07292,
])
return _Flux2BNAdaptor(running_mean=running_mean, running_var=running_var, eps=eps)
def make_ideogram4(
shift: Tensor | None = None,
scale: Tensor | None = None,
) -> LatentFormatAdaptor:
"""Ideogram 4 - 32ch raw VAE ↔ 128ch patched + shift/scale pipeline latent.
Uses the same Flux2 VAE with 2x2 patchify, but applies per-channel
shift+scale normalization instead of BatchNorm.
"""
if shift is None:
shift = torch.tensor([
0.01984364, 0.10149707, 0.29689495, 0.27188619, -0.21445648, -0.15979549,
0.05021099, -0.15083604, -0.15360136, -0.20131799, 0.01922352, 0.06226260,
0.10140969, -0.06739428, 0.37582610, -0.23371200, 0.35164491, -0.02590912,
-0.02719350, -0.10833897, -0.14768480, -0.01130957, -0.22983720, 0.23526423,
-0.10893522, 0.11957631, 0.04047799, 0.31345890, -0.17225064, -0.18646109,
-0.34691978, -0.03571246, 0.02583857, 0.10190072, 0.28402294, 0.26952152,
-0.21634675, -0.17938656, 0.04358909, -0.15007621, -0.15485020, -0.18971131,
0.02710861, 0.05609494, 0.10697846, -0.06854968, 0.38167698, -0.24269937,
0.35705471, -0.03063305, -0.02946109, -0.11244286, -0.14336038, -0.01362137,
-0.21863696, 0.23228983, -0.11739769, 0.11693044, 0.02563311, 0.31356594,
-0.17420591, -0.19006285, -0.34905377, -0.04025005, 0.01924137, 0.07652984,
0.29956080, 0.26280570, -0.22011674, -0.12715361, 0.04879879, -0.14075719,
-0.15935895, -0.21235840, 0.01974813, 0.05523547, 0.10011992, -0.06428964,
0.37781868, -0.21491644, 0.34254215, -0.03153528, -0.03100820, -0.10761415,
-0.14730405, -0.02475182, -0.22855880, 0.25150810, -0.10445128, 0.12446000,
0.07062869, 0.30880162, -0.18016875, -0.18869164, -0.34533499, -0.01291770,
0.02578168, 0.07993659, 0.28642181, 0.26038408, -0.22459419, -0.14820155,
0.04059549, -0.14043529, -0.16111187, -0.20203050, 0.02602069, 0.04852717,
0.10432153, -0.06309942, 0.38402443, -0.22397003, 0.34814481, -0.03774432,
-0.03381438, -0.11245691, -0.14128767, -0.02853208, -0.21752016, 0.24872463,
-0.11399775, 0.12226870, 0.05620835, 0.30917800, -0.18065738, -0.19401479,
-0.34495114, -0.01760592,
])
if scale is None:
scale = torch.tensor([
1.63933691, 1.70204478, 1.73642566, 1.90004803, 1.66753160, 1.69059584,
1.56853198, 1.62314944, 1.89106626, 1.58086668, 1.60822129, 1.60962993,
1.63322129, 1.56074359, 1.73419528, 1.79192650, 1.64040632, 1.66802808,
1.60390303, 1.75480492, 1.63187587, 1.64334594, 1.61722884, 1.60146046,
1.63459219, 1.55291476, 1.68771497, 1.68415657, 1.78966054, 1.66631641,
1.65626686, 1.65976433, 1.63487607, 1.69513249, 1.72933756, 1.91310663,
1.67035057, 1.72286863, 1.56719251, 1.61934825, 1.88628859, 1.56911539,
1.59455129, 1.60829869, 1.62470611, 1.56052853, 1.73677003, 1.77563606,
1.63732541, 1.66370527, 1.59508952, 1.75153949, 1.63029275, 1.64517667,
1.61659342, 1.59722044, 1.64103121, 1.54085310, 1.68610394, 1.67772755,
1.78998563, 1.66621713, 1.65458955, 1.66041308, 1.64710857, 1.68163503,
1.74000294, 1.92784786, 1.67411194, 1.67395548, 1.57406532, 1.62199356,
1.87618195, 1.55843750, 1.57438785, 1.61711053, 1.63094305, 1.55644029,
1.73124302, 1.80666627, 1.64636210, 1.65932006, 1.60816188, 1.75682671,
1.64695873, 1.63121722, 1.61380832, 1.60478651, 1.63396035, 1.53505068,
1.65534289, 1.67132281, 1.80317197, 1.67673140, 1.65700938, 1.68426259,
1.65339716, 1.67540638, 1.73298504, 1.94067348, 1.67893609, 1.70635117,
1.57309060, 1.61928553, 1.87148809, 1.56244866, 1.56697152, 1.61584394,
1.62759496, 1.55480378, 1.73484107, 1.79055143, 1.64688773, 1.66121492,
1.60135887, 1.75254572, 1.64798332, 1.62989921, 1.61381592, 1.60792883,
1.63939668, 1.53075757, 1.65371318, 1.66801185, 1.80029087, 1.67591476,
1.65655173, 1.68533454,
])
return _ShiftScalePatchAdaptor(shift=shift, scale=scale)
def make_wan21(
latents_mean: Tensor | None = None,
latents_std: Tensor | None = None,
) -> LatentFormatAdaptor:
"""Wan21 / Anima / QwenImage - per-channel z-score.
Pipeline latent = ``(VAE_raw - mean) / std``.
Default stats from the Wan21 latent format.
"""
if latents_mean is None:
latents_mean = torch.tensor([
-0.7571, -0.7089, -0.9113, 0.1075, -0.1745, 0.9653, -0.1517, 1.5508,
0.4134, -0.0715, 0.5517, -0.3632, -0.1922, -0.9497, 0.2503, -0.2921,
])
if latents_std is None:
latents_std = torch.tensor([
2.8184, 1.4541, 2.3275, 2.6558, 1.2196, 1.7708, 2.6052, 2.0743,
3.2687, 2.1526, 2.8652, 1.5579, 1.6382, 1.1253, 2.8251, 1.9160,
])
return _ZScoreAdaptor(
external_channels=16,
mean=latents_mean,
std=latents_std,
)
# ── Internal implementations ──────────────────────────────────────────────
class _AffineAdaptor(LatentFormatAdaptor):
"""Affine transform: (z - shift) * scale and inverse.
Used by SDXL and Flux where ComfyUI stores raw VAE latents between nodes
but the diffusion model (and Sesqui) operates on scaled latents.
"""
def __init__(self, external_channels: int, scale: float, shift: float):
super().__init__(external_channels=external_channels)
self.scale = scale
self.shift = shift
def to_vae_latent(self, z: Tensor) -> Tensor:
return (z.float() - self.shift) * self.scale
def from_vae_latent(self, z: Tensor) -> Tensor:
return z.float() / self.scale + self.shift
class _ZScoreAdaptor(LatentFormatAdaptor):
"""Per-channel z-score normalization (Wan21 / Anima / QwenImage)."""
def __init__(self, external_channels: int, mean: Tensor, std: Tensor):
super().__init__(external_channels=external_channels)
self.mean = mean.clone().view(1, -1, 1, 1)
self.std = std.clone().view(1, -1, 1, 1)
def _cast(self, z: Tensor) -> tuple[Tensor, Tensor]:
d = z.device
return (
self.mean.to(dtype=z.dtype, device=d),
self.std.to(dtype=z.dtype, device=d),
)
def to_vae_latent(self, z: Tensor) -> Tensor:
# Pipeline latent = (VAE_raw - mean) / std
# -> VAE_raw = z * std + mean
m, s = self._cast(z)
return z.float() * s + m
def from_vae_latent(self, z: Tensor) -> Tensor:
m, s = self._cast(z)
return (z.float() - m) / s
class _Flux2BNAdaptor(LatentFormatAdaptor):
"""Flux2: 32ch raw VAE ↔ 128ch patched + BN-normalized pipeline."""
def __init__(self, running_mean: Tensor, running_var: Tensor, eps: float = 1e-4):
super().__init__(external_channels=128, spatial_scale=2)
self.bn_mean = running_mean.clone().float().view(1, -1, 1, 1)
self.bn_var = running_var.clone().float().view(1, -1, 1, 1)
self.bn_eps = eps
def to_vae_latent(self, z: Tensor) -> Tensor:
# Pipeline = BN(patchify(VAE_raw))
# Undo BN: x * sqrt(var) + mean
var = self.bn_var.to(dtype=z.dtype, device=z.device)
mean = self.bn_mean.to(dtype=z.dtype, device=z.device)
z = z.float() * torch.sqrt(var + self.bn_eps) + mean
return _unpatchify(z)
def from_vae_latent(self, z: Tensor) -> Tensor:
z = _patchify(z)
var = self.bn_var.to(dtype=z.dtype, device=z.device)
mean = self.bn_mean.to(dtype=z.dtype, device=z.device)
return F.batch_norm(
z,
running_mean=mean.squeeze(),
running_var=var.squeeze(),
momentum=0.0,
eps=self.bn_eps,
)
class _ShiftScalePatchAdaptor(LatentFormatAdaptor):
"""Ideogram 4: 32ch raw VAE <-> 128ch patched + per-channel shift/scale.
Pipeline latent = ``patchify(VAE_raw) * scale + shift``.
"""
def __init__(self, shift: Tensor, scale: Tensor):
super().__init__(external_channels=128, spatial_scale=2)
self.shift = shift.clone().float().view(1, -1, 1, 1)
self.scale = scale.clone().float().view(1, -1, 1, 1)
def _cast(self, z: Tensor) -> tuple[Tensor, Tensor]:
d = z.device
return (
self.shift.to(dtype=z.dtype, device=d),
self.scale.to(dtype=z.dtype, device=d),
)
def to_vae_latent(self, z: Tensor) -> Tensor:
# Pipeline = patchify(VAE_raw) * scale + shift
# -> VAE_raw = unpatchify((z - shift) / scale)
s, sc = self._cast(z)
return _unpatchify((z.float() - s) / sc)
def from_vae_latent(self, z: Tensor) -> Tensor:
s, sc = self._cast(z)
return _patchify(z) * sc + s
+290
View File
@@ -0,0 +1,290 @@
"""
SesquiLSR — Multi-scale latent upscaler.
"""
import math
import torch
import torch.nn as nn
import torch.nn.functional as F
from torch import Tensor
# I like Mish, but any decent activation (like SiLU) works fine.
Activation = nn.Mish
# IR LayerNorm from https://arxiv.org/abs/2504.06629
# Removed bias to reduce chances of mean drift, also it didn't really make
# much of a difference with it in.
class IRLayerNorm(nn.Module):
def __init__(self, channels: int):
super().__init__()
self.weight = nn.Parameter(torch.ones(channels))
def forward(self, x: Tensor) -> tuple[Tensor, Tensor]:
var, mean = torch.var_mean(x, dim=(1, 2, 3), unbiased=False, keepdim=True)
std = torch.sqrt(var + 1e-6)
y = (x - mean) / std * self.weight.view(1, -1, 1, 1)
return y, std
# Inspired by WDSR / wide activation: https://arxiv.org/abs/1808.08718
# Uses a lopsided gate design to maximise values, 8:1 ratio works well
# in practise.
class GatedWDSR(nn.Module):
def __init__(
self,
channels: int,
expand_ratio: float = 2.0,
shrink_ratio: float = 0.8,
values_per_gate: int = 8,
):
super().__init__()
if values_per_gate < 1:
raise ValueError(f"values_per_gate must be >= 1, got {values_per_gate}")
gate_ch = math.ceil(expand_ratio * channels / (values_per_gate + 1))
gate_ch = math.ceil(gate_ch / values_per_gate) * values_per_gate
value_ch = gate_ch * values_per_gate
mid_ch = math.ceil(channels * shrink_ratio)
hidden = value_ch + gate_ch
self.value_ch = value_ch
self.gate_ch = gate_ch
self.values_per_gate = values_per_gate
self.norm = IRLayerNorm(channels)
self.up = nn.Conv2d(channels, hidden, 1, bias=False)
self.down = nn.Conv2d(value_ch, mid_ch, 1, bias=False)
self.mix = nn.Conv2d(
mid_ch, channels, kernel_size=3, padding=1,
padding_mode="reflect", bias=False,
)
self.act = Activation()
def forward(self, x: Tensor) -> Tensor:
y, s = self.norm(x)
v, g = torch.split(self.up(y), [self.value_ch, self.gate_ch], dim=1)
v = self.act(v)
g = torch.tanh(0.5 * g) # Bound residual and slow down saturation.
B, _, H, W = v.shape
v = v.view(B, self.gate_ch, self.values_per_gate, H, W)
y = (v * g[:, :, None, :, :]).reshape(B, self.value_ch, H, W)
y = self.down(y)
y = self.mix(y)
return x + y * s
# Originally the model uses AdaLN in every block for scale conditioning, but this
# ended up exploding. Turns out we can just do conditioning in the head, and the
# model works fine.
class PeriodicEncoder1D(nn.Module):
def __init__(self, n_freqs: int = 4, max_freq: float = 4.0):
super().__init__()
self.out_dim = 2 * n_freqs
self.register_buffer("freqs", 2.0 ** torch.linspace(0, math.log2(max_freq), n_freqs))
def forward(self, x: Tensor) -> Tensor:
angles = x.unsqueeze(-1) * self.freqs * math.pi
return torch.cat([angles.sin(), angles.cos()], dim=-1)
# The "sesqui" part. Inspired by rational/fractional resampling, we use PS to get to 2x,
# then use a learned downsample to reach our target. The original model was a fixed 1.5x;
# sesqui means "one and a half". While the fixed upscale is gone, the name remains.
class LowRankReassemblyHead(nn.Module):
def __init__(self, channels: int, refine_expand: float = 1.5, rank: int = 2):
super().__init__()
self.channels = channels
self.rank = rank
self.K = 4
self.register_buffer(
"tap_offsets", torch.tensor([-1, 0, 1, 2], dtype=torch.long),
persistent=False,
)
self.ps_expand = nn.Conv2d(channels, channels * 4, 1, bias=True)
self._init_pixel_shuffle()
lift_ch = min(64, channels // 2)
self.lift_expand = nn.Conv2d(channels, lift_ch * 4, 1, bias=False)
self.lift_mix = nn.Conv2d(lift_ch, channels, 3, padding=1, padding_mode="reflect", bias=False)
nn.init.trunc_normal_(self.lift_expand.weight, std=0.02)
nn.init.trunc_normal_(self.lift_mix.weight, std=0.02)
# You could probably use a simple conv stack here, but another resblock works well so why not.
self.refine = GatedWDSR(channels, expand_ratio=refine_expand, shrink_ratio=1.0, values_per_gate=8)
taps = torch.zeros(rank, 2, channels, self.K)
taps[0, :, :, :] = torch.tensor([-0.0625, 0.5625, 0.5625, -0.0625], dtype=taps.dtype) # ~Bicubic start
self.base_taps = nn.Parameter(taps)
self.pe = PeriodicEncoder1D(n_freqs=4)
self.axis_emb = nn.Embedding(2, 8)
self.rank_emb = nn.Embedding(rank, 8)
cond_dim = self.pe.out_dim + 1 + 8 + 8
self.filter_net = nn.Sequential(
nn.Linear(cond_dim, 64, bias=True),
Activation(),
nn.Linear(64, channels * self.K, bias=True),
)
nn.init.trunc_normal_(self.filter_net[-1].weight, std=0.02)
nn.init.zeros_(self.filter_net[-1].bias)
self.rank_gain = nn.Parameter(torch.tensor([2.0] + [0.0] * (rank - 1), dtype=torch.float32))
def _init_pixel_shuffle(self) -> None:
with torch.no_grad():
self.ps_expand.weight.zero_()
self.ps_expand.bias.zero_()
w = self.ps_expand.weight.view(self.channels, 4, self.channels)
idx = torch.arange(self.channels, device=w.device)
w[idx, :, idx] = 1.0
@staticmethod
def _axis_coords(n_src: int, n_tgt: int, device: torch.device) -> tuple[Tensor, Tensor]:
pos = (torch.arange(n_tgt, device=device, dtype=torch.float32) + 0.5)
pos = pos * (n_src / n_tgt) - 0.5
base = torch.floor(pos).long().clamp_(0, n_src - 1)
frac = pos - base.float()
return frac, base
def _predict_weights(self, fy: Tensor, fx: Tensor,
log_foot_y: float, log_foot_x: float,
batch: int) -> tuple[Tensor, Tensor]:
ny, nx = fy.numel(), fx.numel()
R = self.rank
n_axis = ny + nx
frac_axis = torch.cat([fy, fx])
log_foot = fy.new_full((n_axis, 1), log_foot_x)
log_foot[:ny] = log_foot_y
axis_id = torch.empty(n_axis, dtype=torch.long, device=fy.device)
axis_id[:ny] = 0
axis_id[ny:] = 1
cond_axis = torch.cat([
self.pe(frac_axis), log_foot, self.axis_emb(axis_id),
], dim=-1)
rank_id = torch.arange(R, device=fy.device).repeat_interleave(n_axis)
cond = torch.cat([cond_axis.repeat(R, 1), self.rank_emb(rank_id)], dim=-1)
cond = cond.to(dtype=self.base_taps.dtype)
residuals = self.filter_net(cond).view(R, n_axis, self.channels, self.K)
residuals = 0.2 * torch.tanh(residuals) # Again, bound the residual otherwise kaboom...
residuals = residuals - residuals.mean(dim=-1, keepdim=True)
wy = residuals[:, :ny] + self.base_taps[:, 0].unsqueeze(1)
wx = residuals[:, ny:] + self.base_taps[:, 1].unsqueeze(1)
ky = wy.permute(0, 2, 1, 3).reshape(R, self.channels * ny, self.K).unsqueeze(1).expand(-1, batch, -1, -1)
kx = wx.permute(0, 2, 1, 3).reshape(R, self.channels * nx, self.K).unsqueeze(1).expand(-1, batch, -1, -1)
return ky, kx
def _filter_axis(self, x: Tensor, centers: Tensor, weights: Tensor, dim: int) -> Tensor:
B, C = x.shape[0], x.shape[1]
n_out = centers.numel()
limit = x.shape[dim] - 1
idx = (centers.unsqueeze(0) + self.tap_offsets.unsqueeze(1)).clamp(0, limit)
w = weights.view(B, C, n_out, self.K)
if dim == 2:
idx_exp = idx.view(1, 1, self.K * n_out, 1).expand(B, C, -1, x.shape[3])
sample = torch.gather(x, 2, idx_exp).view(B, C, self.K, n_out, x.shape[3])
return (sample * w.permute(0, 1, 3, 2).unsqueeze(-1)).sum(dim=2)
idx_exp = idx.view(1, 1, 1, self.K * n_out).expand(B, C, x.shape[2], -1)
sample = torch.gather(x, 3, idx_exp).view(B, C, x.shape[2], self.K, n_out)
return (sample * w.permute(0, 1, 3, 2).unsqueeze(2)).sum(dim=3)
def forward(self, x: Tensor, target_size: tuple[int, int]) -> Tensor:
h_tgt, w_tgt = target_size
batch = x.shape[0]
x_base = F.pixel_shuffle(self.ps_expand(x), 2)
x_lift = self.lift_mix(F.pixel_shuffle(self.lift_expand(x), 2))
x = x_base + x_lift
h_src, w_src = x.shape[-2:]
x = self.refine(x)
fy, iy = self._axis_coords(h_src, h_tgt, x.device)
fx, ix = self._axis_coords(w_src, w_tgt, x.device)
ky, kx = self._predict_weights(
fy, fx,
math.log2(max(h_src / h_tgt, 0.01)),
math.log2(max(w_src / w_tgt, 0.01)),
batch,
)
gains = self.rank_gain.sigmoid()
out = torch.zeros(batch, self.channels, h_tgt, w_tgt, device=x.device, dtype=x.dtype)
for r in range(self.rank):
x_r = self._filter_axis(x, ix, kx[r], dim=3)
x_r = self._filter_axis(x_r, iy, ky[r], dim=2)
out = out + gains[r] * x_r
return out
# Some models/VAEs can get away with very little in terms of trunks. For example, the Qwen/Wan21 VAE
# was completely happy with 2/4 instead of 8/6. However, to keep the arch consistent, we use the same
# structure for all upscalers.
class LatentUpscaler(nn.Module):
def __init__(
self,
in_depth: int = 8,
out_depth: int = 6,
in_channels: int = 4,
width: int = 128,
in_expand: float = 1.5,
out_expand: float = 2.0,
):
super().__init__()
self.head_conv = nn.Sequential(
nn.Conv2d(in_channels, width // 2, 3, padding=1, padding_mode="reflect", bias=False),
Activation(),
nn.Conv2d(width // 2, width, 3, padding=1, padding_mode="reflect", bias=True),
)
# No difference with a slight bottleneck here and smaller expand, saves a few parameters.
self.in_blocks = nn.Sequential(*[
GatedWDSR(width, expand_ratio=in_expand, shrink_ratio=0.75, values_per_gate=8)
for _ in range(in_depth)
])
self.upsample = LowRankReassemblyHead(width, refine_expand=2.0)
# Traditional SR says do most of the work at LR, then upscale and output. The problem is
# latents are delicate structures where no amount of front-loaded work can make up for the
# damage introduced by upscaling. Post-correction is a must, and all variants tested without
# post-correction were much worse.
self.out_blocks = nn.Sequential(*[
GatedWDSR(width, expand_ratio=out_expand, shrink_ratio=1.0, values_per_gate=8)
for _ in range(out_depth)
])
self.tail_conv = nn.Sequential(
nn.Conv2d(width, width // 2, 1, bias=False),
Activation(),
nn.Conv2d(width // 2, in_channels, 3, padding=1, padding_mode="reflect", bias=True),
)
# Give the model a way to do light correction on the skip so the trunk/head can focus on
# detail/damage control.
self.skip_conv = nn.Conv2d(in_channels, in_channels, 5, padding=2, padding_mode="reflect", bias=False)
nn.init.dirac_(self.skip_conv.weight)
def forward(self, x: Tensor, target_size: tuple[int, int]) -> Tensor:
h = self.head_conv(x)
h = self.in_blocks(h)
h = self.upsample(h, target_size)
h = self.out_blocks(h)
h = self.tail_conv(h)
# Magic sauce. Learn correction on crude upsample.
x_up = self.skip_conv(F.interpolate(x, size=target_size, mode="bicubic", align_corners=False))
return h + x_up
@@ -0,0 +1,257 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Prove receiver planning, strict image provenance and source cache dependencies."""
import sys
from copy import deepcopy
from types import ModuleType
from typing import Any
import pytest
from simple_syrup.nodes_v3.ksampler import KSamplerV3
from simple_syrup.runtime.noise_reuse_graph import (
rewrite_noise_reuse_graph,
source_lineage,
)
def graph() -> dict[str, Any]:
"""Build shared source branches without invoking model IO."""
return {
"source": {
"class_type": "KSampler",
"inputs": {
"model": ["loader", 0],
"latent_image": ["empty", 0],
"sampler_name": "euler",
},
},
"decode": {
"class_type": "VAEDecode",
"inputs": {"samples": ["source", 0], "vae": ["loader", 2]},
},
"recover": {
"class_type": "SimpleSyrup.LatentFromImage",
"inputs": {"image": ["decode", 0]},
},
"upscale": {
"class_type": "SimpleSyrup.SimpleUpscaleLatent",
"inputs": {"latent": ["recover", 0], "method": "Auto", "scale_factor": 3.0},
},
"reuse": {
"class_type": "SimpleSyrup.NoiseReuseOptions",
"inputs": {"enabled": True, "finishing_steps": 0},
},
"receive": {
"class_type": "SimpleSyrup.KSampler",
"inputs": {
"model": ["loader", 0],
"latent_image": ["upscale", 0],
"steps": 16,
"sampler_name": "euler",
"scheduler": "normal",
"denoise": 0.2,
"options": ["reuse", 0],
},
},
}
def test_cube_queue_capability_installs_capture_before_source_sampling(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Share exact capture planning with direct Cube queues that omit prompt hooks."""
registry_module = ModuleType("nodes")
monkeypatch.setattr(registry_module, "NODE_CLASS_MAPPINGS", {}, raising=False)
monkeypatch.setitem(sys.modules, "nodes", registry_module)
prompt = graph()
assert KSamplerV3.SUGARCUBES_PROMPT_PREPARATION_API_VERSION == 1
KSamplerV3.prepare_sugarcubes_prompt(prompt)
assert prompt["source"]["inputs"]["model"] == ["source.trajectory_capture", 0]
assert prompt["decode"]["inputs"]["samples"] == ["source.sampling_provenance", 0]
assert (
prompt["source.trajectory_capture"]["class_type"]
== "SimpleSyrup.TrajectoryCaptureModel"
)
first = deepcopy(prompt)
KSamplerV3.prepare_sugarcubes_prompt(prompt)
assert prompt == first
def test_image_bridge_resolves_original_latent_and_both_consumers_share_capture() -> (
None
):
"""Coordinate before source execution and preserve direct latent consumers."""
prompt = graph()
prompt["finish"] = {
"class_type": "SimpleSyrup.NoiseReuseOptions",
"inputs": {"finishing_steps": 1},
}
prompt["receive_finish"] = deepcopy(prompt["receive"])
prompt["receive_finish"]["inputs"]["options"] = ["finish", 0]
assert source_lineage(prompt, ["upscale", 0], {}) == ("source", '[{"scale": 3.0}]')
rewrite_noise_reuse_graph(prompt, {})
captures = [
node
for node in prompt.values()
if node["class_type"] == "SimpleSyrup.TrajectoryCaptureModel"
]
requests = [
node
for node in prompt.values()
if node["class_type"] == "SimpleSyrup.NoiseCaptureRequest"
]
assert len(captures) == 1 and len(requests) == 2
assert requests[1]["inputs"]["previous"] == ["receive.noise_capture_request", 0]
assert prompt["source"]["inputs"]["model"] == ["source.trajectory_capture", 0]
assert prompt["decode"]["inputs"]["samples"] == ["source.sampling_provenance", 0]
assert prompt["source.sampling_provenance"]["inputs"]["latent"] == ["source", 0]
assert requests[0]["inputs"]["source_latent"] == ["empty", 0]
assert not any(node["class_type"] == "VAEEncode" for node in prompt.values())
def test_missing_or_edited_provenance_creates_actionable_error() -> None:
"""Reject pixel edits before any receiving sampler can substitute a fallback."""
prompt = graph()
prompt["decode"]["class_type"] = "ImageScale"
with pytest.raises(ValueError, match="image provenance"):
source_lineage(prompt, ["upscale", 0], {})
rewrite_noise_reuse_graph(prompt, {})
assert prompt["receive"]["inputs"]["latent_image"] == [
"receive.noise_reuse_error",
0,
]
assert (
prompt["receive.noise_reuse_error"]["class_type"]
== "SimpleSyrup.NoiseReuseError"
)
def test_disabled_reuse_leaves_native_graph_unchanged() -> None:
"""Avoid upstream capture or new GPU work when the option is switched off."""
prompt = graph()
prompt["upscale"]["inputs"]["method"] = "bilinear"
prompt["reuse"]["inputs"]["enabled"] = False
before = deepcopy(prompt)
rewrite_noise_reuse_graph(prompt, {})
assert prompt == before
def test_auto_only_requests_format_provenance_without_state_capture() -> None:
"""Identify compatible models without perturbing the original source schedule."""
prompt = graph()
prompt["reuse"]["inputs"]["enabled"] = False
rewrite_noise_reuse_graph(prompt, {})
assert "requests" not in prompt["source.trajectory_capture"]["inputs"]
assert not any(
node["class_type"] == "SimpleSyrup.NoiseCaptureRequest"
for node in prompt.values()
)
def test_auto_without_sampler_lineage_keeps_native_fallback_input() -> None:
"""Allow unknown encoded latents while keeping noise reuse source checks strict."""
prompt = graph()
prompt["reuse"]["inputs"]["enabled"] = False
prompt["upscale"]["inputs"]["latent"] = ["encode", 0]
prompt["encode"] = {"class_type": "VAEEncode", "inputs": {}}
before = deepcopy(prompt)
rewrite_noise_reuse_graph(prompt, {})
assert prompt == before
prompt["reuse"]["inputs"]["enabled"] = True
rewrite_noise_reuse_graph(prompt, {})
assert prompt["upscale"]["inputs"]["latent"] == ["encode", 0]
assert prompt["receive"]["inputs"]["latent_image"] == [
"receive.noise_reuse_error",
0,
]
def test_requeued_coordinated_prompt_is_idempotent() -> None:
"""Keep history requeues from installing duplicate keyed source wrappers."""
prompt = graph()
rewrite_noise_reuse_graph(prompt, {})
before = deepcopy(prompt)
rewrite_noise_reuse_graph(prompt, {})
assert prompt == before
def test_receiving_settings_are_live_source_cache_dependencies() -> None:
"""Invalidate source capture when the receiving schedule or options change."""
prompt = graph()
prompt["receive"]["inputs"]["steps"] = ["live_steps", 0]
prompt["receive"]["inputs"]["denoise"] = ["live_denoise", 0]
rewrite_noise_reuse_graph(prompt, {})
request = prompt["receive.noise_capture_request"]["inputs"]
assert request["steps"] == ["live_steps", 0]
assert request["denoise"] == ["live_denoise", 0]
assert request["options"] == ["reuse", 0]
capture = prompt["source.trajectory_capture"]["inputs"]
assert capture["requests"] == ["receive.noise_capture_request", 0]
@pytest.mark.parametrize(
"kind",
[
"SimpleSyrup.KSamplerAttentionCoupling",
"SimpleSyrup.KSamplerAttentionCouplingContextual",
"SimpleSyrup.KSamplerAttentionCouplingTiled",
"SimpleSyrup.KSamplerPromptByRegion",
"SimpleSyrup.KSamplerPromptByTiledRegion",
"ExternalCompatibleSampler",
],
)
def test_sampling_contract_discovers_regional_and_external_sources(kind: str) -> None:
"""Capture new sampler nodes naturally when they expose the native contract."""
prompt = graph()
prompt["source"]["class_type"] = kind
prompt["source"]["inputs"]["sampler_name"] = "er_sde"
prompt["source"]["_meta"] = {"title": "Original pajamas"}
prompt["receive"]["_meta"] = {"title": "Upscale pajamas"}
rewrite_noise_reuse_graph(prompt, {})
capture = prompt["source.trajectory_capture"]["inputs"]
assert capture["sampler_name"] == "er_sde"
assert "Original pajamas" in capture["graph_context"]
assert "Upscale pajamas" in capture["graph_context"]
assert kind in capture["graph_context"]
@pytest.mark.parametrize("advanced", [False, True])
def test_custom_sampling_capture_preserves_sampler_and_guider_consumers(
advanced: bool,
) -> None:
"""Clone only the source guider branch while observing any connected solver."""
prompt = graph()
source = prompt["source"]
source["class_type"] = "SamplerCustomAdvanced" if advanced else "SamplerCustom"
source["inputs"].pop("sampler_name")
source["inputs"].update(sampler=["select", 0], sigmas=["sigmas", 0])
prompt["select"] = {
"class_type": "KSamplerSelect",
"inputs": {"sampler_name": "dpmpp_2m"},
}
prompt["guidance"] = {
"class_type": "CFGGuider",
"inputs": {"model": ["loader", 0], "cfg": 4.0},
}
original_guider = deepcopy(prompt["guidance"])
if advanced:
source["inputs"].pop("model")
source["inputs"]["guider"] = ["guidance", 0]
rewrite_noise_reuse_graph(prompt, {})
assert prompt["guidance"] == original_guider
assert source["inputs"]["sampler"] == ["select", 0]
capture = prompt["source.trajectory_capture"]["inputs"]
assert capture["sampler_name"] == "dpmpp_2m"
assert capture["sampler"] == ["select", 0]
if advanced:
assert source["inputs"]["guider"] == ["source.trajectory_guider", 0]
assert prompt["source.trajectory_guider"]["inputs"]["model"] == [
"source.trajectory_capture",
0,
]
else:
assert source["inputs"]["model"] == ["source.trajectory_capture", 0]
@@ -0,0 +1,259 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Prove capture preparation precedes native validation for every queue caller."""
from __future__ import annotations
import asyncio
import inspect
import sys
from copy import deepcopy
from types import ModuleType
from typing import Any
import pytest
from simple_syrup.runtime.noise_reuse_validation import (
NativePromptValidator,
prepare_noise_reuse_prompt,
register_noise_reuse_validation,
)
class ValidationModule(ModuleType):
"""Describe the isolated host module's replaceable validation function."""
validate_prompt: NativePromptValidator
async def validate(
execution: ValidationModule,
prompt_id: str,
prompt: dict[str, Any],
partial_targets: list[str] | None,
) -> tuple[bool, object, object, object]:
"""Await the external validator whether it returns a coroutine or awaitable."""
return await execution.validate_prompt(prompt_id, prompt, partial_targets)
def capture_graph() -> dict[str, Any]:
"""Describe direct source lineage and a live receiving capture dependency."""
return {
"source": {
"class_type": "KSampler",
"inputs": {
"model": ["loader", 0],
"latent_image": ["empty", 0],
"sampler_name": "euler_ancestral",
},
},
"reuse": {
"class_type": "SimpleSyrup.NoiseReuseOptions",
"inputs": {"enabled": True},
},
"receive": {
"class_type": "SimpleSyrup.KSampler",
"inputs": {
"model": ["loader", 0],
"latent_image": ["source", 0],
"options": ["reuse", 0],
"steps": 12,
"sampler_name": "euler_ancestral",
"scheduler": "normal",
"denoise": 0.15,
},
},
}
@pytest.fixture
def validation_host(
monkeypatch: pytest.MonkeyPatch,
) -> tuple[ValidationModule, list[tuple[str, dict[str, Any], list[str] | None]]]:
"""Isolate only Comfy's external validation and registration boundaries."""
calls: list[tuple[str, dict[str, Any], list[str] | None]] = []
async def validate_prompt(
prompt_id: str,
prompt: dict[str, Any],
partial_execution_list: list[str] | None,
) -> tuple[bool, object, object, object]:
"""Record the graph exactly as the native validator receives it."""
calls.append((prompt_id, deepcopy(prompt), partial_execution_list))
return True, None, ["receive"], {}
execution = ValidationModule("execution")
execution.validate_prompt = validate_prompt
nodes = ModuleType("nodes")
nodes.NODE_CLASS_MAPPINGS = {} # type: ignore[attr-defined]
monkeypatch.setitem(sys.modules, "execution", execution)
monkeypatch.setitem(sys.modules, "nodes", nodes)
return execution, calls
@pytest.mark.parametrize("partial_targets", [None, ["receive"]])
def test_direct_validation_prepares_capture_without_http_handler(
validation_host: tuple[
ValidationModule, list[tuple[str, dict[str, Any], list[str] | None]]
],
partial_targets: list[str] | None,
) -> None:
"""Cover both full Cube queueing and Comfy partial-output validation."""
execution, calls = validation_host
register_noise_reuse_validation()
prompt = capture_graph()
result = asyncio.run(validate(execution, "run", prompt, partial_targets))
assert result == (True, None, ["receive"], {})
assert len(calls) == 1
prompt_id, validated, targets = calls[0]
assert prompt_id == "run" and targets is partial_targets
assert validated == prompt
assert validated["source"]["inputs"]["model"] == ["source.trajectory_capture", 0]
assert validated["receive"]["inputs"]["latent_image"] == [
"source.sampling_provenance",
0,
]
assert validated["receive.noise_capture_request"]["inputs"]["denoise"] == 0.15
def test_registration_and_history_requeue_do_not_duplicate_capture(
validation_host: tuple[
ValidationModule, list[tuple[str, dict[str, Any], list[str] | None]]
],
) -> None:
"""Preserve graph identity across registration, pre-preparation and history."""
execution, calls = validation_host
register_noise_reuse_validation()
validator = execution.validate_prompt
register_noise_reuse_validation()
assert execution.validate_prompt is validator
prompt = capture_graph()
prepare_noise_reuse_prompt(prompt)
prepared = deepcopy(prompt)
for prompt_id in ("first", "requeue"):
asyncio.run(validate(execution, prompt_id, prompt, None))
assert prompt == prepared and len(calls) == 2
assert all(validated == prepared for _, validated, _ in calls)
def test_registration_preserves_async_signature_and_wrapped_validator(
validation_host: tuple[
ValidationModule, list[tuple[str, dict[str, Any], list[str] | None]]
],
) -> None:
"""Keep native function inspection and extension chaining compatible."""
execution, _ = validation_host
native = execution.validate_prompt
register_noise_reuse_validation()
wrapped = execution.validate_prompt
assert inspect.iscoroutinefunction(wrapped)
assert inspect.signature(wrapped) == inspect.signature(native)
assert inspect.unwrap(wrapped) is native
def test_changed_receiving_controls_remain_source_cache_dependencies(
validation_host: tuple[
ValidationModule, list[tuple[str, dict[str, Any], list[str] | None]]
],
) -> None:
"""Keep fresh submissions sensitive to downstream denoise and live steps."""
execution, calls = validation_host
register_noise_reuse_validation()
for denoise in (0.15, 0.2):
prompt = capture_graph()
prompt["receive"]["inputs"].update(denoise=denoise, steps=["live_steps", 0])
asyncio.run(validate(execution, "run", prompt, None))
requests = [
graph["receive.noise_capture_request"]["inputs"] for _, graph, _ in calls
]
assert [request["denoise"] for request in requests] == [0.15, 0.2]
assert all(request["steps"] == ["live_steps", 0] for request in requests)
def test_unrelated_graph_and_validation_rejection_are_preserved(
validation_host: tuple[
ValidationModule, list[tuple[str, dict[str, Any], list[str] | None]]
],
) -> None:
"""Leave ordinary workflows and native validation error payloads intact."""
execution, _ = validation_host
prompt = {"output": {"class_type": "SaveImage", "inputs": {}}}
before = deepcopy(prompt)
rejection: tuple[bool, object, object, object] = (
False,
{"type": "native_error"},
[],
{"output": {"errors": []}},
)
async def reject(
prompt_id: str, prompt: dict[str, Any], partial_execution_list: list[str] | None
) -> tuple[bool, object, object, object]:
"""Return the host's validation rejection without extension translation."""
return rejection
execution.validate_prompt = reject
register_noise_reuse_validation()
assert asyncio.run(validate(execution, "run", prompt, None)) is rejection
assert prompt == before
def test_native_validation_exception_propagates(
validation_host: tuple[
ValidationModule, list[tuple[str, dict[str, Any], list[str] | None]]
],
) -> None:
"""Keep unexpected host failures visible to the queue that called it."""
execution, _ = validation_host
async def fail(
prompt_id: str, prompt: dict[str, Any], partial_execution_list: list[str] | None
) -> tuple[bool, object, object, object]:
"""Represent a failed external native validator."""
raise RuntimeError("native validation failed")
execution.validate_prompt = fail
register_noise_reuse_validation()
with pytest.raises(RuntimeError, match="native validation failed"):
asyncio.run(validate(execution, "run", capture_graph(), None))
def test_preparation_failure_stops_before_validation(
validation_host: tuple[
ValidationModule, list[tuple[str, dict[str, Any], list[str] | None]]
],
monkeypatch: pytest.MonkeyPatch,
caplog: pytest.LogCaptureFixture,
) -> None:
"""Reject an unexpected planner failure rather than silently skipping capture."""
execution, calls = validation_host
def fail(prompt: dict[str, Any]) -> None:
"""Represent failure at the graph preparation boundary."""
raise RuntimeError("capture planning failed")
monkeypatch.setattr(
"simple_syrup.runtime.noise_reuse_validation.prepare_noise_reuse_prompt", fail
)
register_noise_reuse_validation()
with pytest.raises(RuntimeError, match="capture planning failed"):
asyncio.run(validate(execution, "run", capture_graph(), None))
assert calls == []
assert "capture planning failed" in caplog.text
def test_registration_without_comfy_runtime_is_safe(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Allow tooling imports when Comfy's execution module is unavailable."""
def unavailable(name: str) -> ModuleType:
"""Represent an absent external host installation."""
raise ModuleNotFoundError("No module named execution", name="execution")
monkeypatch.setattr(
"simple_syrup.runtime.noise_reuse_validation.import_module", unavailable
)
register_noise_reuse_validation()
@@ -18,6 +18,18 @@ from support.repository import REPOSITORY_ROOT
BASE_NODE_IDS = [
"SimpleSyrup.AllPromptAttentionSEGS",
"SimpleSyrup.AttentionCouplingOptions",
"SimpleSyrup.ContextualDiffusionOptions",
"SimpleSyrup.NoiseInversionOptions",
"SimpleSyrup.TilingOptions",
"SimpleSyrup.KSampler",
"SimpleSyrup.LatentFromImage",
"SimpleSyrup.SimpleUpscaleLatent",
"SimpleSyrup.NoiseReuseOptions",
"SimpleSyrup.NoiseCaptureRequest",
"SimpleSyrup.TrajectoryCaptureModel",
"SimpleSyrup.AttachSamplingProvenance",
"SimpleSyrup.NoiseReuseError",
"SimpleSyrup.AttentionCaptureModel",
"SimpleSyrup.AttentionMaskedConditioning",
"SimpleSyrup.AttentionRegionMask",
+220
View File
@@ -0,0 +1,220 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Verify native geometry, capture preservation and learned format selection."""
from importlib import import_module
from pathlib import Path
from typing import Any
import pytest
import torch
from simple_syrup.domain.latent_upscale import (
learned_upscale_stages,
recommended_family,
)
from simple_syrup.domain.noise_reuse import PROVENANCE_KEY, SamplingProvenance
from simple_syrup.runtime import latent_upscale_models
from simple_syrup.runtime.latent_upscale_catalog import latent_model_choices
from simple_syrup.runtime.latent_upscale_models import resolve_latent_upscale_model
from simple_syrup.services.latent_upscale_service import LatentUpscaleService
def test_stages_reach_three_times_without_exceeding_trained_range() -> None:
"""Use 2x then 1.5x and preserve exact rounded endpoint dimensions."""
assert learned_upscale_stages((96, 168), (288, 504)) == ((192, 336), (288, 504))
assert learned_upscale_stages((192, 256), (288, 384)) == ((288, 384),)
assert learned_upscale_stages((4, 4), (4, 4)) == ()
with pytest.raises(ValueError, match="enlargement"):
learned_upscale_stages((8, 8), (4, 4))
def test_native_comfy_resize_preserves_provenance_and_resizes_mask() -> None:
"""Exercise the real Comfy native node and preserve capture ownership."""
samples = torch.arange(96).float().reshape(1, 4, 4, 6)
provenance = SamplingProvenance("SDXL", samples)
latent: dict[str, Any] = {
"samples": samples,
PROVENANCE_KEY: provenance,
"noise_mask": torch.ones(1, 4, 6),
}
output = LatentUpscaleService().upscale(latent, "nearest-exact", 1.5)
assert output["samples"].shape == (1, 4, 6, 9)
assert output["noise_mask"].shape == (1, 6, 9)
assert output[PROVENANCE_KEY] is provenance
assert latent["samples"] is samples and latent["noise_mask"].shape == (1, 4, 6)
@pytest.mark.parametrize(
"latent_format,family",
(
("SDXL", "sdxl"),
("Flux", "flux"),
("Flux2", "flux2"),
("Wan21", "wan"),
("Cosmos1CV8x8x8", "wan"),
),
)
def test_auto_uses_verified_format_instead_of_channel_guess(
latent_format: str, family: str
) -> None:
"""Distinguish Flux and Wan even though both can have 16 channels."""
assert recommended_family(latent_format) == family
def test_auto_and_wrong_family_fail_before_download() -> None:
"""Reject ambiguous source identity and incompatible explicit checkpoints."""
with pytest.raises(ValueError, match="provenance"):
resolve_latent_upscale_model({"samples": torch.zeros(1, 16, 4, 4)}, "Auto")
with pytest.raises(ValueError, match="does not match"):
resolve_latent_upscale_model(
{PROVENANCE_KEY: SamplingProvenance("Flux", torch.zeros(1, 16, 4, 4))},
"wan/upscaler_Wan21.safetensors",
)
with pytest.raises(ValueError, match="compatible"):
recommended_family("SD15")
def test_method_choices_deduplicate_registered_windows_paths(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Keep one choice per installed checkpoint while retaining native algorithms."""
folder_paths = import_module("folder_paths")
monkeypatch.setattr(
folder_paths,
"get_filename_list",
lambda category: [
r"wan\upscaler_Wan21.safetensors",
r"custom\example.safetensors",
],
)
choices = latent_model_choices()
assert choices[0] == "Auto"
assert choices.count("wan/upscaler_Wan21.safetensors") == 1
assert "custom/example.safetensors" in choices and "nearest-exact" in choices
def test_invalid_samples_and_masks_fail_before_model_resolution(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Avoid downloads or inference when tensor inputs are invalid."""
from simple_syrup.services import latent_upscale_service
def forbidden(*args: Any) -> Any:
"""Detect side effects before input validation completes."""
raise AssertionError("Model resolution must not occur.")
monkeypatch.setattr(
latent_upscale_service, "resolve_latent_upscale_model", forbidden
)
service = LatentUpscaleService()
with pytest.raises(ValueError, match="finite"):
service.upscale({"samples": torch.full((1, 4, 8, 12), float("nan"))}, "Auto", 2)
with pytest.raises(TypeError, match="mask"):
service.upscale(
{"samples": torch.zeros(1, 4, 8, 12), "noise_mask": "invalid"}, "Auto", 2
)
def test_registered_original_filename_retains_family_identity(
monkeypatch: pytest.MonkeyPatch,
tmp_path: Path,
) -> None:
"""Distinguish same-channel Flux and Wan weights outside the default folders."""
folders = import_module("folder_paths")
utils = import_module("comfy.utils")
monkeypatch.setattr(
folders,
"get_full_path_or_raise",
lambda category, name: str(
tmp_path / "custom_models" / "upscaler_Flux.safetensors"
),
)
monkeypatch.setattr(
utils,
"load_torch_file",
lambda path, **kwargs: {
"head_conv.0.weight": torch.zeros(8, 16, 3, 3),
"upsample.base_taps": torch.ones(1),
},
)
latent = {PROVENANCE_KEY: SamplingProvenance("Wan21", torch.zeros(1, 16, 4, 4))}
with pytest.raises(ValueError, match="does not match"):
latent_upscale_models.resolve_latent_upscale_model(
latent, "custom/upscaler_Flux.safetensors"
)
@pytest.mark.parametrize("latent_format", (None, "SD15", "UnknownLatentFormat"))
@pytest.mark.parametrize("shape", ((2, 16, 4, 6), (1, 16, 2, 4, 6)))
def test_auto_unknown_latents_use_native_bislerp_without_model_io(
monkeypatch: pytest.MonkeyPatch,
latent_format: str | None,
shape: tuple[int, ...],
) -> None:
"""Match native bislerp exactly for unrecognized image and video latents."""
from simple_syrup.services import latent_upscale_service
def forbidden(*args: Any) -> Any:
"""Reject checkpoint lookup or downloads on the native fallback path."""
raise AssertionError("Unknown latents must use native bislerp.")
monkeypatch.setattr(
latent_upscale_service, "resolve_latent_upscale_model", forbidden
)
samples = torch.randn(shape, generator=torch.Generator().manual_seed(53))
latent: dict[str, Any] = {
"samples": samples,
"noise_mask": torch.ones(1, 4, 6),
"batch_index": [0],
}
if latent_format is not None:
latent[PROVENANCE_KEY] = SamplingProvenance(latent_format, samples)
expected = (
import_module("nodes").LatentUpscaleBy().upscale(latent, "bislerp", 1.5)[0]
)
output = LatentUpscaleService().upscale(latent, "Auto", 1.5)
torch.testing.assert_close(output["samples"], expected["samples"], rtol=0, atol=0)
assert output["noise_mask"].shape == (1, 6, 9)
assert output["batch_index"] is latent["batch_index"]
assert output.get(PROVENANCE_KEY) is latent.get(PROVENANCE_KEY)
assert latent["samples"] is samples
@pytest.mark.parametrize(
"latent_format", ("SDXL", "Flux", "Flux2", "Wan21", "Cosmos1CV8x8x8")
)
def test_auto_recognized_latents_keep_learned_model_selection(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path, latent_format: str
) -> None:
"""Keep existing learned Auto selection for every supported format."""
from simple_syrup.services import latent_upscale_service
samples = torch.ones(1, 4, 4, 6)
latent = {
"samples": samples,
PROVENANCE_KEY: SamplingProvenance(latent_format, samples),
}
family = recommended_family(latent_format)
checkpoint = tmp_path / "upscaler.safetensors"
def resolve(value: dict[str, Any], method: str) -> tuple[str, Path]:
"""Verify known formats reach the authoritative learned resolver."""
assert value is latent and method == "Auto"
return family, checkpoint
def infer(
value: torch.Tensor, *, path: Path, family: str, target: tuple[int, int]
) -> torch.Tensor:
"""Verify the learned inference boundary receives the chosen checkpoint."""
assert value is samples and path == checkpoint
assert family == recommended_family(latent_format) and target == (6, 9)
return torch.ones(1, 4, *target)
monkeypatch.setattr(latent_upscale_service, "resolve_latent_upscale_model", resolve)
monkeypatch.setattr(latent_upscale_service, "upscale_with_sesqui", infer)
output = LatentUpscaleService().upscale(latent, "Auto", 1.5)
assert output["samples"].shape == (1, 4, 6, 9)
assert output[PROVENANCE_KEY] is latent[PROVENANCE_KEY]
+58
View File
@@ -0,0 +1,58 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Exercise actual learned inference in Comfy's external latent units and layouts."""
from importlib import import_module
from pathlib import Path
from typing import Any
import pytest
import torch
from safetensors.torch import save_file
from simple_syrup.runtime.sesqui_latent_upscale import upscale_with_sesqui
@pytest.mark.parametrize(
"family,channels,video",
(("sdxl", 4, False), ("flux", 16, False), ("wan", 16, True), ("flux2", 128, False)),
)
def test_learned_adapter_matches_direct_raw_vae_inference(
tmp_path: Path, family: str, channels: int, video: bool
) -> None:
"""Detect extra latent normalization, packed Flux2 mistakes and frame transposes."""
vendor: Any = import_module("simple_syrup.third_party.sesqui_lsr")
raw_channels = 32 if channels == 128 else channels
with torch.random.fork_rng():
torch.manual_seed(42)
network = vendor.LatentUpscaler(in_channels=raw_channels).eval()
samples = torch.randn(
(2, channels, 3, 8, 12) if video else (2, channels, 8, 12)
)
path = tmp_path / "learned.safetensors"
save_file(network.state_dict(), str(path))
flat = (
samples.permute(0, 2, 1, 3, 4).reshape(6, channels, 8, 12) if video else samples
)
adaptor = vendor.make_flux2() if channels == 128 else vendor.make_identity(channels)
with torch.inference_mode():
expected = adaptor.from_vae_latent(
network(adaptor.to_vae_latent(flat), adaptor.vae_target_size((12, 18)))
)
if video:
expected = expected.reshape(2, 3, channels, 12, 18).permute(0, 2, 1, 3, 4)
output = upscale_with_sesqui(samples, path=path, family=family, target=(12, 18))
assert output.shape == expected.shape and output.dtype == samples.dtype
assert torch.equal(output, expected)
def test_invalid_channels_fail_before_checkpoint_loading(tmp_path: Path) -> None:
"""Reject incompatible learned format layouts before model IO occurs."""
with pytest.raises(ValueError, match="16 channels"):
upscale_with_sesqui(
torch.zeros(1, 4, 8, 12),
path=tmp_path / "missing.safetensors",
family="wan",
target=(12, 18),
)
@@ -32,6 +32,22 @@ FORBIDDEN_PATCHER_CALLS = frozenset(
FORBIDDEN_PATCHER_WRITES = frozenset({"forced_hooks", "use_clip_schedule"})
APPROVED_VALUE_CLONES = Counter(
{
("simple_syrup/domain/spectral_noise.py", "spatial_dct(fresh)"): 1,
(
"simple_syrup/runtime/trajectory_capture.py",
"state.detach().float().cpu()",
): 1,
(
"simple_syrup/runtime/trajectory_capture.py",
"result.detach().float().cpu()",
): 1,
(
"simple_syrup/runtime/trajectory_capture.py",
"schedule.detach().float().cpu()",
): 1,
("simple_syrup/domain/inversion_solver.py", "source"): 1,
("simple_syrup/runtime/noise_inversion.py", "zero"): 1,
("simple_syrup/runtime/noise_inversion.py", "noise"): 1,
("simple_syrup/domain/semantic_tiled_diffusion.py", "mask"): 2,
("simple_syrup/image/crop_composite.py", "image"): 1,
(
+77
View File
@@ -0,0 +1,77 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Verify the public latent reuse node contracts and v3 registration."""
from typing import Any
import pytest
from simple_syrup.domain.sampler_options import (
ContextualDiffusionOptions,
SamplerOptions,
)
from simple_syrup.nodes_v3 import get_nodes
from simple_syrup.nodes_v3.latent_from_image import LatentFromImageV3
from simple_syrup.nodes_v3.noise_reuse_options import NoiseReuseOptionsV3
from simple_syrup.nodes_v3.simple_upscale_latent import SimpleUpscaleLatentV3
from simple_syrup.services.latent_provenance_service import LatentProvenanceService
@pytest.mark.parametrize(
"node", (LatentFromImageV3, NoiseReuseOptionsV3, SimpleUpscaleLatentV3)
)
def test_v3_registration_and_tooltips_cover_public_contract(node: Any) -> None:
"""Export all three nodes with descriptions and actionable socket tooltips."""
assert node in get_nodes()
schema = node.define_schema()
assert schema.node_id.startswith("SimpleSyrup.")
assert schema.description
assert all(value.tooltip for value in (*schema.inputs, *schema.outputs))
def test_upscale_exposes_one_dropdown_and_scale_default() -> None:
"""Keep format and model loading choices behind the agreed method control."""
schema = SimpleUpscaleLatentV3.define_schema()
inputs = {value.id: value for value in schema.inputs}
assert list(inputs) == ["latent", "method", "scale_factor"]
assert inputs["method"].default == "Auto"
assert "bislerp" in inputs["method"].tooltip.lower()
assert inputs["scale_factor"].default == 2.0
assert schema.display_name == "Simple Upscale Latent"
def test_reuse_enabled_is_first_widget_and_disabled_passes_same_prefix() -> None:
"""Prove zero finishing is reuse-only and switching off preserves prior options."""
schema = NoiseReuseOptionsV3.define_schema()
inputs = {value.id: value for value in schema.inputs}
assert list(inputs) == [
"options",
"enabled",
"finishing_steps",
"finishing_method",
"finishing_start_fraction",
]
assert inputs["enabled"].default is True
assert inputs["finishing_steps"].default == 0
prefix = SamplerOptions(contextual_diffusion=ContextualDiffusionOptions())
assert NoiseReuseOptionsV3.execute(enabled=False, options=prefix)[0] is prefix
enabled = NoiseReuseOptionsV3.execute(options=prefix)[0]
assert enabled.contextual_diffusion is prefix.contextual_diffusion
assert enabled.noise_reuse is not None and enabled.noise_reuse.finishing_steps == 0
def test_provenance_recovery_has_no_vae_or_widgets_and_never_encodes() -> None:
"""Return graph lineage directly instead of producing a re-encoded approximation."""
schema = LatentFromImageV3.define_schema()
assert [value.id for value in schema.inputs] == ["image"]
service = LatentProvenanceService()
prompt = {
"decode": {
"class_type": "VAEDecode",
"inputs": {"samples": ["sample", 0], "vae": ["loader", 2]},
}
}
assert service.recover_link(prompt, ["decode", 0], {}) == ("sample", 0)
with pytest.raises(ValueError, match="unmodified"):
service.recover_link({}, ["loaded_image", 0], {})
@@ -0,0 +1,98 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Verify optional inversion widgets on implementation-backed V3 sampler nodes."""
from __future__ import annotations
from typing import Any, ClassVar
import pytest
from simple_syrup.domain.noise_inversion import NoiseInversionOptions
from simple_syrup.nodes_v3.legacy_inversion_node_adapter import (
LegacyInversionNodeV3Adapter,
)
from simple_syrup.nodes_v3.legacy_node_wrappers import (
DetailSEGSAsRegionsV3,
DetailSEGSByScaleFactorTiledDiffusionV3,
)
class RecordingImplementation:
"""Expose a minimal maintained declaration and the execution boundary."""
RETURN_TYPES = ("STRING",)
OUTPUT_TOOLTIPS = ("Result.",)
FUNCTION = "run"
CATEGORY = "SimpleSyrup/Test"
DESCRIPTION = "Records sampling inputs."
INPUT_IS_LIST = False
@classmethod
def INPUT_TYPES(cls) -> dict[str, Any]:
"""Keep the persisted input contract independent of inversion widgets."""
return {"required": {"text": ("STRING", {"default": "", "tooltip": "Text."})}}
def run(
self, text: object, noise_inversion: NoiseInversionOptions | None = None
) -> tuple[object, NoiseInversionOptions | None]:
"""Return delegated inputs without invoking Comfy neural execution."""
return text, noise_inversion
class RecordingAdapter(LegacyInversionNodeV3Adapter):
"""Use the production schema and list-mode normalization implementation."""
LEGACY_NODE_CLASS: ClassVar[type[Any]] = RecordingImplementation
NODE_ID = "SimpleSyrup.Recording"
DISPLAY_NAME = "Recording"
@pytest.mark.parametrize("list_mode", [False, True])
def test_inversion_controls_normalize_and_default_on(
monkeypatch: pytest.MonkeyPatch,
list_mode: bool,
) -> None:
"""Default to the accepted recipe and normalize zero-step disable in list mode."""
monkeypatch.setattr(RecordingImplementation, "INPUT_IS_LIST", list_mode)
text = ["prompt"] if list_mode else "prompt"
output, enabled = RecordingAdapter.execute(text=text)
assert output == text and enabled == NoiseInversionOptions()
_, disabled = RecordingAdapter.execute(
text=text,
inversion_steps=[0] if list_mode else 0,
)
assert disabled is None
@pytest.mark.parametrize(
"controls",
[
{"inversion_steps": -1},
{"inversion_resolution_scale": 0},
{"inversion_method": "fireflow"},
{"inversion_switch_fraction": 1},
],
)
def test_invalid_selected_inversion_controls_fail(controls: dict[str, Any]) -> None:
"""Apply domain validation instead of handing malformed controls to a sampler."""
with pytest.raises(ValueError):
RecordingAdapter.execute(text="prompt", **controls)
@pytest.mark.parametrize(
"node",
[DetailSEGSAsRegionsV3, DetailSEGSByScaleFactorTiledDiffusionV3],
)
def test_existing_nodes_append_optional_inversion_without_reordering(node: Any) -> None:
"""Preserve sampler socket ordering before five optional inversion controls."""
schema = node.define_schema()
order = node.WORKFLOW_INPUT_ORDER
assert [item.id for item in schema.inputs[: len(order)]] == list(order)
new = schema.inputs[len(order) :]
assert len(new) == 5 and all(item.optional and item.tooltip for item in new)
assert new[0].id == "inversion_method" and new[0].default == "euler"
steps = next(item for item in new if item.id == "inversion_steps")
assert steps.default == 2 and steps.min == 0
@@ -8,7 +8,7 @@ from __future__ import annotations
from typing import Any, ClassVar
from simple_syrup.nodes_v3.legacy_node_wrappers import LegacyNodeV3Adapter
from simple_syrup.nodes_v3.legacy_node_adapter import LegacyNodeV3Adapter
class _FakeHidden:
@@ -0,0 +1,182 @@
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
# Copyright (C) 2026 Artificial Sweetener and contributors
# SPDX-License-Identifier: AGPL-3.0-or-later
"""Safeguard Krea NegPiP at both supported Comfy model call boundaries."""
from __future__ import annotations
from typing import cast
import pytest
import torch
from comfy.ldm.krea2.model import Attention, SingleStreamDiT
from comfy.model_patcher import ModelPatcher
from simple_syrup.runtime.negpip.krea2 import (
CONDITION_MASK_KEY,
TRANSFORMER_MASK_KEY,
krea2_attn1_negpip,
krea2_diffusion_negpip_wrapper,
)
from simple_syrup.runtime.negpip.krea2_host import (
Krea2HostAttention,
krea2_host_mutations,
)
from simple_syrup.runtime.patcher_lifecycle import PATCHER_LIFECYCLE
@pytest.mark.parametrize("reference_argument", (False, True))
def test_diffusion_wrapper_preserves_both_host_signatures(
reference_argument: bool,
) -> None:
"""Inject one local mask without binding transformer options twice."""
source_options: dict[str, object] = {"existing": True}
mask = torch.tensor([[[-1.0], [1.0]]])
def earlier(
x: object,
timesteps: object,
context: object,
attention_mask: object,
transformer_options: dict[str, object],
**kwargs: object,
) -> dict[str, object]:
"""Expose Comfy 0.28's exact five-positional model boundary."""
del x, timesteps, context, attention_mask, kwargs
return transformer_options
def current(
x: object,
timesteps: object,
context: object,
attention_mask: object,
ref_latents: object,
transformer_options: dict[str, object],
**kwargs: object,
) -> dict[str, object]:
"""Expose Comfy's reference-capable six-positional model boundary."""
del x, timesteps, context, attention_mask, kwargs
assert ref_latents is None
return transformer_options
args = (
(None, None, None, None, None, source_options)
if reference_argument
else (None, None, None, None, source_options)
)
prepared = cast(
dict[str, object],
krea2_diffusion_negpip_wrapper(
current if reference_argument else earlier,
*args,
**{CONDITION_MASK_KEY: mask},
),
)
assert prepared is not source_options
assert prepared[TRANSFORMER_MASK_KEY] is mask
assert prepared["existing"] is True
assert TRANSFORMER_MASK_KEY not in source_options
@pytest.mark.parametrize("negative_sign", (False, True))
def test_host_attention_matches_native_math_and_keeps_call_options_local(
negative_sign: bool,
) -> None:
"""Earlier host hooks reproduce native grouped attention without source mutation."""
torch.manual_seed(4)
attention = Attention(128, 4, kvheads=2, operations=torch.nn)
with torch.no_grad():
attention.qknorm.qnorm.scale.zero_()
attention.qknorm.knorm.scale.zero_()
x = torch.randn(1, 7, 128)
mask = torch.tensor([[[-1.0 if negative_sign else 1.0], [1.0]]])
options = {
TRANSFORMER_MASK_KEY: mask,
"patches": {"attn1_patch": [krea2_attn1_negpip]},
}
native_options = {**options, "block_index": 0, "img_slice": [2, 7]}
with torch.no_grad():
expected = attention(x, transformer_options=native_options)
actual = Krea2HostAttention(attention, 0, 28)(x, transformer_options=options)
assert torch.equal(actual, expected)
assert "block_index" not in options
assert "img_slice" not in options
def test_host_attention_rejects_malformed_token_mask() -> None:
"""Malformed sign metadata must fail before executing attention projections."""
with pytest.raises(ValueError, match="processed token sign mask"):
Krea2HostAttention(None, 0, 28)(
torch.zeros(1, 3, 4), transformer_options={TRANSFORMER_MASK_KEY: "invalid"}
)
def test_host_attention_delegates_unsigned_calls() -> None:
"""Unmarked conditioning retains the host's ordinary attention operation."""
torch.manual_seed(4)
attention = Attention(128, 4, kvheads=2, operations=torch.nn)
with torch.no_grad():
attention.qknorm.qnorm.scale.zero_()
attention.qknorm.knorm.scale.zero_()
x = torch.randn(1, 7, 128)
expected = attention(x)
actual = Krea2HostAttention(attention, 0, 28)(x)
assert torch.equal(actual, expected)
class _EarlierModel(torch.nn.Module):
"""Expose the earlier host boundary around one real Comfy attention module."""
def __init__(self) -> None:
"""Retain a genuine patchable module hierarchy without loading weights."""
super().__init__()
block = torch.nn.Module()
block.attn = Attention(128, 4, kvheads=2, operations=torch.nn)
self.blocks = torch.nn.ModuleList([block])
def _forward(
self,
x: object,
timesteps: object,
context: object,
attention_mask: object = None,
transformer_options: object = None,
**kwargs: object,
) -> object:
"""Represent only the external host argument contract under test."""
del timesteps, context, attention_mask, transformer_options, kwargs
return x
def test_earlier_host_object_patches_restore_on_unload() -> None:
"""Patch only a derived MODEL and restore the shared host method on unload."""
root = torch.nn.Module()
root.diffusion_model = _EarlierModel()
device = torch.device("cpu")
source = ModelPatcher(root, load_device=device, offload_device=device)
attention = root.diffusion_model.blocks[0].attn
assert isinstance(attention, torch.nn.Module)
original = attention.forward
derived = PATCHER_LIFECYCLE.derive_model(
source, krea2_host_mutations(source), operation="test Krea host adaptation"
)
assert source.object_patches == {}
assert attention.forward == original
derived.patch_model(load_weights=False)
try:
assert isinstance(attention.forward, Krea2HostAttention)
finally:
derived.unpatch_model(unpatch_weights=False)
assert attention.forward == original
def test_reference_capable_host_needs_no_object_replacements() -> None:
"""Leave native Krea attention and its reference-image path entirely untouched."""
root = torch.nn.Module()
diffusion = object.__new__(SingleStreamDiT)
torch.nn.Module.__init__(diffusion)
root.diffusion_model = diffusion
device = torch.device("cpu")
model = ModelPatcher(root, load_device=device, offload_device=device)
assert krea2_host_mutations(model) == ()
@@ -11,6 +11,7 @@ from typing import Any, cast
import pytest
import torch
from comfy.ldm.krea2.model import SingleStreamDiT
from comfy.model_base import SDXL, Anima, BaseModel, Krea2, SDXLRefiner
from comfy.model_patcher import ModelPatcher
from comfy.patcher_extension import WrappersMP
@@ -161,6 +162,10 @@ def _model_patcher(model_class: type[BaseModel]) -> ModelPatcher:
model = object.__new__(model_class)
torch.nn.Module.__init__(model)
if model_class is Krea2:
diffusion = object.__new__(SingleStreamDiT)
torch.nn.Module.__init__(diffusion)
model.diffusion_model = diffusion
device = torch.device("cpu")
return ModelPatcher(model, load_device=device, offload_device=device)

Some files were not shown because too many files have changed in this diff Show More