Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d188a3764b | ||
|
|
f1d0630729 | ||
|
|
0cd1032073 | ||
|
|
d01b085082 | ||
|
|
0583ba2675 | ||
|
|
dcc37d7ab6 | ||
|
|
583b22a4bf | ||
|
|
ebe01efc49 | ||
|
|
41e8a2b61c | ||
|
|
22e4a5d202 | ||
|
|
017e3fc7fe | ||
|
|
c298efd8e8 | ||
|
|
7b92a4dafc | ||
|
|
ba13ef3345 | ||
|
|
dfa00d04b6 | ||
|
|
f4e33ff1c7 | ||
|
|
35ffc8b0e3 | ||
|
|
9f4dc7a193 | ||
|
|
1e0f44236e | ||
|
|
af6913e7bf | ||
|
|
b73d3c570f | ||
|
|
c35191de31 | ||
|
|
e8cbc6f724 | ||
|
|
4c13682df3 | ||
|
|
ffee87974e | ||
|
|
4f0d683c56 | ||
|
|
837a675a13 | ||
|
|
02710d2544 | ||
|
|
b80d2bc420 | ||
|
|
d9b92d65b1 | ||
|
|
24878eef89 | ||
|
|
9de6033505 | ||
|
|
36227fa8ff | ||
|
|
4875afdb7e | ||
|
|
dda0a3d326 | ||
|
|
ace5fafda9 | ||
|
|
f031c28589 | ||
|
|
e9cd06e41c | ||
|
|
fc738179f3 | ||
|
|
2dc1331262 | ||
|
|
64ac708ba0 | ||
|
|
60da3c1a48 | ||
|
|
4f4c47ae0a | ||
|
|
8104e81a29 | ||
|
|
e1d051781a | ||
|
|
4a6b8eb15d | ||
|
|
e266880244 | ||
|
|
02784b1ebe | ||
|
|
44cc18b86c | ||
|
|
4a61946919 | ||
|
|
18b7a72ddf | ||
|
|
18097c8fae | ||
|
|
2e5e14f6c2 | ||
|
|
cab3cac8f7 | ||
|
|
be348aeeda | ||
|
|
13a70540fd | ||
|
|
dce4327f9e | ||
|
|
652ae51fc4 | ||
|
|
667073a673 | ||
|
|
823fe209d8 | ||
|
|
6a6c3f8362 | ||
|
|
1972e452dc | ||
|
|
4241d7e97c | ||
|
|
29ee772b1d | ||
|
|
b58539097a | ||
|
|
24492d97a3 | ||
|
|
36081d7771 | ||
|
|
7156dff28f | ||
|
|
546ee5db01 | ||
|
|
d9e20fc601 | ||
|
|
250ee42ea1 | ||
|
|
74f140b2f3 | ||
|
|
9b9f7018fa | ||
|
|
741b5661c6 | ||
|
|
493eeb94b9 | ||
|
|
abd63b6296 | ||
|
|
4e3fd80bb8 | ||
|
|
6470b439b5 | ||
|
|
28bb8f7e88 | ||
|
|
92c1992379 | ||
|
|
acd668f7c6 | ||
|
|
4c7f087ce5 | ||
|
|
c0476b8288 | ||
|
|
b14f997145 | ||
|
|
346ff8b7c4 | ||
|
|
92c53b3493 | ||
|
|
f863fcf645 | ||
|
|
0679b7600b | ||
|
|
4202ca6d46 | ||
|
|
cc2561b557 | ||
|
|
57c4b8c5a4 | ||
|
|
dd514c1aad | ||
|
|
4b6525ce6f |
@@ -52,6 +52,7 @@ jobs:
|
||||
- name: Verify GroundingDINO BERT compatibility
|
||||
run: >-
|
||||
python -m pytest -q
|
||||
--noconftest
|
||||
--rootdir=tests
|
||||
--confcutdir=tests
|
||||
tests/test_grounding_dino_bert_adapter.py
|
||||
@@ -104,7 +105,7 @@ jobs:
|
||||
run: mypy --strict simple_syrup tests
|
||||
|
||||
- name: Verify Python tests
|
||||
run: pytest -n auto -q
|
||||
run: pytest -n auto -q -m "not external_artifact"
|
||||
|
||||
- name: Verify frontend
|
||||
run: npm run check:web
|
||||
|
||||
@@ -260,6 +260,7 @@ npm run build:web
|
||||
## ComfyUI Frontend Rules
|
||||
|
||||
- ComfyUI frontend extensions must be registered from TypeScript source under `web/src`.
|
||||
- If an element belongs to a component, integrate it into that component's structure, layout, input handling, and lifecycle. Do not fake ownership by positioning an unrelated element over the component or synchronizing it through external coordinates. Use detached overlays only for UI that is semantically an overlay, such as menus, tooltips, dialogs, and drag ghosts.
|
||||
- Settings-panel behavior must use Comfy's frontend settings API.
|
||||
- Browser settings are not authoritative for backend behavior.
|
||||
- Any frontend setting that affects backend node declarations or execution must be mirrored through an explicit backend route or persisted backend settings file.
|
||||
|
||||
+60
-36
@@ -1,62 +1,86 @@
|
||||
# [1.5.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.4.0...v1.5.0) (2026-07-14)
|
||||
## [1.9.1](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.9.0...v1.9.1) (2026-09-20)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **groundingdino:** support transformers v4 and v5 ([239070b](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/239070b12d3eecbfd5c47c9410c7ca31ac1402ac))
|
||||
* **contextual-diffusion:** project reference latents into views ([4cd780a](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/4cd780a2451aa472ce826834e4426b65693c46e8))
|
||||
|
||||
# [1.9.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.8.0...v1.9.0) (2026-09-19)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **masking:** expand segmentation tooling and progress ([f70766d](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/f70766dafe096895ad8d6309681fd59270664600))
|
||||
* **prompts:** add automatic NegPiP support ([6d052e9](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/6d052e9800985696972bf431fa7dae4972a56313))
|
||||
|
||||
# [1.4.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.3.0...v1.4.0) (2026-06-02)
|
||||
# [1.8.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.7.1...v1.8.0) (2026-09-19)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **tiled-diffusion:** clamp overlap for small latents ([d7448c6](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/d7448c6ca52ce517b8d0f8ee697249c5def13535))
|
||||
* **downloads:** keep unknown sizes indeterminate ([a31467c](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/a31467cd3a4299e9e3929d44281018dba0322b3e))
|
||||
* **models:** hide installed catalog choices ([d887e87](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/d887e87e03eb6d0d9d5325fe43b67fbd9e47cf71))
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **detailing:** add external llm segs tagging ([b7cd40c](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/b7cd40c85ae9f9514d0b1ff3c3f6007730796752))
|
||||
* **segs:** add regional batching and wd14 tagging nodes ([fef60ad](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/fef60adeca2c8ec7c0641e8106bb0863ee2f195e))
|
||||
* **models:** add curated ultralytics downloads ([b907fa2](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/b907fa2a17afa30170bc22cf1134a750241e55c2))
|
||||
* **models:** prioritize installed ultralytics choices ([d30e04f](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/d30e04f229366d3e1d2388bd4d713b2b47a12a1f))
|
||||
|
||||
# [1.3.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.2.0...v1.3.0) (2026-05-26)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **prompt-control:** add schedule and encode prompt node ([bd515e6](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/bd515e696cedcc78d056af6c23b9193e34f131bc))
|
||||
|
||||
# [1.2.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.1.0...v1.2.0) (2026-05-25)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **nodes:** add VAE options and clone-safe diffusion ([6ff2dc8](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/6ff2dc8c24a6f7ddde3182b81bcbe6aad65427f4))
|
||||
|
||||
# [1.1.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.0.0...v1.1.0) (2026-05-23)
|
||||
## [1.7.1](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.7.0...v1.7.1) (2026-09-11)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **detailers:** align SEGS mask blending behavior ([74c83a6](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/74c83a6a70407aea6a34b0c0e31408f32b929882))
|
||||
* use SimpleSyrup package identity ([4fa5582](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/4fa5582796e9ace2a8805cb1ea2aebce65551e89))
|
||||
* **regional:** preserve shared model patch ancestry ([6059a3f](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/6059a3f913a9502671666e83faeb8686a7a8da27))
|
||||
|
||||
# [1.7.0](https://github.com/Artificial-Sweetener/SimpleSyrup/compare/v1.6.0...v1.7.0) (2026-09-05)
|
||||
|
||||
|
||||
### Bug Fixes
|
||||
|
||||
* **anima:** support regional prompting across Comfy versions ([41a234a](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/41a234a85a4cfcdc4cfce68b80b4b9982719aab4))
|
||||
* **attention:** preserve anchored concept geometry ([1136efd](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/1136efd2ad8708b14320d04eab2f6489efbec282))
|
||||
* **cache:** make integer narrowing checker-independent ([2547767](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/254776791766c41c75a69ceb5b207c54949446c9))
|
||||
* **detailers:** align SEGS mask blending behavior ([a3120ae](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/a3120aebe8834982706d93784a39ab501c6ff40a))
|
||||
* **groundingdino:** support transformers v4 and v5 ([3387c03](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/3387c03eb59f244d56f6a0dfe86cedab1847a8e2))
|
||||
* **mask:** preserve missing-alpha image geometry ([caf7d37](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/caf7d37a154ddc62405807b56550efdaa831d09e))
|
||||
* **media:** stabilize native ordered preview controls ([1235652](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/123565225a1104234a7c9f5c43af0772c7508db8))
|
||||
* **regional:** align prompt batches and LoRA hooks ([656d197](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/656d1970e12be18304b057b07204dcbbe367432b))
|
||||
* **runtime:** centralize Comfy patcher lifecycle ([0e5f513](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/0e5f513ae0f33f40c6a8bd09161043a5af598392))
|
||||
* **sampling:** normalize model-specific latent layouts ([b2084a7](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/b2084a7a9bf04709af51380bc1ef4a09ccb9babc))
|
||||
* **tiled-diffusion:** clamp overlap for small latents ([fecba36](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/fecba36e4e11f0da681c6a5d9d42e18093d741fc))
|
||||
* **tools:** return host-native checkpoint selections ([c7cb8d2](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/c7cb8d29f83ebd44b48038b7ce5e65a0a6f445b4))
|
||||
* use SimpleSyrup package identity ([1659d13](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/1659d131f4215fa0baccc4c70024d63590460e67))
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* **detection:** add keep-only SEGS selection ([8f8ee91](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/8f8ee91dea3ce1a044c0e61b482e571c51b372bc))
|
||||
|
||||
# 1.0.0 (2026-05-22)
|
||||
|
||||
|
||||
### Features
|
||||
|
||||
* initial release ([e513baf](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/e513baf70a20306856e40fbf2afd80b25f5655a6))
|
||||
|
||||
# Changelog
|
||||
|
||||
All notable changes to this project will be documented in this file.
|
||||
* **anima:** add cached quantization profiles ([c20664f](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/c20664f8925305465ccb4c028d0a78a6364a037d))
|
||||
* **attention:** add sampler-derived concept regions ([8ca5d2c](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/8ca5d2c325e1caee822883ba568b25f721e51343))
|
||||
* **attention:** default regional prompts to full weight ([f829321](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/f82932193951bae75d59e1c7c7dba2d187e3c175))
|
||||
* **attention:** improve concept isolation fidelity and speed ([01826ad](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/01826ad1b7c64b11c8af031691403ecf521cb1b5))
|
||||
* **attention:** refine attention-derived region masks ([dff84cc](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/dff84cce2322e64b8a9ada91b84550faf3a5c7a1))
|
||||
* **conditioning:** add regional prompting and SEP-local LoRAs ([d3de028](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/d3de02816b24bea129e77b82568ad47a0fd0ba99))
|
||||
* **conditioning:** support labeled prompt separators ([a708e0b](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/a708e0b4187b0d5f9aeb58f5ab0d276b8a046395))
|
||||
* **detailing:** add external llm segs tagging ([207c449](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/207c4492908371f41aca84c74332c1a1f63d8045))
|
||||
* **detection:** add keep-only SEGS selection ([4f65ae0](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/4f65ae0dd36e43347b40ae64039f40e8a67aea48))
|
||||
* initial release ([4b6525c](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/4b6525ce6ff42f06a7ffd48a54186fcb625f0e21))
|
||||
* **loaders:** add automatic FLUX model loaders ([33bff75](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/33bff75af1b001b716a45f4fadc9e0e0aa20ced1))
|
||||
* **masking:** expand segmentation tooling and progress ([adba198](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/adba1981a4bdb43307496a84cccbfe100ae2174f))
|
||||
* **media:** add native ordered loaders and SEGS preview ([fcf38f2](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/fcf38f2010cfadc76be74410857387c74db3b575))
|
||||
* **nodes:** add VAE options and clone-safe diffusion ([5fc0f3d](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/5fc0f3d8e5ef5fbac0306b1d3c70464a035c396c))
|
||||
* **prompt-control:** add schedule and encode prompt node ([af32cec](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/af32cec99f5bdadcbed9f8c33db0d816ad4b72f0))
|
||||
* **regional:** add native SDXL adapter execution ([068e3db](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/068e3db17d13c875a383c85e6cf931f96459c3b1))
|
||||
* **regional:** add universal attention coupling foundation ([edfc26c](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/edfc26c9122ff0c18d63e9b80832d587594e9e62))
|
||||
* **regional:** build universal adapter execution foundation ([864852d](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/864852dc9e591a3041e23b342346495a7fbf6589))
|
||||
* **regional:** complete capability-routed execution ([fe96a20](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/fe96a206dbf1edcae022b1346f998ed184142062))
|
||||
* **regional:** complete persistent regional LoRA execution ([7b5987d](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/7b5987d6fa66f8deee2655d18b1209bbe20271d6))
|
||||
* **sampling:** add contextual diffusion sampler ([24bf630](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/24bf6309023c552f65947814d9641555a73c8337))
|
||||
* **sampling:** add deterministic seed variation ([7921be3](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/7921be3f9067a26fe5fa47fb8fb370e7cb1679f3))
|
||||
* **sampling:** add regional diffusion sampling ([f7dffca](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/f7dffcacfe104be173793ce412f994519e11e02e))
|
||||
* **sampling:** bypass inactive attention coupling ([b2b8dc4](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/b2b8dc4b016307fd351892f50264c64532898681))
|
||||
* **sampling:** expose evaluated context SEGS ([04a2c3e](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/04a2c3e6e90f9bbb9e3922412844afe5a4e6869f))
|
||||
* **segmentation:** add interactive SEGS preview ([c9e303e](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/c9e303ec4727576af124d9e8ea20d0103f67e7ae))
|
||||
* **segmentation:** add SAM region overlay ([a72c796](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/a72c796d8d6d64a52fed7d281fe53eebc74639ed))
|
||||
* **segmentation:** add SAM-guided tiled diffusion ([968090d](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/968090de87a4fad726fd37d0a08966c01f11a8fd))
|
||||
* **segs:** add regional batching and wd14 tagging nodes ([600d9e3](https://github.com/Artificial-Sweetener/SimpleSyrup/commit/600d9e311b5b8c45762e15c88f54df33454261b5))
|
||||
|
||||
@@ -1,543 +0,0 @@
|
||||
# Plan: Image-Associated Mask to SEGS
|
||||
|
||||
## Goal
|
||||
|
||||
Add a SimpleSyrup node that converts an existing ComfyUI mask into Impact-compatible SEGS while associating every SEG with cropped image data from the source image.
|
||||
|
||||
The node should feel like `Detect SEGS w/ Ultralytics`, but it must take an `IMAGE` and a `MASK` instead of an image and detector model. It should expose the same general region workflow controls where they make sense: size filtering, keep-only limiting, mask dilation, post-dilation, crop factor, sort order, and optional unioning into one SEG.
|
||||
|
||||
## Decisions From Maintainer Discussion
|
||||
|
||||
- This should be a first-class SimpleSyrup node, not just documentation for an Impact Pack workflow chain.
|
||||
- The node should behave like a detector-style SEGS source because downstream users think of this as "pre-detected regions from a mask."
|
||||
- The node must take both `image` and `mask`.
|
||||
- The output SEGS must include full image association by storing each SEG's `cropped_image` from the input image.
|
||||
- The node should support masks with multiple disconnected regions.
|
||||
- The node should let users choose whether disconnected regions become separate SEGs or one combined SEG.
|
||||
- The node should include controls similar to `Detect SEGS w/ Ultralytics` and in a similar order.
|
||||
- The node should not expose detector confidence controls because the input is an existing mask, not detector predictions.
|
||||
- Segment confidence should be fixed at `1.0`.
|
||||
- The existing Impact Pack nodes can already do some of this with `MASK to SEGS` plus `Set Default Image for SEGS`, but that is not the desired UX for SimpleSyrup.
|
||||
|
||||
## Existing Code To Reuse
|
||||
|
||||
Use these SimpleSyrup modules as the main implementation references:
|
||||
|
||||
- `simple_syrup/nodes/detect_segs_with_ultralytics.py`
|
||||
- Existing detector-style node UX and control order.
|
||||
- Current output shape: `RETURN_TYPES = ("SEGS", "MASK")`.
|
||||
- Current list behavior for batched images: `OUTPUT_IS_LIST = (True, False)`.
|
||||
- Applies `limit_segs`, `sort_segs`, and then `build_combined_segs_result`.
|
||||
- `simple_syrup/services/segs_detection_service.py`
|
||||
- Existing service pattern for constructing `Segment` objects from masks and image crops.
|
||||
- Uses `crop_region_for_bbox`, `crop_mask`, `crop_image`, and `dilate_mask`.
|
||||
- `simple_syrup/services/segs_output_service.py`
|
||||
- `build_combined_segs_result()` already creates a one-SEG union with `cropped_image`.
|
||||
- `combined_mask_from_segs()` already produces the mask output from SEGS.
|
||||
- `coerce_cropped_mask()` validates crop-local SEG masks.
|
||||
- This module should become the shared owner for detector-style SEGS output finalization.
|
||||
- `simple_syrup/masking/segs_mask_ops.py`
|
||||
- Reuse image validation, mask crop, image crop, resize, crop factor, and signed dilation behavior.
|
||||
- `simple_syrup/domain/segs.py`
|
||||
- Reuse `Segment`, `BoundingBox`, `CropRegion`, `NativeSegs`, `SORT_ORDER_OPTIONS`, `limit_segs`, `sort_segs`, and `to_impact_compatible_segs`.
|
||||
|
||||
Use these Impact Pack modules only as behavioral references, not as imports:
|
||||
|
||||
- `E:\ComfyUI\custom_nodes\comfyui-impact-pack\modules\impact\segs_nodes.py`
|
||||
- `MaskToSEGS` splits or combines mask regions.
|
||||
- `DefaultImageForSEGS` attaches cropped source image data after mask conversion.
|
||||
- `SEGSMerge`, `SEGSOrderedFilter`, and `DilateMaskInSEGS` show existing user expectations.
|
||||
- `E:\ComfyUI\custom_nodes\comfyui-impact-pack\modules\impact\core.py`
|
||||
- `mask_to_segs()` uses contour detection for disconnected regions.
|
||||
- `batch_mask_to_segs()` handles batched masks for video-style masks.
|
||||
|
||||
Do not import Impact Pack code. SimpleSyrup tests already enforce avoiding external pack imports.
|
||||
|
||||
## Node Name And Contract
|
||||
|
||||
Add a new Comfy v3 node:
|
||||
|
||||
- Node id: `SimpleSyrup.MaskToSEGS`
|
||||
- Display name: `Mask to SEGS`
|
||||
- Category: `SimpleSyrup/Detection`
|
||||
- Search aliases: `mask`, `segs`, `region`, `detect`, `segmentation`
|
||||
- Description: `Converts an existing mask into image-associated SEGS for detail and regional workflows.`
|
||||
|
||||
Outputs:
|
||||
|
||||
- `segs: SEGS`
|
||||
- `mask: MASK`
|
||||
|
||||
Output behavior should match `Detect SEGS w/ Ultralytics`:
|
||||
|
||||
- `segs` is list-output compatible, one SEGS payload per input image/mask pair.
|
||||
- `mask` is a standard ComfyUI batched mask tensor with shape `(B, H, W)`.
|
||||
- The output mask is the union of the retained SEGS after filtering, sorting, and optional combining.
|
||||
|
||||
## Architecture Landing Shape
|
||||
|
||||
Do not add a third copy of the detector-style output pipeline.
|
||||
|
||||
The current code has two nodes that inline the same sequence after SEGS extraction:
|
||||
|
||||
1. `simple_syrup/nodes/detect_segs_with_ultralytics.py`
|
||||
2. `simple_syrup/nodes/prompt_segs_with_sam.py`
|
||||
|
||||
Both nodes currently perform:
|
||||
|
||||
```python
|
||||
segs = limit_segs(segs, keep_only, keep_by)
|
||||
segs = sort_segs(segs, sort_order)
|
||||
combined = build_combined_segs_result(single_image, segs, crop_factor)
|
||||
output_segs = combined.segs if combine_segs else segs
|
||||
segs_outputs.append(to_impact_compatible_segs(output_segs))
|
||||
mask_outputs.append(combined.mask)
|
||||
```
|
||||
|
||||
The new mask node must not duplicate this sequence inline. Instead, implement a shared output-finalization helper in `simple_syrup/services/segs_output_service.py` and refactor the existing detector nodes to use it.
|
||||
|
||||
Recommended shape:
|
||||
|
||||
- `MaskToSEGSService`
|
||||
- Owns only one-image, one-mask extraction into separate native SEGS.
|
||||
- Does not own `keep_only`, `sort_order`, `combine_segs`, Impact-compatible conversion, or output mask construction.
|
||||
- `segs_output_service`
|
||||
- Owns the common detector-style post-processing pipeline:
|
||||
1. limit SEGS
|
||||
2. sort SEGS
|
||||
3. build the combined result
|
||||
4. choose separate or combined output SEGS
|
||||
5. convert output SEGS to Impact-compatible shape
|
||||
6. return the paired mask output
|
||||
- Node classes
|
||||
- Own Comfy-facing schema, batch iteration, and wiring only.
|
||||
- Delegate source-specific extraction to a source service.
|
||||
- Delegate shared final output shaping to `segs_output_service`.
|
||||
|
||||
This keeps ownership strict:
|
||||
|
||||
- Mask-derived region extraction belongs to the mask-to-SEGS service.
|
||||
- Detector model inference belongs to detector services.
|
||||
- Prompt/SAM detection belongs to the SAM prompt service.
|
||||
- Sorting, limiting, combining, and output mask generation belong to the shared SEGS output service.
|
||||
- Domain helpers remain pure SEGS policies and value conversions.
|
||||
|
||||
## Input Schema
|
||||
|
||||
Use this input order:
|
||||
|
||||
1. `image: IMAGE`
|
||||
2. `mask: MASK`
|
||||
3. `mask_threshold: FLOAT`
|
||||
4. `size_threshold: INT`
|
||||
5. `keep_only: INT`
|
||||
6. `mask_dilation: INT`
|
||||
7. `post_dilation: INT`
|
||||
8. `crop_factor: FLOAT`
|
||||
9. `sort_order: SORT_ORDER_OPTIONS`
|
||||
10. `combine_segs: BOOLEAN`
|
||||
11. `label: STRING`
|
||||
|
||||
Detailed input behavior:
|
||||
|
||||
- `image`
|
||||
- Source image used to associate cropped image data with every SEG.
|
||||
- Must be a ComfyUI `IMAGE` tensor shaped `(B, H, W, C)`.
|
||||
- `mask`
|
||||
- Source mask to convert into SEGS.
|
||||
- Must be a ComfyUI `MASK` tensor shaped `(B, H, W)` or a single mask compatible with the image batch.
|
||||
- The implementation may support a single mask for a single image first. If supporting image batches, batch count must either match the image batch or be exactly `1` for reuse across all images.
|
||||
- `mask_threshold`
|
||||
- Default: `0.5`
|
||||
- Min: `0.0`
|
||||
- Max: `1.0`
|
||||
- Step: `0.01`
|
||||
- Converts soft masks into active pixels before region extraction.
|
||||
- Active pixels are `mask >= mask_threshold`.
|
||||
- `size_threshold`
|
||||
- Default: `10`
|
||||
- Min: `1`
|
||||
- Max: `8192`
|
||||
- Drops extracted regions whose bounding box is smaller than this many pixels wide or tall.
|
||||
- Match the wording and role of `Detect SEGS w/ Ultralytics`'s `size_threshold`.
|
||||
- `keep_only`
|
||||
- Default: `0`
|
||||
- Min: `0`
|
||||
- Max: `4096`
|
||||
- Keeps only the largest N regions after size filtering. `0` keeps all regions.
|
||||
- Do not expose a confidence ranking option. There is no detector confidence.
|
||||
- `mask_dilation`
|
||||
- Default: `0`
|
||||
- Min: `-512`
|
||||
- Max: `512`
|
||||
- Signed dilation applied to the full input mask before region extraction.
|
||||
- Positive grows mask regions; negative erodes them.
|
||||
- `post_dilation`
|
||||
- Default: `0`
|
||||
- Min: `-512`
|
||||
- Max: `512`
|
||||
- Signed dilation applied to each final crop-local SEG mask after the crop is chosen.
|
||||
- This mirrors the Ultralytics node's final SEG mask cleanup behavior.
|
||||
- `crop_factor`
|
||||
- Default: `3.0`
|
||||
- Min: `0.0`
|
||||
- Max: `100.0`
|
||||
- Step: `0.1`
|
||||
- `0.0` means use the full image as the crop, matching SimpleSyrup's existing detector convention.
|
||||
- Values greater than or equal to `1.0` expand the SEG crop around the extracted region's bounding box.
|
||||
- Values between `0.0` and `1.0` must fail with an actionable `ValueError`.
|
||||
- `sort_order`
|
||||
- Use `SORT_ORDER_OPTIONS` from `simple_syrup.domain.segs`.
|
||||
- Default: `largest to smallest`.
|
||||
- Sort extracted regions before output and before any combined mask result is built.
|
||||
- `combine_segs`
|
||||
- Default: `False`
|
||||
- `False` returns one SEG per disconnected mask region.
|
||||
- `True` returns one unioned SEG representing all retained mask pixels.
|
||||
- `label`
|
||||
- Default: `mask`
|
||||
- Label applied to extracted SEGs.
|
||||
- If `combine_segs` is true, the combined output label should be `combined` unless there is a strong reason to preserve the user label. Match `build_combined_segs_result()` unless intentionally extending it.
|
||||
|
||||
## Behavior Details
|
||||
|
||||
### Region Extraction
|
||||
|
||||
Add a new service, probably `simple_syrup/services/mask_to_segs_service.py`, with a class such as `MaskToSEGSService`.
|
||||
|
||||
The service should own:
|
||||
|
||||
- Image and mask validation.
|
||||
- Mask thresholding.
|
||||
- Signed full-mask dilation.
|
||||
- Disconnected region extraction.
|
||||
- Per-region bounding box calculation.
|
||||
- Size filtering.
|
||||
- Crop region calculation.
|
||||
- Crop-local mask creation.
|
||||
- Optional post-dilation on each crop-local mask.
|
||||
- Cropped image association.
|
||||
- Native immutable SEGS output.
|
||||
|
||||
The service must not own:
|
||||
|
||||
- `keep_only`
|
||||
- final `sort_order`
|
||||
- `combine_segs`
|
||||
- Impact-compatible conversion
|
||||
- final output mask construction
|
||||
|
||||
Those responsibilities belong to the shared SEGS output finalization helper.
|
||||
|
||||
The node should own only Comfy-facing schema, input ordering, batch iteration, and wiring.
|
||||
|
||||
### Connected Components
|
||||
|
||||
Implement disconnected region extraction inside SimpleSyrup. Do not import Impact Pack.
|
||||
|
||||
Prefer a torch-based or standard-library implementation over adding a new required runtime dependency. A simple deterministic flood-fill or connected-components routine is acceptable because masks are 2D binary tensors and the node is not model-bound.
|
||||
|
||||
Connectivity decision:
|
||||
|
||||
- Use 8-connected components unless tests or existing SimpleSyrup behavior strongly point to 4-connected components.
|
||||
- Document this in the service docstring and tests.
|
||||
- 8-connected behavior usually matches user expectations for painted masks where diagonal contact should remain one region.
|
||||
|
||||
Extraction algorithm outline:
|
||||
|
||||
1. Convert mask to a CPU `torch.bool` active-pixel tensor after threshold and dilation.
|
||||
2. If no pixels are active, return empty native SEGS. The shared output finalization helper is responsible for turning that into a zero mask output.
|
||||
3. Find connected components.
|
||||
4. For each component, compute bbox from active pixel coordinates.
|
||||
5. Drop components where bbox width or height is less than `size_threshold`.
|
||||
6. Create one `Segment` per kept component.
|
||||
|
||||
The service always returns separate native SEGS. `combine_segs` is handled later by the shared output finalization helper so all detector-style SEGS nodes use the same combine behavior.
|
||||
|
||||
### Mask Values
|
||||
|
||||
Use thresholded masks to decide region membership, but preserve useful soft-mask values when forming crop-local masks where possible.
|
||||
|
||||
Recommended behavior:
|
||||
|
||||
- Use the thresholded, dilated mask for topology and bbox extraction.
|
||||
- Use the original normalized mask after `mask_dilation` as the crop-local mask values.
|
||||
- Zero out pixels outside the active component for each separate SEG.
|
||||
- Clamp all final masks to `0.0..1.0`.
|
||||
|
||||
This lets soft masks retain feathered values inside each SEG while still giving deterministic region extraction.
|
||||
|
||||
### Cropped Image Association
|
||||
|
||||
Every returned `Segment` must set:
|
||||
|
||||
- `cropped_image = crop_image(single_image, crop_region).detach().clone()`
|
||||
- `cropped_mask = crop-local mask tensor`
|
||||
- `confidence = 1.0`
|
||||
- `crop_region = CropRegion(...)`
|
||||
- `bbox = BoundingBox(...)`
|
||||
- `label = label`
|
||||
- `control_net_wrapper = None`
|
||||
|
||||
This is the key difference from Impact Pack's `MASK to SEGS`, which creates SEGS without cropped image data and relies on a separate `Set Default Image for SEGS` node.
|
||||
|
||||
### Batch Handling
|
||||
|
||||
Match existing SimpleSyrup detector behavior as closely as possible:
|
||||
|
||||
- Validate the image batch with `validate_image_batch()`.
|
||||
- Iterate image items with `iter_single_images()`.
|
||||
- Produce one SEGS payload per image.
|
||||
- Concatenate mask outputs into one `(B, H, W)` tensor.
|
||||
|
||||
Mask batch rules:
|
||||
|
||||
- If mask batch size equals image batch size, pair by index.
|
||||
- If mask batch size is `1` and image batch size is greater than `1`, reuse the mask for every image.
|
||||
- Otherwise raise `ValueError` explaining the mismatch.
|
||||
- Mask height and width must match the image height and width. Do not silently resize masks for this node unless the maintainer explicitly approves it later.
|
||||
|
||||
## Sorting And Keeping
|
||||
|
||||
Do not expose `keep_by` unless there is a future product decision to add multiple non-confidence policies.
|
||||
|
||||
For `keep_only`, retain the largest regions by crop area before final sorting. Reuse `limit_segs(segs, keep_only, "largest size")` inside the shared output finalization helper for this node. Do not add a new ranking abstraction unless tests show the current domain helper cannot express the behavior clearly.
|
||||
|
||||
Final output order must come from `sort_segs(segs, sort_order)`.
|
||||
|
||||
## Shared Output Finalization
|
||||
|
||||
Add a small result type and helper to `simple_syrup/services/segs_output_service.py`.
|
||||
|
||||
Suggested result type:
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True)
|
||||
class FinalizedSegsOutput:
|
||||
"""Return Impact-compatible SEGS and its paired output mask."""
|
||||
|
||||
segs: object
|
||||
mask: torch.Tensor
|
||||
```
|
||||
|
||||
Suggested helper:
|
||||
|
||||
```python
|
||||
def finalize_detector_segs_output(
|
||||
image: object,
|
||||
segs: NativeSegs,
|
||||
keep_only: int,
|
||||
keep_by: str,
|
||||
crop_factor: float,
|
||||
sort_order: str,
|
||||
combine_segs: bool,
|
||||
) -> FinalizedSegsOutput:
|
||||
"""Apply shared detector-style SEGS output policy."""
|
||||
```
|
||||
|
||||
Behavior:
|
||||
|
||||
1. Apply `limit_segs(segs, keep_only, keep_by)`.
|
||||
2. Apply `sort_segs(segs, sort_order)`.
|
||||
3. Build `combined = build_combined_segs_result(image, segs, crop_factor)`.
|
||||
4. Use `combined.segs` when `combine_segs` is true; otherwise use the sorted separate SEGS.
|
||||
5. Convert chosen SEGS with `to_impact_compatible_segs`.
|
||||
6. Return the converted SEGS and `combined.mask`.
|
||||
|
||||
For the new mask node, call this helper with `keep_by="largest size"` internally.
|
||||
|
||||
Refactor these existing nodes to use the helper as part of this change:
|
||||
|
||||
- `simple_syrup/nodes/detect_segs_with_ultralytics.py`
|
||||
- `simple_syrup/nodes/prompt_segs_with_sam.py`
|
||||
|
||||
Add characterization tests before refactoring or preserve existing tests that already prove:
|
||||
|
||||
- final sorting happens before combined output construction
|
||||
- keep-only limiting happens before final sorting
|
||||
- `combine_segs` chooses the combined SEGS while retaining the same output mask
|
||||
- returned SEGS are Impact-compatible
|
||||
|
||||
The helper should be narrow. Do not move source-specific detection behavior into it.
|
||||
|
||||
## V3 Registration
|
||||
|
||||
Comfy v3 is the only supported export path.
|
||||
|
||||
Preferred implementation:
|
||||
|
||||
- Add a direct v3 node class under `simple_syrup/nodes_v3/mask_to_segs.py`.
|
||||
- Register it in `simple_syrup/nodes_v3/__init__.py::get_nodes()`.
|
||||
- Do not add `NODE_CLASS_MAPPINGS` or legacy export mappings.
|
||||
|
||||
If reusing the legacy adapter pattern would materially reduce risk, it is acceptable to add a legacy-style internal node class under `simple_syrup/nodes/` and wrap it with `LegacyNodeV3Adapter`, but the public export must still be v3-only.
|
||||
|
||||
The implementation should look native to the current codebase and should not add compatibility shims.
|
||||
|
||||
## Tooltips
|
||||
|
||||
Every visible input and output must have concise user-facing tooltip text.
|
||||
|
||||
Required tooltip intent:
|
||||
|
||||
- `image`: source image used for SEG crops.
|
||||
- `mask`: mask whose active regions become SEGS.
|
||||
- `mask_threshold`: threshold used to decide active mask pixels.
|
||||
- `size_threshold`: smallest region width or height to keep, in pixels.
|
||||
- `keep_only`: maximum number of largest regions to keep; `0` keeps all.
|
||||
- `mask_dilation`: grow or shrink the source mask before regions are found.
|
||||
- `post_dilation`: grow or shrink each final SEG mask after cropping.
|
||||
- `crop_factor`: context around each region; `0` uses the full image.
|
||||
- `sort_order`: output ordering for separate SEGS.
|
||||
- `combine_segs`: return one unioned SEG instead of separate regions.
|
||||
- `label`: label stored on extracted SEGs.
|
||||
- `segs` output: image-associated SEGS from the mask.
|
||||
- `mask` output: union of retained SEGS as a ComfyUI mask.
|
||||
|
||||
## Tests To Add
|
||||
|
||||
Add focused tests before or alongside implementation.
|
||||
|
||||
### Shared Output Finalization Tests
|
||||
|
||||
Add tests for the new helper in `tests/test_segs_output_service.py` or extend the existing SEGS output service coverage currently housed in `tests/test_ultralytics_detection_service.py`.
|
||||
|
||||
Cover:
|
||||
|
||||
- Applies `limit_segs` before `sort_segs`.
|
||||
- Builds the combined result from the limited and sorted SEGS.
|
||||
- Returns separate SEGS when `combine_segs` is false.
|
||||
- Returns one combined SEG when `combine_segs` is true.
|
||||
- Always returns the mask from `build_combined_segs_result`.
|
||||
- Converts output SEGS to Impact-compatible tuple/list shape.
|
||||
- Supports `keep_by="largest size"` for mask-derived SEGS.
|
||||
|
||||
After this helper is tested, refactor `Detect SEGS w/ Ultralytics` and `Prompt SEGS w/ SAM` to use it without changing their public behavior.
|
||||
|
||||
### Service Tests
|
||||
|
||||
Create `tests/test_mask_to_segs_service.py`.
|
||||
|
||||
Cover:
|
||||
|
||||
- Single rectangular mask creates one SEG.
|
||||
- `cropped_image` matches the source image crop.
|
||||
- `cropped_mask` matches the mask crop.
|
||||
- Two disconnected regions create two separate SEGs.
|
||||
- Service always returns separate SEGS; combining is covered by shared output finalization tests and node tests.
|
||||
- Empty mask returns empty native SEGS from the service and a zero mask output through the node/output helper.
|
||||
- `mask_threshold` controls active pixels.
|
||||
- `mask_dilation` grows a source mask before region extraction.
|
||||
- Negative `mask_dilation` erodes a source mask.
|
||||
- `post_dilation` changes only crop-local final masks.
|
||||
- `crop_factor` expands crop regions.
|
||||
- `crop_factor = 0.0` uses the full image.
|
||||
- `0.0 < crop_factor < 1.0` raises `ValueError`.
|
||||
- `size_threshold` drops small components.
|
||||
- Soft mask values are clamped and retained inside component masks.
|
||||
- Batch mask and image mismatch raises an actionable error if batch handling is in the service.
|
||||
|
||||
### Node Contract Tests
|
||||
|
||||
Create `tests/test_mask_to_segs_node.py` or a v3-specific equivalent.
|
||||
|
||||
Cover:
|
||||
|
||||
- Node id is `SimpleSyrup.MaskToSEGS`.
|
||||
- Display name is `Mask to SEGS`.
|
||||
- Category is `SimpleSyrup/Detection`.
|
||||
- Outputs are `SEGS` and `MASK`.
|
||||
- The input list order exactly matches the plan.
|
||||
- Every input has a tooltip.
|
||||
- Every output has a tooltip.
|
||||
- No confidence input exists.
|
||||
- No detector model input exists.
|
||||
- No `keep_by` widget exists unless the implementation deliberately adds more non-confidence policies and updates this plan.
|
||||
- Execution returns Impact-compatible SEGS and a `(B, H, W)` mask.
|
||||
- `combine_segs = false` returns separate SEGs.
|
||||
- `combine_segs = true` returns one combined SEG.
|
||||
- `keep_only` keeps the largest mask-derived regions.
|
||||
- `sort_order` orders separate regions using existing domain policies.
|
||||
- Batched images return list-output SEGS and batched masks.
|
||||
|
||||
### Registration Tests
|
||||
|
||||
Update existing registration tests:
|
||||
|
||||
- `SimpleSyrup.MaskToSEGS` appears in `get_nodes()`.
|
||||
- Adding the node does not remove or rename existing nodes.
|
||||
- Root `comfy_entrypoint` still exposes v3 nodes only.
|
||||
|
||||
### Tooltip Coverage Tests
|
||||
|
||||
Update `tests/test_node_tooltips.py` so the new node passes the repository tooltip requirements.
|
||||
|
||||
## Implementation Steps
|
||||
|
||||
- [x] Add or confirm characterization tests for the existing Ultralytics and SAM detector-style output behavior.
|
||||
- Landing note: Existing node tests already covered separate/combined outputs, keep-only limiting, final sorting, combined-builder input order, and batch handling for both detector nodes.
|
||||
- [x] Add shared output finalization tests.
|
||||
- Landing note: `tests/test_segs_output_service.py` now covers limit-before-sort, combined output selection, returned mask preservation, Impact-compatible conversion, and `keep_by="largest size"` for mask-derived SEGS.
|
||||
- [x] Add the shared output finalization helper in `segs_output_service.py`.
|
||||
- Landing note: `FinalizedSegsOutput` and `finalize_detector_segs_output()` now own detector-style limit/sort/combine/conversion/mask finalization.
|
||||
- [x] Refactor `Detect SEGS w/ Ultralytics` and `Prompt SEGS w/ SAM` to use the helper without behavior changes.
|
||||
- Landing note: Both nodes still own source-specific extraction and batch wiring, but delegate shared output shaping to `finalize_detector_segs_output()` with their injectable combined builders.
|
||||
- [x] Add the connected-component helper and mask-to-SEGS service.
|
||||
- Landing note: `mask_components.py` implements deterministic 8-connected component extraction, and `MaskToSEGSService` now converts one image plus one mask into separate native SEGS with cropped image association.
|
||||
- [x] Add direct v3 node schema and execution wrapper.
|
||||
- Landing note: `MaskToSEGSV3` defines the Comfy v3 schema directly, owns image/mask batch pairing, and delegates extraction plus shared output finalization.
|
||||
- [x] Register the node in `simple_syrup/nodes_v3/__init__.py`.
|
||||
- Landing note: `SimpleSyrup.MaskToSEGS` is included in the base v3 node list.
|
||||
- [x] Add or update registration and tooltip tests.
|
||||
- Landing note: Registration expectations include `SimpleSyrup.MaskToSEGS`; the schema-driven tooltip coverage includes every new input and output.
|
||||
- [x] Run focused tests for shared output finalization, existing detector nodes, the new service, the new node, registration, and tooltips.
|
||||
- Landing note: Focused plan checks pass: `51 passed`.
|
||||
- [x] Run full Python gates.
|
||||
- Landing note: `ruff format .`, `ruff check .`, `mypy --strict simple_syrup tests`, and full `pytest -n auto -q` pass. Full test result: `969 passed`.
|
||||
- [x] Do not touch frontend code unless the Comfy v3 UI requires it.
|
||||
- Landing note: No frontend source or generated browser artifact was changed.
|
||||
|
||||
## Verification Commands
|
||||
|
||||
Run all commands from repository root with the ComfyUI virtual environment two directories above this repo.
|
||||
|
||||
Focused checks during development:
|
||||
|
||||
```powershell
|
||||
..\..\venv\Scripts\python.exe -m pytest -n auto -q tests\test_detect_segs_with_ultralytics_node.py tests\test_prompt_segs_with_sam_node.py
|
||||
..\..\venv\Scripts\python.exe -m pytest -n auto -q tests\test_segs_output_service.py
|
||||
..\..\venv\Scripts\python.exe -m pytest -n auto -q tests\test_mask_to_segs_service.py tests\test_mask_to_segs_node.py
|
||||
..\..\venv\Scripts\python.exe -m pytest -n auto -q tests\test_registration.py tests\test_node_tooltips.py
|
||||
```
|
||||
|
||||
Required final gates:
|
||||
|
||||
```powershell
|
||||
..\..\venv\Scripts\ruff.exe format .
|
||||
..\..\venv\Scripts\ruff.exe check .
|
||||
..\..\venv\Scripts\mypy.exe --strict simple_syrup tests
|
||||
..\..\venv\Scripts\python.exe -m pytest -n auto -q
|
||||
```
|
||||
|
||||
If frontend code is touched, also run:
|
||||
|
||||
```powershell
|
||||
npm ci
|
||||
npm run lint:web
|
||||
npm run typecheck:web
|
||||
npm run test:web
|
||||
npm run build:web
|
||||
```
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- A user can plug in an image and a mask and get ready-to-use image-associated SEGS.
|
||||
- A mask with two disconnected regions can produce either two SEGs or one combined SEG.
|
||||
- The node has detector-style controls that feel aligned with `Detect SEGS w/ Ultralytics`.
|
||||
- The node exposes no detector confidence controls.
|
||||
- Each SEG has `cropped_image`, `cropped_mask`, `crop_region`, `bbox`, `label`, and `confidence = 1.0`.
|
||||
- The returned SEGS are Impact-compatible.
|
||||
- The returned mask is the union of retained output SEGS.
|
||||
- The implementation does not import Impact Pack.
|
||||
- The implementation is covered by behavior tests, node contract tests, registration tests, and tooltip tests.
|
||||
- Full Python verification gates pass.
|
||||
@@ -1,192 +1,201 @@
|
||||
# SimpleSyrup
|
||||
|
||||
[](LICENSE) [](https://registry.comfy.org/publishers/artificialsweetener/nodes/SimpleSyrup) [](https://registry.comfy.org/publishers/artificialsweetener/nodes/SimpleSyrup) [](https://www.python.org/downloads/)
|
||||
[](https://registry.comfy.org/publishers/artificialsweetener/nodes/SimpleSyrup) [](https://registry.comfy.org/publishers/artificialsweetener/nodes/SimpleSyrup) [](https://www.python.org/downloads/) [](LICENSE)
|
||||
|
||||
**SimpleSyrup** is a ComfyUI node pack that grew out of moving my A1111/WebUI image workflows into ComfyUI graphs.
|
||||
**SimpleSyrup** is a ComfyUI node pack that grew out of moving my A1111/WebUI image workflows into ComfyUI.
|
||||
|
||||
The WebUI side shows up in the things I kept reaching for: ADetailer-style inline `[SEP]` prompt batches, tiled diffusion, familiar checkpoint loader controls, CLIP skip where WebUI users expect it, and sampler/scheduler extras. The Comfy side matters just as much: the detailers are modeled heavily on ComfyUI Impact Pack's SEGS workflow, and the utility nodes are built for graph readability, ordered data, and explicit runtime behavior.
|
||||
The WebUI influence shows up all over the pack. I missed ADetailer's inline prompt batches, tiled diffusion, CLIP skip beside my checkpoint, and some of the sampler behavior I was used to. I also wanted the regional pieces to work with Impact Pack SEGS so I could use the same regions across detectors and detailers.
|
||||
|
||||
SimpleSyrup pulls from a few different places:
|
||||
The pack now covers model loading, regional prompting and segmentation, high-resolution sampling, image and mask utilities, tagging, and the smaller pieces I need to keep those workflows readable.
|
||||
|
||||
- Moving from **A1111/WebUI** to **ComfyUI** is the reason this pack brings tiled diffusion, familiar checkpoint loader control grouping, ADetailer-style prompt splitting, and A1111-flavored sampler behavior into node graphs.
|
||||
- **ComfyUI Impact Pack** is the main influence for the SEGS and detailer shape: detectors create SEGS, detailers sample cropped areas, masks are feathered, and results are composited back into the source image.
|
||||
- **ADetailer** is the influence for inline `[SEP]` per-segment prompt batches.
|
||||
- The remaining utility pieces cover practical graph needs: GPU Lanczos resizing through TorchLanc, latent provenance helpers, and smaller nodes for image, prompt, and conditioning workflows.
|
||||
[**SugarSubstitute**](https://github.com/Artificial-Sweetener/SugarSubstitute), my native desktop front-end for ComfyUI, can use Cubes built from any ComfyUI nodes available in its connected environment. Its first-party [**Base-Cubes**](https://github.com/Artificial-Sweetener/Base-Cubes) pack uses SimpleSyrup for model loading, regional prompts, segmentation, high-resolution sampling, and other graph work. You can also install SimpleSyrup on its own and use the nodes in normal ComfyUI workflows.
|
||||
|
||||
## Highlights
|
||||
|
||||
- Impact-compatible SEGS detection, sorting, combining, tiling, and detailing.
|
||||
- Detailer nodes modeled heavily on ComfyUI Impact Pack's SEGS workflow.
|
||||
- ADetailer-style inline `[SEP]` per-segment prompt batches.
|
||||
- MultiDiffusion, Mixture of Diffusers, and regional MultiDiffusion sampling paths.
|
||||
- A WebUI-familiar checkpoint loader with CLIP skip and VAE override controls.
|
||||
- A Simple Anima loader that keeps Anima's model, VAE, dtype, and device controls together.
|
||||
- Loaders for SAM, GroundingDINO, ViTMatte, Ultralytics, and WD14.
|
||||
- Tile, tag, and guide workflows that can generate conditioning from WD14 or a configured external vision LLM.
|
||||
- KSampler extras including A1111-style Euler ancestral behavior, AYS, GITS, automatic A1111 scheduling, and beta57.
|
||||
- GPU Lanczos resizing through [TorchLanc](https://github.com/Artificial-Sweetener/TorchLanc), with batch and mask handling.
|
||||
- Normal and tiled VAE encode/decode option nodes with spatial and temporal tiling controls.
|
||||
- Expandable SEGS and conditioning batch helpers for larger regional graphs.
|
||||
- Provenance-aware latent helpers for recovering the latent behind an unmodified decoded image.
|
||||
- Settings-backed model dropdowns that can show known downloadable models or only locally installed ones.
|
||||
- Loaders that keep checkpoints, Anima, FLUX.1, and FLUX.2 models together with the text encoders, VAE, precision, and device choices they need.
|
||||
- My original Contextual Diffusion method for coherent high-resolution edits, plus MultiDiffusion and Mixture of Diffusers tiled sampling.
|
||||
- Impact-compatible SEGS detection, segmentation, interactive preview, batching, and detailers.
|
||||
- ADetailer-style `[SEP]` prompt batches, masked conditioning, and regional samplers, with optional Prompt Control scheduling and LoRA hooks.
|
||||
- WD14 and external vision LLM tagging that stays aligned with the right regions.
|
||||
- Ordered image and mask loading, GPU Lanczos resizing, tiled VAE options, and provenance-aware latent tools.
|
||||
- WebUI-inspired sampling extras including seed variation, A1111 Euler ancestral behavior, AYS, GITS, `automatic_a1111`, and beta57.
|
||||
|
||||
## Installation
|
||||
## Contents
|
||||
|
||||
**Recommended: install through ComfyUI Manager**
|
||||
- [Install](#install)
|
||||
- [Model loading](#model-loading)
|
||||
- [Large images and high-resolution edits](#large-images-and-high-resolution-edits)
|
||||
- [Contextual Diffusion](#contextual-diffusion)
|
||||
- [Tiled Diffusion](#tiled-diffusion)
|
||||
- [SEGS, detailers, and regional prompts](#segs-detailers-and-regional-prompts)
|
||||
- [Tagging images and regions](#tagging-images-and-regions)
|
||||
- [Images, masks, latents, and sampler extras](#images-masks-latents-and-sampler-extras)
|
||||
- [Settings and optional integrations](#settings-and-optional-integrations)
|
||||
- [License, acknowledgements, and research](#license-acknowledgements-and-research)
|
||||
|
||||
Open **Manager** from the ComfyUI toolbar, click **Custom Nodes Manager**, search for **SimpleSyrup**, and click **Install**. Restart ComfyUI after installation.
|
||||
## Install
|
||||
|
||||
**Manual install**
|
||||
### ComfyUI Manager
|
||||
|
||||
If you would rather install it yourself, clone this repo into `ComfyUI/custom_nodes/`, activate your **ComfyUI venv**, and install this node pack's requirements.
|
||||
Open Manager and search the **Node Pack** list for **SimpleSyrup**, then select it and click **Install**. Restart ComfyUI when it finishes.
|
||||
|
||||
ComfyUI still has two Manager interfaces in circulation. In the legacy interface, the search is under **Custom Nodes Manager**.
|
||||
|
||||
### Manual install
|
||||
|
||||
Clone the repository into `ComfyUI/custom_nodes/` and install the requirements with the same Python environment that runs ComfyUI.
|
||||
|
||||
For a normal Windows virtual environment:
|
||||
|
||||
```powershell
|
||||
cd ComfyUI\custom_nodes
|
||||
Set-Location ComfyUI\custom_nodes
|
||||
git clone https://github.com/Artificial-Sweetener/SimpleSyrup.git
|
||||
cd SimpleSyrup
|
||||
pip install -r requirements.txt
|
||||
Set-Location SimpleSyrup
|
||||
..\..\venv\Scripts\python.exe -m pip install -r requirements.txt
|
||||
```
|
||||
|
||||
ComfyUI already provides the heavy shared runtime stack, including PyTorch. SimpleSyrup adds the packages it needs for specific features, including TorchLanc, Ultralytics, ONNX Runtime, Segment Anything, and Hugging Face download helpers.
|
||||
For ComfyUI Windows Portable, run this from the portable installation folder after cloning the repository:
|
||||
|
||||
## The Nodes
|
||||
```powershell
|
||||
.\python_embeded\python.exe -m pip install -r .\ComfyUI\custom_nodes\SimpleSyrup\requirements.txt
|
||||
```
|
||||
|
||||
SimpleSyrup is organized around workflow jobs, not socket types.
|
||||
Restart ComfyUI after installation. SimpleSyrup uses ComfyUI's v3 extension API, so you need a current version of ComfyUI for the nodes to appear.
|
||||
|
||||
### Impact-Style SEGS Workflows
|
||||
ComfyUI already supplies PyTorch and the rest of the shared runtime. SimpleSyrup installs the packages used by its own features, including [TorchLanc](https://github.com/Artificial-Sweetener/TorchLanc), Ultralytics, ONNX Runtime, Segment Anything, and the Hugging Face download helpers.
|
||||
|
||||
SimpleSyrup speaks the Impact Pack SEGS shape on purpose. It can read Impact-style SEGS, sort them, combine them, tile them, and emit SEGS payloads that Impact-style consumers can read.
|
||||
## Model loading
|
||||
|
||||
The detailer nodes are modeled heavily on **ComfyUI Impact Pack**. Detectors create SEGS, SEGS choose the crop areas, crops are sampled, masks are feathered, and the results are composited back into the source image.
|
||||
Loading a checkpoint used to feel like choosing one file. Newer model families can mean a diffusion model, several text encoders, a VAE, and then the precision and device choices for all of them. I made the SimpleSyrup loaders so I could deal with that setup once and get on with the workflow.
|
||||
|
||||
- **Prompt SEGS w/ SAM** uses GroundingDINO to find prompt-matched boxes, SAM to segment them, optional negative prompting to subtract unwanted areas, and optional ViTMatte refinement to clean up mask edges. It returns both SEGS and a combined mask.
|
||||
- **Detect SEGS w/ Ultralytics** runs bbox or segmentation detection, filters by threshold and size, supports label filtering, and returns Impact-compatible SEGS plus a combined mask.
|
||||
- **Batch SEGS** combines multiple SEGS inputs into one ordered SEGS payload.
|
||||
- **Detail SEGS by Scale Factor** upscales each SEG crop, samples it, downsizes it back, and composites it into the original image with feathering and optional denoise masks.
|
||||
- **Detail SEGS by Scale Factor w/ Tiled Diffusion** uses the same crop/detail idea, but samples large crops through SimpleSyrup's tiled diffusion path.
|
||||
- **Detail SEGS as Regions** runs one regional MultiDiffusion pass over the image and pairs every SEG with its matching `CONDITIONING_BATCH` entry.
|
||||
**Simple Load Checkpoint** is the normal checkpoint loader. It has an optional VAE override and keeps CLIP skip beside the model controls where I expect to find it.
|
||||
|
||||
The regional node is different from the per-crop detailers. It samples one full-image latent with regional conditioning, using the global prompt for full-image context while each SEG gets its own positive conditioning.
|
||||
**Simple Load Anima** loads Anima with its Qwen text encoder and Qwen image VAE. You can select every part yourself. If you don't want to, the automatic choices can find and download the known checksum-pinned support files.
|
||||
|
||||
### Per-Segment Prompt Batches
|
||||
**Simple Load FLUX** handles FLUX.1 with CLIP-L, T5-XXL, and its VAE. **Simple Load FLUX.2** inspects the selected diffusion model and chooses the matching text encoder family for FLUX.2 dev, Klein 4B, or Klein 9B/KV conditioning. Both loaders can find or download their known text encoders and VAEs with visible Comfy progress.
|
||||
|
||||
SimpleSyrup layers the ADetailer habit I missed from WebUI on top of the Impact-style detailer shape: writing per-segment prompt batches inline with `[SEP]`.
|
||||
The FLUX loaders only download those revision-locked, checksum-pinned support files. You still install and select the diffusion model. They also expose manual component selection, diffusion weight precision, and text-encoder device placement. Moving text encoding to the CPU can save VRAM, although it will take longer.
|
||||
|
||||
Those prompts become an ordered `CONDITIONING_BATCH`, so prompt 1 stays matched to SEG 1, prompt 2 stays matched to SEG 2, and so on. This keeps the graph readable when each detected item needs its own prompt.
|
||||
## Large images and high-resolution edits
|
||||
|
||||
- **Encode Prompt Batch** splits prompt text with `[SEP]` and encodes ordered positive and negative `CONDITIONING_BATCH` values.
|
||||
- **Conditioning Batch Start** and **Conditioning Batch Append** build ordered conditioning batches for per-segment and regional workflows.
|
||||
- **Batch Region Conditioning** combines normal `CONDITIONING` values and existing `CONDITIONING_BATCH` values into one ordered regional batch.
|
||||
- **Encode Prompt Batch w/ Prompt Control** is exported when Prompt Control is installed. It keeps the `[SEP]` batching workflow while letting Prompt Control handle prompt scheduling and encoding.
|
||||
- **Schedule & Encode Prompts** is also exported when Prompt Control is installed. It schedules Prompt Control LoRA tags, encodes positive and negative prompts, and returns normal conditioning or SimpleSyrup batches depending on whether `[SEP]` is used.
|
||||
Tiled diffusion handles the obvious large-image problem: sometimes the latent is too big to evaluate all at once. There is a worse version. The model has enough memory to run, but the canvas is so far outside its normal working resolution that it starts making terrible decisions anyway.
|
||||
|
||||
### Tile, Tag, and Guide
|
||||
Tiling keeps each evaluation small. It does not make the tiles understand the same complete image. That second problem is why I made Contextual Diffusion.
|
||||
|
||||
**Tile & Tag SEGS** is for workflows where tile regions should carry their own generated prompt guidance.
|
||||
### Contextual Diffusion
|
||||
|
||||
It splits an image into deterministic tile SEGS, crops each tile, runs WD14 tagging on each crop, prefixes your universal positive prompt text, and CLIP-encodes the resulting prompts into a `CONDITIONING_BATCH`. The order matters: the conditioning batch is aligned to the tile SEGS order so downstream per-SEG or regional nodes can pick the right prompt for the right area.
|
||||
**KSampler (Contextual Diffusion)** is an original sampling method I developed for editing and refining oversized latent canvases.
|
||||
|
||||
That is the kind of thing that is easy to do once by hand and annoying to keep correct in a real graph.
|
||||
The first real target was a 2160 × 3072 source image I wanted to edit with FLUX.2 Klein 4B. Downscaling made the edit coherent, but that defeated the point of starting with a high-resolution source. Ordinary tiled diffusion kept much more detail and looked promising at first. Then I looked at the whole image. One tile had found a figure, another had invented a second figure, and different parts of the cathedral had become different buildings. The overlaps were smooth! The scene was still nonsense.
|
||||
|
||||
- **Tag SEGS w/ WD14** tags existing SEGS crops with a connected WD14 tagger and returns conditioning aligned to the original SEGS order.
|
||||
- **Tag SEGS w/ External LLM** sends each SEG crop to a configured vision-capable external LLM, formats the returned tags, and returns aligned conditioning for detail or regional workflows.
|
||||
I needed Klein to see the complete composition and the full-resolution detail during the same denoising process. Contextual Diffusion does that by making overlapping local predictions on the original latent and a second prediction from a smaller, aspect-preserving view of the whole image during the early steps.
|
||||
|
||||
### External LLM Prompting
|
||||
That took some trial and error. Directly blending the whole-image prediction into the tiles made the result blurry. Leaving it active too long produced smears, repeated edges, and other low-resolution garbage in the final detail. What finally worked was subtracting the low-frequency interpretation already present in the tiled prediction and adding only the difference from the whole-image prediction:
|
||||
|
||||
**External LLM Prompt** sends system and user prompts to a configured OpenAI-compatible provider and returns the assistant response as text. It can also include the first image from an optional image input when the selected provider model supports vision.
|
||||
`prediction = local + scheduled_weight × (global_upsampled − local_low_frequency)`
|
||||
|
||||
The external LLM nodes use the endpoint and API key configured in SimpleSyrup settings.
|
||||
The whole-image correction is strongest at the beginning and can decay before the model starts settling fine texture. Distilled Klein models commonly finish in four steps, so even one corrected step is already a quarter of the denoising process.
|
||||
|
||||
### Tiled Sampling
|
||||
Contextual Diffusion is for edits the model already knows how to make at a normal resolution. I use it for clothing, material, color, jewelry, expression, local lighting, and other changes where I want to keep the source pose and composition. If I need a completely new pose, camera, and environment, I establish those at a normal working resolution first and refine the result afterward.
|
||||
|
||||
**KSampler (Tiled Diffusion)** is a KSampler-style node with selectable **MultiDiffusion** and **Mixture of Diffusers** modes.
|
||||
FLUX.2 reference latents stay complete and ordered in every local and whole-image evaluation. This lets one image retain the target composition while other images continue to provide complete subject or style references. The sampler also supports Anima's singleton-depth latent shape, which is useful when refining an illustration after a conventional resize.
|
||||
|
||||
It splits the latent into tiles, denoises tile predictions, and blends them back together during sampling. MultiDiffusion averages overlapping predictions. Mixture of Diffusers uses weighted blending. Both are there because large images and large upscale passes often need a different strategy than normal full-latent denoising.
|
||||
You can connect SEGS to replace the normal grid with a region-guided context plan. This gives you some control over where the local windows fall. Earlier versions ran regular tiles and a second bank of SAM views at the same time because I thought more views of the important objects would help. Instead, I got duplicated hats, extra limbs, repeated garment edges, and other semantic echoes. It was the wrong architecture, so I removed it. The current method uses one local plan at a time and returns its actual windows through `contexts_segs` so you can see what it evaluated.
|
||||
|
||||
Tiled diffusion is here because it was one of the high-resolution workflow tools I kept reaching for in my WebUI setup. SimpleSyrup brings MultiDiffusion and Mixture of Diffusers behavior into normal Comfy sampling nodes, so large latent jobs can be tiled without giving up Comfy's explicit conditioning and graph wiring.
|
||||
The cost is one tiled prediction pass per denoising step and one smaller whole-image evaluation for each step using the correction. UniPC, regional conditioning, ControlNet, and GLIGEN are currently rejected because I haven't validated their spatial behavior across both context sizes.
|
||||
|
||||
This also matters for Anima workflows. Anima can produce beautiful images, but pushing beyond its comfortable native size with untiled diffusion upscale can smear detail instead of improving it. The tiled path gives those workflows another route.
|
||||
I found the formula by comparing the failures and adjusting the method until the whole-image branch could fix composition without taking the detail away from the tiles. After I had implemented it, I learned about [Upsample Guidance](https://arxiv.org/abs/2404.01709). It uses a closely related separation between low-frequency guidance and a high-resolution residual.
|
||||
|
||||
### Diffusion Loaders
|
||||
Upsample Guidance wasn't part of how I developed Contextual Diffusion. There are also practical differences: my high-resolution prediction is assembled from bounded tiles, the complete image is fit into an aspect-preserving context, reference latents stay whole, and the node has its own early-step controls and optional SEGS planning. Still, the mathematical relationship is real. My experiments are qualitative, and I describe the method as independent development of a related multiscale idea instead of claiming priority over that paper.
|
||||
|
||||
**Simple Load Checkpoint** is meant to feel familiar if you come from WebUI, where the common generation controls live near the model selection.
|
||||
### Tiled Diffusion
|
||||
|
||||
- **Simple Load Checkpoint** loads a checkpoint, optionally replaces the checkpoint VAE, and keeps CLIP skip in the same place.
|
||||
- **Simple Load Anima** is for Anima workflows. It loads Anima with the Qwen text encoder and Qwen image VAE it expects. You can choose the files yourself or let SimpleSyrup resolve the known Anima assets automatically. It keeps model, VAE, dtype, and device decisions together so the rest of the graph can get on with the image.
|
||||
**KSampler (Tiled Diffusion)** is the more direct tiled sampler. It divides the latent into overlapping contexts, evaluates them in batches, and combines the predictions during every denoising step.
|
||||
|
||||
### Model and Detector Loaders
|
||||
MultiDiffusion averages the overlapping predictions. Mixture of Diffusers uses Gaussian weights that favor the center of each context. This works well when local evaluation and overlap blending are enough for the image. Contextual Diffusion adds the whole-image correction for edits where the separate contexts lose track of the complete scene.
|
||||
|
||||
These nodes load the models used by detection, segmentation, tagging, matting, and compatibility workflows.
|
||||
The same tiled sampling path is available in **Detail SEGS by Scale Factor w/ Tiled Diffusion** for large detailer crops and **KSampler (Prompt by Tiled Region)** for regional prompts on large canvases.
|
||||
|
||||
- **SAM Model Loader** loads SAM, SAM-HQ, and MobileSAM choices for segmentation workflows.
|
||||
- **GroundingDINO Model Loader** loads GroundingDINO with an explicit BERT text encoder.
|
||||
- **ViTMatte Model Loader** loads ViTMatte for mask edge refinement.
|
||||
- **Load Ultralytics Model** loads an Ultralytics detector and exposes both SimpleSyrup's native detector model and Impact-style compatibility outputs.
|
||||
- **Load WD14 Tagger** loads a SmilingWolf WD14 ONNX model and its tag CSV.
|
||||
- **LayerStyle SAM Models Adapter** splits a LayerStyle `LS_SAM_MODELS` bundle into separate `SAM_MODEL` and `DINO_MODEL` outputs.
|
||||
- **Grounded SAM Model Info** returns JSON metadata for selected SAM and GroundingDINO models.
|
||||
## SEGS, detailers, and regional prompts
|
||||
|
||||
The LayerStyle adapter exists because good ComfyUI workflows should not make you reload the same SAM or GroundingDINO model just because one node pack uses a different socket shape.
|
||||
Impact Pack already had a useful way to represent detected and masked regions: `SEGS`. I built SimpleSyrup around the same shape so regions can move between compatible detectors, these nodes, and Impact workflows without reloading models or rebuilding the masks.
|
||||
|
||||
### Sampler and Scheduler Extras
|
||||
**Prompt SEGS w/ SAM** uses GroundingDINO to find objects from text and SAM to segment them. It also supports negative prompting and optional ViTMatte edge refinement. **Detect SEGS w/ Ultralytics** creates regions from bounding-box or segmentation models with confidence, label, and size filtering. Existing masks can enter the same workflow through **Mask to SEGS**, while **SEGS from SAM Output** runs automatic unprompted segmentation from a connected SAM model.
|
||||
|
||||
**KSampler (Extras)** keeps the normal Comfy sampler shape, but adds sampler and scheduler behavior I wanted available without dragging in a separate sampler stack.
|
||||
**Simple Preview SEGS** shows the regions over the image, lets you select them from an interactive grid, and passes the original SEGS onward. **Batch SEGS** combines several ordered SEGS inputs.
|
||||
|
||||
It includes:
|
||||
The scale-factor detailers work on one crop at a time. They enlarge the crop, sample it, shrink it back, and composite it into the source image with feathering and optional denoise masks. **Detail SEGS as Regions** takes another route: it keeps the full image in one MultiDiffusion pass, uses the global conditioning across the image, and pairs each SEG with its own ordered regional conditioning.
|
||||
|
||||
- `euler_a_a1111`, an A1111/k-diffusion-style Euler ancestral sampler.
|
||||
- **AYS SD1** and **AYS SDXL** schedules.
|
||||
- **GITS**.
|
||||
- **automatic_a1111** scheduler behavior.
|
||||
- **beta57**, a local reimplementation of the RES4LYF beta57 scheduler preset.
|
||||
The prompt batching came directly from ADetailer. **Encode Prompt Batch** splits positive and negative text with `[SEP]`. You can write `[SEP|name]` to keep a long prompt readable; matching still follows the order of the prompts and regions. The first prompt is global, and each later prompt belongs to the corresponding mask or SEG.
|
||||
|
||||
The node still uses Comfy-style seed handling, partial denoise behavior, progress callbacks, and normal positive/negative conditioning inputs.
|
||||
**Conditioning Batch Start**, **Conditioning Batch Append**, and **Batch Region Conditioning** build the same ordered structure from existing conditioning. **Compose Regional Conditioning** converts a global-first prompt batch and ordered masks into normal masked Comfy conditioning. The dedicated **KSampler (Prompt by Region)** and tiled version apply the regional prompt batch during sampling.
|
||||
|
||||
### Image, Prompt, and Latent Utilities
|
||||
If [ComfyUI Prompt Control](https://github.com/asagi4/comfyui-prompt-control) is installed, SimpleSyrup also exports **Encode Prompt Batch w/ Prompt Control** and **Schedule & Encode Prompts**. They preserve Prompt Control scheduling and LoRA hooks across `[SEP]` regions. The rest of the pack loads normally when Prompt Control is absent.
|
||||
|
||||
These nodes handle the smaller jobs that show up all over image workflows.
|
||||
Model loading stays separate from detection. There are loaders for SAM, GroundingDINO, ViTMatte, and Ultralytics. **LayerStyle SAM Models Adapter** accepts a ComfyUI Layer Style Advance `LS_SAM_MODELS` bundle and exposes the loaded SAM and GroundingDINO models through the normal sockets used here.
|
||||
|
||||
- **Resize Image to Target** resizes image batches with stretch, keep-aspect, crop, and pad modes. It can round output dimensions to a divisibility target, anchor crop or pad placement, process batches in chunks, resize a mask with the image, and use GPU Lanczos through [TorchLanc](https://github.com/Artificial-Sweetener/TorchLanc).
|
||||
- **Simple VAE Encode** encodes an image to latent space, but reuses the source latent when the graph proves the image came from an unmodified `VAEDecode`.
|
||||
- **Upscale Latent From Image** finds the latent behind an unmodified decoded image and expands to Comfy's latent upscale behavior.
|
||||
- **VAE Encode (Options)** and **VAE Decode (Options)** wrap ComfyUI's normal and tiled VAE paths behind one explicit tiling toggle, with spatial and temporal tile controls available when tiling is enabled.
|
||||
- **Latent Diagnostics** passes a latent through unchanged while reporting shape, dtype, device, and tiling-fit details.
|
||||
- **Prompt Encode Style** creates Prompt Control style tags from an encode-style selection.
|
||||
- **Prompt Encode Style & Normalization** creates Prompt Control style and normalization tags together.
|
||||
- **Scale Factor** provides a bounded scale multiplier for nodes that expect one.
|
||||
- **Seed** provides a reusable seed value with ComfyUI seed controls.
|
||||
## Tagging images and regions
|
||||
|
||||
The provenance nodes trace the graph. They do not guess from tensor values. If an image has been loaded, edited, cropped, detailed, resized, or otherwise changed, the original latent provenance is broken and the node will not pretend otherwise.
|
||||
**Load WD14 Tagger** loads a SmilingWolf WD14 ONNX model and its tag CSV. **Tag SEGS w/ WD14** runs the tagger on existing SEG crops and keeps the resulting conditioning in the same order. **Tile & Tag SEGS** makes a deterministic set of tile regions, tags each crop, prefixes shared positive text, and returns the SEGS together with their matching conditioning batch.
|
||||
|
||||
## Settings
|
||||
The external LLM nodes use a configured OpenAI-compatible provider. **Tag SEGS w/ External LLM** sends each region crop to a vision-capable model and returns aligned conditioning. **External LLM Prompt** sends system and user prompts and returns the response as text, with an optional image for models that support vision.
|
||||
|
||||
SimpleSyrup adds ComfyUI settings for model visibility and external LLM access:
|
||||
## Images, masks, latents, and sampler extras
|
||||
|
||||
- **SimpleSyrup: Show downloadable models in loader dropdowns** controls whether known downloadable SAM, GroundingDINO, ViTMatte, and WD14 models appear before they are installed locally.
|
||||
- **SimpleSyrup: External LLM endpoint** stores the OpenAI-compatible base URL used by external LLM prompt nodes.
|
||||
- **SimpleSyrup: External LLM API key** stores the API key for that endpoint in OS credential storage.
|
||||
**Load Image List** loads files in selection order as separate image list items, so each image keeps its own dimensions. **Load Mask Batch** loads same-sized files as one `BHW` mask batch and applies the selected channel consistently to every file.
|
||||
|
||||
When downloadable models are shown, supported loaders list known model choices even if the files are not installed yet. When they are hidden, those dropdowns only show models SimpleSyrup can verify locally. Anima's automatic Qwen text encoder and VAE resolution is handled by the Anima loader itself.
|
||||
**Resize Image to Target** handles stretch, keep-aspect, crop, and pad modes. It supports divisibility rounding, anchored crop and pad placement, chunked batches, paired masks, and GPU Lanczos through TorchLanc.
|
||||
|
||||
## License & Acknowledgements
|
||||
**VAE Encode (Options)** and **VAE Decode (Options)** put the normal and tiled VAE paths behind one explicit tiling control, including spatial and temporal tile settings where Comfy supports them.
|
||||
|
||||
**SimpleSyrup** is licensed under the GNU Affero General Public License v3.0 or later (**AGPL-3.0-or-later**). Please read the full [LICENSE](LICENSE) included with this repo.
|
||||
**Simple VAE Encode** can reuse the source latent when the graph proves that its image came directly from an unmodified `VAEDecode`. **Upscale Latent From Image** uses the same provenance to find and resize the original latent. Loading, editing, cropping, detailing, or resizing the image breaks that provenance. These nodes follow the graph instead of trying to identify a latent from the finished tensor.
|
||||
|
||||
AGPL-3.0-or-later is a strong copyleft license. If you convey SimpleSyrup or a modified version, you must provide the corresponding source; and if you let users interact with a modified version over a network, you must offer those users the corresponding source for that modified version.
|
||||
**KSampler (Extras)** adds the A1111/k-diffusion-style `euler_a_a1111` sampler, AYS SD1 and SDXL schedules, GITS, the `automatic_a1111` scheduler, and a local implementation of the RES4LYF beta57 preset. It keeps Comfy's regular seed handling, partial denoise behavior, progress callbacks, and conditioning inputs.
|
||||
|
||||
**Seed Variation** patches a MODEL so Comfy-native samplers mix their normal initial noise toward a second deterministic seed. Strength `0` keeps the sampler seed unchanged, while strength `1` uses variation-seed initial noise. Ancestral and SDE samplers continue to use the sampler seed for additional noise introduced after initialization.
|
||||
|
||||
The remaining utilities are **Latent Diagnostics**, **Scale Factor**, and **Seed**. Latent Diagnostics reports the latent shape, dtype, device, and tiled-sampling compatibility while passing it through unchanged.
|
||||
|
||||
## Settings and optional integrations
|
||||
|
||||
SimpleSyrup adds three ComfyUI settings:
|
||||
|
||||
- **SimpleSyrup: Show downloadable models in loader dropdowns** controls whether known downloadable SAM, GroundingDINO, ViTMatte, and WD14 choices appear before they are installed.
|
||||
- **SimpleSyrup: External LLM endpoint** stores the OpenAI-compatible base URL used to discover provider models and run the external prompt nodes.
|
||||
- **SimpleSyrup: External LLM API key** stores the provider key in OS credential storage.
|
||||
|
||||
With downloadable models enabled, selecting a known missing catalog entry lets its loader download the required files. With the setting disabled, the dropdowns contain models SimpleSyrup can verify locally. Anima, FLUX.1, and FLUX.2 support components are resolved by their own loaders and use checksum-pinned automatic choices.
|
||||
|
||||
Saving the external LLM endpoint and API key refreshes the provider models available in connected SimpleSyrup nodes. Image inputs require a provider model with vision support.
|
||||
|
||||
SimpleSyrup currently interoperates with:
|
||||
|
||||
- [ComfyUI Prompt Control](https://github.com/asagi4/comfyui-prompt-control) for scheduled prompts and regional LoRA hooks.
|
||||
- ComfyUI Impact Pack through compatible `SEGS` values.
|
||||
- ComfyUI Layer Style Advance through its `LS_SAM_MODELS` bundle.
|
||||
|
||||
## License, acknowledgements, and research
|
||||
|
||||
**SimpleSyrup** is licensed under the GNU Affero General Public License v3.0 or later (**AGPL-3.0-or-later**). Please read the full [LICENSE](LICENSE) included with this repository.
|
||||
|
||||
AGPL-3.0-or-later is a strong copyleft license. If you convey SimpleSyrup or a modified version, you must provide the corresponding source. If users interact with a modified version over a network, you must offer those users the corresponding source for that version.
|
||||
|
||||
SimpleSyrup owes a lot to other projects:
|
||||
|
||||
- [ComfyUI Impact Pack](https://github.com/ltdrdata/ComfyUI-Impact-Pack) for the SEGS workflow vocabulary and detailer shape this pack is heavily modeled around.
|
||||
- [ADetailer](https://github.com/Bing-su/adetailer) for the inline `[SEP]` per-segment prompt workflow I missed from WebUI.
|
||||
- [ComfyUI Layer Style Advance](https://github.com/chflame163/ComfyUI_LayerStyle_Advance) for the SAM workflow surface this pack interoperates with.
|
||||
- [Tiled Diffusion & VAE for AUTOMATIC1111](https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111) for practical tiled diffusion and Mixture of Diffusers behavior.
|
||||
- [RES4LYF](https://github.com/ClownsharkBatwing/RES4LYF) for the beta57 scheduler preset reimplemented here.
|
||||
- [ComfyUI](https://github.com/Comfy-Org/ComfyUI) provides the engine and graph ecosystem this pack runs on.
|
||||
- [ComfyUI Impact Pack](https://github.com/ltdrdata/ComfyUI-Impact-Pack) established the SEGS workflow vocabulary and detailer structure used here.
|
||||
- [ADetailer](https://github.com/Bing-su/adetailer) is where the inline `[SEP]` per-segment prompt workflow came from.
|
||||
- [ComfyUI Prompt Control](https://github.com/asagi4/comfyui-prompt-control) provides the scheduled prompt and LoRA-hook behavior used by the optional integration.
|
||||
- [ComfyUI Layer Style Advance](https://github.com/chflame163/ComfyUI_LayerStyle_Advance) provides the SAM model bundle SimpleSyrup can adapt.
|
||||
- [Tiled Diffusion & VAE for AUTOMATIC1111](https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111) informed the practical tiled diffusion and Mixture of Diffusers behavior reimplemented here.
|
||||
- [RES4LYF](https://github.com/ClownsharkBatwing/RES4LYF) is the source of the beta57 scheduler preset reimplemented here.
|
||||
- [ComfyUI-ppm](https://github.com/pamparamm/ComfyUI-ppm) by pamparamm provides the ModelPatcher-based NegPiP behavior adapted here and builds on the [ComfyUI port](https://github.com/laksjdjf/cd-tuner_negpip-ComfyUI) by laksjdjf and the [original WebUI implementation](https://github.com/hako-mikan/sd-webui-negpip) by hako-mikan.
|
||||
|
||||
SimpleSyrup also vendors or reimplements selected third-party behavior for SAM-HQ, MobileSAM, GroundingDINO, AUTOMATIC1111 sampler behavior, k-diffusion, and tiled diffusion behavior. See [third_party/NOTICE.md](third_party/NOTICE.md) for the full third-party notices.
|
||||
SimpleSyrup also vendors or reimplements selected third-party behavior for SAM-HQ, MobileSAM, GroundingDINO, AUTOMATIC1111 sampler behavior, k-diffusion, and tiled diffusion. See [third_party/NOTICE.md](third_party/NOTICE.md) for the complete notices.
|
||||
|
||||
### Research Citations
|
||||
### Research citations
|
||||
|
||||
SimpleSyrup's tiled diffusion behavior is based on ideas from MultiDiffusion and Mixture of Diffusers.
|
||||
SimpleSyrup's tiled diffusion behavior builds on MultiDiffusion and Mixture of Diffusers. Contextual Diffusion was developed independently and was later found to share a related multiscale residual principle with Upsample Guidance.
|
||||
|
||||
```bibtex
|
||||
@article{bar2023multidiffusion,
|
||||
@@ -206,9 +215,17 @@ SimpleSyrup's tiled diffusion behavior is based on ideas from MultiDiffusion and
|
||||
}
|
||||
```
|
||||
|
||||
```bibtex
|
||||
@article{hwang2024upsample,
|
||||
title={Upsample Guidance: Scale Up Diffusion Models without Training},
|
||||
author={Hwang, Juno and Park, Yong-Hyun and Jo, Youngjung},
|
||||
journal={arXiv preprint arXiv:2404.01709},
|
||||
year={2024}
|
||||
}
|
||||
```
|
||||
|
||||
## From the Developer 💖
|
||||
|
||||
|
||||
- **Buy Me a Coffee**: You can help fuel more projects like this at my [Ko-fi page](https://ko-fi.com/artificial_sweetener).
|
||||
- **My Website & Socials**: See my art, poetry, and other dev updates at [artificialsweetener.ai](https://artificialsweetener.ai).
|
||||
- **My Website & Socials**: See my art, poetry, research notes, and other development updates at [artificialsweetener.ai](https://artificialsweetener.ai).
|
||||
- **If you like this project**, it would mean a lot to me if you gave me a star here on GitHub!! ⭐
|
||||
|
||||
+16
@@ -12,9 +12,21 @@ from . import simple_syrup as _simple_syrup_package
|
||||
|
||||
sys.modules.setdefault("simple_syrup", _simple_syrup_package)
|
||||
|
||||
from .simple_syrup.runtime.attention_region_prompt_handler import ( # noqa: E402
|
||||
register_attention_region_prompt_handler,
|
||||
)
|
||||
from .simple_syrup.runtime.comfy_safetensors_dtypes import ( # noqa: E402
|
||||
register_comfy_safetensors_dtypes,
|
||||
)
|
||||
from .simple_syrup.runtime.external_llm_routes import ( # noqa: E402
|
||||
register_external_llm_routes,
|
||||
)
|
||||
from .simple_syrup.runtime.mask_batch_preview_routes import ( # noqa: E402
|
||||
register_mask_batch_preview_routes,
|
||||
)
|
||||
from .simple_syrup.runtime.quant_cache_routes import ( # noqa: E402
|
||||
register_quant_cache_routes,
|
||||
)
|
||||
from .simple_syrup.runtime.settings_routes import register_settings_routes # noqa: E402
|
||||
|
||||
WEB_DIRECTORY = "./web/dist"
|
||||
@@ -39,7 +51,11 @@ async def comfy_entrypoint() -> object:
|
||||
|
||||
|
||||
register_settings_routes()
|
||||
register_comfy_safetensors_dtypes()
|
||||
register_quant_cache_routes()
|
||||
register_external_llm_routes()
|
||||
register_mask_batch_preview_routes()
|
||||
register_attention_region_prompt_handler()
|
||||
|
||||
__all__ = [
|
||||
"WEB_DIRECTORY",
|
||||
|
||||
Generated
+2
-2
@@ -1,12 +1,12 @@
|
||||
{
|
||||
"name": "simple-syrup-comfyui",
|
||||
"version": "1.5.0",
|
||||
"version": "1.9.1",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "simple-syrup-comfyui",
|
||||
"version": "1.5.0",
|
||||
"version": "1.9.1",
|
||||
"license": "AGPL-3.0-or-later",
|
||||
"devDependencies": {
|
||||
"@eslint/js": "^9.39.1",
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "simple-syrup-comfyui",
|
||||
"version": "1.5.0",
|
||||
"version": "1.9.1",
|
||||
"private": true,
|
||||
"license": "AGPL-3.0-or-later",
|
||||
"type": "module",
|
||||
|
||||
+9
-2
@@ -5,7 +5,7 @@ build-backend = "setuptools.build_meta"
|
||||
[project]
|
||||
name = "SimpleSyrup"
|
||||
description = "Workflow-focused ComfyUI extensions for image generation."
|
||||
version = "1.5.0"
|
||||
version = "1.9.1"
|
||||
license = "AGPL-3.0-or-later"
|
||||
license-files = ["LICENSE"]
|
||||
requires-python = ">=3.11"
|
||||
@@ -43,8 +43,11 @@ extend-exclude = [
|
||||
select = ["E", "F", "I", "UP", "B", "C4", "ANN"]
|
||||
ignore = ["ANN401"]
|
||||
|
||||
[tool.ruff.lint.isort]
|
||||
known-first-party = ["simple_syrup"]
|
||||
|
||||
[tool.mypy]
|
||||
python_version = "3.11"
|
||||
python_version = "3.12"
|
||||
warn_return_any = true
|
||||
warn_unused_configs = true
|
||||
disallow_untyped_defs = true
|
||||
@@ -53,6 +56,7 @@ check_untyped_defs = true
|
||||
no_implicit_optional = true
|
||||
strict_equality = true
|
||||
explicit_package_bases = true
|
||||
mypy_path = ["tests"]
|
||||
exclude = [
|
||||
"simple_syrup/third_party/groundingdino_runtime",
|
||||
"simple_syrup/third_party/sam_hq_runtime",
|
||||
@@ -65,6 +69,9 @@ ignore_missing_imports = true
|
||||
[tool.pytest.ini_options]
|
||||
pythonpath = [".", "../.."]
|
||||
testpaths = ["tests"]
|
||||
markers = [
|
||||
"external_artifact: requires a locally installed external source or generated benchmark artifact",
|
||||
]
|
||||
filterwarnings = [
|
||||
"error",
|
||||
"ignore:builtin type SwigPyPacked has no __module__ attribute:DeprecationWarning",
|
||||
|
||||
@@ -6,6 +6,6 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
__version__ = "1.5.0"
|
||||
__version__ = "1.9.1"
|
||||
|
||||
__all__: list[str] = ["__version__"]
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Define versioned, quality-aware Anima quantization profiles."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
|
||||
from .model_quantization import (
|
||||
QuantizationFormat,
|
||||
QuantizationProfile,
|
||||
TensorDescriptor,
|
||||
)
|
||||
|
||||
ORIGINAL_PROFILE = QuantizationProfile("original", "Original", 2, frozenset())
|
||||
FP8_E4M3_PROFILE = QuantizationProfile(
|
||||
"fp8-e4m3",
|
||||
"FP8 E4M3",
|
||||
2,
|
||||
frozenset({QuantizationFormat.FP8_E4M3}),
|
||||
)
|
||||
FP8_E5M2_PROFILE = QuantizationProfile(
|
||||
"fp8-e5m2",
|
||||
"FP8 E5M2",
|
||||
2,
|
||||
frozenset({QuantizationFormat.FP8_E5M2}),
|
||||
)
|
||||
MXFP8_PROFILE = QuantizationProfile(
|
||||
"mxfp8",
|
||||
"MXFP8",
|
||||
2,
|
||||
frozenset({QuantizationFormat.MXFP8}),
|
||||
)
|
||||
NVFP4_MIXED_PROFILE = QuantizationProfile(
|
||||
"nvfp4-mixed",
|
||||
"NVFP4 (Mixed)",
|
||||
3,
|
||||
frozenset({QuantizationFormat.FP8_E4M3, QuantizationFormat.NVFP4}),
|
||||
)
|
||||
_PROFILES = (
|
||||
ORIGINAL_PROFILE,
|
||||
FP8_E4M3_PROFILE,
|
||||
FP8_E5M2_PROFILE,
|
||||
MXFP8_PROFILE,
|
||||
NVFP4_MIXED_PROFILE,
|
||||
)
|
||||
_MAIN_BLOCK_PATTERN = re.compile(
|
||||
r"(?:^|\.)(?:net|diffusion_model)\.blocks\.(?P<index>\d+)\."
|
||||
)
|
||||
_PROTECTED_BLOCKS = {0, 1, 27}
|
||||
_FLOAT_DTYPES = {"F16", "BF16", "F32", "F64"}
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AnimaQuantizationRecipe:
|
||||
"""Assign formats only within Anima's quality-safe DiT block envelope."""
|
||||
|
||||
model_family: str = "Anima"
|
||||
version: int = 2
|
||||
|
||||
@property
|
||||
def profiles(self) -> tuple[QuantizationProfile, ...]:
|
||||
"""Return Anima's stable workflow-facing profile order."""
|
||||
|
||||
return _PROFILES
|
||||
|
||||
def profile_from_selection(self, selection: str) -> QuantizationProfile:
|
||||
"""Parse one current workflow selection into its profile."""
|
||||
|
||||
for profile in self.profiles:
|
||||
if selection in (profile.label, profile.profile_id):
|
||||
return profile
|
||||
valid = ", ".join(profile.label for profile in self.profiles)
|
||||
raise ValueError(f"quantization profile must be one of: {valid}.")
|
||||
|
||||
def policy_for(
|
||||
self,
|
||||
tensor: TensorDescriptor,
|
||||
profile: QuantizationProfile,
|
||||
) -> QuantizationFormat | None:
|
||||
"""Return Anima's per-tensor format while preserving sensitive layers."""
|
||||
|
||||
if profile not in self.profiles:
|
||||
raise ValueError(
|
||||
f"Unknown Anima quantization profile '{profile.profile_id}'."
|
||||
)
|
||||
if profile.is_original or not _is_matrix_weight(tensor):
|
||||
return None
|
||||
if "llm_adapter" in tensor.name or "adaln_modulation" in tensor.name:
|
||||
return None
|
||||
block_match = _MAIN_BLOCK_PATTERN.search(tensor.name)
|
||||
if block_match is None:
|
||||
return None
|
||||
if int(block_match.group("index")) in _PROTECTED_BLOCKS:
|
||||
return None
|
||||
if profile.profile_id == NVFP4_MIXED_PROFILE.profile_id:
|
||||
if "v_proj" in tensor.name or ".mlp." in tensor.name:
|
||||
return QuantizationFormat.FP8_E4M3
|
||||
if any(
|
||||
projection in tensor.name
|
||||
for projection in ("q_proj", "k_proj", "output_proj")
|
||||
):
|
||||
return QuantizationFormat.NVFP4
|
||||
return None
|
||||
return next(iter(profile.required_formats))
|
||||
|
||||
|
||||
def _is_matrix_weight(tensor: TensorDescriptor) -> bool:
|
||||
"""Return whether a tensor is an eligible floating-point matrix weight."""
|
||||
|
||||
return (
|
||||
tensor.dtype_name in _FLOAT_DTYPES
|
||||
and len(tensor.shape) == 2
|
||||
and tensor.name.endswith(".weight")
|
||||
)
|
||||
@@ -0,0 +1,15 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Parse explicit attention concepts without interpreting prompt language."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
def parse_attention_concepts(value: str) -> tuple[str, ...]:
|
||||
"""Return canonical concepts separated only by vertical bars."""
|
||||
|
||||
if not isinstance(value, str):
|
||||
raise TypeError("Attention concepts must be text.")
|
||||
return tuple(part.strip() for part in value.split("|") if part.strip())
|
||||
@@ -0,0 +1,46 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Classify complete Attention Coupling requests before runtime preparation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import StrEnum
|
||||
|
||||
from .conditioning_batch import ConditioningBatch
|
||||
|
||||
|
||||
class AttentionCouplingRequestMode(StrEnum):
|
||||
"""Select ordinary sampling or complete regional Attention Coupling."""
|
||||
|
||||
BYPASS = "bypass"
|
||||
ACTIVE = "active"
|
||||
|
||||
|
||||
def classify_attention_coupling_request(
|
||||
*,
|
||||
positive: object,
|
||||
negative: object,
|
||||
region_masks: object | None,
|
||||
) -> AttentionCouplingRequestMode:
|
||||
"""Return the execution mode or reject a partial regional request."""
|
||||
|
||||
has_conditioning_batch = isinstance(positive, ConditioningBatch) or isinstance(
|
||||
negative,
|
||||
ConditioningBatch,
|
||||
)
|
||||
has_region_masks = region_masks is not None
|
||||
if not has_conditioning_batch and not has_region_masks:
|
||||
return AttentionCouplingRequestMode.BYPASS
|
||||
if has_conditioning_batch and has_region_masks:
|
||||
return AttentionCouplingRequestMode.ACTIVE
|
||||
if has_conditioning_batch:
|
||||
raise ValueError(
|
||||
"Attention Coupling conditioning batches require region_masks. "
|
||||
"Connect ordered masks or use ordinary CONDITIONING on both inputs."
|
||||
)
|
||||
raise ValueError(
|
||||
"Attention Coupling region_masks require a CONDITIONING_BATCH on the "
|
||||
"positive or negative input. Disconnect the masks for ordinary sampling."
|
||||
)
|
||||
@@ -0,0 +1,30 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Resolve explicit two-dimensional geometry for flattened attention maps."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
|
||||
|
||||
def factor_spatial_geometry(
|
||||
token_count: int, *, target_aspect: float
|
||||
) -> tuple[int, int]:
|
||||
"""Return the token factor pair nearest the supplied positive aspect ratio."""
|
||||
|
||||
if type(token_count) is not int or token_count < 1:
|
||||
raise ValueError("Attention spatial token count must be positive.")
|
||||
if target_aspect <= 0.0:
|
||||
raise ValueError("Attention target aspect ratio must be positive.")
|
||||
candidates: list[tuple[float, int, int]] = []
|
||||
for height in range(1, math.isqrt(token_count) + 1):
|
||||
if token_count % height:
|
||||
continue
|
||||
width = token_count // height
|
||||
for candidate_height, candidate_width in ((height, width), (width, height)):
|
||||
error = abs(math.log((candidate_width / candidate_height) / target_aspect))
|
||||
candidates.append((error, candidate_height, candidate_width))
|
||||
_error, height, width = min(candidates)
|
||||
return height, width
|
||||
@@ -0,0 +1,175 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Define immutable requests and plans for attention-region capture."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from enum import StrEnum
|
||||
|
||||
from .attention_spatial_transform import AttentionSpatialTransform
|
||||
from .graph_provenance import GraphLink
|
||||
|
||||
|
||||
class AttentionRegionRequestKind(StrEnum):
|
||||
"""Identify one public attention-region operation."""
|
||||
|
||||
CONCEPT_SEGS = "concept_segs"
|
||||
ALL_PROMPT_SEGS = "all_prompt_segs"
|
||||
REGION_MASK = "region_mask"
|
||||
MASKED_CONDITIONING = "masked_conditioning"
|
||||
|
||||
|
||||
class AttentionCaptureProfile(StrEnum):
|
||||
"""Select the density of attention observations retained during sampling."""
|
||||
|
||||
FAST = "fast"
|
||||
BALANCED = "balanced"
|
||||
EXHAUSTIVE = "exhaustive"
|
||||
|
||||
|
||||
class AttentionEvidenceMode(StrEnum):
|
||||
"""Select honest inspection or derived concept-isolation evidence."""
|
||||
|
||||
CONCEPT = "concept isolation"
|
||||
RAW = "raw attention"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class AttentionRegionControls:
|
||||
"""Hold validated attention-native capture and region-shaping controls."""
|
||||
|
||||
capture_start: float
|
||||
capture_end: float
|
||||
minimum_strength: float
|
||||
minimum_consensus: float
|
||||
split_sensitivity: float
|
||||
minimum_region_size: int
|
||||
profile: AttentionCaptureProfile
|
||||
instance_recall: float = 0.65
|
||||
geometry_recall: float = 0.85
|
||||
keep_only: int = 0
|
||||
keep_by: str = "largest size"
|
||||
combine_segs: bool = False
|
||||
matte_solidity: float = 0.0
|
||||
edge_feather: int = 8
|
||||
evidence_mode: AttentionEvidenceMode = AttentionEvidenceMode.CONCEPT
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require normalized ranges and a non-empty capture interval."""
|
||||
|
||||
normalized = (
|
||||
self.capture_start,
|
||||
self.capture_end,
|
||||
self.minimum_strength,
|
||||
self.minimum_consensus,
|
||||
self.split_sensitivity,
|
||||
self.instance_recall,
|
||||
self.geometry_recall,
|
||||
self.matte_solidity,
|
||||
)
|
||||
if any(
|
||||
isinstance(value, bool) or not isinstance(value, int | float)
|
||||
for value in normalized
|
||||
):
|
||||
raise TypeError("Attention-region controls must be real numbers.")
|
||||
if not 0.0 <= self.capture_start < self.capture_end <= 1.0:
|
||||
raise ValueError("Attention capture start must be below end within 0..1.")
|
||||
if not 0.0 <= self.minimum_strength <= 1.0:
|
||||
raise ValueError("Minimum attention strength must be within 0..1.")
|
||||
if not 0.0 <= self.minimum_consensus <= 1.0:
|
||||
raise ValueError("Minimum attention consensus must be within 0..1.")
|
||||
if not 0.0 <= self.split_sensitivity <= 1.0:
|
||||
raise ValueError("Attention split sensitivity must be within 0..1.")
|
||||
if not 0.0 <= self.instance_recall <= 1.0:
|
||||
raise ValueError("Attention instance recall must be within 0..1.")
|
||||
if not 0.0 <= self.geometry_recall <= 1.0:
|
||||
raise ValueError("Attention geometry recall must be within 0..1.")
|
||||
if type(self.minimum_region_size) is not int or self.minimum_region_size < 1:
|
||||
raise ValueError("Minimum attention region size must be positive.")
|
||||
if type(self.keep_only) is not int or self.keep_only < 0:
|
||||
raise ValueError("Attention keep_only must be non-negative.")
|
||||
if self.keep_by not in ("largest size", "highest confidence"):
|
||||
raise ValueError("Attention keep_by has an invalid policy.")
|
||||
if type(self.combine_segs) is not bool:
|
||||
raise TypeError("Attention combine_segs must be boolean.")
|
||||
if not 0.0 <= self.matte_solidity <= 1.0:
|
||||
raise ValueError("Attention matte solidity must be within 0..1.")
|
||||
if type(self.edge_feather) is not int or self.edge_feather < 0:
|
||||
raise ValueError("Attention edge feather must be non-negative.")
|
||||
if not isinstance(self.profile, AttentionCaptureProfile):
|
||||
raise TypeError("Attention capture profile has an invalid type.")
|
||||
if not isinstance(self.evidence_mode, AttentionEvidenceMode):
|
||||
raise TypeError("Attention evidence mode has an invalid type.")
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class AttentionRegionRequest:
|
||||
"""Bind one public node request to its queries and capture controls."""
|
||||
|
||||
node_id: str
|
||||
kind: AttentionRegionRequestKind
|
||||
queries: tuple[str, ...]
|
||||
controls: AttentionRegionControls
|
||||
sampler_stage: int = 1
|
||||
spatial_transforms: tuple[AttentionSpatialTransform, ...] = ()
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require stable node identity and canonical non-empty query strings."""
|
||||
|
||||
if not self.node_id.strip():
|
||||
raise ValueError("Attention-region request node id cannot be empty.")
|
||||
if not isinstance(self.kind, AttentionRegionRequestKind):
|
||||
raise TypeError("Attention-region request kind has an invalid type.")
|
||||
if any(not query or query != query.strip() for query in self.queries):
|
||||
raise ValueError("Attention-region queries must be canonical strings.")
|
||||
if self.kind is AttentionRegionRequestKind.ALL_PROMPT_SEGS:
|
||||
if self.queries:
|
||||
raise ValueError(
|
||||
"All-prompt attention requests cannot contain queries."
|
||||
)
|
||||
elif not self.queries:
|
||||
raise ValueError("Concept and mask attention requests require concepts.")
|
||||
if type(self.sampler_stage) is not int or self.sampler_stage < -1:
|
||||
raise ValueError("Attention sampler stage must be -1 or greater.")
|
||||
if any(
|
||||
not isinstance(transform, AttentionSpatialTransform)
|
||||
for transform in self.spatial_transforms
|
||||
):
|
||||
raise TypeError("Attention request spatial transforms are invalid.")
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class AttentionCapturePlan:
|
||||
"""Describe one coalesced sampler capture and its graph rewrite authority."""
|
||||
|
||||
sampler_node_id: str
|
||||
model_owner_node_id: str
|
||||
model_input_name: str
|
||||
model_link: GraphLink
|
||||
positive_link: GraphLink
|
||||
requests: tuple[AttentionRegionRequest, ...]
|
||||
prompt_text: str | None = None
|
||||
clip_link: GraphLink | None = None
|
||||
source_aspect: float | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require canonical unique requests and complete graph-edge identity."""
|
||||
|
||||
if not self.sampler_node_id or not self.model_owner_node_id:
|
||||
raise ValueError("Attention capture plan node ids cannot be empty.")
|
||||
if not self.model_input_name:
|
||||
raise ValueError("Attention capture plan model input cannot be empty.")
|
||||
if self.source_aspect is not None and self.source_aspect <= 0.0:
|
||||
raise ValueError("Attention capture source aspect must be positive.")
|
||||
request_ids = tuple(request.node_id for request in self.requests)
|
||||
if not request_ids or request_ids != tuple(sorted(set(request_ids))):
|
||||
raise ValueError("Attention capture requests must be unique and ordered.")
|
||||
|
||||
@property
|
||||
def capture_node_id(self) -> str:
|
||||
"""Return a collision-resistant deterministic injected node id."""
|
||||
|
||||
return f"__simple_syrup_attention_capture__{self.sampler_node_id}"
|
||||
@@ -0,0 +1,21 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Define rendered attention evidence before component and matte shaping."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
import torch
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class AttentionConceptEvidence:
|
||||
"""Hold one concept's alpha, support, and confidence evidence."""
|
||||
|
||||
label: str
|
||||
alpha: torch.Tensor
|
||||
support: torch.Tensor
|
||||
confidence: torch.Tensor
|
||||
@@ -0,0 +1,198 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Model prompt-token spans and compact captured attention observations."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
import torch
|
||||
|
||||
from .attention_spatial_transform import AttentionSpatialTransform
|
||||
from .regional_model_capabilities import RegionalModelFamily
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class AttentionTokenSpan:
|
||||
"""Bind a readable prompt occurrence to exact conditioning token positions."""
|
||||
|
||||
label: str
|
||||
occurrence: int
|
||||
token_indices: tuple[int, ...]
|
||||
head_token_indices: tuple[int, ...] = ()
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require canonical labels and ordered non-negative token indices."""
|
||||
|
||||
if not self.label or self.label != self.label.strip():
|
||||
raise ValueError("Attention token span label must be canonical.")
|
||||
if type(self.occurrence) is not int or self.occurrence < 1:
|
||||
raise ValueError("Attention token span occurrence must be positive.")
|
||||
if (
|
||||
not self.token_indices
|
||||
or self.token_indices != tuple(sorted(set(self.token_indices)))
|
||||
or self.token_indices[0] < 0
|
||||
):
|
||||
raise ValueError("Attention token indices must be ordered and unique.")
|
||||
if self.head_token_indices and (
|
||||
self.head_token_indices != tuple(sorted(set(self.head_token_indices)))
|
||||
or not set(self.head_token_indices).issubset(self.token_indices)
|
||||
):
|
||||
raise ValueError("Attention head tokens must be an ordered span subset.")
|
||||
|
||||
@property
|
||||
def semantic_head_indices(self) -> tuple[int, ...]:
|
||||
"""Return explicit noun-head positions or a safe final-token fallback."""
|
||||
|
||||
return self.head_token_indices or self.token_indices[-1:]
|
||||
|
||||
@property
|
||||
def display_label(self) -> str:
|
||||
"""Disambiguate repeated concepts while keeping first labels concise."""
|
||||
|
||||
return (
|
||||
self.label if self.occurrence == 1 else f"{self.label} #{self.occurrence}"
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class AttentionTokenCatalog:
|
||||
"""Hold readable prompt spans and conditioning sequence length."""
|
||||
|
||||
sequence_length: int
|
||||
spans: tuple[AttentionTokenSpan, ...]
|
||||
token_ids: tuple[object, ...]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require all spans to fit the captured conditioning sequence."""
|
||||
|
||||
if type(self.sequence_length) is not int or self.sequence_length < 1:
|
||||
raise ValueError("Attention token sequence length must be positive.")
|
||||
if len(self.token_ids) != self.sequence_length:
|
||||
raise ValueError("Attention token ids must match the sequence length.")
|
||||
if any(
|
||||
index >= self.sequence_length
|
||||
for span in self.spans
|
||||
for index in span.token_indices
|
||||
):
|
||||
raise ValueError("Attention token span exceeds its conditioning sequence.")
|
||||
|
||||
def exact_matches(self, query: str) -> tuple[AttentionTokenSpan, ...]:
|
||||
"""Return every prompt occurrence whose normalized label equals a query."""
|
||||
|
||||
normalized = _normalized_label(query)
|
||||
return tuple(
|
||||
span for span in self.spans if _normalized_label(span.label) == normalized
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class CapturedAttentionMap:
|
||||
"""Store one head-aggregated token map and its denoising observation identity."""
|
||||
|
||||
label: str
|
||||
values: torch.Tensor
|
||||
progress: float
|
||||
layer_key: str
|
||||
batch_index: int = 0
|
||||
confidence: float = 1.0
|
||||
spatial_height: int | None = None
|
||||
spatial_width: int | None = None
|
||||
spatial_transforms: tuple[AttentionSpatialTransform, ...] = ()
|
||||
concept_values: torch.Tensor | None = None
|
||||
uniform_probability: float = 0.0
|
||||
model_family: RegionalModelFamily = RegionalModelFamily.STANDARD_UNET
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require a finite CPU spatial vector and normalized progress."""
|
||||
|
||||
if not self.label:
|
||||
raise ValueError("Captured attention map label cannot be empty.")
|
||||
if (
|
||||
not isinstance(self.values, torch.Tensor)
|
||||
or self.values.device.type != "cpu"
|
||||
or self.values.ndim != 1
|
||||
or self.values.numel() < 1
|
||||
or not self.values.is_floating_point()
|
||||
or not torch.isfinite(self.values).all().item()
|
||||
):
|
||||
raise ValueError("Captured attention values must be a finite CPU vector.")
|
||||
if not 0.0 <= self.progress <= 1.0:
|
||||
raise ValueError("Captured attention progress must be within 0..1.")
|
||||
if not self.layer_key:
|
||||
raise ValueError("Captured attention layer key cannot be empty.")
|
||||
if type(self.batch_index) is not int or self.batch_index < 0:
|
||||
raise ValueError("Captured attention batch index must be non-negative.")
|
||||
if not 0.0 <= self.confidence <= 1.0:
|
||||
raise ValueError("Captured attention confidence must be within 0..1.")
|
||||
if (self.spatial_height is None) != (self.spatial_width is None):
|
||||
raise ValueError("Captured attention geometry must be complete or absent.")
|
||||
if self.spatial_height is not None and (
|
||||
type(self.spatial_height) is not int
|
||||
or self.spatial_height < 1
|
||||
or type(self.spatial_width) is not int
|
||||
or self.spatial_width < 1
|
||||
or self.spatial_height * self.spatial_width != int(self.values.numel())
|
||||
):
|
||||
raise ValueError("Captured attention geometry must match its values.")
|
||||
if any(
|
||||
not isinstance(transform, AttentionSpatialTransform)
|
||||
for transform in self.spatial_transforms
|
||||
):
|
||||
raise TypeError("Captured attention spatial transforms are invalid.")
|
||||
if self.concept_values is not None and (
|
||||
not isinstance(self.concept_values, torch.Tensor)
|
||||
or self.concept_values.device.type != "cpu"
|
||||
or self.concept_values.shape != self.values.shape
|
||||
or not self.concept_values.is_floating_point()
|
||||
or not torch.isfinite(self.concept_values).all().item()
|
||||
):
|
||||
raise ValueError(
|
||||
"Captured concept evidence must match its finite CPU attention map."
|
||||
)
|
||||
if (
|
||||
isinstance(self.uniform_probability, bool)
|
||||
or not isinstance(self.uniform_probability, int | float)
|
||||
or not 0.0 <= float(self.uniform_probability) <= 1.0
|
||||
):
|
||||
raise ValueError("Captured uniform probability must be within 0..1.")
|
||||
if not isinstance(self.model_family, RegionalModelFamily):
|
||||
raise TypeError("Captured attention model family has an invalid type.")
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class OpenVocabularyContext:
|
||||
"""Hold one query's encoded SDXL context and semantic token positions."""
|
||||
|
||||
label: str
|
||||
values: torch.Tensor
|
||||
token_indices: tuple[int, ...]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require one finite CPU context with valid unique token positions."""
|
||||
|
||||
if not self.label.strip() or self.label != self.label.strip():
|
||||
raise ValueError("Open-vocabulary labels must be canonical strings.")
|
||||
if self.values.ndim != 3 or int(self.values.shape[0]) != 1:
|
||||
raise ValueError("Open-vocabulary context must have shape 1xTxC.")
|
||||
if self.values.device.type != "cpu" or not torch.isfinite(self.values).all():
|
||||
raise ValueError("Open-vocabulary context must be finite CPU storage.")
|
||||
if not self.token_indices or self.token_indices != tuple(
|
||||
sorted(set(self.token_indices))
|
||||
):
|
||||
raise ValueError(
|
||||
"Open-vocabulary token positions must be unique and ordered."
|
||||
)
|
||||
if any(
|
||||
index < 0 or index >= int(self.values.shape[1])
|
||||
for index in self.token_indices
|
||||
):
|
||||
raise ValueError("Open-vocabulary token positions exceed their context.")
|
||||
|
||||
|
||||
def _normalized_label(value: str) -> str:
|
||||
"""Normalize human prompt labels without changing tokenizer semantics."""
|
||||
|
||||
return " ".join(value.casefold().replace("_", " ").split())
|
||||
@@ -0,0 +1,78 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Model ordered sampling stages along one spatial provenance path."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from .attention_spatial_transform import AttentionSpatialTransform
|
||||
from .graph_provenance import GraphLink
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class AttentionSamplerStage:
|
||||
"""Describe one sampler's patch and conditioning authority."""
|
||||
|
||||
sampler_node_id: str
|
||||
model_owner_node_id: str
|
||||
model_link: GraphLink
|
||||
positive_link: GraphLink
|
||||
upstream_link: GraphLink
|
||||
upstream_kind: str
|
||||
forward_transforms: tuple[AttentionSpatialTransform, ...] = ()
|
||||
source_aspect: float | None = None
|
||||
capture_supported: bool = True
|
||||
unsupported_reason: str | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require unsupported stages to explain why capture cannot be projected."""
|
||||
|
||||
if self.capture_supported and self.unsupported_reason is not None:
|
||||
raise ValueError("Supported attention sampler stages cannot have a reason.")
|
||||
if not self.capture_supported and not self.unsupported_reason:
|
||||
raise ValueError("Unsupported attention sampler stages require a reason.")
|
||||
if self.source_aspect is not None and self.source_aspect <= 0.0:
|
||||
raise ValueError("Attention sampler source aspect must be positive.")
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class AttentionSamplerSelection:
|
||||
"""Bind a selected stage to its one-based chronological position."""
|
||||
|
||||
stage: AttentionSamplerStage
|
||||
stage_number: int
|
||||
stage_count: int
|
||||
was_clamped: bool
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class AttentionSamplerLineage:
|
||||
"""Hold sampling stages ordered from oldest to direct provenance."""
|
||||
|
||||
stages: tuple[AttentionSamplerStage, ...]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require at least one uniquely identified stage."""
|
||||
|
||||
identities = tuple(stage.sampler_node_id for stage in self.stages)
|
||||
if not identities or len(set(identities)) != len(identities):
|
||||
raise ValueError("Attention sampler lineage must contain unique stages.")
|
||||
|
||||
def select(self, requested_stage: int) -> AttentionSamplerSelection:
|
||||
"""Resolve one-based selection with 0/-1 aliases for direct provenance."""
|
||||
|
||||
if type(requested_stage) is not int or requested_stage < -1:
|
||||
raise ValueError("Attention sampler stage must be -1 or greater.")
|
||||
count = len(self.stages)
|
||||
if requested_stage in (-1, 0):
|
||||
return AttentionSamplerSelection(self.stages[-1], count, count, False)
|
||||
selected_number = min(requested_stage, count)
|
||||
return AttentionSamplerSelection(
|
||||
self.stages[selected_number - 1],
|
||||
selected_number,
|
||||
count,
|
||||
requested_stage > count,
|
||||
)
|
||||
@@ -0,0 +1,70 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Describe graph-visible full-canvas transformations for attention masks."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from enum import StrEnum
|
||||
|
||||
_ANCHORS = frozenset(
|
||||
{
|
||||
"center",
|
||||
"top-left",
|
||||
"top",
|
||||
"top-right",
|
||||
"left",
|
||||
"right",
|
||||
"bottom-left",
|
||||
"bottom",
|
||||
"bottom-right",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
class AttentionSpatialTransformKind(StrEnum):
|
||||
"""Identify a supported mask-coordinate transformation."""
|
||||
|
||||
RESIZE = "resize"
|
||||
FIT_RESIZE = "fit_resize"
|
||||
SCALE = "scale"
|
||||
COVER_CROP = "cover_crop"
|
||||
FIT_PAD = "fit_pad"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class AttentionSpatialTransform:
|
||||
"""Hold validated resize, crop, or pad parameters from one graph node."""
|
||||
|
||||
kind: AttentionSpatialTransformKind
|
||||
width: int | None = None
|
||||
height: int | None = None
|
||||
scale: float | None = None
|
||||
anchor: str = "center"
|
||||
divisible_by: int = 1
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require complete parameters for the selected transformation kind."""
|
||||
|
||||
if self.anchor not in _ANCHORS:
|
||||
raise ValueError("Attention spatial transform anchor is invalid.")
|
||||
if self.kind is AttentionSpatialTransformKind.SCALE:
|
||||
if self.scale is None or self.scale <= 0.0:
|
||||
raise ValueError("Attention scale transform requires a positive scale.")
|
||||
if self.width is not None or self.height is not None:
|
||||
raise ValueError("Attention scale transform cannot contain a size.")
|
||||
if self.divisible_by != 1:
|
||||
raise ValueError("Attention scale transform cannot set divisibility.")
|
||||
return
|
||||
if (
|
||||
type(self.width) is not int
|
||||
or self.width < 1
|
||||
or type(self.height) is not int
|
||||
or self.height < 1
|
||||
or self.scale is not None
|
||||
):
|
||||
raise ValueError("Attention spatial transform requires a positive size.")
|
||||
if type(self.divisible_by) is not int or self.divisible_by < 1:
|
||||
raise ValueError("Attention spatial divisibility must be positive.")
|
||||
@@ -6,7 +6,6 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
from typing import Any, TypeAlias
|
||||
|
||||
@@ -57,15 +56,6 @@ def batch_conditioning(
|
||||
return ConditioningBatch(tuple(entries))
|
||||
|
||||
|
||||
def split_prompt_batch(text: str, separator: str = "[SEP]") -> tuple[str, ...]:
|
||||
"""Split prompt text into ordered chunks using a configurable separator."""
|
||||
|
||||
if separator == "":
|
||||
raise ValueError("separator must not be empty.")
|
||||
pattern = rf"\s*{re.escape(separator)}\s*"
|
||||
return tuple(re.split(pattern, text))
|
||||
|
||||
|
||||
def select_conditioning(
|
||||
conditioning: Conditioning | ConditioningBatch,
|
||||
index: int,
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Own immutable authored and model-converted conditioning schedule bounds."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ConditioningScheduleRange:
|
||||
"""Retain optional authored percentages and converted sigma boundaries."""
|
||||
|
||||
start_percent: float | None
|
||||
end_percent: float | None
|
||||
timestep_start: float | None
|
||||
timestep_end: float | None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Validate exact optional bounds without inventing absent metadata."""
|
||||
|
||||
start_percent = _optional_finite_float(
|
||||
self.start_percent,
|
||||
name="start_percent",
|
||||
)
|
||||
end_percent = _optional_finite_float(
|
||||
self.end_percent,
|
||||
name="end_percent",
|
||||
)
|
||||
timestep_start = _optional_finite_float(
|
||||
self.timestep_start,
|
||||
name="timestep_start",
|
||||
)
|
||||
timestep_end = _optional_finite_float(
|
||||
self.timestep_end,
|
||||
name="timestep_end",
|
||||
)
|
||||
for name, value in (
|
||||
("start_percent", start_percent),
|
||||
("end_percent", end_percent),
|
||||
):
|
||||
if value is not None and not 0.0 <= value <= 1.0:
|
||||
raise ValueError(f"Conditioning {name} must be in [0, 1].")
|
||||
effective_start = 0.0 if start_percent is None else start_percent
|
||||
effective_end = 1.0 if end_percent is None else end_percent
|
||||
if effective_start > effective_end:
|
||||
raise ValueError("Conditioning start_percent must not exceed end_percent.")
|
||||
if (
|
||||
timestep_start is not None
|
||||
and timestep_end is not None
|
||||
and timestep_start < timestep_end
|
||||
):
|
||||
raise ValueError(
|
||||
"Conditioning timestep_start must not be below timestep_end."
|
||||
)
|
||||
object.__setattr__(self, "start_percent", start_percent)
|
||||
object.__setattr__(self, "end_percent", end_percent)
|
||||
object.__setattr__(self, "timestep_start", timestep_start)
|
||||
object.__setattr__(self, "timestep_end", timestep_end)
|
||||
|
||||
@property
|
||||
def is_time_invariant(self) -> bool:
|
||||
"""Report whether this entry remains admitted for the whole trajectory."""
|
||||
|
||||
if self.start_percent is None and self.timestep_start is not None:
|
||||
return False
|
||||
if self.end_percent is None and self.timestep_end is not None:
|
||||
return False
|
||||
effective_start = 0.0 if self.start_percent is None else self.start_percent
|
||||
effective_end = 1.0 if self.end_percent is None else self.end_percent
|
||||
return effective_start == 0.0 and effective_end == 1.0
|
||||
|
||||
|
||||
def _optional_finite_float(value: object, *, name: str) -> float | None:
|
||||
"""Normalize one optional real boundary without accepting booleans."""
|
||||
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, bool) or not isinstance(value, int | float):
|
||||
raise TypeError(f"Conditioning {name} must be a real number or None.")
|
||||
normalized = float(value)
|
||||
if not math.isfinite(normalized):
|
||||
raise ValueError(f"Conditioning {name} must be finite.")
|
||||
return normalized
|
||||
|
||||
|
||||
UNBOUNDED_CONDITIONING_SCHEDULE = ConditioningScheduleRange(None, None, None, None)
|
||||
@@ -0,0 +1,49 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Own installed-Comfy-equivalent conditioning schedule admission."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
|
||||
from .conditioning_schedule import ConditioningScheduleRange
|
||||
|
||||
|
||||
class ConditioningScheduleSelectionPolicy:
|
||||
"""Match installed Comfy's inclusive converted-sigma admission policy."""
|
||||
|
||||
@staticmethod
|
||||
def is_active(
|
||||
schedule: ConditioningScheduleRange,
|
||||
*,
|
||||
sigma: float,
|
||||
) -> bool:
|
||||
"""Return installed Comfy's inclusive start/end decision."""
|
||||
|
||||
if not isinstance(schedule, ConditioningScheduleRange):
|
||||
raise TypeError("Conditioning selection requires a schedule range.")
|
||||
current_sigma = normalize_conditioning_sigma(sigma)
|
||||
if (
|
||||
schedule.timestep_start is not None
|
||||
and current_sigma > schedule.timestep_start
|
||||
):
|
||||
return False
|
||||
return not (
|
||||
schedule.timestep_end is not None and current_sigma < schedule.timestep_end
|
||||
)
|
||||
|
||||
|
||||
def normalize_conditioning_sigma(value: object) -> float:
|
||||
"""Normalize one finite real current sigma without accepting booleans."""
|
||||
|
||||
if isinstance(value, bool) or not isinstance(value, int | float):
|
||||
raise TypeError("Conditioning selection sigma must be a real number.")
|
||||
sigma = float(value)
|
||||
if not math.isfinite(sigma):
|
||||
raise ValueError("Conditioning selection sigma must be finite.")
|
||||
return sigma
|
||||
|
||||
|
||||
CONDITIONING_SCHEDULE_SELECTION_POLICY = ConditioningScheduleSelectionPolicy()
|
||||
@@ -0,0 +1,170 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Project evaluated latent context windows into lazily materialized SEGS."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from collections.abc import Iterable, Iterator, Sequence
|
||||
from threading import Lock
|
||||
from typing import TypeAlias, overload
|
||||
|
||||
import torch
|
||||
|
||||
from .segs import BoundingBox, CropRegion, Segment, SegsHeader
|
||||
from .tiled_diffusion import TiledDiffusionPlan
|
||||
|
||||
ContextSegs: TypeAlias = tuple[SegsHeader, "ContextSegmentSequence"]
|
||||
|
||||
|
||||
class ContextSegmentSequence(Sequence[Segment]):
|
||||
"""Delay large rectangular mask allocation until a SEGS consumer reads it."""
|
||||
|
||||
def __init__(self, windows: Iterable[CropRegion]) -> None:
|
||||
"""Store deterministic context windows without allocating their masks."""
|
||||
|
||||
self._windows = tuple(windows)
|
||||
self._materialized: tuple[Segment, ...] | None = None
|
||||
self._lock = Lock()
|
||||
|
||||
@property
|
||||
def windows(self) -> tuple[CropRegion, ...]:
|
||||
"""Return immutable projected windows without forcing mask allocation."""
|
||||
|
||||
return self._windows
|
||||
|
||||
@property
|
||||
def is_materialized(self) -> bool:
|
||||
"""Report whether a downstream consumer has requested concrete segments."""
|
||||
|
||||
return self._materialized is not None
|
||||
|
||||
def __len__(self) -> int:
|
||||
"""Return the number of contexts without materializing masks."""
|
||||
|
||||
return len(self._windows)
|
||||
|
||||
@overload
|
||||
def __getitem__(self, index: int) -> Segment: ...
|
||||
|
||||
@overload
|
||||
def __getitem__(self, index: slice) -> tuple[Segment, ...]: ...
|
||||
|
||||
def __getitem__(self, index: int | slice) -> Segment | tuple[Segment, ...]:
|
||||
"""Materialize masks and return one context or a context slice."""
|
||||
|
||||
return self._segments()[index]
|
||||
|
||||
def __iter__(self) -> Iterator[Segment]:
|
||||
"""Materialize masks once and iterate contexts in evaluation order."""
|
||||
|
||||
return iter(self._segments())
|
||||
|
||||
def _segments(self) -> tuple[Segment, ...]:
|
||||
"""Create full rectangular masks once on first downstream access."""
|
||||
|
||||
if self._materialized is not None:
|
||||
return self._materialized
|
||||
with self._lock:
|
||||
if self._materialized is None:
|
||||
self._materialized = tuple(
|
||||
_segment_from_window(window, index)
|
||||
for index, window in enumerate(self._windows, start=1)
|
||||
)
|
||||
return self._materialized
|
||||
|
||||
|
||||
def context_segs_from_tile_plan(
|
||||
plan: TiledDiffusionPlan,
|
||||
*,
|
||||
image_height: int,
|
||||
image_width: int,
|
||||
) -> ContextSegs:
|
||||
"""Return lazy rectangular SEGS for every non-global evaluated tile."""
|
||||
|
||||
_validate_image_dimensions(image_height, image_width)
|
||||
windows = tuple(
|
||||
_project_tile(
|
||||
x=tile.x,
|
||||
y=tile.y,
|
||||
width=tile.width,
|
||||
height=tile.height,
|
||||
latent_width=plan.latent_width,
|
||||
latent_height=plan.latent_height,
|
||||
image_width=image_width,
|
||||
image_height=image_height,
|
||||
)
|
||||
for tile in plan.tiles
|
||||
)
|
||||
return (image_height, image_width), ContextSegmentSequence(windows)
|
||||
|
||||
|
||||
def merge_context_segs(values: Iterable[ContextSegs]) -> ContextSegs:
|
||||
"""Combine batched context windows without duplicates or mask allocation."""
|
||||
|
||||
items = tuple(values)
|
||||
if not items:
|
||||
raise ValueError("Context SEGS requires at least one context plan.")
|
||||
header = items[0][0]
|
||||
if any(item[0] != header for item in items[1:]):
|
||||
raise ValueError(
|
||||
"Context SEGS requires batched image dimensions to match exactly."
|
||||
)
|
||||
windows = list(items[0][1].windows)
|
||||
seen = set(windows)
|
||||
for _header, segments in items[1:]:
|
||||
for window in segments.windows:
|
||||
if window not in seen:
|
||||
windows.append(window)
|
||||
seen.add(window)
|
||||
return header, ContextSegmentSequence(windows)
|
||||
|
||||
|
||||
def _project_tile(
|
||||
*,
|
||||
x: int,
|
||||
y: int,
|
||||
width: int,
|
||||
height: int,
|
||||
latent_width: int,
|
||||
latent_height: int,
|
||||
image_width: int,
|
||||
image_height: int,
|
||||
) -> CropRegion:
|
||||
"""Project one latent rectangle outward into integer image coordinates."""
|
||||
|
||||
left = math.floor(x * image_width / latent_width)
|
||||
top = math.floor(y * image_height / latent_height)
|
||||
right = math.ceil((x + width) * image_width / latent_width)
|
||||
bottom = math.ceil((y + height) * image_height / latent_height)
|
||||
return CropRegion(
|
||||
max(0, min(image_width - 1, left)),
|
||||
max(0, min(image_height - 1, top)),
|
||||
max(1, min(image_width, right)),
|
||||
max(1, min(image_height, bottom)),
|
||||
)
|
||||
|
||||
|
||||
def _segment_from_window(window: CropRegion, index: int) -> Segment:
|
||||
"""Materialize one Impact-compatible full rectangular context mask."""
|
||||
|
||||
return Segment(
|
||||
cropped_image=None,
|
||||
cropped_mask=torch.ones(
|
||||
(window.height, window.width),
|
||||
dtype=torch.uint8,
|
||||
),
|
||||
confidence=1.0,
|
||||
crop_region=window,
|
||||
bbox=BoundingBox(*window),
|
||||
label=f"context_{index:03d}",
|
||||
)
|
||||
|
||||
|
||||
def _validate_image_dimensions(height: int, width: int) -> None:
|
||||
"""Reject image geometry that cannot host projected context rectangles."""
|
||||
|
||||
if height < 1 or width < 1:
|
||||
raise ValueError("Context SEGS image dimensions must be positive.")
|
||||
@@ -0,0 +1,150 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Plan one bounded global context and one authoritative tiled context set."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
import torch
|
||||
|
||||
from .regional_tiled_diffusion import build_region_constrained_tiled_diffusion_plan
|
||||
from .segs import NativeSegs
|
||||
from .segs_tiled_diffusion import build_segs_guided_tiled_diffusion_plan
|
||||
from .spatial_views import SpatialView, SpatialViewKind
|
||||
from .tiled_diffusion import TiledDiffusionPlan, build_tiled_diffusion_plan
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ContextualDiffusionControls:
|
||||
"""Validate workflow controls for contextual diffusion sampling."""
|
||||
|
||||
latent_context_size: int
|
||||
latent_context_overlap: int
|
||||
latent_context_batch_size: int
|
||||
global_weight: float
|
||||
global_steps: int
|
||||
global_decay: float
|
||||
|
||||
def validate(self) -> None:
|
||||
"""Reject controls that cannot produce a stable bounded context plan."""
|
||||
|
||||
if self.latent_context_size < 16:
|
||||
raise ValueError("latent_context_size must be at least 16 latent pixels.")
|
||||
if not 0 <= self.latent_context_overlap < self.latent_context_size:
|
||||
raise ValueError(
|
||||
"latent_context_overlap must be non-negative and smaller than "
|
||||
"latent_context_size."
|
||||
)
|
||||
if self.latent_context_batch_size < 1:
|
||||
raise ValueError("latent_context_batch_size must be at least 1.")
|
||||
if not 0.0 <= self.global_weight <= 2.0:
|
||||
raise ValueError("global_weight must be between 0 and 2.")
|
||||
if self.global_steps < 0:
|
||||
raise ValueError("global_steps must be non-negative.")
|
||||
if not 0.0 <= self.global_decay <= 1.0:
|
||||
raise ValueError("global_decay must be between 0 and 1.")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ContextualDiffusionPlan:
|
||||
"""Own the global context and sole tiled plan for one latent canvas."""
|
||||
|
||||
latent_width: int
|
||||
latent_height: int
|
||||
global_view: SpatialView
|
||||
tile_plan: TiledDiffusionPlan
|
||||
|
||||
|
||||
def build_contextual_diffusion_plan(
|
||||
*,
|
||||
latent_width: int,
|
||||
latent_height: int,
|
||||
controls: ContextualDiffusionControls,
|
||||
segs: NativeSegs | None,
|
||||
region_masks: torch.Tensor | None = None,
|
||||
) -> ContextualDiffusionPlan:
|
||||
"""Return a global context plus the regular or SEGS-guided context plan."""
|
||||
|
||||
controls.validate()
|
||||
global_width, global_height = fit_context_shape(
|
||||
latent_width,
|
||||
latent_height,
|
||||
controls.latent_context_size,
|
||||
)
|
||||
global_view = SpatialView(
|
||||
kind=SpatialViewKind.CONTEXTUAL_GLOBAL,
|
||||
source_x=0,
|
||||
source_y=0,
|
||||
source_width=latent_width,
|
||||
source_height=latent_height,
|
||||
model_width=global_width,
|
||||
model_height=global_height,
|
||||
)
|
||||
if region_masks is not None:
|
||||
tile_plan = build_region_constrained_tiled_diffusion_plan(
|
||||
region_masks=region_masks,
|
||||
segs=segs,
|
||||
latent_width=latent_width,
|
||||
latent_height=latent_height,
|
||||
tile_width=controls.latent_context_size,
|
||||
tile_height=controls.latent_context_size,
|
||||
overlap=controls.latent_context_overlap,
|
||||
tile_batch_size=controls.latent_context_batch_size,
|
||||
)
|
||||
elif segs is not None:
|
||||
tile_plan = build_segs_guided_tiled_diffusion_plan(
|
||||
segs=segs,
|
||||
latent_width=latent_width,
|
||||
latent_height=latent_height,
|
||||
tile_width=controls.latent_context_size,
|
||||
tile_height=controls.latent_context_size,
|
||||
overlap=controls.latent_context_overlap,
|
||||
tile_batch_size=controls.latent_context_batch_size,
|
||||
)
|
||||
else:
|
||||
tile_plan = build_tiled_diffusion_plan(
|
||||
latent_width=latent_width,
|
||||
latent_height=latent_height,
|
||||
tile_width=controls.latent_context_size,
|
||||
tile_height=controls.latent_context_size,
|
||||
overlap=controls.latent_context_overlap,
|
||||
tile_batch_size=controls.latent_context_batch_size,
|
||||
)
|
||||
return ContextualDiffusionPlan(
|
||||
latent_width=latent_width,
|
||||
latent_height=latent_height,
|
||||
global_view=global_view,
|
||||
tile_plan=tile_plan,
|
||||
)
|
||||
|
||||
|
||||
def fit_context_shape(width: int, height: int, max_size: int) -> tuple[int, int]:
|
||||
"""Fit a rectangle inside one maximum latent dimension without boxing it."""
|
||||
|
||||
if width < 1 or height < 1:
|
||||
raise ValueError("Context source dimensions must be positive.")
|
||||
if max_size < 1:
|
||||
raise ValueError("Context maximum size must be positive.")
|
||||
if max(width, height) <= max_size:
|
||||
return width, height
|
||||
scale = max_size / max(width, height)
|
||||
fitted_width = max(2, round(width * scale))
|
||||
fitted_height = max(2, round(height * scale))
|
||||
return _even_at_most(fitted_width, max_size), _even_at_most(
|
||||
fitted_height,
|
||||
max_size,
|
||||
)
|
||||
|
||||
|
||||
def _even_at_most(value: int, maximum: int) -> int:
|
||||
"""Return a positive even model-context dimension within its maximum."""
|
||||
|
||||
bounded = min(maximum, max(2, value))
|
||||
if bounded % 2 == 0:
|
||||
return bounded
|
||||
if bounded == maximum:
|
||||
return max(2, bounded - 1)
|
||||
return bounded + 1
|
||||
@@ -0,0 +1,58 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Classify FLUX model generations from ComfyUI's structural model metadata."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from enum import StrEnum
|
||||
|
||||
|
||||
class FluxGeneration(StrEnum):
|
||||
"""Identify the conditioning generation used by a FLUX diffusion model."""
|
||||
|
||||
FLUX = "flux"
|
||||
FLUX2 = "flux2"
|
||||
|
||||
|
||||
class Flux2TextEncoderProfile(StrEnum):
|
||||
"""Identify the text-encoder family required by a FLUX.2 architecture."""
|
||||
|
||||
DEV = "dev"
|
||||
KLEIN_4B = "klein_4b"
|
||||
KLEIN_9B = "klein_9b"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class FluxModelProfile:
|
||||
"""Describe a structurally detected FLUX generation and encoder profile."""
|
||||
|
||||
generation: FluxGeneration
|
||||
flux2_text_encoder: Flux2TextEncoderProfile | None = None
|
||||
|
||||
|
||||
FLUX2_CONTEXT_DIMENSIONS: dict[int, Flux2TextEncoderProfile] = {
|
||||
15_360: Flux2TextEncoderProfile.DEV,
|
||||
7_680: Flux2TextEncoderProfile.KLEIN_4B,
|
||||
12_288: Flux2TextEncoderProfile.KLEIN_9B,
|
||||
}
|
||||
|
||||
|
||||
def classify_flux_profile(
|
||||
image_model: str | None,
|
||||
context_input_dimension: int | None,
|
||||
) -> FluxModelProfile | None:
|
||||
"""Return the FLUX profile represented by ComfyUI's detected dimensions."""
|
||||
|
||||
if image_model == FluxGeneration.FLUX:
|
||||
return FluxModelProfile(FluxGeneration.FLUX)
|
||||
if image_model != FluxGeneration.FLUX2:
|
||||
return None
|
||||
encoder_profile = (
|
||||
FLUX2_CONTEXT_DIMENSIONS.get(context_input_dimension)
|
||||
if context_input_dimension is not None
|
||||
else None
|
||||
)
|
||||
return FluxModelProfile(FluxGeneration.FLUX2, encoder_profile)
|
||||
@@ -0,0 +1,42 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Select decayed reduced-global authority from denoising timesteps."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import torch
|
||||
|
||||
|
||||
class GlobalContextSchedule:
|
||||
"""Limit whole-image authority to an initial denoising-step fraction."""
|
||||
|
||||
def __init__(
|
||||
self, *, sigmas: torch.Tensor, active_steps: int, decay: float
|
||||
) -> None:
|
||||
"""Capture model-evaluation sigmas and the active initial step count."""
|
||||
|
||||
step_sigmas = sigmas.detach().to(device="cpu", dtype=torch.float64).flatten()
|
||||
if step_sigmas.numel() < 2:
|
||||
raise ValueError(
|
||||
"Contextual Diffusion requires at least one denoising step."
|
||||
)
|
||||
self._step_sigmas = step_sigmas[:-1]
|
||||
self._active_steps = min(len(self._step_sigmas), max(0, active_steps))
|
||||
self._decay = decay
|
||||
|
||||
def scale_for(self, timestep: object) -> float:
|
||||
"""Return the decayed global scale for the nearest scheduled step."""
|
||||
|
||||
if self._active_steps == 0:
|
||||
return 0.0
|
||||
if not isinstance(timestep, torch.Tensor) or timestep.numel() == 0:
|
||||
raise ValueError(
|
||||
"Contextual Diffusion timestep must be a non-empty tensor."
|
||||
)
|
||||
sigma = timestep.detach().flatten()[0].to(device="cpu", dtype=torch.float64)
|
||||
step_index = int(torch.argmin(torch.abs(self._step_sigmas - sigma)).item())
|
||||
if step_index >= self._active_steps:
|
||||
return 0.0
|
||||
return self._decay**step_index
|
||||
@@ -0,0 +1,82 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Define model-independent checkpoint quantization profile contracts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from enum import StrEnum
|
||||
from typing import Protocol
|
||||
|
||||
|
||||
class QuantizationFormat(StrEnum):
|
||||
"""Identify one reusable ComfyUI tensor quantization format."""
|
||||
|
||||
FP8_E4M3 = "float8_e4m3fn"
|
||||
FP8_E5M2 = "float8_e5m2"
|
||||
NVFP4 = "nvfp4"
|
||||
MXFP8 = "mxfp8"
|
||||
|
||||
@property
|
||||
def label(self) -> str:
|
||||
"""Return the concise format label used in diagnostics."""
|
||||
|
||||
return {
|
||||
QuantizationFormat.FP8_E4M3: "FP8 E4M3",
|
||||
QuantizationFormat.FP8_E5M2: "FP8 E5M2",
|
||||
QuantizationFormat.NVFP4: "NVFP4",
|
||||
QuantizationFormat.MXFP8: "MXFP8",
|
||||
}[self]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class QuantizationProfile:
|
||||
"""Describe one workflow-facing, versioned per-tensor policy profile."""
|
||||
|
||||
profile_id: str
|
||||
label: str
|
||||
version: int
|
||||
required_formats: frozenset[QuantizationFormat]
|
||||
|
||||
@property
|
||||
def is_original(self) -> bool:
|
||||
"""Return whether this profile loads the source checkpoint unchanged."""
|
||||
|
||||
return not self.required_formats
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class TensorDescriptor:
|
||||
"""Describe a checkpoint tensor without coupling policy to PyTorch."""
|
||||
|
||||
name: str
|
||||
shape: tuple[int, ...]
|
||||
dtype_name: str
|
||||
|
||||
|
||||
class ModelQuantizationRecipe(Protocol):
|
||||
"""Assign model-specific per-tensor formats for named profiles."""
|
||||
|
||||
@property
|
||||
def model_family(self) -> str:
|
||||
"""Return the stable family identifier used in cache identity."""
|
||||
|
||||
@property
|
||||
def version(self) -> int:
|
||||
"""Return the recipe version used in cache invalidation."""
|
||||
|
||||
@property
|
||||
def profiles(self) -> tuple[QuantizationProfile, ...]:
|
||||
"""Return deterministic workflow profiles owned by this recipe."""
|
||||
|
||||
def profile_from_selection(self, selection: str) -> QuantizationProfile:
|
||||
"""Parse a workflow selection into a recipe-owned profile."""
|
||||
|
||||
def policy_for(
|
||||
self,
|
||||
tensor: TensorDescriptor,
|
||||
profile: QuantizationProfile,
|
||||
) -> QuantizationFormat | None:
|
||||
"""Return the tensor format or ``None`` to preserve source precision."""
|
||||
@@ -0,0 +1,89 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Detect effective negative weights in Comfy-style prompt emphasis."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class _WeightedPromptSegment:
|
||||
"""Retain one parsed prompt fragment and its effective scalar weight."""
|
||||
|
||||
text: str
|
||||
weight: float
|
||||
|
||||
|
||||
def contains_negative_prompt_weight(text: str) -> bool:
|
||||
"""Return whether valid nested emphasis gives any prompt text a negative weight."""
|
||||
|
||||
if not isinstance(text, str):
|
||||
raise TypeError("Negative prompt-weight detection requires text.")
|
||||
escaped = text.replace(r"\)", "\0\1").replace(r"\(", "\0\2")
|
||||
return any(
|
||||
segment.text and segment.weight < 0.0
|
||||
for segment in _weighted_segments(escaped, 1.0)
|
||||
)
|
||||
|
||||
|
||||
def _weighted_segments(
|
||||
text: str,
|
||||
current_weight: float,
|
||||
) -> tuple[_WeightedPromptSegment, ...]:
|
||||
"""Parse emphasis with the same nesting and final-colon rules as ComfyUI."""
|
||||
|
||||
parsed: list[_WeightedPromptSegment] = []
|
||||
for item in _parenthesized_items(text):
|
||||
weight = current_weight
|
||||
if len(item) >= 2 and item[0] == "(" and item[-1] == ")":
|
||||
inner = item[1:-1]
|
||||
delimiter = inner.rfind(":")
|
||||
weight *= 1.1
|
||||
if delimiter > 0:
|
||||
try:
|
||||
weight = float(inner[delimiter + 1 :])
|
||||
except ValueError:
|
||||
pass
|
||||
else:
|
||||
inner = inner[:delimiter]
|
||||
parsed.extend(_weighted_segments(inner, weight))
|
||||
continue
|
||||
parsed.append(
|
||||
_WeightedPromptSegment(
|
||||
item.replace("\0\1", ")").replace("\0\2", "("),
|
||||
current_weight,
|
||||
)
|
||||
)
|
||||
return tuple(parsed)
|
||||
|
||||
|
||||
def _parenthesized_items(text: str) -> tuple[str, ...]:
|
||||
"""Split top-level parenthesized regions while preserving malformed input."""
|
||||
|
||||
result: list[str] = []
|
||||
current = ""
|
||||
nesting = 0
|
||||
for character in text:
|
||||
if character == "(":
|
||||
if nesting == 0:
|
||||
if current:
|
||||
result.append(current)
|
||||
current = "("
|
||||
else:
|
||||
current += character
|
||||
nesting += 1
|
||||
elif character == ")":
|
||||
nesting -= 1
|
||||
if nesting == 0:
|
||||
result.append(f"{current})")
|
||||
current = ""
|
||||
else:
|
||||
current += character
|
||||
else:
|
||||
current += character
|
||||
if current:
|
||||
result.append(current)
|
||||
return tuple(result)
|
||||
@@ -0,0 +1,55 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Domain model for validated, duplicate-preserving file selections."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
from collections.abc import Callable, Sequence
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class OrderedFileSelection:
|
||||
"""Represent non-empty workflow file positions without deduplication."""
|
||||
|
||||
paths: tuple[str, ...]
|
||||
|
||||
@classmethod
|
||||
def require(
|
||||
cls,
|
||||
files: Sequence[str],
|
||||
*,
|
||||
node_name: str,
|
||||
item_name: str,
|
||||
) -> OrderedFileSelection:
|
||||
"""Validate scalar-or-sequence workflow state into ordered paths."""
|
||||
|
||||
ordered = (files,) if isinstance(files, str) else tuple(files)
|
||||
if not ordered:
|
||||
raise ValueError(f"{node_name} requires at least one {item_name} file.")
|
||||
if any(not isinstance(path, str) or not path for path in ordered):
|
||||
raise TypeError(f"{node_name} {item_name} files must be non-empty strings.")
|
||||
return cls(paths=ordered)
|
||||
|
||||
def fingerprint(
|
||||
self,
|
||||
file_fingerprint: Callable[[str], str],
|
||||
*,
|
||||
context: Sequence[str] = (),
|
||||
) -> str:
|
||||
"""Hash context, every path position, and each file's content digest."""
|
||||
|
||||
digest = hashlib.sha256()
|
||||
for value in context:
|
||||
encoded = value.encode("utf-8")
|
||||
digest.update(len(encoded).to_bytes(8, "big"))
|
||||
digest.update(encoded)
|
||||
for path in self.paths:
|
||||
encoded = path.encode("utf-8")
|
||||
digest.update(len(encoded).to_bytes(8, "big"))
|
||||
digest.update(encoded)
|
||||
digest.update(file_fingerprint(path).encode("ascii"))
|
||||
return digest.hexdigest()
|
||||
@@ -0,0 +1,190 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Own immutable processed regional Attention Coupling runtime plans."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
from uuid import UUID
|
||||
|
||||
import torch
|
||||
|
||||
from .conditioning_schedule import ConditioningScheduleRange
|
||||
from .regional_attention import (
|
||||
require_non_negative_regional_attention_index,
|
||||
validate_regional_attention_plan_authorities,
|
||||
)
|
||||
from .regional_lora_plan import RegionalLoraPlan
|
||||
from .regional_mask_bank import RegionalMaskBank
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ProcessedRegionalAttentionEntry:
|
||||
"""Retain one ordered model-ready conditioning entry and Comfy strength."""
|
||||
|
||||
entry_index: int
|
||||
uuid: UUID
|
||||
schedule: ConditioningScheduleRange
|
||||
cross_attention: torch.Tensor
|
||||
strength: float
|
||||
cross_attention_value_multiplier: torch.Tensor | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Validate entry order, model context, and finite scalar strength."""
|
||||
|
||||
require_non_negative_regional_attention_index(
|
||||
self.entry_index,
|
||||
name="entry_index",
|
||||
)
|
||||
if not isinstance(self.uuid, UUID):
|
||||
raise TypeError("Processed conditioning entry UUID must be uuid.UUID.")
|
||||
if not isinstance(self.schedule, ConditioningScheduleRange):
|
||||
raise TypeError(
|
||||
"Processed conditioning entry schedule has an invalid type."
|
||||
)
|
||||
if not isinstance(self.cross_attention, torch.Tensor):
|
||||
raise TypeError("Processed cross_attention must be a torch.Tensor.")
|
||||
if self.cross_attention.ndim != 3:
|
||||
raise ValueError("Processed cross_attention must use BxSxD layout.")
|
||||
if any(int(size) < 1 for size in self.cross_attention.shape):
|
||||
raise ValueError("Processed cross_attention dimensions must be positive.")
|
||||
if not self.cross_attention.is_floating_point():
|
||||
raise TypeError("Processed cross_attention must be floating point.")
|
||||
if not bool(torch.isfinite(self.cross_attention).all().item()):
|
||||
raise ValueError("Processed cross_attention must contain finite values.")
|
||||
if isinstance(self.strength, bool) or not isinstance(
|
||||
self.strength,
|
||||
int | float,
|
||||
):
|
||||
raise TypeError("Processed conditioning strength must be a real number.")
|
||||
if not math.isfinite(float(self.strength)):
|
||||
raise ValueError("Processed conditioning strength must be finite.")
|
||||
object.__setattr__(self, "strength", float(self.strength))
|
||||
multiplier = self.cross_attention_value_multiplier
|
||||
if multiplier is None:
|
||||
return
|
||||
if (
|
||||
not isinstance(multiplier, torch.Tensor)
|
||||
or multiplier.shape != (*self.cross_attention.shape[:2], 1)
|
||||
or not multiplier.is_floating_point()
|
||||
or multiplier.device != self.cross_attention.device
|
||||
or multiplier.dtype != self.cross_attention.dtype
|
||||
or not bool(torch.isfinite(multiplier).all().item())
|
||||
):
|
||||
raise ValueError(
|
||||
"Processed attention value multiplier must be a finite floating "
|
||||
"BxSx1 tensor aligned with cross_attention."
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ProcessedRegionalAttentionContext:
|
||||
"""Retain every ordered processed entry for one authored conditioning."""
|
||||
|
||||
conditioning_index: int
|
||||
region_index: int | None
|
||||
entries: tuple[ProcessedRegionalAttentionEntry, ...]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Validate global/regional ownership and model-ready tensor structure."""
|
||||
|
||||
if self.region_index is None:
|
||||
if self.conditioning_index != 0:
|
||||
raise ValueError(
|
||||
"Processed base attention context must use conditioning index 0."
|
||||
)
|
||||
else:
|
||||
require_non_negative_regional_attention_index(
|
||||
self.region_index,
|
||||
name="region_index",
|
||||
)
|
||||
if self.conditioning_index != self.region_index + 1:
|
||||
raise ValueError(
|
||||
"Processed regional conditioning_index must equal region_index + 1."
|
||||
)
|
||||
if not isinstance(self.entries, tuple) or not self.entries:
|
||||
raise ValueError("Processed attention context requires ordered entries.")
|
||||
if any(
|
||||
not isinstance(entry, ProcessedRegionalAttentionEntry)
|
||||
for entry in self.entries
|
||||
):
|
||||
raise TypeError("Processed attention context contains an invalid entry.")
|
||||
if tuple(entry.entry_index for entry in self.entries) != tuple(
|
||||
range(len(self.entries))
|
||||
):
|
||||
raise ValueError("Processed attention entries must use canonical order.")
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ProcessedRegionalAttentionBranch:
|
||||
"""Retain one processed base context and ordered regional context bank."""
|
||||
|
||||
base_context: ProcessedRegionalAttentionContext
|
||||
regional_contexts: tuple[ProcessedRegionalAttentionContext, ...]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require one global base and canonical regional context order."""
|
||||
|
||||
if not isinstance(self.base_context, ProcessedRegionalAttentionContext):
|
||||
raise TypeError("Processed attention base context has an invalid type.")
|
||||
if self.base_context.region_index is not None:
|
||||
raise ValueError("Processed attention base context must be global.")
|
||||
if not isinstance(self.regional_contexts, tuple):
|
||||
raise TypeError("Processed regional attention contexts must be a tuple.")
|
||||
if any(
|
||||
not isinstance(context, ProcessedRegionalAttentionContext)
|
||||
for context in self.regional_contexts
|
||||
):
|
||||
raise TypeError(
|
||||
"Processed regional attention branch contains an invalid context."
|
||||
)
|
||||
indices = tuple(context.region_index for context in self.regional_contexts)
|
||||
if indices != tuple(range(len(self.regional_contexts))):
|
||||
raise ValueError(
|
||||
"Processed regional attention contexts must use canonical order."
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ProcessedRegionalAttentionPlan:
|
||||
"""Retain processed branches and shared canonical regional authorities."""
|
||||
|
||||
positive: ProcessedRegionalAttentionBranch
|
||||
negative: ProcessedRegionalAttentionBranch
|
||||
mask_bank: RegionalMaskBank
|
||||
lora_plan: RegionalLoraPlan
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Validate plan owners and regional LoRA bounds."""
|
||||
|
||||
if not isinstance(
|
||||
self.positive, ProcessedRegionalAttentionBranch
|
||||
) or not isinstance(self.negative, ProcessedRegionalAttentionBranch):
|
||||
raise TypeError(
|
||||
"Processed regional attention plan contains an invalid branch."
|
||||
)
|
||||
validate_regional_attention_plan_authorities(
|
||||
mask_bank=self.mask_bank,
|
||||
lora_plan=self.lora_plan,
|
||||
positive_region_count=len(self.positive.regional_contexts),
|
||||
negative_region_count=len(self.negative.regional_contexts),
|
||||
)
|
||||
|
||||
@property
|
||||
def is_time_invariant(self) -> bool:
|
||||
"""Report whether contexts and regional LoRA strengths remain fixed."""
|
||||
|
||||
branches = (self.positive, self.negative)
|
||||
contexts = tuple(
|
||||
context
|
||||
for branch in branches
|
||||
for context in (branch.base_context, *branch.regional_contexts)
|
||||
)
|
||||
return self.lora_plan.is_time_invariant and all(
|
||||
entry.schedule.is_time_invariant
|
||||
for context in contexts
|
||||
for entry in context.entries
|
||||
)
|
||||
@@ -0,0 +1,32 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Parse authored prompt text into ordered batch entries."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
DEFAULT_PROMPT_BATCH_SEPARATOR = "[SEP]"
|
||||
_NAMED_DEFAULT_SEPARATOR_PATTERN = r"\[SEP(?:\|[^\r\n\]]+)?\]"
|
||||
|
||||
|
||||
def split_prompt_batch(
|
||||
text: str,
|
||||
separator: str = DEFAULT_PROMPT_BATCH_SEPARATOR,
|
||||
) -> tuple[str, ...]:
|
||||
"""Split prompt text while discarding default-separator labels."""
|
||||
|
||||
if separator == "":
|
||||
raise ValueError("separator must not be empty.")
|
||||
pattern = rf"\s*(?:{_separator_pattern(separator)})\s*"
|
||||
return tuple(re.split(pattern, text))
|
||||
|
||||
|
||||
def _separator_pattern(separator: str) -> str:
|
||||
"""Return labeled default grammar or an escaped custom separator pattern."""
|
||||
|
||||
if separator == DEFAULT_PROMPT_BATCH_SEPARATOR:
|
||||
return _NAMED_DEFAULT_SEPARATOR_PATTERN
|
||||
return re.escape(separator)
|
||||
@@ -9,7 +9,7 @@ from __future__ import annotations
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
|
||||
from .conditioning_batch import split_prompt_batch
|
||||
from .prompt_batch_parser import split_prompt_batch
|
||||
|
||||
PROMPT_TEXT_PATTERN = r"(?:^|>)([^<]+)(?=<|$)"
|
||||
LORA_TAG_PATTERN = r"<[^>]*>"
|
||||
@@ -25,9 +25,15 @@ class PreparedPromptChunk:
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PreparedPromptSide:
|
||||
"""Store ordered prompt chunks and all scheduling tags for one prompt side."""
|
||||
"""Store ordered prompt chunks for one positive or negative prompt side."""
|
||||
|
||||
chunks: tuple[PreparedPromptChunk, ...]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PromptSegmentHookPlan:
|
||||
"""Store the combined LoRA schedule for one aligned SEP position."""
|
||||
|
||||
lora_tags: str
|
||||
|
||||
|
||||
@@ -44,7 +50,7 @@ def extract_lora_tags(text: str) -> str:
|
||||
|
||||
|
||||
def prepare_prompt_side(text: str, separator: str) -> PreparedPromptSide:
|
||||
"""Split a prompt side into cleaned chunks and aggregate LoRA tags."""
|
||||
"""Split a prompt side into ordered cleaned chunks with local LoRA tags."""
|
||||
|
||||
chunks = tuple(
|
||||
PreparedPromptChunk(
|
||||
@@ -53,8 +59,7 @@ def prepare_prompt_side(text: str, separator: str) -> PreparedPromptSide:
|
||||
)
|
||||
for chunk in split_prompt_batch(text, separator)
|
||||
)
|
||||
lora_tags = "\n".join(chunk.lora_tags for chunk in chunks if chunk.lora_tags)
|
||||
return PreparedPromptSide(chunks=chunks, lora_tags=lora_tags)
|
||||
return PreparedPromptSide(chunks=chunks)
|
||||
|
||||
|
||||
def apply_encode_style(encode_style: str, prompt_text: str) -> str:
|
||||
|
||||
@@ -0,0 +1,94 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Plan authored and global-fallback positions across two prompt sides."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import TypeVar
|
||||
|
||||
SegmentValue = TypeVar("SegmentValue")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PromptSegmentSource:
|
||||
"""Identify one effective segment's authored source position."""
|
||||
|
||||
source_index: int
|
||||
authored: bool
|
||||
|
||||
def resolve(self, values: tuple[SegmentValue, ...]) -> SegmentValue:
|
||||
"""Return the authored value selected for this effective position."""
|
||||
|
||||
return values[self.source_index]
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PromptSideAlignment:
|
||||
"""Describe effective positions for one authored prompt side."""
|
||||
|
||||
authored_count: int
|
||||
sources: tuple[PromptSegmentSource, ...]
|
||||
|
||||
def materialize(
|
||||
self,
|
||||
values: tuple[SegmentValue, ...],
|
||||
) -> tuple[SegmentValue, ...]:
|
||||
"""Resolve effective values while rejecting a mismatched authored side."""
|
||||
|
||||
if len(values) != self.authored_count:
|
||||
raise ValueError(
|
||||
"prompt alignment expected "
|
||||
f"{self.authored_count} authored segments but received {len(values)}."
|
||||
)
|
||||
return tuple(source.resolve(values) for source in self.sources)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PromptSegmentAlignment:
|
||||
"""Store matched effective positions for positive and negative prompts."""
|
||||
|
||||
positive: PromptSideAlignment
|
||||
negative: PromptSideAlignment
|
||||
|
||||
@property
|
||||
def segment_count(self) -> int:
|
||||
"""Return the shared number of effective prompt positions."""
|
||||
|
||||
return len(self.positive.sources)
|
||||
|
||||
|
||||
def build_prompt_segment_alignment(
|
||||
*,
|
||||
positive_count: int,
|
||||
negative_count: int,
|
||||
) -> PromptSegmentAlignment:
|
||||
"""Align prompt sides by filling missing positions from each global entry."""
|
||||
|
||||
if positive_count < 1 or negative_count < 1:
|
||||
raise ValueError("each prompt side must contain at least one authored segment.")
|
||||
segment_count = max(positive_count, negative_count)
|
||||
return PromptSegmentAlignment(
|
||||
positive=_build_side_alignment(positive_count, segment_count),
|
||||
negative=_build_side_alignment(negative_count, segment_count),
|
||||
)
|
||||
|
||||
|
||||
def _build_side_alignment(
|
||||
authored_count: int,
|
||||
segment_count: int,
|
||||
) -> PromptSideAlignment:
|
||||
"""Return authored positions followed by global-entry fallback positions."""
|
||||
|
||||
return PromptSideAlignment(
|
||||
authored_count=authored_count,
|
||||
sources=tuple(
|
||||
PromptSegmentSource(
|
||||
source_index=index if index < authored_count else 0,
|
||||
authored=index < authored_count,
|
||||
)
|
||||
for index in range(segment_count)
|
||||
),
|
||||
)
|
||||
@@ -0,0 +1,310 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Define validated identities and records for quantized profile artifacts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
from dataclasses import dataclass
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import cast
|
||||
|
||||
from .model_quantization import QuantizationProfile
|
||||
|
||||
MANIFEST_SCHEMA_VERSION = 2
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SourceCheckpointIdentity:
|
||||
"""Identify an authoritative source checkpoint and its current file state."""
|
||||
|
||||
display_name: str
|
||||
path: Path
|
||||
size_bytes: int
|
||||
modified_ns: int
|
||||
sha256: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class QuantCacheIdentity:
|
||||
"""Identify one profile and recipe derivative of a source checkpoint."""
|
||||
|
||||
source: SourceCheckpointIdentity
|
||||
profile: QuantizationProfile
|
||||
model_family: str
|
||||
recipe_version: int
|
||||
|
||||
@property
|
||||
def stable_key(self) -> str:
|
||||
"""Return the collision-resistant cache key."""
|
||||
|
||||
identity = "\0".join(
|
||||
(
|
||||
self.source.sha256,
|
||||
self.profile.profile_id,
|
||||
str(self.profile.version),
|
||||
self.model_family,
|
||||
str(self.recipe_version),
|
||||
)
|
||||
)
|
||||
return hashlib.sha256(identity.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class QuantCacheManifest:
|
||||
"""Describe one complete SimpleSyrup-managed cache artifact."""
|
||||
|
||||
source_model: str
|
||||
source_path: str
|
||||
source_sha256: str
|
||||
source_size_bytes: int
|
||||
source_modified_ns: int
|
||||
profile_id: str
|
||||
profile_label: str
|
||||
profile_version: int
|
||||
quantization_formats: tuple[str, ...]
|
||||
model_family: str
|
||||
recipe_version: int
|
||||
artifact_file: str
|
||||
artifact_size_bytes: int
|
||||
created_at: str
|
||||
last_used_at: str
|
||||
schema_version: int = MANIFEST_SCHEMA_VERSION
|
||||
managed_by: str = "SimpleSyrup"
|
||||
|
||||
@classmethod
|
||||
def create(
|
||||
cls,
|
||||
identity: QuantCacheIdentity,
|
||||
artifact_file: str,
|
||||
artifact_size_bytes: int,
|
||||
) -> QuantCacheManifest:
|
||||
"""Create a current manifest for a completed artifact."""
|
||||
|
||||
now = datetime.now(UTC).isoformat()
|
||||
return cls(
|
||||
source_model=identity.source.display_name,
|
||||
source_path=str(identity.source.path),
|
||||
source_sha256=identity.source.sha256,
|
||||
source_size_bytes=identity.source.size_bytes,
|
||||
source_modified_ns=identity.source.modified_ns,
|
||||
profile_id=identity.profile.profile_id,
|
||||
profile_label=identity.profile.label,
|
||||
profile_version=identity.profile.version,
|
||||
quantization_formats=tuple(
|
||||
sorted(item.value for item in identity.profile.required_formats)
|
||||
),
|
||||
model_family=identity.model_family,
|
||||
recipe_version=identity.recipe_version,
|
||||
artifact_file=artifact_file,
|
||||
artifact_size_bytes=artifact_size_bytes,
|
||||
created_at=now,
|
||||
last_used_at=now,
|
||||
)
|
||||
|
||||
def matches_current_source(
|
||||
self,
|
||||
source_model: str,
|
||||
source_path: Path,
|
||||
source_size_bytes: int,
|
||||
source_modified_ns: int,
|
||||
profile: QuantizationProfile,
|
||||
model_family: str,
|
||||
recipe_version: int,
|
||||
) -> bool:
|
||||
"""Return whether this artifact derives from the unchanged source file."""
|
||||
|
||||
return (
|
||||
self.source_model == source_model
|
||||
and Path(self.source_path) == source_path
|
||||
and self.source_size_bytes == source_size_bytes
|
||||
and self.source_modified_ns == source_modified_ns
|
||||
and self.profile_id == profile.profile_id
|
||||
and self.profile_version == profile.version
|
||||
and self.model_family == model_family
|
||||
and self.recipe_version == recipe_version
|
||||
)
|
||||
|
||||
def matches_identity(self, identity: QuantCacheIdentity) -> bool:
|
||||
"""Return whether this v2 manifest exactly describes an identity."""
|
||||
|
||||
return (
|
||||
self.schema_version == MANIFEST_SCHEMA_VERSION
|
||||
and self.source_sha256 == identity.source.sha256
|
||||
and self.profile_id == identity.profile.profile_id
|
||||
and self.profile_version == identity.profile.version
|
||||
and self.model_family == identity.model_family
|
||||
and self.recipe_version == identity.recipe_version
|
||||
)
|
||||
|
||||
def touched(self) -> QuantCacheManifest:
|
||||
"""Return a copy with a current explicit LRU timestamp."""
|
||||
|
||||
payload = self.to_payload()
|
||||
payload["last_used_at"] = datetime.now(UTC).isoformat()
|
||||
return QuantCacheManifest.from_payload(payload)
|
||||
|
||||
def to_payload(self) -> dict[str, object]:
|
||||
"""Return the human-readable JSON representation."""
|
||||
|
||||
if self.schema_version == 1:
|
||||
return {
|
||||
"schema_version": 1,
|
||||
"managed_by": self.managed_by,
|
||||
"source_model": self.source_model,
|
||||
"source_path": self.source_path,
|
||||
"source_sha256": self.source_sha256,
|
||||
"source_size_bytes": self.source_size_bytes,
|
||||
"source_modified_ns": self.source_modified_ns,
|
||||
"quantization_format": self.quantization_formats[0],
|
||||
"model_family": self.model_family,
|
||||
"recipe_version": self.recipe_version,
|
||||
"artifact_file": self.artifact_file,
|
||||
"artifact_size_bytes": self.artifact_size_bytes,
|
||||
"created_at": self.created_at,
|
||||
"last_used_at": self.last_used_at,
|
||||
}
|
||||
return {
|
||||
"schema_version": self.schema_version,
|
||||
"managed_by": self.managed_by,
|
||||
"source_model": self.source_model,
|
||||
"source_path": self.source_path,
|
||||
"source_sha256": self.source_sha256,
|
||||
"source_size_bytes": self.source_size_bytes,
|
||||
"source_modified_ns": self.source_modified_ns,
|
||||
"profile_id": self.profile_id,
|
||||
"profile_label": self.profile_label,
|
||||
"profile_version": self.profile_version,
|
||||
"quantization_formats": list(self.quantization_formats),
|
||||
"model_family": self.model_family,
|
||||
"recipe_version": self.recipe_version,
|
||||
"artifact_file": self.artifact_file,
|
||||
"artifact_size_bytes": self.artifact_size_bytes,
|
||||
"created_at": self.created_at,
|
||||
"last_used_at": self.last_used_at,
|
||||
}
|
||||
|
||||
@classmethod
|
||||
def from_payload(cls, payload: object) -> QuantCacheManifest:
|
||||
"""Validate managed manifests, retaining v1 only for cache cleanup."""
|
||||
|
||||
if not isinstance(payload, dict):
|
||||
raise ValueError("Quant cache manifest must be a JSON object.")
|
||||
if payload.get("schema_version") == 1:
|
||||
return cls._from_legacy_payload(payload)
|
||||
required_strings = (
|
||||
"managed_by",
|
||||
"source_model",
|
||||
"source_path",
|
||||
"source_sha256",
|
||||
"profile_id",
|
||||
"profile_label",
|
||||
"model_family",
|
||||
"artifact_file",
|
||||
"created_at",
|
||||
"last_used_at",
|
||||
)
|
||||
for key in required_strings:
|
||||
if not isinstance(payload.get(key), str):
|
||||
raise ValueError(
|
||||
f"Quant cache manifest field '{key}' must be a string."
|
||||
)
|
||||
required_integers = (
|
||||
"schema_version",
|
||||
"source_size_bytes",
|
||||
"source_modified_ns",
|
||||
"profile_version",
|
||||
"recipe_version",
|
||||
"artifact_size_bytes",
|
||||
)
|
||||
for key in required_integers:
|
||||
value = payload.get(key)
|
||||
if not isinstance(value, int) or isinstance(value, bool):
|
||||
raise ValueError(
|
||||
f"Quant cache manifest field '{key}' must be an integer."
|
||||
)
|
||||
raw_formats = payload.get("quantization_formats")
|
||||
if not isinstance(raw_formats, list) or not all(
|
||||
isinstance(item, str) for item in raw_formats
|
||||
):
|
||||
raise ValueError(
|
||||
"Quant cache manifest field 'quantization_formats' must be a "
|
||||
"string list."
|
||||
)
|
||||
if payload["managed_by"] != "SimpleSyrup":
|
||||
raise ValueError("Quant cache manifest is not managed by SimpleSyrup.")
|
||||
if payload["schema_version"] != MANIFEST_SCHEMA_VERSION:
|
||||
raise ValueError("Quant cache manifest schema version is unsupported.")
|
||||
return cls(
|
||||
schema_version=payload["schema_version"],
|
||||
managed_by=payload["managed_by"],
|
||||
source_model=payload["source_model"],
|
||||
source_path=payload["source_path"],
|
||||
source_sha256=payload["source_sha256"],
|
||||
source_size_bytes=payload["source_size_bytes"],
|
||||
source_modified_ns=payload["source_modified_ns"],
|
||||
profile_id=payload["profile_id"],
|
||||
profile_label=payload["profile_label"],
|
||||
profile_version=payload["profile_version"],
|
||||
quantization_formats=tuple(raw_formats),
|
||||
model_family=payload["model_family"],
|
||||
recipe_version=payload["recipe_version"],
|
||||
artifact_file=payload["artifact_file"],
|
||||
artifact_size_bytes=payload["artifact_size_bytes"],
|
||||
created_at=payload["created_at"],
|
||||
last_used_at=payload["last_used_at"],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def _from_legacy_payload(cls, payload: dict[object, object]) -> QuantCacheManifest:
|
||||
"""Decode v1 solely so ordinary LRU and clearing can remove it."""
|
||||
|
||||
required_strings = (
|
||||
"managed_by",
|
||||
"source_model",
|
||||
"source_path",
|
||||
"source_sha256",
|
||||
"quantization_format",
|
||||
"model_family",
|
||||
"artifact_file",
|
||||
"created_at",
|
||||
"last_used_at",
|
||||
)
|
||||
required_integers = (
|
||||
"source_size_bytes",
|
||||
"source_modified_ns",
|
||||
"recipe_version",
|
||||
"artifact_size_bytes",
|
||||
)
|
||||
if any(not isinstance(payload.get(key), str) for key in required_strings):
|
||||
raise ValueError("Legacy quant cache manifest has invalid string fields.")
|
||||
if any(
|
||||
not isinstance(payload.get(key), int) or isinstance(payload.get(key), bool)
|
||||
for key in required_integers
|
||||
):
|
||||
raise ValueError("Legacy quant cache manifest has invalid integer fields.")
|
||||
if payload["managed_by"] != "SimpleSyrup":
|
||||
raise ValueError("Quant cache manifest is not managed by SimpleSyrup.")
|
||||
quantization_format = str(payload["quantization_format"])
|
||||
return cls(
|
||||
schema_version=1,
|
||||
managed_by=str(payload["managed_by"]),
|
||||
source_model=str(payload["source_model"]),
|
||||
source_path=str(payload["source_path"]),
|
||||
source_sha256=str(payload["source_sha256"]),
|
||||
source_size_bytes=cast(int, payload["source_size_bytes"]),
|
||||
source_modified_ns=cast(int, payload["source_modified_ns"]),
|
||||
profile_id=f"legacy-v1-{quantization_format}",
|
||||
profile_label=f"Legacy v1 {quantization_format}",
|
||||
profile_version=1,
|
||||
quantization_formats=(quantization_format,),
|
||||
model_family=str(payload["model_family"]),
|
||||
recipe_version=cast(int, payload["recipe_version"]),
|
||||
artifact_file=str(payload["artifact_file"]),
|
||||
artifact_size_bytes=cast(int, payload["artifact_size_bytes"]),
|
||||
created_at=str(payload["created_at"]),
|
||||
last_used_at=str(payload["last_used_at"]),
|
||||
)
|
||||
@@ -0,0 +1,147 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Own immutable raw regional Attention Coupling authoring plans."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
from .conditioning_batch import ConditioningBatch
|
||||
from .regional_attention import (
|
||||
require_non_negative_regional_attention_index,
|
||||
validate_regional_attention_plan_authorities,
|
||||
)
|
||||
from .regional_lora_plan import EMPTY_REGIONAL_LORA_PLAN, RegionalLoraPlan
|
||||
from .regional_mask_bank import RegionalMaskBank
|
||||
from .regional_prompting import build_regional_conditioning_plan
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RawRegionalAttentionContext:
|
||||
"""Retain one original regional conditioning and its canonical indices."""
|
||||
|
||||
conditioning_index: int
|
||||
region_index: int
|
||||
conditioning: object
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require the established global-first positional relationship."""
|
||||
|
||||
require_non_negative_regional_attention_index(
|
||||
self.region_index,
|
||||
name="region_index",
|
||||
)
|
||||
if self.conditioning_index != self.region_index + 1:
|
||||
raise ValueError(
|
||||
"Regional attention conditioning_index must equal region_index + 1."
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RawRegionalAttentionBranch:
|
||||
"""Retain one base conditioning and ordered regional conditionings."""
|
||||
|
||||
base_conditioning: object
|
||||
regional_contexts: tuple[RawRegionalAttentionContext, ...]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require immutable canonical regional order."""
|
||||
|
||||
if not isinstance(self.regional_contexts, tuple):
|
||||
raise TypeError("Raw regional attention contexts must be a tuple.")
|
||||
if any(
|
||||
not isinstance(context, RawRegionalAttentionContext)
|
||||
for context in self.regional_contexts
|
||||
):
|
||||
raise TypeError(
|
||||
"Raw regional attention branch contains an invalid context."
|
||||
)
|
||||
indices = tuple(context.region_index for context in self.regional_contexts)
|
||||
if indices != tuple(range(len(self.regional_contexts))):
|
||||
raise ValueError(
|
||||
"Raw regional attention contexts must use canonical order."
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RawRegionalAttentionPlan:
|
||||
"""Retain both raw branches and shared canonical regional authorities."""
|
||||
|
||||
positive: RawRegionalAttentionBranch
|
||||
negative: RawRegionalAttentionBranch
|
||||
mask_bank: RegionalMaskBank
|
||||
lora_plan: RegionalLoraPlan
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Validate plan owners and regional LoRA bounds."""
|
||||
|
||||
if not isinstance(self.positive, RawRegionalAttentionBranch) or not isinstance(
|
||||
self.negative, RawRegionalAttentionBranch
|
||||
):
|
||||
raise TypeError("Raw regional attention plan contains an invalid branch.")
|
||||
validate_regional_attention_plan_authorities(
|
||||
mask_bank=self.mask_bank,
|
||||
lora_plan=self.lora_plan,
|
||||
positive_region_count=len(self.positive.regional_contexts),
|
||||
negative_region_count=len(self.negative.regional_contexts),
|
||||
)
|
||||
|
||||
|
||||
def build_raw_regional_attention_plan(
|
||||
*,
|
||||
positive: object,
|
||||
negative: object,
|
||||
mask_bank: RegionalMaskBank,
|
||||
lora_plan: RegionalLoraPlan = EMPTY_REGIONAL_LORA_PLAN,
|
||||
) -> RawRegionalAttentionPlan:
|
||||
"""Build both branches through the authoritative global-first pairing policy."""
|
||||
|
||||
if not isinstance(mask_bank, RegionalMaskBank):
|
||||
raise TypeError("Raw regional attention requires a RegionalMaskBank.")
|
||||
return RawRegionalAttentionPlan(
|
||||
positive=_build_raw_branch(
|
||||
positive,
|
||||
mask_bank=mask_bank,
|
||||
input_name="positive",
|
||||
),
|
||||
negative=_build_raw_branch(
|
||||
negative,
|
||||
mask_bank=mask_bank,
|
||||
input_name="negative",
|
||||
),
|
||||
mask_bank=mask_bank,
|
||||
lora_plan=lora_plan,
|
||||
)
|
||||
|
||||
|
||||
def _build_raw_branch(
|
||||
conditioning: object,
|
||||
*,
|
||||
mask_bank: RegionalMaskBank,
|
||||
input_name: str,
|
||||
) -> RawRegionalAttentionBranch:
|
||||
"""Pair one raw conditioning branch without restating index policy."""
|
||||
|
||||
entries = (
|
||||
conditioning.entries
|
||||
if isinstance(conditioning, ConditioningBatch)
|
||||
else (conditioning,)
|
||||
)
|
||||
pairing = build_regional_conditioning_plan(
|
||||
region_count=mask_bank.region_count,
|
||||
conditioning_count=len(entries),
|
||||
input_name=input_name,
|
||||
)
|
||||
return RawRegionalAttentionBranch(
|
||||
base_conditioning=entries[0],
|
||||
regional_contexts=tuple(
|
||||
RawRegionalAttentionContext(
|
||||
conditioning_index=pair.conditioning_index,
|
||||
region_index=pair.mask_index,
|
||||
conditioning=entries[pair.conditioning_index],
|
||||
)
|
||||
for pair in pairing.pairs
|
||||
),
|
||||
)
|
||||
@@ -0,0 +1,237 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Define model-neutral regional activation geometry and batch alignment."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from enum import StrEnum
|
||||
|
||||
from .spatial_views import SpatialBatchLayout
|
||||
|
||||
|
||||
class RegionalActivationLayout(StrEnum):
|
||||
"""Classify the explicit spatial organization of one adapter activation."""
|
||||
|
||||
DIRECT_CONVOLUTION_1D = "direct_convolution_1d"
|
||||
DIRECT_CONVOLUTION_2D = "direct_convolution_2d"
|
||||
DIRECT_CONVOLUTION_3D = "direct_convolution_3d"
|
||||
FLATTENED_SPATIAL_TOKENS = "flattened_spatial_tokens"
|
||||
CONSUMER_SPATIALIZED = "consumer_spatialized"
|
||||
BRANCH_TOKENS = "branch_tokens"
|
||||
|
||||
|
||||
class RegionalTemporalOwnership(StrEnum):
|
||||
"""Declare how a two-dimensional authored mask owns temporal activations."""
|
||||
|
||||
NONE = "none"
|
||||
REPEAT_SPATIAL_MASK = "repeat_spatial_mask"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RegionalActivationBatchAlignment:
|
||||
"""Retain CFG, latent-batch, and optional view-major alignment evidence."""
|
||||
|
||||
latent_batch_size: int
|
||||
chunk_count: int
|
||||
spatial_layout: SpatialBatchLayout | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require positive counts and one layout over the complete base batch."""
|
||||
|
||||
_positive_integer(self.latent_batch_size, name="latent_batch_size")
|
||||
_positive_integer(self.chunk_count, name="chunk_count")
|
||||
if self.spatial_layout is not None:
|
||||
if not isinstance(self.spatial_layout, SpatialBatchLayout):
|
||||
raise TypeError(
|
||||
"Regional activation spatial_layout must be a SpatialBatchLayout."
|
||||
)
|
||||
if self.spatial_layout.input_batch_size != self.base_batch_size:
|
||||
raise ValueError(
|
||||
"Regional activation spatial layout input batch must match "
|
||||
"CFG chunks times latent batch size."
|
||||
)
|
||||
|
||||
@property
|
||||
def base_batch_size(self) -> int:
|
||||
"""Return the model batch before spatial-view expansion."""
|
||||
|
||||
return self.latent_batch_size * self.chunk_count
|
||||
|
||||
@property
|
||||
def invocation_batch_size(self) -> int:
|
||||
"""Return the active model batch after optional view expansion."""
|
||||
|
||||
if self.spatial_layout is None:
|
||||
return self.base_batch_size
|
||||
return self.spatial_layout.expanded_batch_size
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RegionalActivationGeometry:
|
||||
"""Describe one exact rank activation and its authored-mask correspondence."""
|
||||
|
||||
layout: RegionalActivationLayout
|
||||
invocation_shape: tuple[int, ...]
|
||||
feature_axis: int
|
||||
spatial_height: int
|
||||
spatial_width: int
|
||||
batch_alignment: RegionalActivationBatchAlignment
|
||||
temporal_axis: int | None = None
|
||||
temporal_ownership: RegionalTemporalOwnership = RegionalTemporalOwnership.NONE
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Reject ambiguous axes, batches, tokens, and temporal ownership."""
|
||||
|
||||
if not isinstance(self.layout, RegionalActivationLayout):
|
||||
raise TypeError("Regional activation layout has an invalid type.")
|
||||
if not isinstance(self.invocation_shape, tuple) or not self.invocation_shape:
|
||||
raise ValueError("Regional activation shape must be a nonempty tuple.")
|
||||
for dimension in self.invocation_shape:
|
||||
_positive_integer(dimension, name="shape dimension")
|
||||
if not isinstance(self.batch_alignment, RegionalActivationBatchAlignment):
|
||||
raise TypeError("Regional activation requires batch alignment evidence.")
|
||||
_positive_integer(self.spatial_height, name="spatial_height")
|
||||
_positive_integer(self.spatial_width, name="spatial_width")
|
||||
if self.invocation_shape[0] != self.batch_alignment.invocation_batch_size:
|
||||
raise ValueError(
|
||||
"Regional activation leading batch must match its alignment."
|
||||
)
|
||||
rank = len(self.invocation_shape)
|
||||
_axis(self.feature_axis, rank=rank, name="feature_axis")
|
||||
if self.feature_axis == 0:
|
||||
raise ValueError(
|
||||
"Regional activation feature axis cannot be the batch axis."
|
||||
)
|
||||
if not isinstance(self.temporal_ownership, RegionalTemporalOwnership):
|
||||
raise TypeError(
|
||||
"Regional activation temporal ownership has an invalid type."
|
||||
)
|
||||
self._validate_layout()
|
||||
|
||||
def _validate_layout(self) -> None:
|
||||
"""Match the declared layout to its exact conventional tensor shape."""
|
||||
|
||||
batch = self.batch_alignment.invocation_batch_size
|
||||
features = self.invocation_shape[self.feature_axis]
|
||||
if self.layout is RegionalActivationLayout.DIRECT_CONVOLUTION_1D:
|
||||
self._require_shape((batch, features, self.spatial_width), feature_axis=1)
|
||||
if self.spatial_height != 1:
|
||||
raise ValueError("Direct Conv1d regional geometry requires height one.")
|
||||
self._require_no_temporal_axis()
|
||||
return
|
||||
if self.layout is RegionalActivationLayout.DIRECT_CONVOLUTION_2D:
|
||||
self._require_shape(
|
||||
(batch, features, self.spatial_height, self.spatial_width),
|
||||
feature_axis=1,
|
||||
)
|
||||
self._require_no_temporal_axis()
|
||||
return
|
||||
if self.layout is RegionalActivationLayout.DIRECT_CONVOLUTION_3D:
|
||||
if self.feature_axis != 1 or len(self.invocation_shape) != 5:
|
||||
raise ValueError("Direct Conv3d regional geometry requires B/C/D/H/W.")
|
||||
if self.temporal_axis != 2:
|
||||
raise ValueError(
|
||||
"Direct Conv3d regional geometry requires temporal axis 2."
|
||||
)
|
||||
if self.invocation_shape[3:] != (
|
||||
self.spatial_height,
|
||||
self.spatial_width,
|
||||
):
|
||||
raise ValueError("Direct Conv3d regional H/W must match its tensor.")
|
||||
if (
|
||||
self.temporal_ownership
|
||||
is not RegionalTemporalOwnership.REPEAT_SPATIAL_MASK
|
||||
):
|
||||
raise ValueError(
|
||||
"Direct Conv3d regional geometry requires explicit repeated "
|
||||
"spatial-mask temporal ownership."
|
||||
)
|
||||
return
|
||||
if self.layout in (
|
||||
RegionalActivationLayout.FLATTENED_SPATIAL_TOKENS,
|
||||
RegionalActivationLayout.CONSUMER_SPATIALIZED,
|
||||
):
|
||||
self._require_shape(
|
||||
(
|
||||
batch,
|
||||
self.spatial_height * self.spatial_width,
|
||||
features,
|
||||
),
|
||||
feature_axis=2,
|
||||
)
|
||||
self._require_no_temporal_axis()
|
||||
return
|
||||
if self.layout is RegionalActivationLayout.BRANCH_TOKENS:
|
||||
self._require_shape(
|
||||
(batch, self.spatial_width, features),
|
||||
feature_axis=2,
|
||||
)
|
||||
if self.spatial_height != 1:
|
||||
raise ValueError("Branch-token regional geometry requires height one.")
|
||||
self._require_no_temporal_axis()
|
||||
return
|
||||
raise AssertionError(f"Unhandled regional activation layout: {self.layout}")
|
||||
|
||||
def _require_shape(
|
||||
self,
|
||||
expected: tuple[int, ...],
|
||||
*,
|
||||
feature_axis: int,
|
||||
) -> None:
|
||||
"""Require one conventional shape and feature-axis location."""
|
||||
|
||||
if self.feature_axis != feature_axis or self.invocation_shape != expected:
|
||||
raise ValueError(
|
||||
f"{self.layout.value} regional geometry expected shape {expected} "
|
||||
f"with feature axis {feature_axis}; observed "
|
||||
f"{self.invocation_shape} and axis {self.feature_axis}."
|
||||
)
|
||||
|
||||
def _require_no_temporal_axis(self) -> None:
|
||||
"""Reject temporal claims from non-temporal image activation layouts."""
|
||||
|
||||
if self.temporal_axis is not None:
|
||||
raise ValueError(
|
||||
"Non-temporal regional geometry cannot declare a temporal axis."
|
||||
)
|
||||
if self.temporal_ownership is not RegionalTemporalOwnership.NONE:
|
||||
raise ValueError(
|
||||
"Non-temporal regional geometry cannot claim temporal ownership."
|
||||
)
|
||||
|
||||
@property
|
||||
def temporal_size(self) -> int | None:
|
||||
"""Return the explicit temporal size when this activation owns one."""
|
||||
|
||||
if self.temporal_axis is None:
|
||||
return None
|
||||
return self.invocation_shape[self.temporal_axis]
|
||||
|
||||
def broadcast_mask_shape(self, region_count: int) -> tuple[int, ...]:
|
||||
"""Return the exact region-major multiplier shape for this activation."""
|
||||
|
||||
_positive_integer(region_count, name="region_count")
|
||||
shape = list(self.invocation_shape)
|
||||
shape[self.feature_axis] = 1
|
||||
return (region_count, *shape)
|
||||
|
||||
|
||||
def _positive_integer(value: object, *, name: str) -> None:
|
||||
"""Require one strictly positive non-boolean integer."""
|
||||
|
||||
if isinstance(value, bool) or not isinstance(value, int):
|
||||
raise TypeError(f"Regional activation {name} must be an integer.")
|
||||
if value < 1:
|
||||
raise ValueError(f"Regional activation {name} must be positive.")
|
||||
|
||||
|
||||
def _axis(value: object, *, rank: int, name: str) -> None:
|
||||
"""Require one non-negative axis inside the invocation rank."""
|
||||
|
||||
if isinstance(value, bool) or not isinstance(value, int):
|
||||
raise TypeError(f"Regional activation {name} must be an integer.")
|
||||
if not 0 <= value < rank:
|
||||
raise ValueError(f"Regional activation {name} is outside the tensor rank.")
|
||||
@@ -0,0 +1,69 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Own shared regional Attention Coupling contracts and validation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import StrEnum
|
||||
|
||||
from .regional_lora_plan import RegionalLoraPlan
|
||||
from .regional_mask_bank import RegionalMaskBank
|
||||
|
||||
|
||||
class RegionalAttentionBranch(StrEnum):
|
||||
"""Name the processed conditioning bank selected for one Comfy chunk."""
|
||||
|
||||
POSITIVE = "positive"
|
||||
NEGATIVE = "negative"
|
||||
|
||||
|
||||
def require_non_negative_regional_attention_index(
|
||||
value: object,
|
||||
*,
|
||||
name: str,
|
||||
) -> None:
|
||||
"""Require one non-negative integer regional-attention index."""
|
||||
|
||||
if isinstance(value, bool) or not isinstance(value, int):
|
||||
raise TypeError(f"Regional attention {name} must be an integer.")
|
||||
if value < 0:
|
||||
raise ValueError(f"Regional attention {name} must be non-negative.")
|
||||
|
||||
|
||||
def validate_regional_attention_plan_authorities(
|
||||
*,
|
||||
mask_bank: object,
|
||||
lora_plan: object,
|
||||
positive_region_count: int,
|
||||
negative_region_count: int,
|
||||
) -> None:
|
||||
"""Validate shared mask, LoRA, and branch-count plan authorities."""
|
||||
|
||||
if not isinstance(mask_bank, RegionalMaskBank):
|
||||
raise TypeError("Regional attention plan requires a RegionalMaskBank.")
|
||||
if not isinstance(lora_plan, RegionalLoraPlan):
|
||||
raise TypeError("Regional attention plan requires a RegionalLoraPlan.")
|
||||
for branch_name, region_count in (
|
||||
("positive", positive_region_count),
|
||||
("negative", negative_region_count),
|
||||
):
|
||||
require_non_negative_regional_attention_index(
|
||||
region_count,
|
||||
name=f"{branch_name} region count",
|
||||
)
|
||||
if region_count > mask_bank.region_count:
|
||||
raise ValueError(
|
||||
f"Regional attention {branch_name} branch exceeds the mask bank."
|
||||
)
|
||||
out_of_bounds = tuple(
|
||||
adapter
|
||||
for adapter in lora_plan.adapters
|
||||
if adapter.region_index >= mask_bank.region_count
|
||||
)
|
||||
if out_of_bounds:
|
||||
indices = ", ".join(str(adapter.region_index) for adapter in out_of_bounds)
|
||||
raise ValueError(
|
||||
"Regional attention LoRA region indices exceed the mask bank: " + indices
|
||||
)
|
||||
@@ -0,0 +1,227 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Own immutable chunk-major regional attention batch alignment values."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
|
||||
import torch
|
||||
|
||||
from .regional_attention import RegionalAttentionBranch
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RegionalAttentionChunkBatch:
|
||||
"""Describe one Comfy chunk's contiguous slice of the model batch."""
|
||||
|
||||
chunk_index: int
|
||||
branch: RegionalAttentionBranch
|
||||
batch_start: int
|
||||
batch_stop: int
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Validate one positive non-empty contiguous batch slice."""
|
||||
|
||||
for name, value in (
|
||||
("chunk_index", self.chunk_index),
|
||||
("batch_start", self.batch_start),
|
||||
("batch_stop", self.batch_stop),
|
||||
):
|
||||
if isinstance(value, bool) or not isinstance(value, int):
|
||||
raise TypeError(f"Regional attention {name} must be an integer.")
|
||||
if self.chunk_index < 0 or self.batch_start < 0:
|
||||
raise ValueError("Regional attention chunk indices must be non-negative.")
|
||||
if self.batch_stop <= self.batch_start:
|
||||
raise ValueError("Regional attention chunk batch slice must be non-empty.")
|
||||
if not isinstance(self.branch, RegionalAttentionBranch):
|
||||
raise TypeError("Regional attention chunk branch has an invalid type.")
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class BatchedRegionalAttentionEntry:
|
||||
"""Retain one aligned regional entry and its per-sample Comfy strengths."""
|
||||
|
||||
entry_index: int
|
||||
context: torch.Tensor
|
||||
strengths: tuple[float, ...]
|
||||
cross_attention_value_multiplier: torch.Tensor | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Validate entry order, aligned context, and finite sample strengths."""
|
||||
|
||||
if isinstance(self.entry_index, bool) or not isinstance(self.entry_index, int):
|
||||
raise TypeError("Regional attention entry_index must be an integer.")
|
||||
if self.entry_index < 0:
|
||||
raise ValueError("Regional attention entry_index must be non-negative.")
|
||||
_validate_aligned_context(self.context, name="entry")
|
||||
if not isinstance(self.strengths, tuple):
|
||||
raise TypeError("Regional attention entry strengths must be a tuple.")
|
||||
if len(self.strengths) != int(self.context.shape[0]):
|
||||
raise ValueError(
|
||||
"Regional attention entry strength count must match its batch."
|
||||
)
|
||||
for strength in self.strengths:
|
||||
if isinstance(strength, bool) or not isinstance(strength, int | float):
|
||||
raise TypeError(
|
||||
"Regional attention entry strength must be a real number."
|
||||
)
|
||||
if not math.isfinite(float(strength)):
|
||||
raise ValueError("Regional attention entry strength must be finite.")
|
||||
_validate_value_multiplier(
|
||||
self.cross_attention_value_multiplier,
|
||||
self.context,
|
||||
name="entry",
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class BatchedRegionalAttentionRegion:
|
||||
"""Retain every aligned active conditioning entry for one region."""
|
||||
|
||||
region_index: int
|
||||
entries: tuple[BatchedRegionalAttentionEntry, ...]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require a non-empty canonical entry bank for one region."""
|
||||
|
||||
if isinstance(self.region_index, bool) or not isinstance(
|
||||
self.region_index, int
|
||||
):
|
||||
raise TypeError("Regional attention region_index must be an integer.")
|
||||
if self.region_index < 0:
|
||||
raise ValueError("Regional attention region_index must be non-negative.")
|
||||
if not isinstance(self.entries, tuple) or not self.entries:
|
||||
raise ValueError("Regional attention region requires active entries.")
|
||||
if any(
|
||||
not isinstance(entry, BatchedRegionalAttentionEntry)
|
||||
for entry in self.entries
|
||||
):
|
||||
raise TypeError("Regional attention region contains an invalid entry.")
|
||||
if tuple(entry.entry_index for entry in self.entries) != tuple(
|
||||
range(len(self.entries))
|
||||
):
|
||||
raise ValueError("Regional attention region entries must be canonical.")
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class BatchedRegionalAttentionContexts:
|
||||
"""Retain chunk-major base and per-region active-entry model contexts."""
|
||||
|
||||
latent_batch_size: int
|
||||
chunks: tuple[RegionalAttentionChunkBatch, ...]
|
||||
base_context: torch.Tensor
|
||||
regions: tuple[BatchedRegionalAttentionRegion, ...]
|
||||
base_value_multiplier: torch.Tensor | None = None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Validate complete chunk and tensor alignment."""
|
||||
|
||||
if isinstance(self.latent_batch_size, bool) or not isinstance(
|
||||
self.latent_batch_size, int
|
||||
):
|
||||
raise TypeError("Regional attention latent_batch_size must be an integer.")
|
||||
if self.latent_batch_size < 1:
|
||||
raise ValueError("Regional attention latent_batch_size must be positive.")
|
||||
if not isinstance(self.chunks, tuple) or not self.chunks:
|
||||
raise ValueError("Regional attention batch requires at least one chunk.")
|
||||
if any(
|
||||
not isinstance(chunk, RegionalAttentionChunkBatch) for chunk in self.chunks
|
||||
):
|
||||
raise TypeError("Regional attention batch contains an invalid chunk.")
|
||||
expected_start = 0
|
||||
for chunk_index, chunk in enumerate(self.chunks):
|
||||
if chunk.chunk_index != chunk_index or chunk.batch_start != expected_start:
|
||||
raise ValueError(
|
||||
"Regional attention chunks must be contiguous and ordered."
|
||||
)
|
||||
if chunk.batch_stop - chunk.batch_start != self.latent_batch_size:
|
||||
raise ValueError(
|
||||
"Regional attention chunk size must match latent batch."
|
||||
)
|
||||
expected_start = chunk.batch_stop
|
||||
_validate_aligned_context(
|
||||
self.base_context,
|
||||
expected_batch=expected_start,
|
||||
name="base",
|
||||
)
|
||||
_validate_value_multiplier(
|
||||
self.base_value_multiplier,
|
||||
self.base_context,
|
||||
name="base",
|
||||
)
|
||||
if not isinstance(self.regions, tuple):
|
||||
raise TypeError("Regional attention regions must be a tuple.")
|
||||
if tuple(region.region_index for region in self.regions) != tuple(
|
||||
range(len(self.regions))
|
||||
):
|
||||
raise ValueError("Regional attention regions must use canonical order.")
|
||||
for region in self.regions:
|
||||
for entry in region.entries:
|
||||
_validate_aligned_context(
|
||||
entry.context,
|
||||
expected_batch=expected_start,
|
||||
name=f"region {region.region_index} entry {entry.entry_index}",
|
||||
)
|
||||
if entry.context.shape[1:] != self.base_context.shape[1:]:
|
||||
raise ValueError(
|
||||
"Regional attention context sequence shapes must match."
|
||||
)
|
||||
if (
|
||||
entry.context.device != self.base_context.device
|
||||
or entry.context.dtype != self.base_context.dtype
|
||||
):
|
||||
raise ValueError(
|
||||
"Regional attention context device and dtype must match."
|
||||
)
|
||||
|
||||
|
||||
def _validate_aligned_context(
|
||||
context: object,
|
||||
*,
|
||||
name: str,
|
||||
expected_batch: int | None = None,
|
||||
) -> None:
|
||||
"""Validate one finite floating BxSxD context tensor."""
|
||||
|
||||
if not isinstance(context, torch.Tensor):
|
||||
raise TypeError(f"Regional attention {name} context must be a tensor.")
|
||||
if context.ndim != 3 or (
|
||||
expected_batch is not None and int(context.shape[0]) != expected_batch
|
||||
):
|
||||
raise ValueError(
|
||||
f"Regional attention {name} context has an invalid aligned batch."
|
||||
)
|
||||
if not context.is_floating_point() or not bool(
|
||||
torch.isfinite(context).all().item()
|
||||
):
|
||||
raise ValueError(
|
||||
f"Regional attention {name} context must contain finite floating values."
|
||||
)
|
||||
|
||||
|
||||
def _validate_value_multiplier(
|
||||
multiplier: object,
|
||||
context: torch.Tensor,
|
||||
*,
|
||||
name: str,
|
||||
) -> None:
|
||||
"""Validate one optional value multiplier against its aligned context."""
|
||||
|
||||
if multiplier is None:
|
||||
return
|
||||
if (
|
||||
not isinstance(multiplier, torch.Tensor)
|
||||
or multiplier.shape != (*context.shape[:2], 1)
|
||||
or not multiplier.is_floating_point()
|
||||
or multiplier.device != context.device
|
||||
or multiplier.dtype != context.dtype
|
||||
or not bool(torch.isfinite(multiplier).all().item())
|
||||
):
|
||||
raise ValueError(
|
||||
f"Regional attention {name} value multiplier must be a finite "
|
||||
"floating BxSx1 tensor aligned with its context."
|
||||
)
|
||||
@@ -0,0 +1,15 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Own the immutable spatial execution vocabulary for regional attention."""
|
||||
|
||||
from enum import StrEnum
|
||||
|
||||
|
||||
class RegionalAttentionExecutionMode(StrEnum):
|
||||
"""Identify how one prepared regional model traverses the latent canvas."""
|
||||
|
||||
FULL = "full-context"
|
||||
TILED = "tiled"
|
||||
CONTEXTUAL = "Contextual"
|
||||
@@ -0,0 +1,220 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Select exact active processed Attention Coupling entries for one sigma."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from uuid import UUID
|
||||
|
||||
from .conditioning_schedule_selection import (
|
||||
CONDITIONING_SCHEDULE_SELECTION_POLICY,
|
||||
ConditioningScheduleSelectionPolicy,
|
||||
normalize_conditioning_sigma,
|
||||
)
|
||||
from .processed_regional_attention import (
|
||||
ProcessedRegionalAttentionBranch,
|
||||
ProcessedRegionalAttentionEntry,
|
||||
ProcessedRegionalAttentionPlan,
|
||||
)
|
||||
from .regional_attention import (
|
||||
RegionalAttentionBranch,
|
||||
require_non_negative_regional_attention_index,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ActiveProcessedRegionalAttentionChunk:
|
||||
"""Bind one Comfy chunk UUID to its active base and regional entries."""
|
||||
|
||||
chunk_index: int
|
||||
branch: RegionalAttentionBranch
|
||||
base_entry: ProcessedRegionalAttentionEntry
|
||||
regional_entries: tuple[
|
||||
tuple[ProcessedRegionalAttentionEntry, ...] | None,
|
||||
...,
|
||||
]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Validate canonical immutable active selection state."""
|
||||
|
||||
require_non_negative_regional_attention_index(
|
||||
self.chunk_index,
|
||||
name="chunk_index",
|
||||
)
|
||||
if not isinstance(self.branch, RegionalAttentionBranch):
|
||||
raise TypeError("Regional attention chunk branch has an invalid type.")
|
||||
if not isinstance(self.base_entry, ProcessedRegionalAttentionEntry):
|
||||
raise TypeError("Regional attention chunk base entry has an invalid type.")
|
||||
if not isinstance(self.regional_entries, tuple):
|
||||
raise TypeError("Regional attention chunk regions must be a tuple.")
|
||||
for entries in self.regional_entries:
|
||||
if entries is not None and (
|
||||
not isinstance(entries, tuple)
|
||||
or any(
|
||||
not isinstance(entry, ProcessedRegionalAttentionEntry)
|
||||
for entry in entries
|
||||
)
|
||||
):
|
||||
raise TypeError(
|
||||
"Regional attention chunk contains invalid active entries."
|
||||
)
|
||||
|
||||
|
||||
class RegionalAttentionSelectionService:
|
||||
"""Match Comfy chunk UUIDs to active base and regional entry banks."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
schedule_policy: ConditioningScheduleSelectionPolicy | None = None,
|
||||
) -> None:
|
||||
"""Retain the focused schedule policy collaborator."""
|
||||
|
||||
self._schedule_policy = schedule_policy or ConditioningScheduleSelectionPolicy()
|
||||
|
||||
def select_chunks(
|
||||
self,
|
||||
plan: ProcessedRegionalAttentionPlan,
|
||||
*,
|
||||
cond_or_uncond: object,
|
||||
conditioning_uuids: object,
|
||||
sigma: float,
|
||||
) -> tuple[ActiveProcessedRegionalAttentionChunk, ...]:
|
||||
"""Select exact active entries in Comfy's supplied UUID order."""
|
||||
|
||||
if not isinstance(plan, ProcessedRegionalAttentionPlan):
|
||||
raise TypeError("Regional attention selection requires a processed plan.")
|
||||
selectors = _selectors(cond_or_uncond)
|
||||
identities = _conditioning_uuids(conditioning_uuids)
|
||||
if len(selectors) != len(identities):
|
||||
raise ValueError(
|
||||
"Regional attention selectors and UUIDs must have equal lengths."
|
||||
)
|
||||
current_sigma = normalize_conditioning_sigma(sigma)
|
||||
return tuple(
|
||||
self._select_chunk(
|
||||
plan,
|
||||
chunk_index=chunk_index,
|
||||
selector=selector,
|
||||
conditioning_uuid=conditioning_uuid,
|
||||
sigma=current_sigma,
|
||||
)
|
||||
for chunk_index, (selector, conditioning_uuid) in enumerate(
|
||||
zip(selectors, identities, strict=True)
|
||||
)
|
||||
)
|
||||
|
||||
def _select_chunk(
|
||||
self,
|
||||
plan: ProcessedRegionalAttentionPlan,
|
||||
*,
|
||||
chunk_index: int,
|
||||
selector: object,
|
||||
conditioning_uuid: UUID,
|
||||
sigma: float,
|
||||
) -> ActiveProcessedRegionalAttentionChunk:
|
||||
"""Resolve one branch UUID and all regional schedules atomically."""
|
||||
|
||||
branch, contexts = _branch(plan, selector=selector, chunk_index=chunk_index)
|
||||
matching_base = tuple(
|
||||
entry
|
||||
for entry in contexts.base_context.entries
|
||||
if entry.uuid is conditioning_uuid
|
||||
)
|
||||
if len(matching_base) != 1:
|
||||
raise ValueError(
|
||||
f"Regional attention {branch.value} chunk {chunk_index} UUID "
|
||||
"does not identify exactly one processed base entry."
|
||||
)
|
||||
base_entry = matching_base[0]
|
||||
if not self._schedule_policy.is_active(base_entry.schedule, sigma=sigma):
|
||||
raise ValueError(
|
||||
f"Regional attention {branch.value} chunk {chunk_index} UUID is "
|
||||
f"inactive at sigma {sigma}."
|
||||
)
|
||||
return ActiveProcessedRegionalAttentionChunk(
|
||||
chunk_index=chunk_index,
|
||||
branch=branch,
|
||||
base_entry=base_entry,
|
||||
regional_entries=tuple(
|
||||
self._regional_entries(contexts, region_index, sigma=sigma)
|
||||
for region_index in range(plan.mask_bank.region_count)
|
||||
),
|
||||
)
|
||||
|
||||
def _regional_entries(
|
||||
self,
|
||||
branch: ProcessedRegionalAttentionBranch,
|
||||
region_index: int,
|
||||
*,
|
||||
sigma: float,
|
||||
) -> tuple[ProcessedRegionalAttentionEntry, ...] | None:
|
||||
"""Distinguish absent regions from authored regions with no active entry."""
|
||||
|
||||
if region_index >= len(branch.regional_contexts):
|
||||
return None
|
||||
return self._active_entries(
|
||||
branch.regional_contexts[region_index].entries,
|
||||
sigma=sigma,
|
||||
)
|
||||
|
||||
def _active_entries(
|
||||
self,
|
||||
entries: tuple[ProcessedRegionalAttentionEntry, ...],
|
||||
*,
|
||||
sigma: float,
|
||||
) -> tuple[ProcessedRegionalAttentionEntry, ...]:
|
||||
"""Retain every active entry in authored order at one finite sigma."""
|
||||
|
||||
return tuple(
|
||||
entry
|
||||
for entry in entries
|
||||
if self._schedule_policy.is_active(entry.schedule, sigma=sigma)
|
||||
)
|
||||
|
||||
|
||||
def _branch(
|
||||
plan: ProcessedRegionalAttentionPlan,
|
||||
*,
|
||||
selector: object,
|
||||
chunk_index: int,
|
||||
) -> tuple[RegionalAttentionBranch, ProcessedRegionalAttentionBranch]:
|
||||
"""Narrow one Comfy branch selector without accepting booleans."""
|
||||
|
||||
if isinstance(selector, bool) or not isinstance(selector, int):
|
||||
raise TypeError(
|
||||
f"cond_or_uncond selector {chunk_index} must be integer 0 or 1."
|
||||
)
|
||||
if selector == 0:
|
||||
return RegionalAttentionBranch.POSITIVE, plan.positive
|
||||
if selector == 1:
|
||||
return RegionalAttentionBranch.NEGATIVE, plan.negative
|
||||
raise ValueError(
|
||||
f"cond_or_uncond selector {chunk_index} must be 0 or 1; observed {selector}."
|
||||
)
|
||||
|
||||
|
||||
def _selectors(value: object) -> tuple[object, ...]:
|
||||
"""Require Comfy's ordered selector container."""
|
||||
|
||||
if not isinstance(value, list | tuple):
|
||||
raise TypeError("cond_or_uncond must be a list or tuple of chunk selectors.")
|
||||
return tuple(value)
|
||||
|
||||
|
||||
def _conditioning_uuids(value: object) -> tuple[UUID, ...]:
|
||||
"""Require exact Comfy UUID objects for every supplied chunk."""
|
||||
|
||||
if not isinstance(value, list | tuple):
|
||||
raise TypeError("Conditioning UUIDs must be a list or tuple.")
|
||||
identities = tuple(value)
|
||||
if any(not isinstance(identity, UUID) for identity in identities):
|
||||
raise TypeError("Every conditioning identity must be a Comfy UUID.")
|
||||
return identities
|
||||
|
||||
|
||||
REGIONAL_ATTENTION_SELECTION_SERVICE = RegionalAttentionSelectionService(
|
||||
CONDITIONING_SCHEDULE_SELECTION_POLICY
|
||||
)
|
||||
@@ -0,0 +1,252 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Normalize regional attention weights and blend ordered branch outputs."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
|
||||
import torch
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RegionalAttentionWeights:
|
||||
"""Hold raw base, region, and denominator weights on one query grid."""
|
||||
|
||||
base: torch.Tensor
|
||||
regions: torch.Tensor
|
||||
denominator: torch.Tensor
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require consistent finite non-negative weighting tensors."""
|
||||
|
||||
for name, tensor in (
|
||||
("Base", self.base),
|
||||
("Region", self.regions),
|
||||
("Denominator", self.denominator),
|
||||
):
|
||||
if not isinstance(tensor, torch.Tensor):
|
||||
raise TypeError(f"{name} attention weights must be a torch.Tensor.")
|
||||
if not tensor.is_floating_point():
|
||||
raise TypeError(
|
||||
f"{name} attention weights must use a floating-point dtype."
|
||||
)
|
||||
if not bool(torch.isfinite(tensor).all()):
|
||||
raise ValueError(f"{name} attention weights must be finite.")
|
||||
if not bool((tensor >= 0.0).all()):
|
||||
raise ValueError(f"{name} attention weights must be non-negative.")
|
||||
if self.base.ndim < 1:
|
||||
raise ValueError("Base attention weights require a query-grid dimension.")
|
||||
if self.regions.ndim != self.base.ndim + 1:
|
||||
raise ValueError(
|
||||
"Region attention weights require a leading region dimension."
|
||||
)
|
||||
if int(self.regions.shape[0]) < 1:
|
||||
raise ValueError("Region attention weights require at least one region.")
|
||||
if tuple(self.regions.shape[1:]) != tuple(self.base.shape):
|
||||
raise ValueError("Region attention weights must match the base query grid.")
|
||||
if self.denominator.shape != self.base.shape:
|
||||
raise ValueError("Attention denominator must match the base query grid.")
|
||||
if not (
|
||||
self.base.dtype == self.regions.dtype == self.denominator.dtype
|
||||
and self.base.device == self.regions.device == self.denominator.device
|
||||
):
|
||||
raise ValueError(
|
||||
"Base, region, and denominator weights must share dtype and device."
|
||||
)
|
||||
if not bool((self.denominator > 0.0).all()):
|
||||
raise ValueError("Attention denominator must be strictly positive.")
|
||||
|
||||
@property
|
||||
def normalized_base(self) -> torch.Tensor:
|
||||
"""Return normalized base-complement weights."""
|
||||
|
||||
return self.base / self.denominator
|
||||
|
||||
@property
|
||||
def normalized_regions(self) -> torch.Tensor:
|
||||
"""Return normalized ordered regional weights."""
|
||||
|
||||
return self.regions / self.denominator.unsqueeze(0)
|
||||
|
||||
|
||||
class RegionalAttentionWeightingPolicy:
|
||||
"""Own Comfy-compatible base complement and normalized branch composition."""
|
||||
|
||||
def weights(
|
||||
self,
|
||||
masks: torch.Tensor,
|
||||
*,
|
||||
region_strengths: tuple[float, ...],
|
||||
epsilon: float = 1e-6,
|
||||
) -> RegionalAttentionWeights:
|
||||
"""Return raw weighting terms for ordered region-first query masks."""
|
||||
|
||||
self._validate_mask_structure(masks)
|
||||
strengths = self._validate_strengths(
|
||||
region_strengths,
|
||||
region_count=int(masks.shape[0]),
|
||||
)
|
||||
if (
|
||||
isinstance(epsilon, bool)
|
||||
or not isinstance(epsilon, int | float)
|
||||
or not math.isfinite(float(epsilon))
|
||||
or epsilon <= 0.0
|
||||
):
|
||||
raise ValueError("Regional attention epsilon must be finite and positive.")
|
||||
self._validate_mask_values(masks)
|
||||
|
||||
strength_shape = (len(strengths),) + (1,) * (masks.ndim - 1)
|
||||
strength_tensor = masks.new_tensor(strengths).reshape(strength_shape)
|
||||
region_weights = masks.clamp(0.0, 1.0) * strength_tensor
|
||||
region_sum = region_weights.sum(dim=0)
|
||||
base_weight = torch.relu(1.0 - region_sum)
|
||||
denominator = (base_weight + region_sum).clamp_min(float(epsilon))
|
||||
return RegionalAttentionWeights(
|
||||
base=base_weight,
|
||||
regions=region_weights,
|
||||
denominator=denominator,
|
||||
)
|
||||
|
||||
def blend(
|
||||
self,
|
||||
*,
|
||||
weights: RegionalAttentionWeights,
|
||||
base_output: torch.Tensor,
|
||||
regional_outputs: torch.Tensor,
|
||||
) -> torch.Tensor:
|
||||
"""Blend one base and ordered regional outputs over their query grid."""
|
||||
|
||||
self._validate_outputs(
|
||||
weights=weights,
|
||||
base_output=base_output,
|
||||
regional_outputs=regional_outputs,
|
||||
)
|
||||
feature_dimensions = base_output.ndim - weights.base.ndim
|
||||
feature_shape = (1,) * feature_dimensions
|
||||
base_weight = weights.base.reshape((*weights.base.shape, *feature_shape)).to(
|
||||
dtype=base_output.dtype
|
||||
)
|
||||
region_weights = weights.regions.reshape(
|
||||
(*weights.regions.shape, *feature_shape)
|
||||
).to(dtype=base_output.dtype)
|
||||
denominator = weights.denominator.reshape(
|
||||
(*weights.denominator.shape, *feature_shape)
|
||||
).to(dtype=base_output.dtype)
|
||||
numerator = base_weight * base_output + (region_weights * regional_outputs).sum(
|
||||
dim=0
|
||||
)
|
||||
blended = numerator / denominator
|
||||
if not bool(torch.isfinite(blended).all()):
|
||||
raise ValueError(
|
||||
"Blended regional attention output contains non-finite values."
|
||||
)
|
||||
return blended
|
||||
|
||||
@staticmethod
|
||||
def _validate_mask_structure(masks: torch.Tensor) -> None:
|
||||
"""Validate mask type and shape without inspecting device values."""
|
||||
|
||||
if not isinstance(masks, torch.Tensor):
|
||||
raise TypeError("Regional attention masks must be a torch.Tensor.")
|
||||
if masks.ndim < 2:
|
||||
raise ValueError(
|
||||
"Regional attention masks require region and query-grid dimensions."
|
||||
)
|
||||
if int(masks.shape[0]) < 1 or any(int(size) < 1 for size in masks.shape[1:]):
|
||||
raise ValueError(
|
||||
"Regional attention masks require non-empty region and query grids."
|
||||
)
|
||||
if not masks.is_floating_point():
|
||||
raise TypeError("Regional attention masks must use a floating-point dtype.")
|
||||
|
||||
@staticmethod
|
||||
def _validate_mask_values(masks: torch.Tensor) -> None:
|
||||
"""Reject non-finite mask values before constructing strength tensors."""
|
||||
|
||||
if not bool(torch.isfinite(masks).all()):
|
||||
raise ValueError("Regional attention masks must contain finite values.")
|
||||
|
||||
@staticmethod
|
||||
def _validate_strengths(
|
||||
strengths: tuple[float, ...],
|
||||
*,
|
||||
region_count: int,
|
||||
) -> tuple[float, ...]:
|
||||
"""Validate ordered immutable regional strengths before tensor creation."""
|
||||
|
||||
if not isinstance(strengths, tuple):
|
||||
raise TypeError("Regional attention strengths must be an immutable tuple.")
|
||||
if len(strengths) != region_count:
|
||||
raise ValueError(
|
||||
"Regional attention strength count must match the mask region count."
|
||||
)
|
||||
normalized: list[float] = []
|
||||
for index, strength in enumerate(strengths):
|
||||
if isinstance(strength, bool) or not isinstance(strength, int | float):
|
||||
raise TypeError(
|
||||
f"Regional attention strength {index} must be a real number."
|
||||
)
|
||||
value = float(strength)
|
||||
if not math.isfinite(value) or value < 0.0:
|
||||
raise ValueError(
|
||||
f"Regional attention strength {index} must be finite and "
|
||||
"non-negative."
|
||||
)
|
||||
normalized.append(value)
|
||||
return tuple(normalized)
|
||||
|
||||
@staticmethod
|
||||
def _validate_outputs(
|
||||
*,
|
||||
weights: RegionalAttentionWeights,
|
||||
base_output: torch.Tensor,
|
||||
regional_outputs: torch.Tensor,
|
||||
) -> None:
|
||||
"""Validate branch output ordering, grids, features, and tensor state."""
|
||||
|
||||
if not isinstance(weights, RegionalAttentionWeights):
|
||||
raise TypeError("Regional blend weights must be RegionalAttentionWeights.")
|
||||
if not isinstance(base_output, torch.Tensor) or not isinstance(
|
||||
regional_outputs, torch.Tensor
|
||||
):
|
||||
raise TypeError(
|
||||
"Regional attention branch outputs must be torch.Tensor values."
|
||||
)
|
||||
if (
|
||||
not base_output.is_floating_point()
|
||||
or not regional_outputs.is_floating_point()
|
||||
):
|
||||
raise TypeError(
|
||||
"Regional attention branch outputs must use floating-point dtypes."
|
||||
)
|
||||
if base_output.dtype != regional_outputs.dtype:
|
||||
raise ValueError("Regional attention branch output dtypes must match.")
|
||||
if base_output.device != regional_outputs.device:
|
||||
raise ValueError("Regional attention branch output devices must match.")
|
||||
expected_regional_shape = (int(weights.regions.shape[0]), *base_output.shape)
|
||||
if regional_outputs.shape != expected_regional_shape:
|
||||
raise ValueError(
|
||||
"Regional outputs must contain one ordered branch per region with "
|
||||
"the complete base output shape."
|
||||
)
|
||||
query_dimensions = weights.base.ndim
|
||||
if base_output.ndim < query_dimensions or tuple(
|
||||
base_output.shape[:query_dimensions]
|
||||
) != tuple(weights.base.shape):
|
||||
raise ValueError(
|
||||
"Regional attention outputs must begin with the weighting query grid."
|
||||
)
|
||||
if weights.base.device != base_output.device:
|
||||
raise ValueError(
|
||||
"Regional weights and branch outputs must share one device."
|
||||
)
|
||||
if not bool(torch.isfinite(base_output).all()) or not bool(
|
||||
torch.isfinite(regional_outputs).all()
|
||||
):
|
||||
raise ValueError(
|
||||
"Regional attention branch outputs must contain finite values."
|
||||
)
|
||||
@@ -0,0 +1,105 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Own Comfy-equivalent within-region conditioning-output combination."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
|
||||
import torch
|
||||
|
||||
|
||||
class RegionalConditioningOutputCombiner:
|
||||
"""Combine ordered active-entry outputs with native Comfy strength semantics."""
|
||||
|
||||
def combine(
|
||||
self,
|
||||
outputs: tuple[torch.Tensor, ...],
|
||||
*,
|
||||
strengths: tuple[tuple[float, ...], ...],
|
||||
) -> torch.Tensor:
|
||||
"""Return the ordered strength-weighted output normalized like Comfy."""
|
||||
|
||||
if not isinstance(outputs, tuple) or not outputs:
|
||||
raise ValueError("Regional conditioning combination requires outputs.")
|
||||
if not isinstance(strengths, tuple) or len(strengths) != len(outputs):
|
||||
raise ValueError(
|
||||
"Regional conditioning strengths must align with ordered outputs."
|
||||
)
|
||||
authority = outputs[0]
|
||||
if not isinstance(authority, torch.Tensor) or authority.ndim < 1:
|
||||
raise TypeError("Regional conditioning output must be a tensor batch.")
|
||||
if len(outputs) == 1:
|
||||
entry_strengths = strengths[0]
|
||||
self._validate_entry(
|
||||
authority,
|
||||
entry_strengths,
|
||||
authority=authority,
|
||||
entry_index=0,
|
||||
)
|
||||
if all(strength == 1.0 for strength in entry_strengths):
|
||||
return authority
|
||||
weighted = torch.zeros_like(authority)
|
||||
counts = torch.ones_like(authority) * 1e-37
|
||||
weight_shape = (int(authority.shape[0]),) + (1,) * (authority.ndim - 1)
|
||||
active_rows = torch.zeros(
|
||||
weight_shape,
|
||||
dtype=torch.bool,
|
||||
device=authority.device,
|
||||
)
|
||||
for entry_index, (output, entry_strengths) in enumerate(
|
||||
zip(outputs, strengths, strict=True)
|
||||
):
|
||||
self._validate_entry(
|
||||
output,
|
||||
entry_strengths,
|
||||
authority=authority,
|
||||
entry_index=entry_index,
|
||||
)
|
||||
weights = authority.new_tensor(entry_strengths).reshape(weight_shape)
|
||||
weighted += output * weights
|
||||
counts += weights
|
||||
active_rows |= weights.ne(0)
|
||||
denominator = torch.where(active_rows, counts, torch.ones_like(counts))
|
||||
return weighted / denominator
|
||||
|
||||
@staticmethod
|
||||
def _validate_entry(
|
||||
output: object,
|
||||
strengths: object,
|
||||
*,
|
||||
authority: torch.Tensor,
|
||||
entry_index: int,
|
||||
) -> None:
|
||||
"""Require exact output structure and one finite strength per sample."""
|
||||
|
||||
if not isinstance(output, torch.Tensor):
|
||||
raise TypeError(
|
||||
f"Regional conditioning output {entry_index} must be a tensor."
|
||||
)
|
||||
if (
|
||||
output.shape != authority.shape
|
||||
or output.device != authority.device
|
||||
or output.dtype != authority.dtype
|
||||
):
|
||||
raise ValueError(
|
||||
f"Regional conditioning output {entry_index} must match the first "
|
||||
"output shape, device, and dtype."
|
||||
)
|
||||
if not isinstance(strengths, tuple) or len(strengths) != int(
|
||||
authority.shape[0]
|
||||
):
|
||||
raise ValueError(
|
||||
f"Regional conditioning output {entry_index} strengths must match "
|
||||
"the output batch."
|
||||
)
|
||||
for strength in strengths:
|
||||
if isinstance(strength, bool) or not isinstance(strength, int | float):
|
||||
raise TypeError("Regional conditioning strengths must be real numbers.")
|
||||
if not math.isfinite(float(strength)):
|
||||
raise ValueError("Regional conditioning strengths must be finite.")
|
||||
|
||||
|
||||
REGIONAL_CONDITIONING_OUTPUT_COMBINER = RegionalConditioningOutputCombiner()
|
||||
@@ -0,0 +1,145 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Define immutable regional feature requests, capabilities, and admissions."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from enum import StrEnum
|
||||
|
||||
from .regional_model_capabilities import RegionalModelCapabilities
|
||||
|
||||
|
||||
class RegionalFeature(StrEnum):
|
||||
"""Identify one independently admitted regional sampling feature."""
|
||||
|
||||
FULL_CONTEXT_MASKED_CONDITIONING = "full_context_masked_conditioning"
|
||||
ATTENTION_COUPLING = "attention_coupling"
|
||||
SPATIAL_MODEL_PATCH = "spatial_model_patch"
|
||||
CONTROL = "control"
|
||||
GLIGEN = "gligen"
|
||||
REFERENCE_LATENTS = "reference_latents"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RegionalFeatureRequest:
|
||||
"""Describe the complete immutable regional feature intent for one sample."""
|
||||
|
||||
features: frozenset[RegionalFeature] = frozenset()
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require an immutable set containing only typed regional features."""
|
||||
|
||||
_validate_features(self.features, value_name="Regional feature request")
|
||||
|
||||
def with_feature(self, feature: RegionalFeature) -> RegionalFeatureRequest:
|
||||
"""Return a new request containing one additional typed feature."""
|
||||
|
||||
if not isinstance(feature, RegionalFeature):
|
||||
raise TypeError("Requested regional feature must be a RegionalFeature.")
|
||||
return RegionalFeatureRequest(self.features | {feature})
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RegionalSamplerCapabilities:
|
||||
"""Describe the complete regional feature set implemented by one sampler."""
|
||||
|
||||
features: frozenset[RegionalFeature]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require an immutable set containing only typed regional features."""
|
||||
|
||||
_validate_features(self.features, value_name="Regional sampler capabilities")
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RegionalCapabilityAdmission:
|
||||
"""Record one complete successful request admission for downstream use."""
|
||||
|
||||
request: RegionalFeatureRequest
|
||||
admitted_features: frozenset[RegionalFeature]
|
||||
model_capabilities: RegionalModelCapabilities | None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Reject partial, untyped, or model-incomplete admission state."""
|
||||
|
||||
if not isinstance(self.request, RegionalFeatureRequest):
|
||||
raise TypeError(
|
||||
"Regional admission request must be RegionalFeatureRequest."
|
||||
)
|
||||
_validate_features(
|
||||
self.admitted_features,
|
||||
value_name="Admitted regional features",
|
||||
)
|
||||
if self.admitted_features != self.request.features:
|
||||
raise ValueError("Regional capability admission cannot be partial.")
|
||||
if self.model_capabilities is not None and not isinstance(
|
||||
self.model_capabilities,
|
||||
RegionalModelCapabilities,
|
||||
):
|
||||
raise TypeError(
|
||||
"Regional admission model capabilities must be "
|
||||
"RegionalModelCapabilities."
|
||||
)
|
||||
if self.admitted_features & MODEL_DEPENDENT_REGIONAL_FEATURES:
|
||||
if self.model_capabilities is None:
|
||||
raise ValueError(
|
||||
"Model-dependent regional features require model capabilities."
|
||||
)
|
||||
|
||||
def supports(self, feature: RegionalFeature) -> bool:
|
||||
"""Return whether one typed feature was admitted for this sample."""
|
||||
|
||||
if not isinstance(feature, RegionalFeature):
|
||||
raise TypeError("Regional feature query must be a RegionalFeature.")
|
||||
return feature in self.admitted_features
|
||||
|
||||
|
||||
MODEL_DEPENDENT_REGIONAL_FEATURES = frozenset(
|
||||
{
|
||||
RegionalFeature.ATTENTION_COUPLING,
|
||||
RegionalFeature.SPATIAL_MODEL_PATCH,
|
||||
RegionalFeature.CONTROL,
|
||||
RegionalFeature.GLIGEN,
|
||||
RegionalFeature.REFERENCE_LATENTS,
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _validate_features(
|
||||
features: frozenset[RegionalFeature],
|
||||
*,
|
||||
value_name: str,
|
||||
) -> None:
|
||||
"""Validate one immutable typed regional feature set."""
|
||||
|
||||
if not isinstance(features, frozenset):
|
||||
raise TypeError(f"{value_name} must use an immutable frozenset.")
|
||||
if not all(isinstance(feature, RegionalFeature) for feature in features):
|
||||
raise TypeError(f"{value_name} must contain RegionalFeature values.")
|
||||
|
||||
|
||||
EMPTY_REGIONAL_FEATURE_REQUEST = RegionalFeatureRequest()
|
||||
EMPTY_REGIONAL_CAPABILITY_ADMISSION = RegionalCapabilityAdmission(
|
||||
request=EMPTY_REGIONAL_FEATURE_REQUEST,
|
||||
admitted_features=frozenset(),
|
||||
model_capabilities=None,
|
||||
)
|
||||
CONTEXTUAL_DIFFUSION_REGIONAL_SAMPLER_CAPABILITIES = RegionalSamplerCapabilities(
|
||||
frozenset(
|
||||
{
|
||||
RegionalFeature.FULL_CONTEXT_MASKED_CONDITIONING,
|
||||
RegionalFeature.ATTENTION_COUPLING,
|
||||
}
|
||||
)
|
||||
)
|
||||
TILED_DIFFUSION_REGIONAL_SAMPLER_CAPABILITIES = RegionalSamplerCapabilities(
|
||||
frozenset(
|
||||
{
|
||||
RegionalFeature.FULL_CONTEXT_MASKED_CONDITIONING,
|
||||
RegionalFeature.ATTENTION_COUPLING,
|
||||
}
|
||||
)
|
||||
)
|
||||
@@ -0,0 +1,161 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Own immutable regional model-side LoRA composition plans."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
from enum import StrEnum
|
||||
|
||||
|
||||
class RegionalLoraBranch(StrEnum):
|
||||
"""Identify the conditioning branch that owns one regional adapter use."""
|
||||
|
||||
POSITIVE = "positive"
|
||||
NEGATIVE = "negative"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegionalLoraAdapterIdentity:
|
||||
"""Retain the caller-supplied stable identity of one LoRA artifact."""
|
||||
|
||||
value: str
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Reject identities that cannot distinguish an adapter."""
|
||||
|
||||
if not isinstance(self.value, str) or not self.value.strip():
|
||||
raise ValueError(
|
||||
"Regional LoRA adapter identity must be a non-empty string."
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegionalLoraScheduleBoundary:
|
||||
"""Retain one ordered Comfy HookKeyframe boundary exactly."""
|
||||
|
||||
start_percent: float
|
||||
start_sigma: float
|
||||
strength_multiplier: float
|
||||
guarantee_steps: int
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Validate one finite normalized schedule boundary."""
|
||||
|
||||
_require_finite_float(self.start_percent, name="start_percent")
|
||||
if not 0.0 <= self.start_percent <= 1.0:
|
||||
raise ValueError("Regional LoRA schedule start_percent must be in [0, 1].")
|
||||
_require_finite_float(self.start_sigma, name="start_sigma")
|
||||
if self.start_sigma < 0.0:
|
||||
raise ValueError("Regional LoRA schedule start_sigma must be non-negative.")
|
||||
_require_finite_float(
|
||||
self.strength_multiplier,
|
||||
name="strength_multiplier",
|
||||
)
|
||||
if isinstance(self.guarantee_steps, bool) or not isinstance(
|
||||
self.guarantee_steps, int
|
||||
):
|
||||
raise TypeError("Regional LoRA guarantee_steps must be an integer.")
|
||||
if self.guarantee_steps < 0:
|
||||
raise ValueError("Regional LoRA guarantee_steps must be non-negative.")
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegionalLoraAdapterPlan:
|
||||
"""Describe one ordered regional use of a model-side LoRA adapter."""
|
||||
|
||||
adapter_identity: RegionalLoraAdapterIdentity
|
||||
composition_index: int
|
||||
region_index: int
|
||||
branch: RegionalLoraBranch
|
||||
model_strength: float
|
||||
schedule: tuple[RegionalLoraScheduleBoundary, ...]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Validate adapter ownership, strength, and ordered schedule."""
|
||||
|
||||
if not isinstance(self.adapter_identity, RegionalLoraAdapterIdentity):
|
||||
raise TypeError("Regional LoRA adapter_identity has an invalid type.")
|
||||
_require_non_negative_index(self.composition_index, name="composition_index")
|
||||
_require_non_negative_index(self.region_index, name="region_index")
|
||||
if not isinstance(self.branch, RegionalLoraBranch):
|
||||
raise TypeError("Regional LoRA branch has an invalid type.")
|
||||
_require_finite_float(self.model_strength, name="model_strength")
|
||||
if not isinstance(self.schedule, tuple) or not self.schedule:
|
||||
raise ValueError(
|
||||
"Regional LoRA schedule must contain at least one boundary."
|
||||
)
|
||||
if any(
|
||||
not isinstance(boundary, RegionalLoraScheduleBoundary)
|
||||
for boundary in self.schedule
|
||||
):
|
||||
raise TypeError("Regional LoRA schedule contains an invalid boundary.")
|
||||
starts = tuple(boundary.start_percent for boundary in self.schedule)
|
||||
if starts != tuple(sorted(starts)):
|
||||
raise ValueError("Regional LoRA schedule boundaries must be ordered.")
|
||||
sigmas = tuple(boundary.start_sigma for boundary in self.schedule)
|
||||
if sigmas != tuple(sorted(sigmas, reverse=True)):
|
||||
raise ValueError(
|
||||
"Regional LoRA converted schedule boundaries must be descending."
|
||||
)
|
||||
|
||||
@property
|
||||
def is_time_invariant(self) -> bool:
|
||||
"""Report whether every keyframe retains one effective multiplier."""
|
||||
|
||||
first_multiplier = self.schedule[0].strength_multiplier
|
||||
return all(
|
||||
boundary.strength_multiplier == first_multiplier
|
||||
for boundary in self.schedule[1:]
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegionalLoraPlan:
|
||||
"""Store all adapter uses in authoritative global composition order."""
|
||||
|
||||
adapters: tuple[RegionalLoraAdapterPlan, ...]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require immutable entries with contiguous composition indices."""
|
||||
|
||||
if not isinstance(self.adapters, tuple):
|
||||
raise TypeError("Regional LoRA plan adapters must be a tuple.")
|
||||
if any(
|
||||
not isinstance(adapter, RegionalLoraAdapterPlan)
|
||||
for adapter in self.adapters
|
||||
):
|
||||
raise TypeError("Regional LoRA plan contains an invalid adapter entry.")
|
||||
observed_indices = tuple(adapter.composition_index for adapter in self.adapters)
|
||||
if observed_indices != tuple(range(len(self.adapters))):
|
||||
raise ValueError(
|
||||
"Regional LoRA composition indices must be contiguous and ordered."
|
||||
)
|
||||
|
||||
@property
|
||||
def is_time_invariant(self) -> bool:
|
||||
"""Report whether every regional adapter retains one effective strength."""
|
||||
|
||||
return all(adapter.is_time_invariant for adapter in self.adapters)
|
||||
|
||||
|
||||
def _require_finite_float(value: object, *, name: str) -> None:
|
||||
"""Require one exact finite floating-point value."""
|
||||
|
||||
if not isinstance(value, float) or not math.isfinite(value):
|
||||
raise TypeError(f"Regional LoRA {name} must be a finite float.")
|
||||
|
||||
|
||||
def _require_non_negative_index(value: object, *, name: str) -> None:
|
||||
"""Require one non-negative integer index."""
|
||||
|
||||
if isinstance(value, bool) or not isinstance(value, int):
|
||||
raise TypeError(f"Regional LoRA {name} must be an integer.")
|
||||
if value < 0:
|
||||
raise ValueError(f"Regional LoRA {name} must be non-negative.")
|
||||
|
||||
|
||||
EMPTY_REGIONAL_LORA_PLAN = RegionalLoraPlan(adapters=())
|
||||
@@ -0,0 +1,75 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Define the immutable full-canvas authority for regional masks."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
import torch
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RegionalMaskBank:
|
||||
"""Hold separate planning and conditioning masks on one latent canvas."""
|
||||
|
||||
planning_masks: torch.Tensor
|
||||
conditioning_masks: torch.Tensor
|
||||
canvas_width: int
|
||||
canvas_height: int
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Reject malformed, divergent, or aliased canonical mask tensors."""
|
||||
|
||||
if self.canvas_width < 1 or self.canvas_height < 1:
|
||||
raise ValueError("Regional mask canvas dimensions must be positive.")
|
||||
self._validate_mask_batch(self.planning_masks, name="Planning")
|
||||
self._validate_mask_batch(self.conditioning_masks, name="Conditioning")
|
||||
if self.planning_masks.shape != self.conditioning_masks.shape:
|
||||
raise ValueError(
|
||||
"Regional planning and conditioning mask shapes must match."
|
||||
)
|
||||
if self.planning_masks.dtype != self.conditioning_masks.dtype:
|
||||
raise ValueError(
|
||||
"Regional planning and conditioning mask dtypes must match."
|
||||
)
|
||||
if self.planning_masks.device != self.conditioning_masks.device:
|
||||
raise ValueError(
|
||||
"Regional planning and conditioning mask devices must match."
|
||||
)
|
||||
if (
|
||||
self.planning_masks.untyped_storage().data_ptr()
|
||||
== self.conditioning_masks.untyped_storage().data_ptr()
|
||||
):
|
||||
raise ValueError(
|
||||
"Regional planning and conditioning masks must not share storage."
|
||||
)
|
||||
|
||||
@property
|
||||
def region_count(self) -> int:
|
||||
"""Return the number of ordered authored regions."""
|
||||
|
||||
return int(self.planning_masks.shape[0])
|
||||
|
||||
def _validate_mask_batch(self, masks: torch.Tensor, *, name: str) -> None:
|
||||
"""Validate one normalized floating-point BHW mask batch."""
|
||||
|
||||
if not isinstance(masks, torch.Tensor):
|
||||
raise TypeError(f"{name} regional masks must be a torch.Tensor.")
|
||||
if masks.ndim != 3:
|
||||
raise ValueError(f"{name} regional masks must use BHW layout.")
|
||||
if int(masks.shape[0]) < 1:
|
||||
raise ValueError(f"{name} regional masks require at least one region.")
|
||||
if tuple(masks.shape[1:]) != (self.canvas_height, self.canvas_width):
|
||||
raise ValueError(
|
||||
f"{name} regional masks must match the full latent canvas "
|
||||
f"{self.canvas_width}x{self.canvas_height}."
|
||||
)
|
||||
if not masks.is_floating_point():
|
||||
raise TypeError(f"{name} regional masks must use a floating-point dtype.")
|
||||
if not bool(torch.isfinite(masks).all()):
|
||||
raise ValueError(f"{name} regional masks must contain only finite values.")
|
||||
if not bool(((masks >= 0.0) & (masks <= 1.0)).all()):
|
||||
raise ValueError(f"{name} regional masks must stay within [0, 1].")
|
||||
@@ -0,0 +1,173 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Define immutable capability values for regional attention backends."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from enum import StrEnum
|
||||
|
||||
|
||||
class RegionalModelFamily(StrEnum):
|
||||
"""Identify a defensively admitted regional model family."""
|
||||
|
||||
ANIMA = "anima"
|
||||
STANDARD_UNET = "standard_unet"
|
||||
|
||||
|
||||
class RegionalAttentionBackend(StrEnum):
|
||||
"""Identify the model-specific attention patch backend."""
|
||||
|
||||
ANIMA_OBJECT_PATCH = "anima_object_patch"
|
||||
UNET_ATTN2_PATCH = "unet_attn2_patch"
|
||||
|
||||
|
||||
class RegionalAttentionTopology(StrEnum):
|
||||
"""Identify the image/context token roles owned by an attention backend."""
|
||||
|
||||
SEPARATE_IMAGE_AND_CONTEXT = "separate_image_and_context"
|
||||
SINGLETON_FRAME_SPATIOTEMPORAL = "singleton_frame_spatiotemporal"
|
||||
|
||||
|
||||
class RegionalLatentLayout(StrEnum):
|
||||
"""Identify the latent rank and temporal layout admitted by a backend."""
|
||||
|
||||
ANIMA_SINGLE_FRAME_BCTHW = "anima_single_frame_bcthw"
|
||||
STANDARD_IMAGE_BCHW = "standard_image_bchw"
|
||||
|
||||
|
||||
class RegionalSpatialPatchSupport(StrEnum):
|
||||
"""Report whether one backend can consume canonical spatial views."""
|
||||
|
||||
FULL_AND_SPATIAL_VIEWS = "full_and_spatial_views"
|
||||
|
||||
|
||||
class RegionalControlGligenPolicy(StrEnum):
|
||||
"""Report control and GLIGEN admission for attention coupling."""
|
||||
|
||||
REJECT = "reject"
|
||||
|
||||
|
||||
class RegionalReferenceLatentPolicy(StrEnum):
|
||||
"""Report reference-latent admission for attention coupling."""
|
||||
|
||||
REJECT = "reject"
|
||||
|
||||
|
||||
class RegionalPatchConflict(StrEnum):
|
||||
"""Identify a patch surface that must be collision-free before mutation."""
|
||||
|
||||
DIFFUSION_MODEL_WRAPPER = "diffusion_model_wrapper"
|
||||
CROSS_ATTENTION_OBJECT_PATCH = "cross_attention_object_patch"
|
||||
ATTN2_INPUT_PATCH = "attn2_input_patch"
|
||||
ATTN2_OUTPUT_PATCH = "attn2_output_patch"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RegionalModelCapabilities:
|
||||
"""Describe one admitted backend and every relevant compatibility policy."""
|
||||
|
||||
model_family: RegionalModelFamily
|
||||
attention_backend: RegionalAttentionBackend
|
||||
attention_topology: RegionalAttentionTopology
|
||||
latent_layout: RegionalLatentLayout
|
||||
spatial_patch_support: RegionalSpatialPatchSupport
|
||||
control_gligen_policy: RegionalControlGligenPolicy
|
||||
reference_latent_policy: RegionalReferenceLatentPolicy
|
||||
known_patch_conflicts: tuple[RegionalPatchConflict, ...]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Reject mutable, duplicate, or internally inconsistent capabilities."""
|
||||
|
||||
enum_fields = (
|
||||
("model family", self.model_family, RegionalModelFamily),
|
||||
("attention backend", self.attention_backend, RegionalAttentionBackend),
|
||||
(
|
||||
"attention topology",
|
||||
self.attention_topology,
|
||||
RegionalAttentionTopology,
|
||||
),
|
||||
("latent layout", self.latent_layout, RegionalLatentLayout),
|
||||
(
|
||||
"spatial patch support",
|
||||
self.spatial_patch_support,
|
||||
RegionalSpatialPatchSupport,
|
||||
),
|
||||
(
|
||||
"control/GLIGEN policy",
|
||||
self.control_gligen_policy,
|
||||
RegionalControlGligenPolicy,
|
||||
),
|
||||
(
|
||||
"reference-latent policy",
|
||||
self.reference_latent_policy,
|
||||
RegionalReferenceLatentPolicy,
|
||||
),
|
||||
)
|
||||
for name, value, enum_type in enum_fields:
|
||||
if not isinstance(value, enum_type):
|
||||
raise TypeError(
|
||||
f"Regional {name} must be a {enum_type.__name__} value."
|
||||
)
|
||||
if not isinstance(self.known_patch_conflicts, tuple):
|
||||
raise TypeError(
|
||||
"Known regional patch conflicts must be an immutable tuple."
|
||||
)
|
||||
if not self.known_patch_conflicts:
|
||||
raise ValueError("Regional capabilities require known patch conflicts.")
|
||||
if not all(
|
||||
isinstance(conflict, RegionalPatchConflict)
|
||||
for conflict in self.known_patch_conflicts
|
||||
):
|
||||
raise TypeError(
|
||||
"Known regional patch conflicts must contain "
|
||||
"RegionalPatchConflict values."
|
||||
)
|
||||
if len(set(self.known_patch_conflicts)) != len(self.known_patch_conflicts):
|
||||
raise ValueError(
|
||||
"Known regional patch conflicts must be unique and ordered."
|
||||
)
|
||||
self._validate_family_contract()
|
||||
|
||||
def _validate_family_contract(self) -> None:
|
||||
"""Require the exact backend, layout, and conflict surface for a family."""
|
||||
|
||||
expected: tuple[
|
||||
RegionalAttentionBackend,
|
||||
RegionalAttentionTopology,
|
||||
RegionalLatentLayout,
|
||||
tuple[RegionalPatchConflict, ...],
|
||||
]
|
||||
if self.model_family is RegionalModelFamily.ANIMA:
|
||||
expected = (
|
||||
RegionalAttentionBackend.ANIMA_OBJECT_PATCH,
|
||||
RegionalAttentionTopology.SINGLETON_FRAME_SPATIOTEMPORAL,
|
||||
RegionalLatentLayout.ANIMA_SINGLE_FRAME_BCTHW,
|
||||
(
|
||||
RegionalPatchConflict.DIFFUSION_MODEL_WRAPPER,
|
||||
RegionalPatchConflict.CROSS_ATTENTION_OBJECT_PATCH,
|
||||
RegionalPatchConflict.ATTN2_INPUT_PATCH,
|
||||
RegionalPatchConflict.ATTN2_OUTPUT_PATCH,
|
||||
),
|
||||
)
|
||||
else:
|
||||
expected = (
|
||||
RegionalAttentionBackend.UNET_ATTN2_PATCH,
|
||||
RegionalAttentionTopology.SEPARATE_IMAGE_AND_CONTEXT,
|
||||
RegionalLatentLayout.STANDARD_IMAGE_BCHW,
|
||||
(
|
||||
RegionalPatchConflict.ATTN2_INPUT_PATCH,
|
||||
RegionalPatchConflict.ATTN2_OUTPUT_PATCH,
|
||||
),
|
||||
)
|
||||
if (
|
||||
self.attention_backend,
|
||||
self.attention_topology,
|
||||
self.latent_layout,
|
||||
self.known_patch_conflicts,
|
||||
) != expected:
|
||||
raise ValueError(
|
||||
"Regional model capabilities do not match the model-family contract."
|
||||
)
|
||||
@@ -0,0 +1,74 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Pure policies for global-first regional prompt pairing."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
|
||||
MAX_REGIONAL_PROMPT_WEIGHT = 1.0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegionalConditioningPair:
|
||||
"""Map one regional conditioning entry to its authored mask index."""
|
||||
|
||||
conditioning_index: int
|
||||
mask_index: int
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegionalConditioningPlan:
|
||||
"""Describe valid positional pairing after the global entry."""
|
||||
|
||||
region_count: int
|
||||
pairs: tuple[RegionalConditioningPair, ...]
|
||||
|
||||
|
||||
def validate_regional_prompt_weight(weight: float) -> None:
|
||||
"""Reject regional influence values outside the normalized blend range."""
|
||||
|
||||
if not math.isfinite(weight):
|
||||
raise ValueError("regional_prompt_weight must be finite.")
|
||||
if not 0.0 <= weight <= MAX_REGIONAL_PROMPT_WEIGHT:
|
||||
raise ValueError(
|
||||
"regional_prompt_weight must be between 0.0 and "
|
||||
f"{MAX_REGIONAL_PROMPT_WEIGHT:.1f}."
|
||||
)
|
||||
|
||||
|
||||
def build_regional_conditioning_plan(
|
||||
*,
|
||||
region_count: int,
|
||||
conditioning_count: int,
|
||||
input_name: str,
|
||||
) -> RegionalConditioningPlan:
|
||||
"""Return the global-first positional plan or reject excess prompts."""
|
||||
|
||||
if region_count < 1:
|
||||
raise ValueError("regional prompting requires at least one authored mask.")
|
||||
if conditioning_count < 1:
|
||||
raise ValueError(
|
||||
f"{input_name} conditioning must contain a global entry at index 0."
|
||||
)
|
||||
|
||||
regional_count = conditioning_count - 1
|
||||
if regional_count > region_count:
|
||||
raise ValueError(
|
||||
f"{input_name} conditioning contains {regional_count} regional "
|
||||
f"entries but only {region_count} authored masks were provided."
|
||||
)
|
||||
|
||||
return RegionalConditioningPlan(
|
||||
region_count=region_count,
|
||||
pairs=tuple(
|
||||
RegionalConditioningPair(
|
||||
conditioning_index=mask_index + 1,
|
||||
mask_index=mask_index,
|
||||
)
|
||||
for mask_index in range(regional_count)
|
||||
),
|
||||
)
|
||||
@@ -0,0 +1,106 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Constrain semantic tiled diffusion by authored regional composition masks."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import torch
|
||||
|
||||
from .segs import coerce_segs
|
||||
from .segs_tiled_diffusion import segs_ownership_masks, validate_segs_aspect_ratio
|
||||
from .semantic_tiled_diffusion import build_semantic_tiled_diffusion_plan
|
||||
from .tiled_diffusion import TiledDiffusionPlan
|
||||
|
||||
REGIONAL_PLANNING_THRESHOLD = 0.5
|
||||
|
||||
|
||||
def build_region_constrained_tiled_diffusion_plan(
|
||||
*,
|
||||
region_masks: torch.Tensor,
|
||||
segs: object | None,
|
||||
latent_width: int,
|
||||
latent_height: int,
|
||||
tile_width: int,
|
||||
tile_height: int,
|
||||
overlap: int,
|
||||
tile_batch_size: int,
|
||||
) -> TiledDiffusionPlan:
|
||||
"""Build tiles split wherever regional composition or optional SEGS change."""
|
||||
|
||||
region_ownership = regional_composition_ownership_masks(
|
||||
region_masks,
|
||||
latent_height=latent_height,
|
||||
latent_width=latent_width,
|
||||
)
|
||||
ownership_masks = region_ownership
|
||||
if segs is not None:
|
||||
native_segs = coerce_segs(segs)
|
||||
validate_segs_aspect_ratio(native_segs, latent_height, latent_width)
|
||||
semantic_ownership = segs_ownership_masks(
|
||||
native_segs,
|
||||
latent_height=latent_height,
|
||||
latent_width=latent_width,
|
||||
)
|
||||
ownership_masks = _intersect_partitions(
|
||||
region_ownership,
|
||||
semantic_ownership,
|
||||
)
|
||||
return build_semantic_tiled_diffusion_plan(
|
||||
ownership_masks=ownership_masks,
|
||||
latent_width=latent_width,
|
||||
latent_height=latent_height,
|
||||
tile_width=tile_width,
|
||||
tile_height=tile_height,
|
||||
overlap=overlap,
|
||||
tile_batch_size=tile_batch_size,
|
||||
merge_across_masks=False,
|
||||
)
|
||||
|
||||
|
||||
def regional_composition_ownership_masks(
|
||||
region_masks: torch.Tensor,
|
||||
*,
|
||||
latent_height: int,
|
||||
latent_width: int,
|
||||
) -> tuple[torch.Tensor, ...]:
|
||||
"""Partition the canvas by every distinct active regional-mask combination."""
|
||||
|
||||
if region_masks.ndim != 3:
|
||||
raise ValueError("Regional tile planning requires a BHW mask batch.")
|
||||
if tuple(region_masks.shape[1:]) != (latent_height, latent_width):
|
||||
raise ValueError(
|
||||
"Regional tile planning masks must match latent shape "
|
||||
f"{latent_height}x{latent_width}."
|
||||
)
|
||||
membership = (region_masks.detach().cpu() >= REGIONAL_PLANNING_THRESHOLD).permute(
|
||||
1, 2, 0
|
||||
)
|
||||
flattened = membership.reshape(latent_height * latent_width, -1)
|
||||
signatures, inverse = torch.unique(
|
||||
flattened,
|
||||
dim=0,
|
||||
sorted=True,
|
||||
return_inverse=True,
|
||||
)
|
||||
del signatures
|
||||
labels = inverse.reshape(latent_height, latent_width)
|
||||
return tuple(labels == index for index in range(int(labels.max().item()) + 1))
|
||||
|
||||
|
||||
def _intersect_partitions(
|
||||
first: tuple[torch.Tensor, ...],
|
||||
second: tuple[torch.Tensor, ...],
|
||||
) -> tuple[torch.Tensor, ...]:
|
||||
"""Return non-empty intersections of two complete ownership partitions."""
|
||||
|
||||
intersections = tuple(
|
||||
intersection
|
||||
for first_mask in first
|
||||
for second_mask in second
|
||||
if bool((intersection := torch.logical_and(first_mask, second_mask)).any())
|
||||
)
|
||||
if not intersections:
|
||||
raise ValueError("Regional and SEGS ownership produced no tile coverage.")
|
||||
return intersections
|
||||
@@ -0,0 +1,347 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Define model-neutral resolved regional ordinary-LoRA operation contracts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
from enum import StrEnum
|
||||
|
||||
from .regional_lora_plan import RegionalLoraAdapterPlan
|
||||
|
||||
|
||||
class ResolvedLoraOperationClass(StrEnum):
|
||||
"""Classify normalized low-rank tensor organization before module binding."""
|
||||
|
||||
MATRIX_PAIR = "matrix_pair"
|
||||
CONVOLUTION_1D = "convolution_1d"
|
||||
CONVOLUTION_2D = "convolution_2d"
|
||||
CONVOLUTION_3D = "convolution_3d"
|
||||
UNSUPPORTED = "unsupported"
|
||||
|
||||
|
||||
class RegionalLoraGeometryClass(StrEnum):
|
||||
"""Declare the activation geometry an execution contract requires."""
|
||||
|
||||
TARGET_OPERATION = "target_operation"
|
||||
DIRECT_CONVOLUTION_1D = "direct_convolution_1d"
|
||||
DIRECT_CONVOLUTION_2D = "direct_convolution_2d"
|
||||
DIRECT_CONVOLUTION_3D = "direct_convolution_3d"
|
||||
UNSUPPORTED = "unsupported"
|
||||
|
||||
|
||||
class RegionalLoraExecutionContract(StrEnum):
|
||||
"""Declare exact rank-activation execution or explicit rejection."""
|
||||
|
||||
MATRIX_OR_RESHAPED_CONVOLUTION = "matrix_or_reshaped_convolution"
|
||||
DIRECT_CONVOLUTION = "direct_convolution"
|
||||
UNSUPPORTED = "unsupported"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RegionalLoraTensorShape:
|
||||
"""Retain one immutable positive tensor shape without tensor ownership."""
|
||||
|
||||
dimensions: tuple[int, ...]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require a nonempty tuple of strictly positive integer dimensions."""
|
||||
|
||||
if not isinstance(self.dimensions, tuple) or not self.dimensions:
|
||||
raise ValueError("Regional LoRA tensor shape must be a nonempty tuple.")
|
||||
if any(
|
||||
isinstance(dimension, bool)
|
||||
or not isinstance(dimension, int)
|
||||
or dimension <= 0
|
||||
for dimension in self.dimensions
|
||||
):
|
||||
raise ValueError(
|
||||
"Regional LoRA tensor shape dimensions must be positive integers."
|
||||
)
|
||||
|
||||
@property
|
||||
def rank(self) -> int:
|
||||
"""Return the tensor dimensionality."""
|
||||
|
||||
return len(self.dimensions)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ResolvedRegionalLoraTarget:
|
||||
"""Retain one model-relative parameter path and optional Comfy tensor slice."""
|
||||
|
||||
model_target: str
|
||||
parameter_name: str
|
||||
offset: tuple[int, ...] | None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require an explicit module target, parameter, and valid optional slice."""
|
||||
|
||||
if not isinstance(self.model_target, str) or not self.model_target.strip():
|
||||
raise ValueError("Resolved regional LoRA model target must be nonempty.")
|
||||
if not isinstance(self.parameter_name, str) or not self.parameter_name.strip():
|
||||
raise ValueError("Resolved regional LoRA parameter name must be nonempty.")
|
||||
if self.offset is not None and (
|
||||
not isinstance(self.offset, tuple)
|
||||
or not self.offset
|
||||
or any(
|
||||
isinstance(part, bool) or not isinstance(part, int) or part < 0
|
||||
for part in self.offset
|
||||
)
|
||||
):
|
||||
raise ValueError(
|
||||
"Resolved regional LoRA target offset must contain "
|
||||
"non-negative integers."
|
||||
)
|
||||
|
||||
@property
|
||||
def parameter_path(self) -> str:
|
||||
"""Return the complete model-relative parameter path."""
|
||||
|
||||
return f"{self.model_target}.{self.parameter_name}"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ResolvedRegionalLoraOperation:
|
||||
"""Describe one supported operation or one explicit target rejection."""
|
||||
|
||||
adapter: RegionalLoraAdapterPlan
|
||||
target_index: int
|
||||
target: ResolvedRegionalLoraTarget
|
||||
normalized_operation_type: str
|
||||
operation_class: ResolvedLoraOperationClass
|
||||
down_shape: RegionalLoraTensorShape | None
|
||||
up_shape: RegionalLoraTensorShape | None
|
||||
middle_shape: RegionalLoraTensorShape | None
|
||||
reshape_shape: RegionalLoraTensorShape | None
|
||||
rank: int | None
|
||||
intrinsic_scale: float | None
|
||||
required_geometry: RegionalLoraGeometryClass
|
||||
execution_contract: RegionalLoraExecutionContract
|
||||
rejection_reason: str | None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Reject mutable, incomplete, or mathematically inconsistent metadata."""
|
||||
|
||||
if not isinstance(self.adapter, RegionalLoraAdapterPlan):
|
||||
raise TypeError("Resolved regional LoRA operation requires an adapter.")
|
||||
_require_non_negative_index(self.target_index, name="target_index")
|
||||
if not isinstance(self.target, ResolvedRegionalLoraTarget):
|
||||
raise TypeError("Resolved regional LoRA operation requires a target.")
|
||||
if (
|
||||
not isinstance(self.normalized_operation_type, str)
|
||||
or not self.normalized_operation_type.strip()
|
||||
):
|
||||
raise ValueError(
|
||||
"Normalized regional LoRA operation type must be nonempty."
|
||||
)
|
||||
if not isinstance(self.operation_class, ResolvedLoraOperationClass):
|
||||
raise TypeError("Resolved regional LoRA operation class is invalid.")
|
||||
if not isinstance(self.required_geometry, RegionalLoraGeometryClass):
|
||||
raise TypeError("Resolved regional LoRA geometry class is invalid.")
|
||||
if not isinstance(self.execution_contract, RegionalLoraExecutionContract):
|
||||
raise TypeError("Resolved regional LoRA execution contract is invalid.")
|
||||
if self.execution_contract is RegionalLoraExecutionContract.UNSUPPORTED:
|
||||
self._validate_rejection()
|
||||
return
|
||||
self._validate_supported()
|
||||
|
||||
def _validate_rejection(self) -> None:
|
||||
"""Require one reason and no misleading supported-operation metadata."""
|
||||
|
||||
if self.operation_class is not ResolvedLoraOperationClass.UNSUPPORTED:
|
||||
raise ValueError(
|
||||
"Rejected regional LoRA operation class must be unsupported."
|
||||
)
|
||||
if self.required_geometry is not RegionalLoraGeometryClass.UNSUPPORTED:
|
||||
raise ValueError("Rejected regional LoRA geometry must be unsupported.")
|
||||
if (
|
||||
not isinstance(self.rejection_reason, str)
|
||||
or not self.rejection_reason.strip()
|
||||
):
|
||||
raise ValueError("Rejected regional LoRA operation requires a reason.")
|
||||
if any(
|
||||
value is not None
|
||||
for value in (
|
||||
self.down_shape,
|
||||
self.up_shape,
|
||||
self.middle_shape,
|
||||
self.reshape_shape,
|
||||
self.rank,
|
||||
self.intrinsic_scale,
|
||||
)
|
||||
):
|
||||
raise ValueError(
|
||||
"Rejected regional LoRA operation cannot claim executable metadata."
|
||||
)
|
||||
|
||||
def _validate_supported(self) -> None:
|
||||
"""Require complete shape, scale, geometry, and execution consistency."""
|
||||
|
||||
if self.operation_class is ResolvedLoraOperationClass.UNSUPPORTED:
|
||||
raise ValueError("Supported regional LoRA operation cannot be unsupported.")
|
||||
if self.required_geometry is RegionalLoraGeometryClass.UNSUPPORTED:
|
||||
raise ValueError("Supported regional LoRA geometry cannot be unsupported.")
|
||||
if self.rejection_reason is not None:
|
||||
raise ValueError(
|
||||
"Supported regional LoRA operation cannot have a rejection."
|
||||
)
|
||||
if self.down_shape is None or self.up_shape is None:
|
||||
raise ValueError(
|
||||
"Supported regional LoRA operation requires down/up shapes."
|
||||
)
|
||||
if self.down_shape.rank < 2 or self.up_shape.rank < 2:
|
||||
raise ValueError(
|
||||
"Regional LoRA down/up shapes need at least two dimensions."
|
||||
)
|
||||
if (
|
||||
isinstance(self.rank, bool)
|
||||
or not isinstance(self.rank, int)
|
||||
or self.rank <= 0
|
||||
):
|
||||
raise ValueError("Supported regional LoRA rank must be a positive integer.")
|
||||
if not isinstance(self.intrinsic_scale, float) or not math.isfinite(
|
||||
self.intrinsic_scale
|
||||
):
|
||||
raise ValueError("Regional LoRA intrinsic scale must be finite.")
|
||||
if self.down_shape.dimensions[0] != self.rank:
|
||||
raise ValueError("Regional LoRA down shape must begin with its rank.")
|
||||
if (
|
||||
len(self.up_shape.dimensions) < 2
|
||||
or self.up_shape.dimensions[1] != self.rank
|
||||
):
|
||||
raise ValueError(
|
||||
"Regional LoRA up shape must contain its rank at index one."
|
||||
)
|
||||
if self.middle_shape is not None:
|
||||
if self.middle_shape.rank < 2:
|
||||
raise ValueError(
|
||||
"Regional LoRA middle shape needs at least two dimensions."
|
||||
)
|
||||
if (
|
||||
self.middle_shape.dimensions[0] != self.rank
|
||||
or self.middle_shape.dimensions[1] != self.rank
|
||||
):
|
||||
raise ValueError(
|
||||
"Regional LoRA middle shape must preserve rank channels."
|
||||
)
|
||||
self._validate_operation_shape_contract()
|
||||
|
||||
def _validate_operation_shape_contract(self) -> None:
|
||||
"""Match operation, tensor dimensionality, geometry, and execution policy."""
|
||||
|
||||
if self.operation_class is ResolvedLoraOperationClass.MATRIX_PAIR:
|
||||
if self.down_shape is None or self.up_shape is None:
|
||||
raise AssertionError(
|
||||
"Supported shape validation requires down/up shapes."
|
||||
)
|
||||
if self.down_shape.rank != 2 or self.up_shape.rank != 2:
|
||||
raise ValueError("Matrix-pair regional LoRA tensors must be rank two.")
|
||||
if self.middle_shape is not None:
|
||||
raise ValueError(
|
||||
"Matrix-pair regional LoRA cannot contain middle weights."
|
||||
)
|
||||
if self.required_geometry is not RegionalLoraGeometryClass.TARGET_OPERATION:
|
||||
raise ValueError("Matrix-pair regional LoRA requires target geometry.")
|
||||
if (
|
||||
self.execution_contract
|
||||
is not RegionalLoraExecutionContract.MATRIX_OR_RESHAPED_CONVOLUTION
|
||||
):
|
||||
raise ValueError("Matrix-pair regional LoRA execution is inconsistent.")
|
||||
return
|
||||
spatial_rank = {
|
||||
ResolvedLoraOperationClass.CONVOLUTION_1D: 3,
|
||||
ResolvedLoraOperationClass.CONVOLUTION_2D: 4,
|
||||
ResolvedLoraOperationClass.CONVOLUTION_3D: 5,
|
||||
}[self.operation_class]
|
||||
expected_geometry = {
|
||||
3: RegionalLoraGeometryClass.DIRECT_CONVOLUTION_1D,
|
||||
4: RegionalLoraGeometryClass.DIRECT_CONVOLUTION_2D,
|
||||
5: RegionalLoraGeometryClass.DIRECT_CONVOLUTION_3D,
|
||||
}[spatial_rank]
|
||||
shapes = tuple(
|
||||
shape
|
||||
for shape in (self.down_shape, self.up_shape, self.middle_shape)
|
||||
if shape is not None
|
||||
)
|
||||
if any(shape.rank not in (2, spatial_rank) for shape in shapes):
|
||||
raise ValueError("Convolutional regional LoRA tensor rank is inconsistent.")
|
||||
if not any(shape.rank == spatial_rank for shape in shapes):
|
||||
raise ValueError("Convolutional regional LoRA needs one spatial tensor.")
|
||||
if self.required_geometry is not expected_geometry:
|
||||
raise ValueError("Convolutional regional LoRA geometry is inconsistent.")
|
||||
if (
|
||||
self.execution_contract
|
||||
is not RegionalLoraExecutionContract.DIRECT_CONVOLUTION
|
||||
):
|
||||
raise ValueError("Convolutional regional LoRA execution is inconsistent.")
|
||||
|
||||
@property
|
||||
def authored_strength(self) -> float:
|
||||
"""Return the authored model strength from the authoritative plan owner."""
|
||||
|
||||
return self.adapter.model_strength
|
||||
|
||||
@property
|
||||
def supported(self) -> bool:
|
||||
"""Report whether this target has an executable ordinary-LoRA contract."""
|
||||
|
||||
return self.execution_contract is not RegionalLoraExecutionContract.UNSUPPORTED
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ResolvedRegionalLoraOperationSet:
|
||||
"""Retain canonical adapter/target order across supported and rejected entries."""
|
||||
|
||||
entries: tuple[ResolvedRegionalLoraOperation, ...]
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require immutable entries in contiguous per-adapter target order."""
|
||||
|
||||
if not isinstance(self.entries, tuple):
|
||||
raise TypeError("Resolved regional LoRA entries must be a tuple.")
|
||||
if any(
|
||||
not isinstance(entry, ResolvedRegionalLoraOperation)
|
||||
for entry in self.entries
|
||||
):
|
||||
raise TypeError("Resolved regional LoRA set contains an invalid entry.")
|
||||
observed = tuple(
|
||||
(entry.adapter.composition_index, entry.target_index)
|
||||
for entry in self.entries
|
||||
)
|
||||
if observed != tuple(sorted(observed)):
|
||||
raise ValueError(
|
||||
"Resolved regional LoRA entries must retain declared order."
|
||||
)
|
||||
indices_by_adapter: dict[int, list[int]] = {}
|
||||
for composition_index, target_index in observed:
|
||||
indices_by_adapter.setdefault(composition_index, []).append(target_index)
|
||||
if any(
|
||||
target_indices != list(range(len(target_indices)))
|
||||
for target_indices in indices_by_adapter.values()
|
||||
):
|
||||
raise ValueError(
|
||||
"Resolved regional LoRA target indices must be contiguous per adapter."
|
||||
)
|
||||
|
||||
@property
|
||||
def supported(self) -> tuple[ResolvedRegionalLoraOperation, ...]:
|
||||
"""Return supported entries without changing canonical relative order."""
|
||||
|
||||
return tuple(entry for entry in self.entries if entry.supported)
|
||||
|
||||
@property
|
||||
def rejected(self) -> tuple[ResolvedRegionalLoraOperation, ...]:
|
||||
"""Return rejected entries without changing canonical relative order."""
|
||||
|
||||
return tuple(entry for entry in self.entries if not entry.supported)
|
||||
|
||||
|
||||
def _require_non_negative_index(value: object, *, name: str) -> None:
|
||||
"""Require one non-negative non-boolean integer index."""
|
||||
|
||||
if isinstance(value, bool) or not isinstance(value, int) or value < 0:
|
||||
raise ValueError(f"Resolved regional LoRA {name} must be non-negative.")
|
||||
@@ -0,0 +1,54 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Define validated seed-variation sampling settings."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from math import isfinite
|
||||
|
||||
MIN_SEED = 0
|
||||
MAX_SEED = 0xFFFFFFFFFFFFFFFF
|
||||
MIN_VARIATION_STRENGTH = 0.0
|
||||
MAX_VARIATION_STRENGTH = 1.0
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class SeedVariationSettings:
|
||||
"""Hold one deterministic initial-noise interpolation request."""
|
||||
|
||||
variation_seed: int
|
||||
strength: float
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Reject settings outside ComfyUI's public seed and strength ranges."""
|
||||
|
||||
if isinstance(self.variation_seed, bool) or not isinstance(
|
||||
self.variation_seed,
|
||||
int,
|
||||
):
|
||||
raise TypeError("Variation seed must be an integer.")
|
||||
if not MIN_SEED <= self.variation_seed <= MAX_SEED:
|
||||
raise ValueError(
|
||||
f"Variation seed must be between {MIN_SEED} and {MAX_SEED}."
|
||||
)
|
||||
if isinstance(self.strength, bool) or not isinstance(
|
||||
self.strength,
|
||||
(int, float),
|
||||
):
|
||||
raise TypeError("Variation strength must be a number.")
|
||||
normalized_strength = float(self.strength)
|
||||
if not isfinite(normalized_strength):
|
||||
raise ValueError("Variation strength must be finite.")
|
||||
if (
|
||||
not MIN_VARIATION_STRENGTH
|
||||
<= normalized_strength
|
||||
<= (MAX_VARIATION_STRENGTH)
|
||||
):
|
||||
raise ValueError(
|
||||
"Variation strength must be between "
|
||||
f"{MIN_VARIATION_STRENGTH} and {MAX_VARIATION_STRENGTH}."
|
||||
)
|
||||
object.__setattr__(self, "strength", normalized_strength)
|
||||
@@ -0,0 +1,118 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Plan deterministic visual representations of validated SEGS."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import NamedTuple
|
||||
|
||||
import torch
|
||||
|
||||
from .segs import CropRegion, NativeSegs, Segment, coerce_segment_mask, coerce_segs
|
||||
|
||||
|
||||
class RegionColor(NamedTuple):
|
||||
"""Represent one reusable RGB region color."""
|
||||
|
||||
red: int
|
||||
green: int
|
||||
blue: int
|
||||
|
||||
@property
|
||||
def normalized(self) -> tuple[float, float, float]:
|
||||
"""Return the color as normalized tensor-ready channels."""
|
||||
|
||||
return self.red / 255.0, self.green / 255.0, self.blue / 255.0
|
||||
|
||||
@property
|
||||
def css(self) -> str:
|
||||
"""Return the color as a browser-ready hexadecimal value."""
|
||||
|
||||
return f"#{self.red:02x}{self.green:02x}{self.blue:02x}"
|
||||
|
||||
|
||||
REGION_COLORS: tuple[RegionColor, ...] = (
|
||||
RegionColor(242, 66, 54),
|
||||
RegionColor(33, 150, 243),
|
||||
RegionColor(76, 176, 80),
|
||||
RegionColor(255, 194, 8),
|
||||
RegionColor(156, 39, 176),
|
||||
RegionColor(255, 87, 34),
|
||||
RegionColor(0, 188, 212),
|
||||
RegionColor(232, 31, 99),
|
||||
RegionColor(140, 194, 74),
|
||||
RegionColor(103, 58, 183),
|
||||
RegionColor(255, 153, 0),
|
||||
RegionColor(0, 150, 136),
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class VisualRegion:
|
||||
"""Describe one validated SEG with stable visualization identity."""
|
||||
|
||||
region_id: str
|
||||
index: int
|
||||
segment: Segment
|
||||
mask: torch.Tensor
|
||||
color: RegionColor
|
||||
active_area: int
|
||||
|
||||
@property
|
||||
def crop_region(self) -> CropRegion:
|
||||
"""Return the source crop occupied by this region."""
|
||||
|
||||
return self.segment.crop_region
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class SegVisualizationPlan:
|
||||
"""Own source geometry and ordered visual regions for one SEGS payload."""
|
||||
|
||||
source_height: int
|
||||
source_width: int
|
||||
regions: tuple[VisualRegion, ...]
|
||||
|
||||
|
||||
def build_seg_visualization_plan(segs: NativeSegs) -> SegVisualizationPlan:
|
||||
"""Return deterministic validated regions without mutating the source SEGS."""
|
||||
|
||||
(source_height, source_width), segments = coerce_segs(segs)
|
||||
regions: list[VisualRegion] = []
|
||||
for index, segment in enumerate(segments):
|
||||
_validate_crop_bounds(
|
||||
segment.crop_region,
|
||||
source_height=source_height,
|
||||
source_width=source_width,
|
||||
)
|
||||
mask = coerce_segment_mask(segment).detach().cpu()
|
||||
regions.append(
|
||||
VisualRegion(
|
||||
region_id=f"seg-{index + 1:04d}",
|
||||
index=index,
|
||||
segment=segment,
|
||||
mask=mask,
|
||||
color=REGION_COLORS[index % len(REGION_COLORS)],
|
||||
active_area=int((mask >= 0.5).sum().item()),
|
||||
)
|
||||
)
|
||||
return SegVisualizationPlan(
|
||||
source_height=source_height,
|
||||
source_width=source_width,
|
||||
regions=tuple(regions),
|
||||
)
|
||||
|
||||
|
||||
def _validate_crop_bounds(
|
||||
crop: CropRegion,
|
||||
*,
|
||||
source_height: int,
|
||||
source_width: int,
|
||||
) -> None:
|
||||
"""Reject SEG crops that cannot describe the declared source image."""
|
||||
|
||||
if crop.right > source_width or crop.bottom > source_height:
|
||||
raise ValueError("Segment crop_region must fit inside the SEGS dimensions.")
|
||||
@@ -10,6 +10,8 @@ from collections.abc import Iterable, Sequence
|
||||
from dataclasses import dataclass
|
||||
from typing import NamedTuple, Protocol, TypeAlias, cast
|
||||
|
||||
import torch
|
||||
|
||||
|
||||
class CropRegion(NamedTuple):
|
||||
"""Represent a crop region as left, top, right, bottom coordinates."""
|
||||
@@ -152,6 +154,25 @@ def coerce_segment(value: object) -> Segment:
|
||||
)
|
||||
|
||||
|
||||
def coerce_segment_mask(segment: Segment) -> torch.Tensor:
|
||||
"""Return one validated crop-local HW mask without changing its values."""
|
||||
|
||||
mask = (
|
||||
segment.cropped_mask.float()
|
||||
if isinstance(segment.cropped_mask, torch.Tensor)
|
||||
else torch.as_tensor(segment.cropped_mask, dtype=torch.float32)
|
||||
)
|
||||
if mask.ndim == 3 and int(mask.shape[0]) == 1:
|
||||
mask = mask.squeeze(0)
|
||||
if mask.ndim != 2:
|
||||
raise ValueError("Segment cropped_mask must be HW or singleton BHW shaped.")
|
||||
expected_shape = (segment.crop_region.height, segment.crop_region.width)
|
||||
actual_shape = (int(mask.shape[0]), int(mask.shape[1]))
|
||||
if actual_shape != expected_shape:
|
||||
raise ValueError("Segment cropped_mask must match its crop region.")
|
||||
return mask.clamp(0.0, 1.0)
|
||||
|
||||
|
||||
def to_impact_compatible_segs(segs: NativeSegs) -> ImpactSegs:
|
||||
"""Return raw tuple/list SEGS that Impact-style consumers can read."""
|
||||
|
||||
|
||||
@@ -0,0 +1,212 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Build irregular, SEGS-guided latent tiles for tiled diffusion sampling."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import torch
|
||||
|
||||
from .segs import NativeSegs, Segment, coerce_segment_mask, coerce_segs
|
||||
from .semantic_tiled_diffusion import build_semantic_tiled_diffusion_plan
|
||||
from .tiled_diffusion import TiledDiffusionPlan
|
||||
|
||||
|
||||
def build_segs_guided_tiled_diffusion_plan(
|
||||
*,
|
||||
segs: object,
|
||||
latent_width: int,
|
||||
latent_height: int,
|
||||
tile_width: int,
|
||||
tile_height: int,
|
||||
overlap: int,
|
||||
tile_batch_size: int,
|
||||
) -> TiledDiffusionPlan:
|
||||
"""Build bounded sampling windows whose irregular cores follow supplied SEGS.
|
||||
|
||||
Every latent pixel receives exactly one ownership core. Each core is sampled
|
||||
through a rectangular window, while its local blend mask retains the irregular
|
||||
boundary and shares a feathered overlap with neighboring cores.
|
||||
"""
|
||||
|
||||
native_segs = coerce_segs(segs)
|
||||
validate_segs_aspect_ratio(native_segs, latent_height, latent_width)
|
||||
ownership_masks = segs_ownership_masks(
|
||||
native_segs,
|
||||
latent_height=latent_height,
|
||||
latent_width=latent_width,
|
||||
)
|
||||
return build_semantic_tiled_diffusion_plan(
|
||||
ownership_masks=ownership_masks,
|
||||
latent_width=latent_width,
|
||||
latent_height=latent_height,
|
||||
tile_width=tile_width,
|
||||
tile_height=tile_height,
|
||||
overlap=overlap,
|
||||
tile_batch_size=tile_batch_size,
|
||||
merge_across_masks=True,
|
||||
)
|
||||
|
||||
|
||||
def validate_segs_aspect_ratio(
|
||||
segs: NativeSegs,
|
||||
latent_height: int,
|
||||
latent_width: int,
|
||||
) -> None:
|
||||
"""Reject SEGS that cannot describe the sampled latent's image proportions."""
|
||||
|
||||
source_height, source_width = segs[0]
|
||||
source_ratio = source_width / source_height
|
||||
latent_ratio = latent_width / latent_height
|
||||
if abs(source_ratio - latent_ratio) / source_ratio <= 0.02:
|
||||
return
|
||||
raise ValueError(
|
||||
"SEGS-guided tiled diffusion requires SEGS to match the latent image "
|
||||
f"aspect ratio; SEGS is {source_height}x{source_width}, latent is "
|
||||
f"{latent_height}x{latent_width}."
|
||||
)
|
||||
|
||||
|
||||
def segs_ownership_masks(
|
||||
segs: NativeSegs,
|
||||
*,
|
||||
latent_height: int,
|
||||
latent_width: int,
|
||||
) -> tuple[torch.Tensor, ...]:
|
||||
"""Resolve overlapping SEGS into a deterministic latent ownership partition."""
|
||||
|
||||
source_height, source_width = segs[0]
|
||||
segment_masks = tuple(
|
||||
segment_mask_to_latent(
|
||||
segment,
|
||||
source_height=source_height,
|
||||
source_width=source_width,
|
||||
latent_height=latent_height,
|
||||
latent_width=latent_width,
|
||||
)
|
||||
for segment in segs[1]
|
||||
)
|
||||
ranked_indexes = sorted(
|
||||
range(len(segment_masks)),
|
||||
key=lambda index: (
|
||||
int(segment_masks[index].sum().item()),
|
||||
-float(segs[1][index].confidence),
|
||||
index,
|
||||
),
|
||||
)
|
||||
occupied = torch.zeros((latent_height, latent_width), dtype=torch.bool)
|
||||
ownership_masks: list[torch.Tensor] = []
|
||||
for index in ranked_indexes:
|
||||
owned = torch.logical_and(segment_masks[index], torch.logical_not(occupied))
|
||||
if bool(owned.any()):
|
||||
ownership_masks.append(owned)
|
||||
occupied = torch.logical_or(occupied, segment_masks[index])
|
||||
background = torch.logical_not(occupied)
|
||||
if bool(background.any()):
|
||||
ownership_masks.append(background)
|
||||
if ownership_masks:
|
||||
return tuple(ownership_masks)
|
||||
return (torch.ones((latent_height, latent_width), dtype=torch.bool),)
|
||||
|
||||
|
||||
def segment_mask_to_latent(
|
||||
segment: Segment,
|
||||
*,
|
||||
source_height: int,
|
||||
source_width: int,
|
||||
latent_height: int,
|
||||
latent_width: int,
|
||||
) -> torch.Tensor:
|
||||
"""Restore one crop-local SEG mask and map it to a latent-space mask."""
|
||||
|
||||
return (
|
||||
segment_weight_to_latent(
|
||||
segment,
|
||||
source_height=source_height,
|
||||
source_width=source_width,
|
||||
latent_height=latent_height,
|
||||
latent_width=latent_width,
|
||||
)
|
||||
>= 0.5
|
||||
)
|
||||
|
||||
|
||||
def segment_weight_to_latent(
|
||||
segment: Segment,
|
||||
*,
|
||||
source_height: int,
|
||||
source_width: int,
|
||||
latent_height: int,
|
||||
latent_width: int,
|
||||
) -> torch.Tensor:
|
||||
"""Project one crop-local SEG mask into latent space without binarizing it."""
|
||||
|
||||
crop = segment.crop_region
|
||||
if (
|
||||
crop.left < 0
|
||||
or crop.top < 0
|
||||
or crop.right > source_width
|
||||
or crop.bottom > source_height
|
||||
or crop.width < 1
|
||||
or crop.height < 1
|
||||
):
|
||||
raise ValueError(
|
||||
"SEGS-guided tiled diffusion requires every SEG crop_region to fit "
|
||||
"inside the SEGS header dimensions."
|
||||
)
|
||||
local_mask = coerce_segment_mask(segment).detach().cpu()
|
||||
latent_top, latent_bottom = _latent_sample_range(
|
||||
crop.top,
|
||||
crop.bottom,
|
||||
source_height,
|
||||
latent_height,
|
||||
)
|
||||
latent_left, latent_right = _latent_sample_range(
|
||||
crop.left,
|
||||
crop.right,
|
||||
source_width,
|
||||
latent_width,
|
||||
)
|
||||
latent_mask = torch.zeros((latent_height, latent_width), dtype=torch.float32)
|
||||
if latent_bottom <= latent_top or latent_right <= latent_left:
|
||||
return latent_mask
|
||||
sampled_rows = (
|
||||
torch.div(
|
||||
torch.arange(latent_top, latent_bottom) * source_height,
|
||||
latent_height,
|
||||
rounding_mode="floor",
|
||||
)
|
||||
- crop.top
|
||||
)
|
||||
sampled_columns = (
|
||||
torch.div(
|
||||
torch.arange(latent_left, latent_right) * source_width,
|
||||
latent_width,
|
||||
rounding_mode="floor",
|
||||
)
|
||||
- crop.left
|
||||
)
|
||||
sampled_mask = (
|
||||
local_mask.clamp(0.0, 1.0)
|
||||
.index_select(
|
||||
0,
|
||||
sampled_rows,
|
||||
)
|
||||
.index_select(1, sampled_columns)
|
||||
)
|
||||
latent_mask[latent_top:latent_bottom, latent_left:latent_right] = sampled_mask
|
||||
return latent_mask
|
||||
|
||||
|
||||
def _latent_sample_range(
|
||||
source_start: int,
|
||||
source_end: int,
|
||||
source_limit: int,
|
||||
latent_limit: int,
|
||||
) -> tuple[int, int]:
|
||||
"""Return latent coordinates whose nearest samples fall in a source interval."""
|
||||
|
||||
start = (source_start * latent_limit + source_limit - 1) // source_limit
|
||||
end = (source_end * latent_limit + source_limit - 1) // source_limit
|
||||
return max(0, min(latent_limit, start)), max(0, min(latent_limit, end))
|
||||
@@ -0,0 +1,384 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Build tiled diffusion plans from non-overlapping latent ownership masks."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Sequence
|
||||
from dataclasses import dataclass
|
||||
|
||||
import torch
|
||||
import torch.nn.functional as functional
|
||||
|
||||
from .tiled_diffusion import (
|
||||
LatentTile,
|
||||
TiledDiffusionPlan,
|
||||
batch_latent_tiles,
|
||||
build_tiled_diffusion_plan,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class _OwnershipCore:
|
||||
"""Represent one latent ownership region before sampling-window placement."""
|
||||
|
||||
mask: torch.Tensor
|
||||
bounds: tuple[int, int, int, int]
|
||||
area: int
|
||||
|
||||
|
||||
def build_semantic_tiled_diffusion_plan(
|
||||
*,
|
||||
ownership_masks: Sequence[torch.Tensor],
|
||||
latent_width: int,
|
||||
latent_height: int,
|
||||
tile_width: int,
|
||||
tile_height: int,
|
||||
overlap: int,
|
||||
tile_batch_size: int,
|
||||
merge_across_masks: bool,
|
||||
) -> TiledDiffusionPlan:
|
||||
"""Build bounded windows whose write weights follow ownership masks."""
|
||||
|
||||
base_plan = build_tiled_diffusion_plan(
|
||||
latent_width=latent_width,
|
||||
latent_height=latent_height,
|
||||
tile_width=tile_width,
|
||||
tile_height=tile_height,
|
||||
overlap=overlap,
|
||||
tile_batch_size=tile_batch_size,
|
||||
)
|
||||
normalized_masks = _validate_ownership_masks(
|
||||
ownership_masks,
|
||||
latent_height=latent_height,
|
||||
latent_width=latent_width,
|
||||
)
|
||||
max_core_width = max(1, base_plan.tile_width - base_plan.overlap)
|
||||
max_core_height = max(1, base_plan.tile_height - base_plan.overlap)
|
||||
split_groups = tuple(
|
||||
tuple(
|
||||
split_core
|
||||
for split_core in _split_core(
|
||||
_core_from_mask(mask),
|
||||
max_width=max_core_width,
|
||||
max_height=max_core_height,
|
||||
)
|
||||
)
|
||||
for mask in normalized_masks
|
||||
)
|
||||
if merge_across_masks:
|
||||
cores = _merge_small_cores(
|
||||
tuple(core for group in split_groups for core in group),
|
||||
max_width=max_core_width,
|
||||
max_height=max_core_height,
|
||||
)
|
||||
else:
|
||||
cores = tuple(
|
||||
core
|
||||
for group in split_groups
|
||||
for core in _merge_small_cores(
|
||||
group,
|
||||
max_width=max_core_width,
|
||||
max_height=max_core_height,
|
||||
)
|
||||
)
|
||||
tiles = tuple(
|
||||
sorted(
|
||||
(
|
||||
_tile_for_core(
|
||||
core,
|
||||
latent_width=latent_width,
|
||||
latent_height=latent_height,
|
||||
tile_width=base_plan.tile_width,
|
||||
tile_height=base_plan.tile_height,
|
||||
overlap=base_plan.overlap,
|
||||
)
|
||||
for core in cores
|
||||
),
|
||||
key=lambda tile: (tile.y, tile.x),
|
||||
)
|
||||
)
|
||||
batches, effective_batch_size = batch_latent_tiles(tiles, tile_batch_size)
|
||||
return TiledDiffusionPlan(
|
||||
latent_width=latent_width,
|
||||
latent_height=latent_height,
|
||||
tile_width=base_plan.tile_width,
|
||||
tile_height=base_plan.tile_height,
|
||||
overlap=base_plan.overlap,
|
||||
requested_tile_batch_size=tile_batch_size,
|
||||
tile_batch_size=effective_batch_size,
|
||||
tiles=tiles,
|
||||
batches=batches,
|
||||
)
|
||||
|
||||
|
||||
def _validate_ownership_masks(
|
||||
masks: Sequence[torch.Tensor],
|
||||
*,
|
||||
latent_height: int,
|
||||
latent_width: int,
|
||||
) -> tuple[torch.Tensor, ...]:
|
||||
"""Return non-empty boolean masks that cover the complete latent canvas."""
|
||||
|
||||
normalized: list[torch.Tensor] = []
|
||||
coverage = torch.zeros((latent_height, latent_width), dtype=torch.bool)
|
||||
for index, mask in enumerate(masks):
|
||||
if mask.ndim != 2 or tuple(mask.shape) != (latent_height, latent_width):
|
||||
raise ValueError(
|
||||
"Semantic ownership mask "
|
||||
f"{index} must match latent shape {latent_height}x{latent_width}."
|
||||
)
|
||||
boolean_mask = mask.detach().cpu().bool()
|
||||
if not bool(boolean_mask.any()):
|
||||
continue
|
||||
if bool(torch.logical_and(coverage, boolean_mask).any()):
|
||||
raise ValueError("Semantic ownership masks must not overlap.")
|
||||
normalized.append(boolean_mask)
|
||||
coverage = torch.logical_or(coverage, boolean_mask)
|
||||
if not normalized:
|
||||
raise ValueError("Semantic tiled diffusion requires non-empty ownership.")
|
||||
if not bool(coverage.all()):
|
||||
raise ValueError("Semantic ownership masks must cover the latent canvas.")
|
||||
return tuple(normalized)
|
||||
|
||||
|
||||
def _split_core(
|
||||
core: _OwnershipCore,
|
||||
*,
|
||||
max_width: int,
|
||||
max_height: int,
|
||||
) -> tuple[_OwnershipCore, ...]:
|
||||
"""Recursively divide a core into balanced pieces within its tile budget."""
|
||||
|
||||
left, top, right, bottom = core.bounds
|
||||
width = right - left
|
||||
height = bottom - top
|
||||
if width <= max_width and height <= max_height:
|
||||
return (core,)
|
||||
split_x = width / max_width >= height / max_height
|
||||
first, second = _split_mask_at_balanced_axis(core.mask, core.bounds, split_x)
|
||||
return _split_core(
|
||||
_core_from_mask(first), max_width=max_width, max_height=max_height
|
||||
) + _split_core(_core_from_mask(second), max_width=max_width, max_height=max_height)
|
||||
|
||||
|
||||
def _split_mask_at_balanced_axis(
|
||||
mask: torch.Tensor,
|
||||
bounds: tuple[int, int, int, int],
|
||||
split_x: bool,
|
||||
) -> tuple[torch.Tensor, torch.Tensor]:
|
||||
"""Split one non-empty mask near its active-pixel median on one axis."""
|
||||
|
||||
left, top, right, bottom = bounds
|
||||
counts = (
|
||||
mask[top:bottom, left:right].sum(dim=0)
|
||||
if split_x
|
||||
else mask[top:bottom, left:right].sum(dim=1)
|
||||
)
|
||||
cumulative = torch.cumsum(counts, dim=0)
|
||||
midpoint = int(torch.searchsorted(cumulative, cumulative[-1] / 2, right=False))
|
||||
axis_start = left if split_x else top
|
||||
axis_end = right if split_x else bottom
|
||||
split_at = min(axis_end - 1, max(axis_start + 1, axis_start + midpoint + 1))
|
||||
first = mask.clone()
|
||||
second = mask.clone()
|
||||
if split_x:
|
||||
first[:, split_at:] = False
|
||||
second[:, :split_at] = False
|
||||
else:
|
||||
first[split_at:, :] = False
|
||||
second[:split_at, :] = False
|
||||
if not bool(first.any()) or not bool(second.any()):
|
||||
raise ValueError("Unable to split an oversized semantic tile core.")
|
||||
return first, second
|
||||
|
||||
|
||||
def _merge_small_cores(
|
||||
cores: tuple[_OwnershipCore, ...],
|
||||
*,
|
||||
max_width: int,
|
||||
max_height: int,
|
||||
) -> tuple[_OwnershipCore, ...]:
|
||||
"""Greedily combine nearby cores when one bounded window can hold both."""
|
||||
|
||||
pending = list(cores)
|
||||
minimum_area = max(1, (max_width * max_height) // 4)
|
||||
merged = True
|
||||
while merged:
|
||||
merged = False
|
||||
for index, core in enumerate(tuple(pending)):
|
||||
if core.area >= minimum_area:
|
||||
continue
|
||||
candidate_index = _best_merge_candidate_index(
|
||||
core,
|
||||
pending,
|
||||
excluded_index=index,
|
||||
max_width=max_width,
|
||||
max_height=max_height,
|
||||
)
|
||||
if candidate_index is None:
|
||||
continue
|
||||
candidate = pending[candidate_index]
|
||||
pending[index] = _OwnershipCore(
|
||||
mask=torch.logical_or(core.mask, candidate.mask),
|
||||
bounds=_union_bounds(core.bounds, candidate.bounds),
|
||||
area=core.area + candidate.area,
|
||||
)
|
||||
pending.pop(candidate_index)
|
||||
merged = True
|
||||
break
|
||||
return tuple(pending)
|
||||
|
||||
|
||||
def _best_merge_candidate_index(
|
||||
core: _OwnershipCore,
|
||||
candidates: list[_OwnershipCore],
|
||||
*,
|
||||
excluded_index: int,
|
||||
max_width: int,
|
||||
max_height: int,
|
||||
) -> int | None:
|
||||
"""Return a candidate whose combined bounds fit one ownership budget."""
|
||||
|
||||
eligible: list[tuple[int, int, int]] = []
|
||||
for index, candidate in enumerate(candidates):
|
||||
if index == excluded_index:
|
||||
continue
|
||||
bounds = _union_bounds(core.bounds, candidate.bounds)
|
||||
left, top, right, bottom = bounds
|
||||
width = right - left
|
||||
height = bottom - top
|
||||
if width > max_width or height > max_height:
|
||||
continue
|
||||
distance = _bounds_distance(core.bounds, candidate.bounds)
|
||||
eligible.append((width * height, distance, index))
|
||||
if not eligible:
|
||||
return None
|
||||
return min(eligible, key=lambda item: (item[0], item[1]))[2]
|
||||
|
||||
|
||||
def _bounds_distance(
|
||||
first: tuple[int, int, int, int],
|
||||
second: tuple[int, int, int, int],
|
||||
) -> int:
|
||||
"""Return the axis-aligned gap between two mask bounding boxes."""
|
||||
|
||||
left, top, right, bottom = first
|
||||
other_left, other_top, other_right, other_bottom = second
|
||||
horizontal = max(0, other_left - right, left - other_right)
|
||||
vertical = max(0, other_top - bottom, top - other_bottom)
|
||||
return horizontal + vertical
|
||||
|
||||
|
||||
def _union_bounds(
|
||||
first: tuple[int, int, int, int],
|
||||
second: tuple[int, int, int, int],
|
||||
) -> tuple[int, int, int, int]:
|
||||
"""Return the tight rectangle containing both ownership-core bounds."""
|
||||
|
||||
return (
|
||||
min(first[0], second[0]),
|
||||
min(first[1], second[1]),
|
||||
max(first[2], second[2]),
|
||||
max(first[3], second[3]),
|
||||
)
|
||||
|
||||
|
||||
def _tile_for_core(
|
||||
core: _OwnershipCore,
|
||||
*,
|
||||
latent_width: int,
|
||||
latent_height: int,
|
||||
tile_width: int,
|
||||
tile_height: int,
|
||||
overlap: int,
|
||||
) -> LatentTile:
|
||||
"""Place one bounded sampling window around an irregular ownership core."""
|
||||
|
||||
left, top, right, bottom = core.bounds
|
||||
center_x = (left + right) / 2.0
|
||||
center_y = (top + bottom) / 2.0
|
||||
x = _clamp_window_start(center_x, tile_width, latent_width)
|
||||
y = _clamp_window_start(center_y, tile_height, latent_height)
|
||||
weight_mask = _feathered_tile_weight(
|
||||
core.mask,
|
||||
x=x,
|
||||
y=y,
|
||||
width=tile_width,
|
||||
height=tile_height,
|
||||
overlap=overlap,
|
||||
)
|
||||
if not bool((weight_mask > 0).any()):
|
||||
raise ValueError("Semantic tiled diffusion generated an empty tile weight.")
|
||||
return LatentTile(x, y, tile_width, tile_height, weight_mask)
|
||||
|
||||
|
||||
def _clamp_window_start(center: float, window_size: int, limit: int) -> int:
|
||||
"""Center a fixed sampling window while keeping it inside latent bounds."""
|
||||
|
||||
desired = round(center - window_size / 2.0)
|
||||
return min(max(0, desired), limit - window_size)
|
||||
|
||||
|
||||
def _feathered_tile_weight(
|
||||
mask: torch.Tensor,
|
||||
*,
|
||||
x: int,
|
||||
y: int,
|
||||
width: int,
|
||||
height: int,
|
||||
overlap: int,
|
||||
) -> torch.Tensor:
|
||||
"""Build one feathered tile weight without blurring the full latent mask."""
|
||||
|
||||
if overlap == 0:
|
||||
return mask[y : y + height, x : x + width].float().contiguous()
|
||||
radius = max(1, overlap // 2)
|
||||
source_left = max(0, x - radius)
|
||||
source_top = max(0, y - radius)
|
||||
source_right = min(int(mask.shape[1]), x + width + radius)
|
||||
source_bottom = min(int(mask.shape[0]), y + height + radius)
|
||||
local_weight = (
|
||||
functional.avg_pool2d(
|
||||
mask[source_top:source_bottom, source_left:source_right]
|
||||
.float()
|
||||
.unsqueeze(0)
|
||||
.unsqueeze(0),
|
||||
kernel_size=radius * 2 + 1,
|
||||
stride=1,
|
||||
padding=radius,
|
||||
count_include_pad=False,
|
||||
)
|
||||
.squeeze(0)
|
||||
.squeeze(0)
|
||||
)
|
||||
local_y = y - source_top
|
||||
local_x = x - source_left
|
||||
return local_weight[
|
||||
local_y : local_y + height, local_x : local_x + width
|
||||
].contiguous()
|
||||
|
||||
|
||||
def _core_from_mask(mask: torch.Tensor) -> _OwnershipCore:
|
||||
"""Build one core with bounds and area computed exactly once."""
|
||||
|
||||
bounds = _mask_bounds(mask)
|
||||
if bounds is None:
|
||||
raise ValueError("Semantic tiled diffusion cannot use an empty core.")
|
||||
return _OwnershipCore(mask=mask, bounds=bounds, area=int(mask.sum().item()))
|
||||
|
||||
|
||||
def _mask_bounds(mask: torch.Tensor) -> tuple[int, int, int, int] | None:
|
||||
"""Return left, top, right, bottom bounds for a non-empty boolean mask."""
|
||||
|
||||
y_coords, x_coords = torch.where(mask)
|
||||
if y_coords.numel() == 0:
|
||||
return None
|
||||
return (
|
||||
int(x_coords.min().item()),
|
||||
int(y_coords.min().item()),
|
||||
int(x_coords.max().item()) + 1,
|
||||
int(y_coords.max().item()) + 1,
|
||||
)
|
||||
@@ -0,0 +1,164 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Define immutable spatial model views and view-major batch layouts."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from enum import StrEnum
|
||||
|
||||
|
||||
class SpatialViewKind(StrEnum):
|
||||
"""Identify how one model view relates to the canonical latent canvas."""
|
||||
|
||||
FULL = "full"
|
||||
TILE = "tile"
|
||||
CONTEXTUAL_GLOBAL = "contextual_global"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class SpatialView:
|
||||
"""Describe one source rectangle evaluated at one model spatial shape."""
|
||||
|
||||
kind: SpatialViewKind
|
||||
source_x: int
|
||||
source_y: int
|
||||
source_width: int
|
||||
source_height: int
|
||||
model_width: int
|
||||
model_height: int
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Reject invalid or internally inconsistent view geometry."""
|
||||
|
||||
if not isinstance(self.kind, SpatialViewKind):
|
||||
raise TypeError("Spatial view kind must be a SpatialViewKind value.")
|
||||
if self.source_x < 0 or self.source_y < 0:
|
||||
raise ValueError("Spatial view source coordinates must be non-negative.")
|
||||
if self.source_width < 1 or self.source_height < 1:
|
||||
raise ValueError("Spatial view source dimensions must be positive.")
|
||||
if self.model_width < 1 or self.model_height < 1:
|
||||
raise ValueError("Spatial view model dimensions must be positive.")
|
||||
if self.kind is SpatialViewKind.FULL and (
|
||||
self.source_width != self.model_width
|
||||
or self.source_height != self.model_height
|
||||
):
|
||||
raise ValueError("A full spatial view must preserve its source dimensions.")
|
||||
|
||||
@property
|
||||
def source_right(self) -> int:
|
||||
"""Return the exclusive source rectangle right edge."""
|
||||
|
||||
return self.source_x + self.source_width
|
||||
|
||||
@property
|
||||
def source_bottom(self) -> int:
|
||||
"""Return the exclusive source rectangle bottom edge."""
|
||||
|
||||
return self.source_y + self.source_height
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class SpatialBatchLayout:
|
||||
"""Describe ordered spatial views expanded over one source model batch."""
|
||||
|
||||
canvas_width: int
|
||||
canvas_height: int
|
||||
views: tuple[SpatialView, ...]
|
||||
input_batch_size: int
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Validate canvas containment and homogeneous model-call semantics."""
|
||||
|
||||
if self.canvas_width < 1 or self.canvas_height < 1:
|
||||
raise ValueError("Spatial layout canvas dimensions must be positive.")
|
||||
if self.input_batch_size < 1:
|
||||
raise ValueError("Spatial layout input batch size must be positive.")
|
||||
if not isinstance(self.views, tuple):
|
||||
raise TypeError("Spatial layout views must be an immutable tuple.")
|
||||
if not self.views:
|
||||
raise ValueError("Spatial layout requires at least one view.")
|
||||
if not all(isinstance(view, SpatialView) for view in self.views):
|
||||
raise TypeError("Spatial layout views must contain SpatialView values.")
|
||||
|
||||
view_kind = self.views[0].kind
|
||||
if any(view.kind is not view_kind for view in self.views):
|
||||
raise ValueError("One spatial model call cannot mix view kinds.")
|
||||
for view in self.views:
|
||||
if (
|
||||
view.source_right > self.canvas_width
|
||||
or view.source_bottom > self.canvas_height
|
||||
):
|
||||
raise ValueError(
|
||||
"Spatial view source rectangle must remain inside the canvas."
|
||||
)
|
||||
|
||||
if view_kind in {
|
||||
SpatialViewKind.FULL,
|
||||
SpatialViewKind.CONTEXTUAL_GLOBAL,
|
||||
}:
|
||||
if len(self.views) != 1:
|
||||
raise ValueError("A full-source spatial layout requires one view.")
|
||||
view = self.views[0]
|
||||
if (
|
||||
view.source_x != 0
|
||||
or view.source_y != 0
|
||||
or view.source_width != self.canvas_width
|
||||
or view.source_height != self.canvas_height
|
||||
):
|
||||
raise ValueError(
|
||||
"A full-source spatial view must cover the complete canvas."
|
||||
)
|
||||
|
||||
@property
|
||||
def view_count(self) -> int:
|
||||
"""Return the number of ordered spatial views."""
|
||||
|
||||
return len(self.views)
|
||||
|
||||
@property
|
||||
def expanded_batch_size(self) -> int:
|
||||
"""Return the model batch size after view-major expansion."""
|
||||
|
||||
return self.view_count * self.input_batch_size
|
||||
|
||||
@property
|
||||
def expanded_views(self) -> tuple[SpatialView, ...]:
|
||||
"""Repeat each view for its contiguous source-batch group."""
|
||||
|
||||
return tuple(
|
||||
view
|
||||
for view in self.views
|
||||
for _source_batch_index in range(self.input_batch_size)
|
||||
)
|
||||
|
||||
@property
|
||||
def expanded_view_indices(self) -> tuple[int, ...]:
|
||||
"""Return the view index for every expanded model-batch entry."""
|
||||
|
||||
return tuple(
|
||||
view_index
|
||||
for view_index in range(self.view_count)
|
||||
for _source_batch_index in range(self.input_batch_size)
|
||||
)
|
||||
|
||||
@property
|
||||
def expanded_source_batch_indices(self) -> tuple[int, ...]:
|
||||
"""Return the source-batch index for every expanded model-batch entry."""
|
||||
|
||||
return tuple(
|
||||
source_batch_index
|
||||
for _view in self.views
|
||||
for source_batch_index in range(self.input_batch_size)
|
||||
)
|
||||
|
||||
def expanded_index(self, view_index: int, source_batch_index: int) -> int:
|
||||
"""Return one view-major model-batch index after validating both axes."""
|
||||
|
||||
if not 0 <= view_index < self.view_count:
|
||||
raise IndexError("Spatial view index is outside the layout.")
|
||||
if not 0 <= source_batch_index < self.input_batch_size:
|
||||
raise IndexError("Source batch index is outside the layout.")
|
||||
return view_index * self.input_batch_size + source_batch_index
|
||||
@@ -20,12 +20,13 @@ TILED_DIFFUSION_MODES = ("multidiffusion", "mixture_of_diffusers")
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class LatentTile:
|
||||
"""Describe one rectangular latent-space tile."""
|
||||
"""Describe one rectangular latent-space tile and optional local blend weights."""
|
||||
|
||||
x: int
|
||||
y: int
|
||||
width: int
|
||||
height: int
|
||||
weight_mask: torch.Tensor | None = None
|
||||
|
||||
@property
|
||||
def slicer(self) -> tuple[slice, slice, slice, slice]:
|
||||
@@ -83,7 +84,7 @@ def build_tiled_diffusion_plan(
|
||||
tile_height=effective_tile_height,
|
||||
overlap=effective_overlap,
|
||||
)
|
||||
batches, effective_tile_batch_size = _batch_tiles(tiles, tile_batch_size)
|
||||
batches, effective_tile_batch_size = batch_latent_tiles(tiles, tile_batch_size)
|
||||
return TiledDiffusionPlan(
|
||||
latent_width=latent_width,
|
||||
latent_height=latent_height,
|
||||
@@ -214,7 +215,7 @@ def _split_tiles(
|
||||
return tuple(tiles)
|
||||
|
||||
|
||||
def _batch_tiles(
|
||||
def batch_latent_tiles(
|
||||
tiles: tuple[LatentTile, ...],
|
||||
requested_tile_batch_size: int,
|
||||
) -> tuple[tuple[tuple[LatentTile, ...], ...], int]:
|
||||
|
||||
@@ -2,18 +2,19 @@
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Connected-component helpers for binary mask regions."""
|
||||
"""Extract deterministic connected components from binary mask regions."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from importlib import import_module
|
||||
|
||||
import torch
|
||||
|
||||
from ..domain.segs import BoundingBox
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class MaskComponent:
|
||||
"""Represent one connected mask component and its full-image bbox."""
|
||||
|
||||
@@ -22,62 +23,30 @@ class MaskComponent:
|
||||
|
||||
|
||||
def connected_mask_components(active_mask: torch.Tensor) -> tuple[MaskComponent, ...]:
|
||||
"""Return 8-connected components from an HW active-pixel mask."""
|
||||
"""Return spatially ordered 8-connected components from one HW mask."""
|
||||
|
||||
if active_mask.ndim != 2:
|
||||
if not isinstance(active_mask, torch.Tensor) or active_mask.ndim != 2:
|
||||
raise ValueError("active_mask must be an HW tensor.")
|
||||
|
||||
active = active_mask.detach().to(device="cpu", dtype=torch.bool)
|
||||
height = int(active.shape[0])
|
||||
width = int(active.shape[1])
|
||||
visited = torch.zeros((height, width), dtype=torch.bool)
|
||||
active = active_mask.detach().to(device="cpu", dtype=torch.uint8).numpy()
|
||||
if not active.any():
|
||||
return ()
|
||||
cv2 = import_module("cv2")
|
||||
component_count, labels, stats, _centroids = cv2.connectedComponentsWithStats(
|
||||
active,
|
||||
connectivity=8,
|
||||
)
|
||||
components: list[MaskComponent] = []
|
||||
|
||||
for top in range(height):
|
||||
for left in range(width):
|
||||
if visited[top, left].item() or not active[top, left].item():
|
||||
continue
|
||||
components.append(_trace_component(active, visited, left, top))
|
||||
|
||||
return tuple(components)
|
||||
|
||||
|
||||
def _trace_component(
|
||||
active: torch.Tensor,
|
||||
visited: torch.Tensor,
|
||||
start_left: int,
|
||||
start_top: int,
|
||||
) -> MaskComponent:
|
||||
"""Trace one 8-connected component from its first active pixel."""
|
||||
|
||||
height = int(active.shape[0])
|
||||
width = int(active.shape[1])
|
||||
queue: list[tuple[int, int]] = [(start_top, start_left)]
|
||||
visited[start_top, start_left] = True
|
||||
pixels: list[tuple[int, int]] = []
|
||||
index = 0
|
||||
|
||||
while index < len(queue):
|
||||
top, left = queue[index]
|
||||
index += 1
|
||||
pixels.append((top, left))
|
||||
for neighbor_top in range(max(0, top - 1), min(height, top + 2)):
|
||||
for neighbor_left in range(max(0, left - 1), min(width, left + 2)):
|
||||
if visited[neighbor_top, neighbor_left].item():
|
||||
continue
|
||||
visited[neighbor_top, neighbor_left] = True
|
||||
if active[neighbor_top, neighbor_left].item():
|
||||
queue.append((neighbor_top, neighbor_left))
|
||||
|
||||
top_values = [top for top, _left in pixels]
|
||||
left_values = [left for _top, left in pixels]
|
||||
bbox = BoundingBox(
|
||||
min(left_values),
|
||||
min(top_values),
|
||||
max(left_values) + 1,
|
||||
max(top_values) + 1,
|
||||
for label in range(1, int(component_count)):
|
||||
left = int(stats[label, cv2.CC_STAT_LEFT])
|
||||
top = int(stats[label, cv2.CC_STAT_TOP])
|
||||
width = int(stats[label, cv2.CC_STAT_WIDTH])
|
||||
height = int(stats[label, cv2.CC_STAT_HEIGHT])
|
||||
components.append(
|
||||
MaskComponent(
|
||||
bbox=BoundingBox(left, top, left + width, top + height),
|
||||
mask=torch.from_numpy(labels == label),
|
||||
)
|
||||
)
|
||||
return tuple(
|
||||
sorted(components, key=lambda value: (value.bbox.top, value.bbox.left))
|
||||
)
|
||||
component_mask = torch.zeros((height, width), dtype=torch.bool)
|
||||
for top, left in pixels:
|
||||
component_mask[top, left] = True
|
||||
return MaskComponent(bbox=bbox, mask=component_mask)
|
||||
|
||||
@@ -0,0 +1,194 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Project canonical masks into exact regional activation multiplier shapes."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
import torch
|
||||
|
||||
from ..domain.regional_activation_geometry import (
|
||||
RegionalActivationGeometry,
|
||||
RegionalActivationLayout,
|
||||
)
|
||||
from ..domain.regional_mask_bank import RegionalMaskBank
|
||||
from ..domain.spatial_views import (
|
||||
SpatialBatchLayout,
|
||||
SpatialView,
|
||||
SpatialViewKind,
|
||||
)
|
||||
from .regional_mask_projection import (
|
||||
RegionalMaskForm,
|
||||
RegionalMaskProjectionMode,
|
||||
RegionalMaskProjector,
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RegionalActivationMaskBatch:
|
||||
"""Retain one finite region-major multiplier broadcast over rank activations."""
|
||||
|
||||
multipliers: torch.Tensor
|
||||
geometry: RegionalActivationGeometry
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require exact geometry shape and bounded finite floating multipliers."""
|
||||
|
||||
if not isinstance(self.multipliers, torch.Tensor):
|
||||
raise TypeError("Regional activation multipliers must be a tensor.")
|
||||
if not isinstance(self.geometry, RegionalActivationGeometry):
|
||||
raise TypeError("Regional activation multipliers require geometry.")
|
||||
if not self.multipliers.is_floating_point():
|
||||
raise TypeError("Regional activation multipliers must use floating point.")
|
||||
if self.multipliers.ndim != len(self.geometry.invocation_shape) + 1:
|
||||
raise ValueError("Regional activation multiplier rank is inconsistent.")
|
||||
expected = self.geometry.broadcast_mask_shape(int(self.multipliers.shape[0]))
|
||||
if tuple(self.multipliers.shape) != expected:
|
||||
raise ValueError(
|
||||
"Regional activation multiplier shape must match its geometry."
|
||||
)
|
||||
if not bool(torch.isfinite(self.multipliers).all()):
|
||||
raise ValueError("Regional activation multipliers must be finite.")
|
||||
if not bool(((self.multipliers >= 0.0) & (self.multipliers <= 1.0)).all()):
|
||||
raise ValueError("Regional activation multipliers must stay within [0, 1].")
|
||||
|
||||
|
||||
class RegionalActivationMaskProjector:
|
||||
"""Own geometry-shaped mask projection around existing crop/interpolation."""
|
||||
|
||||
def __init__(self, projector: RegionalMaskProjector | None = None) -> None:
|
||||
"""Retain the canonical mask crop and interpolation authority."""
|
||||
|
||||
self._projector = projector or RegionalMaskProjector()
|
||||
|
||||
def project(
|
||||
self,
|
||||
*,
|
||||
bank: RegionalMaskBank,
|
||||
geometry: RegionalActivationGeometry,
|
||||
form: RegionalMaskForm,
|
||||
mode: RegionalMaskProjectionMode,
|
||||
device: torch.device,
|
||||
dtype: torch.dtype,
|
||||
) -> RegionalActivationMaskBatch:
|
||||
"""Return masks in region/view/chunk/latent activation order."""
|
||||
|
||||
_validate_inputs(bank, geometry, form, mode, device, dtype)
|
||||
layout = geometry.batch_alignment.spatial_layout or _full_layout(
|
||||
bank,
|
||||
input_batch_size=geometry.batch_alignment.base_batch_size,
|
||||
)
|
||||
projected_views = tuple(
|
||||
self._projector.project_query_grid(
|
||||
bank=bank,
|
||||
layout=layout,
|
||||
view_index=view_index,
|
||||
query_height=geometry.spatial_height,
|
||||
query_width=geometry.spatial_width,
|
||||
form=form,
|
||||
mode=mode,
|
||||
)
|
||||
for view_index in range(layout.view_count)
|
||||
)
|
||||
masks = torch.cat(
|
||||
tuple(
|
||||
projected.unsqueeze(1).expand(
|
||||
-1,
|
||||
layout.input_batch_size,
|
||||
-1,
|
||||
-1,
|
||||
)
|
||||
for projected in projected_views
|
||||
),
|
||||
dim=1,
|
||||
).to(device=device, dtype=dtype)
|
||||
multipliers = _reshape_for_activation(masks, geometry)
|
||||
return RegionalActivationMaskBatch(multipliers, geometry)
|
||||
|
||||
|
||||
def _reshape_for_activation(
|
||||
masks: torch.Tensor,
|
||||
geometry: RegionalActivationGeometry,
|
||||
) -> torch.Tensor:
|
||||
"""Place projected H/W masks on the declared operation's non-feature axes."""
|
||||
|
||||
regions, batch, height, width = (int(value) for value in masks.shape)
|
||||
if geometry.layout is RegionalActivationLayout.DIRECT_CONVOLUTION_1D:
|
||||
if height != 1:
|
||||
raise ValueError("Conv1d regional masks require projected height one.")
|
||||
return masks.reshape(regions, batch, 1, width)
|
||||
if geometry.layout is RegionalActivationLayout.DIRECT_CONVOLUTION_2D:
|
||||
return masks.reshape(regions, batch, 1, height, width)
|
||||
if geometry.layout is RegionalActivationLayout.DIRECT_CONVOLUTION_3D:
|
||||
temporal_size = geometry.temporal_size
|
||||
if temporal_size is None:
|
||||
raise ValueError("Conv3d regional masks require explicit temporal size.")
|
||||
return masks.reshape(regions, batch, 1, 1, height, width).expand(
|
||||
-1,
|
||||
-1,
|
||||
-1,
|
||||
temporal_size,
|
||||
-1,
|
||||
-1,
|
||||
)
|
||||
if geometry.layout in (
|
||||
RegionalActivationLayout.FLATTENED_SPATIAL_TOKENS,
|
||||
RegionalActivationLayout.CONSUMER_SPATIALIZED,
|
||||
):
|
||||
return masks.flatten(start_dim=2).unsqueeze(-1)
|
||||
raise AssertionError(f"Unhandled regional activation layout: {geometry.layout}")
|
||||
|
||||
|
||||
def _full_layout(
|
||||
bank: RegionalMaskBank,
|
||||
*,
|
||||
input_batch_size: int,
|
||||
) -> SpatialBatchLayout:
|
||||
"""Represent one full-canvas invocation through the shared layout contract."""
|
||||
|
||||
return SpatialBatchLayout(
|
||||
bank.canvas_width,
|
||||
bank.canvas_height,
|
||||
(
|
||||
SpatialView(
|
||||
SpatialViewKind.FULL,
|
||||
0,
|
||||
0,
|
||||
bank.canvas_width,
|
||||
bank.canvas_height,
|
||||
bank.canvas_width,
|
||||
bank.canvas_height,
|
||||
),
|
||||
),
|
||||
input_batch_size,
|
||||
)
|
||||
|
||||
|
||||
def _validate_inputs(
|
||||
bank: object,
|
||||
geometry: object,
|
||||
form: object,
|
||||
mode: object,
|
||||
device: object,
|
||||
dtype: object,
|
||||
) -> None:
|
||||
"""Validate typed projection inputs without moving or mutating source masks."""
|
||||
|
||||
if not isinstance(bank, RegionalMaskBank):
|
||||
raise TypeError("Regional activation projection requires a mask bank.")
|
||||
if not isinstance(geometry, RegionalActivationGeometry):
|
||||
raise TypeError("Regional activation projection requires geometry.")
|
||||
if not isinstance(form, RegionalMaskForm):
|
||||
raise TypeError("Regional activation mask form has an invalid type.")
|
||||
if not isinstance(mode, RegionalMaskProjectionMode):
|
||||
raise TypeError("Regional activation projection mode has an invalid type.")
|
||||
if not isinstance(device, torch.device):
|
||||
raise TypeError("Regional activation mask device must be a torch.device.")
|
||||
if not isinstance(dtype, torch.dtype) or not dtype.is_floating_point:
|
||||
raise TypeError("Regional activation mask dtype must be floating point.")
|
||||
|
||||
|
||||
REGIONAL_ACTIVATION_MASK_PROJECTOR = RegionalActivationMaskProjector()
|
||||
@@ -0,0 +1,125 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Classify active regional coverage on one projected query grid."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from enum import StrEnum
|
||||
|
||||
import torch
|
||||
|
||||
|
||||
class RegionalCoverageClass(StrEnum):
|
||||
"""Identify model branches required by one regional mask projection."""
|
||||
|
||||
ALL_BASE = "all_base"
|
||||
ALL_SINGLE_REGION = "all_single_region"
|
||||
MIXED = "mixed"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class RegionalMaskActivation:
|
||||
"""Describe ordered active regions and an admitted coverage fast path."""
|
||||
|
||||
coverage_class: RegionalCoverageClass
|
||||
active_region_indices: tuple[int, ...]
|
||||
single_region_index: int | None
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""Require one internally consistent immutable classification."""
|
||||
|
||||
if not isinstance(self.coverage_class, RegionalCoverageClass):
|
||||
raise TypeError(
|
||||
"Regional coverage class must be a RegionalCoverageClass value."
|
||||
)
|
||||
if not isinstance(self.active_region_indices, tuple):
|
||||
raise TypeError("Active regional indices must be an immutable tuple.")
|
||||
if any(index < 0 for index in self.active_region_indices):
|
||||
raise ValueError("Active regional indices must be non-negative.")
|
||||
if tuple(sorted(set(self.active_region_indices))) != self.active_region_indices:
|
||||
raise ValueError("Active regional indices must be unique and ordered.")
|
||||
if self.coverage_class is RegionalCoverageClass.ALL_BASE:
|
||||
if self.active_region_indices or self.single_region_index is not None:
|
||||
raise ValueError("All-base coverage cannot contain an active region.")
|
||||
return
|
||||
if self.coverage_class is RegionalCoverageClass.ALL_SINGLE_REGION:
|
||||
if (
|
||||
len(self.active_region_indices) != 1
|
||||
or self.single_region_index != self.active_region_indices[0]
|
||||
):
|
||||
raise ValueError(
|
||||
"All-single-region coverage requires its one active region index."
|
||||
)
|
||||
return
|
||||
if not self.active_region_indices:
|
||||
raise ValueError("Mixed regional coverage requires an active region.")
|
||||
if self.single_region_index is not None:
|
||||
raise ValueError(
|
||||
"Mixed regional coverage cannot select a fast-path region."
|
||||
)
|
||||
|
||||
|
||||
class RegionalMaskActivationClassifier:
|
||||
"""Own zero-coverage pruning and regional fast-path admission."""
|
||||
|
||||
def classify(
|
||||
self,
|
||||
masks: torch.Tensor,
|
||||
*,
|
||||
tolerance: float = 1e-6,
|
||||
) -> RegionalMaskActivation:
|
||||
"""Return ordered active regions and the exact coverage classification."""
|
||||
|
||||
self._validate_masks(masks)
|
||||
if not 0.0 <= tolerance < 0.5:
|
||||
raise ValueError(
|
||||
"Regional activation tolerance must be at least 0 and below 0.5."
|
||||
)
|
||||
active_region_indices = tuple(
|
||||
index
|
||||
for index in range(int(masks.shape[0]))
|
||||
if bool((masks[index] > tolerance).any())
|
||||
)
|
||||
if not active_region_indices:
|
||||
return RegionalMaskActivation(
|
||||
coverage_class=RegionalCoverageClass.ALL_BASE,
|
||||
active_region_indices=(),
|
||||
single_region_index=None,
|
||||
)
|
||||
if len(active_region_indices) == 1:
|
||||
region_index = active_region_indices[0]
|
||||
if bool((masks[region_index] >= 1.0 - tolerance).all()):
|
||||
return RegionalMaskActivation(
|
||||
coverage_class=RegionalCoverageClass.ALL_SINGLE_REGION,
|
||||
active_region_indices=active_region_indices,
|
||||
single_region_index=region_index,
|
||||
)
|
||||
return RegionalMaskActivation(
|
||||
coverage_class=RegionalCoverageClass.MIXED,
|
||||
active_region_indices=active_region_indices,
|
||||
single_region_index=None,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _validate_masks(masks: torch.Tensor) -> None:
|
||||
"""Validate a projected normalized floating-point BHW mask batch."""
|
||||
|
||||
if not isinstance(masks, torch.Tensor):
|
||||
raise TypeError("Regional activation masks must be a torch.Tensor.")
|
||||
if masks.ndim != 3:
|
||||
raise ValueError("Regional activation masks must use BHW layout.")
|
||||
if int(masks.shape[0]) < 1:
|
||||
raise ValueError("Regional activation requires at least one region.")
|
||||
if int(masks.shape[1]) < 1 or int(masks.shape[2]) < 1:
|
||||
raise ValueError("Regional activation mask grids must be non-empty.")
|
||||
if not masks.is_floating_point():
|
||||
raise TypeError(
|
||||
"Regional activation masks must use a floating-point dtype."
|
||||
)
|
||||
if not bool(torch.isfinite(masks).all()):
|
||||
raise ValueError("Regional activation masks must contain finite values.")
|
||||
if not bool(((masks >= 0.0) & (masks <= 1.0)).all()):
|
||||
raise ValueError("Regional activation masks must stay within [0, 1].")
|
||||
@@ -0,0 +1,176 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Project canonical regional masks into spatial views and query grids."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import StrEnum
|
||||
|
||||
import torch
|
||||
import torch.nn.functional as functional
|
||||
|
||||
from ..domain.regional_mask_bank import RegionalMaskBank
|
||||
from ..domain.spatial_views import SpatialBatchLayout, SpatialView
|
||||
|
||||
|
||||
class RegionalMaskForm(StrEnum):
|
||||
"""Select one canonical regional mask representation."""
|
||||
|
||||
PLANNING = "planning"
|
||||
CONDITIONING = "conditioning"
|
||||
|
||||
|
||||
class RegionalMaskProjectionMode(StrEnum):
|
||||
"""Select the interpolation semantics for one mask projection."""
|
||||
|
||||
CONTINUOUS_COVERAGE = "continuous_coverage"
|
||||
SOFT = "soft"
|
||||
HARD_PRESERVING = "hard_preserving"
|
||||
NEAREST = "nearest"
|
||||
|
||||
|
||||
class RegionalMaskProjector:
|
||||
"""Own canonical-canvas crop and interpolation policy for regional masks."""
|
||||
|
||||
def project_view(
|
||||
self,
|
||||
*,
|
||||
bank: RegionalMaskBank,
|
||||
layout: SpatialBatchLayout,
|
||||
view_index: int,
|
||||
form: RegionalMaskForm,
|
||||
mode: RegionalMaskProjectionMode,
|
||||
) -> torch.Tensor:
|
||||
"""Project one canonical mask form to a view's model dimensions."""
|
||||
|
||||
view = self._view(bank=bank, layout=layout, view_index=view_index)
|
||||
return self.project_query_grid(
|
||||
bank=bank,
|
||||
layout=layout,
|
||||
view_index=view_index,
|
||||
query_height=view.model_height,
|
||||
query_width=view.model_width,
|
||||
form=form,
|
||||
mode=mode,
|
||||
)
|
||||
|
||||
def project_query_grid(
|
||||
self,
|
||||
*,
|
||||
bank: RegionalMaskBank,
|
||||
layout: SpatialBatchLayout,
|
||||
view_index: int,
|
||||
query_height: int,
|
||||
query_width: int,
|
||||
form: RegionalMaskForm,
|
||||
mode: RegionalMaskProjectionMode,
|
||||
) -> torch.Tensor:
|
||||
"""Crop one spatial view and resample it directly to an attention grid."""
|
||||
|
||||
if query_height < 1 or query_width < 1:
|
||||
raise ValueError("Regional mask query-grid dimensions must be positive.")
|
||||
if not isinstance(form, RegionalMaskForm):
|
||||
raise TypeError("Regional mask form must be a RegionalMaskForm value.")
|
||||
if not isinstance(mode, RegionalMaskProjectionMode):
|
||||
raise TypeError(
|
||||
"Regional mask projection mode must be a "
|
||||
"RegionalMaskProjectionMode value."
|
||||
)
|
||||
view = self._view(bank=bank, layout=layout, view_index=view_index)
|
||||
masks = (
|
||||
bank.planning_masks
|
||||
if form is RegionalMaskForm.PLANNING
|
||||
else bank.conditioning_masks
|
||||
)
|
||||
cropped = masks[
|
||||
:,
|
||||
view.source_y : view.source_bottom,
|
||||
view.source_x : view.source_right,
|
||||
]
|
||||
projected = self._resize(
|
||||
cropped,
|
||||
height=query_height,
|
||||
width=query_width,
|
||||
mode=mode,
|
||||
).clamp(0.0, 1.0)
|
||||
if not bool(torch.isfinite(projected).all()):
|
||||
raise ValueError("Projected regional masks contain non-finite values.")
|
||||
return projected
|
||||
|
||||
@staticmethod
|
||||
def _view(
|
||||
*,
|
||||
bank: RegionalMaskBank,
|
||||
layout: SpatialBatchLayout,
|
||||
view_index: int,
|
||||
) -> SpatialView:
|
||||
"""Validate canonical canvas identity and return one indexed view."""
|
||||
|
||||
if (
|
||||
bank.canvas_width != layout.canvas_width
|
||||
or bank.canvas_height != layout.canvas_height
|
||||
):
|
||||
raise ValueError(
|
||||
"Regional mask bank canvas must match the spatial batch layout."
|
||||
)
|
||||
if view_index < 0 or view_index >= layout.view_count:
|
||||
raise IndexError("Regional mask spatial view index is outside the layout.")
|
||||
return layout.views[view_index]
|
||||
|
||||
@staticmethod
|
||||
def _resize(
|
||||
masks: torch.Tensor,
|
||||
*,
|
||||
height: int,
|
||||
width: int,
|
||||
mode: RegionalMaskProjectionMode,
|
||||
) -> torch.Tensor:
|
||||
"""Apply explicit coverage, soft, or hard-preserving interpolation."""
|
||||
|
||||
source_height = int(masks.shape[-2])
|
||||
source_width = int(masks.shape[-1])
|
||||
if (source_height, source_width) == (height, width):
|
||||
return masks
|
||||
batched = masks.unsqueeze(1)
|
||||
if mode is RegionalMaskProjectionMode.HARD_PRESERVING:
|
||||
return functional.interpolate(
|
||||
batched,
|
||||
size=(height, width),
|
||||
mode="nearest-exact",
|
||||
).squeeze(1)
|
||||
if mode is RegionalMaskProjectionMode.NEAREST:
|
||||
return functional.interpolate(
|
||||
batched,
|
||||
size=(height, width),
|
||||
mode="nearest",
|
||||
).squeeze(1)
|
||||
if mode is RegionalMaskProjectionMode.SOFT:
|
||||
return functional.interpolate(
|
||||
batched,
|
||||
size=(height, width),
|
||||
mode="bilinear",
|
||||
align_corners=False,
|
||||
).squeeze(1)
|
||||
|
||||
downsample_height = min(source_height, height)
|
||||
downsample_width = min(source_width, width)
|
||||
projected = batched
|
||||
if (downsample_height, downsample_width) != (
|
||||
source_height,
|
||||
source_width,
|
||||
):
|
||||
projected = functional.interpolate(
|
||||
projected,
|
||||
size=(downsample_height, downsample_width),
|
||||
mode="area",
|
||||
)
|
||||
if tuple(projected.shape[-2:]) != (height, width):
|
||||
projected = functional.interpolate(
|
||||
projected,
|
||||
size=(height, width),
|
||||
mode="bilinear",
|
||||
align_corners=False,
|
||||
)
|
||||
return projected.squeeze(1)
|
||||
@@ -0,0 +1,118 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Mask preparation for full-context regional prompting."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import torch
|
||||
import torch.nn.functional as functional
|
||||
|
||||
from ..domain.regional_mask_bank import RegionalMaskBank
|
||||
from .detailer_masks import gaussian_feather_mask
|
||||
|
||||
|
||||
def prepare_regional_mask_batch(mask: object, feather: int) -> torch.Tensor:
|
||||
"""Return a validated and optionally feathered BHW mask batch."""
|
||||
|
||||
_, conditioning = _prepare_regional_masks(mask, feather)
|
||||
return conditioning
|
||||
|
||||
|
||||
def build_regional_mask_bank(
|
||||
mask: object,
|
||||
*,
|
||||
feather: int,
|
||||
canvas_height: int,
|
||||
canvas_width: int,
|
||||
) -> RegionalMaskBank:
|
||||
"""Build separate planning and conditioning masks on one latent canvas."""
|
||||
|
||||
authored, feathered = _prepare_regional_masks(mask, feather)
|
||||
planning_masks = resize_regional_mask_batch(
|
||||
authored,
|
||||
height=canvas_height,
|
||||
width=canvas_width,
|
||||
)
|
||||
conditioning_masks = resize_regional_mask_batch(
|
||||
feathered,
|
||||
height=canvas_height,
|
||||
width=canvas_width,
|
||||
)
|
||||
return RegionalMaskBank(
|
||||
planning_masks=planning_masks,
|
||||
conditioning_masks=conditioning_masks,
|
||||
canvas_width=canvas_width,
|
||||
canvas_height=canvas_height,
|
||||
)
|
||||
|
||||
|
||||
def _prepare_regional_masks(
|
||||
mask: object,
|
||||
feather: int,
|
||||
) -> tuple[torch.Tensor, torch.Tensor]:
|
||||
"""Return normalized authored masks and a separate feathered tensor."""
|
||||
|
||||
if not isinstance(mask, torch.Tensor):
|
||||
raise TypeError("regional prompting requires a torch MASK tensor.")
|
||||
if feather < 0:
|
||||
raise ValueError("region_mask_feather must be greater than or equal to 0.")
|
||||
|
||||
working = mask.float()
|
||||
if working.ndim == 2:
|
||||
working = working.unsqueeze(0)
|
||||
if working.ndim != 3:
|
||||
raise ValueError("regional prompting requires an HW or BHW MASK tensor.")
|
||||
if int(working.shape[0]) < 1:
|
||||
raise ValueError("regional prompting requires at least one authored mask.")
|
||||
if int(working.shape[1]) < 1 or int(working.shape[2]) < 1:
|
||||
raise ValueError("regional masks must have non-empty height and width.")
|
||||
|
||||
normalized = working.clamp(0.0, 1.0)
|
||||
conditioning = (
|
||||
normalized.to(device=normalized.device, dtype=normalized.dtype, copy=True)
|
||||
if feather == 0
|
||||
else gaussian_feather_mask(normalized, feather)
|
||||
)
|
||||
return normalized, conditioning
|
||||
|
||||
|
||||
def resize_regional_mask_batch(
|
||||
mask_batch: torch.Tensor,
|
||||
*,
|
||||
height: int,
|
||||
width: int,
|
||||
) -> torch.Tensor:
|
||||
"""Resize BHW masks while preserving authored area during downscaling."""
|
||||
|
||||
if height < 1 or width < 1:
|
||||
raise ValueError("regional mask target height and width must be positive.")
|
||||
if tuple(mask_batch.shape[1:]) == (height, width):
|
||||
return mask_batch
|
||||
source_height, source_width = map(int, mask_batch.shape[1:])
|
||||
downscaled_height = min(height, source_height)
|
||||
downscaled_width = min(width, source_width)
|
||||
working = mask_batch.unsqueeze(1)
|
||||
if (downscaled_height, downscaled_width) != (source_height, source_width):
|
||||
working = functional.interpolate(
|
||||
working,
|
||||
size=(downscaled_height, downscaled_width),
|
||||
mode="area",
|
||||
)
|
||||
if (downscaled_height, downscaled_width) != (height, width):
|
||||
working = functional.interpolate(
|
||||
working,
|
||||
size=(height, width),
|
||||
mode="bilinear",
|
||||
align_corners=False,
|
||||
)
|
||||
return working.squeeze(1)
|
||||
|
||||
|
||||
def regional_mask(mask_batch: torch.Tensor, index: int) -> torch.Tensor:
|
||||
"""Return one positional region as a singleton BHW mask."""
|
||||
|
||||
if index < 0 or index >= int(mask_batch.shape[0]):
|
||||
raise IndexError(f"regional mask index {index} is out of range.")
|
||||
return mask_batch[index : index + 1]
|
||||
@@ -8,8 +8,9 @@ from __future__ import annotations
|
||||
|
||||
from typing import Any, ClassVar
|
||||
|
||||
from ..domain.conditioning_batch import split_prompt_batch
|
||||
from ..domain.prompt_batch_parser import DEFAULT_PROMPT_BATCH_SEPARATOR
|
||||
from ..runtime.conditioning_encoding import ComfyConditioningEncoder
|
||||
from ..services.prompt_batch_encoding_service import PromptBatchEncodingService
|
||||
|
||||
|
||||
class EncodePromptBatch:
|
||||
@@ -18,17 +19,22 @@ class EncodePromptBatch:
|
||||
RETURN_TYPES = ("CONDITIONING_BATCH", "CONDITIONING_BATCH")
|
||||
RETURN_NAMES = ("positive", "negative")
|
||||
OUTPUT_TOOLTIPS = (
|
||||
"Positive conditioning entries selected by SEGS order.",
|
||||
"Negative conditioning entries selected by SEGS order.",
|
||||
"Ordered positive conditioning entries for batch-aware consumers.",
|
||||
"Ordered negative conditioning entries for batch-aware consumers.",
|
||||
)
|
||||
FUNCTION = "encode"
|
||||
CATEGORY = "SimpleSyrup/Conditioning"
|
||||
DESCRIPTION = (
|
||||
"Encodes [SEP]-separated prompts into per-segment conditioning batches."
|
||||
"Encodes prompts separated by [SEP] or [SEP|name] into matched "
|
||||
"conditioning batches, reusing each side's global prompt when a "
|
||||
"regional entry is missing."
|
||||
)
|
||||
SEARCH_ALIASES = ["conditioning batch", "prompt batch", "segs prompts"]
|
||||
|
||||
encoder_class: ClassVar[type[ComfyConditioningEncoder]] = ComfyConditioningEncoder
|
||||
service_class: ClassVar[type[PromptBatchEncodingService]] = (
|
||||
PromptBatchEncodingService
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls) -> dict[str, dict[str, tuple[Any, ...]]]:
|
||||
@@ -51,7 +57,9 @@ class EncodePromptBatch:
|
||||
"default": "",
|
||||
"multiline": True,
|
||||
"tooltip": (
|
||||
"Positive prompts in SEGS order, separated by [SEP]."
|
||||
"Ordered positive prompt entries separated by [SEP] "
|
||||
"or [SEP|name]; the global entry fills missing "
|
||||
"positive regions."
|
||||
),
|
||||
},
|
||||
),
|
||||
@@ -61,15 +69,21 @@ class EncodePromptBatch:
|
||||
"default": "",
|
||||
"multiline": True,
|
||||
"tooltip": (
|
||||
"Negative prompts in SEGS order, separated by [SEP]."
|
||||
"Ordered negative prompt entries separated by [SEP] "
|
||||
"or [SEP|name]; the global entry fills missing "
|
||||
"negative regions."
|
||||
),
|
||||
},
|
||||
),
|
||||
"separator": (
|
||||
"STRING",
|
||||
{
|
||||
"default": "[SEP]",
|
||||
"tooltip": "Text marker that separates prompt entries.",
|
||||
"default": DEFAULT_PROMPT_BATCH_SEPARATOR,
|
||||
"tooltip": (
|
||||
"Text marker that separates prompt entries. With the "
|
||||
"default [SEP], use [SEP|name] to add an organizational "
|
||||
"label."
|
||||
),
|
||||
},
|
||||
),
|
||||
}
|
||||
@@ -84,10 +98,9 @@ class EncodePromptBatch:
|
||||
) -> tuple[object, object]:
|
||||
"""Encode positive and negative prompt batches."""
|
||||
|
||||
encoder = self.encoder_class()
|
||||
positive_chunks = split_prompt_batch(positive_prompt, separator)
|
||||
negative_chunks = split_prompt_batch(negative_prompt, separator)
|
||||
return (
|
||||
encoder.encode_batch(clip, positive_chunks),
|
||||
encoder.encode_batch(clip, negative_chunks),
|
||||
return self.service_class(self.encoder_class()).encode(
|
||||
clip=clip,
|
||||
positive_prompt=positive_prompt,
|
||||
negative_prompt=negative_prompt,
|
||||
separator=separator,
|
||||
)
|
||||
|
||||
@@ -8,7 +8,7 @@ from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from ..runtime.model_catalog import grounding_dino_choices, sam_choices
|
||||
from ..runtime.model_choices import ModelChoiceService, default_choice
|
||||
from ..runtime.model_metadata import GroundedSAMModelMetadata
|
||||
from . import tooltips
|
||||
|
||||
@@ -17,6 +17,7 @@ class GroundedSAMModelInfo:
|
||||
"""Expose selected grounded SAM source and local path metadata."""
|
||||
|
||||
_metadata = GroundedSAMModelMetadata()
|
||||
_choices = ModelChoiceService()
|
||||
|
||||
RETURN_TYPES = ("STRING",)
|
||||
RETURN_NAMES = ("model_info",)
|
||||
@@ -31,19 +32,27 @@ class GroundedSAMModelInfo:
|
||||
def INPUT_TYPES(cls) -> dict[str, dict[str, tuple[Any, ...]]]:
|
||||
"""Declare deterministic model metadata inputs."""
|
||||
|
||||
sam_model_choices = cls._choices.sam_choices()
|
||||
grounding_dino_model_choices = cls._choices.grounding_dino_choices()
|
||||
return {
|
||||
"required": {
|
||||
"sam_model": (
|
||||
sam_choices(),
|
||||
sam_model_choices,
|
||||
{
|
||||
"default": "sam_hq_vit_b (379MB)",
|
||||
"default": default_choice(
|
||||
sam_model_choices,
|
||||
"sam_hq_vit_b (379MB)",
|
||||
),
|
||||
"tooltip": tooltips.SAM_MODEL_INPUT,
|
||||
},
|
||||
),
|
||||
"grounding_dino_model": (
|
||||
grounding_dino_choices(),
|
||||
grounding_dino_model_choices,
|
||||
{
|
||||
"default": "GroundingDINO_SwinT_OGC (694MB)",
|
||||
"default": default_choice(
|
||||
grounding_dino_model_choices,
|
||||
"GroundingDINO_SwinT_OGC (694MB)",
|
||||
),
|
||||
"tooltip": tooltips.GROUNDING_DINO_MODEL_INPUT,
|
||||
},
|
||||
),
|
||||
@@ -53,4 +62,6 @@ class GroundedSAMModelInfo:
|
||||
def describe(self, sam_model: str, grounding_dino_model: str) -> tuple[str]:
|
||||
"""Return JSON metadata for selected model entries."""
|
||||
|
||||
self._choices.reject_sentinel(sam_model)
|
||||
self._choices.reject_sentinel(grounding_dino_model)
|
||||
return (self._metadata.describe_selection(sam_model, grounding_dino_model),)
|
||||
|
||||
@@ -6,20 +6,17 @@
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import Any
|
||||
from typing import Any, ClassVar
|
||||
|
||||
import torch
|
||||
|
||||
from ..domain.conditioning_batch import ConditioningBatch, select_conditioning
|
||||
from ..runtime import sampling_samplers, sampling_schedulers
|
||||
from ..services.ksampler_sampling_service import KSamplerSamplingService
|
||||
from . import tooltips
|
||||
|
||||
Latent = dict[str, Any]
|
||||
|
||||
|
||||
class KSamplerExtras:
|
||||
"""Expose KSampler-style sampling with AYS and GITS scheduler options."""
|
||||
"""Expose KSampler-style sampling with extended scheduler options."""
|
||||
|
||||
RETURN_TYPES = ("LATENT",)
|
||||
OUTPUT_TOOLTIPS = (tooltips.DENOISED_LATENT_OUTPUT,)
|
||||
@@ -30,6 +27,7 @@ class KSamplerExtras:
|
||||
"compatible workflows."
|
||||
)
|
||||
SEARCH_ALIASES = ["ksampler", "sampler", "ays", "gits", "lcm"]
|
||||
service_class: ClassVar[type[KSamplerSamplingService]] = KSamplerSamplingService
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls) -> dict[str, dict[str, tuple[Any, ...]]]:
|
||||
@@ -113,151 +111,16 @@ class KSamplerExtras:
|
||||
) -> tuple[Latent]:
|
||||
"""Sample a latent with ComfyUI samplers and extra scheduler sigmas."""
|
||||
|
||||
sampler = sampling_samplers.resolve_sampler(sampler_name)
|
||||
sigmas = sampling_schedulers.calculate_sigmas(
|
||||
output = self.service_class().sample(
|
||||
model=model,
|
||||
scheduler_name=scheduler,
|
||||
sampler_name=sampler_name,
|
||||
seed=seed,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
sampler_name=sampler_name,
|
||||
scheduler=scheduler,
|
||||
positive=positive,
|
||||
negative=negative,
|
||||
latent_image=latent_image,
|
||||
denoise=denoise,
|
||||
).to(model.load_device)
|
||||
|
||||
latent_samples = latent_image["samples"]
|
||||
comfy_sample = _comfy_sample()
|
||||
comfy_utils = _comfy_utils()
|
||||
latent_preview = _latent_preview()
|
||||
|
||||
latent_samples = comfy_sample.fix_empty_latent_channels(
|
||||
model,
|
||||
latent_samples,
|
||||
latent_image.get("downscale_ratio_spacial", None),
|
||||
)
|
||||
|
||||
batch_inds = (
|
||||
latent_image["batch_index"] if "batch_index" in latent_image else None
|
||||
)
|
||||
noise = comfy_sample.prepare_noise(latent_samples, seed, batch_inds)
|
||||
noise_mask = latent_image.get("noise_mask", None)
|
||||
|
||||
callback = latent_preview.prepare_callback(model, steps)
|
||||
disable_pbar = not comfy_utils.PROGRESS_BAR_ENABLED
|
||||
if _uses_conditioning_batch(positive, negative):
|
||||
samples = _sample_conditioning_batch(
|
||||
comfy_sample=comfy_sample,
|
||||
model=model,
|
||||
noise=noise,
|
||||
cfg=cfg,
|
||||
sampler=sampler,
|
||||
sigmas=sigmas,
|
||||
positive=positive,
|
||||
negative=negative,
|
||||
latent_samples=latent_samples,
|
||||
noise_mask=noise_mask,
|
||||
callback=callback,
|
||||
disable_pbar=disable_pbar,
|
||||
seed=seed,
|
||||
)
|
||||
else:
|
||||
samples = comfy_sample.sample_custom(
|
||||
model,
|
||||
noise,
|
||||
cfg,
|
||||
sampler,
|
||||
sigmas,
|
||||
positive,
|
||||
negative,
|
||||
latent_samples,
|
||||
noise_mask=noise_mask,
|
||||
callback=callback,
|
||||
disable_pbar=disable_pbar,
|
||||
seed=seed,
|
||||
)
|
||||
|
||||
output = latent_image.copy()
|
||||
output.pop("downscale_ratio_spacial", None)
|
||||
output["samples"] = samples
|
||||
return (output,)
|
||||
|
||||
|
||||
def _comfy_sample() -> Any:
|
||||
"""Import ComfyUI sample helpers lazily."""
|
||||
|
||||
import comfy.sample
|
||||
|
||||
return comfy.sample
|
||||
|
||||
|
||||
def _uses_conditioning_batch(positive: Any, negative: Any) -> bool:
|
||||
"""Return whether either conditioning input needs per-item selection."""
|
||||
|
||||
return isinstance(positive, ConditioningBatch) or isinstance(
|
||||
negative,
|
||||
ConditioningBatch,
|
||||
)
|
||||
|
||||
|
||||
def _sample_conditioning_batch(
|
||||
*,
|
||||
comfy_sample: Any,
|
||||
model: Any,
|
||||
noise: torch.Tensor,
|
||||
cfg: float,
|
||||
sampler: Any,
|
||||
sigmas: torch.Tensor,
|
||||
positive: Any,
|
||||
negative: Any,
|
||||
latent_samples: torch.Tensor,
|
||||
noise_mask: Any,
|
||||
callback: Any,
|
||||
disable_pbar: bool,
|
||||
seed: int,
|
||||
) -> torch.Tensor:
|
||||
"""Sample each latent batch item with its selected conditioning."""
|
||||
|
||||
sampled: list[torch.Tensor] = []
|
||||
for index in range(int(latent_samples.shape[0])):
|
||||
sampled.append(
|
||||
comfy_sample.sample_custom(
|
||||
model,
|
||||
noise[index : index + 1],
|
||||
cfg,
|
||||
sampler,
|
||||
sigmas,
|
||||
select_conditioning(positive, index),
|
||||
select_conditioning(negative, index),
|
||||
latent_samples[index : index + 1],
|
||||
noise_mask=_slice_noise_mask(noise_mask, index, latent_samples),
|
||||
callback=callback,
|
||||
disable_pbar=disable_pbar,
|
||||
seed=seed,
|
||||
)
|
||||
)
|
||||
return torch.cat(sampled, dim=0)
|
||||
|
||||
|
||||
def _slice_noise_mask(
|
||||
noise_mask: Any,
|
||||
index: int,
|
||||
latent_samples: torch.Tensor,
|
||||
) -> Any:
|
||||
"""Return the noise mask slice matching one latent batch item."""
|
||||
|
||||
if isinstance(noise_mask, torch.Tensor) and noise_mask.shape[0] == int(
|
||||
latent_samples.shape[0],
|
||||
):
|
||||
return noise_mask[index : index + 1]
|
||||
return noise_mask
|
||||
|
||||
|
||||
def _comfy_utils() -> Any:
|
||||
"""Import ComfyUI utility state lazily."""
|
||||
|
||||
import comfy.utils
|
||||
|
||||
return comfy.utils
|
||||
|
||||
|
||||
def _latent_preview() -> Any:
|
||||
"""Import ComfyUI preview helpers lazily."""
|
||||
|
||||
return import_module("latent_preview")
|
||||
|
||||
@@ -1,197 +0,0 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""ComfyUI node declaration for selectable tiled diffusion sampling."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, ClassVar
|
||||
|
||||
from ..domain.tiled_diffusion import TILED_DIFFUSION_MODES
|
||||
from ..runtime import sampling_samplers, sampling_schedulers
|
||||
from ..services.tiled_diffusion_sampling_service import TiledDiffusionSamplingService
|
||||
from . import tooltips
|
||||
|
||||
Latent = dict[str, Any]
|
||||
MAX_LATENT_TILE_SIZE = 512
|
||||
|
||||
|
||||
class KSamplerTiledDiffusion:
|
||||
"""Sample latents with selectable tiled diffusion denoising."""
|
||||
|
||||
RETURN_TYPES = ("LATENT",)
|
||||
OUTPUT_TOOLTIPS = (tooltips.DENOISED_LATENT_OUTPUT,)
|
||||
FUNCTION = "sample"
|
||||
CATEGORY = "SimpleSyrup/Sampling"
|
||||
DESCRIPTION = "Denoises latents with selectable tiled diffusion sampling."
|
||||
SEARCH_ALIASES = [
|
||||
"ksampler",
|
||||
"sampler",
|
||||
"tiled diffusion",
|
||||
"multidiffusion",
|
||||
"multi diffusion",
|
||||
"mixture of diffusers",
|
||||
]
|
||||
|
||||
service_class: ClassVar[type[TiledDiffusionSamplingService]] = (
|
||||
TiledDiffusionSamplingService
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls) -> dict[str, dict[str, tuple[Any, ...]]]:
|
||||
"""Declare ComfyUI inputs for selectable tiled diffusion sampling."""
|
||||
|
||||
return {
|
||||
"required": {
|
||||
"model": ("MODEL", {"tooltip": tooltips.SAMPLING_MODEL}),
|
||||
"seed": (
|
||||
"INT",
|
||||
{
|
||||
"default": 0,
|
||||
"min": 0,
|
||||
"max": 0xFFFFFFFFFFFFFFFF,
|
||||
"control_after_generate": True,
|
||||
"tooltip": tooltips.SAMPLING_SEED,
|
||||
},
|
||||
),
|
||||
"steps": (
|
||||
"INT",
|
||||
{
|
||||
"default": 20,
|
||||
"min": 1,
|
||||
"max": 10000,
|
||||
"tooltip": tooltips.SAMPLING_STEPS,
|
||||
},
|
||||
),
|
||||
"cfg": (
|
||||
"FLOAT",
|
||||
{
|
||||
"default": 8.0,
|
||||
"min": 0.0,
|
||||
"max": 100.0,
|
||||
"step": 0.1,
|
||||
"round": 0.01,
|
||||
"tooltip": tooltips.SAMPLING_CFG,
|
||||
},
|
||||
),
|
||||
"sampler_name": (
|
||||
sampling_samplers.available_samplers(),
|
||||
{"tooltip": tooltips.SAMPLER_NAME},
|
||||
),
|
||||
"scheduler": (
|
||||
sampling_schedulers.available_schedulers(),
|
||||
{"tooltip": tooltips.SCHEDULER},
|
||||
),
|
||||
"positive": (
|
||||
"CONDITIONING,CONDITIONING_BATCH",
|
||||
{"tooltip": tooltips.POSITIVE_CONDITIONING},
|
||||
),
|
||||
"negative": (
|
||||
"CONDITIONING,CONDITIONING_BATCH",
|
||||
{"tooltip": tooltips.NEGATIVE_CONDITIONING},
|
||||
),
|
||||
"latent_image": ("LATENT", {"tooltip": tooltips.LATENT_IMAGE}),
|
||||
"denoise": (
|
||||
"FLOAT",
|
||||
{
|
||||
"default": 1.0,
|
||||
"min": 0.0,
|
||||
"max": 1.0,
|
||||
"step": 0.01,
|
||||
"tooltip": tooltips.DENOISE_STRENGTH,
|
||||
},
|
||||
),
|
||||
"diffusion_mode": (
|
||||
list(TILED_DIFFUSION_MODES),
|
||||
{
|
||||
"default": "multidiffusion",
|
||||
"tooltip": (
|
||||
"Tiled sampling blend method. MultiDiffusion is steady; "
|
||||
"Mixture of Diffusers can blend tile predictions more "
|
||||
"softly."
|
||||
),
|
||||
},
|
||||
),
|
||||
"latent_tile_width": (
|
||||
"INT",
|
||||
{
|
||||
"default": 128,
|
||||
"min": 16,
|
||||
"max": MAX_LATENT_TILE_SIZE,
|
||||
"step": 16,
|
||||
"tooltip": tooltips.LATENT_TILE_WIDTH,
|
||||
},
|
||||
),
|
||||
"latent_tile_height": (
|
||||
"INT",
|
||||
{
|
||||
"default": 128,
|
||||
"min": 16,
|
||||
"max": MAX_LATENT_TILE_SIZE,
|
||||
"step": 16,
|
||||
"tooltip": tooltips.LATENT_TILE_HEIGHT,
|
||||
},
|
||||
),
|
||||
"latent_tile_overlap": (
|
||||
"INT",
|
||||
{
|
||||
"default": 16,
|
||||
"min": 0,
|
||||
"max": 256,
|
||||
"step": 4,
|
||||
"tooltip": tooltips.LATENT_TILE_OVERLAP,
|
||||
},
|
||||
),
|
||||
"latent_tile_batch_size": (
|
||||
"INT",
|
||||
{
|
||||
"default": 4,
|
||||
"min": 1,
|
||||
"max": 8,
|
||||
"step": 1,
|
||||
"tooltip": tooltips.LATENT_TILE_BATCH_SIZE,
|
||||
},
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
def sample(
|
||||
self,
|
||||
model: Any,
|
||||
seed: int,
|
||||
steps: int,
|
||||
cfg: float,
|
||||
sampler_name: str,
|
||||
scheduler: str,
|
||||
positive: Any,
|
||||
negative: Any,
|
||||
latent_image: Latent,
|
||||
denoise: float = 1.0,
|
||||
diffusion_mode: str = "multidiffusion",
|
||||
latent_tile_width: int = 128,
|
||||
latent_tile_height: int = 128,
|
||||
latent_tile_overlap: int = 16,
|
||||
latent_tile_batch_size: int = 4,
|
||||
) -> tuple[Latent]:
|
||||
"""Sample a latent with the selected tiled diffusion method."""
|
||||
|
||||
output = self.service_class().sample(
|
||||
diffusion_mode=diffusion_mode,
|
||||
model=model,
|
||||
seed=seed,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
sampler_name=sampler_name,
|
||||
scheduler=scheduler,
|
||||
positive=positive,
|
||||
negative=negative,
|
||||
latent_image=latent_image,
|
||||
denoise=denoise,
|
||||
latent_tile_width=latent_tile_width,
|
||||
latent_tile_height=latent_tile_height,
|
||||
latent_tile_overlap=latent_tile_overlap,
|
||||
latent_tile_batch_size=latent_tile_batch_size,
|
||||
preview_context=None,
|
||||
)
|
||||
return (output,)
|
||||
@@ -8,6 +8,7 @@ from __future__ import annotations
|
||||
|
||||
from typing import Any, ClassVar
|
||||
|
||||
from ..runtime.model_downloads import ComfyProgressReporter
|
||||
from ..runtime.ultralytics_loader import UltralyticsLoaderService
|
||||
|
||||
|
||||
@@ -40,7 +41,8 @@ class LoadUltralyticsModel:
|
||||
{
|
||||
"default": choices[0],
|
||||
"tooltip": (
|
||||
"Ultralytics model file in the ComfyUI models folder."
|
||||
"A local Ultralytics model or a curated model that "
|
||||
"downloads to ComfyUI's Impact Pack-compatible folders."
|
||||
),
|
||||
},
|
||||
)
|
||||
@@ -50,5 +52,8 @@ class LoadUltralyticsModel:
|
||||
def load(self, model_name: str) -> tuple[object, object, object]:
|
||||
"""Load the selected detector and paired compatibility facades."""
|
||||
|
||||
loaded = self.service_class().load(model_name)
|
||||
loaded = self.service_class().load(
|
||||
model_name,
|
||||
progress=ComfyProgressReporter(),
|
||||
)
|
||||
return loaded.detector_model, loaded.bbox_detector, loaded.segm_detector
|
||||
|
||||
@@ -23,15 +23,16 @@ class ScheduleAndEncodePromptsWithPromptControl:
|
||||
)
|
||||
RETURN_NAMES = ("model", "positive", "negative")
|
||||
OUTPUT_TOOLTIPS = (
|
||||
"Model after LoRA tags from positive and negative prompts are scheduled.",
|
||||
"Model with single-prompt LoRAs applied; SEP-local LoRAs stay on conditioning.",
|
||||
"Positive conditioning or SimpleSyrup conditioning batch.",
|
||||
"Negative conditioning or SimpleSyrup conditioning batch.",
|
||||
)
|
||||
FUNCTION = "execute"
|
||||
CATEGORY = "SimpleSyrup/Conditioning"
|
||||
DESCRIPTION = (
|
||||
"Schedules Prompt-Control LoRAs and encodes prompts, using [SEP] to "
|
||||
"create SimpleSyrup conditioning batches."
|
||||
"Schedules Prompt-Control LoRAs and encodes prompts. [SEP] and "
|
||||
"[SEP|name] create matched conditioning batches using global text for "
|
||||
"missing regions."
|
||||
)
|
||||
SEARCH_ALIASES = ["prompt control", "schedule prompts", "encode prompts"]
|
||||
|
||||
@@ -67,8 +68,9 @@ class ScheduleAndEncodePromptsWithPromptControl:
|
||||
"default": "",
|
||||
"multiline": False,
|
||||
"tooltip": (
|
||||
"Positive Prompt-Control text. [SEP] creates a "
|
||||
"conditioning batch for SimpleSyrup batch-aware nodes."
|
||||
"Positive Prompt-Control text; [SEP] or [SEP|name] "
|
||||
"creates ordered entries, and global text fills missing "
|
||||
"positive regions."
|
||||
),
|
||||
},
|
||||
),
|
||||
@@ -78,8 +80,9 @@ class ScheduleAndEncodePromptsWithPromptControl:
|
||||
"default": "",
|
||||
"multiline": False,
|
||||
"tooltip": (
|
||||
"Negative Prompt-Control text. [SEP] creates a "
|
||||
"conditioning batch for SimpleSyrup batch-aware nodes."
|
||||
"Negative Prompt-Control text; [SEP] or [SEP|name] "
|
||||
"creates ordered entries, and global text fills missing "
|
||||
"negative regions."
|
||||
),
|
||||
},
|
||||
),
|
||||
|
||||
@@ -0,0 +1,124 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""ComfyUI node declaration for automatic SEGS from a SAM model."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from typing import Any, ClassVar
|
||||
|
||||
import torch
|
||||
|
||||
from ..masking.segs_mask_ops import iter_single_images, validate_image_batch
|
||||
from ..runtime.progress import PhaseProgressReporter, create_comfy_phase_progress
|
||||
from ..services.segs_from_sam_output_service import SEGSFromSAMOutputService
|
||||
|
||||
|
||||
class SEGSFromSAMOutput:
|
||||
"""Generate reusable unprompted SEGS from a connected SAM model."""
|
||||
|
||||
service_class: ClassVar[type[SEGSFromSAMOutputService]] = SEGSFromSAMOutputService
|
||||
progress_factory: ClassVar[Callable[..., PhaseProgressReporter]] = (
|
||||
create_comfy_phase_progress
|
||||
)
|
||||
|
||||
RETURN_TYPES = ("SEGS", "IMAGE")
|
||||
RETURN_NAMES = ("segs", "overlay")
|
||||
OUTPUT_IS_LIST = (True, False)
|
||||
OUTPUT_TOOLTIPS = (
|
||||
"Automatic image regions as SEGS for detailing, masking, or tiled diffusion.",
|
||||
"Source images with retained SAM regions shown as translucent colors.",
|
||||
)
|
||||
FUNCTION = "generate"
|
||||
CATEGORY = "SimpleSyrup/Detection"
|
||||
DESCRIPTION = "Creates automatic, unprompted image SEGS from a connected SAM model."
|
||||
SEARCH_ALIASES = ["sam", "automatic", "segment", "segmentation", "segs"]
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls) -> dict[str, dict[str, tuple[Any, ...]]]:
|
||||
"""Declare inputs for automatic SAM-to-SEGS conversion."""
|
||||
|
||||
return {
|
||||
"required": {
|
||||
"image": (
|
||||
"IMAGE",
|
||||
{"tooltip": "Image whose automatic regions become SEGS."},
|
||||
),
|
||||
"sam_model": (
|
||||
"SAM_MODEL",
|
||||
{"tooltip": "SAM model used to find unprompted image regions."},
|
||||
),
|
||||
"segmentation_resolution": (
|
||||
"INT",
|
||||
{
|
||||
"default": 640,
|
||||
"min": 64,
|
||||
"max": 8192,
|
||||
"step": 64,
|
||||
"tooltip": (
|
||||
"Maximum long edge in pixels used for segmentation. "
|
||||
"Lower values run faster and omit smaller details."
|
||||
),
|
||||
},
|
||||
),
|
||||
"minimum_region_area": (
|
||||
"INT",
|
||||
{
|
||||
"default": 0,
|
||||
"min": 0,
|
||||
"max": 268435456,
|
||||
"step": 1,
|
||||
"tooltip": (
|
||||
"Discard masks smaller than this many pixels in the "
|
||||
"original image."
|
||||
),
|
||||
},
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
def generate(
|
||||
self,
|
||||
image: object,
|
||||
sam_model: object,
|
||||
segmentation_resolution: int = 640,
|
||||
minimum_region_area: int = 0,
|
||||
) -> tuple[list[object], torch.Tensor]:
|
||||
"""Return aligned automatic SEGS and a SAM-style overlay image batch."""
|
||||
|
||||
image_batch = validate_image_batch(image, "SEGS from SAM Output")
|
||||
service = self.service_class()
|
||||
phase_progress = type(self).progress_factory(
|
||||
operation="segs_from_sam_output",
|
||||
subject=_sam_model_subject(sam_model),
|
||||
total_phases=int(image_batch.shape[0]) * 4 + 1,
|
||||
)
|
||||
outputs: list[object] = []
|
||||
overlays: list[torch.Tensor] = []
|
||||
try:
|
||||
for single_image in iter_single_images(image_batch):
|
||||
result = service.build(
|
||||
image=single_image,
|
||||
sam_model=sam_model,
|
||||
segmentation_resolution=segmentation_resolution,
|
||||
minimum_region_area=minimum_region_area,
|
||||
phase_progress=phase_progress,
|
||||
)
|
||||
outputs.append(result.segs)
|
||||
overlays.append(result.overlay)
|
||||
except Exception:
|
||||
phase_progress.advance("failed")
|
||||
raise
|
||||
phase_progress.advance("completed")
|
||||
return outputs, torch.cat(overlays, dim=0)
|
||||
|
||||
|
||||
def _sam_model_subject(sam_model: object) -> str:
|
||||
"""Return a concise model identity for Comfy progress diagnostics."""
|
||||
|
||||
model_id = getattr(sam_model, "model_id", None)
|
||||
if isinstance(model_id, str) and model_id:
|
||||
return model_id
|
||||
return type(sam_model).__name__
|
||||
@@ -10,14 +10,17 @@ import importlib
|
||||
from types import ModuleType
|
||||
from typing import Any
|
||||
|
||||
from ..runtime.anima_loader import (
|
||||
from ..domain.anima_quantization import AnimaQuantizationRecipe
|
||||
from ..runtime.diffusion_model_loader import DIFFUSION_WEIGHT_DTYPES
|
||||
from ..runtime.model_downloads import ComfyProgressReporter
|
||||
from ..runtime.quantization_capabilities import QuantizationCapabilityCatalog
|
||||
from ..runtime.quantization_progress import ComfyQuantizationProgressReporter
|
||||
from ..runtime.vae_loader import vae_choices
|
||||
from ..services.anima_loader_service import (
|
||||
AUTO_CHOICE,
|
||||
CLIP_DEVICES,
|
||||
DIFFUSION_WEIGHT_DTYPES,
|
||||
AnimaLoaderService,
|
||||
)
|
||||
from ..runtime.model_downloads import ComfyProgressReporter
|
||||
from ..runtime.vae_loader import vae_choices
|
||||
from . import tooltips
|
||||
|
||||
|
||||
@@ -25,6 +28,8 @@ class SimpleLoadAnima:
|
||||
"""Expose Anima diffusion, text encoder, and VAE loading as one node."""
|
||||
|
||||
_service = AnimaLoaderService()
|
||||
_quantization_recipe = AnimaQuantizationRecipe()
|
||||
_quantization_capabilities = QuantizationCapabilityCatalog()
|
||||
|
||||
RETURN_TYPES = ("MODEL", "CLIP", "VAE")
|
||||
RETURN_NAMES = ("model", "clip", "vae")
|
||||
@@ -53,14 +58,28 @@ class SimpleLoadAnima:
|
||||
)
|
||||
},
|
||||
),
|
||||
"quantization": (
|
||||
cls._quantization_capabilities.selection_labels(
|
||||
cls._quantization_recipe.profiles
|
||||
),
|
||||
{
|
||||
"default": "Original",
|
||||
"advanced": True,
|
||||
"tooltip": (
|
||||
"Creates or reuses a GPU-supported quantized copy in the "
|
||||
"global models/SyrupQuants cache; Original loads the "
|
||||
"selected model unchanged."
|
||||
),
|
||||
},
|
||||
),
|
||||
"diffusion_weight_dtype": (
|
||||
list(DIFFUSION_WEIGHT_DTYPES),
|
||||
{
|
||||
"default": "default",
|
||||
"advanced": True,
|
||||
"tooltip": (
|
||||
"Weight precision for Anima. Lower precision can reduce "
|
||||
"memory use but may slightly change results."
|
||||
"Load-time weight precision used with Original; cached "
|
||||
"quantized copies use their stored quantization format."
|
||||
),
|
||||
},
|
||||
),
|
||||
@@ -68,6 +87,7 @@ class SimpleLoadAnima:
|
||||
_choices_with_auto(folder_paths.get_filename_list("text_encoders")),
|
||||
{
|
||||
"default": AUTO_CHOICE,
|
||||
"advanced": True,
|
||||
"tooltip": (
|
||||
"Qwen text encoder used for Anima prompt understanding. "
|
||||
"Auto selects the expected model."
|
||||
@@ -89,6 +109,7 @@ class SimpleLoadAnima:
|
||||
_choices_with_auto(vae_choices(folder_paths)),
|
||||
{
|
||||
"default": AUTO_CHOICE,
|
||||
"advanced": True,
|
||||
"tooltip": (
|
||||
"VAE used to decode Anima latents. Auto selects the "
|
||||
"expected Qwen image VAE."
|
||||
@@ -101,6 +122,7 @@ class SimpleLoadAnima:
|
||||
def load_models(
|
||||
self,
|
||||
diffusion_model: str,
|
||||
quantization: str,
|
||||
diffusion_weight_dtype: str,
|
||||
text_encoder: str,
|
||||
text_encoder_device: str,
|
||||
@@ -110,11 +132,13 @@ class SimpleLoadAnima:
|
||||
|
||||
return self._service.load_models(
|
||||
diffusion_model=diffusion_model,
|
||||
quantization=quantization,
|
||||
diffusion_weight_dtype=diffusion_weight_dtype,
|
||||
text_encoder=text_encoder,
|
||||
text_encoder_device=text_encoder_device,
|
||||
vae=vae,
|
||||
progress=ComfyProgressReporter(),
|
||||
quantization_progress=ComfyQuantizationProgressReporter(),
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""ComfyUI node declaration for the interactive Simple SEGS inspector."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, ClassVar
|
||||
|
||||
from ..domain.segs import coerce_segs
|
||||
from ..runtime.seg_preview_assets import (
|
||||
SEG_PREVIEW_UI_KEY,
|
||||
ComfySegPreviewAssetPublisher,
|
||||
)
|
||||
from ..services.simple_preview_segs_service import SimplePreviewSEGSService
|
||||
|
||||
|
||||
class SimplePreviewSEGS:
|
||||
"""Expose an interactive overlay and grid inspector for one IMAGE and SEGS."""
|
||||
|
||||
service_class: ClassVar[type[SimplePreviewSEGSService]] = SimplePreviewSEGSService
|
||||
publisher_class: ClassVar[type[ComfySegPreviewAssetPublisher]] = (
|
||||
ComfySegPreviewAssetPublisher
|
||||
)
|
||||
|
||||
RETURN_TYPES = ("SEGS",)
|
||||
RETURN_NAMES = ("segs",)
|
||||
OUTPUT_TOOLTIPS = ("The original SEGS passed through without modification.",)
|
||||
FUNCTION = "preview"
|
||||
CATEGORY = "SimpleSyrup/Preview"
|
||||
OUTPUT_NODE = True
|
||||
DESCRIPTION = (
|
||||
"Interactively inspects SEGS over the original image or in a selectable grid."
|
||||
)
|
||||
SEARCH_ALIASES = ["segs", "regions", "inspect", "overlay", "preview"]
|
||||
|
||||
@classmethod
|
||||
def INPUT_TYPES(cls) -> dict[str, dict[str, tuple[Any, ...]]]:
|
||||
"""Declare the source image and matching SEGS inputs."""
|
||||
|
||||
return {
|
||||
"required": {
|
||||
"image": (
|
||||
"IMAGE",
|
||||
{"tooltip": "Original image described by the connected SEGS."},
|
||||
),
|
||||
"segs": (
|
||||
"SEGS",
|
||||
{"tooltip": "Regions to inspect over the original image."},
|
||||
),
|
||||
}
|
||||
}
|
||||
|
||||
def preview(self, image: object, segs: object) -> dict[str, object]:
|
||||
"""Publish an interactive preview while returning the input SEGS unchanged."""
|
||||
|
||||
native_segs = coerce_segs(segs)
|
||||
document = self.service_class().build(image=image, segs=native_segs)
|
||||
publication = self.publisher_class().publish(document)
|
||||
return {
|
||||
"ui": {
|
||||
"images": list(publication.images),
|
||||
SEG_PREVIEW_UI_KEY: [publication.manifest],
|
||||
},
|
||||
"result": (segs,),
|
||||
}
|
||||
@@ -115,6 +115,20 @@ SAMPLING_SEED = (
|
||||
"Seed used to create sampling noise. Reusing it with matching settings makes "
|
||||
"results repeatable."
|
||||
)
|
||||
SEED_VARIATION_MODEL_INPUT = (
|
||||
"Model whose sampler-provided initial noise will receive seed variation."
|
||||
)
|
||||
VARIATION_SEED = (
|
||||
"Seed for the second noise pattern. Change it to explore another variation "
|
||||
"direction from the sampler's seed."
|
||||
)
|
||||
VARIATION_STRENGTH = (
|
||||
"Noise interpolation strength. 0 keeps the sampler seed unchanged; 1 uses the "
|
||||
"variation seed for initial noise."
|
||||
)
|
||||
SEED_VARIATION_MODEL_OUTPUT = (
|
||||
"Model that interpolates initial sampling noise toward the variation seed."
|
||||
)
|
||||
SAMPLING_STEPS = (
|
||||
"Number of denoising steps. More steps can add refinement but take longer."
|
||||
)
|
||||
@@ -137,6 +151,14 @@ DENOISE_STRENGTH = (
|
||||
"larger changes."
|
||||
)
|
||||
DENOISED_LATENT_OUTPUT = "Denoised latent for VAE decode or more latent processing."
|
||||
CONTEXTUAL_DIFFUSION_CONTEXTS_OUTPUT = (
|
||||
"Rectangular non-global contexts actually evaluated during sampling. Their "
|
||||
"SEGS masks are created only when this output is connected."
|
||||
)
|
||||
TILED_DIFFUSION_MODE = (
|
||||
"Tile overlap blend. MultiDiffusion averages predictions; Mixture of Diffusers "
|
||||
"gives tile centers more influence."
|
||||
)
|
||||
|
||||
LATENT_TILE_WIDTH = (
|
||||
"Width of each latent tile. Larger tiles see more context but use more memory."
|
||||
@@ -152,6 +174,45 @@ LATENT_TILE_BATCH_SIZE = (
|
||||
"Number of latent tiles sampled together. Higher values can be faster but use "
|
||||
"more memory."
|
||||
)
|
||||
LATENT_CONTEXT_SIZE = (
|
||||
"Maximum side of each model context in latent pixels. Larger contexts preserve "
|
||||
"more relationships but use more memory."
|
||||
)
|
||||
LATENT_CONTEXT_OVERLAP = (
|
||||
"Overlap between local latent contexts in latent pixels. Larger overlaps reduce "
|
||||
"seams but increase sampling work."
|
||||
)
|
||||
LATENT_CONTEXT_BATCH_SIZE = (
|
||||
"Number of equal-sized latent contexts sampled together. Higher values can be "
|
||||
"faster but use more memory."
|
||||
)
|
||||
GLOBAL_CONTEXT_WEIGHT = (
|
||||
"Strength of whole-image low-frequency guidance. 1 makes the global context "
|
||||
"authoritative; lower values allow more tile interpretation."
|
||||
)
|
||||
GLOBAL_CONTEXT_STEPS = (
|
||||
"Number of initial denoising steps that use the global context. Fewer steps leave "
|
||||
"more late sampling for local detail."
|
||||
)
|
||||
GLOBAL_CONTEXT_DECAY = (
|
||||
"Multiplier applied to whole-image strength after each global step. Lower "
|
||||
"values hand control to local contexts faster."
|
||||
)
|
||||
CONTEXTUAL_DIFFUSION_SEGS = (
|
||||
"Optional regions that replace the regular grid with SEGS-guided contexts."
|
||||
)
|
||||
OPTIONAL_REGIONAL_MASKS = (
|
||||
"Optional authored masks that activate regional prompting when positive or "
|
||||
"negative conditioning is a batch; SEGS may further subdivide those regions."
|
||||
)
|
||||
OPTIONAL_REGIONAL_PROMPT_WEIGHT = (
|
||||
"Balances regional prompts against the global prompt when regional masks and "
|
||||
"a conditioning batch are connected."
|
||||
)
|
||||
OPTIONAL_REGION_MASK_FEATHER = (
|
||||
"Softens regional conditioning edges by this many source-mask pixels; tile "
|
||||
"boundaries continue to follow the unfeathered authored regions."
|
||||
)
|
||||
|
||||
DETAIL_IMAGE = (
|
||||
"Source image containing the regions to improve. Detailed crops are blended "
|
||||
|
||||
@@ -12,9 +12,26 @@ from ..runtime.prompt_control_availability import prompt_control_is_available
|
||||
def get_nodes() -> list[type[object]]:
|
||||
"""Return v3 nodes that can be advertised in this environment."""
|
||||
|
||||
from .all_prompt_attention_segs import AllPromptAttentionSEGSV3
|
||||
from .attention_capture_model import AttentionCaptureModelV3
|
||||
from .attention_masked_conditioning import AttentionMaskedConditioningV3
|
||||
from .attention_region_mask import AttentionRegionMaskV3
|
||||
from .batch_region_conditioning import BatchRegionConditioningV3
|
||||
from .batch_segs import BatchSEGSV3
|
||||
from .compose_regional_conditioning import ComposeRegionalConditioningV3
|
||||
from .concept_attention_segs import ConceptAttentionSEGSV3
|
||||
from .external_llm_prompt import ExternalLLMPromptV3
|
||||
from .ksampler_attention_coupling import KSamplerAttentionCouplingV3
|
||||
from .ksampler_contextual_attention_coupling import (
|
||||
KSamplerContextualAttentionCouplingV3,
|
||||
)
|
||||
from .ksampler_contextual_diffusion import KSamplerContextualDiffusionV3
|
||||
from .ksampler_prompt_by_region import KSamplerPromptByRegionV3
|
||||
from .ksampler_prompt_by_tiled_region import KSamplerPromptByTiledRegionV3
|
||||
from .ksampler_tiled_attention_coupling import (
|
||||
KSamplerTiledAttentionCouplingV3,
|
||||
)
|
||||
from .ksampler_tiled_diffusion import KSamplerTiledDiffusionV3
|
||||
from .legacy_node_wrappers import (
|
||||
ConditioningBatchAppendV3,
|
||||
ConditioningBatchStartV3,
|
||||
@@ -26,7 +43,6 @@ def get_nodes() -> list[type[object]]:
|
||||
GroundedSAMModelInfoV3,
|
||||
GroundingDINOModelLoaderV3,
|
||||
KSamplerExtrasV3,
|
||||
KSamplerTiledDiffusionV3,
|
||||
LatentDiagnosticsV3,
|
||||
LayerStyleSAMModelsAdapterV3,
|
||||
LoadUltralyticsModelV3,
|
||||
@@ -36,14 +52,21 @@ def get_nodes() -> list[type[object]]:
|
||||
ResizeImageToTargetV3,
|
||||
SAMModelLoaderV3,
|
||||
SeedV3,
|
||||
SEGSFromSAMOutputV3,
|
||||
SimpleLoadAnimaV3,
|
||||
SimplePreviewSEGSV3,
|
||||
SimpleVAEEncodeV3,
|
||||
UpscaleLatentFromImageV3,
|
||||
ViTMatteModelLoaderV3,
|
||||
)
|
||||
from .load_image_list import LoadImageListV3
|
||||
from .load_mask_batch import LoadMaskBatchV3
|
||||
from .mask_to_segs import MaskToSEGSV3
|
||||
from .scale_factor import ScaleFactorV3
|
||||
from .seed_variation import SeedVariationV3
|
||||
from .simple_load_checkpoint import SimpleLoadCheckpointV3
|
||||
from .simple_load_flux import SimpleLoadFluxV3
|
||||
from .simple_load_flux2 import SimpleLoadFlux2V3
|
||||
from .tag_segs_with_external_llm import TagSEGSWithExternalLLMV3
|
||||
from .tag_segs_with_wd14 import TagSEGSWithWD14V3
|
||||
from .tile_and_tag_segs import TileAndTagSEGSV3
|
||||
@@ -52,10 +75,15 @@ def get_nodes() -> list[type[object]]:
|
||||
from .wd14_tagger_loader import WD14TaggerLoaderV3
|
||||
|
||||
nodes: list[type[object]] = [
|
||||
AllPromptAttentionSEGSV3,
|
||||
AttentionCaptureModelV3,
|
||||
AttentionMaskedConditioningV3,
|
||||
AttentionRegionMaskV3,
|
||||
BatchRegionConditioningV3,
|
||||
BatchSEGSV3,
|
||||
ConditioningBatchAppendV3,
|
||||
ConditioningBatchStartV3,
|
||||
ComposeRegionalConditioningV3,
|
||||
DetailSEGSAsRegionsV3,
|
||||
DetailSEGSByScaleFactorTiledDiffusionV3,
|
||||
DetailSEGSByScaleFactorV3,
|
||||
@@ -65,20 +93,34 @@ def get_nodes() -> list[type[object]]:
|
||||
GroundedSAMModelInfoV3,
|
||||
GroundingDINOModelLoaderV3,
|
||||
KSamplerExtrasV3,
|
||||
KSamplerAttentionCouplingV3,
|
||||
KSamplerTiledAttentionCouplingV3,
|
||||
KSamplerContextualAttentionCouplingV3,
|
||||
KSamplerPromptByRegionV3,
|
||||
KSamplerPromptByTiledRegionV3,
|
||||
KSamplerContextualDiffusionV3,
|
||||
KSamplerTiledDiffusionV3,
|
||||
LatentDiagnosticsV3,
|
||||
LayerStyleSAMModelsAdapterV3,
|
||||
LoadUltralyticsModelV3,
|
||||
LoadImageListV3,
|
||||
LoadMaskBatchV3,
|
||||
MaskToSEGSV3,
|
||||
PromptEncodeStyleAndNormalizationV3,
|
||||
PromptEncodeStyleV3,
|
||||
PromptSEGSWithSAMV3,
|
||||
ResizeImageToTargetV3,
|
||||
SAMModelLoaderV3,
|
||||
SEGSFromSAMOutputV3,
|
||||
ScaleFactorV3,
|
||||
ConceptAttentionSEGSV3,
|
||||
SeedV3,
|
||||
SeedVariationV3,
|
||||
SimpleLoadAnimaV3,
|
||||
SimplePreviewSEGSV3,
|
||||
SimpleLoadCheckpointV3,
|
||||
SimpleLoadFluxV3,
|
||||
SimpleLoadFlux2V3,
|
||||
SimpleVAEEncodeV3,
|
||||
TagSEGSWithExternalLLMV3,
|
||||
TagSEGSWithWD14V3,
|
||||
@@ -93,16 +135,26 @@ def get_nodes() -> list[type[object]]:
|
||||
if not prompt_control_is_available():
|
||||
return nodes
|
||||
|
||||
from .apply_automatic_negpip import ApplyAutomaticNegpipV3
|
||||
from .attach_regional_global_conditioning import (
|
||||
AttachRegionalGlobalConditioningV3,
|
||||
)
|
||||
from .encode_prompt_batch_with_prompt_control import (
|
||||
EncodePromptBatchWithPromptControl,
|
||||
)
|
||||
from .label_regional_lora_hooks import LabelRegionalLoraHooksV3
|
||||
from .prepare_regional_lora_hooks import PrepareRegionalLoraHooksV3
|
||||
from .schedule_and_encode_prompts_with_prompt_control import (
|
||||
ScheduleAndEncodePromptsWithPromptControl,
|
||||
)
|
||||
|
||||
return [
|
||||
*nodes,
|
||||
ApplyAutomaticNegpipV3,
|
||||
AttachRegionalGlobalConditioningV3,
|
||||
EncodePromptBatchWithPromptControl,
|
||||
LabelRegionalLoraHooksV3,
|
||||
PrepareRegionalLoraHooksV3,
|
||||
ScheduleAndEncodePromptsWithPromptControl,
|
||||
]
|
||||
|
||||
|
||||
@@ -0,0 +1,129 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Comfy v3 node for exposing every mapped prompt attention region."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..domain.attention_region_capture import AttentionEvidenceMode
|
||||
from ..services.attention_region_node_service import ATTENTION_REGION_NODE_SERVICE
|
||||
from .attention_region_inputs import (
|
||||
attention_region_control_inputs,
|
||||
attention_region_controls,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
hidden: ClassVar[Any]
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class AllPromptAttentionSEGSV3(_ComfyNodeBase):
|
||||
"""Expose separate overlapping regions for all readable positive concepts."""
|
||||
|
||||
GRAPH_PASSTHROUGH_OUTPUTS = {0: "image"}
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the all-prompt downstream attention-SEGS contract."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.AllPromptAttentionSEGS",
|
||||
display_name="All Prompt Attention SEGS",
|
||||
category="SimpleSyrup/Detection",
|
||||
description=(
|
||||
"Returns separate overlapping soft SEGS for every mapped concept "
|
||||
"in the positive prompt that produced the image."
|
||||
),
|
||||
search_aliases=[
|
||||
"all attention heatmaps",
|
||||
"prompt insight",
|
||||
"token regions",
|
||||
],
|
||||
hidden=[_comfy_io.Hidden.unique_id],
|
||||
inputs=[
|
||||
_comfy_io.Image.Input(
|
||||
"image",
|
||||
tooltip=(
|
||||
"Image whose graph provenance identifies the upstream sampler; "
|
||||
"the image is returned unchanged."
|
||||
),
|
||||
),
|
||||
*attention_region_control_inputs(
|
||||
_comfy_io,
|
||||
evidence_mode_default=AttentionEvidenceMode.RAW,
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Image.Output("image", tooltip="Unchanged connected image."),
|
||||
_comfy_io.SEGS.Output(
|
||||
"segs",
|
||||
tooltip="Separate labeled overlapping SEGS for prompt concepts.",
|
||||
is_output_list=True,
|
||||
),
|
||||
_comfy_io.Mask.Output(
|
||||
"mask",
|
||||
tooltip="Soft union of every retained prompt attention region.",
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
image: object,
|
||||
sampler_stage: int,
|
||||
capture_start: float,
|
||||
capture_end: float,
|
||||
minimum_strength: float,
|
||||
minimum_consensus: float,
|
||||
geometry_recall: float,
|
||||
split_sensitivity: float,
|
||||
instance_recall: float,
|
||||
minimum_region_size: int,
|
||||
keep_only: int,
|
||||
keep_by: str,
|
||||
combine_segs: bool,
|
||||
matte_solidity: float,
|
||||
edge_feather: int,
|
||||
capture_profile: str,
|
||||
evidence_mode: str,
|
||||
) -> tuple[object, object, object]:
|
||||
"""Consume all prompt maps and render separate overlapping SEGS."""
|
||||
|
||||
del sampler_stage
|
||||
result = ATTENTION_REGION_NODE_SERVICE.for_image(
|
||||
request_node_id=str(cls.hidden.unique_id),
|
||||
image=image,
|
||||
controls=attention_region_controls(
|
||||
capture_start=capture_start,
|
||||
capture_end=capture_end,
|
||||
minimum_strength=minimum_strength,
|
||||
minimum_consensus=minimum_consensus,
|
||||
geometry_recall=geometry_recall,
|
||||
split_sensitivity=split_sensitivity,
|
||||
instance_recall=instance_recall,
|
||||
minimum_region_size=minimum_region_size,
|
||||
keep_only=keep_only,
|
||||
keep_by=keep_by,
|
||||
combine_segs=combine_segs,
|
||||
matte_solidity=matte_solidity,
|
||||
edge_feather=edge_feather,
|
||||
capture_profile=capture_profile,
|
||||
evidence_mode=evidence_mode,
|
||||
),
|
||||
)
|
||||
return result.image, result.segs, result.mask
|
||||
@@ -0,0 +1,69 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Internal Comfy v3 node for model-family automatic NegPiP preparation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from ..services.negpip_model_service import NEGPIP_MODEL_SERVICE
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
pass
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class ApplyAutomaticNegpipV3(_ComfyNodeBase):
|
||||
"""Patch supported MODEL/CLIP pairs after a negative prompt-weight trigger."""
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the internal runtime patch boundary."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.ApplyAutomaticNegpip",
|
||||
display_name="Apply Automatic NegPiP (Internal)",
|
||||
category="SimpleSyrup/Internal",
|
||||
description=(
|
||||
"Internal model-family NegPiP preparation injected by Schedule & "
|
||||
"Encode Prompts after detecting a negative prompt weight."
|
||||
),
|
||||
is_dev_only=True,
|
||||
inputs=[
|
||||
_comfy_io.Model.Input(
|
||||
"model",
|
||||
tooltip="MODEL inspected and cloned only when NegPiP is supported.",
|
||||
),
|
||||
_comfy_io.Clip.Input(
|
||||
"clip",
|
||||
tooltip="CLIP cloned with the matching NegPiP encoder behavior.",
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Model.Output(
|
||||
"model",
|
||||
tooltip="MODEL carrying one supported NegPiP attention patch set.",
|
||||
),
|
||||
_comfy_io.Clip.Output(
|
||||
"clip",
|
||||
tooltip="CLIP carrying matching negative-weight encoding behavior.",
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(cls, model: object, clip: object) -> tuple[object, object]:
|
||||
"""Return the supported patched pair or the original unsupported pair."""
|
||||
|
||||
return NEGPIP_MODEL_SERVICE.prepare(model, clip)
|
||||
@@ -0,0 +1,71 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Comfy v3 internal node for regional global-prompt companions."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..runtime.regional_conditioning_companion import attach_global_companion
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class AttachRegionalGlobalConditioningV3(_ComfyNodeBase):
|
||||
"""Attach the hooked global share required by one regional LoRA."""
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the internal companion-conditioning schema."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.AttachRegionalGlobalConditioning",
|
||||
display_name="Attach Regional Global Conditioning",
|
||||
category="SimpleSyrup/Internal",
|
||||
description=(
|
||||
"Keeps a region's global and local prompt shares under the same "
|
||||
"regional LoRA."
|
||||
),
|
||||
inputs=[
|
||||
_comfy_io.Conditioning.Input(
|
||||
"conditioning",
|
||||
tooltip="Regional prompt conditioning to preserve.",
|
||||
),
|
||||
_comfy_io.Conditioning.Input(
|
||||
"global_conditioning",
|
||||
tooltip=(
|
||||
"Global prompt encoded with the same regional LoRA hooks."
|
||||
),
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Conditioning.Output(
|
||||
"conditioning",
|
||||
tooltip="Regional conditioning carrying its hooked global share.",
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
conditioning: object,
|
||||
global_conditioning: object,
|
||||
) -> tuple[object]:
|
||||
"""Attach the global companion to one regional conditioning."""
|
||||
|
||||
return (attach_global_companion(conditioning, global_conditioning),)
|
||||
@@ -0,0 +1,96 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Internal Comfy v3 MODEL derivation node injected by prompt provenance."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..services.attention_capture_model_service import ATTENTION_CAPTURE_MODEL_SERVICE
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class AttentionCaptureModelV3(_ComfyNodeBase):
|
||||
"""Derive an observation-only MODEL from a prompt-injected capture plan."""
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the dev-only internal capture node contract."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.AttentionCaptureModel",
|
||||
display_name="Attention Capture Model (Internal)",
|
||||
category="SimpleSyrup/Internal",
|
||||
description=(
|
||||
"Internal prompt-scoped MODEL observer used by downstream "
|
||||
"attention-region nodes."
|
||||
),
|
||||
is_dev_only=True,
|
||||
inputs=[
|
||||
_comfy_io.Model.Input(
|
||||
"model",
|
||||
tooltip="Upstream MODEL cloned for observation-only capture.",
|
||||
),
|
||||
_comfy_io.Conditioning.Input(
|
||||
"positive",
|
||||
tooltip="Positive conditioning whose prompt tokens are mapped.",
|
||||
),
|
||||
_comfy_io.String.Input(
|
||||
"plan_json",
|
||||
multiline=False,
|
||||
tooltip="Prompt-injected capture plan for the target sampler.",
|
||||
),
|
||||
_comfy_io.Clip.Input(
|
||||
"clip",
|
||||
optional=True,
|
||||
tooltip="Graph-visible CLIP used for exact prompt token mapping.",
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Model.Output(
|
||||
"model",
|
||||
tooltip="MODEL carrying one observation-only attention observer.",
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
model: object,
|
||||
positive: object,
|
||||
plan_json: str,
|
||||
clip: object | None = None,
|
||||
) -> tuple[object]:
|
||||
"""Prepare and publish capture state before the target sampler runs."""
|
||||
|
||||
del positive
|
||||
return (
|
||||
ATTENTION_CAPTURE_MODEL_SERVICE.prepare(
|
||||
model=model,
|
||||
plan_json=plan_json,
|
||||
clip=clip,
|
||||
),
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def fingerprint_inputs(cls, **kwargs: object) -> float:
|
||||
"""Force prompt-scoped session publication on every graph execution."""
|
||||
|
||||
del kwargs
|
||||
return float("nan")
|
||||
@@ -0,0 +1,157 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Comfy v3 node for attention-derived regional conditioning of a later sampler."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..domain.attention_concepts import parse_attention_concepts
|
||||
from ..services.attention_region_node_service import ATTENTION_REGION_NODE_SERVICE
|
||||
from .attention_region_inputs import (
|
||||
attention_region_control_inputs,
|
||||
attention_region_controls,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
hidden: ClassVar[Any]
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class AttentionMaskedConditioningV3(_ComfyNodeBase):
|
||||
"""Mask supplied conditioning with attention captured from an earlier sampler."""
|
||||
|
||||
GRAPH_PASSTHROUGH_OUTPUTS = {0: "latent"}
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare later-pass attention-masked conditioning inputs and outputs."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.AttentionMaskedConditioning",
|
||||
display_name="Attention Masked Conditioning",
|
||||
category="SimpleSyrup/Conditioning",
|
||||
description=(
|
||||
"Masks supplied conditioning to regions discovered from an earlier "
|
||||
"sampler so it can control a later sampling pass."
|
||||
),
|
||||
search_aliases=[
|
||||
"regional conditioning",
|
||||
"attention conditioning",
|
||||
"masked prompt",
|
||||
],
|
||||
hidden=[_comfy_io.Hidden.unique_id],
|
||||
inputs=[
|
||||
_comfy_io.Latent.Input(
|
||||
"latent",
|
||||
tooltip=(
|
||||
"Latent produced by the sampler used for localization; it is "
|
||||
"returned unchanged for the later sampler."
|
||||
),
|
||||
),
|
||||
_comfy_io.Conditioning.Input(
|
||||
"conditioning",
|
||||
tooltip="Conditioning to apply only inside the discovered regions.",
|
||||
),
|
||||
_comfy_io.String.Input(
|
||||
"concepts",
|
||||
multiline=True,
|
||||
default="subject",
|
||||
tooltip=("Concepts used to discover regions, separated by |."),
|
||||
),
|
||||
_comfy_io.Float.Input(
|
||||
"conditioning_strength",
|
||||
default=1.0,
|
||||
min=0.0,
|
||||
max=10.0,
|
||||
step=0.05,
|
||||
tooltip="Strength of the supplied conditioning inside the mask.",
|
||||
),
|
||||
*attention_region_control_inputs(_comfy_io),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Latent.Output(
|
||||
"latent", tooltip="Unchanged localization latent."
|
||||
),
|
||||
_comfy_io.Conditioning.Output(
|
||||
"conditioning",
|
||||
tooltip=(
|
||||
"Supplied conditioning carrying the attention-derived mask."
|
||||
),
|
||||
),
|
||||
_comfy_io.Mask.Output(
|
||||
"mask",
|
||||
tooltip="Soft latent-resolution mask attached to the conditioning.",
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
latent: object,
|
||||
conditioning: object,
|
||||
concepts: str,
|
||||
conditioning_strength: float,
|
||||
sampler_stage: int,
|
||||
capture_start: float,
|
||||
capture_end: float,
|
||||
minimum_strength: float,
|
||||
minimum_consensus: float,
|
||||
geometry_recall: float,
|
||||
split_sensitivity: float,
|
||||
instance_recall: float,
|
||||
minimum_region_size: int,
|
||||
keep_only: int,
|
||||
keep_by: str,
|
||||
combine_segs: bool,
|
||||
matte_solidity: float,
|
||||
edge_feather: int,
|
||||
capture_profile: str,
|
||||
evidence_mode: str,
|
||||
) -> tuple[object, object, object]:
|
||||
"""Return later-pass conditioning masked by earlier-sampler attention."""
|
||||
|
||||
del sampler_stage
|
||||
if not parse_attention_concepts(concepts):
|
||||
raise ValueError("Attention Masked Conditioning requires a concept.")
|
||||
result = ATTENTION_REGION_NODE_SERVICE.for_latent(
|
||||
request_node_id=str(cls.hidden.unique_id),
|
||||
latent=latent,
|
||||
controls=attention_region_controls(
|
||||
capture_start=capture_start,
|
||||
capture_end=capture_end,
|
||||
minimum_strength=minimum_strength,
|
||||
minimum_consensus=minimum_consensus,
|
||||
geometry_recall=geometry_recall,
|
||||
split_sensitivity=split_sensitivity,
|
||||
instance_recall=instance_recall,
|
||||
minimum_region_size=minimum_region_size,
|
||||
keep_only=keep_only,
|
||||
keep_by=keep_by,
|
||||
combine_segs=combine_segs,
|
||||
matte_solidity=matte_solidity,
|
||||
edge_feather=edge_feather,
|
||||
capture_profile=capture_profile,
|
||||
evidence_mode=evidence_mode,
|
||||
),
|
||||
)
|
||||
masked = ATTENTION_REGION_NODE_SERVICE.mask_conditioning(
|
||||
conditioning,
|
||||
result.mask,
|
||||
conditioning_strength,
|
||||
)
|
||||
return result.latent, masked, result.mask
|
||||
@@ -0,0 +1,219 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Build shared Comfy v3 inputs and domain controls for attention-region nodes."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from ..domain.attention_region_capture import (
|
||||
AttentionCaptureProfile,
|
||||
AttentionEvidenceMode,
|
||||
AttentionRegionControls,
|
||||
)
|
||||
|
||||
|
||||
def attention_region_control_inputs(
|
||||
io: Any,
|
||||
*,
|
||||
evidence_mode_default: AttentionEvidenceMode = AttentionEvidenceMode.CONCEPT,
|
||||
) -> list[object]:
|
||||
"""Return the complete shared attention-native control schema."""
|
||||
|
||||
return [
|
||||
io.Int.Input(
|
||||
"sampler_stage",
|
||||
default=1,
|
||||
min=-1,
|
||||
max=1024,
|
||||
step=1,
|
||||
tooltip=(
|
||||
"Selects the connected sampling stage: 1 is first, 2 is second, "
|
||||
"and 0 or -1 selects the last; oversized values use the last."
|
||||
),
|
||||
),
|
||||
io.Float.Input(
|
||||
"capture_start",
|
||||
default=0.0,
|
||||
min=0.0,
|
||||
max=0.99,
|
||||
step=0.01,
|
||||
tooltip=(
|
||||
"Start of denoising evidence to include; later values ignore more "
|
||||
"of the initial composition phase."
|
||||
),
|
||||
),
|
||||
io.Float.Input(
|
||||
"capture_end",
|
||||
default=1.0,
|
||||
min=0.01,
|
||||
max=1.0,
|
||||
step=0.01,
|
||||
tooltip=(
|
||||
"End of denoising evidence to include; earlier values ignore more "
|
||||
"late refinement attention."
|
||||
),
|
||||
),
|
||||
io.Float.Input(
|
||||
"minimum_strength",
|
||||
default=0.15,
|
||||
min=0.0,
|
||||
max=1.0,
|
||||
step=0.01,
|
||||
tooltip=(
|
||||
"Minimum normalized attention association retained in a region; "
|
||||
"higher values narrow the silhouette toward its semantic core."
|
||||
),
|
||||
),
|
||||
io.Float.Input(
|
||||
"minimum_consensus",
|
||||
default=0.25,
|
||||
min=0.0,
|
||||
max=1.0,
|
||||
step=0.01,
|
||||
tooltip=(
|
||||
"Fraction of selected observations that must support a pixel; "
|
||||
"higher values keep more persistent regions."
|
||||
),
|
||||
),
|
||||
io.Float.Input(
|
||||
"geometry_recall",
|
||||
default=0.85,
|
||||
min=0.0,
|
||||
max=1.0,
|
||||
step=0.01,
|
||||
tooltip=(
|
||||
"Higher values recover fainter connected object extent from exact "
|
||||
"attention, preserving tails and shafts but admitting more halos."
|
||||
),
|
||||
),
|
||||
io.Float.Input(
|
||||
"split_sensitivity",
|
||||
default=0.0,
|
||||
min=0.0,
|
||||
max=1.0,
|
||||
step=0.01,
|
||||
tooltip=(
|
||||
"Sensitivity to divide one connected region around separate peaks; "
|
||||
"higher values can split a soft silhouette into multiple instances."
|
||||
),
|
||||
),
|
||||
io.Float.Input(
|
||||
"instance_recall",
|
||||
default=0.65,
|
||||
min=0.0,
|
||||
max=1.0,
|
||||
step=0.01,
|
||||
tooltip=(
|
||||
"Higher values retain weaker disconnected instances relative to "
|
||||
"the strongest one, which helps repeated sparse concepts."
|
||||
),
|
||||
),
|
||||
io.Int.Input(
|
||||
"minimum_region_size",
|
||||
default=512,
|
||||
min=1,
|
||||
max=1048576,
|
||||
step=1,
|
||||
tooltip="Discard attention components smaller than this many pixels.",
|
||||
),
|
||||
io.Int.Input(
|
||||
"keep_only",
|
||||
default=1,
|
||||
min=0,
|
||||
max=1024,
|
||||
step=1,
|
||||
tooltip=(
|
||||
"Keep the best N instances per concept; 1 keeps the largest and "
|
||||
"0 keeps all."
|
||||
),
|
||||
),
|
||||
io.Combo.Input(
|
||||
"keep_by",
|
||||
options=["largest size", "highest confidence"],
|
||||
default="largest size",
|
||||
tooltip="Ranks retained instances by area or attention confidence.",
|
||||
),
|
||||
io.Boolean.Input(
|
||||
"combine_segs",
|
||||
default=False,
|
||||
tooltip="Combines retained instances of each concept into one SEG.",
|
||||
),
|
||||
io.Float.Input(
|
||||
"matte_solidity",
|
||||
default=0.75,
|
||||
min=0.0,
|
||||
max=1.0,
|
||||
step=0.01,
|
||||
tooltip=(
|
||||
"Higher values flatten accepted interiors toward fully opaque alpha."
|
||||
),
|
||||
),
|
||||
io.Int.Input(
|
||||
"edge_feather",
|
||||
default=8,
|
||||
min=0,
|
||||
max=4096,
|
||||
step=1,
|
||||
tooltip="Width in output pixels of the matte boundary transition.",
|
||||
),
|
||||
io.Combo.Input(
|
||||
"capture_profile",
|
||||
options=[profile.value for profile in AttentionCaptureProfile],
|
||||
default=AttentionCaptureProfile.FAST.value,
|
||||
tooltip=(
|
||||
"Controls observation density: fast minimizes overhead, balanced "
|
||||
"adds temporal evidence, and exhaustive retains every eligible call."
|
||||
),
|
||||
),
|
||||
io.Combo.Input(
|
||||
"evidence_mode",
|
||||
options=[mode.value for mode in AttentionEvidenceMode],
|
||||
default=evidence_mode_default.value,
|
||||
tooltip=(
|
||||
"Concept isolation derives a cleaner stable region; raw attention "
|
||||
"shows the captured model probabilities without isolation weighting."
|
||||
),
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def attention_region_controls(
|
||||
*,
|
||||
capture_start: float,
|
||||
capture_end: float,
|
||||
minimum_strength: float,
|
||||
minimum_consensus: float,
|
||||
geometry_recall: float,
|
||||
split_sensitivity: float,
|
||||
instance_recall: float,
|
||||
minimum_region_size: int,
|
||||
keep_only: int,
|
||||
keep_by: str,
|
||||
combine_segs: bool,
|
||||
matte_solidity: float,
|
||||
edge_feather: int,
|
||||
capture_profile: str,
|
||||
evidence_mode: str,
|
||||
) -> AttentionRegionControls:
|
||||
"""Build validated domain controls from public node inputs."""
|
||||
|
||||
return AttentionRegionControls(
|
||||
capture_start=capture_start,
|
||||
capture_end=capture_end,
|
||||
minimum_strength=minimum_strength,
|
||||
minimum_consensus=minimum_consensus,
|
||||
split_sensitivity=split_sensitivity,
|
||||
minimum_region_size=minimum_region_size,
|
||||
profile=AttentionCaptureProfile(capture_profile),
|
||||
instance_recall=instance_recall,
|
||||
geometry_recall=geometry_recall,
|
||||
keep_only=keep_only,
|
||||
keep_by=keep_by,
|
||||
combine_segs=combine_segs,
|
||||
matte_solidity=matte_solidity,
|
||||
edge_feather=edge_feather,
|
||||
evidence_mode=AttentionEvidenceMode(evidence_mode),
|
||||
)
|
||||
@@ -0,0 +1,130 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Comfy v3 node for querying upstream attention as a latent-resolution mask."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..domain.attention_concepts import parse_attention_concepts
|
||||
from ..services.attention_region_node_service import ATTENTION_REGION_NODE_SERVICE
|
||||
from .attention_region_inputs import (
|
||||
attention_region_control_inputs,
|
||||
attention_region_controls,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
hidden: ClassVar[Any]
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class AttentionRegionMaskV3(_ComfyNodeBase):
|
||||
"""Return a reusable mask derived from an upstream sampler's attention."""
|
||||
|
||||
GRAPH_PASSTHROUGH_OUTPUTS = {0: "latent"}
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the latent attention-region mask contract."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.AttentionRegionMask",
|
||||
display_name="Attention Region Mask",
|
||||
category="SimpleSyrup/Masking",
|
||||
description=(
|
||||
"Returns a reusable latent-resolution mask for concepts attended "
|
||||
"by the sampler that produced the connected latent."
|
||||
),
|
||||
search_aliases=["latent attention mask", "prompt mask", "region mask"],
|
||||
hidden=[_comfy_io.Hidden.unique_id],
|
||||
inputs=[
|
||||
_comfy_io.Latent.Input(
|
||||
"latent",
|
||||
tooltip=(
|
||||
"Latent whose graph provenance identifies the upstream "
|
||||
"sampler; the latent is returned unchanged."
|
||||
),
|
||||
),
|
||||
_comfy_io.String.Input(
|
||||
"concepts",
|
||||
multiline=True,
|
||||
default="subject",
|
||||
tooltip=(
|
||||
"Concepts whose attention becomes the mask, separated by |."
|
||||
),
|
||||
),
|
||||
*attention_region_control_inputs(_comfy_io),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Latent.Output(
|
||||
"latent", tooltip="Unchanged connected latent."
|
||||
),
|
||||
_comfy_io.Mask.Output(
|
||||
"mask",
|
||||
tooltip="Soft latent-resolution union of matched regions.",
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
latent: object,
|
||||
concepts: str,
|
||||
sampler_stage: int,
|
||||
capture_start: float,
|
||||
capture_end: float,
|
||||
minimum_strength: float,
|
||||
minimum_consensus: float,
|
||||
geometry_recall: float,
|
||||
split_sensitivity: float,
|
||||
instance_recall: float,
|
||||
minimum_region_size: int,
|
||||
keep_only: int,
|
||||
keep_by: str,
|
||||
combine_segs: bool,
|
||||
matte_solidity: float,
|
||||
edge_feather: int,
|
||||
capture_profile: str,
|
||||
evidence_mode: str,
|
||||
) -> tuple[object, object]:
|
||||
"""Consume matching maps and return a reusable latent-resolution mask."""
|
||||
|
||||
del sampler_stage
|
||||
if not parse_attention_concepts(concepts):
|
||||
raise ValueError("Attention Region Mask requires at least one concept.")
|
||||
result = ATTENTION_REGION_NODE_SERVICE.for_latent(
|
||||
request_node_id=str(cls.hidden.unique_id),
|
||||
latent=latent,
|
||||
controls=attention_region_controls(
|
||||
capture_start=capture_start,
|
||||
capture_end=capture_end,
|
||||
minimum_strength=minimum_strength,
|
||||
minimum_consensus=minimum_consensus,
|
||||
geometry_recall=geometry_recall,
|
||||
split_sensitivity=split_sensitivity,
|
||||
instance_recall=instance_recall,
|
||||
minimum_region_size=minimum_region_size,
|
||||
keep_only=keep_only,
|
||||
keep_by=keep_by,
|
||||
combine_segs=combine_segs,
|
||||
matte_solidity=matte_solidity,
|
||||
edge_feather=edge_feather,
|
||||
capture_profile=capture_profile,
|
||||
evidence_mode=evidence_mode,
|
||||
),
|
||||
)
|
||||
return result.latent, result.mask
|
||||
@@ -0,0 +1,83 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Comfy v3 node for composing standard masked regional conditioning."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..services.regional_conditioning_service import RegionalConditioningService
|
||||
from .ksampler_schema import regional_conditioning_inputs
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class ComposeRegionalConditioningV3(_ComfyNodeBase):
|
||||
"""Compose global-first batches and ordered masks for native samplers."""
|
||||
|
||||
conditioning_service_class: ClassVar[type[RegionalConditioningService]] = (
|
||||
RegionalConditioningService
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the standard regional-conditioning composition schema."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.ComposeRegionalConditioning",
|
||||
display_name="Compose Regional Conditioning",
|
||||
category="SimpleSyrup/Conditioning",
|
||||
description=(
|
||||
"Pairs global-first prompt batches with ordered masks and returns "
|
||||
"standard masked conditioning for native Comfy samplers."
|
||||
),
|
||||
search_aliases=["regional prompt", "masked conditioning"],
|
||||
inputs=regional_conditioning_inputs(_comfy_io),
|
||||
outputs=[
|
||||
_comfy_io.Conditioning.Output(
|
||||
"positive",
|
||||
tooltip=(
|
||||
"Standard masked positive conditioning with hooks preserved."
|
||||
),
|
||||
),
|
||||
_comfy_io.Conditioning.Output(
|
||||
"negative",
|
||||
tooltip=(
|
||||
"Standard masked negative conditioning with hooks preserved."
|
||||
),
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
positive: object,
|
||||
negative: object,
|
||||
region_masks: object,
|
||||
regional_prompt_weight: float,
|
||||
region_mask_feather: int,
|
||||
) -> tuple[object, object]:
|
||||
"""Return hook-preserving standard Comfy conditioning values."""
|
||||
|
||||
return cls.conditioning_service_class().assemble(
|
||||
positive=positive,
|
||||
negative=negative,
|
||||
masks=region_masks,
|
||||
regional_prompt_weight=regional_prompt_weight,
|
||||
region_mask_feather=region_mask_feather,
|
||||
)
|
||||
@@ -0,0 +1,135 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Expose explicitly requested upstream attention concepts as SEGS."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..domain.attention_concepts import parse_attention_concepts
|
||||
from ..services.attention_region_node_service import ATTENTION_REGION_NODE_SERVICE
|
||||
from .attention_region_inputs import (
|
||||
attention_region_control_inputs,
|
||||
attention_region_controls,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
hidden: ClassVar[Any]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class ConceptAttentionSEGSV3(_ComfyNodeBase):
|
||||
"""Return attention-derived instances for explicitly named concepts."""
|
||||
|
||||
GRAPH_PASSTHROUGH_OUTPUTS = {0: "image"}
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the downstream concept-attention SEGS contract."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.ConceptAttentionSEGS",
|
||||
display_name="Concept Attention SEGS",
|
||||
category="SimpleSyrup/Detection",
|
||||
description=(
|
||||
"Returns regions associated with explicit concepts from a selected "
|
||||
"sampler in the connected image's provenance."
|
||||
),
|
||||
search_aliases=[
|
||||
"attention heatmap",
|
||||
"prompt segmentation",
|
||||
"attention mask",
|
||||
],
|
||||
hidden=[_comfy_io.Hidden.unique_id],
|
||||
inputs=[
|
||||
_comfy_io.Image.Input(
|
||||
"image",
|
||||
tooltip=(
|
||||
"Image whose graph provenance identifies the sampling chain; "
|
||||
"the image is returned unchanged."
|
||||
),
|
||||
),
|
||||
_comfy_io.String.Input(
|
||||
"concepts",
|
||||
multiline=True,
|
||||
default="subject",
|
||||
tooltip=(
|
||||
"Enter one or more concepts separated by |, such as "
|
||||
"girl | pink hair | cat."
|
||||
),
|
||||
),
|
||||
*attention_region_control_inputs(_comfy_io),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Image.Output("image", tooltip="Unchanged connected image."),
|
||||
_comfy_io.SEGS.Output(
|
||||
"segs",
|
||||
tooltip="Labeled attention-derived instances for the concepts.",
|
||||
is_output_list=True,
|
||||
),
|
||||
_comfy_io.Mask.Output(
|
||||
"mask", tooltip="Union of all retained concept instances."
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
image: object,
|
||||
concepts: str,
|
||||
sampler_stage: int,
|
||||
capture_start: float,
|
||||
capture_end: float,
|
||||
minimum_strength: float,
|
||||
minimum_consensus: float,
|
||||
geometry_recall: float,
|
||||
split_sensitivity: float,
|
||||
instance_recall: float,
|
||||
minimum_region_size: int,
|
||||
keep_only: int,
|
||||
keep_by: str,
|
||||
combine_segs: bool,
|
||||
matte_solidity: float,
|
||||
edge_feather: int,
|
||||
capture_profile: str,
|
||||
evidence_mode: str,
|
||||
) -> tuple[object, object, object]:
|
||||
"""Consume shared capture evidence and render requested concept SEGS."""
|
||||
|
||||
del sampler_stage
|
||||
if not parse_attention_concepts(concepts):
|
||||
raise ValueError("Concept Attention SEGS requires at least one concept.")
|
||||
result = ATTENTION_REGION_NODE_SERVICE.for_image(
|
||||
request_node_id=str(cls.hidden.unique_id),
|
||||
image=image,
|
||||
controls=attention_region_controls(
|
||||
capture_start=capture_start,
|
||||
capture_end=capture_end,
|
||||
minimum_strength=minimum_strength,
|
||||
minimum_consensus=minimum_consensus,
|
||||
geometry_recall=geometry_recall,
|
||||
split_sensitivity=split_sensitivity,
|
||||
instance_recall=instance_recall,
|
||||
minimum_region_size=minimum_region_size,
|
||||
keep_only=keep_only,
|
||||
keep_by=keep_by,
|
||||
combine_segs=combine_segs,
|
||||
matte_solidity=matte_solidity,
|
||||
edge_feather=edge_feather,
|
||||
capture_profile=capture_profile,
|
||||
evidence_mode=evidence_mode,
|
||||
),
|
||||
)
|
||||
return result.image, result.segs, result.mask
|
||||
@@ -9,6 +9,7 @@ from __future__ import annotations
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..domain.prompt_batch_parser import DEFAULT_PROMPT_BATCH_SEPARATOR
|
||||
from ..runtime.prompt_control_batch_graph import PromptControlBatchGraphBuilder
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -47,8 +48,9 @@ class EncodePromptBatchWithPromptControl(_ComfyNodeBase):
|
||||
enable_expand=True,
|
||||
category="SimpleSyrup/Conditioning",
|
||||
description=(
|
||||
"Encodes [SEP]-separated prompts into per-segment Prompt Control "
|
||||
"conditioning batches."
|
||||
"Encodes prompts separated by [SEP] or [SEP|name] into matched "
|
||||
"Prompt Control batches, reusing each side's global text for "
|
||||
"missing regions."
|
||||
),
|
||||
inputs=[
|
||||
_comfy_io.Clip.Input(
|
||||
@@ -64,8 +66,9 @@ class EncodePromptBatchWithPromptControl(_ComfyNodeBase):
|
||||
multiline=True,
|
||||
default="",
|
||||
tooltip=(
|
||||
"Positive Prompt Control prompts in SEGS order, separated "
|
||||
"by the separator text."
|
||||
"Positive Prompt Control prompts separated by [SEP] or "
|
||||
"[SEP|name] in positional order; each segment keeps its "
|
||||
"aligned LoRA hooks, and global text fills missing regions."
|
||||
),
|
||||
),
|
||||
_comfy_io.String.Input(
|
||||
@@ -73,14 +76,20 @@ class EncodePromptBatchWithPromptControl(_ComfyNodeBase):
|
||||
multiline=True,
|
||||
default="",
|
||||
tooltip=(
|
||||
"Negative Prompt Control prompts in SEGS order, separated "
|
||||
"by the separator text."
|
||||
"Negative Prompt Control prompts separated by [SEP] or "
|
||||
"[SEP|name] in positional order; each segment shares hooks "
|
||||
"with the matching positive index, and global text fills "
|
||||
"missing regions."
|
||||
),
|
||||
),
|
||||
_comfy_io.String.Input(
|
||||
"separator",
|
||||
default="[SEP]",
|
||||
tooltip="Text marker that splits prompts into per-SEGS entries.",
|
||||
default=DEFAULT_PROMPT_BATCH_SEPARATOR,
|
||||
tooltip=(
|
||||
"Text marker that splits prompts into per-SEGS entries. "
|
||||
"With the default [SEP], use [SEP|name] to add an "
|
||||
"organizational label."
|
||||
),
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
|
||||
@@ -0,0 +1,117 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Comfy v3 KSampler for full-context Attention Coupling."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..nodes import tooltips
|
||||
from ..services.attention_coupling_sampling_service import (
|
||||
AttentionCouplingSamplingService,
|
||||
)
|
||||
from .ksampler_schema import (
|
||||
ATTENTION_COUPLING_REGIONAL_PROMPT_WEIGHT_DEFAULT,
|
||||
attention_coupling_ksampler_inputs,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class KSamplerAttentionCouplingV3(_ComfyNodeBase):
|
||||
"""Sample supported models with mask-bound regional attention."""
|
||||
|
||||
sampling_service_class: ClassVar[type[AttentionCouplingSamplingService]] = (
|
||||
AttentionCouplingSamplingService
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the full-context Attention Coupling sampler schema."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.KSamplerAttentionCoupling",
|
||||
display_name="KSampler (Attention Coupling)",
|
||||
category="SimpleSyrup/Sampling",
|
||||
description=(
|
||||
"With ordinary conditioning and no masks, denoises through the "
|
||||
"normal KSampler path without Attention Coupling preparation. "
|
||||
"With conditioning batches and masks, denoises supported Anima "
|
||||
"and standard SD/SDXL models through one "
|
||||
"shared trajectory while coupling global and masked regional "
|
||||
"cross-attention. The input MODEL may carry a global LoRA. Anima "
|
||||
"regions may also carry ordered, independently scheduled Prompt "
|
||||
"Control model LoRAs whose overlapping deltas compose in declared "
|
||||
"order. Runtime scales with active adapters, ranks, and targets. "
|
||||
"Standard SD/SDXL regional model-side hooks and unsupported Anima "
|
||||
"adapter targets fail before sampling."
|
||||
),
|
||||
search_aliases=[
|
||||
"attention coupling",
|
||||
"regional lora",
|
||||
"anima regional prompt",
|
||||
"sdxl regional prompt",
|
||||
],
|
||||
inputs=attention_coupling_ksampler_inputs(
|
||||
_comfy_io,
|
||||
region_masks_optional=True,
|
||||
),
|
||||
outputs=[
|
||||
_comfy_io.Latent.Output(
|
||||
"latent",
|
||||
tooltip=tooltips.DENOISED_LATENT_OUTPUT,
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
model: Any,
|
||||
seed: int,
|
||||
steps: int,
|
||||
cfg: float,
|
||||
sampler_name: str,
|
||||
scheduler: str,
|
||||
positive: object,
|
||||
negative: object,
|
||||
latent_image: dict[str, Any],
|
||||
denoise: float,
|
||||
region_masks: object | None = None,
|
||||
regional_prompt_weight: float = (
|
||||
ATTENTION_COUPLING_REGIONAL_PROMPT_WEIGHT_DEFAULT
|
||||
),
|
||||
region_mask_feather: int = 0,
|
||||
) -> tuple[dict[str, Any]]:
|
||||
"""Delegate ordinary or regional sampling to the routing service."""
|
||||
|
||||
output = cls.sampling_service_class().sample(
|
||||
model=model,
|
||||
seed=seed,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
sampler_name=sampler_name,
|
||||
scheduler=scheduler,
|
||||
positive=positive,
|
||||
negative=negative,
|
||||
region_masks=region_masks,
|
||||
regional_prompt_weight=regional_prompt_weight,
|
||||
region_mask_feather=region_mask_feather,
|
||||
latent_image=latent_image,
|
||||
denoise=denoise,
|
||||
)
|
||||
return (output,)
|
||||
@@ -0,0 +1,139 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Native Comfy v3 node for Contextual Attention Coupling."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..nodes import tooltips
|
||||
from ..services.contextual_attention_coupling_sampling_service import (
|
||||
ContextualAttentionCouplingSamplingService,
|
||||
)
|
||||
from .ksampler_schema import (
|
||||
attention_coupling_ksampler_inputs,
|
||||
contextual_diffusion_inputs,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class KSamplerContextualAttentionCouplingV3(_ComfyNodeBase):
|
||||
"""Sample contextual model views with regional attention."""
|
||||
|
||||
sampling_service_class: ClassVar[
|
||||
type[ContextualAttentionCouplingSamplingService]
|
||||
] = ContextualAttentionCouplingSamplingService
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the Contextual Attention Coupling sampler schema."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.KSamplerAttentionCouplingContextual",
|
||||
display_name="KSampler (Attention Coupling + Contextual Diffusion)",
|
||||
category="SimpleSyrup/Sampling",
|
||||
description=(
|
||||
"Preserves large-image composition through Contextual Diffusion "
|
||||
"while coupling regional attention in every local and reduced-global "
|
||||
"Anima or standard SD/SDXL view. Global LoRAs remain on the input "
|
||||
"model. Anima regional LoRA stacks are prepared once, retain "
|
||||
"independent schedules and full quality, and skip inactive work. "
|
||||
"Optional SEGS guide the shared local tile plan. Standard SD/SDXL "
|
||||
"regional model-side hooks and unsupported Anima targets fail before "
|
||||
"sampling."
|
||||
),
|
||||
search_aliases=[
|
||||
"contextual attention coupling",
|
||||
"contextual regional lora",
|
||||
"anima contextual regional prompt",
|
||||
"sdxl contextual regional prompt",
|
||||
"contextual multidiffusion regional lora",
|
||||
"contextual mixture of diffusers regional lora",
|
||||
],
|
||||
inputs=[
|
||||
*attention_coupling_ksampler_inputs(_comfy_io),
|
||||
*contextual_diffusion_inputs(_comfy_io),
|
||||
_comfy_io.SEGS.Input(
|
||||
"segs",
|
||||
optional=True,
|
||||
tooltip=tooltips.CONTEXTUAL_DIFFUSION_SEGS,
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Latent.Output(
|
||||
"latent",
|
||||
tooltip=tooltips.DENOISED_LATENT_OUTPUT,
|
||||
),
|
||||
_comfy_io.SEGS.Output(
|
||||
"contexts_segs",
|
||||
tooltip=tooltips.CONTEXTUAL_DIFFUSION_CONTEXTS_OUTPUT,
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
model: Any,
|
||||
seed: int,
|
||||
steps: int,
|
||||
cfg: float,
|
||||
sampler_name: str,
|
||||
scheduler: str,
|
||||
positive: object,
|
||||
negative: object,
|
||||
region_masks: object,
|
||||
regional_prompt_weight: float,
|
||||
region_mask_feather: int,
|
||||
latent_image: dict[str, Any],
|
||||
denoise: float = 1.0,
|
||||
diffusion_mode: str = "multidiffusion",
|
||||
latent_context_size: int = 96,
|
||||
latent_context_overlap: int = 32,
|
||||
latent_context_batch_size: int = 4,
|
||||
global_weight: float = 1.0,
|
||||
global_steps: int = 1,
|
||||
global_decay: float = 0.5,
|
||||
segs: object | None = None,
|
||||
) -> tuple[dict[str, Any], object]:
|
||||
"""Delegate the complete request to the combined application service."""
|
||||
|
||||
result = cls.sampling_service_class().sample(
|
||||
model=model,
|
||||
seed=seed,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
sampler_name=sampler_name,
|
||||
scheduler=scheduler,
|
||||
positive=positive,
|
||||
negative=negative,
|
||||
region_masks=region_masks,
|
||||
regional_prompt_weight=regional_prompt_weight,
|
||||
region_mask_feather=region_mask_feather,
|
||||
latent_image=latent_image,
|
||||
denoise=denoise,
|
||||
diffusion_mode=diffusion_mode,
|
||||
latent_context_size=latent_context_size,
|
||||
latent_context_overlap=latent_context_overlap,
|
||||
latent_context_batch_size=latent_context_batch_size,
|
||||
global_weight=global_weight,
|
||||
global_steps=global_steps,
|
||||
global_decay=global_decay,
|
||||
segs=segs,
|
||||
)
|
||||
return result.latent, result.contexts
|
||||
@@ -0,0 +1,133 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Native Comfy v3 node for Contextual Diffusion sampling."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..nodes import tooltips
|
||||
from ..services.contextual_diffusion_sampling_service import (
|
||||
ContextualDiffusionSamplingService,
|
||||
)
|
||||
from .ksampler_schema import (
|
||||
contextual_diffusion_inputs,
|
||||
ksampler_inputs,
|
||||
optional_regional_sampling_inputs,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class KSamplerContextualDiffusionV3(_ComfyNodeBase):
|
||||
"""Edit large latents through coordinated global and detailed contexts."""
|
||||
|
||||
service_class: ClassVar[type[ContextualDiffusionSamplingService]] = (
|
||||
ContextualDiffusionSamplingService
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the native Contextual Diffusion KSampler schema."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.KSamplerContextualDiffusion",
|
||||
display_name="KSampler (Contextual Diffusion)",
|
||||
category="SimpleSyrup/Sampling",
|
||||
description=(
|
||||
"Preserves composition while applying appearance and subject-detail "
|
||||
"edits to large latents through global context and optional "
|
||||
"SEGS-guided tiles."
|
||||
),
|
||||
search_aliases=[
|
||||
"ksampler",
|
||||
"contextual diffusion",
|
||||
"contextual tiled diffusion",
|
||||
"high resolution edit",
|
||||
"sam tiled diffusion",
|
||||
],
|
||||
inputs=[
|
||||
*ksampler_inputs(_comfy_io, steps_default=4, cfg_default=1.0),
|
||||
*contextual_diffusion_inputs(_comfy_io),
|
||||
*optional_regional_sampling_inputs(
|
||||
_comfy_io,
|
||||
segs_tooltip=tooltips.CONTEXTUAL_DIFFUSION_SEGS,
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Latent.Output(
|
||||
"latent",
|
||||
tooltip=tooltips.DENOISED_LATENT_OUTPUT,
|
||||
),
|
||||
_comfy_io.SEGS.Output(
|
||||
"contexts_segs",
|
||||
tooltip=tooltips.CONTEXTUAL_DIFFUSION_CONTEXTS_OUTPUT,
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
model: Any,
|
||||
seed: int,
|
||||
steps: int,
|
||||
cfg: float,
|
||||
sampler_name: str,
|
||||
scheduler: str,
|
||||
positive: Any,
|
||||
negative: Any,
|
||||
latent_image: dict[str, Any],
|
||||
denoise: float = 1.0,
|
||||
diffusion_mode: str = "multidiffusion",
|
||||
latent_context_size: int = 96,
|
||||
latent_context_overlap: int = 32,
|
||||
latent_context_batch_size: int = 4,
|
||||
global_weight: float = 1.0,
|
||||
global_steps: int = 1,
|
||||
global_decay: float = 0.5,
|
||||
segs: object | None = None,
|
||||
region_masks: object | None = None,
|
||||
regional_prompt_weight: float = 0.5,
|
||||
region_mask_feather: int = 0,
|
||||
) -> tuple[dict[str, Any], object]:
|
||||
"""Delegate Contextual Diffusion sampling to its application service."""
|
||||
|
||||
result = cls.service_class().sample(
|
||||
model=model,
|
||||
seed=seed,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
sampler_name=sampler_name,
|
||||
scheduler=scheduler,
|
||||
positive=positive,
|
||||
negative=negative,
|
||||
latent_image=latent_image,
|
||||
denoise=denoise,
|
||||
diffusion_mode=diffusion_mode,
|
||||
latent_context_size=latent_context_size,
|
||||
latent_context_overlap=latent_context_overlap,
|
||||
latent_context_batch_size=latent_context_batch_size,
|
||||
global_weight=global_weight,
|
||||
global_steps=global_steps,
|
||||
global_decay=global_decay,
|
||||
segs=segs,
|
||||
region_masks=region_masks,
|
||||
regional_prompt_weight=regional_prompt_weight,
|
||||
region_mask_feather=region_mask_feather,
|
||||
)
|
||||
return result.latent, result.contexts
|
||||
@@ -0,0 +1,103 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Comfy v3 KSampler for full-context regional prompting."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..nodes import tooltips
|
||||
from ..services.ksampler_sampling_service import KSamplerSamplingService
|
||||
from ..services.regional_conditioning_service import RegionalConditioningService
|
||||
from .ksampler_schema import regional_ksampler_inputs
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class KSamplerPromptByRegionV3(_ComfyNodeBase):
|
||||
"""Sample a full latent with global-first ordered regional prompts."""
|
||||
|
||||
conditioning_service_class: ClassVar[type[RegionalConditioningService]] = (
|
||||
RegionalConditioningService
|
||||
)
|
||||
sampling_service_class: ClassVar[type[KSamplerSamplingService]] = (
|
||||
KSamplerSamplingService
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the non-tiled regional KSampler schema."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.KSamplerPromptByRegion",
|
||||
display_name="KSampler (Prompt by Region)",
|
||||
category="SimpleSyrup/Sampling",
|
||||
description=(
|
||||
"Denoises the full latent with one global prompt and ordered "
|
||||
"mask-bound regional prompts."
|
||||
),
|
||||
search_aliases=["ksampler", "regional prompt", "masked prompt"],
|
||||
inputs=regional_ksampler_inputs(_comfy_io),
|
||||
outputs=[
|
||||
_comfy_io.Latent.Output(
|
||||
"latent",
|
||||
tooltip=tooltips.DENOISED_LATENT_OUTPUT,
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
model: Any,
|
||||
seed: int,
|
||||
steps: int,
|
||||
cfg: float,
|
||||
sampler_name: str,
|
||||
scheduler: str,
|
||||
positive: object,
|
||||
negative: object,
|
||||
region_masks: object,
|
||||
regional_prompt_weight: float,
|
||||
region_mask_feather: int,
|
||||
latent_image: dict[str, Any],
|
||||
denoise: float,
|
||||
) -> tuple[dict[str, Any]]:
|
||||
"""Assemble regional conditioning and sample the full latent."""
|
||||
|
||||
assembled_positive, assembled_negative = (
|
||||
cls.conditioning_service_class().assemble(
|
||||
positive=positive,
|
||||
negative=negative,
|
||||
masks=region_masks,
|
||||
regional_prompt_weight=regional_prompt_weight,
|
||||
region_mask_feather=region_mask_feather,
|
||||
)
|
||||
)
|
||||
output = cls.sampling_service_class().sample(
|
||||
model=model,
|
||||
seed=seed,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
sampler_name=sampler_name,
|
||||
scheduler=scheduler,
|
||||
positive=assembled_positive,
|
||||
negative=assembled_negative,
|
||||
latent_image=latent_image,
|
||||
denoise=denoise,
|
||||
)
|
||||
return (output,)
|
||||
@@ -0,0 +1,126 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Comfy v3 tiled KSampler for full-context regional prompting."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..domain.regional_features import RegionalFeature, RegionalFeatureRequest
|
||||
from ..nodes import tooltips
|
||||
from ..services.regional_conditioning_service import RegionalConditioningService
|
||||
from ..services.tiled_diffusion_sampling_service import TiledDiffusionSamplingService
|
||||
from .ksampler_schema import regional_ksampler_inputs, tiled_diffusion_inputs
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class KSamplerPromptByTiledRegionV3(_ComfyNodeBase):
|
||||
"""Sample tiled latents with global-first ordered regional prompts."""
|
||||
|
||||
conditioning_service_class: ClassVar[type[RegionalConditioningService]] = (
|
||||
RegionalConditioningService
|
||||
)
|
||||
sampling_service_class: ClassVar[type[TiledDiffusionSamplingService]] = (
|
||||
TiledDiffusionSamplingService
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the tiled regional KSampler schema."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.KSamplerPromptByTiledRegion",
|
||||
display_name="KSampler (Prompt by Tiled Region)",
|
||||
category="SimpleSyrup/Sampling",
|
||||
description=(
|
||||
"Denoises large latents in overlapping tiles while preserving "
|
||||
"global and ordered mask-bound regional prompts."
|
||||
),
|
||||
search_aliases=[
|
||||
"ksampler",
|
||||
"regional prompt",
|
||||
"tiled regional prompt",
|
||||
"regional hires fix",
|
||||
],
|
||||
inputs=[
|
||||
*regional_ksampler_inputs(_comfy_io),
|
||||
*tiled_diffusion_inputs(_comfy_io),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Latent.Output(
|
||||
"latent",
|
||||
tooltip=tooltips.DENOISED_LATENT_OUTPUT,
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
model: Any,
|
||||
seed: int,
|
||||
steps: int,
|
||||
cfg: float,
|
||||
sampler_name: str,
|
||||
scheduler: str,
|
||||
positive: object,
|
||||
negative: object,
|
||||
region_masks: object,
|
||||
regional_prompt_weight: float,
|
||||
region_mask_feather: int,
|
||||
latent_image: dict[str, Any],
|
||||
denoise: float,
|
||||
diffusion_mode: str,
|
||||
latent_tile_width: int,
|
||||
latent_tile_height: int,
|
||||
latent_tile_overlap: int,
|
||||
latent_tile_batch_size: int,
|
||||
) -> tuple[dict[str, Any]]:
|
||||
"""Assemble regional conditioning and sample overlapping latent tiles."""
|
||||
|
||||
assembled_positive, assembled_negative = (
|
||||
cls.conditioning_service_class().assemble(
|
||||
positive=positive,
|
||||
negative=negative,
|
||||
masks=region_masks,
|
||||
regional_prompt_weight=regional_prompt_weight,
|
||||
region_mask_feather=region_mask_feather,
|
||||
)
|
||||
)
|
||||
output = cls.sampling_service_class().sample(
|
||||
diffusion_mode=diffusion_mode,
|
||||
model=model,
|
||||
seed=seed,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
sampler_name=sampler_name,
|
||||
scheduler=scheduler,
|
||||
positive=assembled_positive,
|
||||
negative=assembled_negative,
|
||||
latent_image=latent_image,
|
||||
denoise=denoise,
|
||||
latent_tile_width=latent_tile_width,
|
||||
latent_tile_height=latent_tile_height,
|
||||
latent_tile_overlap=latent_tile_overlap,
|
||||
latent_tile_batch_size=latent_tile_batch_size,
|
||||
preview_context=None,
|
||||
feature_request=RegionalFeatureRequest(
|
||||
frozenset({RegionalFeature.FULL_CONTEXT_MASKED_CONDITIONING})
|
||||
),
|
||||
)
|
||||
return (output,)
|
||||
@@ -0,0 +1,363 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Own shared native Comfy v3 KSampler input declarations."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
from ..domain.regional_prompting import MAX_REGIONAL_PROMPT_WEIGHT
|
||||
from ..domain.tiled_diffusion import TILED_DIFFUSION_MODES
|
||||
from ..nodes import tooltips
|
||||
from ..runtime import sampling_samplers, sampling_schedulers
|
||||
|
||||
MAX_LATENT_TILE_SIZE = 512
|
||||
MAX_LATENT_CONTEXT_SIZE = 512
|
||||
ATTENTION_COUPLING_REGIONAL_PROMPT_WEIGHT_DEFAULT = 1.0
|
||||
|
||||
|
||||
def ksampler_inputs(
|
||||
comfy_io: Any,
|
||||
*,
|
||||
steps_default: int,
|
||||
cfg_default: float,
|
||||
) -> list[Any]:
|
||||
"""Return standard KSampler inputs with caller-selected defaults."""
|
||||
|
||||
conditioning = comfy_io.Custom("CONDITIONING,CONDITIONING_BATCH")
|
||||
return [
|
||||
comfy_io.Model.Input("model", tooltip=tooltips.SAMPLING_MODEL),
|
||||
comfy_io.Int.Input(
|
||||
"seed",
|
||||
default=0,
|
||||
min=0,
|
||||
max=0xFFFFFFFFFFFFFFFF,
|
||||
control_after_generate=True,
|
||||
tooltip=tooltips.SAMPLING_SEED,
|
||||
),
|
||||
comfy_io.Int.Input(
|
||||
"steps",
|
||||
default=steps_default,
|
||||
min=1,
|
||||
max=10000,
|
||||
tooltip=tooltips.SAMPLING_STEPS,
|
||||
),
|
||||
comfy_io.Float.Input(
|
||||
"cfg",
|
||||
default=cfg_default,
|
||||
min=0.0,
|
||||
max=100.0,
|
||||
step=0.1,
|
||||
round=0.01,
|
||||
tooltip=tooltips.SAMPLING_CFG,
|
||||
),
|
||||
comfy_io.Combo.Input(
|
||||
"sampler_name",
|
||||
options=list(sampling_samplers.available_samplers()),
|
||||
tooltip=tooltips.SAMPLER_NAME,
|
||||
),
|
||||
comfy_io.Combo.Input(
|
||||
"scheduler",
|
||||
options=list(sampling_schedulers.available_schedulers()),
|
||||
tooltip=tooltips.SCHEDULER,
|
||||
),
|
||||
conditioning.Input("positive", tooltip=tooltips.POSITIVE_CONDITIONING),
|
||||
conditioning.Input("negative", tooltip=tooltips.NEGATIVE_CONDITIONING),
|
||||
comfy_io.Latent.Input("latent_image", tooltip=tooltips.LATENT_IMAGE),
|
||||
comfy_io.Float.Input(
|
||||
"denoise",
|
||||
default=1.0,
|
||||
min=0.0,
|
||||
max=1.0,
|
||||
step=0.01,
|
||||
tooltip=tooltips.DENOISE_STRENGTH,
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def tiled_diffusion_inputs(comfy_io: Any) -> list[Any]:
|
||||
"""Return tiled diffusion mode and latent tile controls."""
|
||||
|
||||
return [
|
||||
comfy_io.Combo.Input(
|
||||
"diffusion_mode",
|
||||
options=list(TILED_DIFFUSION_MODES),
|
||||
default="multidiffusion",
|
||||
tooltip=tooltips.TILED_DIFFUSION_MODE,
|
||||
),
|
||||
comfy_io.Int.Input(
|
||||
"latent_tile_width",
|
||||
default=128,
|
||||
min=16,
|
||||
max=MAX_LATENT_TILE_SIZE,
|
||||
step=16,
|
||||
tooltip=tooltips.LATENT_TILE_WIDTH,
|
||||
),
|
||||
comfy_io.Int.Input(
|
||||
"latent_tile_height",
|
||||
default=128,
|
||||
min=16,
|
||||
max=MAX_LATENT_TILE_SIZE,
|
||||
step=16,
|
||||
tooltip=tooltips.LATENT_TILE_HEIGHT,
|
||||
),
|
||||
comfy_io.Int.Input(
|
||||
"latent_tile_overlap",
|
||||
default=16,
|
||||
min=0,
|
||||
max=256,
|
||||
step=4,
|
||||
tooltip=tooltips.LATENT_TILE_OVERLAP,
|
||||
),
|
||||
comfy_io.Int.Input(
|
||||
"latent_tile_batch_size",
|
||||
default=4,
|
||||
min=1,
|
||||
max=8,
|
||||
step=1,
|
||||
tooltip=tooltips.LATENT_TILE_BATCH_SIZE,
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def contextual_diffusion_inputs(comfy_io: Any) -> list[Any]:
|
||||
"""Return Contextual Diffusion layout and global schedule controls."""
|
||||
|
||||
return [
|
||||
comfy_io.Combo.Input(
|
||||
"diffusion_mode",
|
||||
options=list(TILED_DIFFUSION_MODES),
|
||||
default="multidiffusion",
|
||||
tooltip=tooltips.TILED_DIFFUSION_MODE,
|
||||
),
|
||||
comfy_io.Int.Input(
|
||||
"latent_context_size",
|
||||
default=96,
|
||||
min=16,
|
||||
max=MAX_LATENT_CONTEXT_SIZE,
|
||||
step=16,
|
||||
tooltip=tooltips.LATENT_CONTEXT_SIZE,
|
||||
),
|
||||
comfy_io.Int.Input(
|
||||
"latent_context_overlap",
|
||||
default=32,
|
||||
min=0,
|
||||
max=256,
|
||||
step=4,
|
||||
tooltip=tooltips.LATENT_CONTEXT_OVERLAP,
|
||||
),
|
||||
comfy_io.Int.Input(
|
||||
"latent_context_batch_size",
|
||||
default=4,
|
||||
min=1,
|
||||
max=8,
|
||||
step=1,
|
||||
tooltip=tooltips.LATENT_CONTEXT_BATCH_SIZE,
|
||||
),
|
||||
comfy_io.Float.Input(
|
||||
"global_weight",
|
||||
default=1.0,
|
||||
min=0.0,
|
||||
max=2.0,
|
||||
step=0.05,
|
||||
tooltip=tooltips.GLOBAL_CONTEXT_WEIGHT,
|
||||
),
|
||||
comfy_io.Int.Input(
|
||||
"global_steps",
|
||||
default=1,
|
||||
min=0,
|
||||
max=10000,
|
||||
step=1,
|
||||
tooltip=tooltips.GLOBAL_CONTEXT_STEPS,
|
||||
),
|
||||
comfy_io.Float.Input(
|
||||
"global_decay",
|
||||
default=0.5,
|
||||
min=0.0,
|
||||
max=1.0,
|
||||
step=0.05,
|
||||
tooltip=tooltips.GLOBAL_CONTEXT_DECAY,
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def optional_regional_sampling_inputs(
|
||||
comfy_io: Any,
|
||||
*,
|
||||
segs_tooltip: str,
|
||||
) -> list[Any]:
|
||||
"""Return optional SEGS and Regional Conditioning controls."""
|
||||
|
||||
return [
|
||||
comfy_io.SEGS.Input(
|
||||
"segs",
|
||||
optional=True,
|
||||
tooltip=segs_tooltip,
|
||||
),
|
||||
comfy_io.Mask.Input(
|
||||
"region_masks",
|
||||
optional=True,
|
||||
tooltip=tooltips.OPTIONAL_REGIONAL_MASKS,
|
||||
),
|
||||
comfy_io.Float.Input(
|
||||
"regional_prompt_weight",
|
||||
default=0.5,
|
||||
min=0.0,
|
||||
max=MAX_REGIONAL_PROMPT_WEIGHT,
|
||||
step=0.01,
|
||||
round=0.01,
|
||||
optional=True,
|
||||
tooltip=tooltips.OPTIONAL_REGIONAL_PROMPT_WEIGHT,
|
||||
),
|
||||
comfy_io.Int.Input(
|
||||
"region_mask_feather",
|
||||
default=0,
|
||||
min=0,
|
||||
max=512,
|
||||
step=1,
|
||||
optional=True,
|
||||
tooltip=tooltips.OPTIONAL_REGION_MASK_FEATHER,
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def regional_ksampler_inputs(comfy_io: Any) -> list[Any]:
|
||||
"""Return common regional KSampler inputs in workflow order."""
|
||||
|
||||
base = ksampler_inputs(comfy_io, steps_default=20, cfg_default=8.0)
|
||||
return [
|
||||
*base[:6],
|
||||
*regional_conditioning_inputs(comfy_io),
|
||||
*base[8:],
|
||||
]
|
||||
|
||||
|
||||
def attention_coupling_ksampler_inputs(
|
||||
comfy_io: Any,
|
||||
*,
|
||||
region_masks_optional: bool = False,
|
||||
) -> list[Any]:
|
||||
"""Return Attention Coupling inputs with caller-owned bypass availability."""
|
||||
|
||||
base = ksampler_inputs(comfy_io, steps_default=20, cfg_default=8.0)
|
||||
conditioning_batch = comfy_io.Custom("CONDITIONING_BATCH")
|
||||
return [
|
||||
comfy_io.Model.Input(
|
||||
"model",
|
||||
tooltip=(
|
||||
"Supported Anima or standard SD/SDXL model used for one shared "
|
||||
"denoiser trajectory; apply global model LoRAs before connecting it."
|
||||
),
|
||||
),
|
||||
*base[1:6],
|
||||
comfy_io.MultiType.Input(
|
||||
"positive",
|
||||
[comfy_io.Conditioning, conditioning_batch],
|
||||
tooltip=(
|
||||
"Global-first positive conditioning: entry 0 is global and later "
|
||||
"entries pair with masks. Regional Prompt Control WeightHooks may "
|
||||
"contain ordered full-rank Anima LoRA stacks with independent "
|
||||
"schedules; standard SD/SDXL rejects regional model-side hooks."
|
||||
),
|
||||
),
|
||||
comfy_io.MultiType.Input(
|
||||
"negative",
|
||||
[comfy_io.Conditioning, conditioning_batch],
|
||||
tooltip=(
|
||||
"Global-first negative conditioning aligned to the same masks; "
|
||||
"Anima regional LoRA hooks retain their negative-branch ownership "
|
||||
"and independent schedules."
|
||||
),
|
||||
),
|
||||
comfy_io.Mask.Input(
|
||||
"region_masks",
|
||||
optional=region_masks_optional,
|
||||
tooltip=(
|
||||
"Optional ordered masks paired with conditioning entries 1 onward. "
|
||||
"Leave disconnected with ordinary conditioning to bypass Attention "
|
||||
"Coupling. In overlaps, prompt contributions are normalized while "
|
||||
"Anima regional LoRA deltas add in declared adapter and region order."
|
||||
),
|
||||
),
|
||||
comfy_io.Float.Input(
|
||||
"regional_prompt_weight",
|
||||
default=ATTENTION_COUPLING_REGIONAL_PROMPT_WEIGHT_DEFAULT,
|
||||
min=0.0,
|
||||
max=MAX_REGIONAL_PROMPT_WEIGHT,
|
||||
step=0.01,
|
||||
round=0.01,
|
||||
tooltip=(
|
||||
"Balances regional cross-attention against the global prompt from "
|
||||
"0 (global only) to 1 (regional only inside solid masks); regional "
|
||||
"Anima LoRA strength remains controlled by each hook."
|
||||
),
|
||||
),
|
||||
comfy_io.Int.Input(
|
||||
"region_mask_feather",
|
||||
default=0,
|
||||
min=0,
|
||||
max=512,
|
||||
step=1,
|
||||
tooltip=(
|
||||
"Softens Attention Coupling and Anima regional LoRA boundaries by "
|
||||
"this many image pixels; 0 preserves authored mask values."
|
||||
),
|
||||
),
|
||||
*base[8:],
|
||||
]
|
||||
|
||||
|
||||
def regional_conditioning_inputs(comfy_io: Any) -> list[Any]:
|
||||
"""Return the authoritative ordered regional-composition inputs."""
|
||||
|
||||
conditioning_batch = comfy_io.Custom("CONDITIONING_BATCH")
|
||||
return [
|
||||
comfy_io.MultiType.Input(
|
||||
"positive",
|
||||
[comfy_io.Conditioning, conditioning_batch],
|
||||
tooltip=(
|
||||
"Positive conditioning whose first batch entry is global and "
|
||||
"later entries pair with masks in order."
|
||||
),
|
||||
),
|
||||
comfy_io.MultiType.Input(
|
||||
"negative",
|
||||
[comfy_io.Conditioning, conditioning_batch],
|
||||
tooltip=(
|
||||
"Negative conditioning whose first batch entry is global and "
|
||||
"later entries pair with masks in order."
|
||||
),
|
||||
),
|
||||
comfy_io.Mask.Input(
|
||||
"region_masks",
|
||||
tooltip=(
|
||||
"Ordered authored masks; mask 0 pairs with conditioning batch entry 1."
|
||||
),
|
||||
),
|
||||
comfy_io.Float.Input(
|
||||
"regional_prompt_weight",
|
||||
default=0.5,
|
||||
min=0.0,
|
||||
max=MAX_REGIONAL_PROMPT_WEIGHT,
|
||||
step=0.01,
|
||||
round=0.01,
|
||||
tooltip=(
|
||||
"Balances regional prompts against the global prompt; 0 uses "
|
||||
"only global prompting, 1 uses only regional prompting inside "
|
||||
"solid masks, and overlaps reduce the global share further."
|
||||
),
|
||||
),
|
||||
comfy_io.Int.Input(
|
||||
"region_mask_feather",
|
||||
default=0,
|
||||
min=0,
|
||||
max=512,
|
||||
step=1,
|
||||
tooltip=(
|
||||
"Softens regional mask edges by this many image pixels; 0 "
|
||||
"preserves authored mask values."
|
||||
),
|
||||
),
|
||||
]
|
||||
@@ -0,0 +1,135 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Comfy v3 KSampler for tiled Attention Coupling."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..nodes import tooltips
|
||||
from ..services.tiled_attention_coupling_sampling_service import (
|
||||
TiledAttentionCouplingSamplingService,
|
||||
)
|
||||
from .ksampler_schema import (
|
||||
ATTENTION_COUPLING_REGIONAL_PROMPT_WEIGHT_DEFAULT,
|
||||
attention_coupling_ksampler_inputs,
|
||||
tiled_diffusion_inputs,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class KSamplerTiledAttentionCouplingV3(_ComfyNodeBase):
|
||||
"""Sample supported models with tiled regional attention."""
|
||||
|
||||
sampling_service_class: ClassVar[type[TiledAttentionCouplingSamplingService]] = (
|
||||
TiledAttentionCouplingSamplingService
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the tiled Attention Coupling sampler schema."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.KSamplerAttentionCouplingTiled",
|
||||
display_name="KSampler (Attention Coupling + Tiled Diffusion)",
|
||||
category="SimpleSyrup/Sampling",
|
||||
description=(
|
||||
"With ordinary conditioning and no masks, uses normal tiled "
|
||||
"diffusion without Attention Coupling preparation. With conditioning "
|
||||
"batches and masks, denoises large Anima and standard SD/SDXL "
|
||||
"latents in tiles through "
|
||||
"one shared model trajectory per tile batch while coupling global "
|
||||
"and masked regional cross-attention. The input MODEL may carry "
|
||||
"global LoRAs. Anima regions may carry independently scheduled "
|
||||
"regional LoRA stacks; inactive attention and LoRA work is pruned "
|
||||
"without changing quality. MultiDiffusion or Mixture of Diffusers "
|
||||
"fuses restored tile predictions. Standard SD/SDXL regional "
|
||||
"model-side hooks and unsupported Anima targets fail before sampling."
|
||||
),
|
||||
search_aliases=[
|
||||
"attention coupling tiled",
|
||||
"regional lora tiled",
|
||||
"anima tiled regional prompt",
|
||||
"sdxl tiled regional prompt",
|
||||
"multidiffusion regional lora",
|
||||
"mixture of diffusers regional lora",
|
||||
],
|
||||
inputs=[
|
||||
*attention_coupling_ksampler_inputs(
|
||||
_comfy_io,
|
||||
region_masks_optional=True,
|
||||
),
|
||||
*tiled_diffusion_inputs(_comfy_io),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Latent.Output(
|
||||
"latent",
|
||||
tooltip=tooltips.DENOISED_LATENT_OUTPUT,
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
model: Any,
|
||||
seed: int,
|
||||
steps: int,
|
||||
cfg: float,
|
||||
sampler_name: str,
|
||||
scheduler: str,
|
||||
positive: object,
|
||||
negative: object,
|
||||
latent_image: dict[str, Any],
|
||||
denoise: float = 1.0,
|
||||
diffusion_mode: str = "multidiffusion",
|
||||
latent_tile_width: int = 128,
|
||||
latent_tile_height: int = 128,
|
||||
latent_tile_overlap: int = 16,
|
||||
latent_tile_batch_size: int = 4,
|
||||
region_masks: object | None = None,
|
||||
regional_prompt_weight: float = (
|
||||
ATTENTION_COUPLING_REGIONAL_PROMPT_WEIGHT_DEFAULT
|
||||
),
|
||||
region_mask_feather: int = 0,
|
||||
) -> tuple[dict[str, Any]]:
|
||||
"""Delegate ordinary or regional tiled sampling to the routing service."""
|
||||
|
||||
output = cls.sampling_service_class().sample(
|
||||
diffusion_mode=diffusion_mode,
|
||||
model=model,
|
||||
seed=seed,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
sampler_name=sampler_name,
|
||||
scheduler=scheduler,
|
||||
positive=positive,
|
||||
negative=negative,
|
||||
region_masks=region_masks,
|
||||
regional_prompt_weight=regional_prompt_weight,
|
||||
region_mask_feather=region_mask_feather,
|
||||
latent_image=latent_image,
|
||||
denoise=denoise,
|
||||
latent_tile_width=latent_tile_width,
|
||||
latent_tile_height=latent_tile_height,
|
||||
latent_tile_overlap=latent_tile_overlap,
|
||||
latent_tile_batch_size=latent_tile_batch_size,
|
||||
preview_context=None,
|
||||
differential_diffusion=False,
|
||||
)
|
||||
return (output,)
|
||||
@@ -0,0 +1,124 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Native Comfy v3 node for selectable tiled diffusion sampling."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..nodes import tooltips
|
||||
from ..services.tiled_diffusion_sampling_service import TiledDiffusionSamplingService
|
||||
from .ksampler_schema import (
|
||||
ksampler_inputs,
|
||||
optional_regional_sampling_inputs,
|
||||
tiled_diffusion_inputs,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class KSamplerTiledDiffusionV3(_ComfyNodeBase):
|
||||
"""Sample latents with selectable tiled diffusion denoising."""
|
||||
|
||||
service_class: ClassVar[type[TiledDiffusionSamplingService]] = (
|
||||
TiledDiffusionSamplingService
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the native tiled diffusion KSampler schema."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.KSamplerTiledDiffusion",
|
||||
display_name="KSampler (Tiled Diffusion)",
|
||||
category="SimpleSyrup/Sampling",
|
||||
description="Denoises latents with selectable tiled diffusion sampling.",
|
||||
search_aliases=[
|
||||
"ksampler",
|
||||
"sampler",
|
||||
"tiled diffusion",
|
||||
"multidiffusion",
|
||||
"multi diffusion",
|
||||
"mixture of diffusers",
|
||||
],
|
||||
inputs=[
|
||||
*ksampler_inputs(_comfy_io, steps_default=20, cfg_default=8.0),
|
||||
*tiled_diffusion_inputs(_comfy_io),
|
||||
*optional_regional_sampling_inputs(
|
||||
_comfy_io,
|
||||
segs_tooltip=(
|
||||
"Optional image regions that guide irregular tile "
|
||||
"boundaries while preserving the configured overlap."
|
||||
),
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Latent.Output(
|
||||
None,
|
||||
tooltip=tooltips.DENOISED_LATENT_OUTPUT,
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
model: Any,
|
||||
seed: int,
|
||||
steps: int,
|
||||
cfg: float,
|
||||
sampler_name: str,
|
||||
scheduler: str,
|
||||
positive: Any,
|
||||
negative: Any,
|
||||
latent_image: dict[str, Any],
|
||||
denoise: float = 1.0,
|
||||
diffusion_mode: str = "multidiffusion",
|
||||
latent_tile_width: int = 128,
|
||||
latent_tile_height: int = 128,
|
||||
latent_tile_overlap: int = 16,
|
||||
latent_tile_batch_size: int = 4,
|
||||
segs: object | None = None,
|
||||
region_masks: object | None = None,
|
||||
regional_prompt_weight: float = 0.5,
|
||||
region_mask_feather: int = 0,
|
||||
) -> tuple[dict[str, Any]]:
|
||||
"""Delegate tiled diffusion sampling to its application service."""
|
||||
|
||||
output = cls.service_class().sample(
|
||||
diffusion_mode=diffusion_mode,
|
||||
model=model,
|
||||
seed=seed,
|
||||
steps=steps,
|
||||
cfg=cfg,
|
||||
sampler_name=sampler_name,
|
||||
scheduler=scheduler,
|
||||
positive=positive,
|
||||
negative=negative,
|
||||
latent_image=latent_image,
|
||||
denoise=denoise,
|
||||
latent_tile_width=latent_tile_width,
|
||||
latent_tile_height=latent_tile_height,
|
||||
latent_tile_overlap=latent_tile_overlap,
|
||||
latent_tile_batch_size=latent_tile_batch_size,
|
||||
preview_context=None,
|
||||
segs=segs,
|
||||
region_masks=region_masks,
|
||||
regional_prompt_weight=regional_prompt_weight,
|
||||
region_mask_feather=region_mask_feather,
|
||||
)
|
||||
return (output,)
|
||||
@@ -0,0 +1,67 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Expose internal stable identity labeling for regional LoRA hooks."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..runtime.regional_lora_hook_identity import label_regional_lora_hooks
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class LabelRegionalLoraHooksV3(_ComfyNodeBase):
|
||||
"""Attach explicit adapter identities to cloned schedule-bearing hooks."""
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the internal ordered identity boundary."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.LabelRegionalLoraHooks",
|
||||
display_name="Label Regional LoRA Hooks",
|
||||
category="SimpleSyrup/Internal",
|
||||
description=(
|
||||
"Labels scheduled regional LoRA hooks with stable adapter "
|
||||
"identities without changing their weights or keyframes."
|
||||
),
|
||||
inputs=[
|
||||
_comfy_io.Hooks.Input(
|
||||
"hooks",
|
||||
tooltip="Prompt Control hooks whose schedules remain unchanged.",
|
||||
),
|
||||
_comfy_io.String.Input(
|
||||
"adapter_identities_json",
|
||||
tooltip=(
|
||||
"Ordered JSON array with one stable identity per LoRA hook."
|
||||
),
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Hooks.Output(
|
||||
"hooks",
|
||||
tooltip="Cloned hooks carrying the supplied stable identities.",
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(cls, hooks: object, adapter_identities_json: str) -> tuple[object]:
|
||||
"""Return one labeled clone of the supplied HookGroup."""
|
||||
|
||||
return (label_regional_lora_hooks(hooks, adapter_identities_json),)
|
||||
@@ -25,7 +25,6 @@ from ..nodes.grounded_sam_model_info import GroundedSAMModelInfo
|
||||
from ..nodes.grounding_dino_model_loader import GroundingDINOModelLoader
|
||||
from ..nodes.image_resize_to_target import ResizeImageToTarget
|
||||
from ..nodes.ksampler_extras import KSamplerExtras
|
||||
from ..nodes.ksampler_tiled_diffusion import KSamplerTiledDiffusion
|
||||
from ..nodes.latent_diagnostics import LatentDiagnostics
|
||||
from ..nodes.layerstyle_sam_models_adapter import LayerStyleSAMModelsAdapter
|
||||
from ..nodes.load_ultralytics_model import LoadUltralyticsModel
|
||||
@@ -37,7 +36,9 @@ from ..nodes.prompt_segs_with_sam import PromptSEGSWithSAM
|
||||
from ..nodes.provenance_latent import SimpleVAEEncode, UpscaleLatentFromImage
|
||||
from ..nodes.sam_model_loader import SAMModelLoader
|
||||
from ..nodes.seed import Seed
|
||||
from ..nodes.segs_from_sam_output import SEGSFromSAMOutput
|
||||
from ..nodes.simple_load_anima import SimpleLoadAnima
|
||||
from ..nodes.simple_preview_segs import SimplePreviewSEGS
|
||||
from ..nodes.vitmatte_model_loader import ViTMatteModelLoader
|
||||
|
||||
if TYPE_CHECKING:
|
||||
@@ -148,14 +149,6 @@ class KSamplerExtrasV3(LegacyNodeV3Adapter):
|
||||
DISPLAY_NAME = "KSampler (Extras)"
|
||||
|
||||
|
||||
class KSamplerTiledDiffusionV3(LegacyNodeV3Adapter):
|
||||
"""Expose KSampler Tiled Diffusion through Comfy v3 only."""
|
||||
|
||||
LEGACY_NODE_CLASS = KSamplerTiledDiffusion
|
||||
NODE_ID = "SimpleSyrup.KSamplerTiledDiffusion"
|
||||
DISPLAY_NAME = "KSampler (Tiled Diffusion)"
|
||||
|
||||
|
||||
class LayerStyleSAMModelsAdapterV3(LegacyNodeV3Adapter):
|
||||
"""Expose LayerStyle SAM Models Adapter through Comfy v3 only."""
|
||||
|
||||
@@ -254,6 +247,22 @@ class SAMModelLoaderV3(LegacyNodeV3Adapter):
|
||||
DISPLAY_NAME = "SAM Model Loader"
|
||||
|
||||
|
||||
class SEGSFromSAMOutputV3(LegacyNodeV3Adapter):
|
||||
"""Expose automatic SAM-to-SEGS generation through Comfy v3 only."""
|
||||
|
||||
LEGACY_NODE_CLASS = SEGSFromSAMOutput
|
||||
NODE_ID = "SimpleSyrup.SEGSFromSAMOutput"
|
||||
DISPLAY_NAME = "SEGS from SAM Output"
|
||||
|
||||
|
||||
class SimplePreviewSEGSV3(LegacyNodeV3Adapter):
|
||||
"""Expose the interactive Simple Preview SEGS node through Comfy v3 only."""
|
||||
|
||||
LEGACY_NODE_CLASS = SimplePreviewSEGS
|
||||
NODE_ID = "SimpleSyrup.SimplePreviewSEGS"
|
||||
DISPLAY_NAME = "Simple Preview SEGS"
|
||||
|
||||
|
||||
class SeedV3(LegacyNodeV3Adapter):
|
||||
"""Expose Seed through Comfy v3 only."""
|
||||
|
||||
@@ -496,7 +505,6 @@ __all__ = [
|
||||
"GroundedSAMModelInfoV3",
|
||||
"GroundingDINOModelLoaderV3",
|
||||
"KSamplerExtrasV3",
|
||||
"KSamplerTiledDiffusionV3",
|
||||
"LatentDiagnosticsV3",
|
||||
"LayerStyleSAMModelsAdapterV3",
|
||||
"LoadUltralyticsModelV3",
|
||||
@@ -505,8 +513,10 @@ __all__ = [
|
||||
"PromptSEGSWithSAMV3",
|
||||
"ResizeImageToTargetV3",
|
||||
"SAMModelLoaderV3",
|
||||
"SEGSFromSAMOutputV3",
|
||||
"SeedV3",
|
||||
"SimpleLoadAnimaV3",
|
||||
"SimplePreviewSEGSV3",
|
||||
"SimpleVAEEncodeV3",
|
||||
"UpscaleLatentFromImageV3",
|
||||
"ViTMatteModelLoaderV3",
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Comfy v3 node for loading an ordered list of authored images."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..services.load_image_list_service import LoadImageListService
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_api: Any = None if TYPE_CHECKING else import_module("comfy_api.latest")
|
||||
_comfy_io: Any = None if TYPE_CHECKING else _comfy_api.io
|
||||
|
||||
|
||||
class LoadImageListV3(_ComfyNodeBase):
|
||||
"""Load ordered files as a native Comfy IMAGE execution list."""
|
||||
|
||||
service_class: ClassVar[type[LoadImageListService]] = LoadImageListService
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the persisted multi-upload and IMAGE list output."""
|
||||
|
||||
choices = list(cls.service_class().available_files())
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.LoadImageList",
|
||||
display_name="Load Image List",
|
||||
category="SimpleSyrup/Loaders",
|
||||
description=(
|
||||
"Loads an ordered image list without resizing or batching its items."
|
||||
),
|
||||
search_aliases=["load images", "image list", "reference images"],
|
||||
inputs=[
|
||||
_comfy_io.MultiCombo.Input(
|
||||
"image",
|
||||
options=choices,
|
||||
default=[],
|
||||
placeholder="Select one or more images",
|
||||
chip=True,
|
||||
tooltip=(
|
||||
"Select images in order; each file becomes one independent "
|
||||
"IMAGE list item."
|
||||
),
|
||||
extra_dict={
|
||||
"image_upload": True,
|
||||
"image_folder": "input",
|
||||
"allow_batch": True,
|
||||
},
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Image.Output(
|
||||
"images",
|
||||
tooltip=(
|
||||
"Ordered IMAGE list with one independently sized item per file."
|
||||
),
|
||||
is_output_list=True,
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(cls, image: str | list[str]) -> Any:
|
||||
"""Return the selected images as a native Comfy execution list."""
|
||||
|
||||
return _comfy_io.NodeOutput(cls.service_class().load(image))
|
||||
|
||||
@classmethod
|
||||
def validate_inputs(cls, image: str | list[str]) -> bool | str:
|
||||
"""Validate widget values before image loading."""
|
||||
|
||||
try:
|
||||
cls.service_class().validate(image)
|
||||
except (TypeError, ValueError) as error:
|
||||
return str(error)
|
||||
return True
|
||||
|
||||
@classmethod
|
||||
def fingerprint_inputs(cls, image: str | list[str]) -> str:
|
||||
"""Fingerprint ordered filenames and file contents."""
|
||||
|
||||
return cls.service_class().fingerprint(image)
|
||||
@@ -0,0 +1,105 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Comfy v3 node for loading one or many authored masks with native widgets."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..runtime.mask_file_loader import MASK_CHANNELS
|
||||
from ..services.load_mask_batch_service import LoadMaskBatchService
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_api: Any = None if TYPE_CHECKING else import_module("comfy_api.latest")
|
||||
_comfy_io: Any = None if TYPE_CHECKING else _comfy_api.io
|
||||
|
||||
|
||||
class LoadMaskBatchV3(_ComfyNodeBase):
|
||||
"""Load an ordered set of authored files as one Comfy MASK batch."""
|
||||
|
||||
service_class: ClassVar[type[LoadMaskBatchService]] = LoadMaskBatchService
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare a native image-upload combo that accepts one or many files."""
|
||||
|
||||
choices = list(cls.service_class().available_files())
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.LoadMaskBatch",
|
||||
display_name="Load Mask Batch",
|
||||
category="SimpleSyrup/Loaders",
|
||||
description=(
|
||||
"Loads one or many authored mask files in selection order as a "
|
||||
"single mask batch."
|
||||
),
|
||||
search_aliases=["load masks", "mask batch", "regional masks"],
|
||||
inputs=[
|
||||
_comfy_io.MultiCombo.Input(
|
||||
"image",
|
||||
options=choices,
|
||||
default=[],
|
||||
placeholder="Select one or more masks",
|
||||
chip=True,
|
||||
tooltip=(
|
||||
"Select one or more ordered mask files; each selected "
|
||||
"file becomes one regional mask."
|
||||
),
|
||||
extra_dict={
|
||||
"image_upload": True,
|
||||
"image_folder": "input",
|
||||
"allow_batch": True,
|
||||
},
|
||||
),
|
||||
_comfy_io.Combo.Input(
|
||||
"channel",
|
||||
options=list(MASK_CHANNELS),
|
||||
default="alpha",
|
||||
tooltip=(
|
||||
"Image channel read from every file. Missing alpha "
|
||||
"produces zero coverage at the source image dimensions."
|
||||
),
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Mask.Output(
|
||||
"mask",
|
||||
tooltip="Ordered BHW mask batch containing one mask per file.",
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(cls, image: str | list[str], channel: str) -> Any:
|
||||
"""Load the selected files as one same-sized mask batch."""
|
||||
|
||||
mask_batch = cls.service_class().load(image, channel)
|
||||
return _comfy_io.NodeOutput(mask_batch)
|
||||
|
||||
@classmethod
|
||||
def validate_inputs(cls, image: str | list[str], channel: str) -> bool | str:
|
||||
"""Validate native widget values before execution or filesystem reads."""
|
||||
|
||||
try:
|
||||
cls.service_class().validate(image, channel)
|
||||
except (TypeError, ValueError) as error:
|
||||
return str(error)
|
||||
return True
|
||||
|
||||
@classmethod
|
||||
def fingerprint_inputs(cls, image: str | list[str], channel: str) -> str:
|
||||
"""Fingerprint the ordered selected files and shared channel."""
|
||||
|
||||
return cls.service_class().fingerprint(image, channel)
|
||||
@@ -0,0 +1,72 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Comfy v3 internal node for regional LoRA CLIP preparation."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..runtime.regional_lora_hooks import prepare_regional_lora_clip
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class PrepareRegionalLoraHooksV3(_ComfyNodeBase):
|
||||
"""Prepare only the CLIP work required by regional LoRA hooks."""
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the internal hook conversion schema."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.PrepareRegionalLoraHooks",
|
||||
display_name="Prepare Regional LoRA Hooks",
|
||||
category="SimpleSyrup/Internal",
|
||||
description=(
|
||||
"Keeps model-only regional LoRAs away from the text encoder while "
|
||||
"preparing text-encoder patches when present."
|
||||
),
|
||||
inputs=[
|
||||
_comfy_io.Clip.Input(
|
||||
"clip",
|
||||
tooltip="CLIP model used to encode this regional prompt.",
|
||||
),
|
||||
_comfy_io.Hooks.Input(
|
||||
"hooks",
|
||||
tooltip=("Prompt-Control LoRA hooks to inspect and prepare."),
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Clip.Output(
|
||||
"clip",
|
||||
tooltip=(
|
||||
"Original CLIP for model-only LoRAs, or a hook-prepared "
|
||||
"CLIP when text-encoder weights are present."
|
||||
),
|
||||
),
|
||||
_comfy_io.Hooks.Output(
|
||||
"hooks",
|
||||
tooltip=("Regional LoRA hooks to attach after prompt encoding."),
|
||||
),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(cls, clip: Any, hooks: object) -> tuple[Any, object]:
|
||||
"""Return the appropriate encoding CLIP and unchanged model hooks."""
|
||||
|
||||
return prepare_regional_lora_clip(clip, hooks)
|
||||
@@ -49,8 +49,9 @@ class ScheduleAndEncodePromptsWithPromptControl(_ComfyNodeBase):
|
||||
enable_expand=True,
|
||||
category="SimpleSyrup/Conditioning",
|
||||
description=(
|
||||
"Schedules Prompt-Control LoRAs and encodes prompts, using [SEP] "
|
||||
"to create SimpleSyrup conditioning batches."
|
||||
"Schedules Prompt-Control LoRAs and encodes prompts. With [SEP] "
|
||||
"or [SEP|name], both sides are matched using global text for "
|
||||
"missing regions and share each segment's LoRA hooks."
|
||||
),
|
||||
inputs=[
|
||||
_comfy_io.Model.Input(
|
||||
@@ -84,8 +85,9 @@ class ScheduleAndEncodePromptsWithPromptControl(_ComfyNodeBase):
|
||||
multiline=False,
|
||||
default="",
|
||||
tooltip=(
|
||||
"Positive Prompt-Control text. [SEP] creates a "
|
||||
"conditioning batch for SimpleSyrup batch-aware nodes."
|
||||
"Positive Prompt-Control text; [SEP] or [SEP|name] creates "
|
||||
"ordered conditioning entries, and global text fills "
|
||||
"missing positive regions."
|
||||
),
|
||||
),
|
||||
_comfy_io.String.Input(
|
||||
@@ -93,8 +95,9 @@ class ScheduleAndEncodePromptsWithPromptControl(_ComfyNodeBase):
|
||||
multiline=False,
|
||||
default="",
|
||||
tooltip=(
|
||||
"Negative Prompt-Control text. [SEP] creates a "
|
||||
"conditioning batch for SimpleSyrup batch-aware nodes."
|
||||
"Negative Prompt-Control text; [SEP] or [SEP|name] creates "
|
||||
"ordered conditioning entries, and global text fills "
|
||||
"missing negative regions."
|
||||
),
|
||||
),
|
||||
],
|
||||
@@ -102,8 +105,8 @@ class ScheduleAndEncodePromptsWithPromptControl(_ComfyNodeBase):
|
||||
_comfy_io.Model.Output(
|
||||
"model",
|
||||
tooltip=(
|
||||
"Model after LoRA tags from positive and negative prompts "
|
||||
"are scheduled."
|
||||
"Model with single-prompt LoRAs applied globally; SEP-local "
|
||||
"LoRAs travel on their conditioning entries instead."
|
||||
),
|
||||
),
|
||||
MixedConditioningIO.Output(
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Expose seed variation as a Comfy v3 MODEL patch node."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from importlib import import_module
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..domain.seed_variation import (
|
||||
MAX_SEED,
|
||||
MAX_VARIATION_STRENGTH,
|
||||
MIN_SEED,
|
||||
MIN_VARIATION_STRENGTH,
|
||||
)
|
||||
from ..nodes import tooltips
|
||||
from ..services.seed_variation_model_service import SEED_VARIATION_MODEL_SERVICE
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = None if TYPE_CHECKING else import_module("comfy_api.latest").io
|
||||
|
||||
|
||||
class SeedVariationV3(_ComfyNodeBase):
|
||||
"""Derive a MODEL that varies sampler-provided initial noise."""
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the seed-variation MODEL patch contract."""
|
||||
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.SeedVariation",
|
||||
display_name="Seed Variation",
|
||||
category="SimpleSyrup/Sampling",
|
||||
description=(
|
||||
"Creates related generations by mixing sampler noise toward a "
|
||||
"second deterministic seed."
|
||||
),
|
||||
search_aliases=["variation seed", "subseed", "seed interpolation"],
|
||||
inputs=[
|
||||
_comfy_io.Model.Input(
|
||||
"model",
|
||||
tooltip=tooltips.SEED_VARIATION_MODEL_INPUT,
|
||||
),
|
||||
_comfy_io.Int.Input(
|
||||
"variation_seed",
|
||||
default=0,
|
||||
min=MIN_SEED,
|
||||
max=MAX_SEED,
|
||||
control_after_generate=True,
|
||||
tooltip=tooltips.VARIATION_SEED,
|
||||
),
|
||||
_comfy_io.Float.Input(
|
||||
"variation_strength",
|
||||
default=0.0,
|
||||
min=MIN_VARIATION_STRENGTH,
|
||||
max=MAX_VARIATION_STRENGTH,
|
||||
step=0.01,
|
||||
round=0.01,
|
||||
tooltip=tooltips.VARIATION_STRENGTH,
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Model.Output(
|
||||
"model",
|
||||
tooltip=tooltips.SEED_VARIATION_MODEL_OUTPUT,
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
model: object,
|
||||
variation_seed: int,
|
||||
variation_strength: float,
|
||||
) -> tuple[object]:
|
||||
"""Return the source or a MODEL carrying initial-noise variation."""
|
||||
|
||||
return (
|
||||
SEED_VARIATION_MODEL_SERVICE.prepare(
|
||||
model=model,
|
||||
variation_seed=variation_seed,
|
||||
variation_strength=variation_strength,
|
||||
),
|
||||
)
|
||||
@@ -0,0 +1,161 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Expose cohesive FLUX.1 model loading through Comfy's v3 API."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
from types import ModuleType
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..nodes import tooltips
|
||||
from ..runtime.diffusion_model_loader import DIFFUSION_WEIGHT_DTYPES
|
||||
from ..runtime.model_downloads import ComfyProgressReporter
|
||||
from ..runtime.text_encoder_loader import TEXT_ENCODER_DEVICES
|
||||
from ..runtime.vae_loader import vae_choices
|
||||
from ..services.flux_loader_components import AUTO_CHOICE
|
||||
from ..services.flux_loader_service import FluxLoaderService
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = importlib.import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = (
|
||||
None if TYPE_CHECKING else importlib.import_module("comfy_api.latest").io
|
||||
)
|
||||
|
||||
|
||||
class SimpleLoadFluxV3(_ComfyNodeBase):
|
||||
"""Load FLUX.1 diffusion, dual text encoders, and VAE as one node."""
|
||||
|
||||
_service = FluxLoaderService()
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the separate FLUX.1 loader schema."""
|
||||
|
||||
folder_paths = _folder_paths()
|
||||
text_encoder_choices = _choices_with_auto(
|
||||
list(folder_paths.get_filename_list("text_encoders"))
|
||||
)
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.SimpleLoadFlux",
|
||||
display_name="Simple Load FLUX",
|
||||
category="SimpleSyrup/Loaders",
|
||||
description=(
|
||||
"Loads FLUX.1 with CLIP-L, T5-XXL, and its VAE; automatic "
|
||||
"components are downloaded from checksum-pinned Hugging Face files."
|
||||
),
|
||||
search_aliases=["flux", "flux 1", "load flux"],
|
||||
inputs=[
|
||||
_comfy_io.Combo.Input(
|
||||
"diffusion_model",
|
||||
options=list(folder_paths.get_filename_list("diffusion_models")),
|
||||
tooltip=(
|
||||
"FLUX.1 diffusion model to load. This node never downloads "
|
||||
"the diffusion model."
|
||||
),
|
||||
),
|
||||
_comfy_io.Combo.Input(
|
||||
"diffusion_weight_dtype",
|
||||
options=list(DIFFUSION_WEIGHT_DTYPES),
|
||||
default="default",
|
||||
advanced=True,
|
||||
tooltip=(
|
||||
"Weight precision for the diffusion model; FP8 uses less "
|
||||
"memory but can slightly change results."
|
||||
),
|
||||
),
|
||||
_comfy_io.Combo.Input(
|
||||
"clip_l",
|
||||
options=text_encoder_choices,
|
||||
default=AUTO_CHOICE,
|
||||
advanced=True,
|
||||
tooltip=(
|
||||
"CLIP-L encoder for FLUX.1 prompts. Auto finds the exact "
|
||||
"file or downloads it while Comfy shows node progress."
|
||||
),
|
||||
),
|
||||
_comfy_io.Combo.Input(
|
||||
"t5_xxl",
|
||||
options=text_encoder_choices,
|
||||
default=AUTO_CHOICE,
|
||||
advanced=True,
|
||||
tooltip=(
|
||||
"T5-XXL encoder for FLUX.1 prompts. Auto may download the "
|
||||
"large FP16 file and reports progress in Comfy."
|
||||
),
|
||||
),
|
||||
_comfy_io.Combo.Input(
|
||||
"text_encoder_device",
|
||||
options=list(TEXT_ENCODER_DEVICES),
|
||||
default="default",
|
||||
advanced=True,
|
||||
tooltip=(
|
||||
"Device for both text encoders; CPU saves GPU memory but "
|
||||
"makes prompt encoding slower."
|
||||
),
|
||||
),
|
||||
_comfy_io.Combo.Input(
|
||||
"vae",
|
||||
options=_choices_with_auto(vae_choices(folder_paths)),
|
||||
default=AUTO_CHOICE,
|
||||
advanced=True,
|
||||
tooltip=(
|
||||
"VAE used to decode FLUX.1 latents. Auto finds or downloads "
|
||||
"the checksum-pinned FLUX VAE with visible node progress."
|
||||
),
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Model.Output("model", tooltip=tooltips.MODEL_OUTPUT),
|
||||
_comfy_io.Clip.Output("clip", tooltip=tooltips.CLIP_OUTPUT),
|
||||
_comfy_io.Vae.Output("vae", tooltip=tooltips.VAE_OUTPUT),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
diffusion_model: str,
|
||||
diffusion_weight_dtype: str,
|
||||
clip_l: str,
|
||||
t5_xxl: str,
|
||||
text_encoder_device: str,
|
||||
vae: str,
|
||||
) -> tuple[object, object, object]:
|
||||
"""Load and return FLUX.1 MODEL, CLIP, and VAE objects."""
|
||||
|
||||
return cls._service.load_models(
|
||||
diffusion_model=diffusion_model,
|
||||
diffusion_weight_dtype=diffusion_weight_dtype,
|
||||
clip_l=clip_l,
|
||||
t5_xxl=t5_xxl,
|
||||
text_encoder_device=text_encoder_device,
|
||||
vae=vae,
|
||||
progress=ComfyProgressReporter(),
|
||||
)
|
||||
|
||||
|
||||
def _choices_with_auto(choices: list[str]) -> list[str]:
|
||||
"""Return deduplicated choices with automatic selection first."""
|
||||
|
||||
return [AUTO_CHOICE, *(choice for choice in choices if choice != AUTO_CHOICE)]
|
||||
|
||||
|
||||
def _folder_paths() -> ModuleType:
|
||||
"""Import ComfyUI folder paths lazily for schema declaration."""
|
||||
|
||||
module: Any = importlib.import_module("folder_paths")
|
||||
if not isinstance(module, ModuleType):
|
||||
raise TypeError("folder_paths import did not return a module.")
|
||||
return module
|
||||
@@ -0,0 +1,150 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Expose cohesive FLUX.2 model loading through Comfy's v3 API."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
from types import ModuleType
|
||||
from typing import TYPE_CHECKING, Any, ClassVar
|
||||
|
||||
from ..nodes import tooltips
|
||||
from ..runtime.diffusion_model_loader import DIFFUSION_WEIGHT_DTYPES
|
||||
from ..runtime.model_downloads import ComfyProgressReporter
|
||||
from ..runtime.text_encoder_loader import TEXT_ENCODER_DEVICES
|
||||
from ..runtime.vae_loader import vae_choices
|
||||
from ..services.flux2_loader_service import Flux2LoaderService
|
||||
from ..services.flux_loader_components import AUTO_CHOICE
|
||||
|
||||
if TYPE_CHECKING:
|
||||
|
||||
class _ComfyNodeBase:
|
||||
"""Type-checking base for Comfy v3 nodes."""
|
||||
|
||||
RETURN_TYPES: ClassVar[list[str]]
|
||||
RETURN_NAMES: ClassVar[list[str]]
|
||||
|
||||
else:
|
||||
_ComfyNodeBase = importlib.import_module("comfy_api.latest").io.ComfyNode
|
||||
|
||||
_comfy_io: Any = (
|
||||
None if TYPE_CHECKING else importlib.import_module("comfy_api.latest").io
|
||||
)
|
||||
|
||||
|
||||
class SimpleLoadFlux2V3(_ComfyNodeBase):
|
||||
"""Load FLUX.2 diffusion, one profile-specific encoder, and VAE."""
|
||||
|
||||
_service = Flux2LoaderService()
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> Any:
|
||||
"""Declare the separate FLUX.2 loader schema."""
|
||||
|
||||
folder_paths = _folder_paths()
|
||||
return _comfy_io.Schema(
|
||||
node_id="SimpleSyrup.SimpleLoadFlux2",
|
||||
display_name="Simple Load FLUX.2",
|
||||
category="SimpleSyrup/Loaders",
|
||||
description=(
|
||||
"Loads FLUX.2 with its structurally matched text encoder and common "
|
||||
"VAE; automatic components download checksum-pinned Hugging Face "
|
||||
"files when needed."
|
||||
),
|
||||
search_aliases=["flux 2", "flux2", "klein", "load flux 2"],
|
||||
inputs=[
|
||||
_comfy_io.Combo.Input(
|
||||
"diffusion_model",
|
||||
options=list(folder_paths.get_filename_list("diffusion_models")),
|
||||
tooltip=(
|
||||
"FLUX.2 diffusion model to load. This node never downloads "
|
||||
"the diffusion model."
|
||||
),
|
||||
),
|
||||
_comfy_io.Combo.Input(
|
||||
"diffusion_weight_dtype",
|
||||
options=list(DIFFUSION_WEIGHT_DTYPES),
|
||||
default="default",
|
||||
advanced=True,
|
||||
tooltip=(
|
||||
"Weight precision for the diffusion model; FP8 uses less "
|
||||
"memory but can slightly change results."
|
||||
),
|
||||
),
|
||||
_comfy_io.Combo.Input(
|
||||
"text_encoder",
|
||||
options=_choices_with_auto(
|
||||
list(folder_paths.get_filename_list("text_encoders"))
|
||||
),
|
||||
default=AUTO_CHOICE,
|
||||
advanced=True,
|
||||
tooltip=(
|
||||
"Text encoder for FLUX.2. Auto detects dev, Klein 4B, or "
|
||||
"Klein 9B/KV from the loaded model and reports downloads "
|
||||
"through Comfy node progress."
|
||||
),
|
||||
),
|
||||
_comfy_io.Combo.Input(
|
||||
"text_encoder_device",
|
||||
options=list(TEXT_ENCODER_DEVICES),
|
||||
default="default",
|
||||
advanced=True,
|
||||
tooltip=(
|
||||
"Device for the text encoder; CPU saves GPU memory but makes "
|
||||
"prompt encoding slower."
|
||||
),
|
||||
),
|
||||
_comfy_io.Combo.Input(
|
||||
"vae",
|
||||
options=_choices_with_auto(vae_choices(folder_paths)),
|
||||
default=AUTO_CHOICE,
|
||||
advanced=True,
|
||||
tooltip=(
|
||||
"VAE used to decode FLUX.2 latents. Auto finds or downloads "
|
||||
"the common checksum-pinned VAE with visible node progress."
|
||||
),
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
_comfy_io.Model.Output("model", tooltip=tooltips.MODEL_OUTPUT),
|
||||
_comfy_io.Clip.Output("clip", tooltip=tooltips.CLIP_OUTPUT),
|
||||
_comfy_io.Vae.Output("vae", tooltip=tooltips.VAE_OUTPUT),
|
||||
],
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def execute(
|
||||
cls,
|
||||
diffusion_model: str,
|
||||
diffusion_weight_dtype: str,
|
||||
text_encoder: str,
|
||||
text_encoder_device: str,
|
||||
vae: str,
|
||||
) -> tuple[object, object, object]:
|
||||
"""Load and return FLUX.2 MODEL, CLIP, and VAE objects."""
|
||||
|
||||
return cls._service.load_models(
|
||||
diffusion_model=diffusion_model,
|
||||
diffusion_weight_dtype=diffusion_weight_dtype,
|
||||
text_encoder=text_encoder,
|
||||
text_encoder_device=text_encoder_device,
|
||||
vae=vae,
|
||||
progress=ComfyProgressReporter(),
|
||||
)
|
||||
|
||||
|
||||
def _choices_with_auto(choices: list[str]) -> list[str]:
|
||||
"""Return deduplicated choices with automatic selection first."""
|
||||
|
||||
return [AUTO_CHOICE, *(choice for choice in choices if choice != AUTO_CHOICE)]
|
||||
|
||||
|
||||
def _folder_paths() -> ModuleType:
|
||||
"""Import ComfyUI folder paths lazily for schema declaration."""
|
||||
|
||||
module: Any = importlib.import_module("folder_paths")
|
||||
if not isinstance(module, ModuleType):
|
||||
raise TypeError("folder_paths import did not return a module.")
|
||||
return module
|
||||
@@ -0,0 +1,39 @@
|
||||
# SimpleSyrup - workflow-focused ComfyUI extensions for image generation
|
||||
# Copyright (C) 2026 Artificial Sweetener and contributors
|
||||
# SPDX-License-Identifier: AGPL-3.0-or-later
|
||||
|
||||
"""Declare the trusted automatic artifacts used by Anima."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .auto_model_artifact import AutoModelArtifact
|
||||
|
||||
ANIMA_QWEN_TEXT_ENCODER = AutoModelArtifact(
|
||||
cache_id="anima_qwen_text_encoder",
|
||||
filename="qwen_3_06b_base.safetensors",
|
||||
folder_name="text_encoders",
|
||||
canonical_subfolder="qwen",
|
||||
source_url=(
|
||||
"https://huggingface.co/circlestone-labs/Anima/resolve/main/"
|
||||
"split_files/text_encoders/qwen_3_06b_base.safetensors"
|
||||
),
|
||||
source_repo="circlestone-labs/Anima",
|
||||
description="Anima Qwen3 0.6B text encoder",
|
||||
sha256="cd2a512003e2f9f3cd3c32a9c3573f820bb28c940f73c57b1ddaa983d9223eba",
|
||||
)
|
||||
|
||||
ANIMA_QWEN_VAE = AutoModelArtifact(
|
||||
cache_id="anima_qwen_vae",
|
||||
filename="qwen_image_vae.safetensors",
|
||||
folder_name="vae",
|
||||
canonical_subfolder="qwen",
|
||||
source_url=(
|
||||
"https://huggingface.co/circlestone-labs/Anima/resolve/main/"
|
||||
"split_files/vae/qwen_image_vae.safetensors"
|
||||
),
|
||||
source_repo="circlestone-labs/Anima",
|
||||
description="Anima Qwen Image VAE",
|
||||
sha256="a70580f0213e67967ee9c95f05bb400e8fb08307e017a924bf3441223e023d1f",
|
||||
)
|
||||
|
||||
ANIMA_AUTO_ARTIFACTS = (ANIMA_QWEN_TEXT_ENCODER, ANIMA_QWEN_VAE)
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user