Migrate all nodes to the V3 API

Remove obsolete frontend extensions

.github/workflows/publish.yml

@@ -7,14 +7,18 @@ on:
     paths:
       - "pyproject.toml"
 
+permissions:
+  issues: write
+
 jobs:
   publish-node:
     name: Publish Custom Node to registry
     runs-on: ubuntu-latest
+    if: ${{ github.repository_owner == 'Kosinkadink' }}
     steps:
       - name: Check out code
         uses: actions/checkout@v4
       - name: Publish Custom Node
-        uses: Comfy-Org/publish-node-action@main
+        uses: Comfy-Org/publish-node-action@v1
         with:
           personal_access_token: ${{ secrets.REGISTRY_ACCESS_TOKEN }} ## Add your own personal access token to your Github Repository secrets and reference it here.

AGENTS.md

@@ -0,0 +1,272 @@
+# Advanced-ControlNet Contributor Guide
+
+This repository is expected to receive substantial AI-authored code. Treat this
+file as the implementation and verification contract for all changes, especially
+ports of new ControlNet families from ComfyUI.
+
+## Engineering Rules
+
+- Read the relevant ComfyUI implementation and this repository's equivalent
+  control path before editing. Do not design from a model card alone.
+- Make the smallest change that preserves vanilla ComfyUI behavior and adds the
+  established Advanced-ControlNet capabilities.
+- Reuse ComfyUI model classes, patchers, ops, model management, and loaders when
+  possible. Do not maintain a forked copy of core model code without a concrete
+  need.
+- Preserve existing node IDs, inputs, outputs, checkpoint locations, and saved
+  workflow compatibility. New node IDs generally use the `ACN_` prefix; keep
+  existing unprefixed IDs and the aliases in `nodes_deprecated.py` working.
+- Do not add dependencies unless the model cannot be supported with ComfyUI,
+  PyTorch, and the libraries already used by this repository.
+- Match the direct style of the surrounding file. Avoid one-use abstractions,
+  generic framework code, speculative fallbacks, and comments that restate the
+  code.
+- Use plain ASCII punctuation in code, comments, documentation, commit messages,
+  and PR descriptions.
+
+## Architecture Map
+
+- `adv_control/control.py`: standard ControlNet wrappers, conversion of vanilla
+  controls, checkpoint detection, and shared loader dispatch.
+- `adv_control/control_<family>.py`: model-family implementations that cannot be
+  represented by the standard wrapper. Keep family-specific math here.
+- `adv_control/utils.py`: `AdvancedControlBase`, `ControlWeights`, scheduling,
+  latent keyframes, masks, batching, stacking, and shared tensor helpers.
+- `adv_control/nodes_main.py`: standard loaders and Apply nodes.
+- `adv_control/nodes_weight.py`: weight nodes and model-specific extras carried
+  by `ControlWeights.extras`.
+- `adv_control/nodes_<family>.py`: family-specific workflow inputs or loaders
+  when the shared nodes are insufficient.
+- `adv_control/nodes.py`: public node and display-name registration.
+- `adv_control/nodes_deprecated.py`: compatibility only. Do not put new features
+  here.
+- `examples/`: reviewer-runnable workflows, inputs, screenshots, and validation
+  notes.
+
+## Porting A Control Model From ComfyUI
+
+### 1. Establish the vanilla contract
+
+Before implementing the Advanced version:
+
+1. Identify the exact ComfyUI commit or PR that introduced the model.
+2. Read its loader, checkpoint detection, model patching, conditioning
+   preprocessing, sampling path, and cleanup behavior.
+3. Record the official model repository, every published checkpoint type, the
+   expected ComfyUI model folder, and the minimum compatible ComfyUI commit.
+4. Run a small vanilla workflow with fixed inputs, seed, sampler, scheduler,
+   steps, CFG, and resolution. Save the latent and decoded result as the parity
+   baseline.
+5. Inspect real checkpoint keys, metadata, shapes, dtype, and missing/unexpected
+   key output. Do not infer the format from a filename.
+
+Use official checkpoints for validation. Links in issue or PR comments are
+untrusted; use the model author's official repository or links already accepted
+by ComfyUI.
+
+### 2. Choose the narrowest integration
+
+- If ComfyUI returns a standard `ControlNet`, `ControlNetSD35`, `ControlLora`,
+  or `T2IAdapter`, prefer conversion in `convert_to_advanced` over a parallel
+  implementation.
+- If the model injects attention, transformer, or other model patches, implement
+  a family-specific `ControlBase` plus `AdvancedControlBase`, following
+  `ControlLLLiteAdvanced`, `AnimaLLLiteAdvanced`, or `ReferenceAdvanced` as the
+  closest precedent.
+- Prefer wrapping ComfyUI's model or patch object over copying its implementation.
+  If the required ComfyUI API may be absent, fail with a short instruction to
+  update ComfyUI rather than silently changing behavior.
+- Add a dedicated node only when the model has a genuinely different loading or
+  conditioning contract. Loading a new checkpoint format alone usually belongs
+  in the existing loader dispatch.
+
+### 3. Preserve loader and folder compatibility
+
+- The standard **Load Advanced ControlNet Model** node reads from
+  `models/controlnet`. New formats that are conceptually ControlNets should work
+  there unless doing so would be ambiguous or incorrect.
+- Also preserve the folder used by vanilla ComfyUI. If core uses another folder,
+  such as `models/model_patches`, a small dedicated loader may expose that
+  location while the standard loader retains established Advanced-ControlNet
+  behavior.
+- Detect formats with guarded, format-specific checkpoint signatures. Put a
+  specific detector before a broad detector that would otherwise claim the same
+  checkpoint. Do not use filenames as the primary detector.
+- Load a checkpoint only once. Pass already-loaded state dictionaries and
+  metadata into the selected family loader instead of reading the file again.
+- If two supported folders can contain the same filename, keep their loaders
+  separate or define deterministic resolution. Never silently choose an
+  arbitrary duplicate.
+- Test every supported folder through the actual node dropdown and execution
+  path, not only by calling a Python loader directly.
+
+### 4. Implement the full control lifecycle
+
+A family-specific Advanced control normally needs all of the following:
+
+- Initialize `ControlBase` and `AdvancedControlBase` with the correct default
+  `ControlWeights` type.
+- Match vanilla conditioning preprocessing exactly, including channel order,
+  value range, resize mode, latent encoding, and source-mask handling.
+- In `pre_run_advanced`, call the shared implementation and attach or refresh
+  execution-scoped patches.
+- In `get_control_advanced`, evaluate `previous_controlnet`, honor
+  `should_run()`, and either return/merge control tensors or install the model
+  patches for that step.
+- Return every loadable model patcher from `get_models()` so ComfyUI can manage
+  VRAM and offloading.
+- Implement `copy()` using both ComfyUI's `copy_to()` and this repository's
+  `copy_to_advanced()`. Copies must not share mutable execution state that can
+  leak between conditioning branches or queued runs.
+- Clear prepared tensors, patch references, cached shapes, and other
+  execution-scoped state in `cleanup_advanced()`.
+- Use ComfyUI device, dtype, manual-cast, operations, and model-patcher APIs.
+  Do not hardcode CUDA, force float32, or move models manually when ComfyUI
+  already owns that lifecycle.
+- Preserve `previous_controlnet` behavior so same-family and mixed-family
+  controls can be stacked.
+
+## Advanced Feature Contract
+
+A port is not complete merely because default-strength generation works. Unless
+the model architecture makes a capability impossible, verify that it supports:
+
+- Apply-node strength and start/end percentage.
+- Timestep keyframes, including changing strength and inherited values.
+- Latent keyframes on a batch of at least two latents.
+- Apply-node effect masks and timestep-keyframe masks.
+- Default, universal/soft, and architecture-specific per-layer weights.
+- Weight overrides and model-specific weight extras.
+- Conditional/unconditional weighting when the selected weight node exposes it.
+- Stacking with another control, including correct `previous_controlnet` output.
+- Batched conditioning and sliding-context subset indexes where applicable.
+- Repeated execution, copying, cleanup, model offloading, and reload.
+
+Do not claim unsupported features in documentation. If an architecture cannot
+support a feature, document the reason and make incompatible weight types fail
+clearly through `compatible_weights`.
+
+### Masks and model-specific inputs
+
+- `mask_optional` on **Apply Advanced ControlNet** is always an effect mask. It
+  controls where this control influences generation.
+- A model's source mask, control-type selector, or other family-specific data is
+  not an effect mask. Do not overload `mask_optional` with a second meaning.
+- Do not add a model-specific input to the shared Apply node unless it is a
+  coherent capability needed by multiple model families.
+- Prefer a small family-specific extras node that stores auxiliary values in
+  `ControlWeights.extras`, then pass those weights through `weights_override`.
+  Define extras keys next to the model implementation rather than as unrelated
+  strings spread across nodes.
+- Validate required extras where they are first consumed and raise an actionable
+  error that names the exact nodes and connections needed to fix the workflow.
+- Apply effect masks at the actual injection representation. Attention-patch and
+  DiT controls may need token-space masks rather than the normal spatial control
+  tensor path.
+- Verify mask semantics with all-zero, all-one, and half-frame masks. All-zero
+  must equal no control and all-one must equal unmasked full control. Inspect the
+  multiplier at the injection site as well as the final image; global attention
+  can propagate influence outside directly controlled tokens.
+
+### Per-layer weights
+
+- Map custom weights to real architecture blocks in execution order. Confirm the
+  count from the loaded model, not from a model-card claim alone.
+- Default weights must reproduce vanilla output exactly.
+- Universal/soft weights must follow this repository's established progression
+  semantics. Implement a family-specific conversion only when the normal
+  `ControlWeights` layout does not represent the architecture.
+- Ordinary example workflows should use default weights. Do not connect an
+  advanced custom-weight node merely to demonstrate that it exists.
+
+## Required Validation
+
+Python import or compile checks are necessary but are not model validation. Use
+a real local ComfyUI installation, preferably managed by comfy-runner, with this
+repository linked as the custom node.
+
+### Vanilla parity
+
+For every published control type and materially different checkpoint format:
+
+1. Run vanilla ComfyUI and Advanced-ControlNet with identical model files,
+   conditioning, seed, sampler settings, and latent.
+2. Compare latent tensors before decode and decoded pixel arrays.
+3. Target maximum absolute latent difference `0.0` and identical pixels when
+   both paths implement the same math. If exact parity is impossible, explain
+   why and report a justified numerical tolerance plus image metrics.
+4. Confirm strength zero matches no control and strength one matches vanilla.
+5. Check logs for missing/unexpected keys, dtype/device errors, repeated model
+   loads, and cleanup failures.
+
+### Advanced behavior
+
+At minimum, execute focused workflows for:
+
+- A nontrivial start/end schedule or two timestep keyframes.
+- Batch size two with different latent-keyframe strengths.
+- All-zero, all-one, and half-frame effect masks.
+- Default weights and one nonuniform custom or soft-weight configuration.
+- Conditional/unconditional weighting when supported.
+- A stacked control path.
+- Missing required model-specific extras and the resulting readable error.
+- Both the vanilla model folder and any historical Advanced-ControlNet folder.
+- Re-queueing the same workflow to exercise copy and cleanup behavior.
+
+For model families with several control types, test every type. Do not assume
+that lineart, depth, pose, inpainting, union, or channel-count variants share the
+same conditioning contract.
+
+### Basic checks
+
+- Run `python -m compileall adv_control __init__.py` with the target ComfyUI
+  environment.
+- Parse every added workflow JSON.
+- Load each committed workflow in the real frontend, serialize it to an API
+  prompt, and execute that round-tripped prompt. This catches stale node IDs,
+  renamed inputs, invalid widgets, and missing model metadata.
+- Run `git diff --check`.
+
+There is currently no repository unit-test suite. Add focused tests when they
+can exercise pure detection, shape, mask, or scheduling logic without building
+a fake ComfyUI runtime. Do not add a large test framework solely for one port.
+
+## Examples and Review Evidence
+
+Every model-family port must be independently checkable by a reviewer:
+
+- Add one simple workflow for every public control type. Include required input
+  images or masks when licensing permits.
+- Keep ComfyUI's default node names. Use colored groups or regions to explain
+  branches; do not rename nodes, because reviewers need to identify their types.
+- Keep the normal workflows simple. Leave custom per-layer weights disconnected
+  unless a workflow specifically validates those advanced weights.
+- Include direct links to the official base model, encoder, VAE, and control
+  checkpoint repositories, plus the exact destination folder for each file.
+- Include a workflow screenshot and labeled output comparison in the PR. For
+  parity tests, show vanilla, Advanced-ControlNet, and an absolute-difference
+  result when practical.
+- Commit reusable workflows and small review images under `examples/<family>/`.
+  Do not commit model files, latent dumps, or large intermediate artifacts.
+- Document exact seeds/settings, expected numerical results, known limitations,
+  and reproduction steps in the example README and PR description.
+- If final-image interpretation is subtle, include the direct tensor-level
+  evidence needed to distinguish a real bug from model behavior.
+
+## Definition Of Done For A Model Port
+
+- [ ] The official checkpoint is detected without relying on its filename.
+- [ ] Vanilla and historical Advanced-ControlNet model folders are preserved.
+- [ ] Default output matches vanilla for every published control type.
+- [ ] Strength scheduling, keyframes, masks, weights, batching, and stacking are
+      tested or an architectural limitation is documented.
+- [ ] Effect masks are tested at the injection site and in decoded output.
+- [ ] Model-specific inputs use a narrow family boundary, not shared Apply-node
+      expansion.
+- [ ] `get_models`, `copy`, cleanup, dtype/device handling, and repeated runs are
+      verified.
+- [ ] Missing required inputs produce actionable errors.
+- [ ] Simple workflows, input assets, download links, screenshots, and exact
+      reproduction instructions are included.
+- [ ] Real frontend/API execution, compile checks, JSON parsing, and
+      `git diff --check` pass.

README.md

@@ -12,7 +12,10 @@ ControlNet preprocessors are available through [comfyui_controlnet_aux](https://
 - Replicate ***"ControlNet is more important"*** feature from sd-webui-controlnet extension via ***uncond_multiplier*** on ***Soft Weights***
   - uncond_multiplier=0.0 gives identical results of auto1111's feature, but values between 0.0 and 1.0 can be used without issue to granularly control the setting.
 - ControlNet, T2IAdapter, and ControlLoRA support for sliding context windows
-- ControlLLLite support (requires model_optional to be passed into and out of Apply Advanced ControlNet node)
+- ControlLLLite support, including Anima LLLite v2 and inpainting models
+- ControlNet++ support
+- CtrLoRA support
+  - Relevant models linked on [CtrLoRA github page](https://github.com/xyfJASON/ctrlora)
 - SparseCtrl support
 - SVD-ControlNet support
   - Stable Video Diffusion ControlNets trained by **CiaraRowles**: [Depth](https://huggingface.co/CiaraRowles/temporal-controlnet-depth-svd-v1/tree/main/controlnet), [Lineart](https://huggingface.co/CiaraRowles/temporal-controlnet-lineart-svd-v1/tree/main/controlnet)  
@@ -80,6 +83,12 @@ Loads a ControlNet model and converts it into an Advanced version that supports
 ### Outputs
 - 🟪***CONTROL_NET***: loaded Advanced ControlNet
 
+## Anima LLLite v2
+
+Place Anima LLLite v2 files in `ComfyUI/models/model_patches` and load them with **Load Anima LLLite Model**. The regular Advanced ControlNet loader also recognizes these models when they are placed in `ComfyUI/models/controlnet`.
+
+Use the loaded model with **Apply Advanced ControlNet**. For the 4-channel inpainting model, pass the source mask through **Anima LLLite Extras** into the `cn_extras` input of a weights node; `mask_optional` on the Apply node remains the Advanced-ControlNet effect mask. **ControlNet Custom Weights [Anima]** provides one weight for each of Anima's 28 transformer blocks. Timestep keyframes, latent keyframes, soft weights, CFG/unconditional weighting, effect masks, and stacked controls work the same as with other Advanced-ControlNet models.
+
 ## Timestep Keyframe
 ![image](https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet/assets/7365912/404f3cfe-5852-4eed-935b-37e32493d1b5)
 

__init__.py

@@ -1,3 +1,10 @@
-from .adv_control.nodes import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
+from .adv_control.nodes import AdvancedControlNetExtension
+from .adv_control.dinklink import init_dinklink
+from .adv_control.sampling import prepare_dinklink_acn_wrapper
 
-__all__ = ['NODE_CLASS_MAPPINGS', 'NODE_DISPLAY_NAME_MAPPINGS']
+init_dinklink()
+prepare_dinklink_acn_wrapper()
+
+
+async def comfy_entrypoint() -> AdvancedControlNetExtension:
+    return AdvancedControlNetExtension()

adv_control/control.py

@@ -3,40 +3,53 @@ from torch import Tensor
 import torch
 import os
 
+import comfy.model_base
 import comfy.ops
 import comfy.utils
 import comfy.model_management
 import comfy.model_detection
 import comfy.controlnet as comfy_cn
-from comfy.controlnet import ControlBase, ControlNet, ControlLora, T2IAdapter
+from comfy.controlnet import ControlBase, ControlNet, ControlNetSD35, ControlLora, T2IAdapter, StrengthType
 from comfy.model_patcher import ModelPatcher
 
-from .control_sparsectrl import SparseModelPatcher, SparseControlNet, SparseCtrlMotionWrapper, SparseMethod, SparseSettings, SparseSpreadMethod, PreprocSparseRGBWrapper
+from .control_sparsectrl import SparseControlNet, SparseSettings, SparseConst, InterfaceAnimateDiffModel, create_sparse_modelpatcher, load_sparsectrl_motionmodel
-from .control_lllite import LLLiteModule, LLLitePatch
+from .control_lllite import LLLiteModule, LLLitePatch, load_anima_lllite, load_controllllite
 from .control_svd import svd_unet_config_from_diffusers_unet, SVDControlNet, svd_unet_to_diffusers
-from .utils import (AdvancedControlBase, TimestepKeyframeGroup, LatentKeyframeGroup, ControlWeightType, ControlWeights, WeightTypeException,
+from .utils import (AdvancedControlBase, TimestepKeyframeGroup, LatentKeyframeGroup, AbstractPreprocWrapper, ControlWeightType, ControlWeights, WeightTypeException, Extras,
-                    manual_cast_clean_groupnorm, disable_weight_init_clean_groupnorm, prepare_mask_batch, get_properly_arranged_t2i_weights, load_torch_file_with_dict_factory,
+                    manual_cast_clean_groupnorm, disable_weight_init_clean_groupnorm, WrapperConsts, prepare_mask_batch, get_properly_arranged_t2i_weights, load_torch_file_with_dict_factory,
-                    broadcast_image_to_extend, extend_to_batch_size)
+                    broadcast_image_to_extend, extend_to_batch_size, ORIG_PREVIOUS_CONTROLNET, CONTROL_INIT_BY_ACN)
 from .logger import logger
 
 
 class ControlNetAdvanced(ControlNet, AdvancedControlBase):
-    def __init__(self, control_model, timestep_keyframes: TimestepKeyframeGroup, global_average_pooling=False, device=None, load_device=None, manual_cast_dtype=None):
+    def __init__(self, control_model, timestep_keyframes: TimestepKeyframeGroup, global_average_pooling=False, compression_ratio=8, latent_format=None, load_device=None, manual_cast_dtype=None,
-        super().__init__(control_model=control_model, global_average_pooling=global_average_pooling, device=device, load_device=load_device, manual_cast_dtype=manual_cast_dtype)
+                 extra_conds=["y"], strength_type=StrengthType.CONSTANT, concat_mask=False, preprocess_image=lambda a: a):
-        AdvancedControlBase.__init__(self, super(), timestep_keyframes=timestep_keyframes, weights_default=ControlWeights.controlnet())
+        super().__init__(control_model=control_model, global_average_pooling=global_average_pooling, compression_ratio=compression_ratio, latent_format=latent_format, load_device=load_device, manual_cast_dtype=manual_cast_dtype,
+                         extra_conds=extra_conds, strength_type=strength_type, concat_mask=concat_mask, preprocess_image=preprocess_image)
+        AdvancedControlBase.__init__(self, super(type(self), self), timestep_keyframes=timestep_keyframes, weights_default=ControlWeights.controlnet())
+        self.is_flux = False
+        self.x_noisy_shape = None
 
     def get_universal_weights(self) -> ControlWeights:
-        raw_weights = [(self.weights.base_multiplier ** float(12 - i)) for i in range(13)]
+        def cn_weights_func(idx: int, control: dict[str, list[Tensor]], key: str):
-        return self.weights.copy_with_new_weights(raw_weights)
+            if key == "middle":
+                return 1.0 * self.weights.extras.get(Extras.MIDDLE_MULT, 1.0)
+            c_len = len(control[key])
+            raw_weights = [(self.weights.base_multiplier ** float((c_len) - i)) for i in range(c_len+1)]
+            raw_weights = raw_weights[:-1]
+            if key == "input":
+                raw_weights.reverse()
+            return raw_weights[idx]
+        return self.weights.copy_with_new_weights(new_weight_func=cn_weights_func)
 
-    def get_control_advanced(self, x_noisy, t, cond, batched_number):
+    def get_control_advanced(self, x_noisy, t, cond, batched_number, transformer_options):
         # perform special version of get_control that supports sliding context and masks
-        return self.sliding_get_control(x_noisy, t, cond, batched_number)
+        return self.sliding_get_control(x_noisy, t, cond, batched_number, transformer_options)
 
-    def sliding_get_control(self, x_noisy: Tensor, t, cond, batched_number):
+    def sliding_get_control(self, x_noisy: Tensor, t, cond, batched_number, transformer_options):
         control_prev = None
         if self.previous_controlnet is not None:
-            control_prev = self.previous_controlnet.get_control(x_noisy, t, cond, batched_number)
+            control_prev = self.previous_controlnet.get_control(x_noisy, t, cond, batched_number, transformer_options)
 
         if self.timestep_range is not None:
             if t[0] > self.timestep_range[0] or t[0] < self.timestep_range[1]:
@@ -49,21 +62,42 @@ class ControlNetAdvanced(ControlNet, AdvancedControlBase):
         if self.manual_cast_dtype is not None:
             dtype = self.manual_cast_dtype
 
-        output_dtype = x_noisy.dtype
         # make cond_hint appropriate dimensions
         # TODO: change this to not require cond_hint upscaling every step when self.sub_idxs are present
-        if self.sub_idxs is not None or self.cond_hint is None or x_noisy.shape[2] * 8 != self.cond_hint.shape[2] or x_noisy.shape[3] * 8 != self.cond_hint.shape[3]:
+        if self.sub_idxs is not None or self.cond_hint is None or x_noisy.shape[2] * self.real_compression_ratio != self.cond_hint.shape[2] or x_noisy.shape[3] * self.real_compression_ratio != self.cond_hint.shape[3]:
             if self.cond_hint is not None:
                 del self.cond_hint
             self.cond_hint = None
+            self.real_compression_ratio = self.compression_ratio
+            compression_ratio = self.compression_ratio
+            if self.vae is not None and self.mult_by_ratio_when_vae:
+                compression_ratio *= self.vae.downscale_ratio
             # if self.cond_hint_original length greater or equal to real latent count, subdivide it before scaling
             if self.sub_idxs is not None:
                 actual_cond_hint_orig = self.cond_hint_original
                 if self.cond_hint_original.size(0) < self.full_latent_length:
                     actual_cond_hint_orig = extend_to_batch_size(tensor=actual_cond_hint_orig, batch_size=self.full_latent_length)
-                self.cond_hint = comfy.utils.common_upscale(actual_cond_hint_orig[self.sub_idxs], x_noisy.shape[3] * 8, x_noisy.shape[2] * 8, 'nearest-exact', "center").to(dtype).to(self.device)
+                self.cond_hint = comfy.utils.common_upscale(actual_cond_hint_orig[self.sub_idxs], x_noisy.shape[3] * compression_ratio, x_noisy.shape[2] * compression_ratio, self.upscale_algorithm, "center")
             else:
-                self.cond_hint = comfy.utils.common_upscale(self.cond_hint_original, x_noisy.shape[3] * 8, x_noisy.shape[2] * 8, 'nearest-exact', "center").to(dtype).to(self.device)
+                self.cond_hint = comfy.utils.common_upscale(self.cond_hint_original, x_noisy.shape[3] * compression_ratio, x_noisy.shape[2] * compression_ratio, self.upscale_algorithm, "center")
+            self.cond_hint = self.preprocess_image(self.cond_hint)
+            if self.vae is not None:
+                loaded_models = comfy.model_management.loaded_models(only_currently_used=True)
+                self.cond_hint = self.vae.encode(self.cond_hint.movedim(1, -1))
+                comfy.model_management.load_models_gpu(loaded_models)
+                if not self.mult_by_ratio_when_vae:
+                    self.real_compression_ratio = 1
+            if self.latent_format is not None:
+                self.cond_hint = self.latent_format.process_in(self.cond_hint)
+            if len(self.extra_concat_orig) > 0:
+                to_concat = []
+                for c in self.extra_concat_orig:
+                    c = c.to(self.cond_hint.device)
+                    c = comfy.utils.common_upscale(c, self.cond_hint.shape[3], self.cond_hint.shape[2], self.upscale_algorithm, "center")
+                    to_concat.append(comfy.utils.repeat_to_batch_size(c, self.cond_hint.shape[0]))
+                self.cond_hint = torch.cat([self.cond_hint] + to_concat, dim=1)
+
+            self.cond_hint = self.cond_hint.to(device=x_noisy.device, dtype=dtype)
         if x_noisy.shape[0] != self.cond_hint.shape[0]:
             self.cond_hint = broadcast_image_to_extend(self.cond_hint, x_noisy.shape[0], batched_number)
 
@@ -71,28 +105,64 @@ class ControlNetAdvanced(ControlNet, AdvancedControlBase):
         self.prepare_mask_cond_hint(x_noisy=x_noisy, t=t, cond=cond, batched_number=batched_number, dtype=dtype)
 
         context = cond.get('crossattn_controlnet', cond['c_crossattn'])
-        # uses 'y' in new ComfyUI update
+        extra = self.extra_args.copy()
-        y = cond.get('y', None)
+        for c in self.extra_conds:
-        if y is None: # TODO: remove this in the future since no longer used by newest ComfyUI
+            temp = cond.get(c, None)
-            y = cond.get('c_adm', None)
+            if temp is not None:
-        if y is not None:
+                extra[c] = comfy.model_base.convert_tensor(temp, dtype, x_noisy.device)
-            y = y.to(dtype)
+
         timestep = self.model_sampling_current.timestep(t)
         x_noisy = self.model_sampling_current.calculate_input(t, x_noisy)
+        self.x_noisy_shape = x_noisy.shape
 
-        control = self.control_model(x=x_noisy.to(dtype), hint=self.cond_hint, timesteps=timestep.float(), context=context.to(dtype), y=y)
+        control = self.control_model(x=x_noisy.to(dtype), hint=self.cond_hint, timesteps=timestep.to(dtype), context=comfy.model_management.cast_to_device(context, x_noisy.device, dtype), **extra)
-        return self.control_merge(None, control, control_prev, output_dtype)
+        return self.control_merge(control, control_prev, output_dtype=None)
 
-    def copy(self):
+    def pre_run_advanced(self, *args, **kwargs):
-        c = ControlNetAdvanced(self.control_model, self.timestep_keyframes, global_average_pooling=self.global_average_pooling, load_device=self.load_device, manual_cast_dtype=self.manual_cast_dtype)
+        self.is_flux = "Flux" in str(type(self.control_model).__name__)
+        return super().pre_run_advanced(*args, **kwargs)
+
+    def apply_advanced_strengths_and_masks(self, x: Tensor, batched_number: int, flux_shape=None):
+        if self.is_flux:
+            flux_shape = self.x_noisy_shape
+        return super().apply_advanced_strengths_and_masks(x, batched_number, flux_shape)
+
+    def copy(self, subtype=None):
+        if subtype is None:
+            subtype = ControlNetAdvanced
+        c = subtype(self.control_model, self.timestep_keyframes, global_average_pooling=self.global_average_pooling, load_device=self.load_device, manual_cast_dtype=self.manual_cast_dtype)
+        c.control_model = self.control_model
+        c.control_model_wrapped = self.control_model_wrapped
         self.copy_to(c)
         self.copy_to_advanced(c)
         return c
     
+    def cleanup_advanced(self):
+        self.x_noisy_shape = None
+        return super().cleanup_advanced()
+
     @staticmethod
-    def from_vanilla(v: ControlNet, timestep_keyframe: TimestepKeyframeGroup=None) -> 'ControlNetAdvanced':
+    def from_vanilla(v: ControlNet, timestep_keyframe: TimestepKeyframeGroup=None, subtype=None) -> 'ControlNetAdvanced':
-        return ControlNetAdvanced(control_model=v.control_model, timestep_keyframes=timestep_keyframe,
+        if subtype is None:
-                                  global_average_pooling=v.global_average_pooling, device=v.device, load_device=v.load_device, manual_cast_dtype=v.manual_cast_dtype)
+            subtype = ControlNetAdvanced
+        to_return = subtype(control_model=v.control_model, timestep_keyframes=timestep_keyframe,
+                            global_average_pooling=v.global_average_pooling, compression_ratio=v.compression_ratio, latent_format=v.latent_format, load_device=v.load_device,
+                            manual_cast_dtype=v.manual_cast_dtype, extra_conds=v.extra_conds, strength_type=v.strength_type, concat_mask=v.concat_mask, preprocess_image=v.preprocess_image)
+        v.copy_to(to_return)
+        to_return.control_model_wrapped = v.control_model_wrapped.clone() # needed to avoid breaking memory management system (parent tracking)
+        return to_return
+
+
+class ControlNetSD35Advanced(ControlNetSD35, ControlNetAdvanced):
+    def __init__(self, *args, **kwargs):
+        ControlNetAdvanced.__init__(self, *args, **kwargs)
+
+    def copy(self):
+        return ControlNetAdvanced.copy(self, subtype=ControlNetSD35Advanced)
+    
+    @staticmethod
+    def from_vanilla(v: ControlNetSD35, timestep_keyframe=None):
+        return ControlNetAdvanced.from_vanilla(v, timestep_keyframe, subtype=ControlNetSD35Advanced)
 
 
 class T2IAdapterAdvanced(T2IAdapter, AdvancedControlBase):
@@ -100,37 +170,43 @@ class T2IAdapterAdvanced(T2IAdapter, AdvancedControlBase):
         super().__init__(t2i_model=t2i_model, channels_in=channels_in, compression_ratio=compression_ratio, upscale_algorithm=upscale_algorithm, device=device)
         AdvancedControlBase.__init__(self, super(), timestep_keyframes=timestep_keyframes, weights_default=ControlWeights.t2iadapter())
 
-    def control_merge_inject(self, control_input, control_output, control_prev, output_dtype):
+    def control_merge_inject(self, control: dict[str, list[Tensor]], control_prev, output_dtype):
-        # if has uncond multiplier, need to make sure control shapes are the same batch size as expected
+        # match batch_size
-        if self.weights.has_uncond_multiplier or self.weights.has_uncond_mask:
+        # TODO: make this more efficient by modifying the cached self.control_input val instead of doing this every step
-            if control_input is not None:
+        for key in control:
-                for i in range(len(control_input)):
+            control_current = control[key]
-                    x = control_input[i]
+            for i in range(len(control_current)):
-                    if x is not None:
+                x = control_current[i]
-                        if x.size(0) < self.batch_size:
+                if x is not None and x.size(0) == 1 and x.size(0) != self.batch_size:
-                            control_input[i] = x.repeat(self.batched_number, 1, 1, 1)[:self.batch_size]
+                    control_current[i] = x.repeat(self.batch_size, 1, 1, 1)[:self.batch_size]
-            if control_output is not None:
+        return AdvancedControlBase.control_merge_inject(self, control, control_prev, output_dtype)
-                for i in range(len(control_output)):
-                    x = control_output[i]
-                    if x is not None:
-                        if x.size(0) < self.batch_size:
-                            control_output[i] = x.repeat(self.batched_number, 1, 1, 1)[:self.batch_size]
-        return AdvancedControlBase.control_merge_inject(self, control_input, control_output, control_prev, output_dtype)
 
     def get_universal_weights(self) -> ControlWeights:
-        raw_weights = [(self.weights.base_multiplier ** float(7 - i)) for i in range(8)]
+        def t2i_weights_func(idx: int, control: dict[str, list[Tensor]], key: str):
-        raw_weights = [raw_weights[-8], raw_weights[-3], raw_weights[-2], raw_weights[-1]]
+            if key == "middle":
+                return 1.0 * self.weights.extras.get(Extras.MIDDLE_MULT, 1.0)
+            c_len = 8 #len(control[key])
+            raw_weights = [(self.weights.base_multiplier ** float((c_len-1) - i)) for i in range(c_len)]
+            raw_weights = [raw_weights[-c_len], raw_weights[-3], raw_weights[-2], raw_weights[-1]]
             raw_weights = get_properly_arranged_t2i_weights(raw_weights)
-        return self.weights.copy_with_new_weights(raw_weights)
+            if key == "input":
+                raw_weights.reverse()
+            return raw_weights[idx]
+        return self.weights.copy_with_new_weights(new_weight_func=t2i_weights_func)
 
-    def get_calc_pow(self, idx: int, layers: int) -> int:
+    def get_calc_pow(self, idx: int, control: dict[str, list[Tensor]], key: str) -> int:
+        if key == "middle":
+            return 0
         # match how T2IAdapterAdvanced deals with universal weights
-        indeces = [7 - i for i in range(8)]
+        c_len = 8 #len(control[key])
-        indeces = [indeces[-8], indeces[-3], indeces[-2], indeces[-1]]
+        indeces = [(c_len-1) - i for i in range(c_len)]
+        indeces = [indeces[-c_len], indeces[-3], indeces[-2], indeces[-1]]
         indeces = get_properly_arranged_t2i_weights(indeces)
+        if key == "input":
+            indeces.reverse()  # need to reverse to match recent ComfyUI changes
         return indeces[idx]
 
-    def get_control_advanced(self, x_noisy, t, cond, batched_number):
+    def get_control_advanced(self, x_noisy, t, cond, batched_number, transformer_options):
         try:
             # if sub indexes present, replace original hint with subsection
             if self.sub_idxs is not None:
@@ -144,7 +220,7 @@ class T2IAdapterAdvanced(T2IAdapter, AdvancedControlBase):
                 self.cond_hint_original = actual_cond_hint_orig[self.sub_idxs]
             # mask hints
             self.prepare_mask_cond_hint(x_noisy=x_noisy, t=t, cond=cond, batched_number=batched_number)
-            return super().get_control(x_noisy, t, cond, batched_number)
+            return super().get_control(x_noisy, t, cond, batched_number, transformer_options)
         finally:
             if self.sub_idxs is not None:
                 # replace original cond hint
@@ -163,13 +239,15 @@ class T2IAdapterAdvanced(T2IAdapter, AdvancedControlBase):
 
     @staticmethod
     def from_vanilla(v: T2IAdapter, timestep_keyframe: TimestepKeyframeGroup=None) -> 'T2IAdapterAdvanced':
-        return T2IAdapterAdvanced(t2i_model=v.t2i_model, timestep_keyframes=timestep_keyframe, channels_in=v.channels_in,
+        to_return = T2IAdapterAdvanced(t2i_model=v.t2i_model, timestep_keyframes=timestep_keyframe, channels_in=v.channels_in,
                                   compression_ratio=v.compression_ratio, upscale_algorithm=v.upscale_algorithm, device=v.device)
+        v.copy_to(to_return)
+        return to_return
 
 
 class ControlLoraAdvanced(ControlLora, AdvancedControlBase):
-    def __init__(self, control_weights, timestep_keyframes: TimestepKeyframeGroup, global_average_pooling=False, device=None):
+    def __init__(self, control_weights, timestep_keyframes: TimestepKeyframeGroup, global_average_pooling=False):
-        super().__init__(control_weights=control_weights, global_average_pooling=global_average_pooling, device=device)
+        super().__init__(control_weights=control_weights, global_average_pooling=global_average_pooling)
         AdvancedControlBase.__init__(self, super(), timestep_keyframes=timestep_keyframes, weights_default=ControlWeights.controllora())
         # use some functions from ControlNetAdvanced
         self.get_control_advanced = ControlNetAdvanced.get_control_advanced.__get__(self, type(self))
@@ -191,24 +269,26 @@ class ControlLoraAdvanced(ControlLora, AdvancedControlBase):
 
     @staticmethod
     def from_vanilla(v: ControlLora, timestep_keyframe: TimestepKeyframeGroup=None) -> 'ControlLoraAdvanced':
-        return ControlLoraAdvanced(control_weights=v.control_weights, timestep_keyframes=timestep_keyframe,
+        to_return = ControlLoraAdvanced(control_weights=v.control_weights, timestep_keyframes=timestep_keyframe,
-                                   global_average_pooling=v.global_average_pooling, device=v.device)
+                                   global_average_pooling=v.global_average_pooling)
+        v.copy_to(to_return)
+        return to_return
 
 
 class SVDControlNetAdvanced(ControlNetAdvanced):
-    def __init__(self, control_model: SVDControlNet, timestep_keyframes: TimestepKeyframeGroup, global_average_pooling=False, device=None, load_device=None, manual_cast_dtype=None):
+    def __init__(self, control_model: SVDControlNet, timestep_keyframes: TimestepKeyframeGroup, global_average_pooling=False, load_device=None, manual_cast_dtype=None):
-        super().__init__(control_model=control_model, timestep_keyframes=timestep_keyframes, global_average_pooling=global_average_pooling, device=device, load_device=load_device, manual_cast_dtype=manual_cast_dtype)
+        super().__init__(control_model=control_model, timestep_keyframes=timestep_keyframes, global_average_pooling=global_average_pooling, load_device=load_device, manual_cast_dtype=manual_cast_dtype)
 
-    def set_cond_hint(self, *args, **kwargs):
+    def set_cond_hint_inject(self, *args, **kwargs):
-        to_return = super().set_cond_hint(*args, **kwargs)
+        to_return = super().set_cond_hint_inject(*args, **kwargs)
         # cond hint for SVD-ControlNet needs to be scaled between (-1, 1) instead of (0, 1)
         self.cond_hint_original = self.cond_hint_original * 2.0 - 1.0
         return to_return
 
-    def get_control_advanced(self, x_noisy, t, cond, batched_number):
+    def get_control_advanced(self, x_noisy, t, cond, batched_number, transformer_options):
         control_prev = None
         if self.previous_controlnet is not None:
-            control_prev = self.previous_controlnet.get_control(x_noisy, t, cond, batched_number)
+            control_prev = self.previous_controlnet.get_control(x_noisy, t, cond, batched_number, transformer_options)
 
         if self.timestep_range is not None:
             if t[0] > self.timestep_range[0] or t[0] < self.timestep_range[1]:
@@ -233,9 +313,9 @@ class SVDControlNetAdvanced(ControlNetAdvanced):
                 actual_cond_hint_orig = self.cond_hint_original
                 if self.cond_hint_original.size(0) < self.full_latent_length:
                     actual_cond_hint_orig = extend_to_batch_size(tensor=actual_cond_hint_orig, batch_size=self.full_latent_length)
-                self.cond_hint = comfy.utils.common_upscale(actual_cond_hint_orig[self.sub_idxs], x_noisy.shape[3] * 8, x_noisy.shape[2] * 8, 'nearest-exact', "center").to(dtype).to(self.device)
+                self.cond_hint = comfy.utils.common_upscale(actual_cond_hint_orig[self.sub_idxs], x_noisy.shape[3] * 8, x_noisy.shape[2] * 8, 'nearest-exact', "center").to(dtype).to(x_noisy.device)
             else:
-                self.cond_hint = comfy.utils.common_upscale(self.cond_hint_original, x_noisy.shape[3] * 8, x_noisy.shape[2] * 8, 'nearest-exact', "center").to(dtype).to(self.device)
+                self.cond_hint = comfy.utils.common_upscale(self.cond_hint_original, x_noisy.shape[3] * 8, x_noisy.shape[2] * 8, 'nearest-exact', "center").to(dtype).to(x_noisy.device)
         if x_noisy.shape[0] != self.cond_hint.shape[0]:
             self.cond_hint = broadcast_image_to_extend(self.cond_hint, x_noisy.shape[0], batched_number)
 
@@ -246,15 +326,15 @@ class SVDControlNetAdvanced(ControlNetAdvanced):
         # uses 'y' in new ComfyUI update
         y = cond.get('y', None)
         if y is not None:
-            y = y.to(dtype)
+            y = comfy.model_base.convert_tensor(y, dtype, x_noisy.device)
         timestep = self.model_sampling_current.timestep(t)
         x_noisy = self.model_sampling_current.calculate_input(t, x_noisy)
         # concat c_concat if exists (should exist for SVD), doubling channels to 8
         if cond.get('c_concat', None) is not None:
             x_noisy = torch.cat([x_noisy] + [cond['c_concat']], dim=1)
 
-        control = self.control_model(x=x_noisy.to(dtype), hint=self.cond_hint, timesteps=timestep.float(), context=context.to(dtype), y=y, cond=cond)
+        control = self.control_model(x=x_noisy.to(dtype), hint=self.cond_hint, timesteps=timestep.float(), context=comfy.model_management.cast_to_device(context, x_noisy.device, dtype), y=y, cond=cond)
-        return self.control_merge(None, control, control_prev, output_dtype)
+        return self.control_merge(control, control_prev, output_dtype)
 
     def copy(self):
         c = SVDControlNetAdvanced(self.control_model, self.timestep_keyframes, global_average_pooling=self.global_average_pooling, load_device=self.load_device, manual_cast_dtype=self.manual_cast_dtype)
@@ -264,20 +344,37 @@ class SVDControlNetAdvanced(ControlNetAdvanced):
 
 
 class SparseCtrlAdvanced(ControlNetAdvanced):
-    def __init__(self, control_model, timestep_keyframes: TimestepKeyframeGroup, sparse_settings: SparseSettings=None, global_average_pooling=False, device=None, load_device=None, manual_cast_dtype=None):
+    def __init__(self, control_model: SparseControlNet, motion_model: InterfaceAnimateDiffModel,
-        super().__init__(control_model=control_model, timestep_keyframes=timestep_keyframes, global_average_pooling=global_average_pooling, device=device, load_device=load_device, manual_cast_dtype=manual_cast_dtype)
+                 timestep_keyframes: TimestepKeyframeGroup, sparse_settings: SparseSettings=None, global_average_pooling=False, load_device=None, manual_cast_dtype=None):
-        self.control_model_wrapped = SparseModelPatcher(self.control_model, load_device=load_device, offload_device=comfy.model_management.unet_offload_device())
+        super().__init__(control_model=None, timestep_keyframes=timestep_keyframes, global_average_pooling=global_average_pooling, load_device=load_device, manual_cast_dtype=manual_cast_dtype)
+        self.control_model = control_model
+        if control_model is not None:
+            self.control_model_wrapped: ModelPatcher = create_sparse_modelpatcher(self.control_model, motion_model, load_device=load_device, offload_device=comfy.model_management.unet_offload_device())
+            self.prepare_conditioning_info()
         self.add_compatible_weight(ControlWeightType.SPARSECTRL)
-        self.control_model: SparseControlNet = self.control_model  # does nothing except help with IDE hints
+        self.postpone_condhint_latents_check = True
         self.sparse_settings = sparse_settings if sparse_settings is not None else SparseSettings.default()
-        self.latent_format = None
+        self.model_latent_format = None  # latent format for active SD model, NOT controlnet
         self.preprocessed = False
     
-    def get_control_advanced(self, x_noisy: Tensor, t, cond, batched_number: int):
+    def prepare_conditioning_info(self):
+        if self.control_model.use_simplified_conditioning_embedding:
+            # TODO: allow vae_optional to be used instead of preprocessor
+            #self.require_vae = True
+            self.allow_condhint_latents = True
+
+    @property
+    def motion_model(self) -> InterfaceAnimateDiffModel:
+        motion_models = self.control_model_wrapped.get_additional_models_with_key(WrapperConsts.ACN)
+        if len(motion_models) == 0:
+            return None
+        return motion_models[0].model
+
+    def get_control_advanced(self, x_noisy: Tensor, t, cond, batched_number: int, transformer_options):
         # normal ControlNet stuff
         control_prev = None
         if self.previous_controlnet is not None:
-            control_prev = self.previous_controlnet.get_control(x_noisy, t, cond, batched_number)
+            control_prev = self.previous_controlnet.get_control(x_noisy, t, cond, batched_number, transformer_options)
 
         if self.timestep_range is not None:
             if t[0] > self.timestep_range[0] or t[0] < self.timestep_range[1]:
@@ -293,7 +390,8 @@ class SparseCtrlAdvanced(ControlNetAdvanced):
         # set actual input length on motion model
         actual_length = x_noisy.size(0)//batched_number
         full_length = actual_length if self.sub_idxs is None else self.full_latent_length
-        self.control_model.set_actual_length(actual_length=actual_length, full_length=full_length)
+        if self.motion_model is not None:
+            self.motion_model.set_video_length(video_length=actual_length, full_length=full_length)
         # prepare cond_hint, if needed
         dim_mult = 1 if self.control_model.use_simplified_conditioning_embedding else 8
         if self.sub_idxs is not None or self.cond_hint is None or x_noisy.shape[2]*dim_mult != self.cond_hint.shape[2] or x_noisy.shape[3]*dim_mult != self.cond_hint.shape[3]:
@@ -302,35 +400,50 @@ class SparseCtrlAdvanced(ControlNetAdvanced):
                 del self.cond_hint
             self.cond_hint = None
             # first, figure out which cond idxs are relevant, and where they fit in
-            cond_idxs = self.sparse_settings.sparse_method.get_indexes(hint_length=self.cond_hint_original.size(0), full_length=full_length)
+            cond_idxs, hint_order  = self.sparse_settings.sparse_method.get_indexes(hint_length=self.cond_hint_original.size(0), full_length=full_length,
-            
+                                                                                     sub_idxs=self.sub_idxs if self.sparse_settings.is_context_aware() else None)
             range_idxs = list(range(full_length)) if self.sub_idxs is None else self.sub_idxs
             hint_idxs = [] # idxs in cond_idxs
-            local_idxs = []  # idx to pun in final cond_hint
+            local_idxs = []  # idx to put in final cond_hint
             for i,cond_idx in enumerate(cond_idxs):
                 if cond_idx in range_idxs:
                     hint_idxs.append(i)
                     local_idxs.append(range_idxs.index(cond_idx))
+            # log_string = f"cond_idxs: {cond_idxs}, local_idxs: {local_idxs}, hint_idxs: {hint_idxs}, hint_order: {hint_order}"
+            # if self.sub_idxs is not None:
+            #     log_string += f" sub_idxs: {self.sub_idxs[0]}-{self.sub_idxs[-1]}"
+            # logger.warn(log_string)
+            # determine cond/uncond indexes that will get masked
+            self.local_sparse_idxs = []
+            self.local_sparse_idxs_inverse = list(range(x_noisy.size(0)))
+            for batch_idx in range(batched_number):
+                for i in local_idxs:
+                    actual_i = i+(batch_idx*actual_length)
+                    self.local_sparse_idxs.append(actual_i)
+                    if actual_i in self.local_sparse_idxs_inverse:
+                        self.local_sparse_idxs_inverse.remove(actual_i)
             # sub_cond_hint now contains the hints relevant to current x_noisy
-            sub_cond_hint = self.cond_hint_original[hint_idxs].to(dtype).to(self.device)
+            if hint_order is None:
-
+                sub_cond_hint = self.cond_hint_original[hint_idxs].to(dtype).to(x_noisy.device)
+            else:
+                sub_cond_hint = self.cond_hint_original[hint_order][hint_idxs].to(dtype).to(x_noisy.device)
             # scale cond_hints to match noisy input
             if self.control_model.use_simplified_conditioning_embedding:
                 # RGB SparseCtrl; the inputs are latents - use bilinear to avoid blocky artifacts
-                sub_cond_hint = self.latent_format.process_in(sub_cond_hint)  # multiplies by model scale factor
+                sub_cond_hint = self.model_latent_format.process_in(sub_cond_hint)  # multiplies by model scale factor
-                sub_cond_hint = comfy.utils.common_upscale(sub_cond_hint, x_noisy.shape[3], x_noisy.shape[2], "nearest-exact", "center").to(dtype).to(self.device)
+                sub_cond_hint = comfy.utils.common_upscale(sub_cond_hint, x_noisy.shape[3], x_noisy.shape[2], "nearest-exact", "center").to(dtype).to(x_noisy.device)
             else:
                 # other SparseCtrl; inputs are typical images
-                sub_cond_hint = comfy.utils.common_upscale(sub_cond_hint, x_noisy.shape[3] * 8, x_noisy.shape[2] * 8, 'nearest-exact', "center").to(dtype).to(self.device)
+                sub_cond_hint = comfy.utils.common_upscale(sub_cond_hint, x_noisy.shape[3] * 8, x_noisy.shape[2] * 8, 'nearest-exact', "center").to(dtype).to(x_noisy.device)
             # prepare cond_hint (b, c, h ,w)
             cond_shape = list(sub_cond_hint.shape)
             cond_shape[0] = len(range_idxs)
-            self.cond_hint = torch.zeros(cond_shape).to(dtype).to(self.device)
+            self.cond_hint = torch.zeros(cond_shape).to(dtype).to(x_noisy.device)
             self.cond_hint[local_idxs] = sub_cond_hint[:]
             # prepare cond_mask (b, 1, h, w)
             cond_shape[1] = 1
-            cond_mask = torch.zeros(cond_shape).to(dtype).to(self.device)
+            cond_mask = torch.zeros(cond_shape).to(dtype).to(x_noisy.device)
-            cond_mask[local_idxs] = 1.0
+            cond_mask[local_idxs] = self.sparse_settings.sparse_mask_mult * self.weights.extras.get(SparseConst.MASK_MULT, 1.0)
             # combine cond_hint and cond_mask into (b, c+1, h, w)
             if not self.sparse_settings.merged:
                 self.cond_hint = torch.cat([self.cond_hint, cond_mask], dim=1)
@@ -346,152 +459,70 @@ class SparseCtrlAdvanced(ControlNetAdvanced):
         context = cond['c_crossattn']
         y = cond.get('y', None)
         if y is not None:
-            y = y.to(dtype)
+            y = comfy.model_base.convert_tensor(y, dtype, x_noisy.device)
         timestep = self.model_sampling_current.timestep(t)
         x_noisy = self.model_sampling_current.calculate_input(t, x_noisy)
 
-        control = self.control_model(x=x_noisy.to(dtype), hint=self.cond_hint, timesteps=timestep.float(), context=context.to(dtype), y=y)
+        control = self.control_model(x=x_noisy.to(dtype), hint=self.cond_hint, timesteps=timestep.float(), context=comfy.model_management.cast_to_device(context, x_noisy.device, dtype), y=y)
-        return self.control_merge(None, control, control_prev, output_dtype)
+        return self.control_merge(control, control_prev, output_dtype)
+
+    def apply_advanced_strengths_and_masks(self, x: Tensor, batched_number: int, *args, **kwargs):
+        # apply mults to indexes with and without a direct condhint
+        x[self.local_sparse_idxs] *= self.sparse_settings.sparse_hint_mult * self.weights.extras.get(SparseConst.HINT_MULT, 1.0)
+        x[self.local_sparse_idxs_inverse] *= self.sparse_settings.sparse_nonhint_mult * self.weights.extras.get(SparseConst.NONHINT_MULT, 1.0)
+        return super().apply_advanced_strengths_and_masks(x, batched_number, *args, **kwargs)
 
     def pre_run_advanced(self, model, percent_to_timestep_function):
         super().pre_run_advanced(model, percent_to_timestep_function)
-        if type(self.cond_hint_original) == PreprocSparseRGBWrapper:
+        if isinstance(self.cond_hint_original, AbstractPreprocWrapper):
             if not self.control_model.use_simplified_conditioning_embedding:
                 raise ValueError("Any model besides RGB SparseCtrl should NOT have its images go through the RGB SparseCtrl preprocessor.")
             self.cond_hint_original = self.cond_hint_original.condhint
-        self.latent_format = model.latent_format  # LatentFormat object, used to process_in latent cond hint
+        self.model_latent_format = model.latent_format  # LatentFormat object, used to process_in latent cond hint
-        if self.control_model.motion_wrapper is not None:
+        if self.motion_model is not None:
-            self.control_model.motion_wrapper.reset()
+            self.motion_model.cleanup()
-            self.control_model.motion_wrapper.set_strength(self.sparse_settings.motion_strength)
+            self.motion_model.set_effect(self.sparse_settings.motion_strength)
-            self.control_model.motion_wrapper.set_scale_multiplier(self.sparse_settings.motion_scale)
+            self.motion_model.set_scale(self.sparse_settings.motion_scale)
 
     def cleanup_advanced(self):
         super().cleanup_advanced()
-        if self.latent_format is not None:
+        if self.model_latent_format is not None:
-            del self.latent_format
+            del self.model_latent_format
-            self.latent_format = None
+            self.model_latent_format = None
+        self.local_sparse_idxs = None
+        self.local_sparse_idxs_inverse = None
+        if self.motion_model is not None:
+            self.motion_model.cleanup()
 
     def copy(self):
-        c = SparseCtrlAdvanced(self.control_model, self.timestep_keyframes, self.sparse_settings, self.global_average_pooling, self.device, self.load_device, self.manual_cast_dtype)
+        c = SparseCtrlAdvanced(None, None, self.timestep_keyframes, self.sparse_settings, self.global_average_pooling, self.load_device, self.manual_cast_dtype)
+        c.control_model = self.control_model
+        c.control_model_wrapped = self.control_model_wrapped
+        self.prepare_conditioning_info()
         self.copy_to(c)
         self.copy_to_advanced(c)
         return c
 
-
+    def get_models(self):
-class ControlLLLiteAdvanced(ControlBase, AdvancedControlBase):
+        to_return = super().get_models()
-    # This ControlNet is more of an attention patch than a traditional controlnet
+        to_return.extend(self.control_model_wrapped.get_additional_models())
-    def __init__(self, patch_attn1: LLLitePatch, patch_attn2: LLLitePatch, timestep_keyframes: TimestepKeyframeGroup, device=None):
-        super().__init__(device)
-        AdvancedControlBase.__init__(self, super(), timestep_keyframes=timestep_keyframes, weights_default=ControlWeights.controllllite(), require_model=True)
-        self.patch_attn1 = patch_attn1.set_control(self)
-        self.patch_attn2 = patch_attn2.set_control(self)
-        self.latent_dims_div2 = None
-        self.latent_dims_div4 = None
-
-    def patch_model(self, model: ModelPatcher):
-        model.set_model_attn1_patch(self.patch_attn1)
-        model.set_model_attn2_patch(self.patch_attn2)
-
-    def set_cond_hint(self, *args, **kwargs):
-        to_return = super().set_cond_hint(*args, **kwargs)
-        # cond hint for LLLite needs to be scaled between (-1, 1) instead of (0, 1)
-        self.cond_hint_original = self.cond_hint_original * 2.0 - 1.0
         return to_return
 
-    def pre_run_advanced(self, *args, **kwargs):
-        AdvancedControlBase.pre_run_advanced(self, *args, **kwargs)
-        #logger.error(f"in cn: {id(self.patch_attn1)},{id(self.patch_attn2)}")
-        self.patch_attn1.set_control(self)
-        self.patch_attn2.set_control(self)
-        #logger.warn(f"in pre_run_advanced: {id(self)}")
-    
-    def get_control_advanced(self, x_noisy: Tensor, t, cond, batched_number: int):
-        # normal ControlNet stuff
-        control_prev = None
-        if self.previous_controlnet is not None:
-            control_prev = self.previous_controlnet.get_control(x_noisy, t, cond, batched_number)
-
-        if self.timestep_range is not None:
-            if t[0] > self.timestep_range[0] or t[0] < self.timestep_range[1]:
-                return control_prev
-        
-        dtype = x_noisy.dtype
-        # prepare cond_hint
-        if self.sub_idxs is not None or self.cond_hint is None or x_noisy.shape[2] * 8 != self.cond_hint.shape[2] or x_noisy.shape[3] * 8 != self.cond_hint.shape[3]:
-            if self.cond_hint is not None:
-                del self.cond_hint
-            self.cond_hint = None
-            # if self.cond_hint_original length greater or equal to real latent count, subdivide it before scaling
-            if self.sub_idxs is not None:
-                actual_cond_hint_orig = self.cond_hint_original
-                if self.cond_hint_original.size(0) < self.full_latent_length:
-                    actual_cond_hint_orig = extend_to_batch_size(tensor=actual_cond_hint_orig, batch_size=self.full_latent_length)
-                self.cond_hint = comfy.utils.common_upscale(actual_cond_hint_orig[self.sub_idxs], x_noisy.shape[3] * 8, x_noisy.shape[2] * 8, 'nearest-exact', "center").to(dtype).to(self.device)
-            else:
-                self.cond_hint = comfy.utils.common_upscale(self.cond_hint_original, x_noisy.shape[3] * 8, x_noisy.shape[2] * 8, 'nearest-exact', "center").to(dtype).to(self.device)
-        if x_noisy.shape[0] != self.cond_hint.shape[0]:
-            self.cond_hint = broadcast_image_to_extend(self.cond_hint, x_noisy.shape[0], batched_number)
-        # some special logic here compared to other controlnets:
-        # * The cond_emb in attn patches will divide latent dims by 2 or 4, integer
-        # * Due to this loss, the cond_emb will become smaller than x input if latent dims are not divisble by 2 or 4
-        divisible_by_2_h = x_noisy.shape[2]%2==0
-        divisible_by_2_w = x_noisy.shape[3]%2==0
-        if not (divisible_by_2_h and divisible_by_2_w):
-            #logger.warn(f"{x_noisy.shape} not divisible by 2!")
-            new_h = (x_noisy.shape[2]//2)*2
-            new_w = (x_noisy.shape[3]//2)*2
-            if not divisible_by_2_h:
-                new_h += 2
-            if not divisible_by_2_w:
-                new_w += 2
-            self.latent_dims_div2 = (new_h, new_w)
-        divisible_by_4_h = x_noisy.shape[2]%4==0
-        divisible_by_4_w =  x_noisy.shape[3]%4==0
-        if not (divisible_by_4_h and divisible_by_4_w):
-            #logger.warn(f"{x_noisy.shape} not divisible by 4!")
-            new_h = (x_noisy.shape[2]//4)*4
-            new_w = (x_noisy.shape[3]//4)*4
-            if not divisible_by_4_h:
-                new_h += 4
-            if not divisible_by_4_w:
-                new_w += 4
-            self.latent_dims_div4 = (new_h, new_w)
-        # prepare mask
-        self.prepare_mask_cond_hint(x_noisy=x_noisy, t=t, cond=cond, batched_number=batched_number)
-        # done preparing; model patches will take care of everything now.
-        # return normal controlnet stuff
-        return control_prev
-    
-    def cleanup_advanced(self):
-        super().cleanup_advanced()
-        self.patch_attn1.cleanup()
-        self.patch_attn2.cleanup()
-        self.latent_dims_div2 = None
-        self.latent_dims_div4 = None
-    
-    def copy(self):
-        c = ControlLLLiteAdvanced(self.patch_attn1, self.patch_attn2, self.timestep_keyframes)
-        self.copy_to(c)
-        self.copy_to_advanced(c)
-        return c
-    
-    # deepcopy needs to properly keep track of objects to work between model.clone calls!
-    # def __deepcopy__(self, *args, **kwargs):
-    #     self.cleanup_advanced()
-    #     return self
-
-    # def get_models(self):
-    #     # get_models is called once at the start of every KSampler run - use to reset already_patched status
-    #     out = super().get_models()
-    #     logger.error(f"in get_models! {id(self)}")
-    #     return out
-
 
 def load_controlnet(ckpt_path, timestep_keyframe: TimestepKeyframeGroup=None, model=None):
-    controlnet_data = comfy.utils.load_torch_file(ckpt_path, safe_load=True)
+    controlnet_data, metadata = comfy.utils.load_torch_file(ckpt_path, safe_load=True, return_metadata=True)
+    # from pathlib import Path
+    # log_name = ckpt_path.split('\\')[-1]
+    # with open(Path(__file__).parent.parent.parent / rf"keys_{log_name}.txt", "w") as afile:
+    #     for key, value in controlnet_data.items():
+    #         afile.write(f"{key}:\t{value.shape}\n")
     control = None
     # check if a non-vanilla ControlNet
     controlnet_type = ControlWeightType.DEFAULT
+    is_anima_lllite = (
+        "lllite_conditioning1.conv1.weight" in controlnet_data
+        and any(key.startswith("lllite_dit_blocks_") for key in controlnet_data)
+    )
     has_controlnet_key = False
     has_motion_modules_key = False
     has_temporal_res_block_key = False
@@ -508,20 +539,30 @@ def load_controlnet(ckpt_path, timestep_keyframe: TimestepKeyframeGroup=None, mo
         # SVD-ControlNet check
         elif "temporal_res_block" in key:
             has_temporal_res_block_key = True
+        # ControlNet++ check
+        elif "task_embedding" in key:
+            pass
+        # CtrLoRA check
+        elif "lora_layer" in key:
+            controlnet_type = ControlWeightType.CTRLORA
+            break
+
     if has_controlnet_key and has_motion_modules_key:
         controlnet_type = ControlWeightType.SPARSECTRL
     elif has_controlnet_key and has_temporal_res_block_key:
         controlnet_type = ControlWeightType.SVD_CONTROLNET
 
-    if controlnet_type != ControlWeightType.DEFAULT:
+    if is_anima_lllite:
+        control = load_anima_lllite(ckpt_path, controlnet_data=controlnet_data, metadata=metadata, timestep_keyframe=timestep_keyframe)
+    elif controlnet_type != ControlWeightType.DEFAULT:
         if controlnet_type == ControlWeightType.CONTROLLLLITE:
             control = load_controllllite(ckpt_path, controlnet_data=controlnet_data, timestep_keyframe=timestep_keyframe)
         elif controlnet_type == ControlWeightType.SPARSECTRL:
             control = load_sparsectrl(ckpt_path, controlnet_data=controlnet_data, timestep_keyframe=timestep_keyframe, model=model)
         elif controlnet_type == ControlWeightType.SVD_CONTROLNET:
             control = load_svdcontrolnet(ckpt_path, controlnet_data=controlnet_data, timestep_keyframe=timestep_keyframe)
-            #raise Exception(f"SVD-ControlNet is not supported yet!")
+        elif controlnet_type == ControlWeightType.CTRLORA:
-            #control = comfy_cn.load_controlnet(ckpt_path, model=model)
+            raise Exception("This is a CtrLoRA; use the Load CtrLoRA Model node.")
     # otherwise, load vanilla ControlNet
     else:
         try:
@@ -531,6 +572,8 @@ def load_controlnet(ckpt_path, timestep_keyframe: TimestepKeyframeGroup=None, mo
             control = comfy_cn.load_controlnet(ckpt_path, model=model)
         finally:
             comfy.utils.load_torch_file = orig_load_torch_file
+    if control is None:
+        raise Exception(f"Something went wrong when loading '{ckpt_path}'; ControlNet is None.")
     return convert_to_advanced(control, timestep_keyframe=timestep_keyframe)
 
 
@@ -540,7 +583,16 @@ def convert_to_advanced(control, timestep_keyframe: TimestepKeyframeGroup=None):
         return control
     # if exactly ControlNet returned, transform it into ControlNetAdvanced
     if type(control) == ControlNet:
-        return ControlNetAdvanced.from_vanilla(v=control, timestep_keyframe=timestep_keyframe)
+        control = ControlNetAdvanced.from_vanilla(v=control, timestep_keyframe=timestep_keyframe)
+        if is_sd3_advanced_controlnet(control):
+            control.require_vae = True
+        return control
+    # if exactly ControlNetSD35 returned, transform into ControlNetSD35Advanced
+    elif type(control) == ControlNetSD35:
+        control = ControlNetSD35Advanced.from_vanilla(v=control, timestep_keyframe=timestep_keyframe)
+        if is_sd3_advanced_controlnet(control):
+            control.require_vae = True
+        return control
     # if exactly ControlLora returned, transform it into ControlLoraAdvanced
     elif type(control) == ControlLora:
         return ControlLoraAdvanced.from_vanilla(v=control, timestep_keyframe=timestep_keyframe)
@@ -551,10 +603,124 @@ def convert_to_advanced(control, timestep_keyframe: TimestepKeyframeGroup=None):
     return control
 
 
+def convert_all_to_advanced(conds: dict[str, list[dict[str]]]) -> tuple[bool, list]:
+    cache = {}
+    modified = False
+    new_conds = {}
+    for cond_type in conds:
+        converted_cond: list[dict[str]] = None
+        cond = conds[cond_type]
+        if cond is not None:
+            for actual_cond in cond:
+                need_to_convert = False
+                if "control" in actual_cond:
+                    if not are_all_advanced_controlnet(actual_cond["control"]):
+                        need_to_convert = True
+                        break
+            if not need_to_convert:
+                converted_cond = cond
+            else:
+                converted_cond = []
+                for actual_cond in cond:
+                    if not isinstance(actual_cond, dict):
+                        converted_cond.append(actual_cond)
+                        continue
+                    if "control" not in actual_cond:
+                        converted_cond.append(actual_cond)
+                    elif are_all_advanced_controlnet(actual_cond["control"]):
+                        converted_cond.append(actual_cond)
+                    else:
+                        actual_cond = actual_cond.copy()
+                        actual_cond["control"] = _convert_all_control_to_advanced(actual_cond["control"], cache)
+                        converted_cond.append(actual_cond)
+                        modified = True
+        new_conds[cond_type] = converted_cond
+    return modified, new_conds
+
+
+def _convert_all_control_to_advanced(input_object: ControlBase, cache: dict):
+    output_object = input_object
+    # iteratively convert to advanced, if needed
+    next_cn = None
+    curr_cn = input_object
+    iter = 0
+    while curr_cn is not None:
+        if not is_advanced_controlnet(curr_cn):
+            # if already in cache, then conversion was done before, so just link it and exit
+            if curr_cn in cache:
+                new_cn = cache[curr_cn]
+                if next_cn is not None:
+                    setattr(next_cn, ORIG_PREVIOUS_CONTROLNET, next_cn.previous_controlnet)
+                    next_cn.previous_controlnet = new_cn
+                if iter == 0: # if was top-level controlnet, that's the new output
+                    output_object = new_cn
+                break
+            try:
+                # convert to advanced, and assign previous_controlnet (convert doesn't transfer it)
+                new_cn = convert_to_advanced(curr_cn)
+            except Exception as e:
+                raise Exception("Failed to automatically convert a ControlNet to Advanced to support sliding window context.", e)
+            new_cn.previous_controlnet = curr_cn.previous_controlnet
+            if iter == 0: # if was top-level controlnet, that's the new output
+                output_object = new_cn
+            # if next_cn is present, then it needs to be pointed to new_cn
+            if next_cn is not None:
+                setattr(next_cn, ORIG_PREVIOUS_CONTROLNET, next_cn.previous_controlnet)
+                next_cn.previous_controlnet = new_cn
+            # add to cache
+            cache[curr_cn] = new_cn
+            curr_cn = new_cn
+        next_cn = curr_cn
+        curr_cn = curr_cn.previous_controlnet
+        iter += 1
+    return output_object
+
+
+def restore_all_controlnet_conns(conds: dict[str, list[dict[str]]]):
+    # if a cn has an _orig_previous_controlnet property, restore it and delete
+    for cond_type in conds:
+        cond = conds[cond_type]
+        if cond is not None:
+            for actual_cond in cond:
+                if "control" in actual_cond:
+                    # if ACN is the one to have initialized it, delete it
+                    # TODO: maybe check if someone else did a similar hack, and carefully pluck out our stuff?
+                    if CONTROL_INIT_BY_ACN in actual_cond:
+                        actual_cond.pop("control")
+                        actual_cond.pop(CONTROL_INIT_BY_ACN)
+                    else:
+                        _restore_all_controlnet_conns(actual_cond["control"])
+
+
+
+def _restore_all_controlnet_conns(input_object: ControlBase):
+    # restore original previous_controlnet if needed
+    curr_cn = input_object
+    while curr_cn is not None:
+        if hasattr(curr_cn, ORIG_PREVIOUS_CONTROLNET):
+            curr_cn.previous_controlnet = getattr(curr_cn, ORIG_PREVIOUS_CONTROLNET)
+            delattr(curr_cn, ORIG_PREVIOUS_CONTROLNET)
+        curr_cn = curr_cn.previous_controlnet
+
+
+def are_all_advanced_controlnet(input_object: ControlBase):
+    # iteratively check if linked controlnets objects are all advanced
+    curr_cn = input_object
+    while curr_cn is not None:
+        if not is_advanced_controlnet(curr_cn):
+            return False
+        curr_cn = curr_cn.previous_controlnet
+    return True
+
+
 def is_advanced_controlnet(input_object):
     return hasattr(input_object, "sub_idxs")
 
 
+def is_sd3_advanced_controlnet(input_object: ControlNetAdvanced):
+    return type(input_object) in [ControlNetAdvanced, ControlNetSD35Advanced] and input_object.latent_format is not None
+
+
 def load_sparsectrl(ckpt_path: str, controlnet_data: dict[str, Tensor]=None, timestep_keyframe: TimestepKeyframeGroup=None, sparse_settings=SparseSettings.default(), model=None) -> SparseCtrlAdvanced:
     if controlnet_data is None:
         controlnet_data = comfy.utils.load_torch_file(ckpt_path, safe_load=True)
@@ -651,6 +817,7 @@ def load_sparsectrl(ckpt_path: str, controlnet_data: dict[str, Tensor]=None, tim
         controlnet_config["operations"] = manual_cast_clean_groupnorm
     else:
         controlnet_config["operations"] = disable_weight_init_clean_groupnorm
+    controlnet_config["dtype"] = unet_dtype
     controlnet_config.pop("out_channels")
     # get proper hint channels
     if use_simplified_conditioning_embedding:
@@ -685,6 +852,10 @@ def load_sparsectrl(ckpt_path: str, controlnet_data: dict[str, Tensor]=None, tim
         missing, unexpected = control_model.load_state_dict(controlnet_data, strict=False)
     if len(missing) > 0 or len(unexpected) > 0:
         logger.info(f"SparseCtrl ControlNet: {missing}, {unexpected}")
+    # cast control_model to the intended dtype; load_state_dict can leave weights
+    # in their on-disk dtype (e.g. comfy's lazy/zero-copy state dict loading), which
+    # would otherwise mismatch the activations at runtime
+    control_model = control_model.to(unet_dtype)
 
     global_average_pooling = False
     filename = os.path.splitext(ckpt_path)[0]
@@ -692,65 +863,12 @@ def load_sparsectrl(ckpt_path: str, controlnet_data: dict[str, Tensor]=None, tim
         global_average_pooling = True
 
     # actually load motion portion of model now
-    motion_wrapper: SparseCtrlMotionWrapper = SparseCtrlMotionWrapper(motion_data, ops=controlnet_config.get("operations", None)).to(comfy.model_management.unet_dtype())
+    motion_model = load_sparsectrl_motionmodel(ckpt_path=ckpt_path, motion_data=motion_data, ops=controlnet_config.get("operations", None)).to(comfy.model_management.unet_dtype())
-    missing, unexpected = motion_wrapper.load_state_dict(motion_data)
+    # both motion portion and controlnet portions are loaded; ignore motion_model if shouldn't use motion portion
-    if len(missing) > 0 or len(unexpected) > 0:
+    if not sparse_settings.use_motion:
-        logger.info(f"SparseCtrlMotionWrapper: {missing}, {unexpected}")
+        motion_model = None
 
-    # both motion portion and controlnet portions are loaded; bring them together if using motion model
+    control = SparseCtrlAdvanced(control_model, motion_model, timestep_keyframes=timestep_keyframe, sparse_settings=sparse_settings, global_average_pooling=global_average_pooling, load_device=load_device, manual_cast_dtype=manual_cast_dtype)
-    if sparse_settings.use_motion:
-        motion_wrapper.inject(control_model)
-
-    control = SparseCtrlAdvanced(control_model, timestep_keyframes=timestep_keyframe, sparse_settings=sparse_settings, global_average_pooling=global_average_pooling, load_device=load_device, manual_cast_dtype=manual_cast_dtype)
-    return control
-
-
-def load_controllllite(ckpt_path: str, controlnet_data: dict[str, Tensor]=None, timestep_keyframe: TimestepKeyframeGroup=None):
-    if controlnet_data is None:
-        controlnet_data = comfy.utils.load_torch_file(ckpt_path, safe_load=True)
-    # adapted from https://github.com/kohya-ss/ControlNet-LLLite-ComfyUI
-    # first, split weights for each module
-    module_weights = {}
-    for key, value in controlnet_data.items():
-        fragments = key.split(".")
-        module_name = fragments[0]
-        weight_name = ".".join(fragments[1:])
-
-        if module_name not in module_weights:
-            module_weights[module_name] = {}
-        module_weights[module_name][weight_name] = value
-    
-    # next, load each module
-    modules = {}
-    for module_name, weights in module_weights.items():
-        # kohya planned to do something about how these should be chosen, so I'm not touching this
-        # since I am not familiar with the logic for this
-        if "conditioning1.4.weight" in weights:
-            depth = 3
-        elif weights["conditioning1.2.weight"].shape[-1] == 4:
-            depth = 2
-        else:
-            depth = 1
-
-        module = LLLiteModule(
-            name=module_name,
-            is_conv2d=weights["down.0.weight"].ndim == 4,
-            in_dim=weights["down.0.weight"].shape[1],
-            depth=depth,
-            cond_emb_dim=weights["conditioning1.0.weight"].shape[0] * 2,
-            mlp_dim=weights["down.0.weight"].shape[0],
-        )
-        # load weights into module
-        module.load_state_dict(weights)
-        modules[module_name] = module
-        if len(modules) == 1:
-            module.is_first = True
-
-    #logger.info(f"loaded {ckpt_path} successfully, {len(modules)} modules")
-
-    patch_attn1 = LLLitePatch(modules=modules, patch_type=LLLitePatch.ATTN1)
-    patch_attn2 = LLLitePatch(modules=modules, patch_type=LLLitePatch.ATTN2)
-    control = ControlLLLiteAdvanced(patch_attn1=patch_attn1, patch_attn2=patch_attn2, timestep_keyframes=timestep_keyframe)
     return control
 
 
@@ -837,6 +955,7 @@ def load_svdcontrolnet(ckpt_path: str, controlnet_data: dict[str, Tensor]=None,
     manual_cast_dtype = comfy.model_management.unet_manual_cast(unet_dtype, load_device)
     if manual_cast_dtype is not None:
         controlnet_config["operations"] = comfy.ops.manual_cast
+    controlnet_config["dtype"] = unet_dtype
     controlnet_config.pop("out_channels")
     controlnet_config["hint_channels"] = controlnet_data["{}input_hint_block.0.weight".format(prefix)].shape[1]
     control_model = SVDControlNet(**controlnet_config)
@@ -865,6 +984,10 @@ def load_svdcontrolnet(ckpt_path: str, controlnet_data: dict[str, Tensor]=None,
         missing, unexpected = control_model.load_state_dict(controlnet_data, strict=False)
     if len(missing) > 0 or len(unexpected) > 0:
         logger.info(f"SVD-ControlNet: {missing}, {unexpected}")
+    # cast control_model to the intended dtype; load_state_dict can leave weights
+    # in their on-disk dtype (e.g. comfy's lazy/zero-copy state dict loading), which
+    # would otherwise mismatch the activations at runtime
+    control_model = control_model.to(unet_dtype)
 
     global_average_pooling = False
     filename = os.path.splitext(ckpt_path)[0]
@@ -873,4 +996,3 @@ def load_svdcontrolnet(ckpt_path: str, controlnet_data: dict[str, Tensor]=None,
 
     control = SVDControlNetAdvanced(control_model, timestep_keyframes=timestep_keyframe, global_average_pooling=global_average_pooling, load_device=load_device, manual_cast_dtype=manual_cast_dtype)
     return control
-

adv_control/control_ctrlora.py

@@ -0,0 +1,231 @@
+# Core code adapted from CtrLoRA github repo:
+# https://github.com/xyfJASON/ctrlora
+import torch
+from torch import Tensor
+
+from comfy.cldm.cldm import ControlNet as ControlNetCLDM
+import comfy.model_detection
+import comfy.model_management
+import comfy.ops
+import comfy.utils
+
+from comfy.ldm.modules.diffusionmodules.util import (
+    zero_module,
+    timestep_embedding,
+)
+
+from .control import ControlNetAdvanced
+from .utils import TimestepKeyframeGroup
+from .logger import logger
+
+
+class ControlNetCtrLoRA(ControlNetCLDM):
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        # delete input hint block
+        del self.input_hint_block
+    
+    def forward(self, x: Tensor, hint: Tensor, timesteps, context, y=None, **kwargs):
+        t_emb = timestep_embedding(timesteps, self.model_channels, repeat_only=False).to(x.dtype)
+        emb = self.time_embed(t_emb)
+
+        out_output = []
+        out_middle = []
+
+        if self.num_classes is not None:
+            assert y.shape[0] == x.shape[0]
+            emb = emb + self.label_emb(y)
+        
+        h = hint.to(dtype=x.dtype)
+        for module, zero_conv in zip(self.input_blocks, self.zero_convs):
+            h = module(h, emb, context)
+            out_output.append(zero_conv(h, emb, context))
+        
+        h = self.middle_block(h, emb, context)
+        out_middle.append(self.middle_block_out(h, emb, context))
+
+        return {"middle": out_middle, "output": out_output}
+
+
+class CtrLoRAAdvanced(ControlNetAdvanced):
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+        self.preprocess_image = lambda a: (a + 1) / 2.0
+        self.require_vae = True
+        self.mult_by_ratio_when_vae = False
+
+    def pre_run_advanced(self, model, percent_to_timestep_function):
+        super().pre_run_advanced(model, percent_to_timestep_function)
+        self.latent_format = model.latent_format  # LatentFormat object, used to process_in latent cond hint
+
+    def cleanup_advanced(self):
+        super().cleanup_advanced()
+        if self.latent_format is not None:
+            del self.latent_format
+            self.latent_format = None
+
+    def copy(self):
+        c = CtrLoRAAdvanced(self.control_model, self.timestep_keyframes, global_average_pooling=self.global_average_pooling, load_device=self.load_device, manual_cast_dtype=self.manual_cast_dtype)
+        c.control_model = self.control_model
+        c.control_model_wrapped = self.control_model_wrapped
+        self.copy_to(c)
+        self.copy_to_advanced(c)
+        return c
+
+
+def load_ctrlora(base_path: str, lora_path: str,
+                 base_data: dict[str, Tensor]=None, lora_data: dict[str, Tensor]=None,
+                 timestep_keyframe: TimestepKeyframeGroup=None, model=None, model_options={}):
+    if base_data is None:
+        base_data = comfy.utils.load_torch_file(base_path, safe_load=True)
+    controlnet_data = base_data
+
+    # first, check that base_data contains keys with lora_layer
+    contains_lora_layers = False
+    for key in base_data:
+        if "lora_layer" in key:
+            contains_lora_layers = True
+    if not contains_lora_layers:
+        raise Exception(f"File '{base_path}' is not a valid CtrLoRA base model; does not contain any lora_layer keys.")
+    
+    controlnet_config = None
+    supported_inference_dtypes = None
+
+    pth_key = 'control_model.zero_convs.0.0.weight'
+    pth = False
+    key = 'zero_convs.0.0.weight'
+    if pth_key in controlnet_data:
+        pth = True
+        key = pth_key
+        prefix = "control_model."
+    elif key in controlnet_data:
+        prefix = ""
+    else:
+        raise Exception("")
+        net = load_t2i_adapter(controlnet_data, model_options=model_options)
+        if net is None:
+            logging.error("error could not detect control model type.")
+        return net
+
+    if controlnet_config is None:
+        model_config = comfy.model_detection.model_config_from_unet(controlnet_data, prefix, True)
+        supported_inference_dtypes = list(model_config.supported_inference_dtypes)
+        controlnet_config = model_config.unet_config
+
+    unet_dtype = model_options.get("dtype", None)
+    if unet_dtype is None:
+        weight_dtype = comfy.utils.weight_dtype(controlnet_data)
+
+        if supported_inference_dtypes is None:
+            supported_inference_dtypes = [comfy.model_management.unet_dtype()]
+
+        if weight_dtype is not None:
+            supported_inference_dtypes.append(weight_dtype)
+
+        unet_dtype = comfy.model_management.unet_dtype(model_params=-1, supported_dtypes=supported_inference_dtypes)
+
+    load_device = comfy.model_management.get_torch_device()
+
+    manual_cast_dtype = comfy.model_management.unet_manual_cast(unet_dtype, load_device)
+    operations = model_options.get("custom_operations", None)
+    if operations is None:
+        operations = comfy.ops.pick_operations(unet_dtype, manual_cast_dtype)
+
+    controlnet_config["operations"] = operations
+    controlnet_config["dtype"] = unet_dtype
+    controlnet_config["device"] = comfy.model_management.unet_offload_device()
+    controlnet_config.pop("out_channels")
+    controlnet_config["hint_channels"] = 3
+    #controlnet_config["hint_channels"] = controlnet_data["{}input_hint_block.0.weight".format(prefix)].shape[1]
+    control_model = ControlNetCtrLoRA(**controlnet_config)
+
+    if pth:
+        if 'difference' in controlnet_data:
+            if model is not None:
+                comfy.model_management.load_models_gpu([model])
+                model_sd = model.model_state_dict()
+                for x in controlnet_data:
+                    c_m = "control_model."
+                    if x.startswith(c_m):
+                        sd_key = "diffusion_model.{}".format(x[len(c_m):])
+                        if sd_key in model_sd:
+                            cd = controlnet_data[x]
+                            cd += model_sd[sd_key].type(cd.dtype).to(cd.device)
+            else:
+                logger.warning("WARNING: Loaded a diff controlnet without a model. It will very likely not work.")
+
+        class WeightsLoader(torch.nn.Module):
+            pass
+        w = WeightsLoader()
+        w.control_model = control_model
+        missing, unexpected = w.load_state_dict(controlnet_data, strict=False)
+    else:
+        missing, unexpected = control_model.load_state_dict(controlnet_data, strict=False)
+
+    if len(missing) > 0:
+        logger.warning("missing controlnet keys: {}".format(missing))
+
+    if len(unexpected) > 0:
+        logger.debug("unexpected controlnet keys: {}".format(unexpected))
+
+    global_average_pooling = model_options.get("global_average_pooling", False)
+    control = CtrLoRAAdvanced(control_model, timestep_keyframe, global_average_pooling=global_average_pooling,
+                                 load_device=load_device, manual_cast_dtype=manual_cast_dtype)
+    # load lora data onto the controlnet
+    if lora_path is not None:
+        load_lora_data(control, lora_path)
+
+    return control
+
+
+def load_lora_data(control: CtrLoRAAdvanced, lora_path: str, loaded_data: dict[str, Tensor]=None, lora_strength=1.0):
+    if loaded_data is None:
+        loaded_data = comfy.utils.load_torch_file(lora_path, safe_load=True)
+    # check that lora_data contains keys with lora_layer
+    contains_lora_layers = False
+    for key in loaded_data:
+        if "lora_layer" in key:
+            contains_lora_layers = True
+    if not contains_lora_layers:
+        raise Exception(f"File '{lora_path}' is not a valid CtrLoRA lora model; does not contain any lora_layer keys.")
+
+    # now that we know we have a ctrlora file, separate keys into 'set' and 'lora' keys
+    data_set: dict[str, Tensor] = {}
+    data_lora: dict[str, Tensor] = {}
+
+    for key in list(loaded_data.keys()):
+        if 'lora_layer' in key:
+            data_lora[key] = loaded_data.pop(key)
+        else:
+            data_set[key] = loaded_data.pop(key)
+    # no keys should be left over
+    if len(loaded_data) > 0:
+        logger.warning("Not all keys from CtrlLoRA lora model's loaded data were parsed!")
+    
+    # turn set/lora data into corresponding patches;
+    patches = {}
+    # set will replace the values
+    for key, value in data_set.items():
+        # prase model key from key;
+        # remove "control_model."
+        model_key = key.replace("control_model.", "")
+        patches[model_key] = ("set", (value,))
+    # lora will do mm of up and down tensors
+    for down_key in data_lora:
+        # only process lora down keys; we will process both up+down at the same time
+        if ".up." in down_key:
+            continue
+        # get up version of down key
+        up_key = down_key.replace(".down.", ".up.")
+        # get key that will match up with model key;
+        # remove "lora_layer.down." and "control_model."
+        model_key = down_key.replace("lora_layer.down.", "").replace("control_model.", "")
+        
+        weight_down = data_lora[down_key]
+        weight_up = data_lora[up_key]
+        # currently, ComfyUI expects 6 elements in 'lora' type, but for future-proofing add a bunch more with None
+        patches[model_key] = ("lora", (weight_up, weight_down, None, None, None, None,
+                                       None, None, None, None, None, None, None, None))
+    
+    # now that patches are made, add them to model
+    control.control_model_wrapped.add_patches(patches, strength_patch=lora_strength)

adv_control/control_lllite.py

@@ -8,10 +8,43 @@ import torch
 import os
 
 import comfy.utils
+import comfy.ops
+import comfy.model_management
+import comfy.model_patcher
+from comfy.model_patcher import ModelPatcher
 from comfy.controlnet import ControlBase
 
+try:
+    import comfy.ldm.anima.lllite as comfy_anima_lllite
+except ImportError:
+    comfy_anima_lllite = None
+
 from .logger import logger
-from .utils import AdvancedControlBase, deepcopy_with_sharing, prepare_mask_batch
+from .utils import (AdvancedControlBase, TimestepKeyframeGroup, ControlWeights, broadcast_image_to_extend, extend_to_batch_size,
+                    prepare_mask_batch)
+
+
+class AnimaLLLiteConst:
+    INPAINT_MASK = "anima_lllite_inpaint_mask"
+
+
+# based on set_model_patch code in comfy/model_patcher.py
+def set_model_patch(transformer_options, patch, name):
+    to = transformer_options
+    # check if patch was already added
+    if "patches" in to:
+        current_patches = to["patches"].get(name, [])
+        if patch in current_patches:
+            return
+    if "patches" not in to:
+        to["patches"] = {}
+    to["patches"][name] = to["patches"].get(name, []) + [patch]
+
+def set_model_attn1_patch(transformer_options, patch):
+    set_model_patch(transformer_options, patch, "attn1_patch")
+
+def set_model_attn2_patch(transformer_options, patch):
+    set_model_patch(transformer_options, patch, "attn2_patch")
 
 
 def extra_options_to_module_prefix(extra_options):
@@ -92,26 +125,8 @@ class LLLitePatch:
         return LLLitePatch(self.modules, self.patch_type, control)
 
     def cleanup(self):
-        #total_cleaned = 0
         for module in self.modules.values():
             module.cleanup()
-        #    total_cleaned += 1
-        #logger.info(f"cleaned modules: {total_cleaned}, {id(self)}")
-        #logger.error(f"cleanup LLLitePatch: {id(self)}")
-
-    # make sure deepcopy does not copy control, and deepcopied LLLitePatch should be assigned to control
-    def __deepcopy__(self, memo):
-        self.cleanup()
-        to_return: LLLitePatch = deepcopy_with_sharing(self, shared_attribute_names = ['control'], memo=memo)
-        #logger.warn(f"patch {id(self)} turned into {id(to_return)}")
-        try:
-            if self.patch_type == self.ATTN1:
-                to_return.control.patch_attn1 = to_return
-            elif self.patch_type == self.ATTN2:
-                to_return.control.patch_attn2 = to_return
-        except Exception:
-            pass
-        return to_return
 
 
 # TODO: use comfy.ops to support fp8 properly
@@ -252,3 +267,398 @@ class LLLiteModule(torch.nn.Module):
                 if cond_type == 1:
                     cx[actual_length*idx:actual_length*(idx+1)] *= control.weights.uncond_multiplier
         return cx * mask * control.strength * control._current_timestep_keyframe.strength
+
+
+class ControlLLLiteModules(torch.nn.Module):
+    def __init__(self, patch_attn1: LLLitePatch, patch_attn2: LLLitePatch):
+        super().__init__()
+        self.patch_attn1_modules = torch.nn.Sequential(*list(patch_attn1.modules.values()))
+        self.patch_attn2_modules = torch.nn.Sequential(*list(patch_attn2.modules.values()))
+
+
+class ControlLLLiteAdvanced(ControlBase, AdvancedControlBase):
+    # This ControlNet is more of an attention patch than a traditional controlnet
+    def __init__(self, patch_attn1: LLLitePatch, patch_attn2: LLLitePatch, timestep_keyframes: TimestepKeyframeGroup, device, ops: comfy.ops.disable_weight_init):
+        super().__init__()
+        AdvancedControlBase.__init__(self, super(), timestep_keyframes=timestep_keyframes, weights_default=ControlWeights.controllllite())
+        self.device = device
+        self.ops = ops
+        self.patch_attn1 = patch_attn1.clone_with_control(self)
+        self.patch_attn2 = patch_attn2.clone_with_control(self)
+        self.control_model = ControlLLLiteModules(self.patch_attn1, self.patch_attn2)
+        self.control_model_wrapped = ModelPatcher(self.control_model, load_device=device, offload_device=comfy.model_management.unet_offload_device())
+        self.latent_dims_div2 = None
+        self.latent_dims_div4 = None
+
+    def set_cond_hint_inject(self, *args, **kwargs):
+        to_return = super().set_cond_hint_inject(*args, **kwargs)
+        # cond hint for LLLite needs to be scaled between (-1, 1) instead of (0, 1)
+        self.cond_hint_original = self.cond_hint_original * 2.0 - 1.0
+        return to_return
+
+    def pre_run_advanced(self, *args, **kwargs):
+        AdvancedControlBase.pre_run_advanced(self, *args, **kwargs)
+        #logger.error(f"in cn: {id(self.patch_attn1)},{id(self.patch_attn2)}")
+        self.patch_attn1.set_control(self)
+        self.patch_attn2.set_control(self)
+        #logger.warn(f"in pre_run_advanced: {id(self)}")
+    
+    def get_control_advanced(self, x_noisy: Tensor, t, cond, batched_number: int, transformer_options: dict):
+        # normal ControlNet stuff
+        control_prev = None
+        if self.previous_controlnet is not None:
+            control_prev = self.previous_controlnet.get_control(x_noisy, t, cond, batched_number, transformer_options)
+
+        if self.timestep_range is not None:
+            if t[0] > self.timestep_range[0] or t[0] < self.timestep_range[1]:
+                return control_prev
+        
+        dtype = x_noisy.dtype
+        # prepare cond_hint
+        if self.sub_idxs is not None or self.cond_hint is None or x_noisy.shape[2] * 8 != self.cond_hint.shape[2] or x_noisy.shape[3] * 8 != self.cond_hint.shape[3]:
+            if self.cond_hint is not None:
+                del self.cond_hint
+            self.cond_hint = None
+            # if self.cond_hint_original length greater or equal to real latent count, subdivide it before scaling
+            if self.sub_idxs is not None:
+                actual_cond_hint_orig = self.cond_hint_original
+                if self.cond_hint_original.size(0) < self.full_latent_length:
+                    actual_cond_hint_orig = extend_to_batch_size(tensor=actual_cond_hint_orig, batch_size=self.full_latent_length)
+                self.cond_hint = comfy.utils.common_upscale(actual_cond_hint_orig[self.sub_idxs], x_noisy.shape[3] * 8, x_noisy.shape[2] * 8, 'nearest-exact', "center").to(dtype).to(x_noisy.device)
+            else:
+                self.cond_hint = comfy.utils.common_upscale(self.cond_hint_original, x_noisy.shape[3] * 8, x_noisy.shape[2] * 8, 'nearest-exact', "center").to(dtype).to(x_noisy.device)
+        if x_noisy.shape[0] != self.cond_hint.shape[0]:
+            self.cond_hint = broadcast_image_to_extend(self.cond_hint, x_noisy.shape[0], batched_number)
+        # some special logic here compared to other controlnets:
+        # * The cond_emb in attn patches will divide latent dims by 2 or 4, integer
+        # * Due to this loss, the cond_emb will become smaller than x input if latent dims are not divisble by 2 or 4
+        divisible_by_2_h = x_noisy.shape[2]%2==0
+        divisible_by_2_w = x_noisy.shape[3]%2==0
+        if not (divisible_by_2_h and divisible_by_2_w):
+            #logger.warn(f"{x_noisy.shape} not divisible by 2!")
+            new_h = (x_noisy.shape[2]//2)*2
+            new_w = (x_noisy.shape[3]//2)*2
+            if not divisible_by_2_h:
+                new_h += 2
+            if not divisible_by_2_w:
+                new_w += 2
+            self.latent_dims_div2 = (new_h, new_w)
+        divisible_by_4_h = x_noisy.shape[2]%4==0
+        divisible_by_4_w =  x_noisy.shape[3]%4==0
+        if not (divisible_by_4_h and divisible_by_4_w):
+            #logger.warn(f"{x_noisy.shape} not divisible by 4!")
+            new_h = (x_noisy.shape[2]//4)*4
+            new_w = (x_noisy.shape[3]//4)*4
+            if not divisible_by_4_h:
+                new_h += 4
+            if not divisible_by_4_w:
+                new_w += 4
+            self.latent_dims_div4 = (new_h, new_w)
+        # prepare mask
+        self.prepare_mask_cond_hint(x_noisy=x_noisy, t=t, cond=cond, batched_number=batched_number)
+        # done preparing; model patches will take care of everything now
+        set_model_attn1_patch(transformer_options, self.patch_attn1.set_control(self))
+        set_model_attn2_patch(transformer_options, self.patch_attn2.set_control(self))
+        # return normal controlnet stuff
+        return control_prev
+    
+    def get_models(self):
+        to_return: list = super().get_models()
+        to_return.append(self.control_model_wrapped)
+        return to_return
+
+    def cleanup_advanced(self):
+        super().cleanup_advanced()
+        self.patch_attn1.cleanup()
+        self.patch_attn2.cleanup()
+        self.latent_dims_div2 = None
+        self.latent_dims_div4 = None
+    
+    def copy(self):
+        c = ControlLLLiteAdvanced(self.patch_attn1, self.patch_attn2, self.timestep_keyframes, self.device, self.ops)
+        self.copy_to(c)
+        self.copy_to_advanced(c)
+        return c
+
+
+class AnimaLLLiteAdvancedPatch:
+    def __init__(self, model_patch, control: 'AnimaLLLiteAdvanced'=None):
+        self.model_patch = model_patch
+        self.control = control
+
+    def set_control(self, control: 'AnimaLLLiteAdvanced') -> 'AnimaLLLiteAdvancedPatch':
+        self.control = control
+        return self
+
+    def clone_with_control(self, control: 'AnimaLLLiteAdvanced') -> 'AnimaLLLiteAdvancedPatch':
+        return AnimaLLLiteAdvancedPatch(self.model_patch, control)
+
+    def __call__(self, args):
+        if not self.control.should_run():
+            return args
+
+        x = args["x"]
+        if x.shape[2] != 1:
+            raise ValueError(f"Anima LLLite only supports T=1, got T={x.shape[2]}")
+
+        target_height = x.shape[-2] * 8
+        target_width = x.shape[-1] * 8
+        image = self.control.prepare_batched_tensor(self.control.cond_hint_original, x.shape[0])[:, :3]
+        image = comfy.utils.common_upscale(image, target_width, target_height, "bicubic", crop="center").clamp(0.0, 1.0)
+        image = image.to(device=x.device, dtype=x.dtype) * 2.0 - 1.0
+
+        if self.model_patch.model.cond_in_channels == 4:
+            mask = self.control.weights.extras.get(AnimaLLLiteConst.INPAINT_MASK)
+            if mask is None:
+                raise ValueError(
+                    "Anima LLLite inpainting models require an inpaint mask. Connect a MASK to Anima LLLite Extras, "
+                    "connect its cn_extras output to Default Weights, then connect CN_WEIGHTS to weights_override on Apply Advanced ControlNet."
+                )
+            if mask.ndim == 3:
+                mask = mask.unsqueeze(1)
+            if mask.ndim != 4 or mask.shape[1] != 1:
+                raise ValueError(f"Anima LLLite mask must have one channel, got shape {tuple(mask.shape)}")
+            if image.shape[0] > 1:
+                mask = self.control.prepare_batched_tensor(mask, image.shape[0], except_one=False)
+            mask = comfy.utils.common_upscale(mask.float(), target_width, target_height, "nearest-exact", crop="center")
+            if mask.shape[0] != image.shape[0]:
+                if image.shape[0] % mask.shape[0] != 0:
+                    raise ValueError(f"Anima LLLite mask batch {mask.shape[0]} cannot be broadcast to image batch {image.shape[0]}")
+                mask = mask.repeat(image.shape[0] // mask.shape[0], 1, 1, 1)
+            mask = (mask >= 0.5).to(device=x.device, dtype=x.dtype)
+            if self.model_patch.model.inpaint_masked_input:
+                image = image * (mask < 0.5).to(image.dtype)
+            image = torch.cat((image, mask * 2.0 - 1.0), dim=1)
+
+        cond_emb = self.model_patch.model.encode_conditioning(image)
+        multiplier, weight_mask = self.control.prepare_multiplier(args["img"])
+        args["transformer_options"]["model_patch_data"][self] = (cond_emb, multiplier, weight_mask)
+        return args
+
+    def to(self, device_or_dtype):
+        return self
+
+    def models(self):
+        return [self.model_patch]
+
+
+class AnimaLLLiteAdvancedAttentionPatch:
+    def __init__(self, patch: AnimaLLLiteAdvancedPatch, targets):
+        self.patch = patch
+        self.targets = targets
+
+    def __call__(self, q, k, v, pe=None, attn_mask=None, extra_options=None):
+        patch_data = extra_options["model_patch_data"].get(self.patch)
+        if patch_data is None:
+            return {"q": q, "k": k, "v": v, "pe": pe, "attn_mask": attn_mask}
+
+        cond_emb, multiplier, weight_mask = patch_data
+        block_index = extra_options["block_index"]
+        strength = self.patch.control.get_block_strength(block_index, multiplier, weight_mask)
+        values = {"q": q, "k": k, "v": v}
+        for value_name, target in self.targets.items():
+            values[value_name] = self.patch.model_patch.model.apply(values[value_name], cond_emb, block_index, target, strength)
+        return {"q": values["q"], "k": values["k"], "v": values["v"], "pe": pe, "attn_mask": attn_mask}
+
+
+class AnimaLLLiteAdvancedMLPPatch:
+    def __init__(self, patch: AnimaLLLiteAdvancedPatch):
+        self.patch = patch
+
+    def __call__(self, args):
+        patch_data = args["transformer_options"]["model_patch_data"].get(self.patch)
+        if patch_data is None:
+            return args
+
+        cond_emb, multiplier, weight_mask = patch_data
+        block_index = args["transformer_options"]["block_index"]
+        strength = self.patch.control.get_block_strength(block_index, multiplier, weight_mask)
+        args["x"] = self.patch.model_patch.model.apply(args["x"], cond_emb, block_index, "mlp_layer1", strength)
+        return args
+
+
+class AnimaLLLiteAdvanced(ControlBase, AdvancedControlBase):
+    def __init__(self, model_patch, timestep_keyframes: TimestepKeyframeGroup):
+        ControlBase.__init__(self)
+        AdvancedControlBase.__init__(self, super(), timestep_keyframes=timestep_keyframes, weights_default=ControlWeights.controllllite())
+        self.model_patch = model_patch
+        self.patch = AnimaLLLiteAdvancedPatch(model_patch, self)
+        self.patch_attn1 = AnimaLLLiteAdvancedAttentionPatch(
+            self.patch,
+            {"q": "self_attn_q_proj", "k": "self_attn_k_proj", "v": "self_attn_v_proj"},
+        )
+        self.patch_attn2 = AnimaLLLiteAdvancedAttentionPatch(self.patch, {"q": "cross_attn_q_proj"})
+        self.patch_mlp = AnimaLLLiteAdvancedMLPPatch(self.patch)
+
+    def prepare_batched_tensor(self, tensor: Tensor, target_batch: int, except_one=True) -> Tensor:
+        if self.sub_idxs is not None:
+            if tensor.shape[0] < self.full_latent_length:
+                tensor = extend_to_batch_size(tensor, self.full_latent_length)
+            tensor = tensor[self.sub_idxs]
+        if tensor.shape[0] != target_batch:
+            tensor = broadcast_image_to_extend(tensor, target_batch, self.batched_number, except_one=except_one)
+        return tensor
+
+    def prepare_effect_mask(self, mask: Tensor, img: Tensor) -> Tensor:
+        mask = self.prepare_batched_tensor(mask, img.shape[0], except_one=False)
+        mask = torch.nn.functional.interpolate(
+            mask.reshape((-1, 1, mask.shape[-2], mask.shape[-1])).float(),
+            size=(img.shape[2], img.shape[3]),
+            mode="bilinear",
+        )
+        return mask.flatten(2).transpose(1, 2).to(device=img.device, dtype=img.dtype)
+
+    def prepare_multiplier(self, img: Tensor):
+        multiplier = self.strength * self._current_timestep_keyframe.strength
+        token_multiplier = None
+
+        masks = [self.mask_cond_hint_original, self._current_timestep_keyframe.mask_hint_orig]
+        for mask in masks:
+            if mask is not None:
+                mask_multiplier = self.prepare_effect_mask(mask, img)
+                token_multiplier = mask_multiplier if token_multiplier is None else token_multiplier * mask_multiplier
+
+        flat_img = img.flatten(1, 3)
+        if self.latent_keyframes is not None:
+            latent_multiplier = self.calc_latent_keyframe_mults(flat_img, self.batched_number)
+            token_multiplier = latent_multiplier if token_multiplier is None else token_multiplier * latent_multiplier
+
+        if self.weights.has_uncond_multiplier and self.cond_or_uncond is not None:
+            batch_multiplier = torch.ones((img.shape[0], 1, 1), dtype=img.dtype, device=img.device)
+            actual_length = img.shape[0] // self.batched_number
+            for idx, cond_type in enumerate(self.cond_or_uncond):
+                if cond_type == 1:
+                    batch_multiplier[actual_length * idx:actual_length * (idx + 1)] *= self.weights.uncond_multiplier
+            token_multiplier = batch_multiplier if token_multiplier is None else token_multiplier * batch_multiplier
+
+        weight_mask = None
+        if self.weights.weight_mask is not None:
+            weight_mask = self.prepare_effect_mask(self.weights.weight_mask, img)
+
+        if token_multiplier is not None:
+            multiplier = token_multiplier * multiplier
+        return multiplier, weight_mask
+
+    def get_block_strength(self, block_index: int, multiplier, weight_mask):
+        block_weight = 1.0
+        if self.weights.weight_type == "universal":
+            exponent = self.model_patch.model.block_count - block_index
+            if weight_mask is not None:
+                block_weight = torch.pow(weight_mask, exponent)
+            else:
+                block_weight = self.weights.base_multiplier ** exponent
+        elif self.weights.weights_input is not None and block_index < len(self.weights.weights_input):
+            block_weight = self.weights.weights_input[block_index]
+        return multiplier * block_weight
+
+    def pre_run_advanced(self, *args, **kwargs):
+        AdvancedControlBase.pre_run_advanced(self, *args, **kwargs)
+        self.patch.set_control(self)
+
+    def get_control_advanced(self, x_noisy: Tensor, t, cond, batched_number: int, transformer_options: dict):
+        control_prev = None
+        if self.previous_controlnet is not None:
+            control_prev = self.previous_controlnet.get_control(x_noisy, t, cond, batched_number, transformer_options)
+        if not self.should_run():
+            return control_prev
+
+        set_model_patch(transformer_options, self.patch.set_control(self), "post_input")
+        set_model_attn1_patch(transformer_options, self.patch_attn1)
+        set_model_attn2_patch(transformer_options, self.patch_attn2)
+        set_model_patch(transformer_options, self.patch_mlp, "mlp_patch")
+        return control_prev
+
+    def get_models(self):
+        models = super().get_models()
+        models.append(self.model_patch)
+        return models
+
+    def copy(self):
+        copied = AnimaLLLiteAdvanced(self.model_patch, self.timestep_keyframes)
+        self.copy_to(copied)
+        self.copy_to_advanced(copied)
+        return copied
+
+
+def load_anima_lllite(ckpt_path: str, controlnet_data: dict[str, Tensor]=None, metadata=None, timestep_keyframe: TimestepKeyframeGroup=None):
+    if comfy_anima_lllite is None:
+        raise RuntimeError("Anima LLLite requires a newer version of ComfyUI. Please update ComfyUI.")
+    if controlnet_data is None or metadata is None:
+        loaded_data, loaded_metadata = comfy.utils.load_torch_file(ckpt_path, safe_load=True, return_metadata=True)
+        if controlnet_data is None:
+            controlnet_data = loaded_data
+        metadata = loaded_metadata
+
+    dtype = comfy.utils.weight_dtype(controlnet_data)
+    model = comfy_anima_lllite.AnimaLLLite(
+        controlnet_data,
+        metadata,
+        device=comfy.model_management.unet_offload_device(),
+        dtype=dtype,
+        operations=comfy.ops.manual_cast,
+    )
+    patcher_type = getattr(comfy.model_patcher, "CoreModelPatcher", ModelPatcher)
+    model_patcher = patcher_type(
+        model,
+        load_device=comfy.model_management.get_torch_device(),
+        offload_device=comfy.model_management.unet_offload_device(),
+    )
+    is_dynamic = getattr(model_patcher, "is_dynamic", lambda: False)()
+    model.load_state_dict(controlnet_data, assign=is_dynamic)
+    return AnimaLLLiteAdvanced(model_patcher, timestep_keyframe)
+
+
+def load_controllllite(ckpt_path: str, controlnet_data: dict[str, Tensor]=None, timestep_keyframe: TimestepKeyframeGroup=None):
+    if controlnet_data is None:
+        controlnet_data = comfy.utils.load_torch_file(ckpt_path, safe_load=True)
+    # adapted from https://github.com/kohya-ss/ControlNet-LLLite-ComfyUI
+    # first, split weights for each module
+    module_weights = {}
+    for key, value in controlnet_data.items():
+        fragments = key.split(".")
+        module_name = fragments[0]
+        weight_name = ".".join(fragments[1:])
+
+        if module_name not in module_weights:
+            module_weights[module_name] = {}
+        module_weights[module_name][weight_name] = value
+
+    unet_dtype = comfy.model_management.unet_dtype()
+    load_device = comfy.model_management.get_torch_device()
+    manual_cast_dtype = comfy.model_management.unet_manual_cast(unet_dtype, load_device)
+    ops = comfy.ops.disable_weight_init
+    if manual_cast_dtype is not None:
+        ops = comfy.ops.manual_cast
+
+    # next, load each module
+    modules = {}
+    for module_name, weights in module_weights.items():
+        # kohya planned to do something about how these should be chosen, so I'm not touching this
+        # since I am not familiar with the logic for this
+        if "conditioning1.4.weight" in weights:
+            depth = 3
+        elif weights["conditioning1.2.weight"].shape[-1] == 4:
+            depth = 2
+        else:
+            depth = 1
+
+        module = LLLiteModule(
+            name=module_name,
+            is_conv2d=weights["down.0.weight"].ndim == 4,
+            in_dim=weights["down.0.weight"].shape[1],
+            depth=depth,
+            cond_emb_dim=weights["conditioning1.0.weight"].shape[0] * 2,
+            mlp_dim=weights["down.0.weight"].shape[0],
+        )
+        # load weights into module
+        module.load_state_dict(weights)
+        modules[module_name] = module.to(dtype=unet_dtype)
+        if len(modules) == 1:
+            module.is_first = True
+
+    #logger.info(f"loaded {ckpt_path} successfully, {len(modules)} modules")
+
+    patch_attn1 = LLLitePatch(modules=modules, patch_type=LLLitePatch.ATTN1)
+    patch_attn2 = LLLitePatch(modules=modules, patch_type=LLLitePatch.ATTN2)
+    control = ControlLLLiteAdvanced(patch_attn1=patch_attn1, patch_attn2=patch_attn2, timestep_keyframes=timestep_keyframe, device=load_device, ops=ops)
+    return control

adv_control/control_plusplus.py

@@ -0,0 +1,486 @@
+# Code ported and modified from the diffusers ControlNetPlus repo by Qi Xin:
+# https://github.com/xinsir6/ControlNetPlus/blob/main/models/controlnet_union.py
+from typing import Union
+
+import os
+import torch
+import torch as th
+import torch.nn as nn
+from torch import Tensor
+from collections import OrderedDict
+
+
+from comfy.ldm.modules.diffusionmodules.util import (zero_module, timestep_embedding)
+
+from comfy.cldm.cldm import ControlNet as ControlNetCLDM
+import comfy.cldm.cldm
+from comfy.controlnet import ControlNet
+#from comfy.t2i_adapter.adapter import ResidualAttentionBlock
+from comfy.ldm.modules.attention import optimized_attention
+import comfy.ops
+import comfy.model_base
+import comfy.model_management
+import comfy.model_detection
+import comfy.utils
+
+from .utils import (AdvancedControlBase, ControlWeights, ControlWeightType, TimestepKeyframeGroup, AbstractPreprocWrapper, Extras,
+                    extend_to_batch_size, broadcast_image_to_extend)
+from .logger import logger
+
+
+class PlusPlusType:
+    OPENPOSE = "openpose"
+    DEPTH = "depth"
+    THICKLINE = "hed/pidi/scribble/ted"
+    THINLINE = "canny/lineart/mlsd"
+    NORMAL = "normal"
+    SEGMENT = "segment"
+    TILE = "tile"
+    REPAINT = "inpaint/outpaint"
+    NONE = "none"
+    _LIST_WITH_NONE = [OPENPOSE, DEPTH, THICKLINE, THINLINE, NORMAL, SEGMENT, TILE, REPAINT, NONE]
+    _LIST = [OPENPOSE, DEPTH, THICKLINE, THINLINE, NORMAL, SEGMENT, TILE, REPAINT]
+    _DICT = {OPENPOSE: 0, DEPTH: 1, THICKLINE: 2, THINLINE: 3, NORMAL: 4, SEGMENT: 5, TILE: 6, REPAINT: 7, NONE: -1}
+
+    @classmethod
+    def to_idx(cls, control_type: str):
+        try:
+            return cls._DICT[control_type]
+        except KeyError:
+            raise Exception(f"Unknown control type '{control_type}'.")
+
+
+class PlusPlusInput:
+    def __init__(self, image: Tensor, control_type: str, strength: float):
+        self.image = image
+        self.control_type = control_type
+        self.strength = strength
+
+    def clone(self):
+        return PlusPlusInput(self.image, self.control_type, self.strength)
+
+
+class PlusPlusInputGroup:
+    def __init__(self):
+        self.controls: dict[str, PlusPlusInput] = {}
+    
+    def add(self, pp_input: PlusPlusInput):
+        if pp_input.control_type in self.controls:
+            raise Exception(f"Control type '{pp_input.control_type}' is already present; ControlNet++ does not allow more than 1 of each type.")
+        self.controls[pp_input.control_type] = pp_input
+    
+    def clone(self) -> 'PlusPlusInputGroup':
+        cloned = PlusPlusInputGroup()
+        for key, value in self.controls.items():
+            cloned.controls[key] = value.clone()
+        return cloned
+
+
+class PlusPlusImageWrapper(AbstractPreprocWrapper):
+    error_msg = error_msg = "Invalid use of ControlNet++ Image Wrapper. The output of ControlNet++ Image Wrapper is NOT a usual image, but an object holding the images and extra info - you must connect the output directly to an Apply Advanced ControlNet node. It cannot be used for anything else that accepts IMAGE input."
+    def __init__(self, condhint: PlusPlusInputGroup):
+        super().__init__(condhint)
+        # just an IDE type hint
+        self.condhint: PlusPlusInputGroup
+
+    def movedim(self, source: int, destination: int):
+        condhint = self.condhint.clone()
+        for pp_input in condhint.controls.values():
+            pp_input.image = pp_input.image.movedim(source, destination)
+        return PlusPlusImageWrapper(condhint)
+
+# parts taken from comfy/cldm/cldm.py
+class OptimizedAttention(nn.Module):
+    def __init__(self, c, nhead, dropout=0.0, dtype=None, device=None, operations=None):
+        super().__init__()
+        self.heads = nhead
+        self.c = c
+
+        self.in_proj = operations.Linear(c, c * 3, bias=True, dtype=dtype, device=device)
+        self.out_proj = operations.Linear(c, c, bias=True, dtype=dtype, device=device)
+
+    def forward(self, x):
+        x = self.in_proj(x)
+        q, k, v = x.split(self.c, dim=2)
+        out = optimized_attention(q, k, v, self.heads)
+        return self.out_proj(out)
+
+class QuickGELU(nn.Module):
+    def forward(self, x: torch.Tensor):
+        return x * torch.sigmoid(1.702 * x)
+
+class ResBlockUnionControlnet(nn.Module):
+    def __init__(self, dim, nhead, dtype=None, device=None, operations=None):
+        super().__init__()
+        self.attn = OptimizedAttention(dim, nhead, dtype=dtype, device=device, operations=operations)
+        self.ln_1 = operations.LayerNorm(dim, dtype=dtype, device=device)
+        self.mlp = nn.Sequential(
+            OrderedDict([("c_fc", operations.Linear(dim, dim * 4, dtype=dtype, device=device)), ("gelu", QuickGELU()),
+                         ("c_proj", operations.Linear(dim * 4, dim, dtype=dtype, device=device))]))
+        self.ln_2 = operations.LayerNorm(dim, dtype=dtype, device=device)
+
+    def attention(self, x: torch.Tensor):
+        return self.attn(x)
+
+    def forward(self, x: torch.Tensor):
+        x = x + self.attention(self.ln_1(x))
+        x = x + self.mlp(self.ln_2(x))
+        return x
+
+
+class ControlAddEmbeddingAdv(nn.Module):
+    def __init__(self, in_dim, out_dim, num_control_type, dtype=None, device=None, operations: comfy.ops.disable_weight_init=None):
+        super().__init__()
+        self.num_control_type = num_control_type
+        self.in_dim = in_dim
+        self.linear_1 = operations.Linear(in_dim * num_control_type, out_dim, dtype=dtype, device=device)
+        self.linear_2 = operations.Linear(out_dim, out_dim, dtype=dtype, device=device)
+
+    def forward(self, control_type, dtype, device):
+        if control_type is None:
+            control_type = torch.zeros((self.num_control_type,), device=device)
+        c_type = timestep_embedding(control_type.flatten(), self.in_dim, repeat_only=False).to(dtype).reshape((-1, self.num_control_type * self.in_dim))
+        return self.linear_2(torch.nn.functional.silu(self.linear_1(c_type)))
+
+
+class ControlNetPlusPlus(ControlNetCLDM):
+    def __init__(self, *args,**kwargs):
+        super().__init__(*args, **kwargs)
+
+        operations: comfy.ops.disable_weight_init = kwargs.get("operations", comfy.ops.disable_weight_init)
+        device = kwargs.get("device", None)
+
+        time_embed_dim = self.model_channels * 4
+        control_add_embed_dim = 256
+
+        self.control_add_embedding = ControlAddEmbeddingAdv(control_add_embed_dim, time_embed_dim, self.num_control_type, dtype=self.dtype, device=device, operations=operations)
+
+    def union_controlnet_merge(self, hint: list[Tensor], control_type, emb, context):
+        # Equivalent to: https://github.com/xinsir6/ControlNetPlus/tree/main
+        indexes = torch.nonzero(control_type[0])
+        inputs = []
+        condition_list = []
+
+        for idx in range(indexes.shape[0]):
+            controlnet_cond = self.input_hint_block(hint[indexes[idx][0]], emb, context)
+            feat_seq = torch.mean(controlnet_cond, dim=(2, 3))
+            if idx < indexes.shape[0]:
+                feat_seq += self.task_embedding[indexes[idx][0]].to(dtype=feat_seq.dtype, device=feat_seq.device)
+
+            inputs.append(feat_seq.unsqueeze(1))
+            condition_list.append(controlnet_cond)
+
+        x = torch.cat(inputs, dim=1)
+        x = self.transformer_layes(x)
+
+        controlnet_cond_fuser = None
+        for idx in range(indexes.shape[0]):
+            alpha = self.spatial_ch_projs(x[:, idx])
+            alpha = alpha.unsqueeze(-1).unsqueeze(-1)
+            o = condition_list[idx] + alpha
+            if controlnet_cond_fuser is None:
+                controlnet_cond_fuser = o
+            else:
+                controlnet_cond_fuser += o
+        return controlnet_cond_fuser
+
+    def forward(self, x: Tensor, hint: list[Tensor], timesteps, context, y: Tensor=None, **kwargs):
+        t_emb = timestep_embedding(timesteps, self.model_channels, repeat_only=False).to(x.dtype)
+        emb = self.time_embed(t_emb)
+
+        guided_hint = None
+        if self.control_add_embedding is not None:
+            control_type = kwargs.get("control_type", None)
+
+            emb += self.control_add_embedding(control_type, emb.dtype, emb.device)
+            if control_type is not None:
+                guided_hint = self.union_controlnet_merge(hint, control_type, emb, context)
+
+        if guided_hint is None:
+            guided_hint = self.input_hint_block(hint[0], emb, context)
+
+        out_output = []
+        out_middle = []
+
+        hs = []
+        if self.num_classes is not None:
+            assert y.shape[0] == x.shape[0]
+            emb = emb + self.label_emb(y)
+
+        h = x
+        for module, zero_conv in zip(self.input_blocks, self.zero_convs):
+            if guided_hint is not None:
+                h = module(h, emb, context)
+                h += guided_hint
+                guided_hint = None
+            else:
+                h = module(h, emb, context)
+            out_output.append(zero_conv(h, emb, context))
+
+        h = self.middle_block(h, emb, context)
+        out_middle.append(self.middle_block_out(h, emb, context))
+
+        return {"middle": out_middle, "output": out_output}
+
+
+class ControlNetPlusPlusAdvanced(ControlNet, AdvancedControlBase):
+    def __init__(self, control_model: ControlNetPlusPlus, timestep_keyframes: TimestepKeyframeGroup, global_average_pooling=False, load_device=None, manual_cast_dtype=None):
+        super().__init__(control_model=control_model, global_average_pooling=global_average_pooling, load_device=load_device, manual_cast_dtype=manual_cast_dtype)
+        AdvancedControlBase.__init__(self, super(), timestep_keyframes=timestep_keyframes, weights_default=ControlWeights.controlnet())
+        self.add_compatible_weight(ControlWeightType.CONTROLNETPLUSPLUS)
+        # for IDE type hint purposes
+        self.control_model: ControlNetPlusPlus
+        self.cond_hint_original: Union[PlusPlusImageWrapper, PlusPlusInputGroup]
+        self.cond_hint: list[Union[Tensor, None]]
+        self.cond_hint_shape: Tensor = None
+        self.cond_hint_types: Tensor = None
+        # in case it is using the single loader
+        self.single_control_type: str = None
+
+    def get_universal_weights(self) -> ControlWeights:
+        def cn_weights_func(idx: int, control: dict[str, list[Tensor]], key: str):
+            if key == "middle":
+                return 1.0 * self.weights.extras.get(Extras.MIDDLE_MULT, 1.0)
+            c_len = len(control[key])
+            raw_weights = [(self.weights.base_multiplier ** float((c_len) - i)) for i in range(c_len+1)]
+            raw_weights = raw_weights[:-1]
+            if key == "input":
+                raw_weights.reverse()
+            return raw_weights[idx]
+        return self.weights.copy_with_new_weights(new_weight_func=cn_weights_func)
+
+    def verify_control_type(self, model_name: str, pp_group: PlusPlusInputGroup=None):
+        if pp_group is not None:
+            for pp_input in pp_group.controls.values():
+                if PlusPlusType.to_idx(pp_input.control_type) >= self.control_model.num_control_type:
+                    raise Exception(f"ControlNet++ model '{model_name}' does not support control_type '{pp_input.control_type}'.")
+        if self.single_control_type is not None:
+            if PlusPlusType.to_idx(self.single_control_type) >= self.control_model.num_control_type:
+                raise Exception(f"ControlNet++ model '{model_name}' does not support control_type '{self.single_control_type}'.")
+
+    def set_cond_hint_inject(self, *args, **kwargs):
+        to_return = super().set_cond_hint_inject(*args, **kwargs)
+        # if not single_control_type, expect PlusPlusImageWrapper
+        if self.single_control_type is None:
+            # check that cond_hint is wrapped, and unwrap it
+            if type(self.cond_hint_original) != PlusPlusImageWrapper:
+                raise Exception("ControlNet++ (Multi) expects image input from the Load ControlNet++ Model node, NOT from anything else. Images are provided to that node via ControlNet++ Input nodes.")
+            self.cond_hint_original = self.cond_hint_original.condhint.clone()
+        # otherwise, expect single image input (AKA, usual controlnet input)
+        else:
+            # check that cond_hint is not a PlusPlusImageWrapper
+            if type(self.cond_hint_original) == PlusPlusImageWrapper:
+                raise Exception("ControlNet++ (Single) expects usual image input, NOT the image input from a Load ControlNet++ Model (Multi) node.")
+            pp_group = PlusPlusInputGroup()
+            pp_input = PlusPlusInput(self.cond_hint_original, self.single_control_type, 1.0)
+            pp_group.add(pp_input)
+            self.cond_hint_original = pp_group
+        return to_return
+
+    def get_control_advanced(self, x_noisy: Tensor, t, cond, batched_number, transformer_options):
+        control_prev = None
+        if self.previous_controlnet is not None:
+            control_prev = self.previous_controlnet.get_control(x_noisy, t, cond, batched_number, transformer_options)
+
+        if self.timestep_range is not None:
+            if t[0] > self.timestep_range[0] or t[0] < self.timestep_range[1]:
+                if control_prev is not None:
+                    return control_prev
+                else:
+                    return None
+
+        dtype = self.control_model.dtype
+        if self.manual_cast_dtype is not None:
+            dtype = self.manual_cast_dtype
+
+        output_dtype = x_noisy.dtype
+
+        # make all cond_hints appropriate dimensions
+        # TODO: change this to not require cond_hint upscaling every step when self.sub_idxs is present
+        if self.sub_idxs is not None or self.cond_hint is None or x_noisy.shape[2] * self.compression_ratio != self.cond_hint_shape[2] or x_noisy.shape[3] * self.compression_ratio != self.cond_hint_shape[3]:
+            if self.cond_hint is not None:
+                del self.cond_hint
+            self.cond_hint = [None] * self.control_model.num_control_type
+            self.cond_hint_types = torch.tensor([0.0] * self.control_model.num_control_type)
+            self.cond_hint_shape = None
+            compression_ratio = self.compression_ratio
+            # unlike normal controlnet, need to handle each input image tensor (for each type)
+            for pp_type, pp_input in self.cond_hint_original.controls.items():
+                pp_idx = PlusPlusType.to_idx(pp_type)
+                # if negative, means no type should be selected (single only)
+                if pp_idx < 0:
+                    pp_idx = 0
+                else:
+                    self.cond_hint_types[pp_idx] = pp_input.strength
+                # if self.cond_hint_original lengths greater or equal to latent count, subdivide
+                if self.sub_idxs is not None:
+                    actual_cond_hint_orig = pp_input.image
+                    if pp_input.image.size(0) < self.full_latent_length:
+                        actual_cond_hint_orig = extend_to_batch_size(tensor=actual_cond_hint_orig, batch_size=self.full_latent_length)
+                    self.cond_hint[pp_idx] = comfy.utils.common_upscale(actual_cond_hint_orig[self.sub_idxs], x_noisy.shape[3] * compression_ratio, x_noisy.shape[2] * compression_ratio, 'nearest-exact', "center")
+                else:
+                    self.cond_hint[pp_idx] = comfy.utils.common_upscale(pp_input.image, x_noisy.shape[3] * compression_ratio, x_noisy.shape[2] * compression_ratio, 'nearest-exact', "center")
+                self.cond_hint[pp_idx] = self.cond_hint[pp_idx].to(device=x_noisy.device, dtype=dtype)
+                self.cond_hint_shape = self.cond_hint[pp_idx].shape
+            # prepare cond_hint_controls to match batchsize
+            if self.cond_hint_types.count_nonzero() == 0:
+                self.cond_hint_types = None
+            else:
+                self.cond_hint_types = self.cond_hint_types.unsqueeze(0).to(device=x_noisy.device, dtype=dtype).repeat(x_noisy.shape[0], 1)
+        for i in range(len(self.cond_hint)):
+            if self.cond_hint[i] is not None:
+                if x_noisy.shape[0] != self.cond_hint[i].shape[0]:
+                    self.cond_hint[i] = broadcast_image_to_extend(self.cond_hint[i], x_noisy.shape[0], batched_number)
+        if self.cond_hint_types is not None and x_noisy.shape[0] != self.cond_hint_types.shape[0]:
+            self.cond_hint_types = broadcast_image_to_extend(self.cond_hint_types, x_noisy.shape[0], batched_number, False)
+        
+        # prepare mask_cond_hint
+        self.prepare_mask_cond_hint(x_noisy=x_noisy, t=t, cond=cond, batched_number=batched_number, dtype=dtype)
+
+        context = cond.get('crossattn_controlnet', cond['c_crossattn'])
+        y = cond.get('y', None)
+        if y is not None:
+            y = comfy.model_base.convert_tensor(y, dtype, x_noisy.device)
+        timestep = self.model_sampling_current.timestep(t)
+        x_noisy = self.model_sampling_current.calculate_input(t, x_noisy)
+
+        control = self.control_model(x=x_noisy.to(dtype), hint=self.cond_hint, timesteps=timestep.float(), context=comfy.model_management.cast_to_device(context, x_noisy.device, dtype), y=y, control_type=self.cond_hint_types)
+        return self.control_merge(control, control_prev, output_dtype)
+
+    def copy(self):
+        c = ControlNetPlusPlusAdvanced(self.control_model, self.timestep_keyframes, global_average_pooling=self.global_average_pooling, load_device=self.load_device, manual_cast_dtype=self.manual_cast_dtype)
+        self.copy_to(c)
+        self.copy_to_advanced(c)
+        c.single_control_type = self.single_control_type
+        return c
+
+
+def load_controlnetplusplus(ckpt_path: str, timestep_keyframe: TimestepKeyframeGroup=None, model=None):
+    controlnet_data = comfy.utils.load_torch_file(ckpt_path, safe_load=True)
+    # check that actually is ControlNet++ model
+    if "task_embedding" not in controlnet_data:
+        raise Exception(f"'{ckpt_path}' is not a valid ControlNet++ model.")
+
+    controlnet_config = None
+    supported_inference_dtypes = None
+
+    if "controlnet_cond_embedding.conv_in.weight" in controlnet_data: #diffusers format
+        controlnet_config = comfy.model_detection.unet_config_from_diffusers_unet(controlnet_data)
+        diffusers_keys = comfy.utils.unet_to_diffusers(controlnet_config)
+        diffusers_keys["controlnet_mid_block.weight"] = "middle_block_out.0.weight"
+        diffusers_keys["controlnet_mid_block.bias"] = "middle_block_out.0.bias"
+
+        count = 0
+        loop = True
+        while loop:
+            suffix = [".weight", ".bias"]
+            for s in suffix:
+                k_in = "controlnet_down_blocks.{}{}".format(count, s)
+                k_out = "zero_convs.{}.0{}".format(count, s)
+                if k_in not in controlnet_data:
+                    loop = False
+                    break
+                diffusers_keys[k_in] = k_out
+            count += 1
+
+        count = 0
+        loop = True
+        while loop:
+            suffix = [".weight", ".bias"]
+            for s in suffix:
+                if count == 0:
+                    k_in = "controlnet_cond_embedding.conv_in{}".format(s)
+                else:
+                    k_in = "controlnet_cond_embedding.blocks.{}{}".format(count - 1, s)
+                k_out = "input_hint_block.{}{}".format(count * 2, s)
+                if k_in not in controlnet_data:
+                    k_in = "controlnet_cond_embedding.conv_out{}".format(s)
+                    loop = False
+                diffusers_keys[k_in] = k_out
+            count += 1
+
+        new_sd = {}
+        for k in diffusers_keys:
+            if k in controlnet_data:
+                new_sd[diffusers_keys[k]] = controlnet_data.pop(k)
+        
+        if "control_add_embedding.linear_1.bias" in controlnet_data: #Union Controlnet
+            controlnet_config["union_controlnet_num_control_type"] = controlnet_data["task_embedding"].shape[0]
+            for k in list(controlnet_data.keys()):
+                new_k = k.replace('.attn.in_proj_', '.attn.in_proj.')
+                new_sd[new_k] = controlnet_data.pop(k)
+
+        leftover_keys = controlnet_data.keys()
+        if len(leftover_keys) > 0:
+            logger.warning("leftover ControlNet++ keys: {}".format(leftover_keys))
+        controlnet_data = new_sd
+    elif "controlnet_blocks.0.weight" in controlnet_data: #SD3 diffusers format
+        raise Exception("Unexpected SD3 diffusers format for ControlNet++ model. Something is very wrong.")
+
+    pth_key = 'control_model.zero_convs.0.0.weight'
+    pth = False
+    key = 'zero_convs.0.0.weight'
+    if pth_key in controlnet_data:
+        pth = True
+        key = pth_key
+        prefix = "control_model."
+    elif key in controlnet_data:
+        prefix = ""
+    else:
+        raise Exception("Unexpected T2IAdapter format for ControlNet++ model. Something is very wrong.")
+
+    if controlnet_config is None:
+        model_config = comfy.model_detection.model_config_from_unet(controlnet_data, prefix, True)
+        supported_inference_dtypes = model_config.supported_inference_dtypes
+        controlnet_config = model_config.unet_config
+
+    load_device = comfy.model_management.get_torch_device()
+    if supported_inference_dtypes is None:
+        unet_dtype = comfy.model_management.unet_dtype()
+    else:
+        unet_dtype = comfy.model_management.unet_dtype(supported_dtypes=supported_inference_dtypes)
+
+    manual_cast_dtype = comfy.model_management.unet_manual_cast(unet_dtype, load_device)
+    if manual_cast_dtype is not None:
+        controlnet_config["operations"] = comfy.ops.manual_cast
+    controlnet_config["dtype"] = unet_dtype
+    controlnet_config.pop("out_channels")
+    controlnet_config["hint_channels"] = controlnet_data["{}input_hint_block.0.weight".format(prefix)].shape[1]
+    control_model = ControlNetPlusPlus(**controlnet_config)
+
+    if pth:
+        if 'difference' in controlnet_data:
+            if model is not None:
+                comfy.model_management.load_models_gpu([model])
+                model_sd = model.model_state_dict()
+                for x in controlnet_data:
+                    c_m = "control_model."
+                    if x.startswith(c_m):
+                        sd_key = "diffusion_model.{}".format(x[len(c_m):])
+                        if sd_key in model_sd:
+                            cd = controlnet_data[x]
+                            cd += model_sd[sd_key].type(cd.dtype).to(cd.device)
+            else:
+                logger.warning("WARNING: Loaded a diff controlnet without a model. It will very likely not work.")
+
+        class WeightsLoader(torch.nn.Module):
+            pass
+        w = WeightsLoader()
+        w.control_model = control_model
+        missing, unexpected = w.load_state_dict(controlnet_data, strict=False)
+    else:
+        missing, unexpected = control_model.load_state_dict(controlnet_data, strict=False)
+
+    if len(missing) > 0:
+        logger.warning("missing ControlNet++ keys: {}".format(missing))
+
+    if len(unexpected) > 0:
+        logger.debug("unexpected ControlNet++ keys: {}".format(unexpected))
+
+    global_average_pooling = False
+    filename = os.path.splitext(ckpt_path)[0]
+    if filename.endswith("_shuffle") or filename.endswith("_shuffle_fp16"): #TODO: smarter way of enabling global_average_pooling
+        global_average_pooling = True
+
+    control = ControlNetPlusPlusAdvanced(control_model, timestep_keyframes=timestep_keyframe, global_average_pooling=global_average_pooling, load_device=load_device, manual_cast_dtype=manual_cast_dtype)
+    return control

adv_control/control_svd.py

@@ -311,7 +311,8 @@ class SVDControlNet(nn.Module):
 
         guided_hint = self.input_hint_block(hint, emb, context, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator)
 
-        outs = []
+        out_output = []
+        out_middle = []
 
         hs = []
         if self.num_classes is not None:
@@ -326,12 +327,12 @@ class SVDControlNet(nn.Module):
                 guided_hint = None
             else:
                 h = module(h, emb, context, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator)
-            outs.append(zero_conv(h, emb, context, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator))
+            out_output.append(zero_conv(h, emb, context, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator))
 
         h = self.middle_block(h, emb, context, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator)
-        outs.append(self.middle_block_out(h, emb, context, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator))
+        out_middle.append(self.middle_block_out(h, emb, context, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator))
 
-        return outs
+        return {"middle": out_middle, "output": out_output}
 
 
 TEMPORAL_TRANSFORMER_BLOCKS = {

adv_control/dinklink.py

@@ -0,0 +1,112 @@
+####################################################################################################
+# DinkLink is my method of sharing classes/functions between my nodes.
+#
+# My DinkLink-compatible nodes will inject comfy.hooks with a __DINKLINK attr
+# that stores a dictionary, where any of my node packs can store their stuff.
+#
+# It is not intended to be accessed by node packs that I don't develop, so things may change
+# at any time.
+#
+# DinkLink also serves as a proof-of-concept for a future ComfyUI implementation of
+# purposely exposing node pack classes/functions with other node packs.
+####################################################################################################
+from __future__ import annotations
+from typing import Union
+from torch import Tensor, nn
+
+from comfy.model_patcher import ModelPatcher
+import comfy.hooks
+
+DINKLINK = "__DINKLINK"
+
+
+def init_dinklink():
+    create_dinklink()
+    prepare_dinklink()
+
+def create_dinklink():
+    if not hasattr(comfy.hooks, DINKLINK):
+        setattr(comfy.hooks, DINKLINK, {})
+
+def get_dinklink() -> dict[str, dict[str]]:
+    create_dinklink()
+    return getattr(comfy.hooks, DINKLINK)
+
+
+class DinkLinkConst:
+    VERSION = "version"
+    # ADE
+    ADE = "ADE"
+    ADE_ANIMATEDIFFMODEL = "AnimateDiffModel"
+    ADE_ANIMATEDIFFINFO = "AnimateDiffInfo"
+    ADE_CREATE_MOTIONMODELPATCHER = "create_MotionModelPatcher"
+
+def prepare_dinklink():
+    pass
+
+
+class InterfaceAnimateDiffInfo:
+    '''Class only used for IDE type hints; interface of ADE's AnimateDiffInfo'''
+    def __init__(self, sd_type: str, mm_format: str, mm_version: str, mm_name: str):
+        self.sd_type = sd_type
+        self.mm_format = mm_format
+        self.mm_version = mm_version
+        self.mm_name = mm_name
+
+
+class InterfaceAnimateDiffModel(nn.Module):
+    '''Class only used for IDE type hints; interface of ADE's AnimateDiffModel'''
+    def __init__(self, mm_state_dict: dict[str, Tensor], mm_info: InterfaceAnimateDiffInfo, init_kwargs: dict[str]={}):
+        pass
+
+    def set_video_length(self, video_length: int, full_length: int) -> None:
+        raise NotImplemented()
+
+    def set_scale(self, scale: Union[float, Tensor, None], per_block_list: Union[list, None]=None) -> None:
+        raise NotImplemented()
+
+    def set_effect(self, multival: Union[float, Tensor, None], per_block_list: Union[list, None]=None) -> None:
+        raise NotImplemented()
+
+    def cleanup(self):
+        raise NotImplemented()
+
+    def inject(self, model: ModelPatcher):
+        pass
+
+    def eject(self, model: ModelPatcher):
+        pass
+
+
+def get_CreateMotionModelPatcher(throw_exception=True):
+    d = get_dinklink()
+    try:
+        link_ade = d[DinkLinkConst.ADE]
+        return link_ade[DinkLinkConst.ADE_CREATE_MOTIONMODELPATCHER]
+    except KeyError:
+        if throw_exception:
+            raise Exception("Could not get create_MotionModelPatcher function. AnimateDiff-Evolved nodes need to be installed to use SparseCtrl; " + \
+                            "they are either not installed or are of an insufficient version.")
+    return None
+
+def get_AnimateDiffModel(throw_exception=True):
+    d = get_dinklink()
+    try:
+        link_ade = d[DinkLinkConst.ADE]
+        return link_ade[DinkLinkConst.ADE_ANIMATEDIFFMODEL]
+    except KeyError:
+        if throw_exception:
+            raise Exception("Could not get AnimateDiffModel class. AnimateDiff-Evolved nodes need to be installed to use SparseCtrl; " + \
+                            "they are either not installed or are of an insufficient version.")
+    return None
+
+def get_AnimateDiffInfo(throw_exception=True) -> InterfaceAnimateDiffInfo:
+    d = get_dinklink()
+    try:
+        link_ade = d[DinkLinkConst.ADE]
+        return link_ade[DinkLinkConst.ADE_ANIMATEDIFFINFO]
+    except KeyError:
+        if throw_exception:
+            raise Exception("Could not get AnimateDiffInfo class - AnimateDiff-Evolved nodes need to be installed to use SparseCtrl; " + \
+                            "they are either not installed or are of an insufficient version.")
+    return None

adv_control/nodes.py

@@ -1,235 +1,73 @@
-import numpy as np
+from comfy_api.latest import ComfyExtension, io
-from torch import Tensor
 
-import folder_paths
+from .nodes_main import (ControlNetLoaderAdvanced, DiffControlNetLoaderAdvanced, AnimaLLLiteLoaderAdvanced,
-from comfy.model_patcher import ModelPatcher
+                         AdvancedControlNetApply, AdvancedControlNetApplySingle)
-
+from .nodes_weight import (DefaultWeights, ScaledSoftMaskedUniversalWeights, ScaledSoftUniversalWeights,
-from .control import load_controlnet, convert_to_advanced, is_advanced_controlnet
+                           SoftControlNetWeightsSD15, CustomControlNetWeightsSD15, CustomControlNetWeightsFlux,
-from .utils import ControlWeights, LatentKeyframeGroup, TimestepKeyframeGroup, BIGMAX
+                           CustomControlNetWeightsAnima, SoftT2IAdapterWeights, CustomT2IAdapterWeights, ExtrasMiddleMultNode,
-from .nodes_weight import (DefaultWeights, ScaledSoftMaskedUniversalWeights, ScaledSoftUniversalWeights, SoftControlNetWeights, CustomControlNetWeights,
+                           AnimaLLLiteExtras)
-    SoftT2IAdapterWeights, CustomT2IAdapterWeights)
 from .nodes_keyframes import (LatentKeyframeGroupNode, LatentKeyframeInterpolationNode, LatentKeyframeBatchedGroupNode, LatentKeyframeNode,
                               TimestepKeyframeNode, TimestepKeyframeInterpolationNode, TimestepKeyframeFromStrengthListNode)
-from .nodes_sparsectrl import SparseCtrlMergedLoaderAdvanced, SparseCtrlLoaderAdvanced, SparseIndexMethodNode, SparseSpreadMethodNode, RgbSparseCtrlPreprocessor
+from .nodes_sparsectrl import SparseCtrlMergedLoaderAdvanced, SparseCtrlLoaderAdvanced, SparseIndexMethodNode, SparseSpreadMethodNode, RgbSparseCtrlPreprocessor, SparseWeightExtras
 from .nodes_reference import ReferenceControlNetNode, ReferenceControlFinetune, ReferencePreprocessorNode
-from .nodes_loosecontrol import ControlNetLoaderWithLoraAdvanced
+from .nodes_plusplus import PlusPlusLoaderAdvanced, PlusPlusLoaderSingle, PlusPlusInputNode
-from .nodes_deprecated import LoadImagesFromDirectory
+from .nodes_ctrlora import CtrLoRALoader
-from .logger import logger
+from .nodes_deprecated import (LoadImagesFromDirectory, ScaledSoftUniversalWeightsDeprecated,
+                               SoftControlNetWeightsDeprecated, CustomControlNetWeightsDeprecated, 
+                               SoftT2IAdapterWeightsDeprecated, CustomT2IAdapterWeightsDeprecated,
+                               AdvancedControlNetApplyDEPR, AdvancedControlNetApplySingleDEPR,
+                               ControlNetLoaderAdvancedDEPR, DiffControlNetLoaderAdvancedDEPR)
 
 
-class ControlNetLoaderAdvanced:
-    @classmethod
-    def INPUT_TYPES(s):
-        return {
-            "required": {
-                "control_net_name": (folder_paths.get_filename_list("controlnet"), ),
-            },
-            "optional": {
-                "timestep_keyframe": ("TIMESTEP_KEYFRAME", ),
-            }
-        }
-
-    RETURN_TYPES = ("CONTROL_NET", )
-    FUNCTION = "load_controlnet"
-
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝"
-
-    def load_controlnet(self, control_net_name,
-                        timestep_keyframe: TimestepKeyframeGroup=None
-                        ):
-        controlnet_path = folder_paths.get_full_path("controlnet", control_net_name)
-        controlnet = load_controlnet(controlnet_path, timestep_keyframe)
-        return (controlnet,)
 
 
-class DiffControlNetLoaderAdvanced:
+class AdvancedControlNetExtension(ComfyExtension):
-    @classmethod
+    async def get_node_list(self) -> list[type[io.ComfyNode]]:
-    def INPUT_TYPES(s):
+        return [
-        return {
+            TimestepKeyframeNode,
-            "required": {
+            TimestepKeyframeInterpolationNode,
-                "model": ("MODEL",),
+            TimestepKeyframeFromStrengthListNode,
-                "control_net_name": (folder_paths.get_filename_list("controlnet"), )
+            LatentKeyframeNode,
-            },
+            LatentKeyframeInterpolationNode,
-            "optional": {
+            LatentKeyframeBatchedGroupNode,
-                "timestep_keyframe": ("TIMESTEP_KEYFRAME", ),
+            LatentKeyframeGroupNode,
-            }
+            AdvancedControlNetApply,
-        }
+            AdvancedControlNetApplySingle,
-    
+            ControlNetLoaderAdvanced,
-    RETURN_TYPES = ("CONTROL_NET", )
+            DiffControlNetLoaderAdvanced,
-    FUNCTION = "load_controlnet"
+            AnimaLLLiteLoaderAdvanced,
-
+            ScaledSoftUniversalWeights,
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝"
+            ScaledSoftMaskedUniversalWeights,
-
+            SoftControlNetWeightsSD15,
-    def load_controlnet(self, control_net_name, model,
+            CustomControlNetWeightsSD15,
-                        timestep_keyframe: TimestepKeyframeGroup=None
+            CustomControlNetWeightsFlux,
-                        ):
+            CustomControlNetWeightsAnima,
-        controlnet_path = folder_paths.get_full_path("controlnet", control_net_name)
+            SoftT2IAdapterWeights,
-        controlnet = load_controlnet(controlnet_path, timestep_keyframe, model)
+            CustomT2IAdapterWeights,
-        if is_advanced_controlnet(controlnet):
+            DefaultWeights,
-            controlnet.verify_all_weights()
+            ExtrasMiddleMultNode,
-        return (controlnet,)
+            AnimaLLLiteExtras,
-
+            RgbSparseCtrlPreprocessor,
-
+            SparseCtrlLoaderAdvanced,
-class AdvancedControlNetApply:
+            SparseCtrlMergedLoaderAdvanced,
-    @classmethod
+            SparseIndexMethodNode,
-    def INPUT_TYPES(s):
+            SparseSpreadMethodNode,
-        return {
+            SparseWeightExtras,
-            "required": {
+            PlusPlusLoaderSingle,
-                "positive": ("CONDITIONING", ),
+            PlusPlusLoaderAdvanced,
-                "negative": ("CONDITIONING", ),
+            PlusPlusInputNode,
-                "control_net": ("CONTROL_NET", ),
+            CtrLoRALoader,
-                "image": ("IMAGE", ),
+            ReferencePreprocessorNode,
-                "strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.01}),
+            ReferenceControlNetNode,
-                "start_percent": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}),
+            ReferenceControlFinetune,
-                "end_percent": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001})
+            LoadImagesFromDirectory,
-            },
+            ScaledSoftUniversalWeightsDeprecated,
-            "optional": {
+            SoftControlNetWeightsDeprecated,
-                "mask_optional": ("MASK", ),
+            CustomControlNetWeightsDeprecated,
-                "timestep_kf": ("TIMESTEP_KEYFRAME", ),
+            SoftT2IAdapterWeightsDeprecated,
-                "latent_kf_override": ("LATENT_KEYFRAME", ),
+            CustomT2IAdapterWeightsDeprecated,
-                "weights_override": ("CONTROL_NET_WEIGHTS", ),
+            AdvancedControlNetApplyDEPR,
-                "model_optional": ("MODEL",),
+            AdvancedControlNetApplySingleDEPR,
-            }
+            ControlNetLoaderAdvancedDEPR,
-        }
+            DiffControlNetLoaderAdvancedDEPR
-
+        ]
-    RETURN_TYPES = ("CONDITIONING","CONDITIONING","MODEL",)
-    RETURN_NAMES = ("positive", "negative", "model_opt")
-    FUNCTION = "apply_controlnet"
-
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝"
-
-    def apply_controlnet(self, positive, negative, control_net, image, strength, start_percent, end_percent,
-                         mask_optional: Tensor=None, model_optional: ModelPatcher=None,
-                         timestep_kf: TimestepKeyframeGroup=None, latent_kf_override: LatentKeyframeGroup=None,
-                         weights_override: ControlWeights=None):
-        if strength == 0:
-            return (positive, negative, model_optional)
-        if model_optional:
-            model_optional = model_optional.clone()
-
-        control_hint = image.movedim(-1,1)
-        cnets = {}
-
-        out = []
-        for conditioning in [positive, negative]:
-            c = []
-            for t in conditioning:
-                d = t[1].copy()
-
-                prev_cnet = d.get('control', None)
-                if prev_cnet in cnets:
-                    c_net = cnets[prev_cnet]
-                else:
-                    # copy, convert to advanced if needed, and set cond
-                    c_net = convert_to_advanced(control_net.copy()).set_cond_hint(control_hint, strength, (start_percent, end_percent))
-                    if is_advanced_controlnet(c_net):
-                        # disarm node check
-                        c_net.disarm()
-                        # if model required, verify model is passed in, and if so patch it
-                        if c_net.require_model:
-                            if not model_optional:
-                                raise Exception(f"Type '{type(c_net).__name__}' requires model_optional input, but got None.")
-                            c_net.patch_model(model=model_optional)
-                        # apply optional parameters and overrides, if provided
-                        if timestep_kf is not None:
-                            c_net.set_timestep_keyframes(timestep_kf)
-                        if latent_kf_override is not None:
-                            c_net.latent_keyframe_override = latent_kf_override
-                        if weights_override is not None:
-                            c_net.weights_override = weights_override
-                        # verify weights are compatible
-                        c_net.verify_all_weights()
-                        # set cond hint mask
-                        if mask_optional is not None:
-                            mask_optional = mask_optional.clone()
-                            # if not in the form of a batch, make it so
-                            if len(mask_optional.shape) < 3:
-                                mask_optional = mask_optional.unsqueeze(0)
-                            c_net.set_cond_hint_mask(mask_optional)
-                    c_net.set_previous_controlnet(prev_cnet)
-                    cnets[prev_cnet] = c_net
-
-                d['control'] = c_net
-                d['control_apply_to_uncond'] = False
-                n = [t[0], d]
-                c.append(n)
-            out.append(c)
-        return (out[0], out[1], model_optional)
-
-
-# NODE MAPPING
-NODE_CLASS_MAPPINGS = {
-    # Keyframes
-    "TimestepKeyframe": TimestepKeyframeNode,
-    "ACN_TimestepKeyframeInterpolation": TimestepKeyframeInterpolationNode,
-    "ACN_TimestepKeyframeFromStrengthList": TimestepKeyframeFromStrengthListNode,
-    "LatentKeyframe": LatentKeyframeNode,
-    "LatentKeyframeTiming": LatentKeyframeInterpolationNode,
-    "LatentKeyframeBatchedGroup": LatentKeyframeBatchedGroupNode,
-    "LatentKeyframeGroup": LatentKeyframeGroupNode,
-    # Conditioning
-    "ACN_AdvancedControlNetApply": AdvancedControlNetApply,
-    # Loaders
-    "ControlNetLoaderAdvanced": ControlNetLoaderAdvanced,
-    "DiffControlNetLoaderAdvanced": DiffControlNetLoaderAdvanced,
-    # Weights
-    "ScaledSoftControlNetWeights": ScaledSoftUniversalWeights,
-    "ScaledSoftMaskedUniversalWeights": ScaledSoftMaskedUniversalWeights,
-    "SoftControlNetWeights": SoftControlNetWeights,
-    "CustomControlNetWeights": CustomControlNetWeights,
-    "SoftT2IAdapterWeights": SoftT2IAdapterWeights,
-    "CustomT2IAdapterWeights": CustomT2IAdapterWeights,
-    "ACN_DefaultUniversalWeights": DefaultWeights,
-    # SparseCtrl
-    "ACN_SparseCtrlRGBPreprocessor": RgbSparseCtrlPreprocessor,
-    "ACN_SparseCtrlLoaderAdvanced": SparseCtrlLoaderAdvanced,
-    "ACN_SparseCtrlMergedLoaderAdvanced": SparseCtrlMergedLoaderAdvanced,
-    "ACN_SparseCtrlIndexMethodNode": SparseIndexMethodNode,
-    "ACN_SparseCtrlSpreadMethodNode": SparseSpreadMethodNode,
-    # Reference
-    "ACN_ReferencePreprocessor": ReferencePreprocessorNode,
-    "ACN_ReferenceControlNet": ReferenceControlNetNode,
-    "ACN_ReferenceControlNetFinetune": ReferenceControlFinetune,
-    # LOOSEControl
-    #"ACN_ControlNetLoaderWithLoraAdvanced": ControlNetLoaderWithLoraAdvanced,
-    # Deprecated
-    "LoadImagesFromDirectory": LoadImagesFromDirectory,
-}
-
-NODE_DISPLAY_NAME_MAPPINGS = {
-    # Keyframes
-    "TimestepKeyframe": "Timestep Keyframe 🛂🅐🅒🅝",
-    "ACN_TimestepKeyframeInterpolation": "Timestep Keyframe Interpolation 🛂🅐🅒🅝",
-    "ACN_TimestepKeyframeFromStrengthList": "Timestep Keyframe From List 🛂🅐🅒🅝",
-    "LatentKeyframe": "Latent Keyframe 🛂🅐🅒🅝",
-    "LatentKeyframeTiming": "Latent Keyframe Interpolation 🛂🅐🅒🅝",
-    "LatentKeyframeBatchedGroup": "Latent Keyframe From List 🛂🅐🅒🅝",
-    "LatentKeyframeGroup": "Latent Keyframe Group 🛂🅐🅒🅝",
-    # Conditioning
-    "ACN_AdvancedControlNetApply": "Apply Advanced ControlNet 🛂🅐🅒🅝",
-    # Loaders
-    "ControlNetLoaderAdvanced": "Load Advanced ControlNet Model 🛂🅐🅒🅝",
-    "DiffControlNetLoaderAdvanced": "Load Advanced ControlNet Model (diff) 🛂🅐🅒🅝",
-    # Weights
-    "ScaledSoftControlNetWeights": "Scaled Soft Weights 🛂🅐🅒🅝",
-    "ScaledSoftMaskedUniversalWeights": "Scaled Soft Masked Weights 🛂🅐🅒🅝",
-    "SoftControlNetWeights": "ControlNet Soft Weights 🛂🅐🅒🅝",
-    "CustomControlNetWeights": "ControlNet Custom Weights 🛂🅐🅒🅝",
-    "SoftT2IAdapterWeights": "T2IAdapter Soft Weights 🛂🅐🅒🅝",
-    "CustomT2IAdapterWeights": "T2IAdapter Custom Weights 🛂🅐🅒🅝",
-    "ACN_DefaultUniversalWeights": "Force Default Weights 🛂🅐🅒🅝",
-    # SparseCtrl
-    "ACN_SparseCtrlRGBPreprocessor": "RGB SparseCtrl 🛂🅐🅒🅝",
-    "ACN_SparseCtrlLoaderAdvanced": "Load SparseCtrl Model 🛂🅐🅒🅝",
-    "ACN_SparseCtrlMergedLoaderAdvanced": "🧪Load Merged SparseCtrl Model 🛂🅐🅒🅝",
-    "ACN_SparseCtrlIndexMethodNode": "SparseCtrl Index Method 🛂🅐🅒🅝",
-    "ACN_SparseCtrlSpreadMethodNode": "SparseCtrl Spread Method 🛂🅐🅒🅝",
-    # Reference
-    "ACN_ReferencePreprocessor": "Reference Preproccessor 🛂🅐🅒🅝",
-    "ACN_ReferenceControlNet": "Reference ControlNet 🛂🅐🅒🅝",
-    "ACN_ReferenceControlNetFinetune": "Reference ControlNet (Finetune) 🛂🅐🅒🅝",
-    # LOOSEControl
-    #"ACN_ControlNetLoaderWithLoraAdvanced": "Load Adv. ControlNet Model w/ LoRA 🛂🅐🅒🅝",
-    # Deprecated
-    "LoadImagesFromDirectory": "🚫Load Images [DEPRECATED] 🛂🅐🅒🅝",
-}

adv_control/nodes_ctrlora.py

@@ -0,0 +1,28 @@
+from comfy_api.latest import io
+import folder_paths
+
+from .control_ctrlora import load_ctrlora
+
+class CtrLoRALoader(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_CtrLoRALoader',
+            display_name='Load CtrLoRA Model 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝/CtrLoRA',
+            inputs=[
+                io.Combo.Input('base', options=folder_paths.get_filename_list("controlnet")),
+                io.Combo.Input('lora', options=folder_paths.get_filename_list("controlnet"))
+            ],
+            outputs=[
+                io.ControlNet.Output('CONTROL_NET', is_output_list=False)
+            ]
+        )
+    
+
+    @classmethod
+    def execute(cls, base: str, lora: str):
+        base_path = folder_paths.get_full_path("controlnet", base)
+        lora_path = folder_paths.get_full_path("controlnet", lora)
+        controlnet = load_ctrlora(base_path, lora_path)
+        return io.NodeOutput(controlnet,)

adv_control/nodes_deprecated.py

@@ -1,32 +1,38 @@
+from comfy_api.latest import io
 import os
 
 import torch
+import folder_paths
 
 import numpy as np
 from PIL import Image, ImageOps
-from .utils import BIGMAX
+from .control import load_controlnet, is_advanced_controlnet
-from .logger import logger
+from .nodes_main import AdvancedControlNetApply
+from .utils import ControlWeights, TimestepKeyframeGroup, TimestepKeyframe, get_properly_arranged_t2i_weights
 
-
+class LoadImagesFromDirectory(io.ComfyNode):
-class LoadImagesFromDirectory:
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='LoadImagesFromDirectory',
-                "directory": ("STRING", {"default": ""}),
+            display_name='🚫Load Images [DEPRECATED] 🛂🅐🅒🅝',
-            },
+            category='',
-            "optional": {
+            inputs=[
-                "image_load_cap": ("INT", {"default": 0, "min": 0, "max": BIGMAX, "step": 1}),
+                io.String.Input('directory', default=''),
-                "start_index": ("INT", {"default": 0, "min": 0, "max": BIGMAX, "step": 1}),
+                io.Int.Input('image_load_cap', optional=True, default=0, max=9007199254740991, min=0, step=1),
-            }
+                io.Int.Input('start_index', optional=True, default=0, max=9007199254740991, min=0, step=1)
-        }
+            ],
+            outputs=[
+                io.Image.Output('IMAGE', is_output_list=False),
+                io.Mask.Output('MASK', is_output_list=False),
+                io.Int.Output('INT', is_output_list=False)
+            ],
+            is_deprecated=True
+        )
     
-    RETURN_TYPES = ("IMAGE", "MASK", "INT")
-    FUNCTION = "load_images"
 
-    CATEGORY = ""
+    @classmethod
-
+    def execute(cls, directory: str, image_load_cap: int = 0, start_index: int = 0):
-    def load_images(self, directory: str, image_load_cap: int = 0, start_index: int = 0):
         if not os.path.isdir(directory):
             raise FileNotFoundError(f"Directory '{directory} cannot be found.'")
         dir_files = os.listdir(directory)
@@ -68,4 +74,321 @@ class LoadImagesFromDirectory:
         if len(images) == 0:
             raise FileNotFoundError(f"No images could be loaded from directory '{directory}'.")
 
-        return (torch.cat(images, dim=0), torch.stack(masks, dim=0), image_count)
+        return io.NodeOutput(torch.cat(images, dim=0), torch.stack(masks, dim=0), image_count)
+
+class ScaledSoftUniversalWeightsDeprecated(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ScaledSoftControlNetWeights',
+            display_name='Scaled Soft Weights 🛂🅐🅒🅝',
+            category='',
+            inputs=[
+                io.Float.Input('base_multiplier', default=0.825, max=1.0, min=0.0, step=0.001),
+                io.Boolean.Input('flip_weights', default=False),
+                io.Float.Input('uncond_multiplier', optional=True, default=1.0, max=1.0, min=0.0, step=0.01),
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
+            ],
+            outputs=[
+                io.Custom('CONTROL_NET_WEIGHTS').Output('CN_WEIGHTS', is_output_list=False),
+                io.Custom('TIMESTEP_KEYFRAME').Output('TK_SHORTCUT', is_output_list=False)
+            ]
+        )
+    
+
+    @classmethod
+    def execute(cls, base_multiplier, flip_weights, uncond_multiplier: float=1.0, cn_extras: dict[str]={}):
+        weights = ControlWeights.universal(base_multiplier=base_multiplier, uncond_multiplier=uncond_multiplier, extras=cn_extras)
+        return io.NodeOutput(weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
+
+class SoftControlNetWeightsDeprecated(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='SoftControlNetWeights',
+            display_name='ControlNet Soft Weights 🛂🅐🅒🅝',
+            category='',
+            inputs=[
+                io.Float.Input('weight_00', default=0.09941396206337118, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_01', default=0.12050177219802567, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_02', default=0.14606275417942507, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_03', default=0.17704576264172736, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_04', default=0.214600924414215, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_05', default=0.26012233262329093, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_06', default=0.3152997971191405, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_07', default=0.3821815722656249, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_08', default=0.4632503906249999, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_09', default=0.561515625, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_10', default=0.6806249999999999, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_11', default=0.825, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_12', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Boolean.Input('flip_weights', default=False),
+                io.Float.Input('uncond_multiplier', optional=True, default=1.0, max=1.0, min=0.0, step=0.01),
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
+            ],
+            outputs=[
+                io.Custom('CONTROL_NET_WEIGHTS').Output('CN_WEIGHTS', is_output_list=False),
+                io.Custom('TIMESTEP_KEYFRAME').Output('TK_SHORTCUT', is_output_list=False)
+            ],
+            is_deprecated=True
+        )
+    
+
+    @classmethod
+    def execute(cls, weight_00, weight_01, weight_02, weight_03, weight_04, weight_05, weight_06,
+                     weight_07, weight_08, weight_09, weight_10, weight_11, weight_12, flip_weights,
+                     uncond_multiplier: float=1.0, cn_extras: dict[str]={}):
+        weights_output = [weight_00, weight_01, weight_02, weight_03, weight_04, weight_05, weight_06, 
+                   weight_07, weight_08, weight_09, weight_10, weight_11]
+        weights_middle = [weight_12]
+        weights = ControlWeights.controlnet(weights_output=weights_output, weights_middle=weights_middle, uncond_multiplier=uncond_multiplier, extras=cn_extras)
+        return io.NodeOutput(weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
+
+class CustomControlNetWeightsDeprecated(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='CustomControlNetWeights',
+            display_name='ControlNet Custom Weights 🛂🅐🅒🅝',
+            category='',
+            inputs=[
+                io.Float.Input('weight_00', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_01', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_02', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_03', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_04', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_05', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_06', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_07', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_08', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_09', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_10', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_11', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_12', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Boolean.Input('flip_weights', default=False),
+                io.Float.Input('uncond_multiplier', optional=True, default=1.0, max=1.0, min=0.0, step=0.01),
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
+            ],
+            outputs=[
+                io.Custom('CONTROL_NET_WEIGHTS').Output('CN_WEIGHTS', is_output_list=False),
+                io.Custom('TIMESTEP_KEYFRAME').Output('TK_SHORTCUT', is_output_list=False)
+            ],
+            is_deprecated=True
+        )
+    
+
+    @classmethod
+    def execute(cls, weight_00, weight_01, weight_02, weight_03, weight_04, weight_05, weight_06,
+                     weight_07, weight_08, weight_09, weight_10, weight_11, weight_12, flip_weights,
+                     uncond_multiplier: float=1.0, cn_extras: dict[str]={}):
+        weights_output = [weight_00, weight_01, weight_02, weight_03, weight_04, weight_05, weight_06, 
+                   weight_07, weight_08, weight_09, weight_10, weight_11]
+        weights_middle = [weight_12]
+        weights = ControlWeights.controlnet(weights_output=weights_output, weights_middle=weights_middle, uncond_multiplier=uncond_multiplier, extras=cn_extras)
+        return io.NodeOutput(weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
+
+class SoftT2IAdapterWeightsDeprecated(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='SoftT2IAdapterWeights',
+            display_name='T2IAdapter Soft Weights 🛂🅐🅒🅝',
+            category='',
+            inputs=[
+                io.Float.Input('weight_00', default=0.25, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_01', default=0.62, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_02', default=0.825, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_03', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Boolean.Input('flip_weights', default=False),
+                io.Float.Input('uncond_multiplier', optional=True, default=1.0, max=1.0, min=0.0, step=0.01),
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
+            ],
+            outputs=[
+                io.Custom('CONTROL_NET_WEIGHTS').Output('CN_WEIGHTS', is_output_list=False),
+                io.Custom('TIMESTEP_KEYFRAME').Output('TK_SHORTCUT', is_output_list=False)
+            ],
+            is_deprecated=True
+        )
+    
+
+    @classmethod
+    def execute(cls, weight_00, weight_01, weight_02, weight_03, flip_weights,
+                     uncond_multiplier: float=1.0, cn_extras: dict[str]={}):
+        weights = [weight_00, weight_01, weight_02, weight_03]
+        weights = get_properly_arranged_t2i_weights(weights)
+        weights = ControlWeights.t2iadapter(weights_input=weights, uncond_multiplier=uncond_multiplier, extras=cn_extras)
+        return io.NodeOutput(weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
+
+class CustomT2IAdapterWeightsDeprecated(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='CustomT2IAdapterWeights',
+            display_name='T2IAdapter Custom Weights 🛂🅐🅒🅝',
+            category='',
+            inputs=[
+                io.Float.Input('weight_00', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_01', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_02', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('weight_03', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Boolean.Input('flip_weights', default=False),
+                io.Float.Input('uncond_multiplier', optional=True, default=1.0, max=1.0, min=0.0, step=0.01),
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
+            ],
+            outputs=[
+                io.Custom('CONTROL_NET_WEIGHTS').Output('CN_WEIGHTS', is_output_list=False),
+                io.Custom('TIMESTEP_KEYFRAME').Output('TK_SHORTCUT', is_output_list=False)
+            ],
+            is_deprecated=True
+        )
+    
+
+    @classmethod
+    def execute(cls, weight_00, weight_01, weight_02, weight_03, flip_weights,
+                     uncond_multiplier: float=1.0, cn_extras: dict[str]={}):
+        weights = [weight_00, weight_01, weight_02, weight_03]
+        weights = get_properly_arranged_t2i_weights(weights)
+        weights = ControlWeights.t2iadapter(weights_input=weights, uncond_multiplier=uncond_multiplier, extras=cn_extras)
+        return io.NodeOutput(weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
+
+class AdvancedControlNetApplyDEPR(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_AdvancedControlNetApply',
+            display_name='Apply Advanced ControlNet 🛂🅐🅒🅝',
+            category='',
+            inputs=[
+                io.Conditioning.Input('positive'),
+                io.Conditioning.Input('negative'),
+                io.ControlNet.Input('control_net'),
+                io.Image.Input('image'),
+                io.Float.Input('strength', default=1.0, max=10.0, min=0.0, step=0.01),
+                io.Float.Input('start_percent', default=0.0, max=1.0, min=0.0, step=0.001),
+                io.Float.Input('end_percent', default=1.0, max=1.0, min=0.0, step=0.001),
+                io.Mask.Input('mask_optional', optional=True),
+                io.Custom('TIMESTEP_KEYFRAME').Input('timestep_kf', optional=True),
+                io.Custom('LATENT_KEYFRAME').Input('latent_kf_override', optional=True),
+                io.Custom('CONTROL_NET_WEIGHTS').Input('weights_override', optional=True),
+                io.Model.Input('model_optional', optional=True),
+                io.Vae.Input('vae_optional', optional=True)
+            ],
+            outputs=[
+                io.Conditioning.Output('positive', is_output_list=False),
+                io.Conditioning.Output('negative', is_output_list=False),
+                io.Model.Output('model_opt', is_output_list=False)
+            ],
+            is_deprecated=True
+        )
+
+    @classmethod
+    def execute(cls, positive, negative, control_net, image, strength, start_percent, end_percent,
+                         mask_optional=None, model_optional=None, vae_optional=None,
+                         timestep_kf: TimestepKeyframeGroup=None, latent_kf_override=None,
+                         weights_override: ControlWeights=None, control_apply_to_uncond=False):
+        new_positive, new_negative = AdvancedControlNetApply.execute(positive=positive, negative=negative, control_net=control_net, image=image,
+                                                          strength=strength, start_percent=start_percent, end_percent=end_percent,
+                                                          mask_optional=mask_optional, vae_optional=vae_optional,
+                                                          timestep_kf=timestep_kf, latent_kf_override=latent_kf_override, weights_override=weights_override,).args
+        return io.NodeOutput(new_positive, new_negative, model_optional)
+
+class AdvancedControlNetApplySingleDEPR(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_AdvancedControlNetApplySingle',
+            display_name='Apply Advanced ControlNet(1) 🛂🅐🅒🅝',
+            category='',
+            inputs=[
+                io.Conditioning.Input('conditioning'),
+                io.ControlNet.Input('control_net'),
+                io.Image.Input('image'),
+                io.Float.Input('strength', default=1.0, max=10.0, min=0.0, step=0.01),
+                io.Float.Input('start_percent', default=0.0, max=1.0, min=0.0, step=0.001),
+                io.Float.Input('end_percent', default=1.0, max=1.0, min=0.0, step=0.001),
+                io.Mask.Input('mask_optional', optional=True),
+                io.Custom('TIMESTEP_KEYFRAME').Input('timestep_kf', optional=True),
+                io.Custom('LATENT_KEYFRAME').Input('latent_kf_override', optional=True),
+                io.Custom('CONTROL_NET_WEIGHTS').Input('weights_override', optional=True),
+                io.Model.Input('model_optional', optional=True),
+                io.Vae.Input('vae_optional', optional=True)
+            ],
+            outputs=[
+                io.Conditioning.Output('CONDITIONING', is_output_list=False),
+                io.Model.Output('model_opt', is_output_list=False)
+            ],
+            is_deprecated=True
+        )
+
+    @classmethod
+    def execute(cls, conditioning, control_net, image, strength, start_percent, end_percent,
+                         mask_optional=None, model_optional=None, vae_optional=None,
+                         timestep_kf: TimestepKeyframeGroup=None, latent_kf_override=None,
+                         weights_override: ControlWeights=None):
+        values = AdvancedControlNetApply.execute(positive=conditioning, negative=None, control_net=control_net, image=image,
+                                                          strength=strength, start_percent=start_percent, end_percent=end_percent,
+                                                          mask_optional=mask_optional, vae_optional=vae_optional,
+                                                          timestep_kf=timestep_kf, latent_kf_override=latent_kf_override, weights_override=weights_override,
+                                                          control_apply_to_uncond=True)
+        return io.NodeOutput(values.args[0], model_optional)
+
+class ControlNetLoaderAdvancedDEPR(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ControlNetLoaderAdvanced',
+            display_name='Load Advanced ControlNet Model 🛂🅐🅒🅝',
+            category='',
+            inputs=[
+                io.Combo.Input('control_net_name', options=folder_paths.get_filename_list("controlnet")),
+                io.Custom('TIMESTEP_KEYFRAME').Input('tk_optional', optional=True)
+            ],
+            outputs=[
+                io.ControlNet.Output('CONTROL_NET', is_output_list=False)
+            ],
+            is_deprecated=True
+        )
+
+    @classmethod
+    def execute(cls, control_net_name,
+                        tk_optional: TimestepKeyframeGroup=None,
+                        timestep_keyframe: TimestepKeyframeGroup=None,
+                        ):
+        if timestep_keyframe is not None: # backwards compatibility
+            tk_optional = timestep_keyframe
+        controlnet_path = folder_paths.get_full_path("controlnet", control_net_name)
+        controlnet = load_controlnet(controlnet_path, tk_optional)
+        return io.NodeOutput(controlnet,)
+    
+
+class DiffControlNetLoaderAdvancedDEPR(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='DiffControlNetLoaderAdvanced',
+            display_name='Load Advanced ControlNet Model (diff) 🛂🅐🅒🅝',
+            category='',
+            inputs=[
+                io.Model.Input('model'),
+                io.Combo.Input('control_net_name', options=folder_paths.get_filename_list("controlnet")),
+                io.Custom('TIMESTEP_KEYFRAME').Input('tk_optional', optional=True)
+            ],
+            outputs=[
+                io.ControlNet.Output('CONTROL_NET', is_output_list=False)
+            ],
+            is_deprecated=True
+        )
+    
+
+    @classmethod
+    def execute(cls, control_net_name, model,
+                        tk_optional: TimestepKeyframeGroup=None,
+                        timestep_keyframe: TimestepKeyframeGroup=None
+                        ):
+        if timestep_keyframe is not None: # backwards compatibility
+            tk_optional = timestep_keyframe
+        controlnet_path = folder_paths.get_full_path("controlnet", control_net_name)
+        controlnet = load_controlnet(controlnet_path, tk_optional, model)
+        if is_advanced_controlnet(controlnet):
+            controlnet.verify_all_weights()
+        return io.NodeOutput(controlnet,)

adv_control/nodes_keyframes.py

@@ -1,40 +1,40 @@
+from comfy_api.latest import io
 from typing import Union
 import numpy as np
 from collections.abc import Iterable
 
-from .utils import ControlWeights, TimestepKeyframe, TimestepKeyframeGroup, LatentKeyframe, LatentKeyframeGroup, BIGMIN, BIGMAX
+from .utils import ControlWeights, TimestepKeyframe, TimestepKeyframeGroup, LatentKeyframe, LatentKeyframeGroup
 from .utils import StrengthInterpolation as SI
 from .logger import logger
 
-
+class TimestepKeyframeNode(io.ComfyNode):
-class TimestepKeyframeNode:
     OUTDATED_DUMMY = -39
 
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='TimestepKeyframe',
-                "start_percent": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}, ),
+            display_name='Timestep Keyframe 🛂🅐🅒🅝',
-            },
+            category='Adv-ControlNet 🛂🅐🅒🅝/keyframes',
-            "optional": {
+            inputs=[
-                "prev_timestep_kf": ("TIMESTEP_KEYFRAME", ),
+                io.Float.Input('start_percent', default=0.0, max=1.0, min=0.0, step=0.001),
-                "strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Custom('TIMESTEP_KEYFRAME').Input('prev_timestep_kf', optional=True),
-                "cn_weights": ("CONTROL_NET_WEIGHTS", ),
+                io.Float.Input('strength', optional=True, default=1.0, max=10.0, min=0.0, step=0.001),
-                "latent_keyframe": ("LATENT_KEYFRAME", ),
+                io.Custom('CONTROL_NET_WEIGHTS').Input('cn_weights', optional=True),
-                "null_latent_kf_strength": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Custom('LATENT_KEYFRAME').Input('latent_keyframe', optional=True),
-                "inherit_missing": ("BOOLEAN", {"default": True}, ),
+                io.Float.Input('null_latent_kf_strength', optional=True, default=0.0, max=10.0, min=0.0, step=0.001),
-                "guarantee_steps": ("INT", {"default": 1, "min": 0, "max": BIGMAX}),
+                io.Boolean.Input('inherit_missing', optional=True, default=True),
-                "mask_optional": ("MASK", ),
+                io.Int.Input('guarantee_steps', optional=True, default=1, max=9007199254740991, min=0),
-            }
+                io.Mask.Input('mask_optional', optional=True)
-        }
+            ],
+            outputs=[
+                io.Custom('TIMESTEP_KEYFRAME').Output('TIMESTEP_KF', is_output_list=False)
+            ]
+        )
     
-    RETURN_NAMES = ("TIMESTEP_KF", )
-    RETURN_TYPES = ("TIMESTEP_KEYFRAME", )
-    FUNCTION = "load_keyframe"
 
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/keyframes"
+    @classmethod
-
+    def execute(cls,
-    def load_keyframe(self,
                       start_percent: float,
                       strength: float=1.0,
                       cn_weights: ControlWeights=None, control_net_weights: ControlWeights=None, # old name
@@ -46,7 +46,7 @@ class TimestepKeyframeNode:
                       guarantee_usage=True, # old input
                       mask_optional=None,):
         # if using outdated dummy value, means node on workflow is outdated and should appropriately convert behavior
-        if guarantee_steps == self.OUTDATED_DUMMY:
+        if guarantee_steps == cls.OUTDATED_DUMMY:
             guarantee_steps = int(guarantee_usage)
         control_net_weights = control_net_weights if control_net_weights else cn_weights
         prev_timestep_keyframe = prev_timestep_keyframe if prev_timestep_keyframe else prev_timestep_kf
@@ -58,39 +58,39 @@ class TimestepKeyframeNode:
                                     control_weights=control_net_weights, latent_keyframes=latent_keyframe, inherit_missing=inherit_missing,
                                     guarantee_steps=guarantee_steps, mask_hint_orig=mask_optional)
         prev_timestep_keyframe.add(keyframe)
-        return (prev_timestep_keyframe,)
+        return io.NodeOutput(prev_timestep_keyframe,)
     
 
-class TimestepKeyframeInterpolationNode:
+class TimestepKeyframeInterpolationNode(io.ComfyNode):
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='ACN_TimestepKeyframeInterpolation',
-                "start_percent": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001},),
+            display_name='Timestep Keyframe Interp. 🛂🅐🅒🅝',
-                "end_percent": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}),
+            category='Adv-ControlNet 🛂🅐🅒🅝/keyframes',
-                "strength_start": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001},),
+            inputs=[
-                "strength_end": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001},),
+                io.Float.Input('start_percent', default=0.0, max=1.0, min=0.0, step=0.001),
-                "interpolation": (SI._LIST, ),
+                io.Float.Input('end_percent', default=1.0, max=1.0, min=0.0, step=0.001),
-                "intervals": ("INT", {"default": 50, "min": 2, "max": 100, "step": 1}),
+                io.Float.Input('strength_start', default=1.0, max=10.0, min=0.0, step=0.001),
-            },
+                io.Float.Input('strength_end', default=1.0, max=10.0, min=0.0, step=0.001),
-            "optional": {
+                io.Combo.Input('interpolation', options=['linear', 'ease-in', 'ease-out', 'ease-in-out']),
-                "prev_timestep_kf": ("TIMESTEP_KEYFRAME", ),
+                io.Int.Input('intervals', default=50, max=100, min=2, step=1),
-                "cn_weights": ("CONTROL_NET_WEIGHTS", ),
+                io.Custom('TIMESTEP_KEYFRAME').Input('prev_timestep_kf', optional=True),
-                "latent_keyframe": ("LATENT_KEYFRAME", ),
+                io.Custom('CONTROL_NET_WEIGHTS').Input('cn_weights', optional=True),
-                "null_latent_kf_strength": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 10.0, "step": 0.001},),
+                io.Custom('LATENT_KEYFRAME').Input('latent_keyframe', optional=True),
-                "inherit_missing": ("BOOLEAN", {"default": True},),
+                io.Float.Input('null_latent_kf_strength', optional=True, default=0.0, max=10.0, min=0.0, step=0.001),
-                "mask_optional": ("MASK", ),
+                io.Boolean.Input('inherit_missing', optional=True, default=True),
-                "print_keyframes": ("BOOLEAN", {"default": False}),
+                io.Mask.Input('mask_optional', optional=True),
-            }
+                io.Boolean.Input('print_keyframes', optional=True, default=False)
-        }
+            ],
+            outputs=[
+                io.Custom('TIMESTEP_KEYFRAME').Output('TIMESTEP_KF', is_output_list=False)
+            ]
+        )
     
-    RETURN_NAMES = ("TIMESTEP_KF", )
-    RETURN_TYPES = ("TIMESTEP_KEYFRAME", )
-    FUNCTION = "load_keyframe"
 
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/keyframes"
+    @classmethod
-
+    def execute(cls,
-    def load_keyframe(self,
                       start_percent: float, end_percent: float,
                       strength_start: float, strength_end: float, interpolation: str, intervals: int,
                       cn_weights: ControlWeights=None,
@@ -119,36 +119,35 @@ class TimestepKeyframeInterpolationNode:
                                     guarantee_steps=guarantee_steps, mask_hint_orig=mask_optional))
             if print_keyframes:
                 logger.info(f"TimestepKeyframe - start_percent:{percent} = {strength}")
-        return (prev_timestep_kf,)
+        return io.NodeOutput(prev_timestep_kf,)
 
-
+class TimestepKeyframeFromStrengthListNode(io.ComfyNode):
-class TimestepKeyframeFromStrengthListNode:
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='ACN_TimestepKeyframeFromStrengthList',
-                "float_strengths": ("FLOAT", {"default": -1, "min": -1, "step": 0.001, "forceInput": True}),
+            display_name='Timestep Keyframe From List 🛂🅐🅒🅝',
-                "start_percent": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001},),
+            category='Adv-ControlNet 🛂🅐🅒🅝/keyframes',
-                "end_percent": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}),
+            inputs=[
-            },
+                io.Float.Input('float_strengths', default=-1, force_input=True, min=-1, step=0.001),
-            "optional": {
+                io.Float.Input('start_percent', default=0.0, max=1.0, min=0.0, step=0.001),
-                "prev_timestep_kf": ("TIMESTEP_KEYFRAME", ),
+                io.Float.Input('end_percent', default=1.0, max=1.0, min=0.0, step=0.001),
-                "cn_weights": ("CONTROL_NET_WEIGHTS", ),
+                io.Custom('TIMESTEP_KEYFRAME').Input('prev_timestep_kf', optional=True),
-                "latent_keyframe": ("LATENT_KEYFRAME", ),
+                io.Custom('CONTROL_NET_WEIGHTS').Input('cn_weights', optional=True),
-                "null_latent_kf_strength": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 10.0, "step": 0.001},),
+                io.Custom('LATENT_KEYFRAME').Input('latent_keyframe', optional=True),
-                "inherit_missing": ("BOOLEAN", {"default": True},),
+                io.Float.Input('null_latent_kf_strength', optional=True, default=0.0, max=10.0, min=0.0, step=0.001),
-                "mask_optional": ("MASK", ),
+                io.Boolean.Input('inherit_missing', optional=True, default=True),
-                "print_keyframes": ("BOOLEAN", {"default": False}),
+                io.Mask.Input('mask_optional', optional=True),
-            }
+                io.Boolean.Input('print_keyframes', optional=True, default=False)
-        }
+            ],
+            outputs=[
+                io.Custom('TIMESTEP_KEYFRAME').Output('TIMESTEP_KF', is_output_list=False)
+            ]
+        )
     
-    RETURN_NAMES = ("TIMESTEP_KF", )
-    RETURN_TYPES = ("TIMESTEP_KEYFRAME", )
-    FUNCTION = "load_keyframe"
 
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/keyframes"
+    @classmethod
-
+    def execute(cls,
-    def load_keyframe(self,
                       start_percent: float, end_percent: float,
                       float_strengths: float,
                       cn_weights: ControlWeights=None,
@@ -182,29 +181,27 @@ class TimestepKeyframeFromStrengthListNode:
                                     guarantee_steps=guarantee_steps, mask_hint_orig=mask_optional))
             if print_keyframes:
                 logger.info(f"TimestepKeyframe - start_percent:{percent} = {strength}")
-        return (prev_timestep_kf,)
+        return io.NodeOutput(prev_timestep_kf,)
 
-
+class LatentKeyframeNode(io.ComfyNode):
-class LatentKeyframeNode:
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='LatentKeyframe',
-                "batch_index": ("INT", {"default": 0, "min": BIGMIN, "max": BIGMAX, "step": 1}),
+            display_name='Latent Keyframe 🛂🅐🅒🅝',
-                "strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+            category='Adv-ControlNet 🛂🅐🅒🅝/keyframes',
-            },
+            inputs=[
-            "optional": {
+                io.Int.Input('batch_index', default=0, max=9007199254740991, min=-9007199254740991, step=1),
-                "prev_latent_kf": ("LATENT_KEYFRAME", ),
+                io.Float.Input('strength', default=1.0, max=10.0, min=0.0, step=0.001),
-            }
+                io.Custom('LATENT_KEYFRAME').Input('prev_latent_kf', optional=True)
-        }
+            ],
+            outputs=[
+                io.Custom('LATENT_KEYFRAME').Output('LATENT_KF', is_output_list=False)
+            ]
+        )
 
-    RETURN_NAMES = ("LATENT_KF", )
+    @classmethod
-    RETURN_TYPES = ("LATENT_KEYFRAME", )
+    def execute(cls,
-    FUNCTION = "load_keyframe"
-
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/keyframes"
-
-    def load_keyframe(self,
                       batch_index: int,
                       strength: float,
                       prev_latent_kf: LatentKeyframeGroup=None,
@@ -217,30 +214,29 @@ class LatentKeyframeNode:
             prev_latent_keyframe = prev_latent_keyframe.clone()
         keyframe = LatentKeyframe(batch_index, strength)
         prev_latent_keyframe.add(keyframe)
-        return (prev_latent_keyframe,)
+        return io.NodeOutput(prev_latent_keyframe,)
 
-
+class LatentKeyframeGroupNode(io.ComfyNode):
-class LatentKeyframeGroupNode:
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='LatentKeyframeGroup',
-                "index_strengths": ("STRING", {"multiline": True, "default": ""}),
+            display_name='Latent Keyframe Group 🛂🅐🅒🅝',
-            },
+            category='Adv-ControlNet 🛂🅐🅒🅝/keyframes',
-            "optional": {
+            inputs=[
-                "prev_latent_kf": ("LATENT_KEYFRAME", ),
+                io.String.Input('index_strengths', default='', multiline=True),
-                "latent_optional": ("LATENT", ),
+                io.Custom('LATENT_KEYFRAME').Input('prev_latent_kf', optional=True),
-                "print_keyframes": ("BOOLEAN", {"default": False})
+                io.Latent.Input('latent_optional', optional=True),
-            }
+                io.Boolean.Input('print_keyframes', optional=True, default=False)
-        }
+            ],
+            outputs=[
+                io.Custom('LATENT_KEYFRAME').Output('LATENT_KF', is_output_list=False)
+            ]
+        )
     
-    RETURN_NAMES = ("LATENT_KF", )
-    RETURN_TYPES = ("LATENT_KEYFRAME", )
-    FUNCTION = "load_keyframes"
 
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/keyframes"
+    @staticmethod
-
+    def validate_index(index: int, latent_count: int = 0, is_range: bool = False, allow_negative = False) -> int:
-    def validate_index(self, index: int, latent_count: int = 0, is_range: bool = False, allow_negative = False) -> int:
         # if part of range, do nothing
         if is_range:
             return index
@@ -258,13 +254,15 @@ class LatentKeyframeGroupNode:
             index = conv_index
         return index
 
-    def convert_to_index_int(self, raw_index: str, latent_count: int = 0, is_range: bool = False, allow_negative = False) -> int:
+    @classmethod
+    def convert_to_index_int(cls, raw_index: str, latent_count: int = 0, is_range: bool = False, allow_negative = False) -> int:
         try:
-            return self.validate_index(int(raw_index), latent_count=latent_count, is_range=is_range, allow_negative=allow_negative)
+            return cls.validate_index(int(raw_index), latent_count=latent_count, is_range=is_range, allow_negative=allow_negative)
         except ValueError as e:
             raise ValueError(f"index '{raw_index}' must be an integer.", e)
 
-    def convert_to_latent_keyframes(self, latent_indeces: str, latent_count: int) -> set[LatentKeyframe]:
+    @classmethod
+    def convert_to_latent_keyframes(cls, latent_indeces: str, latent_count: int) -> set[LatentKeyframe]:
         if not latent_indeces:
             return set()
         int_latent_indeces = [i for i in range(0, latent_count)]
@@ -289,8 +287,8 @@ class LatentKeyframeGroupNode:
             if ':' in g:
                 index_range = g.split(":", 1)
                 index_range = [r.strip() for r in index_range]
-                start_index = self.convert_to_index_int(index_range[0], latent_count=latent_count, is_range=True, allow_negative=allow_negative)
+                start_index = cls.convert_to_index_int(index_range[0], latent_count=latent_count, is_range=True, allow_negative=allow_negative)
-                end_index = self.convert_to_index_int(index_range[1], latent_count=latent_count, is_range=True, allow_negative=allow_negative)
+                end_index = cls.convert_to_index_int(index_range[1], latent_count=latent_count, is_range=True, allow_negative=allow_negative)
                 # if latents were passed in, base indeces on known latent count
                 if len(int_latent_indeces) > 0:
                     for i in int_latent_indeces[start_index:end_index]:
@@ -301,14 +299,16 @@ class LatentKeyframeGroupNode:
                         chosen_indeces.add(LatentKeyframe(i, strength))
             # parse individual indeces
             else:
-                chosen_indeces.add(LatentKeyframe(self.convert_to_index_int(g, latent_count=latent_count, allow_negative=allow_negative), strength))
+                chosen_indeces.add(LatentKeyframe(cls.convert_to_index_int(g, latent_count=latent_count, allow_negative=allow_negative), strength))
         return chosen_indeces
 
-    def load_keyframes(self,
+    @classmethod
+    def execute(cls,
                        index_strengths: str,
                        prev_latent_kf: LatentKeyframeGroup=None,
                        prev_latent_keyframe: LatentKeyframeGroup=None, # old name
-                       latent_image_opt=None,
+                       latent_optional=None,
+                       latent_image_opt=None, # old name
                        print_keyframes=False):
         prev_latent_keyframe = prev_latent_keyframe if prev_latent_keyframe else prev_latent_kf
         if not prev_latent_keyframe:
@@ -317,10 +317,11 @@ class LatentKeyframeGroupNode:
             prev_latent_keyframe = prev_latent_keyframe.clone()
         curr_latent_keyframe = LatentKeyframeGroup()
 
+        latent_image_opt = latent_image_opt if latent_image_opt is not None else latent_optional
         latent_count = -1
         if latent_image_opt:
             latent_count = latent_image_opt['samples'].size()[0]
-        latent_keyframes = self.convert_to_latent_keyframes(index_strengths, latent_count=latent_count)
+        latent_keyframes = cls.convert_to_latent_keyframes(index_strengths, latent_count=latent_count)
 
         for latent_keyframe in latent_keyframes:
             curr_latent_keyframe.add(latent_keyframe)
@@ -333,32 +334,32 @@ class LatentKeyframeGroupNode:
         for latent_keyframe in prev_latent_keyframe.keyframes:
             curr_latent_keyframe.add(latent_keyframe)
 
-        return (curr_latent_keyframe,)
+        return io.NodeOutput(curr_latent_keyframe,)
 
         
-class LatentKeyframeInterpolationNode:
+class LatentKeyframeInterpolationNode(io.ComfyNode):
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='LatentKeyframeTiming',
-                "batch_index_from": ("INT", {"default": 0, "min": BIGMIN, "max": BIGMAX, "step": 1}),
+            display_name='Latent Keyframe Interp. 🛂🅐🅒🅝',
-                "batch_index_to_excl": ("INT", {"default": 0, "min": BIGMIN, "max": BIGMAX, "step": 1}),
+            category='Adv-ControlNet 🛂🅐🅒🅝/keyframes',
-                "strength_from": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+            inputs=[
-                "strength_to": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Int.Input('batch_index_from', default=0, max=9007199254740991, min=-9007199254740991, step=1),
-                "interpolation": (SI._LIST, ),
+                io.Int.Input('batch_index_to_excl', default=0, max=9007199254740991, min=-9007199254740991, step=1),
-            },
+                io.Float.Input('strength_from', default=1.0, max=10.0, min=0.0, step=0.001),
-            "optional": {
+                io.Float.Input('strength_to', default=1.0, max=10.0, min=0.0, step=0.001),
-                "prev_latent_kf": ("LATENT_KEYFRAME", ),
+                io.Combo.Input('interpolation', options=['linear', 'ease-in', 'ease-out', 'ease-in-out']),
-                "print_keyframes": ("BOOLEAN", {"default": False})
+                io.Custom('LATENT_KEYFRAME').Input('prev_latent_kf', optional=True),
-            }
+                io.Boolean.Input('print_keyframes', optional=True, default=False)
-        }
+            ],
+            outputs=[
+                io.Custom('LATENT_KEYFRAME').Output('LATENT_KF', is_output_list=False)
+            ]
+        )
 
-    RETURN_NAMES = ("LATENT_KF", )
+    @classmethod
-    RETURN_TYPES = ("LATENT_KEYFRAME", )
+    def execute(cls,
-    FUNCTION = "load_keyframe"
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/keyframes"
-
-    def load_keyframe(self,
                         batch_index_from: int,
                         strength_from: float,
                         batch_index_to_excl: int,
@@ -407,28 +408,27 @@ class LatentKeyframeInterpolationNode:
         for latent_keyframe in prev_latent_keyframe.keyframes:
             curr_latent_keyframe.add(latent_keyframe)
 
-        return (curr_latent_keyframe,)
+        return io.NodeOutput(curr_latent_keyframe,)
 
-
+class LatentKeyframeBatchedGroupNode(io.ComfyNode):
-class LatentKeyframeBatchedGroupNode:
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='LatentKeyframeBatchedGroup',
-                "float_strengths": ("FLOAT", {"default": -1, "min": -1, "step": 0.001, "forceInput": True}),
+            display_name='Latent Keyframe From List 🛂🅐🅒🅝',
-            },
+            category='Adv-ControlNet 🛂🅐🅒🅝/keyframes',
-            "optional": {
+            inputs=[
-                "prev_latent_kf": ("LATENT_KEYFRAME", ),
+                io.Float.Input('float_strengths', default=-1, force_input=True, min=-1, step=0.001),
-                "print_keyframes": ("BOOLEAN", {"default": False})
+                io.Custom('LATENT_KEYFRAME').Input('prev_latent_kf', optional=True),
-            }
+                io.Boolean.Input('print_keyframes', optional=True, default=False)
-        }
+            ],
+            outputs=[
+                io.Custom('LATENT_KEYFRAME').Output('LATENT_KF', is_output_list=False)
+            ]
+        )
 
-    RETURN_NAMES = ("LATENT_KF", )
+    @classmethod
-    RETURN_TYPES = ("LATENT_KEYFRAME", )
+    def execute(cls, float_strengths: Union[float, list[float]],
-    FUNCTION = "load_keyframe"
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/keyframes"
-
-    def load_keyframe(self, float_strengths: Union[float, list[float]],
                       prev_latent_kf: LatentKeyframeGroup=None,
                       prev_latent_keyframe: LatentKeyframeGroup=None, # old name
                       print_keyframes=False):
@@ -458,4 +458,4 @@ class LatentKeyframeBatchedGroupNode:
         for latent_keyframe in prev_latent_keyframe.keyframes:
             curr_latent_keyframe.add(latent_keyframe)
 
-        return (curr_latent_keyframe,)
+        return io.NodeOutput(curr_latent_keyframe,)

adv_control/nodes_main.py

@@ -0,0 +1,222 @@
+from comfy_api.latest import io
+from torch import Tensor
+
+import folder_paths
+
+from .control import load_controlnet, convert_to_advanced, is_advanced_controlnet, is_sd3_advanced_controlnet
+from .control_lllite import load_anima_lllite
+from .utils import ControlWeights, LatentKeyframeGroup, TimestepKeyframeGroup, AbstractPreprocWrapper
+
+class ControlNetLoaderAdvanced(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_ControlNetLoaderAdvanced',
+            display_name='Load Advanced ControlNet Model 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝',
+            inputs=[
+                io.Combo.Input('cnet', options=folder_paths.get_filename_list("controlnet")),
+                io.Custom('TIMESTEP_KEYFRAME').Input('_tk_opt', optional=True)
+            ],
+            outputs=[
+                io.ControlNet.Output('CONTROL_NET', is_output_list=False)
+            ]
+        )
+
+    @classmethod
+    def execute(cls, cnet,
+                        _tk_opt: TimestepKeyframeGroup=None,
+                        ):
+        controlnet_path = folder_paths.get_full_path("controlnet", cnet)
+        controlnet = load_controlnet(controlnet_path, _tk_opt)
+        return io.NodeOutput(controlnet,)
+    
+
+class DiffControlNetLoaderAdvanced(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_DiffControlNetLoaderAdvanced',
+            display_name='Load Advanced ControlNet Model (diff) 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝',
+            inputs=[
+                io.Model.Input('model'),
+                io.Combo.Input('cnet', options=folder_paths.get_filename_list("controlnet")),
+                io.Custom('TIMESTEP_KEYFRAME').Input('_tk_opt', optional=True)
+            ],
+            outputs=[
+                io.ControlNet.Output('CONTROL_NET', is_output_list=False)
+            ]
+        )
+    
+
+    @classmethod
+    def execute(cls, cnet, model,
+                        _tk_opt: TimestepKeyframeGroup=None,
+                        ):
+        controlnet_path = folder_paths.get_full_path("controlnet", cnet)
+        controlnet = load_controlnet(controlnet_path, _tk_opt, model)
+        if is_advanced_controlnet(controlnet):
+            controlnet.verify_all_weights()
+        return io.NodeOutput(controlnet,)
+
+class AnimaLLLiteLoaderAdvanced(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_AnimaLLLiteLoaderAdvanced',
+            display_name='Load Anima LLLite Model 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝/loaders',
+            inputs=[
+                io.Combo.Input('model_patch', options=folder_paths.get_filename_list("model_patches")),
+                io.Custom('TIMESTEP_KEYFRAME').Input('timestep_kf', optional=True)
+            ],
+            outputs=[
+                io.ControlNet.Output('CONTROL_NET', is_output_list=False)
+            ]
+        )
+
+    @classmethod
+    def execute(cls, model_patch, timestep_kf: TimestepKeyframeGroup=None):
+        model_patch_path = folder_paths.get_full_path_or_raise("model_patches", model_patch)
+        return io.NodeOutput(load_anima_lllite(model_patch_path, timestep_keyframe=timestep_kf),)
+
+class AdvancedControlNetApply(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_AdvancedControlNetApply_v2',
+            display_name='Apply Advanced ControlNet 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝',
+            inputs=[
+                io.Conditioning.Input('positive'),
+                io.Conditioning.Input('negative'),
+                io.ControlNet.Input('control_net'),
+                io.Image.Input('image'),
+                io.Float.Input('strength', default=1.0, max=10.0, min=0.0, step=0.01),
+                io.Float.Input('start_percent', default=0.0, max=1.0, min=0.0, step=0.001),
+                io.Float.Input('end_percent', default=1.0, max=1.0, min=0.0, step=0.001),
+                io.Mask.Input('mask_optional', optional=True),
+                io.Custom('TIMESTEP_KEYFRAME').Input('timestep_kf', optional=True),
+                io.Custom('LATENT_KEYFRAME').Input('latent_kf_override', optional=True),
+                io.Custom('CONTROL_NET_WEIGHTS').Input('weights_override', optional=True),
+                io.Vae.Input('vae_optional', optional=True)
+            ],
+            outputs=[
+                io.Conditioning.Output('positive', is_output_list=False),
+                io.Conditioning.Output('negative', is_output_list=False)
+            ]
+        )
+
+    @classmethod
+    def execute(cls, positive, negative, control_net, image, strength, start_percent, end_percent,
+                         mask_optional: Tensor=None, vae_optional=None,
+                         timestep_kf: TimestepKeyframeGroup=None, latent_kf_override: LatentKeyframeGroup=None,
+                         weights_override: ControlWeights=None, control_apply_to_uncond=False):
+        if strength == 0:
+            return io.NodeOutput(positive, negative)
+
+        control_hint = image.movedim(-1,1)
+        cnets = {}
+
+        out = []
+        for conditioning in [positive, negative]:
+            c = []
+            if conditioning is not None:
+                for t in conditioning:
+                    d = t[1].copy()
+
+                    prev_cnet = d.get('control', None)
+                    if prev_cnet in cnets:
+                        c_net = cnets[prev_cnet]
+                    else:
+                        # make sure control_net is not None to avoid confusing error messages
+                        if control_net is None:
+                            raise Exception("Passed in control_net is None; something must have went wrong when loading it from a Load ControlNet node.")
+                        # copy, convert to advanced if needed, and set cond
+                        c_net = convert_to_advanced(control_net.copy()).set_cond_hint(control_hint, strength, (start_percent, end_percent), vae_optional)
+                        if is_advanced_controlnet(c_net):
+                            # disarm node check
+                            c_net.disarm()
+                            # check for allow_condhint_latents where vae_optional can't handle it itself
+                            if c_net.allow_condhint_latents and not c_net.require_vae:
+                                if not isinstance(control_hint, AbstractPreprocWrapper):
+                                    raise Exception(f"Type '{type(c_net).__name__}' requires proc_IMAGE input via a corresponding preprocessor, but received a normal Image instead.")
+                            else:
+                                if isinstance(control_hint, AbstractPreprocWrapper) and not c_net.postpone_condhint_latents_check:
+                                    raise Exception(f"Type '{type(c_net).__name__}' requires a normal Image input, but received a proc_IMAGE input instead.")
+                            # if vae required, verify vae is passed in
+                            if c_net.require_vae:
+                                # if controlnet can accept preprocced condhint latents and is the case, ignore vae requirement
+                                if c_net.allow_condhint_latents and isinstance(control_hint, AbstractPreprocWrapper):
+                                    pass
+                                elif not vae_optional:
+                                    # make sure SD3 ControlNet will get a special message instead of generic type mention
+                                    if is_sd3_advanced_controlnet(c_net):
+                                        raise Exception(f"SD3 ControlNet requires vae_optional input, but got None.")
+                                    else:
+                                        raise Exception(f"Type '{type(c_net).__name__}' requires vae_optional input, but got None.")
+                            # apply optional parameters and overrides, if provided
+                            if timestep_kf is not None:
+                                c_net.set_timestep_keyframes(timestep_kf)
+                            if latent_kf_override is not None:
+                                c_net.latent_keyframe_override = latent_kf_override
+                            if weights_override is not None:
+                                c_net.weights_override = weights_override
+                            # verify weights are compatible
+                            c_net.verify_all_weights()
+                            # set cond hint mask
+                            if mask_optional is not None:
+                                mask_optional = mask_optional.clone()
+                                # if not in the form of a batch, make it so
+                                if len(mask_optional.shape) < 3:
+                                    mask_optional = mask_optional.unsqueeze(0)
+                                c_net.set_cond_hint_mask(mask_optional)
+                        c_net.set_previous_controlnet(prev_cnet)
+                        cnets[prev_cnet] = c_net
+
+                    d['control'] = c_net
+                    d['control_apply_to_uncond'] = control_apply_to_uncond
+                    n = [t[0], d]
+                    c.append(n)
+            out.append(c)
+        return io.NodeOutput(out[0], out[1])
+    
+
+class AdvancedControlNetApplySingle(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_AdvancedControlNetApplySingle_v2',
+            display_name='Apply Advanced ControlNet(1) 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝',
+            inputs=[
+                io.Conditioning.Input('conditioning'),
+                io.ControlNet.Input('control_net'),
+                io.Image.Input('image'),
+                io.Float.Input('strength', default=1.0, max=10.0, min=0.0, step=0.01),
+                io.Float.Input('start_percent', default=0.0, max=1.0, min=0.0, step=0.001),
+                io.Float.Input('end_percent', default=1.0, max=1.0, min=0.0, step=0.001),
+                io.Mask.Input('mask_optional', optional=True),
+                io.Custom('TIMESTEP_KEYFRAME').Input('timestep_kf', optional=True),
+                io.Custom('LATENT_KEYFRAME').Input('latent_kf_override', optional=True),
+                io.Custom('CONTROL_NET_WEIGHTS').Input('weights_override', optional=True),
+                io.Vae.Input('vae_optional', optional=True)
+            ],
+            outputs=[
+                io.Conditioning.Output('CONDITIONING', is_output_list=False),
+                io.Model.Output('model_opt', is_output_list=False)
+            ]
+        )
+
+    @classmethod
+    def execute(cls, conditioning, control_net, image, strength, start_percent, end_percent,
+                         mask_optional: Tensor=None, vae_optional=None,
+                         timestep_kf: TimestepKeyframeGroup=None, latent_kf_override: LatentKeyframeGroup=None,
+                         weights_override: ControlWeights=None):
+        values = AdvancedControlNetApply.execute(positive=conditioning, negative=None, control_net=control_net, image=image,
+                                                          strength=strength, start_percent=start_percent, end_percent=end_percent,
+                                                          mask_optional=mask_optional, vae_optional=vae_optional,
+                                                          timestep_kf=timestep_kf, latent_kf_override=latent_kf_override, weights_override=weights_override,
+                                                          control_apply_to_uncond=True)
+        return io.NodeOutput(values.args[0], None)

adv_control/nodes_plusplus.py

@@ -0,0 +1,89 @@
+from comfy_api.latest import io
+from torch import Tensor
+import math
+
+import folder_paths
+
+from .control_plusplus import load_controlnetplusplus, PlusPlusInput, PlusPlusInputGroup, PlusPlusImageWrapper
+
+class PlusPlusLoaderAdvanced(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_ControlNet++LoaderAdvanced',
+            display_name='Load ControlNet++ Model (Multi) 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝/ControlNet++',
+            inputs=[
+                io.Custom('PLUS_INPUT').Input('plus_input'),
+                io.Combo.Input('name', options=folder_paths.get_filename_list("controlnet"))
+            ],
+            outputs=[
+                io.ControlNet.Output('CONTROL_NET', is_output_list=False),
+                io.Image.Output('IMAGE', is_output_list=False)
+            ]
+        )
+    
+
+    @classmethod
+    def execute(cls, plus_input: PlusPlusInputGroup, name: str):
+        controlnet_path = folder_paths.get_full_path("controlnet", name)
+        controlnet = load_controlnetplusplus(controlnet_path)
+        controlnet.verify_control_type(name, plus_input)
+        controlnet.allow_condhint_latents = True
+        return io.NodeOutput(controlnet, PlusPlusImageWrapper(plus_input),)
+
+class PlusPlusLoaderSingle(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_ControlNet++LoaderSingle',
+            display_name='Load ControlNet++ Model (Single) 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝/ControlNet++',
+            inputs=[
+                io.Combo.Input('name', options=folder_paths.get_filename_list("controlnet")),
+                io.Combo.Input('control_type', options=['openpose', 'depth', 'hed/pidi/scribble/ted', 'canny/lineart/mlsd', 'normal', 'segment', 'tile', 'inpaint/outpaint', 'none'], default='none')
+            ],
+            outputs=[
+                io.ControlNet.Output('CONTROL_NET', is_output_list=False)
+            ]
+        )
+    
+
+    @classmethod
+    def execute(cls, name: str, control_type: str):
+        controlnet_path = folder_paths.get_full_path("controlnet", name)
+        controlnet = load_controlnetplusplus(controlnet_path)
+        controlnet.single_control_type = control_type
+        controlnet.verify_control_type(name)
+        return io.NodeOutput(controlnet,)
+
+class PlusPlusInputNode(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_ControlNet++InputNode',
+            display_name='ControlNet++ Input 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝/ControlNet++',
+            inputs=[
+                io.Image.Input('image'),
+                io.Combo.Input('control_type', options=['openpose', 'depth', 'hed/pidi/scribble/ted', 'canny/lineart/mlsd', 'normal', 'segment', 'tile', 'inpaint/outpaint']),
+                io.Custom('PLUS_INPUT').Input('prev_plus_input', optional=True)
+            ],
+            outputs=[
+                io.Custom('PLUS_INPUT').Output('PLUS_INPUT', is_output_list=False)
+            ]
+        )
+    
+
+    @classmethod
+    def execute(cls, image: Tensor, control_type: str, strength=1.0, prev_plus_input: PlusPlusInputGroup=None):
+        if prev_plus_input is None:
+            prev_plus_input = PlusPlusInputGroup()
+        prev_plus_input = prev_plus_input.clone()
+
+        if math.isclose(strength, 0.0):
+            strength = 0.0000001
+        pp_input = PlusPlusInput(image, control_type, strength)
+        prev_plus_input.add(pp_input)
+
+        return io.NodeOutput(prev_plus_input,)

adv_control/nodes_reference.py

@@ -1,3 +1,4 @@
+from comfy_api.latest import io
 from torch import Tensor
 
 from nodes import VAEEncode
@@ -6,77 +7,81 @@ from comfy.sd import VAE
 
 from .control_reference import ReferenceAdvanced, ReferenceOptions, ReferenceType, ReferencePreprocWrapper
 
-
 # node for ReferenceCN
-class ReferenceControlNetNode:
+class ReferenceControlNetNode(io.ComfyNode):
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='ACN_ReferenceControlNet',
-                "reference_type": (ReferenceType._LIST,),
+            display_name='Reference ControlNet 🛂🅐🅒🅝',
-                "style_fidelity": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01}),
+            category='Adv-ControlNet 🛂🅐🅒🅝/Reference',
-                "ref_weight": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}),
+            inputs=[
-            },
+                io.Combo.Input('reference_type', options=['reference_attn', 'reference_adain', 'reference_attn+adain']),
-        }
+                io.Float.Input('style_fidelity', default=0.5, max=1.0, min=0.0, step=0.01),
+                io.Float.Input('ref_weight', default=1.0, max=1.0, min=0.0, step=0.01)
+            ],
+            outputs=[
+                io.ControlNet.Output('CONTROL_NET', is_output_list=False)
+            ]
+        )
     
-    RETURN_TYPES = ("CONTROL_NET", )
-    FUNCTION = "load_controlnet"
 
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/Reference"
+    @classmethod
-
+    def execute(cls, reference_type: str, style_fidelity: float, ref_weight: float):
-    def load_controlnet(self, reference_type: str, style_fidelity: float, ref_weight: float):
         ref_opts = ReferenceOptions.create_combo(reference_type=reference_type, style_fidelity=style_fidelity, ref_weight=ref_weight)
         controlnet = ReferenceAdvanced(ref_opts=ref_opts, timestep_keyframes=None)
-        return (controlnet,)
+        return io.NodeOutput(controlnet,)
 
-
+class ReferenceControlFinetune(io.ComfyNode):
-class ReferenceControlFinetune:
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='ACN_ReferenceControlNetFinetune',
-                "attn_style_fidelity": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01}),
+            display_name='Reference ControlNet (Finetune) 🛂🅐🅒🅝',
-                "attn_ref_weight": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}),
+            category='Adv-ControlNet 🛂🅐🅒🅝/Reference',
-                "attn_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}),
+            inputs=[
-                "adain_style_fidelity": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 1.0, "step": 0.01}),
+                io.Float.Input('attn_style_fidelity', default=0.5, max=1.0, min=0.0, step=0.01),
-                "adain_ref_weight": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}),
+                io.Float.Input('attn_ref_weight', default=1.0, max=1.0, min=0.0, step=0.01),
-                "adain_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}),
+                io.Float.Input('attn_strength', default=1.0, max=1.0, min=0.0, step=0.01),
-            },
+                io.Float.Input('adain_style_fidelity', default=0.5, max=1.0, min=0.0, step=0.01),
-        }
+                io.Float.Input('adain_ref_weight', default=1.0, max=1.0, min=0.0, step=0.01),
+                io.Float.Input('adain_strength', default=1.0, max=1.0, min=0.0, step=0.01)
+            ],
+            outputs=[
+                io.ControlNet.Output('CONTROL_NET', is_output_list=False)
+            ]
+        )
     
-    RETURN_TYPES = ("CONTROL_NET", )
-    FUNCTION = "load_controlnet"
 
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/Reference"
+    @classmethod
-
+    def execute(cls,
-    def load_controlnet(self,
                         attn_style_fidelity: float, attn_ref_weight: float, attn_strength: float,
                         adain_style_fidelity: float, adain_ref_weight: float, adain_strength: float):
         ref_opts = ReferenceOptions(reference_type=ReferenceType.ATTN_ADAIN,
                                     attn_style_fidelity=attn_style_fidelity, attn_ref_weight=attn_ref_weight, attn_strength=attn_strength,
                                     adain_style_fidelity=adain_style_fidelity, adain_ref_weight=adain_ref_weight, adain_strength=adain_strength)
         controlnet = ReferenceAdvanced(ref_opts=ref_opts, timestep_keyframes=None)
-        return (controlnet,)
+        return io.NodeOutput(controlnet,)
 
-
+class ReferencePreprocessorNode(io.ComfyNode):
-class ReferencePreprocessorNode:
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='ACN_ReferencePreprocessor',
-                "image": ("IMAGE", ),
+            display_name='Reference Preproccessor 🛂🅐🅒🅝',
-                "vae": ("VAE", ),
+            category='Adv-ControlNet 🛂🅐🅒🅝/Reference/preprocess',
-                "latent_size": ("LATENT", ),
+            inputs=[
-            }
+                io.Image.Input('image'),
-        }
+                io.Vae.Input('vae'),
+                io.Latent.Input('latent_size')
+            ],
+            outputs=[
+                io.Image.Output('proc_IMAGE', is_output_list=False)
+            ]
+        )
 
-    RETURN_TYPES = ("IMAGE",)
+    @classmethod
-    RETURN_NAMES = ("proc_IMAGE",)
+    def execute(cls, vae: VAE, image: Tensor, latent_size: Tensor):
-    FUNCTION = "preprocess_images"
-
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/Reference/preprocess"
-
-    def preprocess_images(self, vae: VAE, image: Tensor, latent_size: Tensor):
         # first, resize image to match latents
         image = image.movedim(-1,1)
         image = comfy.utils.common_upscale(image, latent_size["samples"].shape[3] * 8, latent_size["samples"].shape[2] * 8, 'nearest-exact', "center")
@@ -87,4 +92,4 @@ class ReferencePreprocessorNode:
         except Exception:
             image = VAEEncode.vae_encode_crop_pixels(image)
         encoded = vae.encode(image[:,:,:,:3])
-        return (ReferencePreprocWrapper(condhint=encoded),)
+        return io.NodeOutput(ReferencePreprocWrapper(condhint=encoded),)

adv_control/nodes_sparsectrl.py

@@ -1,3 +1,4 @@
+from comfy_api.latest import io
 from torch import Tensor
 
 import folder_paths
@@ -6,62 +7,69 @@ import comfy.utils
 from comfy.sd import VAE
 
 from .utils import TimestepKeyframeGroup
-from .control_sparsectrl import SparseMethod, SparseIndexMethod, SparseSettings, SparseSpreadMethod, PreprocSparseRGBWrapper
+from .control_sparsectrl import SparseMethod, SparseIndexMethod, SparseSettings, SparseSpreadMethod, PreprocSparseRGBWrapper, SparseConst, SparseContextAware, get_idx_list_from_str
-from .control import load_sparsectrl, load_controlnet, ControlNetAdvanced, SparseCtrlAdvanced
+from .control import load_sparsectrl, load_controlnet, ControlNetAdvanced
-
 
 # node for SparseCtrl loading
-class SparseCtrlLoaderAdvanced:
+class SparseCtrlLoaderAdvanced(io.ComfyNode):
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='ACN_SparseCtrlLoaderAdvanced',
-                "sparsectrl_name": (folder_paths.get_filename_list("controlnet"), ),
+            display_name='Load SparseCtrl Model 🛂🅐🅒🅝',
-                "use_motion": ("BOOLEAN", {"default": True}, ),
+            category='Adv-ControlNet 🛂🅐🅒🅝/SparseCtrl',
-                "motion_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+            inputs=[
-                "motion_scale": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Combo.Input('sparsectrl_name', options=folder_paths.get_filename_list("controlnet")),
-            },
+                io.Boolean.Input('use_motion', default=True),
-            "optional": {
+                io.Float.Input('motion_strength', default=1.0, max=10.0, min=0.0, step=0.001),
-                "sparse_method": ("SPARSE_METHOD", ),
+                io.Float.Input('motion_scale', default=1.0, max=10.0, min=0.0, step=0.001),
-                "tk_optional": ("TIMESTEP_KEYFRAME", ),
+                io.Custom('SPARSE_METHOD').Input('sparse_method', optional=True),
-            }
+                io.Custom('TIMESTEP_KEYFRAME').Input('tk_optional', optional=True),
-        }
+                io.Combo.Input('context_aware', optional=True, options=['nearest_hint', 'off']),
+                io.Float.Input('sparse_hint_mult', optional=True, default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('sparse_nonhint_mult', optional=True, default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('sparse_mask_mult', optional=True, default=1.0, max=10.0, min=0.0, step=0.001)
+            ],
+            outputs=[
+                io.ControlNet.Output('CONTROL_NET', is_output_list=False)
+            ]
+        )
     
-    RETURN_TYPES = ("CONTROL_NET", )
-    FUNCTION = "load_controlnet"
 
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/SparseCtrl"
+    @classmethod
-
+    def execute(cls, sparsectrl_name: str, use_motion: bool, motion_strength: float, motion_scale: float, sparse_method: SparseMethod=SparseSpreadMethod(), tk_optional: TimestepKeyframeGroup=None,
-    def load_controlnet(self, sparsectrl_name: str, use_motion: bool, motion_strength: float, motion_scale: float, sparse_method: SparseMethod=SparseSpreadMethod(), tk_optional: TimestepKeyframeGroup=None):
+                        context_aware=SparseContextAware.NEAREST_HINT, sparse_hint_mult=1.0, sparse_nonhint_mult=1.0, sparse_mask_mult=1.0):
         sparsectrl_path = folder_paths.get_full_path("controlnet", sparsectrl_name)
-        sparse_settings = SparseSettings(sparse_method=sparse_method, use_motion=use_motion, motion_strength=motion_strength, motion_scale=motion_scale)
+        sparse_settings = SparseSettings(sparse_method=sparse_method, use_motion=use_motion, motion_strength=motion_strength, motion_scale=motion_scale,
+                                         context_aware=context_aware,
+                                         sparse_mask_mult=sparse_mask_mult, sparse_hint_mult=sparse_hint_mult, sparse_nonhint_mult=sparse_nonhint_mult)
         sparsectrl = load_sparsectrl(sparsectrl_path, timestep_keyframe=tk_optional, sparse_settings=sparse_settings)
-        return (sparsectrl,)
+        return io.NodeOutput(sparsectrl,)
 
-
+class SparseCtrlMergedLoaderAdvanced(io.ComfyNode):
-class SparseCtrlMergedLoaderAdvanced:
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='ACN_SparseCtrlMergedLoaderAdvanced',
-                "sparsectrl_name": (folder_paths.get_filename_list("controlnet"), ),
+            display_name='🧪Load Merged SparseCtrl Model 🛂🅐🅒🅝',
-                "control_net_name": (folder_paths.get_filename_list("controlnet"), ),
+            category='Adv-ControlNet 🛂🅐🅒🅝/SparseCtrl/experimental',
-                "use_motion": ("BOOLEAN", {"default": True}, ),
+            inputs=[
-                "motion_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Combo.Input('sparsectrl_name', options=folder_paths.get_filename_list("controlnet")),
-                "motion_scale": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Combo.Input('control_net_name', options=folder_paths.get_filename_list("controlnet")),
-            },
+                io.Boolean.Input('use_motion', default=True),
-            "optional": {
+                io.Float.Input('motion_strength', default=1.0, max=10.0, min=0.0, step=0.001),
-                "sparse_method": ("SPARSE_METHOD", ),
+                io.Float.Input('motion_scale', default=1.0, max=10.0, min=0.0, step=0.001),
-                "tk_optional": ("TIMESTEP_KEYFRAME", ),
+                io.Custom('SPARSE_METHOD').Input('sparse_method', optional=True),
-            }
+                io.Custom('TIMESTEP_KEYFRAME').Input('tk_optional', optional=True)
-        }
+            ],
+            outputs=[
+                io.ControlNet.Output('CONTROL_NET', is_output_list=False)
+            ]
+        )
     
-    RETURN_TYPES = ("CONTROL_NET", )
-    FUNCTION = "load_controlnet"
 
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/SparseCtrl/experimental"
+    @classmethod
-
+    def execute(cls, sparsectrl_name: str, control_net_name: str, use_motion: bool, motion_strength: float, motion_scale: float, sparse_method: SparseMethod=SparseSpreadMethod(), tk_optional: TimestepKeyframeGroup=None):
-    def load_controlnet(self, sparsectrl_name: str, control_net_name: str, use_motion: bool, motion_strength: float, motion_scale: float, sparse_method: SparseMethod=SparseSpreadMethod(), tk_optional: TimestepKeyframeGroup=None):
         sparsectrl_path = folder_paths.get_full_path("controlnet", sparsectrl_name)
         controlnet_path = folder_paths.get_full_path("controlnet", control_net_name)
         sparse_settings = SparseSettings(sparse_method=sparse_method, use_motion=use_motion, motion_strength=motion_strength, motion_scale=motion_scale, merged=True)
@@ -78,78 +86,68 @@ class SparseCtrlMergedLoaderAdvanced:
             new_state_dict[key] = value
         # now, reload sparsectrl with real settings
         sparsectrl = load_sparsectrl(sparsectrl_path, controlnet_data=new_state_dict, timestep_keyframe=tk_optional, sparse_settings=sparse_settings)
-        return (sparsectrl,)
+        return io.NodeOutput(sparsectrl,)
 
-
+class SparseIndexMethodNode(io.ComfyNode):
-class SparseIndexMethodNode:
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='ACN_SparseCtrlIndexMethodNode',
-                "indexes": ("STRING", {"default": "0"}),
+            display_name='SparseCtrl Index Method 🛂🅐🅒🅝',
-            }
+            category='Adv-ControlNet 🛂🅐🅒🅝/SparseCtrl',
-        }
+            inputs=[
-    
+                io.String.Input('indexes', default='0')
-    RETURN_TYPES = ("SPARSE_METHOD",)
+            ],
-    FUNCTION = "get_method"
+            outputs=[
-
+                io.Custom('SPARSE_METHOD').Output('SPARSE_METHOD', is_output_list=False)
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/SparseCtrl"
+            ]
-
+        )
-    def get_method(self, indexes: str):
-        idxs = []
-        unique_idxs = set()
-        # get indeces from string
-        str_idxs = [x.strip() for x in indexes.strip().split(",")]
-        for str_idx in str_idxs:
-            try:
-                idx = int(str_idx)
-                if idx in unique_idxs:
-                    raise ValueError(f"'{idx}' is duplicated; indexes must be unique.")
-                idxs.append(idx)
-                unique_idxs.add(idx)
-            except ValueError:
-                raise ValueError(f"'{str_idx}' is not a valid integer index.")
-        if len(idxs) == 0:
-            raise ValueError(f"No indexes were listed in Sparse Index Method.")
-        return (SparseIndexMethod(idxs),)
     
 
-class SparseSpreadMethodNode:
     @classmethod
-    def INPUT_TYPES(s):
+    def execute(cls, indexes: str):
-        return {
+        idxs = get_idx_list_from_str(indexes)
-            "required": {
+        return io.NodeOutput(SparseIndexMethod(idxs),)
-                "spread": (SparseSpreadMethod.LIST,),
-            }
-        }
 
-    RETURN_TYPES = ("SPARSE_METHOD",)
+class SparseSpreadMethodNode(io.ComfyNode):
-    FUNCTION = "get_method"
-
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/SparseCtrl"
-
-    def get_method(self, spread: str):
-        return (SparseSpreadMethod(spread=spread),)
-
-
-class RgbSparseCtrlPreprocessor:
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='ACN_SparseCtrlSpreadMethodNode',
-                "image": ("IMAGE", ),
+            display_name='SparseCtrl Spread Method 🛂🅐🅒🅝',
-                "vae": ("VAE", ),
+            category='Adv-ControlNet 🛂🅐🅒🅝/SparseCtrl',
-                "latent_size": ("LATENT", ),
+            inputs=[
-            }
+                io.Combo.Input('spread', options=['uniform', 'starting', 'ending', 'center'])
-        }
+            ],
+            outputs=[
+                io.Custom('SPARSE_METHOD').Output('SPARSE_METHOD', is_output_list=False)
+            ]
+        )
     
-    RETURN_TYPES = ("IMAGE",)
-    RETURN_NAMES = ("proc_IMAGE",)
-    FUNCTION = "preprocess_images"
 
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/SparseCtrl/preprocess"
+    @classmethod
+    def execute(cls, spread: str):
+        return io.NodeOutput(SparseSpreadMethod(spread=spread),)
 
-    def preprocess_images(self, vae: VAE, image: Tensor, latent_size: Tensor):
+class RgbSparseCtrlPreprocessor(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_SparseCtrlRGBPreprocessor',
+            display_name='RGB SparseCtrl 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝/SparseCtrl/preprocess',
+            inputs=[
+                io.Image.Input('image'),
+                io.Vae.Input('vae'),
+                io.Latent.Input('latent_size')
+            ],
+            outputs=[
+                io.Image.Output('proc_IMAGE', is_output_list=False)
+            ]
+        )
+
+    @classmethod
+    def execute(cls, vae: VAE, image: Tensor, latent_size: Tensor):
         # first, resize image to match latents
         image = image.movedim(-1,1)
         image = comfy.utils.common_upscale(image, latent_size["samples"].shape[3] * 8, latent_size["samples"].shape[2] * 8, 'nearest-exact', "center")
@@ -160,4 +158,31 @@ class RgbSparseCtrlPreprocessor:
         except Exception:
             image = VAEEncode.vae_encode_crop_pixels(image)
         encoded = vae.encode(image[:,:,:,:3])
-        return (PreprocSparseRGBWrapper(condhint=encoded),)
+        return io.NodeOutput(PreprocSparseRGBWrapper(condhint=encoded),)
+
+class SparseWeightExtras(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_SparseCtrlWeightExtras',
+            display_name='SparseCtrl Weight Extras 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝/SparseCtrl/extras',
+            inputs=[
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True),
+                io.Float.Input('sparse_hint_mult', optional=True, default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('sparse_nonhint_mult', optional=True, default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('sparse_mask_mult', optional=True, default=1.0, max=10.0, min=0.0, step=0.001)
+            ],
+            outputs=[
+                io.Custom('CN_WEIGHTS_EXTRAS').Output('cn_extras', is_output_list=False)
+            ]
+        )
+    
+
+    @classmethod
+    def execute(cls, cn_extras: dict[str]={}, sparse_hint_mult=1.0, sparse_nonhint_mult=1.0, sparse_mask_mult=1.0):
+        cn_extras = cn_extras.copy()
+        cn_extras[SparseConst.HINT_MULT] = sparse_hint_mult
+        cn_extras[SparseConst.NONHINT_MULT] = sparse_nonhint_mult
+        cn_extras[SparseConst.MASK_MULT] = sparse_mask_mult
+        return io.NodeOutput(cn_extras, )

adv_control/nodes_weight.py

@@ -1,53 +1,57 @@
+from comfy_api.latest import io
 from torch import Tensor
 import torch
-from .utils import TimestepKeyframe, TimestepKeyframeGroup, ControlWeights, get_properly_arranged_t2i_weights, linear_conversion
+from .utils import TimestepKeyframe, TimestepKeyframeGroup, ControlWeights, Extras, get_properly_arranged_t2i_weights, linear_conversion
-from .logger import logger
+from .control_lllite import AnimaLLLiteConst
-
 
 WEIGHTS_RETURN_NAMES = ("CN_WEIGHTS", "TK_SHORTCUT")
 
-
+class DefaultWeights(io.ComfyNode):
-class DefaultWeights:
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-        }
+            node_id='ACN_DefaultUniversalWeights',
-    
+            display_name='Default Weights 🛂🅐🅒🅝',
-    RETURN_TYPES = ("CONTROL_NET_WEIGHTS", "TIMESTEP_KEYFRAME",)
+            category='Adv-ControlNet 🛂🅐🅒🅝/weights',
-    RETURN_NAMES = WEIGHTS_RETURN_NAMES
+            inputs=[
-    FUNCTION = "load_weights"
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
-
+            ],
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/weights"
+            outputs=[
-
+                io.Custom('CONTROL_NET_WEIGHTS').Output('CN_WEIGHTS', is_output_list=False),
-    def load_weights(self):
+                io.Custom('TIMESTEP_KEYFRAME').Output('TK_SHORTCUT', is_output_list=False)
-        weights = ControlWeights.default()
+            ]
-        return (weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights))) 
+        )
     
 
-class ScaledSoftMaskedUniversalWeights:
     @classmethod
-    def INPUT_TYPES(s):
+    def execute(cls, cn_extras: dict[str]={}):
-        return {
+        weights = ControlWeights.default(extras=cn_extras)
-            "required": {
+        return io.NodeOutput(weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
-                "mask": ("MASK", ),
-                "min_base_multiplier": ("FLOAT", {"default": 0.0, "min": 0.0, "max": 1.0, "step": 0.001}, ),
-                "max_base_multiplier": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.001}, ),
-                #"lock_min": ("BOOLEAN", {"default": False}, ),
-                #"lock_max": ("BOOLEAN", {"default": False}, ),
-            },
-            "optional": {
-                "uncond_multiplier": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}, ),
-            }
-        }
 
-    RETURN_TYPES = ("CONTROL_NET_WEIGHTS", "TIMESTEP_KEYFRAME",)
+class ScaledSoftMaskedUniversalWeights(io.ComfyNode):
-    RETURN_NAMES = WEIGHTS_RETURN_NAMES
+    @classmethod
-    FUNCTION = "load_weights"
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ScaledSoftMaskedUniversalWeights',
+            display_name='Scaled Soft Masked Weights 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝/weights',
+            inputs=[
+                io.Mask.Input('mask'),
+                io.Float.Input('min_base_multiplier', default=0.0, max=1.0, min=0.0, step=0.001),
+                io.Float.Input('max_base_multiplier', default=1.0, max=1.0, min=0.0, step=0.001),
+                io.Float.Input('uncond_multiplier', optional=True, default=1.0, max=1.0, min=0.0, step=0.01),
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
+            ],
+            outputs=[
+                io.Custom('CONTROL_NET_WEIGHTS').Output('CN_WEIGHTS', is_output_list=False),
+                io.Custom('TIMESTEP_KEYFRAME').Output('TK_SHORTCUT', is_output_list=False)
+            ]
+        )
     
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/weights"
 
-    def load_weights(self, mask: Tensor, min_base_multiplier: float, max_base_multiplier: float, lock_min=False, lock_max=False,
+    @classmethod
-                     uncond_multiplier: float=1.0):
+    def execute(cls, mask: Tensor, min_base_multiplier: float, max_base_multiplier: float, lock_min=False, lock_max=False,
+                     uncond_multiplier: float=1.0, cn_extras: dict[str]={}):
         # normalize mask
         mask = mask.clone()
         x_min = 0.0 if lock_min else mask.min()
@@ -56,169 +60,319 @@ class ScaledSoftMaskedUniversalWeights:
             mask = torch.ones_like(mask) * max_base_multiplier
         else:
             mask = linear_conversion(mask, x_min, x_max, min_base_multiplier, max_base_multiplier)
-        weights = ControlWeights.universal_mask(weight_mask=mask, uncond_multiplier=uncond_multiplier)
+        weights = ControlWeights.universal_mask(weight_mask=mask, uncond_multiplier=uncond_multiplier, extras=cn_extras)
-        return (weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
+        return io.NodeOutput(weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
 
-
+class ScaledSoftUniversalWeights(io.ComfyNode):
-class ScaledSoftUniversalWeights:
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='ACN_ScaledSoftControlNetWeights',
-                "base_multiplier": ("FLOAT", {"default": 0.825, "min": 0.0, "max": 1.0, "step": 0.001}, ),
+            display_name='Scaled Soft Weights 🛂🅐🅒🅝',
-                "flip_weights": ("BOOLEAN", {"default": False}),
+            category='Adv-ControlNet 🛂🅐🅒🅝/weights',
-            },
+            inputs=[
-            "optional": {
+                io.Float.Input('base_multiplier', default=0.825, max=1.0, min=0.0, step=0.001),
-                "uncond_multiplier": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}, ),
+                io.Float.Input('uncond_multiplier', optional=True, default=1.0, max=1.0, min=0.0, step=0.01),
-            }
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
-        }
+            ],
-    
+            outputs=[
-    RETURN_TYPES = ("CONTROL_NET_WEIGHTS", "TIMESTEP_KEYFRAME",)
+                io.Custom('CONTROL_NET_WEIGHTS').Output('CN_WEIGHTS', is_output_list=False),
-    RETURN_NAMES = WEIGHTS_RETURN_NAMES
+                io.Custom('TIMESTEP_KEYFRAME').Output('TK_SHORTCUT', is_output_list=False)
-    FUNCTION = "load_weights"
+            ]
-
+        )
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/weights"
-
-    def load_weights(self, base_multiplier, flip_weights, uncond_multiplier: float=1.0):
-        weights = ControlWeights.universal(base_multiplier=base_multiplier, flip_weights=flip_weights, uncond_multiplier=uncond_multiplier)
-        return (weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights))) 
     
 
-class SoftControlNetWeights:
     @classmethod
-    def INPUT_TYPES(s):
+    def execute(cls, base_multiplier, uncond_multiplier: float=1.0, cn_extras: dict[str]={}):
-        return {
+        weights = ControlWeights.universal(base_multiplier=base_multiplier, uncond_multiplier=uncond_multiplier, extras=cn_extras)
-            "required": {
+        return io.NodeOutput(weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
-                "weight_00": ("FLOAT", {"default": 0.09941396206337118, "min": 0.0, "max": 10.0, "step": 0.001}, ),
-                "weight_01": ("FLOAT", {"default": 0.12050177219802567, "min": 0.0, "max": 10.0, "step": 0.001}, ),
-                "weight_02": ("FLOAT", {"default": 0.14606275417942507, "min": 0.0, "max": 10.0, "step": 0.001}, ),
-                "weight_03": ("FLOAT", {"default": 0.17704576264172736, "min": 0.0, "max": 10.0, "step": 0.001}, ),
-                "weight_04": ("FLOAT", {"default": 0.214600924414215, "min": 0.0, "max": 10.0, "step": 0.001}, ),
-                "weight_05": ("FLOAT", {"default": 0.26012233262329093, "min": 0.0, "max": 10.0, "step": 0.001}, ),
-                "weight_06": ("FLOAT", {"default": 0.3152997971191405, "min": 0.0, "max": 10.0, "step": 0.001}, ),
-                "weight_07": ("FLOAT", {"default": 0.3821815722656249, "min": 0.0, "max": 10.0, "step": 0.001}, ),
-                "weight_08": ("FLOAT", {"default": 0.4632503906249999, "min": 0.0, "max": 10.0, "step": 0.001}, ),
-                "weight_09": ("FLOAT", {"default": 0.561515625, "min": 0.0, "max": 10.0, "step": 0.001}, ),
-                "weight_10": ("FLOAT", {"default": 0.6806249999999999, "min": 0.0, "max": 10.0, "step": 0.001}, ),
-                "weight_11": ("FLOAT", {"default": 0.825, "min": 0.0, "max": 10.0, "step": 0.001}, ),
-                "weight_12": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
-                "flip_weights": ("BOOLEAN", {"default": False}),
-            },
-            "optional": {
-                "uncond_multiplier": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}, ),
-            }
-        }
 
-    RETURN_TYPES = ("CONTROL_NET_WEIGHTS", "TIMESTEP_KEYFRAME",)
+class SoftControlNetWeightsSD15(io.ComfyNode):
-    RETURN_NAMES = WEIGHTS_RETURN_NAMES
-    FUNCTION = "load_weights"
-
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/weights/ControlNet"
-
-    def load_weights(self, weight_00, weight_01, weight_02, weight_03, weight_04, weight_05, weight_06, 
-                     weight_07, weight_08, weight_09, weight_10, weight_11, weight_12, flip_weights,
-                     uncond_multiplier: float=1.0):
-        weights = [weight_00, weight_01, weight_02, weight_03, weight_04, weight_05, weight_06, 
-                   weight_07, weight_08, weight_09, weight_10, weight_11, weight_12]
-        weights = ControlWeights.controlnet(weights, flip_weights=flip_weights, uncond_multiplier=uncond_multiplier)
-        return (weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
-
-
-class CustomControlNetWeights:
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='ACN_SoftControlNetWeightsSD15',
-                "weight_00": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+            display_name='ControlNet Soft Weights [SD1.5] 🛂🅐🅒🅝',
-                "weight_01": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+            category='Adv-ControlNet 🛂🅐🅒🅝/weights/ControlNet',
-                "weight_02": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+            inputs=[
-                "weight_03": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Float.Input('output_0', default=0.09941396206337118, max=10.0, min=0.0, step=0.001),
-                "weight_04": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Float.Input('output_1', default=0.12050177219802567, max=10.0, min=0.0, step=0.001),
-                "weight_05": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Float.Input('output_2', default=0.14606275417942507, max=10.0, min=0.0, step=0.001),
-                "weight_06": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Float.Input('output_3', default=0.17704576264172736, max=10.0, min=0.0, step=0.001),
-                "weight_07": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Float.Input('output_4', default=0.214600924414215, max=10.0, min=0.0, step=0.001),
-                "weight_08": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Float.Input('output_5', default=0.26012233262329093, max=10.0, min=0.0, step=0.001),
-                "weight_09": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Float.Input('output_6', default=0.3152997971191405, max=10.0, min=0.0, step=0.001),
-                "weight_10": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Float.Input('output_7', default=0.3821815722656249, max=10.0, min=0.0, step=0.001),
-                "weight_11": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Float.Input('output_8', default=0.4632503906249999, max=10.0, min=0.0, step=0.001),
-                "weight_12": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Float.Input('output_9', default=0.561515625, max=10.0, min=0.0, step=0.001),
-                "flip_weights": ("BOOLEAN", {"default": False}),
+                io.Float.Input('output_10', default=0.6806249999999999, max=10.0, min=0.0, step=0.001),
-            },
+                io.Float.Input('output_11', default=0.825, max=10.0, min=0.0, step=0.001),
-            "optional": {
+                io.Float.Input('middle_0', default=1.0, max=10.0, min=0.0, step=0.001),
-                "uncond_multiplier": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}, ),
+                io.Float.Input('uncond_multiplier', optional=True, default=1.0, max=1.0, min=0.0, step=0.01),
-            }
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
-        }
+            ],
-    
+            outputs=[
-    RETURN_TYPES = ("CONTROL_NET_WEIGHTS", "TIMESTEP_KEYFRAME",)
+                io.Custom('CONTROL_NET_WEIGHTS').Output('CN_WEIGHTS', is_output_list=False),
-    RETURN_NAMES = WEIGHTS_RETURN_NAMES
+                io.Custom('TIMESTEP_KEYFRAME').Output('TK_SHORTCUT', is_output_list=False)
-    FUNCTION = "load_weights"
+            ]
-
+        )
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/weights/ControlNet"
-
-    def load_weights(self, weight_00, weight_01, weight_02, weight_03, weight_04, weight_05, weight_06, 
-                     weight_07, weight_08, weight_09, weight_10, weight_11, weight_12, flip_weights,
-                     uncond_multiplier: float=1.0):
-        weights = [weight_00, weight_01, weight_02, weight_03, weight_04, weight_05, weight_06, 
-                   weight_07, weight_08, weight_09, weight_10, weight_11, weight_12]
-        weights = ControlWeights.controlnet(weights, flip_weights=flip_weights, uncond_multiplier=uncond_multiplier)
-        return (weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
     
 
-class SoftT2IAdapterWeights:
     @classmethod
-    def INPUT_TYPES(s):
+    def execute(cls, output_0, output_1, output_2, output_3, output_4, output_5, output_6,
-        return {
+                     output_7, output_8, output_9, output_10, output_11, middle_0,
-            "required": {
+                     uncond_multiplier: float=1.0, cn_extras: dict[str]={}):
-                "weight_00": ("FLOAT", {"default": 0.25, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+        return CustomControlNetWeightsSD15.execute(
-                "weight_01": ("FLOAT", {"default": 0.62, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                                                        output_0=output_0, output_1=output_1, output_2=output_2, output_3=output_3,
-                "weight_02": ("FLOAT", {"default": 0.825, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                                                        output_4=output_4, output_5=output_5, output_6=output_6, output_7=output_7,
-                "weight_03": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                                                        output_8=output_8, output_9=output_9, output_10=output_10, output_11=output_11,
-                "flip_weights": ("BOOLEAN", {"default": False}),
+                                                        middle_0=middle_0,
-            },
+                                                        uncond_multiplier=uncond_multiplier, cn_extras=cn_extras)
-            "optional": {
-                "uncond_multiplier": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}, ),
-            }
-        }
 
-    RETURN_TYPES = ("CONTROL_NET_WEIGHTS", "TIMESTEP_KEYFRAME",)
+class CustomControlNetWeightsSD15(io.ComfyNode):
-    RETURN_NAMES = WEIGHTS_RETURN_NAMES
+    @classmethod
-    FUNCTION = "load_weights"
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_CustomControlNetWeightsSD15',
+            display_name='ControlNet Custom Weights [SD1.5] 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝/weights/ControlNet',
+            inputs=[
+                io.Float.Input('output_0', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('output_1', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('output_2', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('output_3', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('output_4', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('output_5', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('output_6', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('output_7', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('output_8', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('output_9', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('output_10', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('output_11', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('middle_0', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('uncond_multiplier', optional=True, default=1.0, max=1.0, min=0.0, step=0.01),
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
+            ],
+            outputs=[
+                io.Custom('CONTROL_NET_WEIGHTS').Output('CN_WEIGHTS', is_output_list=False),
+                io.Custom('TIMESTEP_KEYFRAME').Output('TK_SHORTCUT', is_output_list=False)
+            ]
+        )
     
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/weights/T2IAdapter"
 
-    def load_weights(self, weight_00, weight_01, weight_02, weight_03, flip_weights,
+    @classmethod
-                     uncond_multiplier: float=1.0):
+    def execute(cls, output_0, output_1, output_2, output_3, output_4, output_5, output_6,
-        weights = [weight_00, weight_01, weight_02, weight_03]
+                     output_7, output_8, output_9, output_10, output_11, middle_0,
+                     uncond_multiplier: float=1.0, cn_extras: dict[str]={}):
+        weights_output = [output_0, output_1, output_2, output_3, output_4, output_5, output_6,
+                          output_7, output_8, output_9, output_10, output_11]
+        weights_middle = [middle_0]
+        weights = ControlWeights.controlnet(weights_output=weights_output, weights_middle=weights_middle, uncond_multiplier=uncond_multiplier,
+                                            extras=cn_extras, disable_applied_to=True)
+        return io.NodeOutput(weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
+
+class CustomControlNetWeightsFlux(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_CustomControlNetWeightsFlux',
+            display_name='ControlNet Custom Weights [Flux] 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝/weights/ControlNet',
+            inputs=[
+                io.Float.Input('input_0', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_1', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_2', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_3', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_4', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_5', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_6', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_7', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_8', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_9', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_10', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_11', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_12', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_13', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_14', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_15', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_16', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_17', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_18', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('uncond_multiplier', optional=True, default=1.0, max=1.0, min=0.0, step=0.01),
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
+            ],
+            outputs=[
+                io.Custom('CONTROL_NET_WEIGHTS').Output('CN_WEIGHTS', is_output_list=False),
+                io.Custom('TIMESTEP_KEYFRAME').Output('TK_SHORTCUT', is_output_list=False)
+            ]
+        )
+    
+
+    @classmethod
+    def execute(cls, input_0, input_1, input_2, input_3, input_4, input_5, input_6,
+                     input_7, input_8, input_9, input_10, input_11, input_12, input_13,
+                     input_14, input_15, input_16, input_17, input_18,
+                     uncond_multiplier: float=1.0, cn_extras: dict[str]={}):
+        weights_input = [input_0, input_1, input_2, input_3, input_4, input_5,
+                         input_6, input_7, input_8, input_9, input_10, input_11,
+                         input_12, input_13, input_14, input_15, input_16, input_17, input_18]
+        weights = ControlWeights.controlnet(weights_input=weights_input, uncond_multiplier=uncond_multiplier, extras=cn_extras, disable_applied_to=True)
+        return io.NodeOutput(weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
+
+class CustomControlNetWeightsAnima(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_CustomControlNetWeightsAnima',
+            display_name='ControlNet Custom Weights [Anima] 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝/weights/ControlNet',
+            inputs=[
+                io.Float.Input('block_0', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_1', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_2', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_3', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_4', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_5', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_6', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_7', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_8', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_9', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_10', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_11', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_12', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_13', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_14', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_15', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_16', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_17', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_18', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_19', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_20', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_21', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_22', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_23', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_24', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_25', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_26', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('block_27', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('uncond_multiplier', optional=True, default=1.0, max=1.0, min=0.0, step=0.01),
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
+            ],
+            outputs=[
+                io.Custom('CONTROL_NET_WEIGHTS').Output('CN_WEIGHTS', is_output_list=False),
+                io.Custom('TIMESTEP_KEYFRAME').Output('TK_SHORTCUT', is_output_list=False)
+            ]
+        )
+
+    @classmethod
+    def execute(cls, uncond_multiplier: float=1.0, cn_extras: dict[str]={}, **kwargs):
+        weights = [kwargs[f"block_{index}"] for index in range(28)]
+        control_weights = ControlWeights.controllllite(
+            weights_input=weights,
+            uncond_multiplier=uncond_multiplier,
+            extras=cn_extras,
+        )
+        return io.NodeOutput(control_weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=control_weights)))
+
+class SoftT2IAdapterWeights(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_SoftT2IAdapterWeights',
+            display_name='T2IAdapter Soft Weights 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝/weights/T2IAdapter',
+            inputs=[
+                io.Float.Input('input_0', default=0.25, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_1', default=0.62, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_2', default=0.825, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_3', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('uncond_multiplier', optional=True, default=1.0, max=1.0, min=0.0, step=0.01),
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
+            ],
+            outputs=[
+                io.Custom('CONTROL_NET_WEIGHTS').Output('CN_WEIGHTS', is_output_list=False),
+                io.Custom('TIMESTEP_KEYFRAME').Output('TK_SHORTCUT', is_output_list=False)
+            ]
+        )
+    
+
+    @classmethod
+    def execute(cls, input_0, input_1, input_2, input_3,
+                     uncond_multiplier: float=1.0, cn_extras: dict[str]={}):
+        return CustomT2IAdapterWeights.execute(input_0=input_0, input_1=input_1, input_2=input_2, input_3=input_3,
+                                                    uncond_multiplier=uncond_multiplier, cn_extras=cn_extras)
+
+class CustomT2IAdapterWeights(io.ComfyNode):
+    @classmethod
+    def define_schema(cls) -> io.Schema:
+        return io.Schema(
+            node_id='ACN_CustomT2IAdapterWeights',
+            display_name='T2IAdapter Custom Weights 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝/weights/T2IAdapter',
+            inputs=[
+                io.Float.Input('input_0', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_1', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_2', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('input_3', default=1.0, max=10.0, min=0.0, step=0.001),
+                io.Float.Input('uncond_multiplier', optional=True, default=1.0, max=1.0, min=0.0, step=0.01),
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
+            ],
+            outputs=[
+                io.Custom('CONTROL_NET_WEIGHTS').Output('CN_WEIGHTS', is_output_list=False),
+                io.Custom('TIMESTEP_KEYFRAME').Output('TK_SHORTCUT', is_output_list=False)
+            ]
+        )
+    
+
+    @classmethod
+    def execute(cls, input_0, input_1, input_2, input_3,
+                     uncond_multiplier: float=1.0, cn_extras: dict[str]={}):
+        weights = [input_0, input_1, input_2, input_3]
         weights = get_properly_arranged_t2i_weights(weights)
-        weights = ControlWeights.t2iadapter(weights, flip_weights=flip_weights, uncond_multiplier=uncond_multiplier)
+        weights = ControlWeights.t2iadapter(weights_input=weights, uncond_multiplier=uncond_multiplier, extras=cn_extras, disable_applied_to=True)
-        return (weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
+        return io.NodeOutput(weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
 
-
+class ExtrasMiddleMultNode(io.ComfyNode):
-class CustomT2IAdapterWeights:
     @classmethod
-    def INPUT_TYPES(s):
+    def define_schema(cls) -> io.Schema:
-        return {
+        return io.Schema(
-            "required": {
+            node_id='ACN_ExtrasMiddleMult',
-                "weight_00": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+            display_name='Middle Weight Extras 🛂🅐🅒🅝',
-                "weight_01": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+            category='Adv-ControlNet 🛂🅐🅒🅝/weights/extras',
-                "weight_02": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+            inputs=[
-                "weight_03": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 10.0, "step": 0.001}, ),
+                io.Float.Input('middle_mult', default=1.0, max=10.0, min=0.0, step=0.001),
-                "flip_weights": ("BOOLEAN", {"default": False}),
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
-            },
+            ],
-            "optional": {
+            outputs=[
-                "uncond_multiplier": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.0, "step": 0.01}, ),
+                io.Custom('CN_WEIGHTS_EXTRAS').Output('cn_extras', is_output_list=False)
-            }
+            ]
-        }
+        )
     
-    RETURN_TYPES = ("CONTROL_NET_WEIGHTS", "TIMESTEP_KEYFRAME",)
-    RETURN_NAMES = WEIGHTS_RETURN_NAMES
-    FUNCTION = "load_weights"
 
-    CATEGORY = "Adv-ControlNet 🛂🅐🅒🅝/weights/T2IAdapter"
+    @classmethod
+    def execute(cls, middle_mult: float, cn_extras: dict[str]={}):
+        cn_extras = cn_extras.copy()
+        cn_extras[Extras.MIDDLE_MULT] = middle_mult
+        return io.NodeOutput(cn_extras,)
 
-    def load_weights(self, weight_00, weight_01, weight_02, weight_03, flip_weights,
+class AnimaLLLiteExtras(io.ComfyNode):
-                     uncond_multiplier: float=1.0):
+    @classmethod
-        weights = [weight_00, weight_01, weight_02, weight_03]
+    def define_schema(cls) -> io.Schema:
-        weights = get_properly_arranged_t2i_weights(weights)
+        return io.Schema(
-        weights = ControlWeights.t2iadapter(weights, flip_weights=flip_weights, uncond_multiplier=uncond_multiplier)
+            node_id='ACN_AnimaLLLiteExtras',
-        return (weights, TimestepKeyframeGroup.default(TimestepKeyframe(control_weights=weights)))
+            display_name='Anima LLLite Extras 🛂🅐🅒🅝',
+            category='Adv-ControlNet 🛂🅐🅒🅝/weights/extras',
+            inputs=[
+                io.Mask.Input('inpaint_mask'),
+                io.Custom('CN_WEIGHTS_EXTRAS').Input('cn_extras', optional=True)
+            ],
+            outputs=[
+                io.Custom('CN_WEIGHTS_EXTRAS').Output('cn_extras', is_output_list=False)
+            ]
+        )
+
+    @classmethod
+    def execute(cls, inpaint_mask: Tensor, cn_extras: dict[str]={}):
+        cn_extras = cn_extras.copy()
+        cn_extras[AnimaLLLiteConst.INPAINT_MASK] = inpaint_mask.clone()
+        return io.NodeOutput(cn_extras,)

adv_control/sampling.py

@@ -0,0 +1,225 @@
+from typing import Callable, Union
+
+import comfy.hooks
+import comfy.model_patcher
+import comfy.patcher_extension
+import comfy.sample
+import comfy.samplers
+from comfy.model_patcher import ModelPatcher
+from comfy.controlnet import ControlBase
+from comfy.ldm.modules.attention import BasicTransformerBlock
+
+
+from .control import convert_all_to_advanced, restore_all_controlnet_conns
+from .control_reference import (ReferenceAdvanced, ReferenceInjections,
+                                RefBasicTransformerBlock, RefTimestepEmbedSequential,
+                                InjectionBasicTransformerBlockHolder, InjectionTimestepEmbedSequentialHolder,
+                                _forward_inject_BasicTransformerBlock,
+                                handle_context_ref_setup, handle_reference_injection,
+                                REF_CONTROL_LIST_ALL, CONTEXTREF_CLEAN_FUNC)
+from .dinklink import get_dinklink
+from .utils import torch_dfs, WrapperConsts, CURRENT_WRAPPER_VERSION
+
+def prepare_dinklink_acn_wrapper():
+    # expose acn_sampler_sample_wrapper
+    d = get_dinklink()
+    link_acn = d.setdefault(WrapperConsts.ACN, {})
+    link_acn[WrapperConsts.VERSION] = CURRENT_WRAPPER_VERSION
+    link_acn[WrapperConsts.ACN_CREATE_SAMPLER_SAMPLE_WRAPPER] = (comfy.patcher_extension.WrappersMP.OUTER_SAMPLE,
+                                                                 WrapperConsts.ACN_OUTER_SAMPLE_WRAPPER_KEY,
+                                                                 acn_outer_sample_wrapper)
+
+
+def support_sliding_context_windows(conds) -> tuple[bool, list[dict]]:
+    # convert to advanced, with report if anything was actually modified
+    modified, new_conds = convert_all_to_advanced(conds)
+    return modified, new_conds
+
+
+def has_sliding_context_windows(model: ModelPatcher):
+    params = model.get_attachment("ADE_params")
+    if params is None:
+        # backwards compatibility
+        params = getattr(model, "motion_injection_params", None)
+        if params is None:
+            return False
+    context_options = getattr(params, "context_options")
+    return context_options.context_length is not None
+
+
+def get_contextref_obj(model: ModelPatcher):
+    params = model.get_attachment("ADE_params")
+    if params is None:
+        # backwards compatibility
+        params = getattr(model, "motion_injection_params", None)
+        if params is None:
+            return None
+    context_options = getattr(params, "context_options")
+    extras = getattr(context_options, "extras", None)
+    if extras is None:
+        return None
+    return getattr(extras, "context_ref", None)
+
+
+def get_refcn(control: ControlBase, order: int=-1):
+    ref_set: set[ReferenceAdvanced] = set()
+    if control is None:
+        return ref_set
+    if type(control) == ReferenceAdvanced and not control.is_context_ref:
+        control.order = order
+        order -= 1
+        ref_set.add(control)
+    ref_set.update(get_refcn(control.previous_controlnet, order=order))
+    return ref_set
+
+
+def should_register_outer_sample_wrapper(hook, model, model_options: dict, target, registered: list):
+    wrappers = comfy.patcher_extension.get_wrappers_with_key(comfy.patcher_extension.WrappersMP.OUTER_SAMPLE,
+                                                  WrapperConsts.ACN_OUTER_SAMPLE_WRAPPER_KEY,
+                                                  model_options, is_model_options=True)
+    return len(wrappers) == 0
+
+def create_wrapper_hooks():
+    wrappers = {}
+    comfy.patcher_extension.add_wrapper_with_key(comfy.patcher_extension.WrappersMP.OUTER_SAMPLE,
+                                                 WrapperConsts.ACN_OUTER_SAMPLE_WRAPPER_KEY,
+                                                 acn_outer_sample_wrapper,
+                                                 transformer_options=wrappers)
+    hooks = comfy.hooks.HookGroup()
+    hook = comfy.hooks.WrapperHook(wrappers)
+    hook.hook_id = WrapperConsts.ACN_OUTER_SAMPLE_WRAPPER_KEY
+    hook.custom_should_register = should_register_outer_sample_wrapper
+    hooks.add(hook)
+    return hooks
+
+def acn_outer_sample_wrapper(executor, *args, **kwargs):
+    controlnets_modified = False
+    guider: comfy.samplers.CFGGuider = executor.class_obj
+    model = guider.model_patcher
+    orig_conds = guider.conds
+    orig_model_options = guider.model_options
+    try:
+        new_model_options = orig_model_options
+        # if context options present, perform some special actions that may be required
+        context_refs = []
+        if has_sliding_context_windows(guider.model_patcher):
+            new_model_options = comfy.model_patcher.create_model_options_clone(new_model_options)
+            # convert all CNs to Advanced if needed
+            controlnets_modified, conds = support_sliding_context_windows(orig_conds)
+            if controlnets_modified:
+                guider.conds = conds
+            # enable ContextRef, if requested
+            existing_contextref_obj = get_contextref_obj(guider.model_patcher)
+            if existing_contextref_obj is not None:
+                context_refs = handle_context_ref_setup(existing_contextref_obj, new_model_options["transformer_options"], guider.conds)
+                controlnets_modified = True
+        # look for Advanced ControlNets that will require intervention to work
+        ref_set = set()
+        for outer_cond in guider.conds.values():
+            for cond in outer_cond:
+                if "control" in cond:
+                    ref_set.update(get_refcn(cond["control"]))
+        # if no ref cn found, do original function immediately
+        if len(ref_set) == 0 and len(context_refs) == 0:
+            return executor(*args, **kwargs)
+        # otherwise, injection time
+        try:
+            # inject
+            # storage for all Reference-related injections
+            reference_injections = ReferenceInjections()
+
+            # first, handle attn module injection
+            all_modules = torch_dfs(model.model)
+            attn_modules: list[RefBasicTransformerBlock] = []
+            for module in all_modules:
+                if isinstance(module, BasicTransformerBlock):
+                    attn_modules.append(module)
+            attn_modules = [module for module in all_modules if isinstance(module, BasicTransformerBlock)]
+            attn_modules = sorted(attn_modules, key=lambda x: -x.norm1.normalized_shape[0])
+            for i, module in enumerate(attn_modules):
+                injection_holder = InjectionBasicTransformerBlockHolder(block=module, idx=i)
+                injection_holder.attn_weight = float(i) / float(len(attn_modules))
+                if hasattr(module, "_forward"): # backward compatibility
+                    module._forward = _forward_inject_BasicTransformerBlock.__get__(module, type(module))
+                else:
+                    module.forward = _forward_inject_BasicTransformerBlock.__get__(module, type(module))
+                module.injection_holder = injection_holder
+                reference_injections.attn_modules.append(module)
+            # figure out which module is middle block
+            if hasattr(model.model.diffusion_model, "middle_block"):
+                mid_modules = torch_dfs(model.model.diffusion_model.middle_block)
+                mid_attn_modules: list[RefBasicTransformerBlock] = [module for module in mid_modules if isinstance(module, BasicTransformerBlock)]
+                for module in mid_attn_modules:
+                    module.injection_holder.is_middle = True
+
+            # next, handle gn module injection (TimestepEmbedSequential)
+            # TODO: figure out the logic behind these hardcoded indexes
+            if type(model.model).__name__ == "SDXL":
+                input_block_indices = [4, 5, 7, 8]
+                output_block_indices = [0, 1, 2, 3, 4, 5]
+            else:
+                input_block_indices = [4, 5, 7, 8, 10, 11]
+                output_block_indices = [0, 1, 2, 3, 4, 5, 6, 7]
+            if hasattr(model.model.diffusion_model, "middle_block"):
+                module = model.model.diffusion_model.middle_block
+                injection_holder = InjectionTimestepEmbedSequentialHolder(block=module, idx=0, is_middle=True)
+                injection_holder.gn_weight = 0.0
+                module.injection_holder = injection_holder
+                reference_injections.gn_modules.append(module)
+            for w, i in enumerate(input_block_indices):
+                module = model.model.diffusion_model.input_blocks[i]
+                injection_holder = InjectionTimestepEmbedSequentialHolder(block=module, idx=i, is_input=True)
+                injection_holder.gn_weight = 1.0 - float(w) / float(len(input_block_indices))
+                module.injection_holder = injection_holder
+                reference_injections.gn_modules.append(module)
+            for w, i in enumerate(output_block_indices):
+                module = model.model.diffusion_model.output_blocks[i]
+                injection_holder = InjectionTimestepEmbedSequentialHolder(block=module, idx=i, is_output=True)
+                injection_holder.gn_weight = float(w) / float(len(output_block_indices))
+                module.injection_holder = injection_holder
+                reference_injections.gn_modules.append(module)
+            # hack gn_module forwards and update weights
+            for i, module in enumerate(reference_injections.gn_modules):
+                module.injection_holder.gn_weight *= 2
+
+            # store ordered ref cns in model's transformer options
+            new_model_options = comfy.model_patcher.create_model_options_clone(new_model_options)
+            # handle diffusion_model forward injection
+            handle_reference_injection(new_model_options, reference_injections)
+
+            ref_list: list[ReferenceAdvanced] = list(ref_set)
+            new_model_options["transformer_options"][REF_CONTROL_LIST_ALL] = sorted(ref_list, key=lambda x: x.order)
+            new_model_options["transformer_options"][CONTEXTREF_CLEAN_FUNC] = reference_injections.clean_contextref_module_mem
+            guider.model_options = new_model_options
+            # continue with original function
+            return executor(*args, **kwargs)
+        finally:
+            # cleanup injections
+            # restore attn modules
+            attn_modules: list[RefBasicTransformerBlock] = reference_injections.attn_modules
+            for module in attn_modules:
+                module.injection_holder.restore(module)
+                module.injection_holder.clean_all()
+                del module.injection_holder
+            del attn_modules
+            # restore gn modules
+            gn_modules: list[RefTimestepEmbedSequential] = reference_injections.gn_modules
+            for module in gn_modules:
+                module.injection_holder.restore(module)
+                module.injection_holder.clean_all()
+                del module.injection_holder
+            del gn_modules
+            # cleanup
+            reference_injections.cleanup()
+    finally:
+        # restore model_options
+        guider.model_options = orig_model_options
+        # restore guider.conds
+        guider.conds = orig_conds
+        # restore controlnets in conds, if needed
+        if controlnets_modified:
+            restore_all_controlnet_conns(guider.conds)
+        del orig_conds
+        del orig_model_options
+        del model
+        del guider

adv_control/utils.py

@@ -3,22 +3,29 @@ from typing import Callable, Union
 import torch
 from torch import Tensor
 import torch.nn.functional
+from einops import rearrange
 import numpy as np
 import math
 
 import comfy.ops
 import comfy.utils
-import comfy.sample
-import comfy.samplers
-import comfy.model_base
 
 from comfy.controlnet import ControlBase
 from comfy.model_patcher import ModelPatcher
+from comfy.sd import VAE
 
 from .logger import logger
 
 BIGMIN = -(2**53-1)
 BIGMAX = (2**53-1)
+BIGMAX_TENSOR = torch.tensor(9999999999.9)
+
+ORIG_PREVIOUS_CONTROLNET = "_orig_previous_controlnet"
+CONTROL_INIT_BY_ACN = "_control_init_by_ACN"
+
+class Extras:
+    MIDDLE_MULT = "middle_mult"
+
 
 def load_torch_file_with_dict_factory(controlnet_data: dict[str, Tensor], orig_load_torch_file: Callable):
     def load_torch_file_with_dict(*args, **kwargs):
@@ -27,107 +34,14 @@ def load_torch_file_with_dict_factory(controlnet_data: dict[str, Tensor], orig_l
         return controlnet_data
     return load_torch_file_with_dict
 
-# wrapping len function so that it will save the thing len is trying to get the length of;
-# this will be assumed to be the cond_or_uncond variable;
-# automatically restores len to original function after running
-def wrapper_len_factory(orig_len: Callable) -> Callable:
-    def wrapper_len(*args, **kwargs):
-        cond_or_uncond = args[0]
-        real_length = orig_len(*args, **kwargs)
-        if real_length > 0 and type(cond_or_uncond) == list and (cond_or_uncond[0] in [0, 1]):
-            try:
-                to_return = IntWithCondOrUncond(real_length)
-                setattr(to_return, "cond_or_uncond", cond_or_uncond)
-                return to_return
-            finally:
-                __builtins__["len"] = orig_len
-        else:
-            return real_length
-    return wrapper_len
 
-# wrapping cond_cat function so that it will wrap around len function to get cond_or_uncond variable value
+CURRENT_WRAPPER_VERSION = 10002
-# from comfy.samplers.calc_conds_batch
-def wrapper_cond_cat_factory(orig_cond_cat: Callable):
-    def wrapper_cond_cat(*args, **kwargs):
-        __builtins__["len"] = wrapper_len_factory(__builtins__["len"])
-        return orig_cond_cat(*args, **kwargs)
-    return wrapper_cond_cat
-orig_cond_cat = comfy.samplers.cond_cat
-comfy.samplers.cond_cat = wrapper_cond_cat_factory(orig_cond_cat)
-
-
-# wrapping apply_model so that len function will be cleaned up fairly soon after being injected
-def apply_model_uncond_cleanup_factory(orig_apply_model, orig_len):
-    def apply_model_uncond_cleanup_wrapper(self, *args, **kwargs):
-        __builtins__["len"] = orig_len
-        return orig_apply_model(self, *args, **kwargs)
-    return apply_model_uncond_cleanup_wrapper
-global_orig_len = __builtins__["len"]
-orig_apply_model = comfy.model_base.BaseModel.apply_model
-comfy.model_base.BaseModel.apply_model = apply_model_uncond_cleanup_factory(orig_apply_model, global_orig_len)
-
-
-def uncond_multiplier_check_cn_sample_factory(orig_comfy_sample: Callable, is_custom=False) -> Callable:
-    def contains_uncond_multiplier(control: Union[ControlBase, 'AdvancedControlBase']):
-        if control is None:
-            return False
-        if not isinstance(control, AdvancedControlBase):
-            return contains_uncond_multiplier(control.previous_controlnet)
-        # check if weights_override has an uncond_multiplier
-        if control.weights_override is not None and control.weights_override.has_uncond_multiplier:
-            return True
-        # check if any timestep_keyframes have an uncond_multiplier on their weights
-        if control.timestep_keyframes is not None:
-            for tk in control.timestep_keyframes.keyframes:
-                if tk.has_control_weights() and tk.control_weights.has_uncond_multiplier:
-                    return True
-        return contains_uncond_multiplier(control.previous_controlnet)
-
-    # check if positive or negative conds contain Adv. Cns that use multiply_negative on weights
-    def uncond_multiplier_check_cn_sample(model: ModelPatcher, *args, **kwargs):
-        positive = args[-3]
-        negative = args[-2]
-        has_uncond_multiplier = False
-        if positive is not None:
-            for cond in positive:
-                if "control" in cond[1]:
-                    has_uncond_multiplier = contains_uncond_multiplier(cond[1]["control"])
-                    if has_uncond_multiplier:
-                        break
-        if negative is not None and not has_uncond_multiplier:
-            for cond in negative:
-                if "control" in cond[1]:
-                    has_uncond_multiplier = contains_uncond_multiplier(cond[1]["control"])
-                    if has_uncond_multiplier:
-                        break
-        try:
-            # if uncond_multiplier found, continue to use wrapped version of function
-            if has_uncond_multiplier:
-                return orig_comfy_sample(model, *args, **kwargs)
-            # otherwise, use original version of function to prevent even the smallest of slowdowns (0.XX%)
-            try:
-                wrapped_cond_cat = comfy.samplers.cond_cat
-                comfy.samplers.cond_cat = orig_cond_cat
-                return orig_comfy_sample(model, *args, **kwargs)
-            finally:
-                comfy.samplers.cond_cat = wrapped_cond_cat
-        finally:
-            # make sure len function is unwrapped by the time sampling is done, just in case
-            __builtins__["len"] = global_orig_len
-    return uncond_multiplier_check_cn_sample
-# inject sample functions
-comfy.sample.sample = uncond_multiplier_check_cn_sample_factory(comfy.sample.sample)
-comfy.sample.sample_custom = uncond_multiplier_check_cn_sample_factory(comfy.sample.sample_custom, is_custom=True)
-
-
-class IntWithCondOrUncond(int):
-    def __new__(cls, *args, **kwargs):
-        return super(IntWithCondOrUncond, cls).__new__(cls, *args, **kwargs)
-
-    def __init__(self, *args, **kwargs):
-        super().__init__()
-        self.cond_or_uncond = None
 
+class WrapperConsts:
+    ACN = "ACN"
+    VERSION = "version"
+    ACN_OUTER_SAMPLE_WRAPPER_KEY = "ACN_outer_sample_wrapper"
+    ACN_CREATE_SAMPLER_SAMPLE_WRAPPER = "create_outer_sample_wrapper"
 
 
 def get_properly_arranged_t2i_weights(initial_weights: list[float]):
@@ -144,76 +58,91 @@ class ControlWeightType:
     UNIVERSAL = "universal"
     T2IADAPTER = "t2iadapter"
     CONTROLNET = "controlnet"
+    CONTROLNETPLUSPLUS = "controlnet++"
     CONTROLLORA = "controllora"
     CONTROLLLLITE = "controllllite"
     SVD_CONTROLNET = "svd_controlnet"
     SPARSECTRL = "sparsectrl"
+    CTRLORA = "ctrlora"
 
 
 class ControlWeights:
-    def __init__(self, weight_type: str, base_multiplier: float=1.0, flip_weights: bool=False, weights: list[float]=None, weight_mask: Tensor=None,
+    def __init__(self, weight_type: str, base_multiplier: float=1.0,
-                 uncond_multiplier=1.0, uncond_mask: Tensor=None):
+                 weights_input: list[float]=None, weights_middle: list[float]=None, weights_output: list[float]=None,
+                 weight_func: Callable=None, weight_mask: Tensor=None,
+                 uncond_multiplier=1.0, uncond_mask: Tensor=None,
+                 extras: dict[str]={}, disable_applied_to=False):
         self.weight_type = weight_type
         self.base_multiplier = base_multiplier
-        self.flip_weights = flip_weights
+        self.weights_input = weights_input
-        self.weights = weights
+        self.weights_middle = weights_middle
-        if self.weights is not None and self.flip_weights:
+        self.weights_output = weights_output
-            self.weights.reverse()
+        self.weight_func = weight_func
         self.weight_mask = weight_mask
         self.uncond_multiplier = float(uncond_multiplier)
         self.has_uncond_multiplier = not math.isclose(self.uncond_multiplier, 1.0)
         self.uncond_mask = uncond_mask if uncond_mask is not None else 1.0
         self.has_uncond_mask = uncond_mask is not None
+        self.extras = extras.copy()
+        self.disable_applied_to = disable_applied_to
 
-    def get(self, idx: int, default=1.0) -> Union[float, Tensor]:
+    def get(self, idx: int, control: dict[str, list[Tensor]], key: str, default=1.0) -> Union[float, Tensor]:
+        # if weight_func present, use it
+        if self.weight_func is not None:
+            return self.weight_func(idx=idx, control=control, key=key)
+        effective_mult = 1.0
         # if weights is not none, return index
-        if self.weights is not None:
+        relevant_weights = None
-            # this implies weights list is not aligning with expectations - will need to adjust code
+        if key == "middle":
-            if idx >= len(self.weights):
+            relevant_weights = self.weights_middle
-                return default
+            effective_mult *= self.extras.get(Extras.MIDDLE_MULT, 1.0)
-            return self.weights[idx]
+        elif key == "input":
-        return 1.0
+            relevant_weights = self.weights_input
+            if relevant_weights is not None:
+                relevant_weights = list(reversed(relevant_weights))
+        else:
+            relevant_weights = self.weights_output
+        if relevant_weights is None:
+            return default * effective_mult
+        elif idx >= len(relevant_weights):
+            return default * effective_mult
+        return relevant_weights[idx] * effective_mult
 
-    def copy_with_new_weights(self, new_weights: list[float]):
+    def copy_with_new_weights(self, new_weights_input: list[float]=None, new_weights_middle: list[float]=None, new_weights_output: list[float]=None,
-        return ControlWeights(weight_type=self.weight_type, base_multiplier=self.base_multiplier, flip_weights=self.flip_weights,
+                              new_weight_func: Callable=None):
-                              weights=new_weights, weight_mask=self.weight_mask, uncond_multiplier=self.uncond_multiplier)
+        return ControlWeights(weight_type=self.weight_type, base_multiplier=self.base_multiplier,
+                              weights_input=new_weights_input, weights_middle=new_weights_middle, weights_output=new_weights_output,
+                              weight_func=new_weight_func, weight_mask=self.weight_mask,
+                              uncond_multiplier=self.uncond_multiplier,
+                              extras=self.extras, disable_applied_to=self.disable_applied_to)
 
     @classmethod
-    def default(cls):
+    def default(cls, extras: dict[str]={}):
-        return cls(ControlWeightType.DEFAULT)
+        return cls(ControlWeightType.DEFAULT, extras=extras)
 
     @classmethod
-    def universal(cls, base_multiplier: float, flip_weights: bool=False, uncond_multiplier: float=1.0):
+    def universal(cls, base_multiplier: float, uncond_multiplier: float=1.0, extras: dict[str]={}):
-        return cls(ControlWeightType.UNIVERSAL, base_multiplier=base_multiplier, flip_weights=flip_weights, uncond_multiplier=uncond_multiplier)
+        return cls(ControlWeightType.UNIVERSAL, base_multiplier=base_multiplier, uncond_multiplier=uncond_multiplier, disable_applied_to=True, extras=extras)
     
     @classmethod
-    def universal_mask(cls, weight_mask: Tensor, uncond_multiplier: float=1.0):
+    def universal_mask(cls, weight_mask: Tensor, uncond_multiplier: float=1.0, extras: dict[str]={}):
-        return cls(ControlWeightType.UNIVERSAL, weight_mask=weight_mask, uncond_multiplier=uncond_multiplier)
+        return cls(ControlWeightType.UNIVERSAL, weight_mask=weight_mask, uncond_multiplier=uncond_multiplier, disable_applied_to=True, extras=extras)
 
     @classmethod
-    def t2iadapter(cls, weights: list[float]=None, flip_weights: bool=False, uncond_multiplier: float=1.0):
+    def t2iadapter(cls, weights_input: list[float]=None, uncond_multiplier: float=1.0, extras: dict[str]={}, disable_applied_to=False):
-        if weights is None:
+        return cls(ControlWeightType.T2IADAPTER, weights_input=weights_input, uncond_multiplier=uncond_multiplier, extras=extras, disable_applied_to=disable_applied_to)
-            weights = [1.0]*12
-        return cls(ControlWeightType.T2IADAPTER, weights=weights,flip_weights=flip_weights, uncond_multiplier=uncond_multiplier)
 
     @classmethod
-    def controlnet(cls, weights: list[float]=None, flip_weights: bool=False, uncond_multiplier: float=1.0):
+    def controlnet(cls, weights_output: list[float]=None, weights_middle: list[float]=None, weights_input: list[float]=None, uncond_multiplier: float=1.0, extras: dict[str]={}, disable_applied_to=False):
-        if weights is None:
+        return cls(ControlWeightType.CONTROLNET, weights_output=weights_output, weights_middle=weights_middle, weights_input=weights_input, uncond_multiplier=uncond_multiplier, extras=extras, disable_applied_to=disable_applied_to)
-            weights = [1.0]*13
-        return cls(ControlWeightType.CONTROLNET, weights=weights, flip_weights=flip_weights, uncond_multiplier=uncond_multiplier)
     
     @classmethod
-    def controllora(cls, weights: list[float]=None, flip_weights: bool=False, uncond_multiplier: float=1.0):
+    def controllora(cls, weights_output: list[float]=None, weights_middle: list[float]=None, weights_input: list[float]=None, uncond_multiplier: float=1.0, extras: dict[str]={}, disable_applied_to=False):
-        if weights is None:
+        return cls(ControlWeightType.CONTROLLORA, weights_output=weights_output, weights_middle=weights_middle, weights_input=weights_input, uncond_multiplier=uncond_multiplier, extras=extras, disable_applied_to=disable_applied_to)
-            weights = [1.0]*10
-        return cls(ControlWeightType.CONTROLLORA, weights=weights, flip_weights=flip_weights, uncond_multiplier=uncond_multiplier)
     
     @classmethod
-    def controllllite(cls, weights: list[float]=None, flip_weights: bool=False, uncond_multiplier: float=1.0):
+    def controllllite(cls, weights_output: list[float]=None, weights_middle: list[float]=None, weights_input: list[float]=None, uncond_multiplier: float=1.0, extras: dict[str]={}, disable_applied_to=False):
-        if weights is None:
+        return cls(ControlWeightType.CONTROLLLLITE, weights_output=weights_output, weights_middle=weights_middle, weights_input=weights_input, uncond_multiplier=uncond_multiplier, extras=extras, disable_applied_to=disable_applied_to)
-            # TODO: make this have a real value
-            weights = [1.0]*200
-        return cls(ControlWeightType.CONTROLLLLITE, weights=weights, flip_weights=flip_weights, uncond_multiplier=uncond_multiplier)
 
 
 class StrengthInterpolation:
@@ -318,6 +247,11 @@ class TimestepKeyframe:
     def has_mask_hint(self):
         return self.mask_hint_orig is not None
     
+    def get_effective_guarantee_steps(self, max_sigma: torch.Tensor):
+        '''If keyframe starts before current sampling range (max_sigma), treat as 0.'''
+        if self.start_t > max_sigma:
+            return 0
+        return self.guarantee_steps
     
     @staticmethod
     def default() -> 'TimestepKeyframe':
@@ -326,8 +260,9 @@ class TimestepKeyframe:
 
 # always maintain sorted state (by start_percent of TimestepKeyFrame)
 class TimestepKeyframeGroup:
-    def __init__(self) -> None:
+    def __init__(self, add_default=True) -> None:
         self.keyframes: list[TimestepKeyframe] = []
+        if add_default:
             self.keyframes.append(TimestepKeyframe.default())
 
     def add(self, keyframe: TimestepKeyframe) -> None:
@@ -354,7 +289,7 @@ class TimestepKeyframeGroup:
         return len(self.keyframes) == 0
     
     def clone(self) -> 'TimestepKeyframeGroup':
-        cloned = TimestepKeyframeGroup()
+        cloned = TimestepKeyframeGroup(add_default=False)
         # already sorted, so don't use add function to make cloning quicker
         for tk in self.keyframes:
             cloned.keyframes.append(tk)
@@ -369,7 +304,7 @@ class TimestepKeyframeGroup:
 
 class AbstractPreprocWrapper:
     error_msg = "Invalid use of [InsertHere] output. The output of [InsertHere] preprocessor is NOT a usual image, but a latent pretending to be an image - you must connect the output directly to an Apply ControlNet node (advanced or otherwise). It cannot be used for anything else that accepts IMAGE input."
-    def __init__(self, condhint: Tensor):
+    def __init__(self, condhint):
         self.condhint = condhint
     
     def movedim(self, *args, **kwargs):
@@ -403,8 +338,10 @@ class AbstractPreprocWrapper:
 class disable_weight_init_clean_groupnorm(comfy.ops.disable_weight_init):
     class GroupNorm(comfy.ops.disable_weight_init.GroupNorm):
         def forward_comfy_cast_weights(self, input):
-            weight, bias = comfy.ops.cast_bias_weight(self, input)
+            weight, bias, offload_stream = comfy.ops.cast_bias_weight(self, input, offloadable=True)
-            return torch.nn.functional.group_norm(input, self.num_groups, weight, bias, self.eps)
+            x = torch.nn.functional.group_norm(input, self.num_groups, weight, bias, self.eps)
+            comfy.ops.uncast_bias_weight(self, weight, bias, offload_stream)
+            return x
 
         def forward(self, input):
             if self.comfy_cast_weights:
@@ -418,11 +355,20 @@ class manual_cast_clean_groupnorm(comfy.ops.manual_cast):
 
 
 # adapted from comfy/sample.py
-def prepare_mask_batch(mask: Tensor, shape: Tensor, multiplier: int=1, match_dim1=False):
+def prepare_mask_batch(mask: Tensor, shape: Tensor, multiplier: int=1, match_dim1=False, match_shape=False, flux_shape=None):
     mask = mask.clone()
-    mask = torch.nn.functional.interpolate(mask.reshape((-1, 1, mask.shape[-2], mask.shape[-1])), size=(shape[2]*multiplier, shape[3]*multiplier), mode="bilinear")
+    if flux_shape is not None:
+        multiplier = multiplier * 0.5
+        mask = torch.nn.functional.interpolate(mask.reshape((-1, 1, mask.shape[-2], mask.shape[-1])), size=(round(flux_shape[-2]*multiplier), round(flux_shape[-1]*multiplier)), mode="bilinear")
+        mask = rearrange(mask, "b c h w -> b (h w) c")
+    else:
+        mask = torch.nn.functional.interpolate(mask.reshape((-1, 1, mask.shape[-2], mask.shape[-1])), size=(round(shape[-2]*multiplier), round(shape[-1]*multiplier)), mode="bilinear")
     if match_dim1:
+        if match_shape and len(shape) < 4:
+            raise Exception(f"match_dim1 cannot be True if shape is under 4 dims; was {len(shape)}.")
         mask = torch.cat([mask] * shape[1], dim=1)
+    if match_shape and len(shape) == 3 and len(mask.shape) != 3:
+        mask = mask.squeeze(1)
     return mask
 
 
@@ -524,15 +470,25 @@ def get_sorted_list_via_attr(objects: list, attr: str) -> list:
     return sorted_list
 
 
+# DFS Search for Torch.nn.Module, Written by Lvmin
+def torch_dfs(model: torch.nn.Module):
+    result = [model]
+    for child in model.children():
+        result += torch_dfs(child)
+    return result
+
+
 class WeightTypeException(TypeError):
     "Raised when weight not compatible with AdvancedControlBase object"
     pass
 
 
 class AdvancedControlBase:
-    def __init__(self, base: ControlBase, timestep_keyframes: TimestepKeyframeGroup, weights_default: ControlWeights, require_model=False):
+    ACN_VERSION = CURRENT_WRAPPER_VERSION
+    
+    def __init__(self, base: ControlBase, timestep_keyframes: TimestepKeyframeGroup, weights_default: ControlWeights, require_vae=False, allow_condhint_latents=False):
         self.base = base
-        self.compatible_weights = [ControlWeightType.UNIVERSAL]
+        self.compatible_weights = [ControlWeightType.UNIVERSAL, ControlWeightType.DEFAULT]
         self.add_compatible_weight(weights_default.weight_type)
         # mask for which parts of controlnet output to keep
         self.mask_cond_hint_original = None
@@ -545,9 +501,11 @@ class AdvancedControlBase:
         self.full_latent_length = 0
         self.context_length = 0
         # timesteps
-        self.t: Tensor = None
+        self.t: float = None
-        self.batched_number: Union[int, IntWithCondOrUncond] = None
+        self.prev_t: float = None
+        self.batched_number: int = None
         self.batch_size: int = 0
+        self.cond_or_uncond: list[int] = None
         # weights + override
         self.weights: ControlWeights = None
         self.weights_default: ControlWeights = weights_default
@@ -563,13 +521,18 @@ class AdvancedControlBase:
         self.pre_run = self.pre_run_inject
         self.cleanup = self.cleanup_inject
         self.set_previous_controlnet = self.set_previous_controlnet_inject
-        # require model to be passed into Apply Advanced ControlNet 🛂🅐🅒🅝 node
+        self.set_cond_hint = self.set_cond_hint_inject
-        self.require_model = require_model
+        # vae to store
+        self.adv_vae = None
+        self.mult_by_ratio_when_vae = True
+        # compression ratio stuff
+        self.real_compression_ratio = None
+        # require model/vae to be passed into Apply Advanced ControlNet 🛂🅐🅒🅝 node
+        self.require_vae = require_vae
+        self.allow_condhint_latents = allow_condhint_latents
+        self.postpone_condhint_latents_check = False
         # disarm - when set to False, used to force usage of Apply Advanced ControlNet 🛂🅐🅒🅝 node (which will set it to True)
-        self.disarmed = not require_model
+        self.disarmed = True
-    
-    def patch_model(self, model: ModelPatcher):
-        pass
 
     def add_compatible_weight(self, control_weight_type: str):
         self.compatible_weights.append(control_weight_type)
@@ -598,15 +561,17 @@ class AdvancedControlBase:
         self.weights = None
         self.latent_keyframes = None
 
-    def prepare_current_timestep(self, t: Tensor, batched_number: int):
+    def prepare_current_timestep(self, t: Tensor, transformer_options: dict[str, torch.Tensor]):
         self.t = float(t[0])
-        self.batched_number = batched_number
+        # check if t has changed (otherwise do nothing, as step already accounted for)
-        self.batch_size = len(t)
+        if self.t == self.prev_t:
+            return
         # get current step percent
         curr_t: float = self.t
         prev_index = self._current_timestep_index
+        max_sigma = torch.max(transformer_options.get("sample_sigmas", BIGMAX_TENSOR))
         # if met guaranteed steps (or no current keyframe), look for next keyframe in case need to switch
-        if self._current_timestep_keyframe is None or self._current_used_steps >= self._current_timestep_keyframe.guarantee_steps:
+        if self._current_timestep_keyframe is None or self._current_used_steps >= self._current_timestep_keyframe.get_effective_guarantee_steps(max_sigma):
             # if has next index, loop through and see if need to switch
             if self.timestep_keyframes.has_index(self._current_timestep_index+1):
                 for i in range(self._current_timestep_index+1, len(self.timestep_keyframes)):
@@ -632,12 +597,13 @@ class AdvancedControlBase:
                             del self.tk_mask_cond_hint_original
                             self.tk_mask_cond_hint_original = None
                         # if guarantee_steps greater than zero, stop searching for other keyframes
-                        if self._current_timestep_keyframe.guarantee_steps > 0:
+                        if self._current_timestep_keyframe.get_effective_guarantee_steps(max_sigma) > 0:
                             break
                     # if eval_tk is outside of percent range, stop looking further
                     else:
                         break
-        
+        # update prev_t
+        self.prev_t = self.t
         # update steps current keyframe is used
         self._current_used_steps += 1
         # if index changed, apply overrides
@@ -652,7 +618,7 @@ class AdvancedControlBase:
         self.prepare_weights()
     
     def prepare_weights(self):
-        if self.weights is None or self.weights.weight_type == ControlWeightType.DEFAULT:
+        if self.weights is None:
             self.weights = self.weights_default
         elif self.weights.weight_type == ControlWeightType.UNIVERSAL:
             # if universal and weight_mask present, no need to convert
@@ -667,6 +633,23 @@ class AdvancedControlBase:
         self.mask_cond_hint_original = mask_hint
         return self
 
+    def set_cond_hint_inject(self, *args, **kwargs):
+        to_return = self.base.set_cond_hint(*args, **kwargs)
+        # if vae required, look in args and kwargs for it
+        if self.require_vae:
+            # check args first, as that's the default way vae param is used in ComfyUI
+            for arg in args:
+                if isinstance(arg, VAE):
+                    self.adv_vae = arg
+                    self.vae = arg
+                    break
+            # if not in args, check kwargs now
+            if self.adv_vae is None:
+                if 'vae' in kwargs:
+                    self.adv_vae = kwargs['vae']
+                    self.vae = kwargs['vae']
+        return to_return
+
     def pre_run_inject(self, model, percent_to_timestep_function):
         self.base.pre_run(model, percent_to_timestep_function)
         self.pre_run_advanced(model, percent_to_timestep_function)
@@ -675,6 +658,9 @@ class AdvancedControlBase:
         # for each timestep keyframe, calculate the start_t
         for tk in self.timestep_keyframes.keyframes:
             tk.start_t = percent_to_timestep_function(tk.start_percent)
+        # set real_compression_ratio to compression_ratio
+        if hasattr(self, "compression_ratio"):
+            self.real_compression_ratio = self.compression_ratio
         # clear variables
         self.cleanup_advanced()
 
@@ -695,34 +681,49 @@ class AdvancedControlBase:
                 return False
         return True
 
-    def get_control_inject(self, x_noisy, t, cond, batched_number):
+    def get_control_inject(self, x_noisy, t, cond, batched_number, transformer_options: dict):
+        self.batched_number = batched_number
+        self.batch_size = len(t)
+        self.cond_or_uncond = transformer_options.get("cond_or_uncond", None)
+        # fill out ad_param-related fields, if present
+        if "ad_params" in transformer_options:
+            self.sub_idxs = transformer_options["ad_params"]["sub_idxs"]
+            self.full_latent_length = transformer_options["ad_params"]["full_length"]
+            self.context_length = transformer_options["ad_params"]["context_length"]
         # prepare timestep and everything related
-        self.prepare_current_timestep(t=t, batched_number=batched_number)
+        self.prepare_current_timestep(t=t, transformer_options=transformer_options)
         # if should not perform any actions for the controlnet, exit without doing any work
         if self.strength == 0.0 or self._current_timestep_keyframe.strength == 0.0:
-            return self.default_control_actions(x_noisy, t, cond, batched_number)
+            return self.default_control_actions(x_noisy, t, cond, batched_number, transformer_options)
         # otherwise, perform normal function
-        return self.get_control_advanced(x_noisy, t, cond, batched_number)
+        return self.get_control_advanced(x_noisy, t, cond, batched_number, transformer_options)
 
-    def get_control_advanced(self, x_noisy, t, cond, batched_number):
+    def get_control_advanced(self, x_noisy, t, cond, batched_number, transformer_options):
-        return self.default_control_actions(x_noisy, t, cond, batched_number)
+        return self.default_control_actions(x_noisy, t, cond, batched_number, transformer_options)
 
-    def default_control_actions(self, x_noisy, t, cond, batched_number):
+    def default_control_actions(self, x_noisy, t, cond, batched_number, transformer_options):
         control_prev = None
         if self.previous_controlnet is not None:
-            control_prev = self.previous_controlnet.get_control(x_noisy, t, cond, batched_number)
+            control_prev = self.previous_controlnet.get_control(x_noisy, t, cond, batched_number, transformer_options)
         return control_prev
 
-    def calc_weight(self, idx: int, x: Tensor, layers: int) -> Union[float, Tensor]:
+    def calc_weight(self, idx: int, x: Tensor, control: dict[str, list[Tensor]], key: str) -> Union[float, Tensor]:
         if self.weights.weight_mask is not None:
             # prepare weight mask
             self.prepare_weight_mask_cond_hint(x, self.batched_number)
             # adjust mask for current layer and return
-            return torch.pow(self.weight_mask_cond_hint, self.get_calc_pow(idx=idx, layers=layers))
+            return torch.pow(self.weight_mask_cond_hint, self.get_calc_pow(idx=idx, control=control, key=key))
-        return self.weights.get(idx=idx)
+        return self.weights.get(idx=idx, control=control, key=key)
     
-    def get_calc_pow(self, idx: int, layers: int) -> int:
+    def get_calc_pow(self, idx: int, control: dict[str, list[Tensor]], key: str) -> int:
-        return (layers-1)-idx
+        if key == "middle":
+            return 0
+        else:
+            c_len = len(control[key])
+            real_idx = c_len-idx
+            if key == "input":
+                real_idx = c_len - real_idx + 1
+            return real_idx
 
     def calc_latent_keyframe_mults(self, x: Tensor, batched_number: int) -> Tensor:
         # apply strengths, and get batch indeces to null out
@@ -772,12 +773,11 @@ class AdvancedControlBase:
             final_tensor = final_tensor.unsqueeze(-1)
         return final_tensor
 
-    def apply_advanced_strengths_and_masks(self, x: Tensor, batched_number: int):
+    def apply_advanced_strengths_and_masks(self, x: Tensor, batched_number: int, flux_shape: tuple=None):
         # handle weight's uncond_multiplier, if applicable
         if self.weights.has_uncond_multiplier:
-            cond_or_uncond = self.batched_number.cond_or_uncond
             actual_length = x.size(0) // batched_number
-            for idx, cond_type in enumerate(cond_or_uncond):
+            for idx, cond_type in enumerate(self.cond_or_uncond):
                 # if uncond, set to weight's uncond_multiplier
                 if cond_type == 1:
                     x[actual_length*idx:actual_length*(idx+1)] *= self.weights.uncond_multiplier
@@ -788,50 +788,41 @@ class AdvancedControlBase:
             x[:] = x[:] * self.calc_latent_keyframe_mults(x=x, batched_number=batched_number)
         # apply masks, resizing mask to required dims
         if self.mask_cond_hint is not None:
-            masks = prepare_mask_batch(self.mask_cond_hint, x.shape)
+            masks = prepare_mask_batch(self.mask_cond_hint, x.shape, match_shape=True, flux_shape=flux_shape)
             x[:] = x[:] * masks
         if self.tk_mask_cond_hint is not None:
-            masks = prepare_mask_batch(self.tk_mask_cond_hint, x.shape)
+            masks = prepare_mask_batch(self.tk_mask_cond_hint, x.shape, match_shape=True, flux_shape=flux_shape)
             x[:] = x[:] * masks
         # apply timestep keyframe strengths
         if self._current_timestep_keyframe.strength != 1.0:
             x[:] *= self._current_timestep_keyframe.strength
     
-    def control_merge_inject(self: 'AdvancedControlBase', control_input, control_output, control_prev, output_dtype):
+    def control_merge_inject(self: 'AdvancedControlBase', control: dict[str, list[Tensor]], control_prev: dict, output_dtype):
         out = {'input':[], 'middle':[], 'output': []}
 
-        if control_input is not None:
+        for key in control:
-            for i in range(len(control_input)):
+            control_output = control[key]
-                key = 'input'
+            applied_to = set()
-                x = control_input[i]
-                if x is not None:
-                    self.apply_advanced_strengths_and_masks(x, self.batched_number)
-
-                    x *= self.strength * self.calc_weight(i, x, len(control_input))
-                    if x.dtype != output_dtype:
-                        x = x.to(output_dtype)
-                out[key].insert(0, x)
-
-        if control_output is not None:
             for i in range(len(control_output)):
-                if i == (len(control_output) - 1):
-                    key = 'middle'
-                    index = 0
-                else:
-                    key = 'output'
-                    index = i
                 x = control_output[i]
                 if x is not None:
-                    self.apply_advanced_strengths_and_masks(x, self.batched_number)
-
                     if self.global_average_pooling:
                         x = torch.mean(x, dim=(2, 3), keepdim=True).repeat(1, 1, x.shape[2], x.shape[3])
 
-                    x *= self.strength * self.calc_weight(i, x, len(control_output))
+                    # if should disable applied_to optimization, clone the weight if in applied_to
-                    if x.dtype != output_dtype:
+                    if self.weights.disable_applied_to and x in applied_to:
+                        x = x.clone()
+
+                    if x not in applied_to: #memory saving strategy, allow shared tensors and only apply strength to shared tensors once
+                        applied_to.add(x)
+                        self.apply_advanced_strengths_and_masks(x, self.batched_number)
+                        x *= self.strength * self.calc_weight(i, x, control, key)
+
+                    if output_dtype is not None and x.dtype != output_dtype:
                         x = x.to(output_dtype)
 
                 out[key].append(x)
+
         if control_prev is not None:
             for x in ['input', 'middle', 'output']:
                 o = out[x]
@@ -846,7 +837,7 @@ class AdvancedControlBase:
                             if o[i].shape[0] < prev_val.shape[0]:
                                 o[i] = prev_val + o[i]
                             else:
-                                o[i] += prev_val
+                                o[i] = prev_val + o[i]  # TODO from base ComfyUI: change back to inplace add if shared tensors stop being an issue
         return out
 
     def prepare_mask_cond_hint(self, x_noisy: Tensor, t, cond, batched_number, dtype=None, direct_attn=False):
@@ -869,7 +860,7 @@ class AdvancedControlBase:
                 del out_mask
                 # TODO: perform upscale on only the sub_idxs masks at a time instead of all to conserve RAM
                 # resize mask and match batch count
-                out_mask = prepare_mask_batch(orig_mask, x_noisy.shape, multiplier=multiplier)
+                out_mask = prepare_mask_batch(orig_mask, x_noisy.shape, multiplier=multiplier, match_shape=True)
                 actual_latent_length = x_noisy.shape[0] // batched_number
                 out_mask = extend_to_batch_size(out_mask, actual_latent_length if self.sub_idxs is None else self.full_latent_length)
                 if self.sub_idxs is not None:
@@ -880,7 +871,7 @@ class AdvancedControlBase:
             # default dtype to be same as x_noisy
             if dtype is None:
                 dtype = x_noisy.dtype
-            setattr(self, attr_name, out_mask.to(dtype=dtype).to(self.device))
+            setattr(self, attr_name, out_mask.to(dtype=dtype).to(x_noisy.device))
             del out_mask
 
     def _reset_attr(self, attr_name, new_value=None):
@@ -897,10 +888,14 @@ class AdvancedControlBase:
         self.full_latent_length = 0
         self.context_length = 0
         self.t = None
+        self.prev_t = None
         self.batched_number = None
         self.batch_size = 0
         self.weights = None
         self.latent_keyframes = None
+        # set effective_compression_ratio to compression_ratio
+        if hasattr(self, "compression_ratio"):
+            self.real_compression_ratio = self.compression_ratio
         # timestep stuff
         self._current_timestep_keyframe = None
         self._current_timestep_index = -1
@@ -923,4 +918,9 @@ class AdvancedControlBase:
         copied.mask_cond_hint_original = self.mask_cond_hint_original
         copied.weights_override = self.weights_override
         copied.latent_keyframe_override = self.latent_keyframe_override
+        copied.adv_vae = self.adv_vae
+        copied.mult_by_ratio_when_vae = self.mult_by_ratio_when_vae
+        copied.require_vae = self.require_vae
+        copied.allow_condhint_latents = self.allow_condhint_latents
+        copied.postpone_condhint_latents_check = self.postpone_condhint_latents_check
         copied.disarmed = self.disarmed

examples/anima_lllite_v2/README.md

@@ -0,0 +1,144 @@
+# Anima LLLite validation workflows
+
+This folder contains simple Advanced-ControlNet workflows for the two Anima
+LLLite v2 checkpoints: the five conditioning inputs documented for the v2
+any-test-like model and the v2 inpainting model. It also includes vanilla parity
+and effect-mask validation workflows.
+
+![Simple workflows for all six v2 examples](https://ampcode.com/attachments/3ef84a8ad6947c624cd82b5ac6fb9c664b224678c774a5eba79f4f7e6e8d5c3f.jpeg)
+
+![Control images and tested results for all six v2 examples](https://ampcode.com/attachments/addd1fdb85aa331cc622ef55a9397a996f34c5243e268a1b9bcc42ca39a25120.jpeg)
+
+## Simple v2 workflows
+
+| Type | Workflow | Input image | Checkpoint |
+| --- | --- | --- | --- |
+| Any - Grayscale A | [`anima_lllite_any_grayscale_a.json`](anima_lllite_any_grayscale_a.json) | [`anima_lllite_any_grayscale_a_control.png`](https://ampcode.com/attachments/911b31c414bcf5c9aeaa254356c89f04e1133ce2181d0414cd8b7e9a7fa7847c.png) | `anima-lllite-any-test-like-v2.safetensors` |
+| Any - Grayscale B | [`anima_lllite_any_grayscale_b.json`](anima_lllite_any_grayscale_b.json) | [`anima_lllite_any_grayscale_b_control.png`](https://ampcode.com/attachments/6950b3c8a47ff62311cc8ac147e2c8aa6aad9176abe7046db3a1bd6cf97cf705.png) | `anima-lllite-any-test-like-v2.safetensors` |
+| Any - Lineart | [`anima_lllite_any_lineart.json`](anima_lllite_any_lineart.json) | [`anima_lllite_any_lineart_control.png`](https://ampcode.com/attachments/338c2f07267d564ca9fb60bf79c229b5689509202734875f0cb674efb8d691b1.png) | `anima-lllite-any-test-like-v2.safetensors` |
+| Any - HED scribble | [`anima_lllite_any_hed_scribble.json`](anima_lllite_any_hed_scribble.json) | [`anima_lllite_any_hed_scribble_control.png`](https://ampcode.com/attachments/a52217f09c5852652667152cb901b98c3472c2e4cbf86a6cae53a36ec7e84192.png) | `anima-lllite-any-test-like-v2.safetensors` |
+| Any - PiDiNet scribble | [`anima_lllite_any_pidinet_scribble.json`](anima_lllite_any_pidinet_scribble.json) | [`anima_lllite_any_pidinet_scribble_control.png`](https://ampcode.com/attachments/57bc6b0d8cba0df17f92115917bfc6ccfce83107e56af624f0be1db32c402579.png) | `anima-lllite-any-test-like-v2.safetensors` |
+| Inpainting | [`anima_lllite_inpainting.json`](anima_lllite_inpainting.json) | [`anima_lllite_v2_control.png`](https://ampcode.com/attachments/3220b2f533a014619e3c0422eca8a3343e8488eb113795af5bae5f05d38a8ada.png) | `anima-lllite-inpainting-v2.safetensors` |
+
+Download the selected input image from the table and save it under the displayed
+filename in `ComfyUI/input`. Load its workflow and queue it unchanged. Results
+are saved under `ComfyUI/output/acn_anima_examples`. Binary inputs and
+screenshots are linked externally instead of being committed to this repository.
+
+These examples use default node names and do not connect the custom 28-layer
+Anima weights node. The inpainting workflow uses Anima LLLite Extras and
+Default Weights only because the model's source mask must be carried through
+`cn_extras`. Strength, start/end scheduling, effect masks, timestep keyframes,
+latent keyframes, and stacking remain available on the standard
+Advanced-ControlNet nodes.
+
+The model author trained the v2 any-test-like checkpoint on five conditioning
+types: HED scribble, PiDiNet scribble, Grayscale A, Grayscale B, and lineart,
+all with heavy augmentation. The examples above exercise each input type using
+that one v2 checkpoint instead of the lower-quality Preview3 depth, pose,
+lineart, and scribble checkpoints.
+
+The HED and PiDiNet controls were prepared with the
+[comfyui_controlnet_aux](https://github.com/Fannovel16/comfyui_controlnet_aux)
+HED and Scribble PiDiNet preprocessors. The HED soft-edge output was thresholded
+at 80 to make the documented white-on-black scribble representation. The model
+card names two grayscale generation patterns but does not define their
+individual construction; Grayscale A is the author's sample and Grayscale B
+demonstrates the documented inversion, contrast, and blur augmentation.
+
+## Requirements
+
+Official Hugging Face repositories:
+
+- [circlestone-labs/Anima](https://huggingface.co/circlestone-labs/Anima) - base model, text encoder, and VAE
+- [kohya-ss/Anima-LLLite](https://huggingface.co/kohya-ss/Anima-LLLite) - Anima LLLite control models
+
+Use ComfyUI commit `0f42ba514631` or later and place these files in the listed
+model folders. These are direct downloads from the official model repositories:
+
+| Download | ComfyUI model folder |
+| --- | --- |
+| [`anima-base-v1.0.safetensors`](https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/diffusion_models/anima-base-v1.0.safetensors?download=true) | `models/diffusion_models` |
+| [`qwen_3_06b_base.safetensors`](https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/text_encoders/qwen_3_06b_base.safetensors?download=true) | `models/text_encoders` |
+| [`qwen_image_vae.safetensors`](https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/vae/qwen_image_vae.safetensors?download=true) | `models/vae` |
+| [`anima-lllite-inpainting-v2.safetensors`](https://huggingface.co/kohya-ss/Anima-LLLite/resolve/main/anima-lllite-inpainting-v2.safetensors?download=true) | `models/model_patches` |
+| [`anima-lllite-any-test-like-v2.safetensors`](https://huggingface.co/kohya-ss/Anima-LLLite/resolve/main/anima-lllite-any-test-like-v2.safetensors?download=true) (3-channel any-type model) | `models/model_patches` |
+
+The included workflows use **Load Anima LLLite Model**, so their LLLite files
+belong in `models/model_patches`, matching vanilla ComfyUI. For compatibility
+with existing Advanced-ControlNet LLLite workflows, the files may instead be
+placed in `models/controlnet` and loaded with **Load Advanced ControlNet
+Model**. The standard loader automatically distinguishes Anima checkpoints
+from older SDXL LLLite checkpoints.
+
+## Run the inpainting comparison
+
+This validation workflow runs the inpainting model through vanilla ComfyUI and
+Advanced-ControlNet with identical inputs and sampling settings. It saves both
+decoded results, both latent tensors, and an absolute pixel-difference image.
+
+![Workflow showing the vanilla and Advanced-ControlNet branches](https://ampcode.com/attachments/f056cae89132c45bc133f456a2832a81255fee9c0782968a24adce2095b2fe1b.jpeg)
+
+![Bit-exact result comparison](https://ampcode.com/attachments/fcf073d1253ea721a2d46d34efde0e45ff6ccfb751fbd295a438ab459cc26037.jpeg)
+
+1. Download [`anima_lllite_v2_control.png`](https://ampcode.com/attachments/3220b2f533a014619e3c0422eca8a3343e8488eb113795af5bae5f05d38a8ada.png)
+   to `ComfyUI/input`.
+2. Load `anima_lllite_v2_inpaint_comparison.json` in ComfyUI.
+3. Queue the workflow without changing its settings.
+4. Inspect `ComfyUI/output/acn_anima_pr`.
+
+The control PNG contains a transparent edit region. ComfyUI's Load Image node
+provides that alpha channel as the source inpainting mask.
+
+The vanilla branch passes the mask directly to Apply Anima LLLite. The
+Advanced-ControlNet branch passes it through Anima LLLite Extras, the
+`cn_extras` input on Default Weights, and `weights_override` on Apply Advanced
+ControlNet.
+
+If an inpainting checkpoint reaches sampling without that source mask,
+Advanced-ControlNet raises an error that describes these connections instead
+of silently substituting an empty mask.
+
+With the included seed and settings, the expected results are:
+
+- Identical latent tensors with maximum and mean absolute differences of `0.0`.
+- Identical decoded PNG pixels.
+- A completely black `absolute_difference` image.
+
+## Run the any-type effect-mask comparison
+
+This workflow verifies the official 3-channel any-type checkpoint against
+vanilla ComfyUI and applies an Advanced-ControlNet effect mask to only the left
+half of the image. Its nodes retain their default names; the colored regions
+identify the comparison branches.
+
+![Any-type effect-mask workflow](https://ampcode.com/attachments/3a547b4914a84f0d578b0223149d3ed14e9b22542093e4bc51b6501aeb18f29f.jpeg)
+
+![Any-type parity and effect-mask results](https://ampcode.com/attachments/340c286b28d74d943f03a95d71208f2408c94934f0809fbb060cc774ae8c1b67.jpeg)
+
+1. Download [`anima_lllite_v2_any_control.png`](https://ampcode.com/attachments/db18d41c476324bdf5a5b6117476ed117cc590f8f48e193f6316d08b959bc49c.png)
+   and [`anima_lllite_v2_left_half_mask.png`](https://ampcode.com/attachments/e4fe3a88415357f29315c9afb124ee975a48dee8abccad1fe1903dd5d21b91b6.png)
+   to `ComfyUI/input`.
+2. Load `anima_lllite_v2_any_effect_mask.json` in ComfyUI.
+3. Queue the workflow without changing its settings.
+4. Inspect `ComfyUI/output/acn_anima_any_mask`.
+
+The expected results are:
+
+- Advanced-ControlNet full control and vanilla full control have identical
+  latent tensors and decoded pixels, with maximum absolute difference `0.0`.
+- A completely black effect mask produces the no-control latent exactly.
+- A completely white effect mask produces the unmasked full-control latent
+  exactly.
+- At the model token resolution, the included half mask is exactly `1.0` on
+  the left and `0.0` on the right. Direct LLLite injection is therefore zero
+  for every right-side token.
+- In the final image, the masked right side is closer to the no-control
+  baseline than full control: PSNR improves from `16.39` to `17.69`, and SSIM
+  improves from `0.554` to `0.600`.
+
+The final right half is not pixel-identical to the no-control image. This model
+patches self-attention Q, so controlled left-side tokens can influence
+right-side tokens through global self-attention and later diffusion steps.
+The effect mask guarantees local control injection, not hard image-space
+isolation after attention.

examples/anima_lllite_v2/anima_lllite_any_grayscale_a.json

@@ -0,0 +1,941 @@
+{
+  "id": "00000000-0000-0000-0000-000000000000",
+  "revision": 0,
+  "last_node_id": 12,
+  "last_link_id": 26,
+  "nodes": [
+    {
+      "id": 1,
+      "type": "UNETLoader",
+      "pos": [
+        50,
+        100
+      ],
+      "size": [
+        270,
+        82
+      ],
+      "flags": {},
+      "order": 0,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "unet_name",
+          "name": "unet_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "unet_name"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "weight_dtype",
+          "name": "weight_dtype",
+          "type": "COMBO",
+          "widget": {
+            "name": "weight_dtype"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "MODEL",
+          "name": "MODEL",
+          "type": "MODEL",
+          "links": [
+            20
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "UNETLoader",
+        "models": [
+          {
+            "name": "anima-base-v1.0.safetensors",
+            "url": "https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/diffusion_models/anima-base-v1.0.safetensors",
+            "directory": "diffusion_models"
+          }
+        ]
+      },
+      "widgets_values": [
+        "anima-base-v1.0.safetensors",
+        "default"
+      ]
+    },
+    {
+      "id": 2,
+      "type": "CLIPLoader",
+      "pos": [
+        50,
+        250
+      ],
+      "size": [
+        270,
+        106
+      ],
+      "flags": {},
+      "order": 1,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "clip_name",
+          "name": "clip_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "clip_name"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "type",
+          "name": "type",
+          "type": "COMBO",
+          "widget": {
+            "name": "type"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "device",
+          "name": "device",
+          "shape": 7,
+          "type": "COMBO",
+          "widget": {
+            "name": "device"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CLIP",
+          "name": "CLIP",
+          "type": "CLIP",
+          "links": [
+            14,
+            15
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPLoader",
+        "models": [
+          {
+            "name": "qwen_3_06b_base.safetensors",
+            "url": "https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/text_encoders/qwen_3_06b_base.safetensors",
+            "directory": "text_encoders"
+          }
+        ]
+      },
+      "widgets_values": [
+        "qwen_3_06b_base.safetensors",
+        "qwen_image",
+        "default"
+      ]
+    },
+    {
+      "id": 3,
+      "type": "CLIPTextEncode",
+      "pos": [
+        360,
+        190
+      ],
+      "size": [
+        400,
+        200
+      ],
+      "flags": {},
+      "order": 6,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "clip",
+          "name": "clip",
+          "type": "CLIP",
+          "link": 14
+        },
+        {
+          "localized_name": "text",
+          "name": "text",
+          "type": "STRING",
+          "widget": {
+            "name": "text"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CONDITIONING",
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            16
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        "anime illustration of a female knight in ornate silver plate armor holding a sword, side profile in a dense forest, detailed"
+      ]
+    },
+    {
+      "id": 4,
+      "type": "CLIPTextEncode",
+      "pos": [
+        360,
+        440
+      ],
+      "size": [
+        400,
+        200
+      ],
+      "flags": {},
+      "order": 7,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "clip",
+          "name": "clip",
+          "type": "CLIP",
+          "link": 15
+        },
+        {
+          "localized_name": "text",
+          "name": "text",
+          "type": "STRING",
+          "widget": {
+            "name": "text"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CONDITIONING",
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            17
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        ""
+      ]
+    },
+    {
+      "id": 5,
+      "type": "EmptySD3LatentImage",
+      "pos": [
+        50,
+        470
+      ],
+      "size": [
+        270,
+        106
+      ],
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "width",
+          "name": "width",
+          "type": "INT",
+          "widget": {
+            "name": "width"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "height",
+          "name": "height",
+          "type": "INT",
+          "widget": {
+            "name": "height"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "batch_size",
+          "name": "batch_size",
+          "type": "INT",
+          "widget": {
+            "name": "batch_size"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "LATENT",
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            23
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "EmptySD3LatentImage"
+      },
+      "widgets_values": [
+        512,
+        512,
+        1
+      ]
+    },
+    {
+      "id": 6,
+      "type": "LoadImage",
+      "pos": [
+        50,
+        650
+      ],
+      "size": [
+        282.798828125,
+        314
+      ],
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "image",
+          "name": "image",
+          "type": "COMBO",
+          "widget": {
+            "name": "image"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "choose file to upload",
+          "name": "upload",
+          "type": "IMAGEUPLOAD",
+          "widget": {
+            "name": "upload"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "IMAGE",
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            19
+          ]
+        },
+        {
+          "localized_name": "MASK",
+          "name": "MASK",
+          "type": "MASK",
+          "links": null
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "LoadImage"
+      },
+      "widgets_values": [
+        "anima_lllite_any_grayscale_a_control.png",
+        "image"
+      ]
+    },
+    {
+      "id": 7,
+      "type": "VAELoader",
+      "pos": [
+        360,
+        700
+      ],
+      "size": [
+        270,
+        58
+      ],
+      "flags": {},
+      "order": 4,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "vae_name",
+          "name": "vae_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "vae_name"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "VAE",
+          "name": "VAE",
+          "type": "VAE",
+          "links": [
+            25
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAELoader",
+        "models": [
+          {
+            "name": "qwen_image_vae.safetensors",
+            "url": "https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/vae/qwen_image_vae.safetensors",
+            "directory": "vae"
+          }
+        ]
+      },
+      "widgets_values": [
+        "qwen_image_vae.safetensors"
+      ]
+    },
+    {
+      "id": 8,
+      "type": "ACN_AnimaLLLiteLoaderAdvanced",
+      "pos": [
+        860,
+        120
+      ],
+      "size": [
+        315,
+        58
+      ],
+      "flags": {},
+      "order": 5,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "timestep_kf",
+          "name": "timestep_kf",
+          "shape": 7,
+          "type": "TIMESTEP_KEYFRAME",
+          "link": null
+        },
+        {
+          "localized_name": "model_patch",
+          "name": "model_patch",
+          "type": "COMBO",
+          "widget": {
+            "name": "model_patch"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CONTROL_NET",
+          "name": "CONTROL_NET",
+          "type": "CONTROL_NET",
+          "links": [
+            18
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ACN_AnimaLLLiteLoaderAdvanced",
+        "models": [
+          {
+            "name": "anima-lllite-any-test-like-v2.safetensors",
+            "url": "https://huggingface.co/kohya-ss/Anima-LLLite/resolve/main/anima-lllite-any-test-like-v2.safetensors",
+            "directory": "model_patches"
+          }
+        ]
+      },
+      "widgets_values": [
+        "anima-lllite-any-test-like-v2.safetensors"
+      ]
+    },
+    {
+      "id": 9,
+      "type": "ACN_AdvancedControlNetApply_v2",
+      "pos": [
+        1210,
+        130
+      ],
+      "size": [
+        287.86183776855466,
+        266
+      ],
+      "flags": {},
+      "order": 8,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "positive",
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 16
+        },
+        {
+          "localized_name": "negative",
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 17
+        },
+        {
+          "localized_name": "control_net",
+          "name": "control_net",
+          "type": "CONTROL_NET",
+          "link": 18
+        },
+        {
+          "localized_name": "image",
+          "name": "image",
+          "type": "IMAGE",
+          "link": 19
+        },
+        {
+          "localized_name": "mask_optional",
+          "name": "mask_optional",
+          "shape": 7,
+          "type": "MASK",
+          "link": null
+        },
+        {
+          "localized_name": "timestep_kf",
+          "name": "timestep_kf",
+          "shape": 7,
+          "type": "TIMESTEP_KEYFRAME",
+          "link": null
+        },
+        {
+          "localized_name": "latent_kf_override",
+          "name": "latent_kf_override",
+          "shape": 7,
+          "type": "LATENT_KEYFRAME",
+          "link": null
+        },
+        {
+          "localized_name": "weights_override",
+          "name": "weights_override",
+          "shape": 7,
+          "type": "CONTROL_NET_WEIGHTS",
+          "link": null
+        },
+        {
+          "localized_name": "vae_optional",
+          "name": "vae_optional",
+          "shape": 7,
+          "type": "VAE",
+          "link": null
+        },
+        {
+          "localized_name": "strength",
+          "name": "strength",
+          "type": "FLOAT",
+          "widget": {
+            "name": "strength"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "start_percent",
+          "name": "start_percent",
+          "type": "FLOAT",
+          "widget": {
+            "name": "start_percent"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "end_percent",
+          "name": "end_percent",
+          "type": "FLOAT",
+          "widget": {
+            "name": "end_percent"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "positive",
+          "name": "positive",
+          "type": "CONDITIONING",
+          "links": [
+            21
+          ]
+        },
+        {
+          "localized_name": "negative",
+          "name": "negative",
+          "type": "CONDITIONING",
+          "links": [
+            22
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ACN_AdvancedControlNetApply_v2"
+      },
+      "widgets_values": [
+        1,
+        0,
+        1
+      ]
+    },
+    {
+      "id": 10,
+      "type": "KSampler",
+      "pos": [
+        1580,
+        110
+      ],
+      "size": [
+        270,
+        262
+      ],
+      "flags": {},
+      "order": 9,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "model",
+          "name": "model",
+          "type": "MODEL",
+          "link": 20
+        },
+        {
+          "localized_name": "positive",
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 21
+        },
+        {
+          "localized_name": "negative",
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 22
+        },
+        {
+          "localized_name": "latent_image",
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 23
+        },
+        {
+          "localized_name": "seed",
+          "name": "seed",
+          "type": "INT",
+          "widget": {
+            "name": "seed"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "steps",
+          "name": "steps",
+          "type": "INT",
+          "widget": {
+            "name": "steps"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "cfg",
+          "name": "cfg",
+          "type": "FLOAT",
+          "widget": {
+            "name": "cfg"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "sampler_name",
+          "name": "sampler_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "sampler_name"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "scheduler",
+          "name": "scheduler",
+          "type": "COMBO",
+          "widget": {
+            "name": "scheduler"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "denoise",
+          "name": "denoise",
+          "type": "FLOAT",
+          "widget": {
+            "name": "denoise"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "LATENT",
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            24
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "KSampler"
+      },
+      "widgets_values": [
+        82000,
+        "fixed",
+        12,
+        1,
+        "euler",
+        "simple",
+        1
+      ]
+    },
+    {
+      "id": 11,
+      "type": "VAEDecode",
+      "pos": [
+        1940,
+        140
+      ],
+      "size": [
+        140,
+        46
+      ],
+      "flags": {},
+      "order": 10,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "samples",
+          "name": "samples",
+          "type": "LATENT",
+          "link": 24
+        },
+        {
+          "localized_name": "vae",
+          "name": "vae",
+          "type": "VAE",
+          "link": 25
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "IMAGE",
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            26
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAEDecode"
+      }
+    },
+    {
+      "id": 12,
+      "type": "SaveImage",
+      "pos": [
+        2190,
+        110
+      ],
+      "size": [
+        270,
+        58
+      ],
+      "flags": {},
+      "order": 11,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "images",
+          "name": "images",
+          "type": "IMAGE",
+          "link": 26
+        },
+        {
+          "localized_name": "filename_prefix",
+          "name": "filename_prefix",
+          "type": "STRING",
+          "widget": {
+            "name": "filename_prefix"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "images",
+          "name": "images",
+          "type": "IMAGE",
+          "links": null
+        }
+      ],
+      "properties": {},
+      "widgets_values": [
+        "acn_anima_examples/any_grayscale_a"
+      ]
+    }
+  ],
+  "links": [
+    [
+      14,
+      2,
+      0,
+      3,
+      0,
+      "CLIP"
+    ],
+    [
+      15,
+      2,
+      0,
+      4,
+      0,
+      "CLIP"
+    ],
+    [
+      16,
+      3,
+      0,
+      9,
+      0,
+      "CONDITIONING"
+    ],
+    [
+      17,
+      4,
+      0,
+      9,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      18,
+      8,
+      0,
+      9,
+      2,
+      "CONTROL_NET"
+    ],
+    [
+      19,
+      6,
+      0,
+      9,
+      3,
+      "IMAGE"
+    ],
+    [
+      20,
+      1,
+      0,
+      10,
+      0,
+      "MODEL"
+    ],
+    [
+      21,
+      9,
+      0,
+      10,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      22,
+      9,
+      1,
+      10,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      23,
+      5,
+      0,
+      10,
+      3,
+      "LATENT"
+    ],
+    [
+      24,
+      10,
+      0,
+      11,
+      0,
+      "LATENT"
+    ],
+    [
+      25,
+      7,
+      0,
+      11,
+      1,
+      "VAE"
+    ],
+    [
+      26,
+      11,
+      0,
+      12,
+      0,
+      "IMAGE"
+    ]
+  ],
+  "groups": [
+    {
+      "id": 20,
+      "title": "SHARED INPUTS",
+      "bounding": [
+        10,
+        35,
+        740,
+        900
+      ],
+      "color": "#3f789e",
+      "font_size": 32,
+      "flags": {}
+    },
+    {
+      "id": 21,
+      "title": "ANIMA LLLITE V2 ANY - GRAYSCALE A",
+      "bounding": [
+        810,
+        35,
+        1660,
+        900
+      ],
+      "color": "#6f4d86",
+      "font_size": 32,
+      "flags": {}
+    }
+  ],
+  "config": {},
+  "extra": {
+    "ds": {
+      "scale": 1,
+      "offset": [
+        0,
+        0
+      ]
+    },
+    "info": {
+      "name": "Anima LLLite v2 any - grayscale A",
+      "author": "Kosinkadink",
+      "description": "Advanced-ControlNet example for Anima LLLite v2 any with grayscale A conditioning."
+    }
+  },
+  "version": 0.4
+}

examples/anima_lllite_v2/anima_lllite_any_grayscale_b.json

@@ -0,0 +1,941 @@
+{
+  "id": "00000000-0000-0000-0000-000000000000",
+  "revision": 0,
+  "last_node_id": 12,
+  "last_link_id": 26,
+  "nodes": [
+    {
+      "id": 1,
+      "type": "UNETLoader",
+      "pos": [
+        50,
+        100
+      ],
+      "size": [
+        270,
+        82
+      ],
+      "flags": {},
+      "order": 0,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "unet_name",
+          "name": "unet_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "unet_name"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "weight_dtype",
+          "name": "weight_dtype",
+          "type": "COMBO",
+          "widget": {
+            "name": "weight_dtype"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "MODEL",
+          "name": "MODEL",
+          "type": "MODEL",
+          "links": [
+            20
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "UNETLoader",
+        "models": [
+          {
+            "name": "anima-base-v1.0.safetensors",
+            "url": "https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/diffusion_models/anima-base-v1.0.safetensors",
+            "directory": "diffusion_models"
+          }
+        ]
+      },
+      "widgets_values": [
+        "anima-base-v1.0.safetensors",
+        "default"
+      ]
+    },
+    {
+      "id": 2,
+      "type": "CLIPLoader",
+      "pos": [
+        50,
+        250
+      ],
+      "size": [
+        270,
+        106
+      ],
+      "flags": {},
+      "order": 1,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "clip_name",
+          "name": "clip_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "clip_name"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "type",
+          "name": "type",
+          "type": "COMBO",
+          "widget": {
+            "name": "type"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "device",
+          "name": "device",
+          "shape": 7,
+          "type": "COMBO",
+          "widget": {
+            "name": "device"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CLIP",
+          "name": "CLIP",
+          "type": "CLIP",
+          "links": [
+            14,
+            15
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPLoader",
+        "models": [
+          {
+            "name": "qwen_3_06b_base.safetensors",
+            "url": "https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/text_encoders/qwen_3_06b_base.safetensors",
+            "directory": "text_encoders"
+          }
+        ]
+      },
+      "widgets_values": [
+        "qwen_3_06b_base.safetensors",
+        "qwen_image",
+        "default"
+      ]
+    },
+    {
+      "id": 3,
+      "type": "CLIPTextEncode",
+      "pos": [
+        360,
+        190
+      ],
+      "size": [
+        400,
+        200
+      ],
+      "flags": {},
+      "order": 6,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "clip",
+          "name": "clip",
+          "type": "CLIP",
+          "link": 14
+        },
+        {
+          "localized_name": "text",
+          "name": "text",
+          "type": "STRING",
+          "widget": {
+            "name": "text"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CONDITIONING",
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            16
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        "anime illustration of a female knight in ornate silver plate armor holding a sword, side profile in a dense forest, detailed"
+      ]
+    },
+    {
+      "id": 4,
+      "type": "CLIPTextEncode",
+      "pos": [
+        360,
+        440
+      ],
+      "size": [
+        400,
+        200
+      ],
+      "flags": {},
+      "order": 7,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "clip",
+          "name": "clip",
+          "type": "CLIP",
+          "link": 15
+        },
+        {
+          "localized_name": "text",
+          "name": "text",
+          "type": "STRING",
+          "widget": {
+            "name": "text"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CONDITIONING",
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            17
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        ""
+      ]
+    },
+    {
+      "id": 5,
+      "type": "EmptySD3LatentImage",
+      "pos": [
+        50,
+        470
+      ],
+      "size": [
+        270,
+        106
+      ],
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "width",
+          "name": "width",
+          "type": "INT",
+          "widget": {
+            "name": "width"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "height",
+          "name": "height",
+          "type": "INT",
+          "widget": {
+            "name": "height"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "batch_size",
+          "name": "batch_size",
+          "type": "INT",
+          "widget": {
+            "name": "batch_size"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "LATENT",
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            23
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "EmptySD3LatentImage"
+      },
+      "widgets_values": [
+        512,
+        512,
+        1
+      ]
+    },
+    {
+      "id": 6,
+      "type": "LoadImage",
+      "pos": [
+        50,
+        650
+      ],
+      "size": [
+        282.798828125,
+        314
+      ],
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "image",
+          "name": "image",
+          "type": "COMBO",
+          "widget": {
+            "name": "image"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "choose file to upload",
+          "name": "upload",
+          "type": "IMAGEUPLOAD",
+          "widget": {
+            "name": "upload"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "IMAGE",
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            19
+          ]
+        },
+        {
+          "localized_name": "MASK",
+          "name": "MASK",
+          "type": "MASK",
+          "links": null
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "LoadImage"
+      },
+      "widgets_values": [
+        "anima_lllite_any_grayscale_b_control.png",
+        "image"
+      ]
+    },
+    {
+      "id": 7,
+      "type": "VAELoader",
+      "pos": [
+        360,
+        700
+      ],
+      "size": [
+        270,
+        58
+      ],
+      "flags": {},
+      "order": 4,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "vae_name",
+          "name": "vae_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "vae_name"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "VAE",
+          "name": "VAE",
+          "type": "VAE",
+          "links": [
+            25
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAELoader",
+        "models": [
+          {
+            "name": "qwen_image_vae.safetensors",
+            "url": "https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/vae/qwen_image_vae.safetensors",
+            "directory": "vae"
+          }
+        ]
+      },
+      "widgets_values": [
+        "qwen_image_vae.safetensors"
+      ]
+    },
+    {
+      "id": 8,
+      "type": "ACN_AnimaLLLiteLoaderAdvanced",
+      "pos": [
+        860,
+        120
+      ],
+      "size": [
+        315,
+        58
+      ],
+      "flags": {},
+      "order": 5,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "timestep_kf",
+          "name": "timestep_kf",
+          "shape": 7,
+          "type": "TIMESTEP_KEYFRAME",
+          "link": null
+        },
+        {
+          "localized_name": "model_patch",
+          "name": "model_patch",
+          "type": "COMBO",
+          "widget": {
+            "name": "model_patch"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CONTROL_NET",
+          "name": "CONTROL_NET",
+          "type": "CONTROL_NET",
+          "links": [
+            18
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ACN_AnimaLLLiteLoaderAdvanced",
+        "models": [
+          {
+            "name": "anima-lllite-any-test-like-v2.safetensors",
+            "url": "https://huggingface.co/kohya-ss/Anima-LLLite/resolve/main/anima-lllite-any-test-like-v2.safetensors",
+            "directory": "model_patches"
+          }
+        ]
+      },
+      "widgets_values": [
+        "anima-lllite-any-test-like-v2.safetensors"
+      ]
+    },
+    {
+      "id": 9,
+      "type": "ACN_AdvancedControlNetApply_v2",
+      "pos": [
+        1210,
+        130
+      ],
+      "size": [
+        287.86183776855466,
+        266
+      ],
+      "flags": {},
+      "order": 8,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "positive",
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 16
+        },
+        {
+          "localized_name": "negative",
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 17
+        },
+        {
+          "localized_name": "control_net",
+          "name": "control_net",
+          "type": "CONTROL_NET",
+          "link": 18
+        },
+        {
+          "localized_name": "image",
+          "name": "image",
+          "type": "IMAGE",
+          "link": 19
+        },
+        {
+          "localized_name": "mask_optional",
+          "name": "mask_optional",
+          "shape": 7,
+          "type": "MASK",
+          "link": null
+        },
+        {
+          "localized_name": "timestep_kf",
+          "name": "timestep_kf",
+          "shape": 7,
+          "type": "TIMESTEP_KEYFRAME",
+          "link": null
+        },
+        {
+          "localized_name": "latent_kf_override",
+          "name": "latent_kf_override",
+          "shape": 7,
+          "type": "LATENT_KEYFRAME",
+          "link": null
+        },
+        {
+          "localized_name": "weights_override",
+          "name": "weights_override",
+          "shape": 7,
+          "type": "CONTROL_NET_WEIGHTS",
+          "link": null
+        },
+        {
+          "localized_name": "vae_optional",
+          "name": "vae_optional",
+          "shape": 7,
+          "type": "VAE",
+          "link": null
+        },
+        {
+          "localized_name": "strength",
+          "name": "strength",
+          "type": "FLOAT",
+          "widget": {
+            "name": "strength"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "start_percent",
+          "name": "start_percent",
+          "type": "FLOAT",
+          "widget": {
+            "name": "start_percent"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "end_percent",
+          "name": "end_percent",
+          "type": "FLOAT",
+          "widget": {
+            "name": "end_percent"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "positive",
+          "name": "positive",
+          "type": "CONDITIONING",
+          "links": [
+            21
+          ]
+        },
+        {
+          "localized_name": "negative",
+          "name": "negative",
+          "type": "CONDITIONING",
+          "links": [
+            22
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ACN_AdvancedControlNetApply_v2"
+      },
+      "widgets_values": [
+        1,
+        0,
+        1
+      ]
+    },
+    {
+      "id": 10,
+      "type": "KSampler",
+      "pos": [
+        1580,
+        110
+      ],
+      "size": [
+        270,
+        262
+      ],
+      "flags": {},
+      "order": 9,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "model",
+          "name": "model",
+          "type": "MODEL",
+          "link": 20
+        },
+        {
+          "localized_name": "positive",
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 21
+        },
+        {
+          "localized_name": "negative",
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 22
+        },
+        {
+          "localized_name": "latent_image",
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 23
+        },
+        {
+          "localized_name": "seed",
+          "name": "seed",
+          "type": "INT",
+          "widget": {
+            "name": "seed"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "steps",
+          "name": "steps",
+          "type": "INT",
+          "widget": {
+            "name": "steps"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "cfg",
+          "name": "cfg",
+          "type": "FLOAT",
+          "widget": {
+            "name": "cfg"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "sampler_name",
+          "name": "sampler_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "sampler_name"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "scheduler",
+          "name": "scheduler",
+          "type": "COMBO",
+          "widget": {
+            "name": "scheduler"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "denoise",
+          "name": "denoise",
+          "type": "FLOAT",
+          "widget": {
+            "name": "denoise"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "LATENT",
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            24
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "KSampler"
+      },
+      "widgets_values": [
+        82000,
+        "fixed",
+        12,
+        1,
+        "euler",
+        "simple",
+        1
+      ]
+    },
+    {
+      "id": 11,
+      "type": "VAEDecode",
+      "pos": [
+        1940,
+        140
+      ],
+      "size": [
+        140,
+        46
+      ],
+      "flags": {},
+      "order": 10,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "samples",
+          "name": "samples",
+          "type": "LATENT",
+          "link": 24
+        },
+        {
+          "localized_name": "vae",
+          "name": "vae",
+          "type": "VAE",
+          "link": 25
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "IMAGE",
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            26
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAEDecode"
+      }
+    },
+    {
+      "id": 12,
+      "type": "SaveImage",
+      "pos": [
+        2190,
+        110
+      ],
+      "size": [
+        270,
+        58
+      ],
+      "flags": {},
+      "order": 11,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "images",
+          "name": "images",
+          "type": "IMAGE",
+          "link": 26
+        },
+        {
+          "localized_name": "filename_prefix",
+          "name": "filename_prefix",
+          "type": "STRING",
+          "widget": {
+            "name": "filename_prefix"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "images",
+          "name": "images",
+          "type": "IMAGE",
+          "links": null
+        }
+      ],
+      "properties": {},
+      "widgets_values": [
+        "acn_anima_examples/any_grayscale_b"
+      ]
+    }
+  ],
+  "links": [
+    [
+      14,
+      2,
+      0,
+      3,
+      0,
+      "CLIP"
+    ],
+    [
+      15,
+      2,
+      0,
+      4,
+      0,
+      "CLIP"
+    ],
+    [
+      16,
+      3,
+      0,
+      9,
+      0,
+      "CONDITIONING"
+    ],
+    [
+      17,
+      4,
+      0,
+      9,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      18,
+      8,
+      0,
+      9,
+      2,
+      "CONTROL_NET"
+    ],
+    [
+      19,
+      6,
+      0,
+      9,
+      3,
+      "IMAGE"
+    ],
+    [
+      20,
+      1,
+      0,
+      10,
+      0,
+      "MODEL"
+    ],
+    [
+      21,
+      9,
+      0,
+      10,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      22,
+      9,
+      1,
+      10,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      23,
+      5,
+      0,
+      10,
+      3,
+      "LATENT"
+    ],
+    [
+      24,
+      10,
+      0,
+      11,
+      0,
+      "LATENT"
+    ],
+    [
+      25,
+      7,
+      0,
+      11,
+      1,
+      "VAE"
+    ],
+    [
+      26,
+      11,
+      0,
+      12,
+      0,
+      "IMAGE"
+    ]
+  ],
+  "groups": [
+    {
+      "id": 20,
+      "title": "SHARED INPUTS",
+      "bounding": [
+        10,
+        35,
+        740,
+        900
+      ],
+      "color": "#3f789e",
+      "font_size": 32,
+      "flags": {}
+    },
+    {
+      "id": 21,
+      "title": "ANIMA LLLITE V2 ANY - GRAYSCALE B",
+      "bounding": [
+        810,
+        35,
+        1660,
+        900
+      ],
+      "color": "#6f4d86",
+      "font_size": 32,
+      "flags": {}
+    }
+  ],
+  "config": {},
+  "extra": {
+    "ds": {
+      "scale": 1,
+      "offset": [
+        0,
+        0
+      ]
+    },
+    "info": {
+      "name": "Anima LLLite v2 any - grayscale B",
+      "author": "Kosinkadink",
+      "description": "Advanced-ControlNet example for Anima LLLite v2 any with grayscale B conditioning."
+    }
+  },
+  "version": 0.4
+}

examples/anima_lllite_v2/anima_lllite_any_hed_scribble.json

@@ -0,0 +1,941 @@
+{
+  "id": "00000000-0000-0000-0000-000000000000",
+  "revision": 0,
+  "last_node_id": 12,
+  "last_link_id": 26,
+  "nodes": [
+    {
+      "id": 1,
+      "type": "UNETLoader",
+      "pos": [
+        50,
+        100
+      ],
+      "size": [
+        270,
+        82
+      ],
+      "flags": {},
+      "order": 0,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "unet_name",
+          "name": "unet_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "unet_name"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "weight_dtype",
+          "name": "weight_dtype",
+          "type": "COMBO",
+          "widget": {
+            "name": "weight_dtype"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "MODEL",
+          "name": "MODEL",
+          "type": "MODEL",
+          "links": [
+            20
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "UNETLoader",
+        "models": [
+          {
+            "name": "anima-base-v1.0.safetensors",
+            "url": "https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/diffusion_models/anima-base-v1.0.safetensors",
+            "directory": "diffusion_models"
+          }
+        ]
+      },
+      "widgets_values": [
+        "anima-base-v1.0.safetensors",
+        "default"
+      ]
+    },
+    {
+      "id": 2,
+      "type": "CLIPLoader",
+      "pos": [
+        50,
+        250
+      ],
+      "size": [
+        270,
+        106
+      ],
+      "flags": {},
+      "order": 1,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "clip_name",
+          "name": "clip_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "clip_name"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "type",
+          "name": "type",
+          "type": "COMBO",
+          "widget": {
+            "name": "type"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "device",
+          "name": "device",
+          "shape": 7,
+          "type": "COMBO",
+          "widget": {
+            "name": "device"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CLIP",
+          "name": "CLIP",
+          "type": "CLIP",
+          "links": [
+            14,
+            15
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPLoader",
+        "models": [
+          {
+            "name": "qwen_3_06b_base.safetensors",
+            "url": "https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/text_encoders/qwen_3_06b_base.safetensors",
+            "directory": "text_encoders"
+          }
+        ]
+      },
+      "widgets_values": [
+        "qwen_3_06b_base.safetensors",
+        "qwen_image",
+        "default"
+      ]
+    },
+    {
+      "id": 3,
+      "type": "CLIPTextEncode",
+      "pos": [
+        360,
+        190
+      ],
+      "size": [
+        400,
+        200
+      ],
+      "flags": {},
+      "order": 6,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "clip",
+          "name": "clip",
+          "type": "CLIP",
+          "link": 14
+        },
+        {
+          "localized_name": "text",
+          "name": "text",
+          "type": "STRING",
+          "widget": {
+            "name": "text"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CONDITIONING",
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            16
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        "anime illustration of a female knight in ornate silver plate armor holding a sword, side profile in a dense forest, detailed"
+      ]
+    },
+    {
+      "id": 4,
+      "type": "CLIPTextEncode",
+      "pos": [
+        360,
+        440
+      ],
+      "size": [
+        400,
+        200
+      ],
+      "flags": {},
+      "order": 7,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "clip",
+          "name": "clip",
+          "type": "CLIP",
+          "link": 15
+        },
+        {
+          "localized_name": "text",
+          "name": "text",
+          "type": "STRING",
+          "widget": {
+            "name": "text"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CONDITIONING",
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            17
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        ""
+      ]
+    },
+    {
+      "id": 5,
+      "type": "EmptySD3LatentImage",
+      "pos": [
+        50,
+        470
+      ],
+      "size": [
+        270,
+        106
+      ],
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "width",
+          "name": "width",
+          "type": "INT",
+          "widget": {
+            "name": "width"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "height",
+          "name": "height",
+          "type": "INT",
+          "widget": {
+            "name": "height"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "batch_size",
+          "name": "batch_size",
+          "type": "INT",
+          "widget": {
+            "name": "batch_size"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "LATENT",
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            23
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "EmptySD3LatentImage"
+      },
+      "widgets_values": [
+        512,
+        512,
+        1
+      ]
+    },
+    {
+      "id": 6,
+      "type": "LoadImage",
+      "pos": [
+        50,
+        650
+      ],
+      "size": [
+        282.798828125,
+        314
+      ],
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "image",
+          "name": "image",
+          "type": "COMBO",
+          "widget": {
+            "name": "image"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "choose file to upload",
+          "name": "upload",
+          "type": "IMAGEUPLOAD",
+          "widget": {
+            "name": "upload"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "IMAGE",
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            19
+          ]
+        },
+        {
+          "localized_name": "MASK",
+          "name": "MASK",
+          "type": "MASK",
+          "links": null
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "LoadImage"
+      },
+      "widgets_values": [
+        "anima_lllite_any_hed_scribble_control.png",
+        "image"
+      ]
+    },
+    {
+      "id": 7,
+      "type": "VAELoader",
+      "pos": [
+        360,
+        700
+      ],
+      "size": [
+        270,
+        58
+      ],
+      "flags": {},
+      "order": 4,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "vae_name",
+          "name": "vae_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "vae_name"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "VAE",
+          "name": "VAE",
+          "type": "VAE",
+          "links": [
+            25
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAELoader",
+        "models": [
+          {
+            "name": "qwen_image_vae.safetensors",
+            "url": "https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/vae/qwen_image_vae.safetensors",
+            "directory": "vae"
+          }
+        ]
+      },
+      "widgets_values": [
+        "qwen_image_vae.safetensors"
+      ]
+    },
+    {
+      "id": 8,
+      "type": "ACN_AnimaLLLiteLoaderAdvanced",
+      "pos": [
+        860,
+        120
+      ],
+      "size": [
+        315,
+        58
+      ],
+      "flags": {},
+      "order": 5,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "timestep_kf",
+          "name": "timestep_kf",
+          "shape": 7,
+          "type": "TIMESTEP_KEYFRAME",
+          "link": null
+        },
+        {
+          "localized_name": "model_patch",
+          "name": "model_patch",
+          "type": "COMBO",
+          "widget": {
+            "name": "model_patch"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CONTROL_NET",
+          "name": "CONTROL_NET",
+          "type": "CONTROL_NET",
+          "links": [
+            18
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ACN_AnimaLLLiteLoaderAdvanced",
+        "models": [
+          {
+            "name": "anima-lllite-any-test-like-v2.safetensors",
+            "url": "https://huggingface.co/kohya-ss/Anima-LLLite/resolve/main/anima-lllite-any-test-like-v2.safetensors",
+            "directory": "model_patches"
+          }
+        ]
+      },
+      "widgets_values": [
+        "anima-lllite-any-test-like-v2.safetensors"
+      ]
+    },
+    {
+      "id": 9,
+      "type": "ACN_AdvancedControlNetApply_v2",
+      "pos": [
+        1210,
+        130
+      ],
+      "size": [
+        287.86183776855466,
+        266
+      ],
+      "flags": {},
+      "order": 8,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "positive",
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 16
+        },
+        {
+          "localized_name": "negative",
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 17
+        },
+        {
+          "localized_name": "control_net",
+          "name": "control_net",
+          "type": "CONTROL_NET",
+          "link": 18
+        },
+        {
+          "localized_name": "image",
+          "name": "image",
+          "type": "IMAGE",
+          "link": 19
+        },
+        {
+          "localized_name": "mask_optional",
+          "name": "mask_optional",
+          "shape": 7,
+          "type": "MASK",
+          "link": null
+        },
+        {
+          "localized_name": "timestep_kf",
+          "name": "timestep_kf",
+          "shape": 7,
+          "type": "TIMESTEP_KEYFRAME",
+          "link": null
+        },
+        {
+          "localized_name": "latent_kf_override",
+          "name": "latent_kf_override",
+          "shape": 7,
+          "type": "LATENT_KEYFRAME",
+          "link": null
+        },
+        {
+          "localized_name": "weights_override",
+          "name": "weights_override",
+          "shape": 7,
+          "type": "CONTROL_NET_WEIGHTS",
+          "link": null
+        },
+        {
+          "localized_name": "vae_optional",
+          "name": "vae_optional",
+          "shape": 7,
+          "type": "VAE",
+          "link": null
+        },
+        {
+          "localized_name": "strength",
+          "name": "strength",
+          "type": "FLOAT",
+          "widget": {
+            "name": "strength"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "start_percent",
+          "name": "start_percent",
+          "type": "FLOAT",
+          "widget": {
+            "name": "start_percent"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "end_percent",
+          "name": "end_percent",
+          "type": "FLOAT",
+          "widget": {
+            "name": "end_percent"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "positive",
+          "name": "positive",
+          "type": "CONDITIONING",
+          "links": [
+            21
+          ]
+        },
+        {
+          "localized_name": "negative",
+          "name": "negative",
+          "type": "CONDITIONING",
+          "links": [
+            22
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ACN_AdvancedControlNetApply_v2"
+      },
+      "widgets_values": [
+        1,
+        0,
+        1
+      ]
+    },
+    {
+      "id": 10,
+      "type": "KSampler",
+      "pos": [
+        1580,
+        110
+      ],
+      "size": [
+        270,
+        262
+      ],
+      "flags": {},
+      "order": 9,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "model",
+          "name": "model",
+          "type": "MODEL",
+          "link": 20
+        },
+        {
+          "localized_name": "positive",
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 21
+        },
+        {
+          "localized_name": "negative",
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 22
+        },
+        {
+          "localized_name": "latent_image",
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 23
+        },
+        {
+          "localized_name": "seed",
+          "name": "seed",
+          "type": "INT",
+          "widget": {
+            "name": "seed"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "steps",
+          "name": "steps",
+          "type": "INT",
+          "widget": {
+            "name": "steps"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "cfg",
+          "name": "cfg",
+          "type": "FLOAT",
+          "widget": {
+            "name": "cfg"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "sampler_name",
+          "name": "sampler_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "sampler_name"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "scheduler",
+          "name": "scheduler",
+          "type": "COMBO",
+          "widget": {
+            "name": "scheduler"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "denoise",
+          "name": "denoise",
+          "type": "FLOAT",
+          "widget": {
+            "name": "denoise"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "LATENT",
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            24
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "KSampler"
+      },
+      "widgets_values": [
+        82000,
+        "fixed",
+        12,
+        1,
+        "euler",
+        "simple",
+        1
+      ]
+    },
+    {
+      "id": 11,
+      "type": "VAEDecode",
+      "pos": [
+        1940,
+        140
+      ],
+      "size": [
+        140,
+        46
+      ],
+      "flags": {},
+      "order": 10,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "samples",
+          "name": "samples",
+          "type": "LATENT",
+          "link": 24
+        },
+        {
+          "localized_name": "vae",
+          "name": "vae",
+          "type": "VAE",
+          "link": 25
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "IMAGE",
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            26
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAEDecode"
+      }
+    },
+    {
+      "id": 12,
+      "type": "SaveImage",
+      "pos": [
+        2190,
+        110
+      ],
+      "size": [
+        270,
+        58
+      ],
+      "flags": {},
+      "order": 11,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "images",
+          "name": "images",
+          "type": "IMAGE",
+          "link": 26
+        },
+        {
+          "localized_name": "filename_prefix",
+          "name": "filename_prefix",
+          "type": "STRING",
+          "widget": {
+            "name": "filename_prefix"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "images",
+          "name": "images",
+          "type": "IMAGE",
+          "links": null
+        }
+      ],
+      "properties": {},
+      "widgets_values": [
+        "acn_anima_examples/any_hed_scribble"
+      ]
+    }
+  ],
+  "links": [
+    [
+      14,
+      2,
+      0,
+      3,
+      0,
+      "CLIP"
+    ],
+    [
+      15,
+      2,
+      0,
+      4,
+      0,
+      "CLIP"
+    ],
+    [
+      16,
+      3,
+      0,
+      9,
+      0,
+      "CONDITIONING"
+    ],
+    [
+      17,
+      4,
+      0,
+      9,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      18,
+      8,
+      0,
+      9,
+      2,
+      "CONTROL_NET"
+    ],
+    [
+      19,
+      6,
+      0,
+      9,
+      3,
+      "IMAGE"
+    ],
+    [
+      20,
+      1,
+      0,
+      10,
+      0,
+      "MODEL"
+    ],
+    [
+      21,
+      9,
+      0,
+      10,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      22,
+      9,
+      1,
+      10,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      23,
+      5,
+      0,
+      10,
+      3,
+      "LATENT"
+    ],
+    [
+      24,
+      10,
+      0,
+      11,
+      0,
+      "LATENT"
+    ],
+    [
+      25,
+      7,
+      0,
+      11,
+      1,
+      "VAE"
+    ],
+    [
+      26,
+      11,
+      0,
+      12,
+      0,
+      "IMAGE"
+    ]
+  ],
+  "groups": [
+    {
+      "id": 20,
+      "title": "SHARED INPUTS",
+      "bounding": [
+        10,
+        35,
+        740,
+        900
+      ],
+      "color": "#3f789e",
+      "font_size": 32,
+      "flags": {}
+    },
+    {
+      "id": 21,
+      "title": "ANIMA LLLITE V2 ANY - HED SCRIBBLE",
+      "bounding": [
+        810,
+        35,
+        1660,
+        900
+      ],
+      "color": "#6f4d86",
+      "font_size": 32,
+      "flags": {}
+    }
+  ],
+  "config": {},
+  "extra": {
+    "ds": {
+      "scale": 1,
+      "offset": [
+        0,
+        0
+      ]
+    },
+    "info": {
+      "name": "Anima LLLite v2 any - HED scribble",
+      "author": "Kosinkadink",
+      "description": "Advanced-ControlNet example for Anima LLLite v2 any with HED scribble conditioning."
+    }
+  },
+  "version": 0.4
+}

examples/anima_lllite_v2/anima_lllite_any_lineart.json

@@ -0,0 +1,941 @@
+{
+  "id": "00000000-0000-0000-0000-000000000000",
+  "revision": 0,
+  "last_node_id": 12,
+  "last_link_id": 26,
+  "nodes": [
+    {
+      "id": 1,
+      "type": "UNETLoader",
+      "pos": [
+        50,
+        100
+      ],
+      "size": [
+        270,
+        82
+      ],
+      "flags": {},
+      "order": 0,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "unet_name",
+          "name": "unet_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "unet_name"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "weight_dtype",
+          "name": "weight_dtype",
+          "type": "COMBO",
+          "widget": {
+            "name": "weight_dtype"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "MODEL",
+          "name": "MODEL",
+          "type": "MODEL",
+          "links": [
+            20
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "UNETLoader",
+        "models": [
+          {
+            "name": "anima-base-v1.0.safetensors",
+            "url": "https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/diffusion_models/anima-base-v1.0.safetensors",
+            "directory": "diffusion_models"
+          }
+        ]
+      },
+      "widgets_values": [
+        "anima-base-v1.0.safetensors",
+        "default"
+      ]
+    },
+    {
+      "id": 2,
+      "type": "CLIPLoader",
+      "pos": [
+        50,
+        250
+      ],
+      "size": [
+        270,
+        106
+      ],
+      "flags": {},
+      "order": 1,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "clip_name",
+          "name": "clip_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "clip_name"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "type",
+          "name": "type",
+          "type": "COMBO",
+          "widget": {
+            "name": "type"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "device",
+          "name": "device",
+          "shape": 7,
+          "type": "COMBO",
+          "widget": {
+            "name": "device"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CLIP",
+          "name": "CLIP",
+          "type": "CLIP",
+          "links": [
+            14,
+            15
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPLoader",
+        "models": [
+          {
+            "name": "qwen_3_06b_base.safetensors",
+            "url": "https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/text_encoders/qwen_3_06b_base.safetensors",
+            "directory": "text_encoders"
+          }
+        ]
+      },
+      "widgets_values": [
+        "qwen_3_06b_base.safetensors",
+        "qwen_image",
+        "default"
+      ]
+    },
+    {
+      "id": 3,
+      "type": "CLIPTextEncode",
+      "pos": [
+        360,
+        190
+      ],
+      "size": [
+        400,
+        200
+      ],
+      "flags": {},
+      "order": 6,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "clip",
+          "name": "clip",
+          "type": "CLIP",
+          "link": 14
+        },
+        {
+          "localized_name": "text",
+          "name": "text",
+          "type": "STRING",
+          "widget": {
+            "name": "text"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CONDITIONING",
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            16
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        "anime illustration of a female knight in ornate silver plate armor holding a sword, side profile in a dense forest, detailed"
+      ]
+    },
+    {
+      "id": 4,
+      "type": "CLIPTextEncode",
+      "pos": [
+        360,
+        440
+      ],
+      "size": [
+        400,
+        200
+      ],
+      "flags": {},
+      "order": 7,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "clip",
+          "name": "clip",
+          "type": "CLIP",
+          "link": 15
+        },
+        {
+          "localized_name": "text",
+          "name": "text",
+          "type": "STRING",
+          "widget": {
+            "name": "text"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CONDITIONING",
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            17
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        ""
+      ]
+    },
+    {
+      "id": 5,
+      "type": "EmptySD3LatentImage",
+      "pos": [
+        50,
+        470
+      ],
+      "size": [
+        270,
+        106
+      ],
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "width",
+          "name": "width",
+          "type": "INT",
+          "widget": {
+            "name": "width"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "height",
+          "name": "height",
+          "type": "INT",
+          "widget": {
+            "name": "height"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "batch_size",
+          "name": "batch_size",
+          "type": "INT",
+          "widget": {
+            "name": "batch_size"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "LATENT",
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            23
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "EmptySD3LatentImage"
+      },
+      "widgets_values": [
+        512,
+        512,
+        1
+      ]
+    },
+    {
+      "id": 6,
+      "type": "LoadImage",
+      "pos": [
+        50,
+        650
+      ],
+      "size": [
+        282.798828125,
+        314
+      ],
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "image",
+          "name": "image",
+          "type": "COMBO",
+          "widget": {
+            "name": "image"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "choose file to upload",
+          "name": "upload",
+          "type": "IMAGEUPLOAD",
+          "widget": {
+            "name": "upload"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "IMAGE",
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            19
+          ]
+        },
+        {
+          "localized_name": "MASK",
+          "name": "MASK",
+          "type": "MASK",
+          "links": null
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "LoadImage"
+      },
+      "widgets_values": [
+        "anima_lllite_any_lineart_control.png",
+        "image"
+      ]
+    },
+    {
+      "id": 7,
+      "type": "VAELoader",
+      "pos": [
+        360,
+        700
+      ],
+      "size": [
+        270,
+        58
+      ],
+      "flags": {},
+      "order": 4,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "vae_name",
+          "name": "vae_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "vae_name"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "VAE",
+          "name": "VAE",
+          "type": "VAE",
+          "links": [
+            25
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAELoader",
+        "models": [
+          {
+            "name": "qwen_image_vae.safetensors",
+            "url": "https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/vae/qwen_image_vae.safetensors",
+            "directory": "vae"
+          }
+        ]
+      },
+      "widgets_values": [
+        "qwen_image_vae.safetensors"
+      ]
+    },
+    {
+      "id": 8,
+      "type": "ACN_AnimaLLLiteLoaderAdvanced",
+      "pos": [
+        860,
+        120
+      ],
+      "size": [
+        315,
+        58
+      ],
+      "flags": {},
+      "order": 5,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "timestep_kf",
+          "name": "timestep_kf",
+          "shape": 7,
+          "type": "TIMESTEP_KEYFRAME",
+          "link": null
+        },
+        {
+          "localized_name": "model_patch",
+          "name": "model_patch",
+          "type": "COMBO",
+          "widget": {
+            "name": "model_patch"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CONTROL_NET",
+          "name": "CONTROL_NET",
+          "type": "CONTROL_NET",
+          "links": [
+            18
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ACN_AnimaLLLiteLoaderAdvanced",
+        "models": [
+          {
+            "name": "anima-lllite-any-test-like-v2.safetensors",
+            "url": "https://huggingface.co/kohya-ss/Anima-LLLite/resolve/main/anima-lllite-any-test-like-v2.safetensors",
+            "directory": "model_patches"
+          }
+        ]
+      },
+      "widgets_values": [
+        "anima-lllite-any-test-like-v2.safetensors"
+      ]
+    },
+    {
+      "id": 9,
+      "type": "ACN_AdvancedControlNetApply_v2",
+      "pos": [
+        1210,
+        130
+      ],
+      "size": [
+        287.86183776855466,
+        266
+      ],
+      "flags": {},
+      "order": 8,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "positive",
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 16
+        },
+        {
+          "localized_name": "negative",
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 17
+        },
+        {
+          "localized_name": "control_net",
+          "name": "control_net",
+          "type": "CONTROL_NET",
+          "link": 18
+        },
+        {
+          "localized_name": "image",
+          "name": "image",
+          "type": "IMAGE",
+          "link": 19
+        },
+        {
+          "localized_name": "mask_optional",
+          "name": "mask_optional",
+          "shape": 7,
+          "type": "MASK",
+          "link": null
+        },
+        {
+          "localized_name": "timestep_kf",
+          "name": "timestep_kf",
+          "shape": 7,
+          "type": "TIMESTEP_KEYFRAME",
+          "link": null
+        },
+        {
+          "localized_name": "latent_kf_override",
+          "name": "latent_kf_override",
+          "shape": 7,
+          "type": "LATENT_KEYFRAME",
+          "link": null
+        },
+        {
+          "localized_name": "weights_override",
+          "name": "weights_override",
+          "shape": 7,
+          "type": "CONTROL_NET_WEIGHTS",
+          "link": null
+        },
+        {
+          "localized_name": "vae_optional",
+          "name": "vae_optional",
+          "shape": 7,
+          "type": "VAE",
+          "link": null
+        },
+        {
+          "localized_name": "strength",
+          "name": "strength",
+          "type": "FLOAT",
+          "widget": {
+            "name": "strength"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "start_percent",
+          "name": "start_percent",
+          "type": "FLOAT",
+          "widget": {
+            "name": "start_percent"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "end_percent",
+          "name": "end_percent",
+          "type": "FLOAT",
+          "widget": {
+            "name": "end_percent"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "positive",
+          "name": "positive",
+          "type": "CONDITIONING",
+          "links": [
+            21
+          ]
+        },
+        {
+          "localized_name": "negative",
+          "name": "negative",
+          "type": "CONDITIONING",
+          "links": [
+            22
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ACN_AdvancedControlNetApply_v2"
+      },
+      "widgets_values": [
+        1,
+        0,
+        1
+      ]
+    },
+    {
+      "id": 10,
+      "type": "KSampler",
+      "pos": [
+        1580,
+        110
+      ],
+      "size": [
+        270,
+        262
+      ],
+      "flags": {},
+      "order": 9,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "model",
+          "name": "model",
+          "type": "MODEL",
+          "link": 20
+        },
+        {
+          "localized_name": "positive",
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 21
+        },
+        {
+          "localized_name": "negative",
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 22
+        },
+        {
+          "localized_name": "latent_image",
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 23
+        },
+        {
+          "localized_name": "seed",
+          "name": "seed",
+          "type": "INT",
+          "widget": {
+            "name": "seed"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "steps",
+          "name": "steps",
+          "type": "INT",
+          "widget": {
+            "name": "steps"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "cfg",
+          "name": "cfg",
+          "type": "FLOAT",
+          "widget": {
+            "name": "cfg"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "sampler_name",
+          "name": "sampler_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "sampler_name"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "scheduler",
+          "name": "scheduler",
+          "type": "COMBO",
+          "widget": {
+            "name": "scheduler"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "denoise",
+          "name": "denoise",
+          "type": "FLOAT",
+          "widget": {
+            "name": "denoise"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "LATENT",
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            24
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "KSampler"
+      },
+      "widgets_values": [
+        82000,
+        "fixed",
+        12,
+        1,
+        "euler",
+        "simple",
+        1
+      ]
+    },
+    {
+      "id": 11,
+      "type": "VAEDecode",
+      "pos": [
+        1940,
+        140
+      ],
+      "size": [
+        140,
+        46
+      ],
+      "flags": {},
+      "order": 10,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "samples",
+          "name": "samples",
+          "type": "LATENT",
+          "link": 24
+        },
+        {
+          "localized_name": "vae",
+          "name": "vae",
+          "type": "VAE",
+          "link": 25
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "IMAGE",
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            26
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAEDecode"
+      }
+    },
+    {
+      "id": 12,
+      "type": "SaveImage",
+      "pos": [
+        2190,
+        110
+      ],
+      "size": [
+        270,
+        58
+      ],
+      "flags": {},
+      "order": 11,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "images",
+          "name": "images",
+          "type": "IMAGE",
+          "link": 26
+        },
+        {
+          "localized_name": "filename_prefix",
+          "name": "filename_prefix",
+          "type": "STRING",
+          "widget": {
+            "name": "filename_prefix"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "images",
+          "name": "images",
+          "type": "IMAGE",
+          "links": null
+        }
+      ],
+      "properties": {},
+      "widgets_values": [
+        "acn_anima_examples/any_lineart"
+      ]
+    }
+  ],
+  "links": [
+    [
+      14,
+      2,
+      0,
+      3,
+      0,
+      "CLIP"
+    ],
+    [
+      15,
+      2,
+      0,
+      4,
+      0,
+      "CLIP"
+    ],
+    [
+      16,
+      3,
+      0,
+      9,
+      0,
+      "CONDITIONING"
+    ],
+    [
+      17,
+      4,
+      0,
+      9,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      18,
+      8,
+      0,
+      9,
+      2,
+      "CONTROL_NET"
+    ],
+    [
+      19,
+      6,
+      0,
+      9,
+      3,
+      "IMAGE"
+    ],
+    [
+      20,
+      1,
+      0,
+      10,
+      0,
+      "MODEL"
+    ],
+    [
+      21,
+      9,
+      0,
+      10,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      22,
+      9,
+      1,
+      10,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      23,
+      5,
+      0,
+      10,
+      3,
+      "LATENT"
+    ],
+    [
+      24,
+      10,
+      0,
+      11,
+      0,
+      "LATENT"
+    ],
+    [
+      25,
+      7,
+      0,
+      11,
+      1,
+      "VAE"
+    ],
+    [
+      26,
+      11,
+      0,
+      12,
+      0,
+      "IMAGE"
+    ]
+  ],
+  "groups": [
+    {
+      "id": 20,
+      "title": "SHARED INPUTS",
+      "bounding": [
+        10,
+        35,
+        740,
+        900
+      ],
+      "color": "#3f789e",
+      "font_size": 32,
+      "flags": {}
+    },
+    {
+      "id": 21,
+      "title": "ANIMA LLLITE V2 ANY - LINEART",
+      "bounding": [
+        810,
+        35,
+        1660,
+        900
+      ],
+      "color": "#6f4d86",
+      "font_size": 32,
+      "flags": {}
+    }
+  ],
+  "config": {},
+  "extra": {
+    "ds": {
+      "scale": 1,
+      "offset": [
+        0,
+        0
+      ]
+    },
+    "info": {
+      "name": "Anima LLLite v2 any - lineart",
+      "author": "Kosinkadink",
+      "description": "Advanced-ControlNet example for Anima LLLite v2 any with lineart conditioning."
+    }
+  },
+  "version": 0.4
+}

examples/anima_lllite_v2/anima_lllite_any_pidinet_scribble.json

@@ -0,0 +1,941 @@
+{
+  "id": "00000000-0000-0000-0000-000000000000",
+  "revision": 0,
+  "last_node_id": 12,
+  "last_link_id": 26,
+  "nodes": [
+    {
+      "id": 1,
+      "type": "UNETLoader",
+      "pos": [
+        50,
+        100
+      ],
+      "size": [
+        270,
+        82
+      ],
+      "flags": {},
+      "order": 0,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "unet_name",
+          "name": "unet_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "unet_name"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "weight_dtype",
+          "name": "weight_dtype",
+          "type": "COMBO",
+          "widget": {
+            "name": "weight_dtype"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "MODEL",
+          "name": "MODEL",
+          "type": "MODEL",
+          "links": [
+            20
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "UNETLoader",
+        "models": [
+          {
+            "name": "anima-base-v1.0.safetensors",
+            "url": "https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/diffusion_models/anima-base-v1.0.safetensors",
+            "directory": "diffusion_models"
+          }
+        ]
+      },
+      "widgets_values": [
+        "anima-base-v1.0.safetensors",
+        "default"
+      ]
+    },
+    {
+      "id": 2,
+      "type": "CLIPLoader",
+      "pos": [
+        50,
+        250
+      ],
+      "size": [
+        270,
+        106
+      ],
+      "flags": {},
+      "order": 1,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "clip_name",
+          "name": "clip_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "clip_name"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "type",
+          "name": "type",
+          "type": "COMBO",
+          "widget": {
+            "name": "type"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "device",
+          "name": "device",
+          "shape": 7,
+          "type": "COMBO",
+          "widget": {
+            "name": "device"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CLIP",
+          "name": "CLIP",
+          "type": "CLIP",
+          "links": [
+            14,
+            15
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPLoader",
+        "models": [
+          {
+            "name": "qwen_3_06b_base.safetensors",
+            "url": "https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/text_encoders/qwen_3_06b_base.safetensors",
+            "directory": "text_encoders"
+          }
+        ]
+      },
+      "widgets_values": [
+        "qwen_3_06b_base.safetensors",
+        "qwen_image",
+        "default"
+      ]
+    },
+    {
+      "id": 3,
+      "type": "CLIPTextEncode",
+      "pos": [
+        360,
+        190
+      ],
+      "size": [
+        400,
+        200
+      ],
+      "flags": {},
+      "order": 6,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "clip",
+          "name": "clip",
+          "type": "CLIP",
+          "link": 14
+        },
+        {
+          "localized_name": "text",
+          "name": "text",
+          "type": "STRING",
+          "widget": {
+            "name": "text"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CONDITIONING",
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            16
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        "anime illustration of a female knight in ornate silver plate armor holding a sword, side profile in a dense forest, detailed"
+      ]
+    },
+    {
+      "id": 4,
+      "type": "CLIPTextEncode",
+      "pos": [
+        360,
+        440
+      ],
+      "size": [
+        400,
+        200
+      ],
+      "flags": {},
+      "order": 7,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "clip",
+          "name": "clip",
+          "type": "CLIP",
+          "link": 15
+        },
+        {
+          "localized_name": "text",
+          "name": "text",
+          "type": "STRING",
+          "widget": {
+            "name": "text"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CONDITIONING",
+          "name": "CONDITIONING",
+          "type": "CONDITIONING",
+          "links": [
+            17
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "CLIPTextEncode"
+      },
+      "widgets_values": [
+        ""
+      ]
+    },
+    {
+      "id": 5,
+      "type": "EmptySD3LatentImage",
+      "pos": [
+        50,
+        470
+      ],
+      "size": [
+        270,
+        106
+      ],
+      "flags": {},
+      "order": 2,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "width",
+          "name": "width",
+          "type": "INT",
+          "widget": {
+            "name": "width"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "height",
+          "name": "height",
+          "type": "INT",
+          "widget": {
+            "name": "height"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "batch_size",
+          "name": "batch_size",
+          "type": "INT",
+          "widget": {
+            "name": "batch_size"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "LATENT",
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            23
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "EmptySD3LatentImage"
+      },
+      "widgets_values": [
+        512,
+        512,
+        1
+      ]
+    },
+    {
+      "id": 6,
+      "type": "LoadImage",
+      "pos": [
+        50,
+        650
+      ],
+      "size": [
+        282.798828125,
+        314
+      ],
+      "flags": {},
+      "order": 3,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "image",
+          "name": "image",
+          "type": "COMBO",
+          "widget": {
+            "name": "image"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "choose file to upload",
+          "name": "upload",
+          "type": "IMAGEUPLOAD",
+          "widget": {
+            "name": "upload"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "IMAGE",
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            19
+          ]
+        },
+        {
+          "localized_name": "MASK",
+          "name": "MASK",
+          "type": "MASK",
+          "links": null
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "LoadImage"
+      },
+      "widgets_values": [
+        "anima_lllite_any_pidinet_scribble_control.png",
+        "image"
+      ]
+    },
+    {
+      "id": 7,
+      "type": "VAELoader",
+      "pos": [
+        360,
+        700
+      ],
+      "size": [
+        270,
+        58
+      ],
+      "flags": {},
+      "order": 4,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "vae_name",
+          "name": "vae_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "vae_name"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "VAE",
+          "name": "VAE",
+          "type": "VAE",
+          "links": [
+            25
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAELoader",
+        "models": [
+          {
+            "name": "qwen_image_vae.safetensors",
+            "url": "https://huggingface.co/circlestone-labs/Anima/resolve/main/split_files/vae/qwen_image_vae.safetensors",
+            "directory": "vae"
+          }
+        ]
+      },
+      "widgets_values": [
+        "qwen_image_vae.safetensors"
+      ]
+    },
+    {
+      "id": 8,
+      "type": "ACN_AnimaLLLiteLoaderAdvanced",
+      "pos": [
+        860,
+        120
+      ],
+      "size": [
+        315,
+        58
+      ],
+      "flags": {},
+      "order": 5,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "timestep_kf",
+          "name": "timestep_kf",
+          "shape": 7,
+          "type": "TIMESTEP_KEYFRAME",
+          "link": null
+        },
+        {
+          "localized_name": "model_patch",
+          "name": "model_patch",
+          "type": "COMBO",
+          "widget": {
+            "name": "model_patch"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "CONTROL_NET",
+          "name": "CONTROL_NET",
+          "type": "CONTROL_NET",
+          "links": [
+            18
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ACN_AnimaLLLiteLoaderAdvanced",
+        "models": [
+          {
+            "name": "anima-lllite-any-test-like-v2.safetensors",
+            "url": "https://huggingface.co/kohya-ss/Anima-LLLite/resolve/main/anima-lllite-any-test-like-v2.safetensors",
+            "directory": "model_patches"
+          }
+        ]
+      },
+      "widgets_values": [
+        "anima-lllite-any-test-like-v2.safetensors"
+      ]
+    },
+    {
+      "id": 9,
+      "type": "ACN_AdvancedControlNetApply_v2",
+      "pos": [
+        1210,
+        130
+      ],
+      "size": [
+        287.86183776855466,
+        266
+      ],
+      "flags": {},
+      "order": 8,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "positive",
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 16
+        },
+        {
+          "localized_name": "negative",
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 17
+        },
+        {
+          "localized_name": "control_net",
+          "name": "control_net",
+          "type": "CONTROL_NET",
+          "link": 18
+        },
+        {
+          "localized_name": "image",
+          "name": "image",
+          "type": "IMAGE",
+          "link": 19
+        },
+        {
+          "localized_name": "mask_optional",
+          "name": "mask_optional",
+          "shape": 7,
+          "type": "MASK",
+          "link": null
+        },
+        {
+          "localized_name": "timestep_kf",
+          "name": "timestep_kf",
+          "shape": 7,
+          "type": "TIMESTEP_KEYFRAME",
+          "link": null
+        },
+        {
+          "localized_name": "latent_kf_override",
+          "name": "latent_kf_override",
+          "shape": 7,
+          "type": "LATENT_KEYFRAME",
+          "link": null
+        },
+        {
+          "localized_name": "weights_override",
+          "name": "weights_override",
+          "shape": 7,
+          "type": "CONTROL_NET_WEIGHTS",
+          "link": null
+        },
+        {
+          "localized_name": "vae_optional",
+          "name": "vae_optional",
+          "shape": 7,
+          "type": "VAE",
+          "link": null
+        },
+        {
+          "localized_name": "strength",
+          "name": "strength",
+          "type": "FLOAT",
+          "widget": {
+            "name": "strength"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "start_percent",
+          "name": "start_percent",
+          "type": "FLOAT",
+          "widget": {
+            "name": "start_percent"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "end_percent",
+          "name": "end_percent",
+          "type": "FLOAT",
+          "widget": {
+            "name": "end_percent"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "positive",
+          "name": "positive",
+          "type": "CONDITIONING",
+          "links": [
+            21
+          ]
+        },
+        {
+          "localized_name": "negative",
+          "name": "negative",
+          "type": "CONDITIONING",
+          "links": [
+            22
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "ACN_AdvancedControlNetApply_v2"
+      },
+      "widgets_values": [
+        1,
+        0,
+        1
+      ]
+    },
+    {
+      "id": 10,
+      "type": "KSampler",
+      "pos": [
+        1580,
+        110
+      ],
+      "size": [
+        270,
+        262
+      ],
+      "flags": {},
+      "order": 9,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "model",
+          "name": "model",
+          "type": "MODEL",
+          "link": 20
+        },
+        {
+          "localized_name": "positive",
+          "name": "positive",
+          "type": "CONDITIONING",
+          "link": 21
+        },
+        {
+          "localized_name": "negative",
+          "name": "negative",
+          "type": "CONDITIONING",
+          "link": 22
+        },
+        {
+          "localized_name": "latent_image",
+          "name": "latent_image",
+          "type": "LATENT",
+          "link": 23
+        },
+        {
+          "localized_name": "seed",
+          "name": "seed",
+          "type": "INT",
+          "widget": {
+            "name": "seed"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "steps",
+          "name": "steps",
+          "type": "INT",
+          "widget": {
+            "name": "steps"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "cfg",
+          "name": "cfg",
+          "type": "FLOAT",
+          "widget": {
+            "name": "cfg"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "sampler_name",
+          "name": "sampler_name",
+          "type": "COMBO",
+          "widget": {
+            "name": "sampler_name"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "scheduler",
+          "name": "scheduler",
+          "type": "COMBO",
+          "widget": {
+            "name": "scheduler"
+          },
+          "link": null
+        },
+        {
+          "localized_name": "denoise",
+          "name": "denoise",
+          "type": "FLOAT",
+          "widget": {
+            "name": "denoise"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "LATENT",
+          "name": "LATENT",
+          "type": "LATENT",
+          "links": [
+            24
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "KSampler"
+      },
+      "widgets_values": [
+        82000,
+        "fixed",
+        12,
+        1,
+        "euler",
+        "simple",
+        1
+      ]
+    },
+    {
+      "id": 11,
+      "type": "VAEDecode",
+      "pos": [
+        1940,
+        140
+      ],
+      "size": [
+        140,
+        46
+      ],
+      "flags": {},
+      "order": 10,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "samples",
+          "name": "samples",
+          "type": "LATENT",
+          "link": 24
+        },
+        {
+          "localized_name": "vae",
+          "name": "vae",
+          "type": "VAE",
+          "link": 25
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "IMAGE",
+          "name": "IMAGE",
+          "type": "IMAGE",
+          "links": [
+            26
+          ]
+        }
+      ],
+      "properties": {
+        "Node name for S&R": "VAEDecode"
+      }
+    },
+    {
+      "id": 12,
+      "type": "SaveImage",
+      "pos": [
+        2190,
+        110
+      ],
+      "size": [
+        270,
+        58
+      ],
+      "flags": {},
+      "order": 11,
+      "mode": 0,
+      "inputs": [
+        {
+          "localized_name": "images",
+          "name": "images",
+          "type": "IMAGE",
+          "link": 26
+        },
+        {
+          "localized_name": "filename_prefix",
+          "name": "filename_prefix",
+          "type": "STRING",
+          "widget": {
+            "name": "filename_prefix"
+          },
+          "link": null
+        }
+      ],
+      "outputs": [
+        {
+          "localized_name": "images",
+          "name": "images",
+          "type": "IMAGE",
+          "links": null
+        }
+      ],
+      "properties": {},
+      "widgets_values": [
+        "acn_anima_examples/any_pidinet_scribble"
+      ]
+    }
+  ],
+  "links": [
+    [
+      14,
+      2,
+      0,
+      3,
+      0,
+      "CLIP"
+    ],
+    [
+      15,
+      2,
+      0,
+      4,
+      0,
+      "CLIP"
+    ],
+    [
+      16,
+      3,
+      0,
+      9,
+      0,
+      "CONDITIONING"
+    ],
+    [
+      17,
+      4,
+      0,
+      9,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      18,
+      8,
+      0,
+      9,
+      2,
+      "CONTROL_NET"
+    ],
+    [
+      19,
+      6,
+      0,
+      9,
+      3,
+      "IMAGE"
+    ],
+    [
+      20,
+      1,
+      0,
+      10,
+      0,
+      "MODEL"
+    ],
+    [
+      21,
+      9,
+      0,
+      10,
+      1,
+      "CONDITIONING"
+    ],
+    [
+      22,
+      9,
+      1,
+      10,
+      2,
+      "CONDITIONING"
+    ],
+    [
+      23,
+      5,
+      0,
+      10,
+      3,
+      "LATENT"
+    ],
+    [
+      24,
+      10,
+      0,
+      11,
+      0,
+      "LATENT"
+    ],
+    [
+      25,
+      7,
+      0,
+      11,
+      1,
+      "VAE"
+    ],
+    [
+      26,
+      11,
+      0,
+      12,
+      0,
+      "IMAGE"
+    ]
+  ],
+  "groups": [
+    {
+      "id": 20,
+      "title": "SHARED INPUTS",
+      "bounding": [
+        10,
+        35,
+        740,
+        900
+      ],
+      "color": "#3f789e",
+      "font_size": 32,
+      "flags": {}
+    },
+    {
+      "id": 21,
+      "title": "ANIMA LLLITE V2 ANY - PIDINET SCRIBBLE",
+      "bounding": [
+        810,
+        35,
+        1660,
+        900
+      ],
+      "color": "#6f4d86",
+      "font_size": 32,
+      "flags": {}
+    }
+  ],
+  "config": {},
+  "extra": {
+    "ds": {
+      "scale": 1,
+      "offset": [
+        0,
+        0
+      ]
+    },
+    "info": {
+      "name": "Anima LLLite v2 any - PiDiNet scribble",
+      "author": "Kosinkadink",
+      "description": "Advanced-ControlNet example for Anima LLLite v2 any with PiDiNet scribble conditioning."
+    }
+  },
+  "version": 0.4
+}

pyproject.toml

@@ -1,8 +1,8 @@
 [project]
 name = "comfyui-advanced-controlnet"
 description = "Nodes for scheduling ControlNet strength across timesteps and batched latents, as well as applying custom weights and attention masks."
-version = "1.0.2"
+version = "1.5.8"
-license = "LICENSE"
+license = { file = "LICENSE" }
 dependencies = []
 
 [project.urls]
@@ -13,3 +13,4 @@ Repository = "https://github.com/Kosinkadink/ComfyUI-Advanced-ControlNet"
 PublisherId = "kosinkadink"
 DisplayName = "ComfyUI-Advanced-ControlNet"
 Icon = ""
+requires-comfyui = ">=0.3.68"