Chore: Update pyproject.toml

This node loads multiple LoRAs in one action to prevent re-quantizing the model in Comfy. Good for quantized formats like FP8, Int8, and NVFP4.

README.md

@@ -0,0 +1,232 @@
+# ComfyUI-ClybsChromaNodes
+
+A small collection of custom nodes for [ComfyUI](https://github.com/comfyanonymous/ComfyUI), designed primarily for use with [Lodestone Rock's Chroma](https://huggingface.co/lodestones/Chroma) model (and compatible flow-matching architectures like FLUX and SD3). The package bundles custom guidance, samplers, schedulers, and an adaptive multi-LoRA loader, all of which integrate as standard ComfyUI nodes.
+
+## Installation
+
+Clone this repository into your ComfyUI `custom_nodes` directory and restart ComfyUI:
+
+```bash
+cd /path/to/ComfyUI/custom_nodes
+git clone https://github.com/Clybius/ComfyUI-ClybsChromaNodes.git
+```
+
+No additional Python dependencies are required beyond a working ComfyUI install. The frontend extension is picked up automatically via `WEB_DIRECTORY = "./js"`.
+
+## Node overview
+
+The package registers **8 nodes**, organized into four groups:
+
+| Group | Nodes |
+|---|---|
+| Guidance | `ClybGuidance` |
+| Samplers | `SamplerClyb_BDF`, `SamplerTaylorFlow`, `SamplerWrapperCFGPP` |
+| Schedulers | `InverseSquaredScheduler`, `PrintSigmas` |
+| LoRA Loaders | `ClybAdaptiveLoraLoader`, `ClybAdaptiveLoraLoaderModelOnly` |
+
+In addition, the `chroma_NAG.py` module ships a `ChromaNAG` class (Normalized Attention Guidance for Chroma's `DoubleStreamBlock`) that is **not currently registered** in the node mappings — the class is available in code but does not appear in the ComfyUI node browser.
+
+---
+
+## Guidance
+
+### `ClybGuidance`
+*File: `clyb_Guidance.py`*
+*Category: `sampling/custom_sampling`*
+
+A pre-CFG model patch that rewires how the conditional and unconditional predictions are combined at every sampling step. Stacks the following features on top of standard CFG:
+
+- **Project-and-scale (`eta`)** — the guidance vector is split into a component parallel to the conditional and a component orthogonal to it. The parallel component is scaled by `eta`, the orthogonal component is left alone. `eta = 1.0` recovers default CFG.
+- **Norm clamping (`norm_threshold`)** — if the L2 norm of the guided output exceeds the conditional's norm times `norm_threshold`, the guided output is rescaled back down. Disabled at `0.0`.
+- **Momentum (`momentum`, `momentum_beta`, `momentum_renorm`)** — adds a fraction of a running-average guidance vector to the current guidance, optionally re-normalized back to its original norm. `momentum = 0` disables it.
+- **Scalar projection (`scalar_projection`, `scalar_logsumexp`)** — projects the conditional onto the unconditional as a scalar (`logsumexp` or `sum` reduction), then scales the unconditional by that scalar.
+- **STD/var rescale (`rescale_phi`, `var_rescale`)** — blends the guided output toward an output whose standard deviation (or variance) matches the conditional's. `rescale_phi = 0` disables it.
+- **Sine-bell schedule (`scale_up_ratio`, `scale_up_shift`)** — animates the effective CFG scale from `1.0` at the start, up to the configured CFG scale at the middle of diffusion, and back down to `1.0` at the end. `scale_up_ratio = 0` disables it. `scale_up_shift < 1.0` shifts the bell later, `> 1.0` earlier.
+- **atan2/sin blend (`atan2sin_ratio`)** — blends the unconditional with `uncond.atan().sin() / cond.atan().cos()` (a Chroma-specific twist on the guidance direction).
+
+| Input | Type | Default | Range | Description |
+|---|---|---|---|---|
+| `model` | MODEL | — | — | Model to patch |
+| `eta` | FLOAT | 1.0 | -50, 50 | Parallel guidance scale |
+| `norm_threshold` | FLOAT | 0.0 | 0, 50 | Norm clamp (0 = off) |
+| `momentum` | FLOAT | 0.0 | -10, 10 | Momentum weight (0 = off) |
+| `momentum_beta` | FLOAT | 0.75 | 0, 0.999 | Running-average smoothing |
+| `momentum_renorm` | FLOAT | 1.0 | 0, 1 | Renormalize after momentum |
+| `scalar_projection` | BOOLEAN | False | — | Scalar projection of cond onto uncond |
+| `scalar_logsumexp` | BOOLEAN | False | — | Use `logsumexp` (else `sum`) |
+| `rescale_phi` | FLOAT | 0.0 | 0, 1 | STD-rescale blend (0 = off) |
+| `var_rescale` | BOOLEAN | False | — | Use `var` (else `std`) for rescale |
+| `scale_up_ratio` | FLOAT | 0.0 | 0, 1 | Sine-bell CFG weight (0 = off) |
+| `scale_up_shift` | FLOAT | 1.0 | 0.1, 10 | Sine-bell schedule shift |
+| `atan2sin_ratio` | FLOAT | 0.0 | -100, 100 | atan2/sin blend (0 = off) |
+
+**Returns:** `MODEL` (patched).
+
+---
+
+## Samplers
+
+All three nodes return a `SAMPLER` object intended to be plugged into the `sampler` input of `KSampler` (or any node that accepts a sampler).
+
+### `SamplerClyb_BDF`
+*File: `clyb_Samplers.py`*
+*Category: `sampling/custom_sampling/samplers`*
+
+A backward-differentiation-formula-style sampler that takes a single model evaluation at the start of the step, then synthesizes a refined denoised prediction at the `sigma_down` point and combines them with one of three scalar fusions:
+
+- `projection` — projects the half-step prediction back onto the line spanned by the full-step prediction.
+- `atan2sin` — uses `atan2(sin(half), cos(full))` to blend the two predictions in angle space.
+- `atan2sin+projection` (default) — the atan2/sin blend followed by a projection rescaling, combining both stabilizations.
+
+The sampler detects flow-matching models (FLUX, SD3, Chroma) automatically and switches to the flow-style ancestral update with `alpha_ip1`/`alpha_down`/`renoise_coeff`.
+
+| Input | Type | Default | Range | Description |
+|---|---|---|---|---|
+| `scalar` | ENUM | `atan2sin+projection` | `projection`, `atan2sin`, `atan2sin+projection` | Scalar fusion mode |
+| `eta` | FLOAT | 1.0 | 0, 100 | Ancestral stochasticity |
+| `s_noise` | FLOAT | 1.0 | 0, 100 | Noise scaling factor |
+
+### `SamplerTaylorFlow`
+*File: `clyb_Samplers.py`*
+*Category: `sampling/custom_sampling/samplers`*
+
+A multi-step Taylor-expansion sampler (registered as `taylor_flow` in the ComfyUI sampler list). Implements the algorithm from *"Leveraging Previous Steps: A Training-free Fast Solver for Flow Diffusion"* (Nov 2024):
+
+1. Maintain a rolling history of `(sigma, denoised)` pairs from the previous `order` steps.
+2. At each step, perform **one** model evaluation at the current state.
+3. Build a Vandermonde matrix from the historical `sigma` values.
+4. Solve for Taylor coefficients `B` that predict the latent at `sigma_next`.
+5. Apply the Euler step plus a correction term built from the history.
+6. Inject ancestral noise as usual.
+
+The four `sigma_calc` modes control how the `sigma_down` / `sigma_up` pair is computed for the ancestral noise injection:
+
+- `clyb` (default) — logarithmic scaling, the original Clyb scheme.
+- `taylor-expansion` — exponential factor with a quadratic correction based on the step ratio.
+- `ancestral` — standard k-diffusion `get_ancestral_step`.
+- `adaptive` — converges toward the standard scheme based on a normalized variance of the denoised history (small history variance ⇒ less noise).
+
+The Vandermonde solve supports two methods internally (iterative two-sided equilibration with Tikhonov regularization, and diagonal-dominant extraction) and falls back to a `lstsq` solve if the regularized system is singular.
+
+| Input | Type | Default | Range | Description |
+|---|---|---|---|---|
+| `order` | INT | 8 | 1, 16 | Taylor expansion order (history length) |
+| `eta` | FLOAT | 1.0 | 0, 1 | Ancestral stochasticity |
+| `s_noise` | FLOAT | 1.0 | 0, 2 | Noise scaling factor |
+| `sigma_calc` | ENUM | `clyb` | `clyb`, `taylor-expansion`, `ancestral`, `adaptive` | Ancestral sigma calculation method |
+
+### `SamplerWrapperCFGPP`
+*File: `clyb_Samplers.py`*
+*Category: `sampling/custom_sampling/samplers`*
+
+A *sampler wrapper* — takes any other `SAMPLER` as input and returns a new sampler that runs the inner sampler but with a CFG++-style denoised recomputation. CFG++ replaces the standard CFG blend with a closed-form denoised that uses the unconditional prediction from the *next* sigma:
+
+```
+denoised_star = (sigma * alpha_t * denoised_guided
+                 - sigma_next * alpha_s * uncond_denoised) / (sigma - sigma_next)
+```
+
+where `alpha_s = sigma * exp(lambda(sigma))` and `alpha_t = sigma_next * exp(lambda(sigma_next))`, with `lambda(s) = sigma_to_half_log_snr(s, model_sampling)`.
+
+The wrapper installs a `post_cfg_function` hook on the model to capture `uncond_denoised` from the inner sampler, then uses a `CFGPPProxyModel` to perform the recombination at every step. The wrapper itself is **not** added to the standard KSampler dropdown — it is only reachable through this node (or any node that constructs it via `comfy.samplers.ksampler("cfgpp", {...})`).
+
+| Input | Type | Description |
+|---|---|---|
+| `sampler` | SAMPLER | Inner sampler to wrap with CFG++ |
+
+---
+
+## Schedulers
+
+### `InverseSquaredScheduler`
+*File: `clyb_Schedulers.py`*
+*Category: `sampling/custom_sampling/schedulers`*
+
+A sigma scheduler that biases the schedule toward the *end* of diffusion. It uses `(1 - t²)²` (i.e. the inverse of `t` mapped through `(1-t)²`) to pick sigma indices — fine-grained near the end, coarser at the start. The scheduler is also registered into `SCHEDULER_HANDLERS` under the name `inverse_squared`, so it can be used as a string in any node that accepts a scheduler name.
+
+| Input | Type | Default | Range | Description |
+|---|---|---|---|---|
+| `model` | MODEL | — | — | Model to derive the sigma range from |
+| `steps` | INT | 20 | 3, 1000 | Number of steps |
+| `denoise` | FLOAT | 1.0 | 0, 1 | Denoise strength (< 1.0 enables img2img-style short schedules) |
+
+**Returns:** `SIGMAS`.
+
+### `PrintSigmas`
+*File: `clyb_Schedulers.py`*
+*Category: `sampling/custom_sampling/schedulers`*
+
+A debug helper that prints the incoming `SIGMAS` tensor to the console and passes it through unchanged. Useful for inspecting schedules from other nodes without modifying them.
+
+| Input | Type | Description |
+|---|---|---|
+| `sigmas` | SIGMAS | Sigma tensor to print |
+
+**Returns:** `SIGMAS` (passthrough).
+
+---
+
+## LoRA Loaders
+
+Both loaders share a frontend extension (`js/clyb_adaptive_lora.js`) that dynamically reveals the next `lora_name_N` / `strength_*_N` triplet only after the previous `lora_name_M` is set to a non-`"none"` value (up to a cap of 20 LoRAs). Setting a slot back to `"none"` hides the trailing widgets, and serialization / deserialization are handled correctly so that hidden widget values survive workflow save/load.
+
+### `ClybAdaptiveLoraLoader`
+*File: `clyb_ModelLoader.py`*
+*Category: `loaders`*
+
+Apply up to 20 LoRAs to a `(MODEL, CLIP)` pair, in order, by cloning the model once and merging all patches into that single clone. The clone-per-call approach is cheaper than the per-LoRA clone done by ComfyUI's built-in chain loader. LoRA file contents are cached in `self.loaded_loras` keyed by slot index — the cache is invalidated when a different LoRA is selected for that slot.
+
+| Input | Type | Default | Range | Description |
+|---|---|---|---|---|
+| `model` | MODEL | — | — | Diffusion model to patch |
+| `clip` | CLIP | — | — | CLIP model to patch |
+| `lora_name_1` | ENUM | — | loras list | First LoRA (set to `"none"` to skip) |
+| `strength_model_1` | FLOAT | 1.0 | -100, 100 | Diffusion-model strength (negative allowed) |
+| `strength_clip_1` | FLOAT | 1.0 | -100, 100 | CLIP strength (negative allowed) |
+| `lora_name_2..20` | ENUM | `"none"` | loras list | Additional LoRAs (revealed as you fill slots) |
+| `strength_model_2..20` | FLOAT | 1.0 | -100, 100 | Per-slot diffusion-model strength |
+| `strength_clip_2..20` | FLOAT | 1.0 | -100, 100 | Per-slot CLIP strength |
+
+**Returns:** `MODEL`, `CLIP`.
+
+### `ClybAdaptiveLoraLoaderModelOnly`
+*File: `clyb_ModelLoader.py`*
+*Category: `loaders`*
+
+Same as `ClybAdaptiveLoraLoader`, but the `CLIP` input is omitted from the schema and the internal `strength_clip_*` is forced to `0.0` for every slot, leaving only the diffusion-model patches applied. Use this for `MODEL`-only pipelines (e.g. unconditional sampling, flows without a text encoder, or cases where CLIP is wired in separately).
+
+| Input | Type | Default | Range | Description |
+|---|---|---|---|---|
+| `model` | MODEL | — | — | Diffusion model to patch |
+| `lora_name_1` | ENUM | — | loras list | First LoRA (set to `"none"` to skip) |
+| `strength_model_1` | FLOAT | 1.0 | -100, 100 | Diffusion-model strength (negative allowed) |
+| `lora_name_2..20` | ENUM | `"none"` | loras list | Additional LoRAs (revealed as you fill slots) |
+| `strength_model_2..20` | FLOAT | 1.0 | -100, 100 | Per-slot diffusion-model strength |
+
+**Returns:** `MODEL`.
+
+---
+
+## Project layout
+
+```
+ComfyUI-ClybsChromaNodes/
+├── __init__.py                  # Entry point: imports modules, registers nodes, exposes WEB_DIRECTORY
+├── pyproject.toml               # Package metadata (v1.0.5)
+├── LICENSE                      # Apache License 2.0
+├── js/
+│   └── clyb_adaptive_lora.js    # Frontend extension for the dynamic LoRA widget behavior
+├── chroma_NAG.py                # ChromaNAG class (currently unregistered)
+├── clyb_Guidance.py             # ClybGuidance model patch
+├── clyb_Samplers.py             # clyb_bdf, taylor_flow samplers + cfgpp wrapper + 3 node classes
+├── clyb_Schedulers.py           # InverseSquaredScheduler, PrintSigmas + scheduler registration
+└── clyb_ModelLoader.py          # ClybAdaptiveLoraLoader, ClybAdaptiveLoraLoaderModelOnly
+```
+
+## License
+
+This project is licensed under the [Apache License 2.0](LICENSE).
+
+## Repository
+
+[https://github.com/Clybius/ComfyUI-ClybsChromaNodes](https://github.com/Clybius/ComfyUI-ClybsChromaNodes)

__init__.py

@@ -2,19 +2,40 @@ from . import chroma_NAG
 from . import clyb_Guidance
 from . import clyb_Samplers
 from . import clyb_Schedulers
+from . import clyb_ModelLoader
 
 clyb_Samplers.add_samplers()
 
 NODE_CLASS_MAPPINGS = {
+    # Guidance
     "ClybGuidance": clyb_Guidance.ClybGuidance,
+    # Samplers
     "SamplerClyb_BDF": clyb_Samplers.SamplerClyb_BDF,
+    "SamplerTaylorFlow": clyb_Samplers.SamplerTaylorFlow,
+    "SamplerClyb_GeomExtrap": clyb_Samplers.SamplerClyb_GeomExtrap,
+    "SamplerWrapperCFGPP": clyb_Samplers.SamplerWrapperCFGPP,
+    # Schedulers
     "InverseSquaredScheduler": clyb_Schedulers.InverseSquaredScheduler,
     "PrintSigmas": clyb_Schedulers.PrintSigmas,
+    # LoraLoaders
+    "ClybAdaptiveLoraLoader": clyb_ModelLoader.ClybAdaptiveLoraLoader,
+    "ClybAdaptiveLoraLoaderModelOnly": clyb_ModelLoader.ClybAdaptiveLoraLoaderModelOnly,
 }
 
 NODE_DISPLAY_NAME_MAPPINGS = {
+    # Guidance
     "ClybGuidance": "ClybGuidance",
+    # Samplers
     "SamplerClyb_BDF": "SamplerClyb_BDF",
+    "SamplerTaylorFlow": "SamplerTaylorFlow",
+    "SamplerClyb_GeomExtrap": "SamplerClyb_GeomExtrap",
+    "SamplerWrapperCFGPP": "SamplerWrapperCFGPP",
+    # Schedulers
     "InverseSquaredScheduler": "InverseSquaredScheduler",
     "PrintSigmas": "PrintSigmas",
-}
+    # LoraLoaders
+    "ClybAdaptiveLoraLoader": "ClybAdaptiveLoraLoader",
+    "ClybAdaptiveLoraLoaderModelOnly": "ClybAdaptiveLoraLoaderModelOnly",
+}
+
+WEB_DIRECTORY = "./js"

clyb_Guidance.py

@@ -203,9 +203,14 @@ class ClybGuidance:
 
             # 1. Move to Frequency domain using 2D Fast Fourier Transform
             # We use norm='ortho' to ensure the transform is unitary and preserves energy.
-            fft_cond = torch.fft.fftshift(torch.fft.fftn(cond.to(torch.float64), norm='ortho'))
-            fft_uncond = torch.fft.fftshift(torch.fft.fftn(uncond.to(torch.float64), norm='ortho'))
-            fft_out = torch.fft.fftshift(torch.fft.fftn(out.to(torch.float64), norm='ortho'))
+            device = cond.device
+            is_mps = device.type == "mps" if isinstance(device, torch.device) else "mps" in str(device)
+            precision_dtype = torch.float32 if is_mps else torch.float64
+            complex_dtype = torch.cfloat if is_mps else torch.cdouble
+
+            fft_cond = torch.fft.fftshift(torch.fft.fftn(cond.to(precision_dtype), norm='ortho'))
+            fft_uncond = torch.fft.fftshift(torch.fft.fftn(uncond.to(precision_dtype), norm='ortho'))
+            fft_out = torch.fft.fftshift(torch.fft.fftn(out.to(precision_dtype), norm='ortho'))
 
             # 1. Create the 2D Hann window kernel for convolution
             hann_1d = torch.signal.windows.hann(5, device=cond.device)
@@ -226,7 +231,7 @@ class ClybGuidance:
             fft_cond_real = fft_cond_flat.real
             fft_uncond_real = fft_uncond_flat.real
             #guidance_direction = (cond - uncond)
-            local_avg_magnitude = F.conv1d(fft_cond_real, kernel.to(torch.float64), padding='same')
+            local_avg_magnitude = F.conv1d(fft_cond_real, kernel.to(precision_dtype), padding='same')
 
             # 3. Normalize the magnitude map for each image in the batch to the [0, 1] range
             # This makes the `strength` parameter behave consistently across different images.
@@ -250,7 +255,7 @@ class ClybGuidance:
 
             print(local_scale)
 
-            guided_tensor = fft_cond + (local_scale.to(torch.cdouble) * (fft_cond_flat - fft_uncond_flat)).view(fft_cond.shape)
+            guided_tensor = fft_cond + (local_scale.to(complex_dtype) * (fft_cond_flat - fft_uncond_flat)).view(fft_cond.shape)
 
             guided_tensor = torch.fft.ifftshift(guided_tensor)
             guided_tensor = torch.fft.ifftn(guided_tensor, norm='ortho').real

clyb_ModelLoader.py

@@ -0,0 +1,125 @@
+import torch
+import logging
+import comfy.sd
+import folder_paths
+import comfy.utils
+import comfy.lora
+import comfy.lora_convert
+
+
+class ClybAdaptiveLoraLoader:
+    def __init__(self):
+        self.loaded_loras = {}
+
+    @classmethod
+    def INPUT_TYPES(s):
+        file_list = folder_paths.get_filename_list("loras")
+        file_list.insert(0, "none")
+
+        inputs = {
+            "required": {
+                "model": ("MODEL", {"tooltip": "The diffusion model the LoRA will be applied to."}),
+                "clip": ("CLIP", {"tooltip": "The CLIP model the LoRA will be applied to."}),
+                "lora_name_1": (file_list, {"tooltip": "The name of the LoRA."}),
+                "strength_model_1": ("FLOAT", {"default": 1.0, "min": -100.0, "max": 100.0, "step": 0.01, "tooltip": "How strongly to modify the diffusion model. This value can be negative."}),
+                "strength_clip_1": ("FLOAT", {"default": 1.0, "min": -100.0, "max": 100.0, "step": 0.01, "tooltip": "How strongly to modify the CLIP model. This value can be negative."}),
+            },
+            "optional": {}
+        }
+
+        for i in range(2, 21):
+            inputs["optional"][f"lora_name_{i}"] = (file_list, {"default": "none"})
+            inputs["optional"][f"strength_model_{i}"] = ("FLOAT", {"default": 1.0, "min": -100.0, "max": 100.0, "step": 0.01})
+            inputs["optional"][f"strength_clip_{i}"] = ("FLOAT", {"default": 1.0, "min": -100.0, "max": 100.0, "step": 0.01})
+
+        return inputs
+
+    RETURN_TYPES = ("MODEL", "CLIP")
+    OUTPUT_TOOLTIPS = ("The modified diffusion model.", "The modified CLIP model.")
+    FUNCTION = "load_lora"
+
+    CATEGORY = "loaders"
+    DESCRIPTION = "Apply multiple LoRAs adaptively by merging patches into a single model clone."
+    EXPERIMENTAL = True
+
+    def load_lora(self, model, clip, **kwargs):
+        lora_keys = [k for k in kwargs.keys() if k.startswith("lora_name_")]
+        try:
+            lora_keys.sort(key=lambda x: int(x.split("_")[-1]))
+        except:
+            pass
+
+        model_lora = model.clone() if model is not None else None
+        clip_lora = clip.clone() if clip is not None else None
+
+        for k in lora_keys:
+            lora_name = kwargs[k]
+            if lora_name == "none":
+                continue
+
+            idx = k.split("_")[-1]
+            strength_model = kwargs.get(f"strength_model_{idx}", 1.0)
+            strength_clip = kwargs.get(f"strength_clip_{idx}", 1.0)
+
+            if strength_model == 0 and strength_clip == 0:
+                continue
+
+            lora_path = folder_paths.get_full_path_or_raise("loras", lora_name)
+            lora = None
+            if idx in self.loaded_loras:
+                if self.loaded_loras[idx][0] == lora_path:
+                    lora = self.loaded_loras[idx][1]
+                else:
+                    self.loaded_loras.pop(idx, None)
+
+            if lora is None:
+                lora = comfy.utils.load_torch_file(lora_path, safe_load=True)
+                self.loaded_loras[idx] = (lora_path, lora)
+
+            key_map = {}
+            if model_lora is not None:
+                key_map = comfy.lora.model_lora_keys_unet(model_lora.model, key_map)
+            if clip_lora is not None:
+                key_map = comfy.lora.model_lora_keys_clip(clip_lora.cond_stage_model, key_map)
+
+            lora_converted = comfy.lora_convert.convert_lora(lora)
+            loaded = comfy.lora.load_lora(lora_converted, key_map)
+
+            if model_lora is not None:
+                model_lora.add_patches(loaded, strength_model)
+            if clip_lora is not None:
+                clip_lora.add_patches(loaded, strength_clip)
+
+        return (model_lora, clip_lora)
+
+
+class ClybAdaptiveLoraLoaderModelOnly(ClybAdaptiveLoraLoader):
+    @classmethod
+    def INPUT_TYPES(s):
+        file_list = folder_paths.get_filename_list("loras")
+        file_list.insert(0, "none")
+        inputs = {
+            "required": {
+                "model": ("MODEL",),
+                "lora_name_1": (file_list, ),
+                "strength_model_1": ("FLOAT", {"default": 1.0, "min": -100.0, "max": 100.0, "step": 0.01}),
+            },
+            "optional": {}
+        }
+        for i in range(2, 21):
+            inputs["optional"][f"lora_name_{i}"] = (file_list, {"default": "none"})
+            inputs["optional"][f"strength_model_{i}"] = ("FLOAT", {"default": 1.0, "min": -100.0, "max": 100.0, "step": 0.01})
+
+        return inputs
+
+    RETURN_TYPES = ("MODEL",)
+    FUNCTION = "load_lora_model_only"
+
+    def load_lora_model_only(self, model, **kwargs):
+        new_kwargs = kwargs.copy()
+        lora_keys = [k for k in kwargs.keys() if k.startswith("lora_name_")]
+        for k in lora_keys:
+            idx = k.split("_")[-1]
+            new_kwargs[f"strength_clip_{idx}"] = 0.0
+
+        return (self.load_lora(model, None, **new_kwargs)[0],)

clyb_Samplers.py

@@ -1,9 +1,15 @@
+import collections
 import math
 
 import torch
 from tqdm.auto import trange
 
-from comfy.k_diffusion.sampling import default_noise_sampler
+import comfy.model_patcher
+from comfy.k_diffusion.sampling import (
+    default_noise_sampler,
+    get_ancestral_step,
+    sigma_to_half_log_snr,
+)
 import comfy.samplers
 
 @torch.no_grad()
@@ -78,11 +84,724 @@ def sample_clyb_bdf(model, x, sigmas, extra_args=None, callback=None, disable=No
         flow = True
     return sampler_clyb_bdf(model, x, sigmas, extra_args=extra_args, callback=callback, disable=disable, scalar=scalar, eta=eta, s_noise=s_noise, noise_sampler=noise_sampler, flow=flow)
 
+# =============================================================================
+# TAYLOR FLOW SAMPLER - Multi-step sampler using Taylor expansion on
+# previous denoised predictions to approximate higher-order derivatives.
+# Based on "Leveraging Previous Steps" (Nov 2024).
+# =============================================================================
+
+
+def _construct_vandermonde_flow(history, sigma_ref, max_order, device, dtype):
+    """
+    Build Vandermonde matrix R_p for polynomial interpolation.
+
+    R_p[m, i] = (sigma_{n-1-m} - sigma_ref)^i
+
+    Args:
+        history: list of (sigma, denoised) tuples (oldest to newest)
+        sigma_ref: reference sigma (current timestep t_{n-1})
+        max_order: maximum polynomial degree (number of previous steps to use)
+        device: torch device
+        dtype: torch dtype (float64 for numerical stability)
+
+    Returns:
+        R: (k, k) Vandermonde matrix where k = min(len(history), max_order)
+    """
+    k = min(len(history), max_order)
+
+    # Use most recent k points from history
+    recent_history = list(history)[-k:]
+
+    # Build matrix: R[m, i] = (sigma_m - sigma_ref)^i
+    R = torch.zeros((k, k), device=device, dtype=dtype)
+
+    for m, (sigma_m, _) in enumerate(reversed(recent_history)):
+        h_m = sigma_m - sigma_ref  # Time difference (negative for past points)
+
+        for i in range(k):
+            R[m, i] = h_m**i
+
+    return R
+
+
+def _solve_flow_coefficients(R, h_n, method="diagonal", diag_weight=1.0):
+    """
+    Solve for B coefficients using either two-sided equilibration or diagonal-dominant regularization.
+
+    Args:
+        R: Vandermonde matrix (k, k)
+        h_n: step size (sigma_next - sigma_cur)
+        method: "equilibration" (default) or "diagonal"
+        diag_weight: Weight for diagonal component (0.0 to 1.0, default 1.0 for original behavior)
+
+    Returns:
+        B: coefficient vector (k,)
+    """
+    k = R.shape[0]
+    device = R.device
+    dtype = R.dtype
+
+    # Handle edge cases
+    if k == 0:
+        return torch.tensor([], device=device, dtype=dtype)
+    if k == 1:
+        # Simple case: just use the diagonal element
+        return torch.tensor([h_n], device=device, dtype=dtype)
+
+    # Compute C vector: C_i = h_n^{i+1} / (i+1)
+    C = torch.zeros(k, device=device, dtype=dtype)
+    for i in range(k):
+        C[i] = (h_n ** (i + 1)) / (i + 1)
+
+    if method == "equilibration":
+        # Iteratively balance row and column norms to equilibrate the matrix
+        D = torch.eye(k, device=device, dtype=dtype)
+        E = torch.eye(k, device=device, dtype=dtype)
+        R_work = R.clone()
+
+        for _ in range(5):  # 5 iterations typically sufficient for convergence
+            # Row scaling: normalize rows to unit infinity-norm
+            row_norms = torch.norm(R_work, dim=1, p=float('inf'))
+            D_scale = torch.diag(1.0 / torch.sqrt(row_norms + 1e-10))
+            R_work = D_scale @ R_work
+            D = D_scale @ D
+
+            # Column scaling: normalize columns to unit infinity-norm
+            col_norms = torch.norm(R_work, dim=0, p=float('inf'))
+            E_scale = torch.diag(1.0 / torch.sqrt(col_norms + 1e-10))
+            R_work = R_work @ E_scale
+            E = E @ E_scale
+
+        # Apply row scaling to C
+        C_eq = D @ C
+
+        # Minimal Tikhonov regularization on equilibrated system
+        lambda_reg = 0.0001
+        R_reg = R_work + lambda_reg * torch.eye(k, device=device, dtype=dtype)
+
+        # Solve and unscale
+        try:
+            B_eq = torch.linalg.solve(R_reg, C_eq)
+            B = E @ B_eq
+        except torch.linalg.LinAlgError:
+            B_eq = torch.linalg.lstsq(R_reg, C_eq, rcond=1e-10).solution
+            B = E @ B_eq
+
+    elif method == "diagonal":
+        # Diagonal-Dominant Extraction
+        R_diag = torch.diag(torch.diag(R))
+        R_weighted = diag_weight * R_diag + (1.0 - diag_weight) * R
+
+        lambda_reg = 1e-16
+        R_reg = R_weighted  # + lambda_reg * torch.eye(k, device=device, dtype=dtype)
+
+        try:
+            B = torch.linalg.solve(R_reg, C)
+        except torch.linalg.LinAlgError:
+            B = torch.linalg.lstsq(R_reg, C, rcond=1e-10).solution
+
+    else:
+        # Unknown method: fallback to equilibration
+        return _solve_flow_coefficients(R, h_n, method="equilibration", diag_weight=diag_weight)
+
+    return B
+
+
+@torch.no_grad()
+def sampler_taylor_flow(
+    model,
+    x,
+    sigmas,
+    extra_args=None,
+    callback=None,
+    disable=None,
+    order=8,
+    eta=1.0,
+    s_noise=1.0,
+    noise_sampler=None,
+    flow=False,
+    sigma_calc="clyb",
+):
+    """
+    Taylor Flow sampler - Multi-step sampler using Taylor expansion on previous
+    denoised predictions to approximate higher-order derivatives.
+
+    Based on "Leveraging Previous Steps: A Training-free Fast Solver for
+    Flow Diffusion" (Nov 2024). Achieves O(h^p) approximation error with only
+    1 function evaluation per step by reusing cached historical predictions.
+
+    Args:
+        model: Diffusion model
+        x: Initial latent
+        sigmas: Sigma schedule
+        extra_args: Extra arguments for model
+        callback: Progress callback
+        disable: Disable progress bar
+        order: Taylor expansion order (1-16). Higher = more accurate but uses more history
+        eta: Ancestral sampling eta (stochasticity)
+        s_noise: Noise scale
+        noise_sampler: Noise sampler function
+        flow: Whether using flow-based model (FLUX, SD3, Chroma)
+        sigma_calc: Ancestral sigma calculation method ("clyb", "taylor-expansion", "ancestral", "adaptive")
+
+    Returns:
+        Denoised latent tensor
+    """
+    extra_args = {} if extra_args is None else extra_args
+    seed = extra_args.get("seed", None)
+    noise_sampler = (
+        default_noise_sampler(x, seed=seed) if noise_sampler is None else noise_sampler
+    )
+    s_in = x.new_ones([x.shape[0]])
+    device = x.device
+
+    if len(sigmas) <= 1:
+        return x
+
+    # Rolling history buffer for (sigma, denoised) pairs
+    history = collections.deque(maxlen=order)
+
+    for i in trange(len(sigmas) - 1, disable=disable):
+        sigma_cur, sigma_next = sigmas[i], sigmas[i + 1]
+        h_n = sigma_next - sigma_cur  # Step size
+
+        # Ancestral sigma calculation - selectable method
+        if sigma_calc == "clyb":
+            # Original Clyb implementation (logarithmic scaling)
+            sigma_down = (
+                sigma_next**2 / (1 + math.log(1.0 + abs(sigma_next - sigma_cur)) * eta)
+            ) ** 0.5
+            sigma_up = (sigma_next**2 - sigma_down**2) ** 0.5
+        elif sigma_calc == "taylor-expansion":
+            # Taylor-Expansion-Matched: exponential integral with quadratic correction
+            step_ratio = abs(h_n) / max(sigma_next, 1e-8)
+            taylor_factor = math.exp(-eta * step_ratio)
+            quadratic_correction = 1 - eta * 0.5 * step_ratio ** 2
+            sigma_down = sigma_next * taylor_factor * quadratic_correction
+            sigma_up = sigma_next * max(0.0, 1 - taylor_factor**2) ** 0.5
+        elif sigma_calc == "ancestral":
+            # Standard k-diffusion ancestral step
+            sigma_down, sigma_up = get_ancestral_step(sigma_cur, sigma_next, eta)
+        elif sigma_calc == "adaptive":
+            window_size = min(order, len(history))
+            if window_size >= 2:
+                history_list = list(history)
+                recent = history_list[-window_size:]
+                denoised_list = [d.float() for _, d in recent]
+                stacked = torch.stack(denoised_list)
+                mean_d = stacked.mean(dim=0)
+                var_val = ((stacked - mean_d) ** 2).mean().item()
+                norm_val = mean_d.pow(2).mean().item()
+                eps = 1e-8
+                if math.isfinite(var_val) and math.isfinite(norm_val):
+                    normalized_metric = var_val / (var_val + abs(norm_val) + eps)
+                    normalized_metric = min(1.0, max(0.0, normalized_metric))
+                else:
+                    normalized_metric = 0.0
+                sigma_down = sigma_next * (1.0 - eta * normalized_metric)
+                sigma_down = max(0.0, min(sigma_next, sigma_down))
+                sigma_up = math.sqrt(max(0.0, sigma_next**2 - sigma_down**2))
+            else:
+                sigma_down = (
+                    sigma_next**2 / (1 + math.log(1.0 + abs(sigma_next - sigma_cur)) * eta)
+                ) ** 0.5
+                sigma_up = (sigma_next**2 - sigma_down**2) ** 0.5
+        else:
+            # Default to clyb if unknown method
+            sigma_down = (
+                sigma_next**2 / (1 + math.log(1.0 + abs(sigma_next - sigma_cur)) * eta)
+            ) ** 0.5
+            sigma_up = (sigma_next**2 - sigma_down**2) ** 0.5
+
+        # Flow model coefficients
+        alpha_ip1 = None
+        alpha_down = None
+        renoise_coeff = None
+        alpha_ratio = 1.0
+        if flow:
+            alpha_ip1 = 1 - sigma_next
+            alpha_down = 1 - sigma_down
+            renoise_coeff = (
+                sigma_next**2 - sigma_down**2 * alpha_ip1**2 / alpha_down**2
+            ) ** 0.5
+            alpha_ratio = alpha_ip1 / alpha_down if alpha_down != 0 else 1.0
+
+        # =========================================================================
+        # TAYLOR EXPANSION PHASE (main algorithm)
+        # =========================================================================
+        # 1. Single model evaluation at current state
+        denoised_cur = model(x, sigma_cur * s_in, **extra_args)
+
+        # 2. Build Vandermonde matrix from historical timesteps
+        precision_dtype = torch.float32 if (device.type == "mps" if isinstance(device, torch.device) else "mps" in str(device)) else torch.float64
+        R_p = _construct_vandermonde_flow(
+            history, sigma_cur, order, device, precision_dtype
+        )
+
+        # 3. Solve for B coefficients
+        B = _solve_flow_coefficients(R_p, h_n)
+        B = B.to(dtype=x.dtype)
+
+        # 4. Compute D_m differences: D_m = v_history[m] - v_current
+        D_list = []
+        for _, denoised_prev in reversed(list(history)[-len(B) :]):
+            D_m = denoised_prev - denoised_cur
+            D_list.append(D_m)
+
+        # 5. Predictor step: x_pred = Euler + sum(B_m * D_m)
+        w_next = 1.0 - sigma_down / sigma_cur
+        euler_step = x.lerp(denoised_cur, weight=w_next)
+
+        if len(D_list) > 0:
+            correction = sum(B[m] * D_list[m] for m in range(len(D_list)))
+        else:
+            correction = 0
+
+        x_next = euler_step + correction
+
+        # 6. Update rolling history
+        history.append((sigma_cur, denoised_cur))
+
+        x = x_next
+
+        # =========================================================================
+        # ANCESTRAL NOISE INJECTION
+        # =========================================================================
+        if sigma_next > 0 and eta > 0:
+            noise = noise_sampler(sigma_cur, sigma_next) * s_noise
+            if flow:
+                x = alpha_ratio * x + noise * renoise_coeff
+            else:
+                x = x + noise * sigma_up
+
+        # Callback
+        if callback is not None:
+            callback(
+                {
+                    "x": x,
+                    "i": i,
+                    "sigma": sigma_cur,
+                    "sigma_hat": sigma_cur,
+                    "denoised": denoised_cur,
+                }
+            )
+
+    return x
+
+
+@torch.no_grad()
+def sample_taylor_flow(
+    model,
+    x,
+    sigmas,
+    extra_args=None,
+    callback=None,
+    disable=None,
+    order=8,
+    eta=1.0,
+    s_noise=1.0,
+    noise_sampler=None,
+    sigma_calc="clyb",
+):
+    """Wrapper with flow model detection."""
+    flow = False
+    if isinstance(
+        model.inner_model.inner_model.model_sampling, comfy.model_sampling.CONST
+    ):
+        flow = True
+    return sampler_taylor_flow(
+        model,
+        x,
+        sigmas,
+        extra_args=extra_args,
+        callback=callback,
+        disable=disable,
+        order=order,
+        eta=eta,
+        s_noise=s_noise,
+        noise_sampler=noise_sampler,
+        flow=flow,
+        sigma_calc=sigma_calc,
+    )
+
+
+# =============================================================================
+# CFG++ SAMPLER WRAPPER - Captures uncond_denoised via post-CFG hook and
+# recomputes a CFG++-style denoised using sigma_to_half_log_snr. Wraps any
+# inner SAMPLER (KSampler-style).
+# =============================================================================
+
+
+class CFGPPProxyModel:
+    def __init__(self, model, sigmas):
+        self.model = model
+        self.sigmas = sigmas
+        self.uncond_denoised = None
+
+    def __call__(self, x, sigma, **kwargs):
+        model_options = kwargs.get("model_options", {}).copy()
+
+        def post_cfg_function(args):
+            self.uncond_denoised = args["uncond_denoised"]
+            return args["denoised"]
+
+        kwargs["model_options"] = comfy.model_patcher.set_model_options_post_cfg_function(
+            model_options, post_cfg_function, disable_cfg1_optimization=True
+        )
+
+        denoised_guided = self.model(x, sigma, **kwargs)
+
+        if self.uncond_denoised is None:
+            return denoised_guided
+
+        sigma_val = sigma.flatten()[0].item()
+        idx = (self.sigmas - sigma_val).abs().argmin().item()
+        sigma_next_val = float(self.sigmas[idx + 1]) if idx + 1 < len(self.sigmas) else 0.0
+
+        if sigma_next_val == 0:
+            return denoised_guided
+
+        model_sampling = self.model.inner_model.model_patcher.get_model_object("model_sampling")
+        lambda_fn = lambda s: sigma_to_half_log_snr(s, model_sampling)
+
+        alpha_s = sigma_val * lambda_fn(torch.tensor(sigma_val)).exp().item()
+        alpha_t = sigma_next_val * lambda_fn(torch.tensor(sigma_next_val)).exp().item()
+
+        denoised_star = (
+            sigma_val * alpha_t * denoised_guided
+            - sigma_next_val * alpha_s * self.uncond_denoised
+        ) / (sigma_val - sigma_next_val)
+
+        return denoised_star
+
+    def __getattr__(self, name):
+        return getattr(self.model, name)
+
+
+@torch.no_grad()
+def sample_cfgpp(model, x, sigmas, extra_args=None, callback=None, disable=None,
+                 sampler=None):
+    extra_args = {} if extra_args is None else extra_args
+    proxy = CFGPPProxyModel(model, sigmas)
+    return sampler.sampler_function(
+        proxy, x, sigmas,
+        extra_args=extra_args,
+        callback=callback,
+        disable=disable,
+        **sampler.extra_options,
+    )
+
+
+# =============================================================================
+# GEOM_EXTRAP SAMPLER - 3-NFE per step sampler that uses two geometric
+# midpoints between sigma_cur and sigma_down to approximate a higher-order
+# denoised prediction, then integrates that prediction into the noisy x
+# latent. Uses the same selectable sigma_calc branches as SamplerTaylorFlow.
+# =============================================================================
+
+
+def _compute_ancestral_sigmas(sigma_cur, sigma_next, eta, sigma_calc, history, order):
+    """
+    Compute (sigma_down, sigma_up) for the requested sigma_calc method.
+    Replicates the four branches from sampler_taylor_flow (clyb_Samplers.py:268-314)
+    verbatim so behaviour matches that sampler.
+    """
+    h_n = sigma_next - sigma_cur
+    if sigma_calc == "clyb":
+        sigma_down = (
+            sigma_next ** 2 / (1.0 + math.log(1.0 + abs(sigma_next - sigma_cur)) * eta)
+        ) ** 0.5
+        sigma_up = (sigma_next ** 2 - sigma_down ** 2) ** 0.5
+    elif sigma_calc == "taylor-expansion":
+        step_ratio = abs(h_n) / max(sigma_next, 1e-8)
+        taylor_factor = math.exp(-eta * step_ratio)
+        quadratic_correction = 1.0 - eta * 0.5 * step_ratio ** 2
+        sigma_down = sigma_next * taylor_factor * quadratic_correction
+        sigma_up = sigma_next * max(0.0, 1.0 - taylor_factor ** 2) ** 0.5
+    elif sigma_calc == "ancestral":
+        sigma_down, sigma_up = get_ancestral_step(sigma_cur, sigma_next, eta)
+    elif sigma_calc == "adaptive":
+        window_size = min(order, len(history))
+        if window_size >= 2:
+            history_list = list(history)
+            recent = history_list[-window_size:]
+            denoised_list = [d.float() for _, d in recent]
+            stacked = torch.stack(denoised_list)
+            mean_d = stacked.mean(dim=0)
+            var_val = ((stacked - mean_d) ** 2).mean().item()
+            norm_val = mean_d.pow(2).mean().item()
+            eps = 1e-8
+            if math.isfinite(var_val) and math.isfinite(norm_val):
+                normalized_metric = var_val / (var_val + abs(norm_val) + eps)
+                normalized_metric = min(1.0, max(0.0, normalized_metric))
+            else:
+                normalized_metric = 0.0
+            sigma_down = sigma_next * (1.0 - eta * normalized_metric)
+            sigma_down = max(0.0, min(sigma_next, sigma_down))
+            sigma_up = math.sqrt(max(0.0, sigma_next ** 2 - sigma_down ** 2))
+        else:
+            sigma_down = (
+                sigma_next ** 2
+                / (1.0 + math.log(1.0 + abs(sigma_next - sigma_cur)) * eta)
+            ) ** 0.5
+            sigma_up = (sigma_next ** 2 - sigma_down ** 2) ** 0.5
+    else:
+        sigma_down = (
+            sigma_next ** 2 / (1.0 + math.log(1.0 + abs(sigma_next - sigma_cur)) * eta)
+        ) ** 0.5
+        sigma_up = (sigma_next ** 2 - sigma_down ** 2) ** 0.5
+    return sigma_down, sigma_up
+
+
+@torch.no_grad()
+def sampler_geom_extrap(
+    model,
+    x,
+    sigmas,
+    extra_args=None,
+    callback=None,
+    disable=None,
+    eta=1.0,
+    s_noise=1.0,
+    noise_sampler=None,
+    flow=False,
+    sigma_calc="clyb",
+):
+    """
+    Geometric-Midpoint Extrapolation sampler.
+
+    Per-step procedure (3 NFEs per step):
+      1. Sample at sigma_cur.
+      2. Integrate into x_mid1 at sigma_gm1 = sqrt(sigma_cur * sigma_down).
+      3. Sample at sigma_gm1.
+      4. Linearly extrapolate through (denoised1, denoised2) to predict at sigma_down.
+      5. Integrate denoised_pred into x_mid2 at sigma_gm2 = sqrt(sigma_gm1 * sigma_down).
+      6. Sample at sigma_gm2.
+      7. If not the final step, do a quadratic (3-point divided-difference) extrapolation
+         through (denoised1, denoised2, denoised3) to predict at sigma_down and integrate
+         that into x. If the final step, integrate denoised3 directly into x.
+
+    All intermediate sigmas (sigma_gm1, sigma_gm2) and lerp weight denominators are
+    clamped to >= 1e-4 for numerical stability near sigma = 0.
+    """
+    extra_args = {} if extra_args is None else extra_args
+    seed = extra_args.get("seed", None)
+    noise_sampler = (
+        default_noise_sampler(x, seed=seed) if noise_sampler is None else noise_sampler
+    )
+    s_in = x.new_ones([x.shape[0]])
+
+    if len(sigmas) <= 1:
+        return x
+
+    # History buffer is needed for the "adaptive" sigma_calc branch.
+    history = collections.deque(maxlen=16)
+    # The last iteration index is len(sigmas) - 2 (since the loop goes 0..len(sigmas)-2,
+    # and sigmas[-1] is 0). On that step sigma_next == 0 and sigma_down == 0, so the
+    # 3-point extrapolation degenerates (denominators collapse). We skip extrapolation
+    # there and use denoised3 directly.
+    is_final_step = len(sigmas) - 2 if len(sigmas) >= 2 else 0
+
+    for i in trange(len(sigmas) - 1, disable=disable):
+        sigma_cur, sigma_next = sigmas[i], sigmas[i + 1]
+        h_n = sigma_next - sigma_cur
+
+        # ---- Ancestral sigma_down / sigma_up via the same branches as taylor_flow ----
+        sigma_down, sigma_up = _compute_ancestral_sigmas(
+            sigma_cur, sigma_next, eta, sigma_calc, history, order=16
+        )
+
+        # ---- Flow model coefficients (identical to sampler_taylor_flow lines 316-327) ----
+        alpha_ip1 = None
+        alpha_down = None
+        renoise_coeff = None
+        alpha_ratio = 1.0
+        if flow:
+            alpha_ip1 = 1.0 - sigma_next
+            alpha_down = 1.0 - sigma_down
+            renoise_coeff = (
+                sigma_next ** 2 - sigma_down ** 2 * alpha_ip1 ** 2 / alpha_down ** 2
+            ) ** 0.5
+            alpha_ratio = alpha_ip1 / alpha_down if alpha_down != 0 else 1.0
+
+        # ---- Geometric midpoints, clamped for numerical stability ----
+        sigma_gm1 = (sigma_cur * sigma_down).clamp_min(1e-4).sqrt()
+        sigma_gm2 = (sigma_gm1 * sigma_down).clamp_min(1e-4).sqrt()
+
+        # ---- NFE 1: sample at sigma_cur ----
+        denoised1 = model(x, sigma_cur * s_in, **extra_args)
+
+        # ---- Integrate denoised1 into a noisy latent at sigma_gm1 ----
+        w_gm1 = (sigma_gm1 / sigma_cur).clamp_min(1e-4)
+        x_mid1 = denoised1.lerp(x, weight=w_gm1)
+
+        # ---- NFE 2: sample at sigma_gm1 ----
+        denoised2 = model(x_mid1, sigma_gm1 * s_in, **extra_args)
+
+        # ---- Linear extrapolation through (denoised1, denoised2) to predict at sigma_down ----
+        slope_12 = (denoised2 - denoised1) / (sigma_gm1 - sigma_cur)
+        denoised_pred = denoised2 + slope_12 * (sigma_down - sigma_gm1)
+
+        # ---- Integrate denoised_pred into a noisy latent at sigma_gm2 (from original x) ----
+        w_gm2 = (sigma_gm2 / sigma_gm1).clamp_min(1e-4)
+        x_mid2 = denoised_pred.lerp(x_mid1, weight=w_gm2)
+
+        # ---- NFE 3: sample at sigma_gm2 ----
+        denoised3 = model(x_mid2, sigma_gm2 * s_in, **extra_args)
+
+        # ---- Final vs non-final step ----
+        if i < is_final_step:
+            # Quadratic (3-point) extrapolation via Newton divided differences
+            d1 = (denoised2 - denoised1) / (sigma_gm1 - sigma_cur)
+            d2 = (denoised3 - denoised2) / (sigma_gm2 - sigma_gm1)
+            d2_div = (d2 - d1) / (sigma_gm2 - sigma_cur)
+            denoised_final = (
+                denoised3
+                + d2 * (sigma_down - sigma_gm2)
+                + d2_div * (sigma_down - sigma_gm2) * (sigma_down - sigma_gm1)
+            )
+        else:
+            # Final step: sigma_down = 0, just use denoised3 directly.
+            denoised_final = denoised3
+
+        # ---- Integrate the final denoised prediction into x at sigma_down ----
+        w_down = (sigma_down / sigma_gm2).clamp_min(1e-4)
+        x = denoised_final.lerp(x_mid2, weight=w_down)
+
+        # ---- Update history for "adaptive" sigma_calc branch on subsequent steps ----
+        history.append((sigma_cur, denoised_final))
+
+        # ---- Ancestral noise injection (identical to sampler_taylor_flow lines 369-374) ----
+        if sigma_next > 0 and eta > 0:
+            noise = noise_sampler(sigma_cur, sigma_next) * s_noise
+            if flow:
+                x = alpha_ratio * x + noise * renoise_coeff
+            else:
+                x = x + noise * sigma_up
+
+        # ---- Callback ----
+        if callback is not None:
+            callback(
+                {
+                    "x": x,
+                    "i": i,
+                    "sigma": sigma_cur,
+                    "sigma_hat": sigma_cur,
+                    "denoised": denoised_final,
+                }
+            )
+
+    return x
+
+
+@torch.no_grad()
+def sample_geom_extrap(
+    model,
+    x,
+    sigmas,
+    extra_args=None,
+    callback=None,
+    disable=None,
+    eta=1.0,
+    s_noise=1.0,
+    noise_sampler=None,
+    sigma_calc="clyb",
+):
+    """Wrapper that detects flow vs non-flow then calls sampler_geom_extrap."""
+    flow = False
+    if isinstance(
+        model.inner_model.inner_model.model_sampling, comfy.model_sampling.CONST
+    ):
+        flow = True
+    return sampler_geom_extrap(
+        model,
+        x,
+        sigmas,
+        extra_args=extra_args,
+        callback=callback,
+        disable=disable,
+        eta=eta,
+        s_noise=s_noise,
+        noise_sampler=noise_sampler,
+        flow=flow,
+        sigma_calc=sigma_calc,
+    )
+
+
+class SamplerClyb_GeomExtrap:
+    """
+    Geometric-Midpoint Extrapolation sampler.
+
+    3-NFE per step sampler that uses two geometric midpoints between sigma_cur and
+    sigma_down to build a 3-point quadratic extrapolation of the denoised prediction
+    at sigma_down, then integrates that prediction into the noisy x latent.
+
+    Compatible with both flow-matching (FLUX, SD3, Chroma) and non-flow models.
+
+    Parameters:
+    - eta: Ancestral sampling stochasticity (0=deterministic, 1=full stochastic)
+    - s_noise: Noise scaling factor
+    - sigma_calc: Ancestral sigma calculation method (clyb, taylor-expansion, ancestral, adaptive)
+    """
+
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": {
+                "eta": (
+                    "FLOAT",
+                    {
+                        "default": 1.0,
+                        "min": 0.0,
+                        "max": 100.0,
+                        "step": 0.01,
+                        "tooltip": "Ancestral sampling stochasticity (0=deterministic, 1=full stochastic)",
+                    },
+                ),
+                "s_noise": (
+                    "FLOAT",
+                    {
+                        "default": 1.0,
+                        "min": 0.0,
+                        "max": 100.0,
+                        "step": 0.01,
+                        "tooltip": "Noise scaling factor",
+                    },
+                ),
+                "sigma_calc": (
+                    ["clyb", "taylor-expansion", "ancestral", "adaptive"],
+                    {
+                        "default": "clyb",
+                        "tooltip": "Ancestral sigma calculation method: clyb (original log-based), taylor-expansion (exponential+quadratic), ancestral (standard k-diffusion), adaptive (history-based convergence-aware)",
+                    },
+                ),
+            }
+        }
+
+    RETURN_TYPES = ("SAMPLER",)
+    CATEGORY = "sampling/custom_sampling/samplers"
+    FUNCTION = "get_sampler"
+
+    def get_sampler(self, eta, s_noise, sigma_calc):
+        sampler = comfy.samplers.ksampler(
+            "geom_extrap",
+            {
+                "eta": eta,
+                "s_noise": s_noise,
+                "sigma_calc": sigma_calc,
+            },
+        )
+        return (sampler,)
+
+
 # The following function adds the samplers during initialization, in __init__.py
 def add_samplers():
     from comfy.samplers import KSampler, k_diffusion_sampling
     if hasattr(KSampler, "DISCARD_PENULTIMATE_SIGMA_SAMPLERS"):
         KSampler.DISCARD_PENULTIMATE_SIGMA_SAMPLERS |= discard_penultimate_sigma_samplers
+
+    # ---- Top-level samplers: registered into BOTH KSampler.SAMPLERS (dropdown)
+    # AND k_diffusion_sampling (function lookup) ----
     added = 0
     for sampler in extra_samplers: #getattr(self, "sample_{}".format(extra_samplers))
         if sampler not in KSampler.SAMPLERS:
@@ -97,8 +816,27 @@ def add_samplers():
         import importlib
         importlib.reload(k_diffusion_sampling)
 
+    # ---- Sampler wrappers: registered into k_diffusion_sampling ONLY (function
+    # lookup). They are NOT added to KSampler.SAMPLERS, so they will NOT appear
+    # in the standard KSampler node's sampler_name dropdown. They are only
+    # accessible through their dedicated wrapper node (e.g. SamplerWrapperCFGPP),
+    # which calls comfy.samplers.ksampler("<name>", {"sampler": inner_sampler}).
+    # comfy.samplers.ksampler() resolves the function via
+    # getattr(k_diffusion_sampling, "sample_<name>"), which is what we set here.
+    for name, func in extra_sampler_wrappers.items():
+        if not hasattr(k_diffusion_sampling, "sample_{}".format(name)):
+            setattr(k_diffusion_sampling, "sample_{}".format(name), func)
+
 extra_samplers = {
     "clyb_bdf": sample_clyb_bdf,
+    "taylor_flow": sample_taylor_flow,
+    "geom_extrap": sample_geom_extrap,
+}
+
+# Wrappers are NOT in the standard sampler dropdown. They are only reachable
+# via dedicated wrapper nodes (e.g. SamplerWrapperCFGPP).
+extra_sampler_wrappers = {
+    "cfgpp": sample_cfgpp,
 }
 
 discard_penultimate_sigma_samplers = set(())
@@ -120,4 +858,126 @@ class SamplerClyb_BDF:
 
     def get_sampler(self, scalar, eta, s_noise):
         sampler = comfy.samplers.ksampler("clyb_bdf", {"scalar": scalar, "eta": eta, "s_noise": s_noise})
-        return (sampler, )
+        return (sampler, )
+
+
+class SamplerTaylorFlow:
+    """
+    Taylor Flow sampler - Multi-step sampler using Taylor expansion on previous
+    denoised predictions to approximate higher-order derivatives.
+
+    Based on "Leveraging Previous Steps: A Training-free Fast Solver for
+    Flow Diffusion" (Nov 2024). Achieves O(h^p) approximation error with only
+    1 function evaluation per step by reusing cached historical predictions.
+
+    Key features:
+    - Leverages previous steps via rolling history buffer
+    - Polynomial interpolation via Vandermonde matrix
+    - Compatible with both flow models (FLUX, SD3, Chroma) and diffusion models
+    - Multiple ancestral sigma calculation methods
+
+    Parameters:
+    - order (1-16): Taylor expansion order. Higher = more accurate but uses more history
+    - eta: Ancestral sampling stochasticity (0=deterministic, 1=full stochastic)
+    - s_noise: Noise scaling factor
+    - sigma_calc: Ancestral sigma calculation method (clyb, taylor-expansion, ancestral, adaptive)
+
+    Recommended for:
+    - High-quality generation with fewer steps
+    - Flow-based models (FLUX, SD3, Chroma) with ancestral sampling
+    - Balancing speed (fewer NFEs) and quality (higher-order accuracy)
+    - Experimenting with different ancestral noise schedules
+    """
+
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": {
+                "order": (
+                    "INT",
+                    {
+                        "default": 8,
+                        "min": 1,
+                        "max": 16,
+                        "step": 1,
+                        "tooltip": "Taylor expansion order (1-16). Higher = more accurate but uses more history",
+                    },
+                ),
+                "eta": (
+                    "FLOAT",
+                    {
+                        "default": 1.0,
+                        "min": 0.0,
+                        "max": 100.0,
+                        "step": 0.01,
+                        "tooltip": "Ancestral sampling stochasticity (0=deterministic, 1=full stochastic)",
+                    },
+                ),
+                "s_noise": (
+                    "FLOAT",
+                    {
+                        "default": 1.0,
+                        "min": 0.0,
+                        "max": 100.0,
+                        "step": 0.01,
+                        "tooltip": "Noise scaling factor",
+                    },
+                ),
+                "sigma_calc": (
+                    ["clyb", "taylor-expansion", "ancestral", "adaptive"],
+                    {
+                        "default": "clyb",
+                        "tooltip": "Ancestral sigma calculation method: clyb (original log-based), taylor-expansion (exponential+quadratic), ancestral (standard k-diffusion), adaptive (history-based convergence-aware)",
+                    },
+                ),
+            }
+        }
+
+    RETURN_TYPES = ("SAMPLER",)
+    CATEGORY = "sampling/custom_sampling/samplers"
+    FUNCTION = "get_sampler"
+
+    def get_sampler(self, order, eta, s_noise, sigma_calc):
+        sampler = comfy.samplers.ksampler(
+            "taylor_flow",
+            {
+                "order": order,
+                "eta": eta,
+                "s_noise": s_noise,
+                "sigma_calc": sigma_calc,
+            },
+        )
+        return (sampler,)
+
+
+class SamplerWrapperCFGPP:
+    """
+    CFG++ Sampler Wrapper.
+
+    Wraps an inner SAMPLER and intercepts its model call via a proxy that
+    captures the uncond_denoised output through a post-CFG hook, then
+    recomputes a CFG++-style denoised. Implementation follows the standard
+    CFG++ paper formulation: the uncond output is taken from a model sampling
+    at the next sigma, and the final denoised_star is computed as
+        (sigma * alpha_t * denoised_guided
+         - sigma_next * alpha_s * uncond_denoised) / (sigma - sigma_next).
+    """
+
+    @classmethod
+    def INPUT_TYPES(s):
+        return {
+            "required": {
+                "sampler": ("SAMPLER",),
+            }
+        }
+
+    RETURN_TYPES = ("SAMPLER",)
+    CATEGORY = "sampling/custom_sampling/samplers"
+    FUNCTION = "get_sampler"
+
+    def get_sampler(self, sampler):
+        sampler = comfy.samplers.ksampler(
+            "cfgpp",
+            {"sampler": sampler},
+        )
+        return (sampler,)

js/clyb_adaptive_lora.js

@@ -0,0 +1,84 @@
+import { app } from "../../scripts/app.js";
+
+app.registerExtension({
+    name: "ClybsChromaNodes.AdaptiveLora",
+    async beforeRegisterNodeDef(nodeType, nodeData, app) {
+        if (nodeData.name === "ClybAdaptiveLoraLoader" ||
+            nodeData.name === "ClybAdaptiveLoraLoaderModelOnly") {
+
+            const onNodeCreated = nodeType.prototype.onNodeCreated;
+            nodeType.prototype.onNodeCreated = function () {
+                const r = onNodeCreated ? onNodeCreated.apply(this, arguments) : undefined;
+
+                if (!this.all_widgets) {
+                    this.all_widgets = [...this.widgets];
+                }
+
+                this.updateWidgets = function() {
+                    let lastActiveIdx = 1;
+                    for (let i = 1; i <= 20; i++) {
+                        const w = this.all_widgets.find(w => w.name === `lora_name_${i}`);
+                        if (w && w.value && w.value !== "none") {
+                            lastActiveIdx = i + 1;
+                        }
+                    }
+                    if (lastActiveIdx > 20) lastActiveIdx = 20;
+
+                    const newWidgets = [];
+                    for (let w of this.all_widgets) {
+                        const name = w.name;
+                        if (name.startsWith("lora_name_") || name.startsWith("strength_model_") || name.startsWith("strength_clip_")) {
+                            const idx = parseInt(name.split("_").pop());
+                            if (idx <= lastActiveIdx) {
+                                newWidgets.push(w);
+                            }
+                        } else {
+                            newWidgets.push(w);
+                        }
+                    }
+                    this.widgets = newWidgets;
+                    this.computeSize();
+                    app.graph.setDirtyCanvas(true, true);
+                };
+
+                for (let w of this.all_widgets) {
+                    if (w.name.startsWith("lora_name_")) {
+                        const oldCallback = w.callback;
+                        w.callback = (v) => {
+                            if (oldCallback) oldCallback.apply(this, [v]);
+                            this.updateWidgets();
+                        };
+                    }
+                }
+
+                this.updateWidgets();
+                return r;
+            };
+
+            // Ensure all widgets are serialized even if hidden
+            const onSerialize = nodeType.prototype.onSerialize;
+            nodeType.prototype.onSerialize = function(o) {
+                if (onSerialize) onSerialize.apply(this, arguments);
+                if (this.all_widgets) {
+                    o.widgets_values = this.all_widgets.map(w => w.value);
+                }
+            };
+
+            // Ensure we can load all widgets correctly
+            const onConfigure = nodeType.prototype.onConfigure;
+            nodeType.prototype.onConfigure = function(o) {
+                if (this.all_widgets && o.widgets_values) {
+                    // Pre-fill all_widgets with loaded values before updateWidgets runs
+                    for (let i = 0; i < this.all_widgets.length; i++) {
+                        if (o.widgets_values[i] !== undefined) {
+                            this.all_widgets[i].value = o.widgets_values[i];
+                        }
+                    }
+                }
+                const r = onConfigure ? onConfigure.apply(this, arguments) : undefined;
+                if (this.updateWidgets) this.updateWidgets();
+                return r;
+            };
+        }
+    }
+});

pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "clybschromanodes"
 description = "A small collection of nodes intended for use with Lodestone Rock's Chroma model, for ComfyUI."
-version = "1.0.3"
+version = "1.0.6"
 license = {file = "LICENSE"}
 
 [project.urls]