# ComfyUI-AutoGuidance Adds an **AutoGuidance + CFG** **GUIDER** node for **ComfyUI**, based on: - **Guiding a Diffusion Model with a Bad Version of Itself** (arXiv:2406.02507) https://arxiv.org/abs/2406.02507 This node lets you run normal CFG on a “good” model while also using a “bad” model to compute an additional **AutoGuidance delta** that is added on top of the CFG output (optionally capped and ramped over steps). You provide: - a **good_model** (what you actually want), - a **bad_model** (intentionally different/worse), - and this guider mixes them according to the selected **delta mode**, **cap**, and **ramp**. --- ## Install Clone into: `ComfyUI/custom_nodes/ComfyUI-AutoGuidance` Restart ComfyUI. --- ## Node - **AutoGuidance CFG Guider (good+bad)** → outputs a **GUIDER** Plug it into **SamplerCustomAdvanced** (GUIDER input). --- ## Quick start (how to wire it) 1) Load your two models: - `good_model`: checkpoint + your normal LoRA stack - `bad_model`: intentionally different/worse setup (examples below) 2) Add: - **AutoGuidance CFG Guider (good+bad)** 3) Connect: - its **GUIDER** output → **SamplerCustomAdvanced** → **GUIDER** That’s it. --- ## Extremely important: `dual_models_2x_vram` requires a second checkpoint file path If you pick **`swap_mode = dual_models_2x_vram`**, you must ensure ComfyUI loads **two distinct model instances**. If both loaders point to the same checkpoint path, ComfyUI’s caching can reuse the same underlying model object. In that situation, `dual_models_2x_vram` cannot behave as intended. ### Do this (reliable) Make a second checkpoint file path and load each one: - Copy the checkpoint file (simple + reliable), e.g. `model.safetensors` → `model_copy.safetensors` Then in ComfyUI: - `CheckpointLoaderSimple` → load `model.safetensors` (good path) - `CheckpointLoaderSimple` → load `model_copy.safetensors` (bad path) **VRAM note:** dual-model mode uses ~2× the base checkpoint VRAM (plus LoRAs, plus activations). --- ## Choosing a useful “bad_model” AutoGuidance only has leverage if **good_model and bad_model produce meaningfully different denoiser predictions**. If they’re too similar, the effect will be subtle regardless of ramp settings. Ways to make `bad_model` meaningfully different (examples, not promises): - Remove some LoRAs on the bad path (good has them, bad doesn’t) - Use a different LoRA stack on the bad path (style/character/etc.) - Use the same LoRA at a deliberately different weight (including negative if your workflow supports it) - Change conditioning strategy on the bad path (e.g. different prompt/conditioning pipeline) The point isn’t “worse” aesthetically—it’s **different** in a way that changes predictions. --- ## Node parameters ### Core knobs #### `cfg` Normal CFG scale. #### `w_autoguide` Paper-style parameterization: - `w_autoguide = 1.0` → AutoGuidance delta effectively off - `w_autoguide > 1.0` → strength increases with `(w_autoguide - 1)` Internally this is used as a multiplier for the AutoGuidance delta direction (added on top of CFG). --- ## Swap modes (performance + correctness) ### `shared_safe_low_vram` (default) - Lowest VRAM overhead - Most compatibility / correctness across Comfy variants - Can be **very slow** because it must swap LoRA stacks on a shared model safely ### `shared_fast_extra_vram` - Faster swapping than safe mode (uses inplace updates / device-friendly behavior) - Still shares one model object, so there’s still unavoidable overhead - Uses extra VRAM ### `dual_models_2x_vram` - Fastest **when you truly load two separate checkpoint instances** (see the “second checkpoint path” section above) - Uses ~2× checkpoint VRAM - Avoids the expensive shared swapping path entirely --- ## AutoGuidance delta mode (`ag_delta_mode`) ### `bad_conditional` (recommended default) Uses the most “LoRA-sensitive” direction in practice: - compare **good conditional output** vs **bad conditional output** This tends to show the biggest differences when you change LoRAs on the bad path. ### `raw_delta` Uses a raw difference direction between guided outputs. Can be harsher / less predictable. ### `project_cfg` Projects the “push-away-from-bad” direction onto the **actual CFG update direction**. This keeps changes more aligned with CFG, often more conservative. ### `reject_cfg` Removes the component of the “push-away-from-bad” direction that is parallel to the **actual CFG update direction**. This can increase composition changes when AG otherwise behaves like “CFG++”. --- ## Safety cap: `ag_max_ratio` AutoGuidance is capped relative to the magnitude of the **actual CFG update** (`cfg_out - uncond_good`): - Higher `ag_max_ratio` = stronger visible effect (up to destabilization if extreme) - Lower `ag_max_ratio` = subtler If your output looks “basically like normal CFG”, this cap (and/or the ramp) is the first place to inspect. > This repo includes `debug_metrics` specifically so you can see whether you are being clamped (look at `scale` and `n_delta_applied`). --- ## Ramp over steps (this controls “composition vs detail”) These parameters scale **the cap** (`ag_max_ratio`) over denoise progress. Important concept: - Progress is defined as **0 early (high sigma)** → **1 late (low sigma)** - In this implementation, progress is tracked by step index: `prog = step / (total_steps - 1)` ### `ag_ramp_mode` - `flat` Constant cap across all steps. **Use this if you want composition changes.** - `detail_late` Weak early, strong late → mostly affects fine detail. If `ag_ramp_floor = 0`, early steps can get near-zero AG and the image composition can stay close to normal CFG. - `compose_early` Strong early, weaker late → pushes structure/composition more than detail. - `mid_peak` Strongest mid-steps, weaker at ends. ### `ag_ramp_power` Controls steepness of the selected ramp curve. ### `ag_ramp_floor` Minimum always-on fraction of the cap. - `0.0` means the ramp can reduce AG close to zero in parts of the schedule. - If you use `detail_late` but still want some early influence, raise `ag_ramp_floor`. > This README intentionally does not give “magic” preset numbers. Use `debug_metrics` to confirm what’s actually being applied in your workflow. --- ## Advanced / safety toggles ### `safe_force_clean_swap` Only relevant in shared-model modes. Forces clean swaps between patch stacks to avoid state leakage / washed-out results on some Comfy builds. - Safer - Slower ### `uuid_only_noop` Debug option. Treat “same patches_uuid” as a no-op even if ownership tracking is imperfect. Use only for debugging. ### `debug_swap` Prints swap/patch diagnostics (patch counts, uuids, etc.). ### `debug_metrics` Prints direction / magnitude diagnostics (useful for confirming AG is actually being applied). It also logs active `sampler_post_cfg_function` hooks at step 0, which helps diagnose post-CFG rescale/clamp behavior that can suppress visible AG effects. --- ## Troubleshooting ### “It looks almost the same as normal CFG” Common causes: 1) **Your ramp/cap is effectively zero early**, so composition won’t move. - If you used `detail_late` with `ag_ramp_floor = 0`, early AG can be near-zero by design. - Fix: use `flat` or `compose_early`, or raise `ag_ramp_floor`. 2) **Bad model is not actually “bad enough”** - Make the bad path deliberately different for one test (remove key LoRAs, use an intentionally different stack, etc.). - Confirm with `debug_metrics` that `n_good_minus_bad_cond` is meaningfully non-zero. 3) **You’re not truly in dual-model mode** - If you selected `dual_models_2x_vram` but did not load the checkpoint via two different file paths, you may not actually have two model instances. 4) **Post-CFG hooks are suppressing the visible effect** - `debug_metrics` prints active `sampler_post_cfg_function` hooks at step 0. - Some workflows rescale/normalize/clamp after CFG, which can reduce the visible impact of AG. ### “dual_models_2x_vram is insanely slow” That can happen if both loaders resolve to the same underlying model instance (same checkpoint path / caching). Fix it by creating a second checkpoint file path and loading both. ### “My LoRA changes on the bad path do nothing” Check `debug_swap` / `patch_info`: - If patch counts change when you edit bad LoRAs, the patches are applied. - If the image still doesn’t move, inspect `debug_metrics`: - are you being heavily clamped (`scale` near 0)? - is `n_good_minus_bad_cond` meaningfully > 0? - do you have post-CFG hooks that might normalize the change? --- ## Credits - Paper: https://arxiv.org/abs/2406.02507 - This repository implements an AutoGuidance-style guider node for ComfyUI’s custom sampler pipeline.