Replicates the krea2_controlnet_pose.json reference workflow inside this node
(ostris nodes are no longer required):
- _krea2_make_ref_kv_forward: replaces diffusion_model.forward so reference
latents run in isolated kv_cache mode — one t=0 ref-only pass precomputes
every block's K/V (refs attend only to each other), reused as extra
attention keys each denoising step. This matches the ai-toolkit kv_cache
training convention the openpose control LoRA was trained with
- New helpers: _krea2_patchify, _krea2_ref_pack (RoPE axis-0 index i+1),
_krea2_ref_attn_kv (K/V capture/inject), _krea2_ref_block_forward,
_krea2_ref_precompute_kv, _krea2_forward_with_cached_refs,
_krea2_ref_fingerprint (content-keyed cache)
- New optional node input use_kv_cache (default True); when False the stock
joint-mode ref path is used instead
- extra_conds ref injection retained: skipped when conditioning already
carries reference_latents (TextEncodeKrea2OstrisEdit compat), so the VLM
vision path can still be combined without double injection
- depth / zimage / nunchaku_qwenimage / qwenimage_standard routes unchanged
The openpose control LoRA is a Kontext-style conditioning LoRA (blocks +
txtfusion, no expanded first projection). The previous openpose path only
applied the LoRA patches and never injected the pose image, so generation had
no pose control at all.
- _apply_krea2_openpose_control now VAE-encodes the control image (fit ~1MP,
snapped /16, ai-toolkit convention) and injects it as a reference latent
via patched extra_conds / extra_conds_shapes with
ref_latents_method=index_timestep_zero (native Krea2 ref pathway)
- Injection is skipped when the conditioning already carries
reference_latents (e.g. TextEncodeKrea2OstrisEdit), so the VLM vision path
wins and no double injection occurs
- depth / zimage / nunchaku_qwenimage / qwenimage_standard routes unchanged
- Add _classify_krea2_control_type() to detect depth vs openpose by first.weight presence
- Add _apply_krea2_openpose_control() for pure block LoRA (no first projection, no control latent)
- Route txtfusion.* keys in _krea2_target_key for layerwise/refiner blocks
- Split krea2 route into depth (unchanged) / openpose (new) exclusive branches
- No changes to zimage/nunchaku/qwenimage_standard routes or loader node
Post-merge additional fixes for PR #52 (per md/PR52_MERGE_PLAN.md):
- compose_loras_v2: restore _cached_first_lora_state_dict so the first
LoRA state dict loaded during section-3 debug inspection is reused in
the section-4 main loop. Without this the first LoRA is read from disk
twice whenever the precompiled cache is disabled (default).
- qwenimage_v2.py IS_CHANGED: drop the `model` arg (ComfyUI never passes
MODEL inputs to IS_CHANGED; the TypeError -> NaN path forced re-compose
every run) and remove str(model) from the hash (object repr changes
every run, which also forced constant re-execution).
Traced full data flow across Stage 1 (application loop) and Stage 3 (_apply_lora_to_module nunchaku branch). Two-stage optimization is internally consistent — no device mismatch. Previous concern was incorrect and has been replaced with validation results.
Referenced loader classes do not exist in ComfyUI-nunchaku. CPU offload already available via standard ComfyUI mechanisms. Adoption count revised from 8 to 7 (1 rejected).
ussoewwin