Generalize the grounded encode to N reference images (scene + subject):
both go through the vision tower with a per-image <vision> block, spliced
and deepstack-injected at their own token positions, with multi-image
get_rope_index. Validated vs full torch Qwen3-VL — two-image grounded tap
matches to cosine 1.0, mean|Δ| 3.1e-5 (single-image unchanged). generate_edit
now feeds both image + image_b through grounding when grounding_px > 0.
Fix a ZeroDivisionError in preprocess: max_pixels is now clamped so a tiny
grounding_px (e.g. from a stale workflow whose widgets predate the
grounding_px control) can't zero the area ceiling. The example edit
workflow gains the grounding_px widget so its values align with the node.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes rung B — the semantic path of krea2_edit. Krea2Edit gains a
grounding_px control: >0 runs the source image through the (now wired-up)
Qwen3-VL vision tower and reads the instruction *while looking at the
image*, matching what the identity LoRA was trained with; 0 keeps plain
text conditioning.
Integration (krea2_engine/text_encoder.py): splice merged vision tokens
at <image_pad>, inject the 3 deepstack features at decoder layers 0/1/2,
apply interleaved multimodal RoPE (3D t/h/w positions, mrope_section
24/20/20), and tap the same 12 layers. Torch-free at runtime; the vision
tower is loaded lazily on first grounded use.
Validated against the full PyTorch Qwen3-VL model end-to-end: the grounded
12-layer tap matches to cosine 1.0, mean|Δ| 2.7e-5 (a deepstack recording-
order bug found and fixed via the per-layer diff). txt2img stays
pixel-identical to the parity reference; grounded edit runs end-to-end and
preserves the source scene better than the plain path (see tools/).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pad prompts only to the longest in the batch instead of always to 541
tokens; the 5 suffix tokens keep their absolute RoPE positions from the
padded layout (positions are index-assigned and only trailing pad columns
are dropped, so every real token's rotation is unchanged — verified:
embeddings layer bit-exact, divergence enters only via kernel
reassociation at ~1e-6 f32 and grows smoothly, no structural jump).
Encoder work drops ~12x for short prompts. KREA2_EXACT_LEGACY=1 restores
always-max padding.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
mx.fast.scaled_dot_product_attention handles grouped KV heads (48Q/12KV)
natively; passing them directly is bit-identical to the mx.repeat
workaround (parity: max|d|=0.0 incl. LoRA case) while skipping the 4x
K/V materialization in both the DiT and the Qwen3 text encoder.
Wall-clock neutral at 512² (quantized matmuls dominate); saves memory
traffic and peak footprint, which matters on 32 GB machines.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Krea2ModelLoader / Krea2LoRA / Krea2Generate / Krea2Unload nodes running
Krea-2-Turbo on Apple MLX in-process, returning ComfyUI IMAGE tensors.
The krea2_engine/ pipeline is vendored directly in this repo (no external
git dependency) — see NOTICE for the full attribution chain back to the
original Krea 2 model and the MLX port it builds on.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
