Architecture overview (the MLX island), editing guide (grounding_px
semantics, aspect-ratio rule, two-ref order and caveats, Turbo vs
Raw-removals limitation), example-workflow index, environment-variable
reference, tools/harness section.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes rung B — the semantic path of krea2_edit. Krea2Edit gains a
grounding_px control: >0 runs the source image through the (now wired-up)
Qwen3-VL vision tower and reads the instruction *while looking at the
image*, matching what the identity LoRA was trained with; 0 keeps plain
text conditioning.
Integration (krea2_engine/text_encoder.py): splice merged vision tokens
at <image_pad>, inject the 3 deepstack features at decoder layers 0/1/2,
apply interleaved multimodal RoPE (3D t/h/w positions, mrope_section
24/20/20), and tap the same 12 layers. Torch-free at runtime; the vision
tower is loaded lazily on first grounded use.
Validated against the full PyTorch Qwen3-VL model end-to-end: the grounded
12-layer tap matches to cosine 1.0, mean|Δ| 2.7e-5 (a deepstack recording-
order bug found and fixed via the per-layer diff). txt2img stays
pixel-identical to the parity reference; grounded edit runs end-to-end and
preserves the source scene better than the plain path (see tools/).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New additive node for identity-preserving edits: the clean VAE-encoded
source rides the sequence as a frame≠0 reference token block while a
fresh-noise target is denoised, and only the target is decoded — the
appearance path from lbouaraba/comfyui-krea2edit, mapped onto our engine
(sequence [text | src(frame=1) | target(frame=0)], mirroring the 3-axis
RoPE frame scheme that was already present but unused).
Engine additions are isolated: transformer gains prepare_edit +
denoise_step_edit, sampling gains sample_edit + build_edit_positions,
pipeline gains generate_edit; the txt2img/img2img sample() path is
untouched (verified txt2img stays pixel-identical to the parity ref).
Pairs with the krea2_edit identity LoRA (conradlocke/krea2-identity-edit),
which loads through the existing Krea2LoRA node unchanged (ai-toolkit
reference naming, 28 blocks, 0 unmatched keys). Validated: source fox +
"red scarf" keeps the fox's identity while applying the edit; the LoRA
sharpens edit coherence markedly (see tools/test_edit.py). Second
reference image supported for multi-ref edit LoRAs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New additive node (existing four untouched): VAE-encodes a source image
and denoises from a noised version of it. Rectified-flow interpolant
x_t=(1-t)·z0+t·noise, entering the schedule at t=denoise; denoise=1.0
takes the exact txt2img code path (verified pixel-identical to the
parity reference — the hard guard that generation is undisturbed).
Engine changes are additive: sample() gains init_latent/strength
(strength>=1.0 is a byte-identical no-op), pipeline gains
generate_img2img() reusing the prompt cache + LoRA path, nodes.py gains
_from_image_tensor. VAE encode input range verified [-1,1] (round-trip
45 dB). Denoise sweep 0.3→0.9 is monotonic and coherent; contact sheet
and strength=1.0 guard in tools/test_img2img.py.
First rung of the image-conditioning ladder (img2img → in-context edit
→ grounded vision); reuses the VAE-encode plumbing the edit path needs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
README documents the measured speedups (512² ~35% / 1024² ~15% on
M1 Max), the prompt-embedding cache, and the KREA2_EXACT_LEGACY seed-
reproducibility flag. NOTICE records the performance modifications to
the vendored engine, as its attribution chain previously stated the
engine was otherwise unmodified.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Krea2ModelLoader / Krea2LoRA / Krea2Generate / Krea2Unload nodes running
Krea-2-Turbo on Apple MLX in-process, returning ComfyUI IMAGE tensors.
The krea2_engine/ pipeline is vendored directly in this repo (no external
git dependency) — see NOTICE for the full attribution chain back to the
original Krea 2 model and the MLX port it builds on.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
