Weights now live in the Comfy Desktop shared store
(~/ComfyUI-Shared/models/{krea2,loras}) rather than a project-local Models/
folder bridged by per-file symlinks. The test harness searches there first,
falls back to the legacy Models/ location, and honors the KREA2_TEST_* env
overrides.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
_shared gains default_model()/default_lora() (env overrides
KREA2_TEST_MODEL/KREA2_TEST_LORA, else <repo>/../Models discovery with
clear errors) and fox_source()/ref_uint8() for the reference source image
boilerplate repeated across scripts. test_img2img/test_edit/
test_edit_grounded no longer hardcode machine paths; all tools compile
and resolve on a fresh checkout.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Generalize the grounded encode to N reference images (scene + subject):
both go through the vision tower with a per-image <vision> block, spliced
and deepstack-injected at their own token positions, with multi-image
get_rope_index. Validated vs full torch Qwen3-VL — two-image grounded tap
matches to cosine 1.0, mean|Δ| 3.1e-5 (single-image unchanged). generate_edit
now feeds both image + image_b through grounding when grounding_px > 0.
Fix a ZeroDivisionError in preprocess: max_pixels is now clamped so a tiny
grounding_px (e.g. from a stale workflow whose widgets predate the
grounding_px control) can't zero the area ceiling. The example edit
workflow gains the grounding_px widget so its values align with the node.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes rung B — the semantic path of krea2_edit. Krea2Edit gains a
grounding_px control: >0 runs the source image through the (now wired-up)
Qwen3-VL vision tower and reads the instruction *while looking at the
image*, matching what the identity LoRA was trained with; 0 keeps plain
text conditioning.
Integration (krea2_engine/text_encoder.py): splice merged vision tokens
at <image_pad>, inject the 3 deepstack features at decoder layers 0/1/2,
apply interleaved multimodal RoPE (3D t/h/w positions, mrope_section
24/20/20), and tap the same 12 layers. Torch-free at runtime; the vision
tower is loaded lazily on first grounded use.
Validated against the full PyTorch Qwen3-VL model end-to-end: the grounded
12-layer tap matches to cosine 1.0, mean|Δ| 2.7e-5 (a deepstack recording-
order bug found and fixed via the per-layer diff). txt2img stays
pixel-identical to the parity reference; grounded edit runs end-to-end and
preserves the source scene better than the plain path (see tools/).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Groundwork for image-grounded conditioning (rung B). Vendors mlx-vlm's
pure-MLX Qwen3-VL vision tower into krea2_engine/vision/ (self-contained:
inlined SDPA helper, local VisionConfig) and adds torch-free image
preprocessing (smart-resize + CLIP normalize + patch flatten).
Both validated against the PyTorch reference (tools/test_vision_tower.py):
- tower vs transformers Qwen3VLVisionModel: merged embeds + all 3 deepstack
features match to max|Δ|~2e-4, cosine 1.0 (bf16 rounding).
- preprocessing vs transformers image processor: max|Δ|~2e-7, grid exact.
Loads the visual.* weights already present in the Krea-2-Turbo text
encoder (315 tensors, 0 missing). Not yet wired into any node — the
grounded-encode integration (M-RoPE, deepstack injection, 12-layer tap)
is the next step. mlx-vlm is MIT; attribution added to NOTICE.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New additive node for identity-preserving edits: the clean VAE-encoded
source rides the sequence as a frame≠0 reference token block while a
fresh-noise target is denoised, and only the target is decoded — the
appearance path from lbouaraba/comfyui-krea2edit, mapped onto our engine
(sequence [text | src(frame=1) | target(frame=0)], mirroring the 3-axis
RoPE frame scheme that was already present but unused).
Engine additions are isolated: transformer gains prepare_edit +
denoise_step_edit, sampling gains sample_edit + build_edit_positions,
pipeline gains generate_edit; the txt2img/img2img sample() path is
untouched (verified txt2img stays pixel-identical to the parity ref).
Pairs with the krea2_edit identity LoRA (conradlocke/krea2-identity-edit),
which loads through the existing Krea2LoRA node unchanged (ai-toolkit
reference naming, 28 blocks, 0 unmatched keys). Validated: source fox +
"red scarf" keeps the fox's identity while applying the edit; the LoRA
sharpens edit coherence markedly (see tools/test_edit.py). Second
reference image supported for multi-ref edit LoRAs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
New additive node (existing four untouched): VAE-encodes a source image
and denoises from a noised version of it. Rectified-flow interpolant
x_t=(1-t)·z0+t·noise, entering the schedule at t=denoise; denoise=1.0
takes the exact txt2img code path (verified pixel-identical to the
parity reference — the hard guard that generation is undisturbed).
Engine changes are additive: sample() gains init_latent/strength
(strength>=1.0 is a byte-identical no-op), pipeline gains
generate_img2img() reusing the prompt cache + LoRA path, nodes.py gains
_from_image_tensor. VAE encode input range verified [-1,1] (round-trip
45 dB). Denoise sweep 0.3→0.9 is monotonic and coherent; contact sheet
and strength=1.0 guard in tools/test_img2img.py.
First rung of the image-conditioning ladder (img2img → in-context edit
→ grounded vision); reuses the VAE-encode plumbing the edit path needs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
bench.py attributes generation time to encode / per-step / decode stages
via the pipeline's own step_callback; parity.py snapshots decoded float32
pixels for a fixed prompt/seed/size (+LoRA) matrix and asserts later runs
stay uint8-identical — the quality gate for every optimization commit.
Reference outputs and results land in tools/reference/ (gitignored).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
