# ComfyUI-Krea2-Reference

Reference images for **Krea 2** in ComfyUI — ChatGPT-style: *"the character
from Picture 1, in the exact art style of Picture 2"*. Character and style
transfer through native vision tokens, not img2img.

Krea 2's text encoder is **Qwen3-VL-4B**, a vision-language model. This node
interleaves your reference images into the prompt as native vision tokens
(`Picture N:`), so the DiT reads character/style information from the same
hidden states it was trained on. Paired with the **reference LoRA** (below),
the model follows the scene text instead of collaging the reference.

## Examples

Everything below was generated by Krea 2 itself (Turbo, 8 steps, cfg 1.0).
The pipeline: one root character and two style images generated text-only,
then combined through this node + the `krea2_reference_v1` LoRA (layered:
text 1.0 / dit 0.0).

**Step 1 — generate the ingredients (text-only, no references):**

| Root character | Style image A (ukiyo-e) | Style image B (rubber-hose) |
|---|---|---|
| ![root](examples/root_girl.png) | ![ukiyoe](examples/style_ref_ukiyoe.png) | ![rubberhose](examples/style_ref_rubberhose.png) |

**Step 2 — combine them:** `"the character from Picture 1, in the exact art
style of Picture 2, <scene>"`

| Girl × ukiyo-e | Girl × rubber-hose |
|---|---|
| ![girl ukiyoe](examples/girl_in_ukiyoe.png) | ![girl rubberhose](examples/girl_in_rubberhose.png) |

**Style by text instead of image** (single reference, style described in the
prompt) **and multi-panel:**

| Watercolor (text style) | Retro 90s cel (text style) | 4-panel comic |
|---|---|---|
| ![watercolor](examples/style_watercolor.png) | ![retro](examples/style_retro90s.png) | ![comic](examples/comic.png) |

Same braid, star hairpin, freckles, bell choker, sunflower dress, cat
backpack — across styles and panels. With two references (v1), the style
image is adopted strongly by the scene and only partially by the subject;
describing the style in text as well pushes it further onto the character.

> **Drag & drop:** every PNG in `examples/` embeds its full ComfyUI graph.
> Drop `examples/girl_in_ukiyoe.png` into ComfyUI to get the two-reference
> workflow (2× Load Image → Krea 2 Reference Encode → layered LoRA →
> KSampler), or `examples/style_watercolor.png` for the single-reference one.

## Install

```bash
cd ComfyUI/custom_nodes
git clone https://github.com/KonokoAz/ComfyUI-Krea2-Reference
```

Models (ComfyUI format): Krea 2 checkpoint (`UNETLoader`), `qwen3vl_4b`
**bf16** text encoder (`CLIPLoader`, type `krea2` — fp8 breaks the vision
tower), `qwen_image_vae` (`VAELoader`). Download the reference LoRA from
Civitai / Hugging Face and put it in `models/loras/`.

## Nodes

### Krea 2 Reference Encode (`TextEncodeKrea2Reference`)

Prompt + up to 4 reference images. **Roles live in the prompt**: write
`character1..4` for identity references and `style1..4` for style-only
references (their patches are shuffled so the subject can't leak; `characterN`
wins if both appear). Both expand to the exact clauses the LoRA trains on —
`"The character1 driving a car in the style2"` is a complete prompt. Plain
`image1..4` / `Picture 1..4` also work; connected images never mentioned get a
character clause appended automatically.

| Input | Description |
|---|---|
| `imageN_detail` | vision tokens per ref: low/normal/high/max = 256/512/1024/1280px. High for characters, low for style/pose hints |
| `imageN_mask` | crop to mask bbox + white-out background before encoding |
| `bg_removal` | wire `LoadBackgroundRemovalModel`: auto-cleans content refs |
| `layout_split` | first fraction of steps conditioned on text only (0.25 kills collage layouts zero-shot; unnecessary with the LoRA) |

### Krea 2 Reference LoRA (layered) (`Krea2ReferenceLoraLoader`)

Loads `krea2_reference_v1.safetensors` with independent strengths per layer
group. Reference addressing, anti-bleed and style transfer live in the
conditioning path (`text_fusion`/`txt_in`); the DiT-block tensors mostly add
identity drift.

- **`strength_text_fusion 1.0` + `strength_dit_body 0.0`** — recommended.
  Base model rendering untouched, references followed. Best identity, best
  for comics.
- Raise `strength_dit_body` to ~0.3 for extra scene adherence on hard prompts.
- A regular `LoraLoaderModelOnly` at strength 0.7–1.4 also works (single
  knob, slight identity drift at high strength).


## Reference LoRA v1

Trained on ~2.7k reference-following triplets (teacher-generated character
sets, judged for consistency by Qwen3-VL). What it fixes vs zero-shot:

- Scene text followed instead of ignored (no more white-background
  character-sheet collages).
- `"the character from Picture 1"` / `"in the exact art style of Picture 1"`
  addressing.
- Style transfer decouples the reference's palette/layout from the scene.
- Plain text-to-image unaffected (ref-dropout training rows).

Known limits (v2 in progress): character-sheet references induce multi-view
clones (use single-pose refs); style refs strongly tied to a franchise drag
their cast along; two-reference prompts (character + style) work only
partially.

## Tips

- **The reference's colors ARE the character's identity.** To restyle an
  attribute, change it in the reference, not the prompt: a grayscale copy of
  the character ref + "the entire image is black and white" produces a fully
  monochrome scene (`examples/girl_in_rubberhose_bw.png`); with the color
  ref, the character keeps her palette no matter what the prompt says.
  Raising `strength_dit_body` does NOT help here — it pulls toward the LoRA's
  (colorful) training distribution.
- CFG 1.0 on Turbo skips the negative prompt entirely — it's not a bug.
- Wide canvases encourage figure duplication; 1024×1024 is the sweet spot.
- For exact pose: pose image as init latent, denoise 0.70–0.78.