69 lines
3.6 KiB
Markdown
69 lines
3.6 KiB
Markdown
# Krea2 Reference Encoder
|
||
|
||
Prepares the reference latents for a Krea2-patched model and appends them to your existing text conditioning. The adapter uses RoPE (Rotary Position Embedding) to attend to the references during diffusion without overwriting your text prompt.
|
||
|
||
## How the slots work
|
||
|
||
The adapter was trained with a fixed slot convention:
|
||
|
||
| Slot | Input | Resolution | Role |
|
||
| --- | --- | --- | --- |
|
||
| 1 | `base_latent` | **Must match output** | The image being edited, inpainted, or outpainted |
|
||
| 2 | `reference_latent_2` | Any | Style / content reference (optional) |
|
||
| 3 | `reference_latent_3` | Any | Style / content reference (optional) |
|
||
| 4 | `reference_latent_4` | Any | Style / content reference (optional) |
|
||
|
||
Slot 1 is always the **base image** — the canvas the model will modify. Slots 2–4 are optional visual references that guide style or content.
|
||
|
||
## Inputs
|
||
|
||
| Name | Type | Required | Description |
|
||
| --- | --- | --- | --- |
|
||
| `positive` | CONDITIONING | ✅ | Positive text conditioning from a CLIP Text Encode node. The reference payload is **appended** — your prompt is never overwritten. |
|
||
| `negative` | CONDITIONING | ✅ | Negative text conditioning. Receives the same reference payload as `positive`. |
|
||
| `base_latent` | LATENT | ✅ | The image to edit / inpaint, VAE-encoded. **Must be the same resolution as the generation output.** |
|
||
| `strength` | FLOAT | ✅ | Scales all reference latents before storing them. Range: `0.0 – 2.0`, default `1.0`. |
|
||
| `reference_latent_2` | LATENT | ➖ | Optional style / content reference, VAE-encoded. |
|
||
| `reference_latent_3` | LATENT | ➖ | Optional style / content reference, VAE-encoded. |
|
||
| `reference_latent_4` | LATENT | ➖ | Optional style / content reference, VAE-encoded. |
|
||
|
||
## Outputs
|
||
|
||
| Name | Type | Description |
|
||
| --- | --- | --- |
|
||
| `positive` | CONDITIONING | Positive conditioning with all reference latents embedded under `reference_latents`. |
|
||
| `negative` | CONDITIONING | Negative conditioning with the same reference payload. |
|
||
|
||
## Usage
|
||
|
||
### Instruct-style editing
|
||
|
||
Edit an existing image by describing only the change you want.
|
||
|
||
1. **Load Image** → **VAE Encode** → `base_latent`
|
||
2. Write a prompt describing the change (e.g. *"replace the sky with a stormy sunset"*)
|
||
3. Connect `positive` / `negative` to a **KSampler** whose model comes from **Krea2 Model Patcher**
|
||
|
||
### Inpainting / outpainting
|
||
|
||
Fill a masked or extended region while keeping the rest of the image intact.
|
||
|
||
1. Prepare the source image with the unknown region masked (neutral grey or noise)
|
||
2. **VAE Encode** the masked image → `base_latent`
|
||
3. Prompt for the content that should fill the masked area
|
||
|
||
### Reference-guided generation
|
||
|
||
Generate a new image that draws style or content from reference images.
|
||
|
||
1. Provide a blank or noise latent (same size as the desired output) → `base_latent`
|
||
2. Load style / content reference images → **VAE Encode** → `reference_latent_2` (and optionally `_3`, `_4`)
|
||
3. Write a prompt describing the desired output
|
||
|
||
## Notes
|
||
|
||
- `base_latent` **must match the output resolution** — this is a hard constraint from the training data. Mismatched resolutions will produce degraded results.
|
||
- `strength` above `1.0` amplifies the reference signal and can cause the model to over-fit to the references at the expense of prompt adherence.
|
||
- Chaining two **Krea2 Reference Encoder** nodes is supported — the second call accumulates its references onto the first via `torch.cat`.
|
||
- The downstream sampler **must** use a model patched with **Krea2 Model Patcher**. An unpatched model will silently ignore the `reference_latents` key in the conditioning metadata.
|