kgilper-krea-reference/docs/deepstack-layers/README.md

The layers Values: What They Scale, and Where They Came From

This folder is the authoritative, reproducible account of the 12 layers values in the Krea reference recipes - what they physically scale (verified from the model), where the specific numbers came from (traced through the development history), and what is still unmeasured. It is deliberately honest about the difference between those three things, because "trust me, I swept it" would be wrong: the numbers were not swept.

1. What the 12 values scale (verified from ComfyUI source + the model weights)

Krea 2 is unusual in how it conditions on text. Its ComfyUI text encoder (comfy/text_encoders/krea2.py) states it directly:

Krea 2 (K2) text encoder: Qwen3-VL-4B, 12-layer tap. K2 conditions on a stack of hidden states from 12 layers of Qwen3-VL-4B (reference taps hidden_states[2,5,8,...,35]), kept as a (B, 12, seq, 2560) tensor ... flattened to (B, seq, 12*2560).

So the conditioning Krea 2 consumes is 12 text-encoder layer taps (layers 2, 5, 8, ... 35 - every third layer), each 2560 wide, flattened to width 12 x 2560 = 30720. The Krea 2 diffusion model then fuses them with a learned weighting - its weights include txtfusion.projector [1, 12] and per-tap layerwise_blocks at width 2560 (visible in the checkpoint header).

The node (kg_krea_v9/conditioning.py) splits the conditioning-delta's width into layer_count (= table length = 12) equal chunks. On Krea 2: flat = 30720, 30720 % 12 == 0, layer_dim = 2560 - the split lands exactly on the 12 layer taps, the divisibility check passes, and the per-layer gains genuinely apply (no flat-average fallback). layers[i] scales tap i (hidden_states[2+3i]), on top of the model's own learned fusion. Position is depth: layers[0] is the shallowest tap (layer 2), layers[11] the deepest (layer 35).

Terminology warning - "deepstack" is a misnomer here. The node and the V9 paper call these "deepstack chunks." That name is inherited but wrong: Qwen3-VL's actual deepstack is 3 vision taps (qwen3vl.py: deepstack_visual_indexes=[5, 11, 17]) used to inject image-embed features, not text. The node's 12 chunks are the 12 K2 text-layer taps above. Read "deepstack chunk N" as "text-layer tap N" throughout.

The landing math per tap: effective scale = clamp(strength x phase x shape x layers[i], -6, +6), compose weight = scale - 1. Gains act on the token channel only; the pooled/global channel (global) is independent. Derivation for authoring: custom_recipes/README.md.

2. Where the specific numbers came from (traced, not guessed)

The provenance was reconstructed from the node's development history (Kevin's OpenAI Codex sessions, 2026-06-30 to 07-01; internal record in local_records/2026-07-03-deepstack-layer-determination/). The honest story:

1. Early versions (v1-v5) applied a single uniform scalar across the whole tap stack - no per-tap tables. "Roles" were prose appended to the prompt.
2. The per-tap tables were adopted, not swept. The 12-value shape came from a third-party node, ComfyUI-ConditioningKrea2Rebalance (GitHub nova452), whose DEFAULT_WEIGHTS = "1,1,1,1,1,1,1,2.5,5.0,1.1,4.0,1.0" spike the deep taps (strongest at tap 8, then 10, then 7). The shipped STYLE_LAYER_PULL tail is that template almost verbatim (... 2.5, 5.0, 1.1, 4.0, 1.2); PALETTE, MATERIAL, and LIGHTING are per-role scalings of the same template; the suppressed early ramp (taps 0-4) was added on the principle that shallow layers carry structure a look-borrowing card should drop.
3. What was tuned empirically were the scalar knobs, not the vector: the shape pull (0.45 -> 0.35) and the strength cap (-> 0.9), judged against seeded before/after renders. The 12-value arrays themselves were held fixed and never validated tap-by-tap.

So the numbers rest on: a correct structural premise (12 real taps), a borrowed spike template, and a sound general principle (shallow = structure, deep = appearance) - but not a per-tap measurement on Krea 2.

3. What the tables' shape looks like (a designed pattern, not independent evidence)

python docs/deepstack-layers/analyze_tables.py prints the five tables from the live code. Their shape is coherent and worth seeing:

Taps	All appearance tables	Reading
0-4	suppress, smooth ramp 0.15 -> 0.85	shallow layers (structure) turned down
5-6	neutral 1.0	transition
7 / 8 / 10	spike (mean 2.4x / 4.8x / 3.9x)	deep layers (appearance) turned up
9, 11	mild 1.1-1.4	gentle

Note: the four appearance tables agreeing does not independently confirm the per-tap semantics - they agree because they are scalings of one borrowed template (section 2), so their consensus reflects a single design, not four independent measurements. It is a self-consistent, principled design; treat it as such.

3.5 The authoritative design basis (Krea 2 report + UniFusion) - and how it points to improvement

The Krea 2 Technical Report states the design directly: rather than the last VLM layer, Krea 2 "introduce[s] a shallow attention layer that aggregates hidden features across layers" to "dynamically select coarse-to-fine text representations," explicitly "inspired by UniFusion," because "the last-layer features of an autoregressive LLM are suboptimal... optimized for next-token prediction rather than image generation."

UniFusion (the method Krea 2 builds on) names the mechanism Layerwise Attention Pooling (LAP) - a learned attention over "every third layer" (matching Krea 2's hidden_states[2,5,8,...,35]) - and reports the load-bearing empirical finding:

"higher weights to early-to-middle VLM layers... semantic information useful for downstream finetuning lives in the earlier part of a VLM." and "the penultimate VLM layer contributes the least."

So the "correct" per-layer weighting is the one the model learned (its txtfusion LAP), and the design favors early-to-middle layers, with the deepest/penultimate contributing least. (Qwen3-VL's own "DeepStack" is a separate mechanism - 3 vision taps - which is why "deepstack" here is a misnomer for these text-layer taps.)

This is misaligned with the shipped tables, which spike the deep taps. Mapping taps to layers (layer = 2 + 3*tap) and comparing the model's learned emphasis (|txtfusion.projector|, Stage 0) to the shipped STYLE table:

• Both value tap 7 (layer 23) and drop the earliest taps - aligned.
• The table's biggest spike is tap 8 (layer 26) - only the model's #5.
• The table suppresses tap 4 (layer 14) - the model's #3 and squarely in UniFusion's favored band.
• The table over-weights tap 11 (layer 35, penultimate) - which the model and UniFusion rank nearly last.

The table's emphasis is shifted deeper than the design intends (a legacy of the borrowed deep-spike template). Improvement directions (all validated via the render harness below, since render measurement is the only reliable judge):

1. Track the model's learned weighting - lift taps 4 and 9, moderate the tap-8 over-spike, drop tap 11 (a table proportional to the projector prior, model_prior.json).
2. Use the coarse-to-fine axis to differentiate recipes - broad/semantic jobs emphasize the early-to-middle taps; fine-detail jobs allow more of the deep taps. The borrowed template used ~one spike shape for every role.
3. Consider per-tap magnitude normalization in the node - render tests show the delta-reweighting is magnitude-dominated (deep taps move the image, shallow taps barely do), so gains on the semantically-useful early taps have little effect as-is. Normalizing per-tap magnitude before the gain would let the tables emphasize early-middle layers as the design intends. This is the biggest lever, and a node-mechanism change rather than a table edit.

4. How to derive the values reliably (the methodology + harness)

Deriving these reliably - and re-deriving them for any future checkpoint - is a four-stage pipeline, from cheapest/most-targeted to definitive. Stages 0-1 are built and have been run (Stage 1 once, see its result below); Stages 2-3 add render-based validation and optimization on a V10 box. This replaces the way the shipped values were actually set (borrow a template + eyeball), which Stages 0-1 show does not match how the model behaves.

Stage 0 - the model's own prior (free, no rendering) - DONE

Krea 2's diffusion model already learned a per-tap weighting (txtfusion.projector [1,12]). Extract it and compare the shipped tables to it:

python docs/deepstack-layers/extract_model_prior.py --model <krea2-*.safetensors>  # re-extract
python docs/deepstack-layers/compare_to_model_prior.py                             # tables vs. prior

Result (2026-07-03): the model leans hardest on taps 7 > 9 > 4 > 10 > 8; the shipped tables spike 8 > 10 > 7. Rank-correlation of each table with the model's emphasis is only +0.58 to +0.64, with consistent gaps at taps 4 (model #3, tables suppress it), 8 (tables' #1 spike, model #5), and 11. So the borrowed template does not track how Krea 2 actually uses the taps - concrete motivation to re-derive. (Caveat: the projector weights processed taps while the node's gains hit the raw delta, so this is a prior/smell-test, not ground truth.) Prior values: model_prior.json.

Stage 1 - representational selectivity (cheap, no rendering) - RUN ONCE

Measure, at the conditioning level, what each tap encodes. Build a controlled reference set (vary one attribute at a time), encode each through a neutral V10 card, capture the per-tap signature with the probe node, and decompose the variance per tap per attribute.

python docs/deepstack-layers/probe_selectivity.py --selftest              # validate the math (passes)
python docs/deepstack-layers/generate_probe_graphs.py --dry-run           # build encode graphs
# then on a V10+probe box: --server URL, copy signatures back, then:
python docs/deepstack-layers/probe_selectivity.py --probes <dir> --manifest probe_out/probe-manifest.json

Measured (2026-07-03, on the Krea 2 turbo model), in three passes:

1. Mechanism confirmed live. The probe reported feature width 30720, divisible by 12, tap_dim 2560 - exactly the 12-tap stack; the split engages. Tap magnitude rises steeply with depth (tap 0 ~29 -> tap 11 ~293).
2. Conditioning-level selectivity does exist (once the stimuli are clean: one colorful base, each attribute changed by a single pure transform). In proportion, shallow taps 0-4 lean palette and middle taps 5-10 lean texture (cross-tap std-dev ~0.10-0.12, well above the flat first attempt).
3. But render validation flipped the practical conclusion - and this is the important part. Isolating taps in an actual render (shape=1, global=0, spike the taps, render, measure) showed the deep taps do essentially all the visible work: spiking taps 7/8/10 turned a plain bowl into the colorful reference vase entirely, while spiking the shallow taps 0-4 did nothing. The deep taps' huge magnitude dominates the pixel effect regardless of their proportional attribute mix, so the shallow "palette" taps are practically irrelevant, and the effective (deep) taps move palette + texture + subject together - they do not cleanly separate.

Net: the shipped tables spiking taps 7/8/10 is directionally reasonable (those are the high-leverage taps, which the model's own projector also emphasizes). What is not supported is the idea that the per-tap values cleanly route distinct attributes - and whether the specific per-role numbers are optimal cannot be answered at the conditioning level at all.

The load-bearing lesson: conditioning-level selectivity misleads on its own, because tap magnitude and the nonlinear diffusion path dominate the actual effect. The layers can only be derived reliably from renders. Full arc, raw signatures, and the validation images: local_records/2026-07-03-deepstack-layer-determination/stage1-measurement/.

Stage 2 - metric-scored render validation (moderate) - PROVEN OUT, formalize next

The decisive test above (isolate taps -> render -> measure palette/texture) is Stage 2 in miniature and is why the conclusion is trustworthy. Formalizing it means a fixed seed/prompt grid scored with objective, attribute-specific metrics (color-distribution distance for palette, LPIPS/SSIM for structure, high-frequency energy for texture, style/identity distances), not by eye. SCORING.md is the interim manual rubric.

Stage 3 - derive the table from renders (definitive) - RUN ONCE for the style role

Turn a role's intent into a measurable objective and pick/optimize the 12-gain vector by rendering candidates and scoring them. Done here for the style role: hold the real style-recipe settings fixed and vary only layers; render (colorful reference -> plain-bowl prompt) and score look-transfer minus subject-leak.

Result (2026-07-03): among seven candidate tables, the shipped STYLE table scored best (objective 0.106), with the shallow-tap table clearly worst

• the render evidence supports spiking the deep taps. But the differences are small and every candidate's output was nearly the same bowl, so for the style role the layer table is a second-order knob: the shipped values hold up, and the primary look-vs-subject control is strength and the shape/global (token/pooled) split, not the per-tap vector. The layers visibly bite only in the aggressive isolation test (Stage 1 result above), where the deep taps dominate - which the shipped tables already emphasize.

To go further (more roles, references, a real CMA-ES/Bayesian optimizer with CLIP style/identity metrics), the harness and objective are in place; this run is the demonstration. Full result + images: local_records/.../stage1-measurement/stage3-derivation/.

Bottom line

The reliable, measured way to set the layers is render-based (Stages 2-3); conditioning analysis alone misleads (Stage 1). Applying it shows the shipped tables are directionally right (they spike the effective, high-magnitude deep taps, which the model's own projector also leans on) and, for the style role, near-optimal and second-order. The durable deliverable is the harness itself, which can re-derive or verify the layers for any future checkpoint.

Building the controlled reference sets

Stage 1's reliability depends on the sets being controlled. Make them by generating variants that hold everything fixed but one attribute (e.g. the same scene re-rendered in N palettes; the same palette over N subjects). These can be produced with Krea/Ideogram to a dedicated folder. The probe manifest records each reference's attribute levels for the variance decomposition.

Files

File	What	Runs
analyze_tables.py	Prints the five tables' designed shape from live code.	now, local
extract_model_prior.py	Extracts the model's learned per-tap weighting from a checkpoint.	now (reads model file)
compare_to_model_prior.py	Stage 0: tables vs. the model prior.	now, local
probe_node/	ComfyUI node that saves per-tap conditioning signatures.	install on box
probe_selectivity.py	Stage 1: tap x attribute selectivity + self-test.	now (selftest), needs data (real)
generate_probe_graphs.py	Builds the encode-only probe graphs.	now (dry-run), needs box (encode)
generate_sweep.py	Stage 2: single-tap render sweep.	needs V10 box
SCORING.md	Manual scoring rubric for the render sweep.	reference

Cross-references

• V9 technical paper 5.2 - the channel math (inherits the "deepstack" naming corrected here).
• V9 technical paper 16.4 - the render-sweep methodology.
• custom_recipes/README.md - authoring a layers array from this structure.