Files
kgilper-krea-reference/docs
Kevin Gilper aa8dc49f72 Document the authoritative design basis (Krea 2 report + UniFusion) for the layers
Researched how Krea 2's 12-layer text conditioning is designed, from
authoritative sources, so the tables can be improved against the real design:

- Krea 2 Technical Report: the 12 taps are a learned "shallow attention layer
  that aggregates hidden features across layers" for "coarse-to-fine text
  representations," explicitly inspired by UniFusion.
- UniFusion (the method): Layerwise Attention Pooling over "every third layer"
  (matches hidden_states[2,5,8,...,35]); empirically favors EARLY-TO-MIDDLE
  layers, with the penultimate contributing LEAST.
- This is misaligned with the shipped tables, which spike the deep taps:
  the table's biggest spike (tap 8/layer 26) is only the model's #5, it
  suppresses tap 4/layer 14 (the model's #3 and a UniFusion-favored layer),
  and over-weights tap 11/layer 35 (penultimate, ranked nearly last). The
  model's own learned projector (Stage 0) corroborates.

README section 3.5 adds the sourced design basis and three improvement
directions (track the learned weighting, differentiate recipes along the
coarse-to-fine axis, and add per-tap magnitude normalization so early-layer
gains actually bite). Full research + per-tap comparison in local_records.
2026-07-03 08:28:13 -04:00
..