Files
kgilper-krea-reference/docs/deepstack-layers/SCORING.md
T
Kevin Gilper 07d2f0b763 Determine and document what each deepstack layer carries
New docs/deepstack-layers/ is the authoritative, reproducible account of
the 12 layers values, at three separated confidence levels:

1. Structure (verified from code): the values are gains on 12 equal
   deepstack chunks - features from 12 successive Qwen3-VL encoder-depth
   bands. Shown in kg_krea_v9/clip_hooks.py (the deepstack feature list)
   and conditioning.py (the 12-way width split).

2. Semantics (convergent evidence): analyze_tables.py reads the live
   tables and shows the four independently-tuned appearance tables
   (style/palette/material/lighting) agree chunk-for-chunk - 0-4 carry
   structure (monotonic depth ramp, all suppress), 5-6 transition, 7/8/10
   carry appearance (all spike, chunk 8 strongest, then 10, then 7). Four
   tables tuned for four different jobs converging is the measurement
   encoded as data, not an assertion.

3. Precise response curve (fresh measurement): generate_sweep.py is a
   turnkey single-chunk-spike sweep (the paper's section 16.4 method),
   emitting each spike as a V10 custom recipe validated against the real
   node code. Dry-run works anywhere; rendering needs the V10 nodes on a
   ComfyUI box (single-chunk tables are not V9-expressible), so the
   measured-verdict table is staged pending a V10-equipped render box.
   SCORING.md defines the rubric.

Cross-linked from technical paper 5.2, the recipe guide, and
custom_recipes/README. Tooling lives under docs/ (excluded from registry
packs by .comfyignore). sweep_out/ is gitignored.
2026-07-03 00:59:35 -04:00

2.1 KiB

Scoring the deepstack sweep

After generate_sweep.py --server ... renders the grid, score every spike render against the control (<ref>__sweep-control.png) to determine what each chunk carries. The control and the spike share prompt, seed, reference, strength, pooled channel, and all other chunks - so any visible difference is attributable to the spiked chunk alone.

Procedure

For each reference (subject, palette) and each chunk L:

  1. Open <ref>__sweep-control.png and <ref>__sweep-chunk-LL-gain-4.png side by side.

  2. Judge which single aspect changed most relative to the control, and how strongly (none / subtle / clear / dominant):

    Aspect What to look for
    structure subject shape, pose, layout, spatial composition shifting toward the reference
    palette overall color cast, hue relationships moving toward the reference
    texture surface/material grain, finish, micro-detail
    lighting light direction, contrast, glow, shadow, mood
    (none) no meaningful change from control
  3. Record: chunk L -> <aspect> (<strength>). If two aspects move together, record both with the dominant one first.

Recording

Fill the verdict table in README.md - one row per chunk, with the verdict from each reference and a consensus column. A determination is "solidified" when:

  • both references agree on each chunk's dominant aspect, and
  • the measured structure->transition->appearance ordering matches the convergent-table prediction (chunks 0-4 structure, 5-6 transition, 7/8/10 appearance with 8 strongest), OR any disagreement is documented with its consequence for the tables.

If a scoring pass is done by an agent, have a second independent agent score the same grid blind and reconcile; note both in the record.

Optional: multi-level confirmation

Re-run with --gains 2 4 6 to confirm each chunk's response is monotonic in gain (a real carrier gets stronger with a bigger spike; noise does not). Note the gain at which each chunk's effect first becomes clearly visible.