digital-garbage and Claude Sonnet 5
1e4553e658
fix: H3 attention decoupling was scaling K magnitude, which barely moves softmax
...
Live 0-1 strength sweep came back flat -- confirmed the mechanism, not just the
value, was wrong. H3's Q/K get RMSNorm'd before attention, so scaling a key
toward zero pushes its logit toward the *mean* of an already near-zero-centered
distribution, not toward excluded. Switched to what actually suppresses a
softmax entry: an additive negative bias on the targeted logits, still passed
through the same per-query-group split (still no S^2 mask, still same total
FLOPs -- the bias is a per-key vector broadcast over the group's queries).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com >
2026-09-05 02:23:30 +03:00
..
2026-08-22 23:18:36 +03:00
2026-08-03 11:48:58 +03:00
2026-08-11 03:53:11 +03:00
2026-06-21 17:14:06 +03:00
2026-08-25 13:36:23 +03:00
2026-09-04 18:10:55 +03:00
2026-09-04 23:01:27 +03:00
2026-08-25 13:34:08 +03:00
2026-08-25 14:06:52 +03:00
2026-08-26 23:10:27 +03:00
2026-07-31 18:10:10 +03:00
2026-08-11 23:47:51 +03:00
2026-09-04 19:56:38 +03:00
2026-08-11 23:39:47 +03:00
2026-08-23 04:35:41 +03:00
2026-08-25 17:53:45 +03:00
2026-08-25 17:45:13 +03:00
2026-09-03 23:22:35 +03:00
2026-08-25 13:32:44 +03:00
2026-08-23 04:32:44 +03:00
2026-08-04 22:48:15 +03:00
2026-08-26 23:39:32 +03:00
2026-08-23 08:12:43 +03:00
2026-09-05 02:23:30 +03:00
2026-08-27 00:38:46 +03:00
2026-08-25 21:48:28 +03:00
2026-08-21 00:03:43 +03:00
2026-08-25 21:48:28 +03:00
2026-09-05 00:03:56 +03:00
2026-09-05 01:22:23 +03:00
2026-09-03 16:58:34 +03:00
2026-08-27 00:16:43 +03:00
2026-08-23 04:17:41 +03:00
2026-07-29 17:28:35 +03:00
2026-06-24 19:04:32 +03:00
2026-08-11 03:53:11 +03:00
2026-08-23 08:12:43 +03:00
2026-08-25 14:42:05 +03:00
2026-07-15 21:55:22 +03:00
2026-08-26 23:39:32 +03:00
2026-08-11 22:36:38 +03:00
2026-08-03 11:48:58 +03:00
2026-08-27 14:32:20 +03:00
2026-08-23 09:12:22 +03:00
2026-08-26 23:20:31 +03:00
2026-08-23 03:52:45 +03:00
2026-08-26 23:10:27 +03:00
2026-08-06 19:41:18 +03:00
2026-08-26 17:48:07 +03:00
2026-06-10 16:50:19 +03:00
2026-08-23 00:08:57 +03:00
2026-08-26 23:10:27 +03:00
2026-08-23 09:55:21 +03:00
2026-09-03 18:46:30 +03:00
2026-07-31 18:10:10 +03:00
2026-08-27 14:32:20 +03:00
2026-08-20 19:14:24 +03:00
2026-06-24 17:14:25 +03:00
2026-08-11 23:39:47 +03:00
2026-08-12 00:05:52 +03:00
2026-08-23 03:25:10 +03:00
2026-08-27 14:41:10 +03:00
2026-07-06 05:26:21 +03:00
2026-08-20 23:36:01 +03:00
2026-08-25 14:32:11 +03:00
2026-08-25 13:48:35 +03:00
2026-08-25 13:26:43 +03:00
2026-08-25 13:54:01 +03:00
2026-08-27 00:32:17 +03:00
2026-09-03 14:41:00 +03:00
2026-09-03 14:47:26 +03:00
2026-09-03 18:56:28 +03:00
2026-08-25 14:34:26 +03:00