[perf] FastH3 two-Spark: gather packed FP4 inputs before head projections
Opt-in h3_vsa_fp4.sp_input_gather (default false; requires sp_projection_overlap) gathers the already-packed FP4 inputs and rotary tables, then computes each rank's assigned Q/K/V heads locally; the gate projection overlaps the smaller transfer. Supported on the measured GB10/cuDNN shape only; other shapes keep the existing exchange. Enabled in the Spark-pair example. T2AV on 2 Sparks: 19.35 -> 18.76 s/clip (merge gate check, same SP reduction order rules). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 5.5
parent
6397822a5b
commit
be8a13b749
@@ -52,6 +52,7 @@ generator:
|
||||
sp_pooled_key_mean: true
|
||||
sp_direct_keys: true
|
||||
sp_projection_overlap: true
|
||||
sp_input_gather: true
|
||||
VSA_sparsity: 0.8
|
||||
VSA_tile_size: 64
|
||||
h3_sequential_load: false
|
||||
|
||||
Reference in New Issue
Block a user