Files
ULT7RAandCopilot 82bdde669f FLUXNATION: Fused spike attention + step cache CUDA kernel for FLUX.1
Fused FP8 CUDA kernel for FLUX.1 SingleStreamBlock with spike attention
and step caching. Replaces ComfyUI's entire SingleStreamBlock forward pass
with a single torch.ops call — modulation, dual GEMMs, QKV, RoPE, attention,
gating and residual all fused. 30%+ faster than stock ComfyUI on RTX 4090.

Spike attention: block-sparse Triton kernel that scores attention blocks
by dot-product similarity and keeps only the top 45% after image structure
forms. Step caching replays cached attention output on alternating spike
steps for zero attention compute every other step.

Works with all 38 ComfyUI samplers at any step count.
FP16 ports included for 10-series, 20-series, and 30-series GPUs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-04-05 17:56:53 -06:00
..