Files
hao-ai-lab-FastVideo/scripts/distill
Aryan Kumar 72f4928065 [feat]: dense FP4 attention QAD for Pika H3 Flex (FA4 NVFP4 Q/K numerics)
fp4_dense_qat routes FlashAttention layers through the FP4 training kernel with every
key block listed (Q/K NVFP4, P and V BF16, straight-through gradients), matching the
FlashAttention-4 NVFP4 deployment; qad.fp4_attention enables it in the Flex profile.
check_fp4_attention_parity.py compares it with the real FA4 NVFP4 kernel on a GPU.
2026-10-31 21:26:55 -07:00
..