sglang-flash-attn3

Pre-built Flash Attention 3 (forward-only) CUDA kernels from sgl-flash-attn, packaged for the HuggingFace kernels library. Requires Hopper (sm_90+).

Kernel source: kernels-community/sgl-flash-attn3

Usage

pip install kernels

from kernels import get_kernel

fa3 = get_kernel("kernels-community/sgl-flash-attn3", revision="v1")

fa3.flash_attn_varlen_func(q, k, v, cu_seqlens_q, cu_seqlens_k, causal=True)
fa3.flash_attn_with_kvcache(q, k_cache, v_cache, cache_seqlens=cache_seqlens, causal=True)
fa3.is_fa3_supported()  # True on H100/H200

SGLang Integration

Entry point: python/sglang/srt/layers/attention/flashattention_backend.py

Original:

from sgl_kernel.flash_attn import flash_attn_varlen_func as flash_attn_varlen_func_fa3
from sgl_kernel.flash_attn import flash_attn_with_kvcache as flash_attn_with_kvcache_fa3

Replace with:

from kernels import get_kernel
_fa3_mod = get_kernel("kernels-community/sgl-flash-attn3", revision="v1")
flash_attn_varlen_func_fa3 = _fa3_mod.flash_attn_varlen_func
flash_attn_with_kvcache_fa3 = _fa3_mod.flash_attn_with_kvcache

Same pattern in 5 other files:

• dual_chunk_flashattention_backend.py
• nsa_backend.py
• xpu_backend.py
• vision.py
• multimodal_gen/runtime/layers/attention/backends/flash_attn.py

Benchmarks

H100 NVL, Qwen2.5-3B-Instruct, FA3. All deltas within noise - zero performance regression.

Credits

• Tri Dao - Flash Attention 3
• SGLang - sgl_kernel FA3 implementation
• HuggingFace - kernel-builder infrastructure