sgl-project/sglang · error · ValueError

flashinfer_sparse_mla supports only GLM DSA with FP8 KV cach

Error message

flashinfer_sparse_mla supports only GLM DSA with FP8 KV cache on NVIDIA SM120/SM121; got model_arch={model_arch!r}, sm_major={device_sm_major}, kv_cache_dtype={kv_cache_dtype}, prefill_impl={prefill_impl!r}, decode_impl={decode_impl!r}.

What it means

The attention backend selector detected that flashinfer_sparse_mla was requested (via prefill/decode impl choice), but this implementation is only valid for GLM DSA-architecture models running on NVIDIA SM120/SM121 GPUs with an FP8 (float8_e4m3fn) KV cache. Any other combination is rejected at config validation time.

Source

Thrown at python/sglang/kernels/ops/attention/flash_mla_sm120.py:589

def _validate_flashinfer_sparse_mla_backend(
    *,
    model_arch: str,
    device_sm_major: int,
    kv_cache_dtype: torch.dtype,
    prefill_impl: str,
    decode_impl: str,
) -> bool:
    selected = {prefill_impl, decode_impl}
    uses_flashinfer_sparse_mla = "flashinfer_sparse_mla" in selected
    is_glm_sm12_fp8 = (
        model_arch in _GLM_DSA_MODEL_ARCHS
        and device_sm_major == 12
        and kv_cache_dtype == torch.float8_e4m3fn
        and not _is_hip
    )
    if uses_flashinfer_sparse_mla and not is_glm_sm12_fp8:
        raise ValueError(
            "flashinfer_sparse_mla supports only GLM DSA with FP8 KV cache "
            "on NVIDIA SM120/SM121; "
            f"got model_arch={model_arch!r}, sm_major={device_sm_major}, "
            f"kv_cache_dtype={kv_cache_dtype}, prefill_impl={prefill_impl!r}, "
            f"decode_impl={decode_impl!r}."
        )
    if is_glm_sm12_fp8:
        unsupported = selected - {"flashinfer_sparse_mla"}
        if unsupported:
            raise ValueError(
                "GLM DSA with FP8 KV cache on NVIDIA SM120/SM121 supports "
                "only flashinfer_sparse_mla, "
                f"but got {sorted(unsupported)}."
            )
    return uses_flashinfer_sparse_mla


def flashinfer_sparse_mla_forward(

View on GitHub (pinned to 0132848349)

Solutions

1. Switch attention backend (prefill/decode impl) away from flashinfer_sparse_mla
2. Or use a GLM DSA model with --kv-cache-dtype fp8_e4m3 on SM120/SM121 hardware
3. Verify device compute capability (needs SM major 12)

Example fix

# before
server_args.prefill_impl = "flashinfer_sparse_mla"  # bf16 cache, non-GLM model
# after
server_args.prefill_impl = "fa3"  # or set --kv-cache-dtype fp8_e4m3 with GLM DSA

assert not (impl == 'flashinfer_sparse_mla' and not (arch in GLM_DSA_ARCHS and sm_major == 12 and kv_dtype == torch.float8_e4m3fn)), 'flashinfer_sparse_mla requires GLM DSA + FP8 KV + SM120'

def sparse_mla_supported(model_arch: str, sm_major: int, kv_dtype: torch.dtype) -> bool:
    return (model_arch in _GLM_DSA_MODEL_ARCHS and sm_major == 12
            and kv_dtype == torch.float8_e4m3fn)

try:
    backend = AttentionBackend(...)
except ValueError as e:
    if 'flashinfer_sparse_mla' in str(e):
        fallback_to_default_backend()

When it happens

Trigger: Setting attention impl to flashinfer_sparse_mla while model_arch is not in the GLM DSA set, or device SM major != 12, or kv_cache_dtype != float8_e4m3fn, or on HIP.

Common situations: Defaulting a non-GLM model to sparse MLA on a 50-series (SM120) GPU; using bf16 KV cache with sparse MLA; selecting the backend on AMD.

Related errors

• GLM DSA with FP8 KV cache on NVIDIA SM120/SM121 supports onl
• The requested FlashAttention forward configuration exceeds S
• fp8_blockwise_scaled_mm JIT kernel requires SM120 (Blackwell
• Attention backend '{selected_backend}' is not supported by t
• Ring Attention requires one of the ring-capable backends ({'

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d599955ef9a4c628. Report an issue: GitHub.