sgl-project/sglang · error · ValueError

GLM DSA with FP8 KV cache on NVIDIA SM120/SM121 supports onl

Error message

GLM DSA with FP8 KV cache on NVIDIA SM120/SM121 supports only flashinfer_sparse_mla, but got {sorted(unsupported)}.

What it means

The inverse constraint of error 327: when running GLM DSA with FP8 KV cache on SM120/SM121, the only supported attention implementation is flashinfer_sparse_mla. The validator rejects any other selected impl for this hardware/model/dtype combination.

Source

Thrown at python/sglang/kernels/ops/attention/flash_mla_sm120.py:599

    uses_flashinfer_sparse_mla = "flashinfer_sparse_mla" in selected
    is_glm_sm12_fp8 = (
        model_arch in _GLM_DSA_MODEL_ARCHS
        and device_sm_major == 12
        and kv_cache_dtype == torch.float8_e4m3fn
        and not _is_hip
    )
    if uses_flashinfer_sparse_mla and not is_glm_sm12_fp8:
        raise ValueError(
            "flashinfer_sparse_mla supports only GLM DSA with FP8 KV cache "
            "on NVIDIA SM120/SM121; "
            f"got model_arch={model_arch!r}, sm_major={device_sm_major}, "
            f"kv_cache_dtype={kv_cache_dtype}, prefill_impl={prefill_impl!r}, "
            f"decode_impl={decode_impl!r}."
        )
    if is_glm_sm12_fp8:
        unsupported = selected - {"flashinfer_sparse_mla"}
        if unsupported:
            raise ValueError(
                "GLM DSA with FP8 KV cache on NVIDIA SM120/SM121 supports "
                "only flashinfer_sparse_mla, "
                f"but got {sorted(unsupported)}."
            )
    return uses_flashinfer_sparse_mla


def flashinfer_sparse_mla_forward(
    q: torch.Tensor,
    kv_cache: torch.Tensor,
    indices: torch.Tensor,
    seq_lens: torch.Tensor,
    workspace_buffer: torch.Tensor,
    *,
    page_size: int,
    kv_cache_dim: int,
    qk_nope_head_dim: int,
    kv_lora_rank: int,

View on GitHub (pinned to 0132848349)

Solutions

1. Set the attention impl to flashinfer_sparse_mla (remove overrides)
2. Or change kv_cache_dtype back to bf16/fp16 if another impl is required
3. Ensure the sparse-mla backend is installed/enabled in the build

Example fix

# before
server_args.decode_impl = "flashmla"  # GLM DSA + fp8 on SM120
# after
server_args.decode_impl = "flashinfer_sparse_mla"

if is_glm_sm12_fp8:
    assert selected <= {'flashinfer_sparse_mla'}, f'unsupported impls for GLM+FP8+SM120: {selected}'

When it happens

Trigger: Running GLM DSA with kv_cache_dtype=fp8_e4m3 on SM120 while selected impls include anything besides flashinfer_sparse_mla.

Common situations: Explicitly overriding prefill/decode impl to fa3/flashmla/triton for GLM on a 50-series GPU; stale config from another model carried over.

Related errors

• flashinfer_sparse_mla supports only GLM DSA with FP8 KV cach
• The requested FlashAttention forward configuration exceeds S
• fp8_blockwise_scaled_mm JIT kernel requires SM120 (Blackwell
• Attention backend '{selected_backend}' is not supported by t
• Ring Attention requires one of the ring-capable backends ({'

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2a7079e3535eeff3. Report an issue: GitHub.