sgl-project/sglang · error · ValueError

tokenspeed_mla backend is only supported on Blackwell GPUs (

Error message

tokenspeed_mla backend is only supported on Blackwell GPUs (SM100/SM12x).

What it means

SGLang rejects the tokenspeed_mla attention backend on non-Blackwell hardware. The TokenSpeed MLA kernels are compiled for SM100/SM12x (Blackwell) only, so enabling the backend on Hopper or older GPUs fails during server-args validation.

Source

Thrown at python/sglang/srt/arg_groups/overrides.py:2503

    the view)."""
    if (
        view.attention_backend == "trtllm_mla"
        or view.decode_attention_backend == "trtllm_mla"
    ):
        if not is_blackwell_supported():
            raise ValueError(
                "TRTLLM MLA backend is only supported on Blackwell GPUs (SM100/SM12x). Please use a different backend."
            )
        if view.kv_cache_dtype not in ["fp8_e4m3", "fp4_e2m1", "bf16", "auto"]:
            raise ValueError(
                "TensorRT-LLM MLA backend only supports kv-cache-dtype of fp8_e4m3, fp4_e2m1, bf16, or auto."
            )
    if (
        view.attention_backend == "tokenspeed_mla"
        or view.decode_attention_backend == "tokenspeed_mla"
    ):
        if not is_blackwell_supported():
            raise ValueError(
                "tokenspeed_mla backend is only supported on Blackwell GPUs (SM100/SM12x)."
            )
        if view.kv_cache_dtype not in ["fp8_e4m3"]:
            raise ValueError(
                "tokenspeed_mla backend requires kv-cache-dtype=fp8_e4m3, "
                f"got {view.kv_cache_dtype}."
            )
    return {}


@register_post_process
def _hisparse_validation(view: Any) -> dict:
    """Read-only validation pass: --enable-hisparse constraints (model class,
    radix cache, kv dtype, DSA backends) read the resolved values through the
    view."""
    from sglang.srt.arg_groups.hisparse_hook import validate_hisparse

    validate_hisparse(view)

View on GitHub (pinned to 0132848349)

Solutions

1. Run on a Blackwell GPU (B200/GB200/SM100 or SM12x)
2. Switch to an attention backend supported by your hardware, e.g. --attention-backend flashmla or fa3

Example fix

# before (on H100)
--attention-backend tokenspeed_mla
# after
--attention-backend flashmla --kv-cache-dtype fp8_e4m3

import torch
cc = torch.cuda.get_device_capability()
assert cc >= (10, 0), "tokenspeed_mla requires Blackwell SM100/SM12x"

When it happens

Trigger: Running with --attention-backend tokenspeed_mla or --decode-attention-backend tokenspeed_mla on a GPU whose compute capability is below SM100 (e.g. H100, A100, L40).

Common situations: Developers testing a Blackwell-targeted config on an H100 dev box, or CI runners without B200 access.

Related errors

• tokenspeed_mla backend requires kv-cache-dtype=fp8_e4m3, got
• CuteDSL MLA backend is only supported on Blackwell GPUs (SM1
• --quantization nvfp4_online is supported only on NVIDIA Blac
• TensorRT-LLM MLA backend only supports kv-cache-dtype of fp8
• CuteDSL MLA backend only supports kv-cache-dtype of fp8_e4m3

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/47e526efeedccbb0. Report an issue: GitHub.