sgl-project/sglang · error · ValueError

tokenspeed_mla backend can only be used with MLA models.

Error message

tokenspeed_mla backend can only be used with MLA models.

What it means

The tokenspeed_mla attention backend is only implemented for MLA (multi-head latent attention) model architectures. The factory checks runner.use_mla_backend and rejects non-MLA models before constructing TokenspeedMLABackend, because the kernels it dispatches to assume absorbed MLA KV projections.

Source

Thrown at python/sglang/srt/layers/attention/attention_registry.py:93

        raise ValueError("trtllm_mla backend can only be used with MLA models.")
    if get_parallel().dcp_enabled and get_spec().speculative_algorithm is not None:
        _, decode_backend = runner.server_args.get_attention_backends()
        if decode_backend == "trtllm_mla":
            raise ValueError(
                "trtllm_mla cannot serve decode context parallelism with speculative "
                "decoding: it does not forward the cyclic DCP metadata to its decode "
                "kernel and returns no rank-local LSE for the cross-rank merge. "
                "Select cutedsl_mla or tokenspeed_mla."
            )
    from sglang.srt.layers.attention.trtllm_mla_backend import TRTLLMMLABackend

    return TRTLLMMLABackend(runner)


@register_attention_backend("tokenspeed_mla")
def create_tokenspeed_mla_backend(runner):
    if not runner.use_mla_backend:
        raise ValueError("tokenspeed_mla backend can only be used with MLA models.")
    from sglang.srt.layers.attention.tokenspeed_mla_backend import (
        TokenspeedMLABackend,
    )

    return TokenspeedMLABackend(runner)


@register_attention_backend("cutedsl_mla")
def create_cutedsl_mla_backend(runner):
    if not runner.use_mla_backend:
        raise ValueError("cutedsl_mla backend can only be used with MLA models.")
    from sglang.srt.layers.attention.cutedsl_mla_backend import CuteDslMLABackend

    return CuteDslMLABackend(runner)


@register_attention_backend("aiter")
def create_aiter_backend(runner):

View on GitHub (pinned to 0132848349)

Solutions

  1. Remove the explicit --attention-backend tokenspeed_mla and let SGLang auto-select a backend for the model
  2. Switch to an MLA model (DeepSeek-V2/V3 family) if you specifically want tokenspeed_mla

Example fix

# before
--model-model-path qwen2.5-7b --attention-backend tokenspeed_mla
# after
--model-model-path qwen2.5-7b  # let auto-selection pick a GQA backend
Defensive patterns

Strategy: validation

Validate before calling

if not model_runner.use_mla_backend and server_args.attention_backend == "tokenspeed_mla":
    raise SystemExit("tokenspeed_mla requires an MLA model; remove the override")

Type guard

def is_mla_model(runner) -> bool:
    return bool(getattr(runner, "use_mla_backend", False))

Prevention

When it happens

Trigger: Creating a server/model runner with attention backend 'tokenspeed_mla' for a model whose architecture does not set use_mla_backend (any GQA/MHA model such as Llama, Qwen).

Common situations: Copying a launch flag from a DeepSeek/V3 setup to a non-MLA model, or explicitly forcing --attention-backend tokenspeed_mla on a standard transformer.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1f2bd480c921bf78. Report an issue: GitHub.