sgl-project/sglang · error · ValueError

trtllm_mla backend can only be used with MLA models.

Error message

trtllm_mla backend can only be used with MLA models.

What it means

The trtllm_mla attention backend is only valid for MLA-architecture models (DeepSeek-style multi-head latent attention). create_trtllm_mla_mla checks runner.use_mla_backend at registration time and raises ValueError if the model is not MLA.

Source

Thrown at python/sglang/srt/layers/attention/attention_registry.py:75

                not hasattr(runner, "plan_stream_for_flashinfer")
                or not runner.plan_stream_for_flashinfer
            ):
                runner.plan_stream_for_flashinfer = torch.cuda.Stream()
        return FlashInferAttnBackend(
            runner, init_new_workspace=runner.init_new_workspace
        )
    else:
        from sglang.srt.layers.attention.flashinfer_mla_backend import (
            FlashInferMLAAttnBackend,
        )

        return FlashInferMLAAttnBackend(runner)


@register_attention_backend("trtllm_mla")
def create_trtllm_mla_backend(runner):
    if not runner.use_mla_backend:
        raise ValueError("trtllm_mla backend can only be used with MLA models.")
    if get_parallel().dcp_enabled and get_spec().speculative_algorithm is not None:
        _, decode_backend = runner.server_args.get_attention_backends()
        if decode_backend == "trtllm_mla":
            raise ValueError(
                "trtllm_mla cannot serve decode context parallelism with speculative "
                "decoding: it does not forward the cyclic DCP metadata to its decode "
                "kernel and returns no rank-local LSE for the cross-rank merge. "
                "Select cutedsl_mla or tokenspeed_mla."
            )
    from sglang.srt.layers.attention.trtllm_mla_backend import TRTLLMMLABackend

    return TRTLLMMLABackend(runner)


@register_attention_backend("tokenspeed_mla")
def create_tokenspeed_mla_backend(runner):
    if not runner.use_mla_backend:
        raise ValueError("tokenspeed_mla backend can only be used with MLA models.")

View on GitHub (pinned to 0132848349)

Solutions

1. Use trtllm_mla only with MLA models (DeepSeek V2/V3, etc.)
2. For non-MLA models switch to a supported backend: flashmla for MLA, or flashinfer/fa3/triton for GQA models
3. Remove the explicit --attention-backend trtllm_mla and let SGLang auto-select the backend for the model

Example fix

# before
python -m sglang.launch_server --model qwen/Qwen2.5-7B --attention-backend trtllm_mla
# after
python -m sglang.launch_server --model qwen/Qwen2.5-7B --attention-backend flashinfer

from sglang.srt.server_args import ServerArgs
if server_args.attention_backend == 'trtllm_mla' and not model_config.is_mla:
    server_args.attention_backend = 'flashinfer'  # or raise early with a clear message

def backend_ok_for_model(backend: str, use_mla: bool) -> bool:
    return backend != 'trtllm_mla' or use_mla

When it happens

Trigger: Starting sglang with --attention-backend trtllm_mla (or attn_backend='trtllm_mla') on a non-MLA model such as Llama/Qwen/Mistral, where use_mla_backend is False.

Common situations: Copy-pasting launch flags from a DeepSeek deployment to a GQA model; forcing a backend override via server_args without checking model architecture.

Related errors

• MiniMax-H3 ring parallelism requires the FlashAttention back
• TensorRT-LLM MLA backend only supports kv-cache-dtype of fp8
• speculative_eagle_topk > 1 with page_size > 1 is only suppor
• trtllm_mla cannot serve decode context parallelism with spec
• tokenspeed_mla backend can only be used with MLA models.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1760fbb64ae487e4. Report an issue: GitHub.