sgl-project/sglang · error · ValueError

trtllm_mha backend can only be used with non-MLA models.

Error message

trtllm_mha backend can only be used with non-MLA models.

What it means

The trtllm_mha backend is the mirror of the MLA backends: it only serves non-MLA (standard multi-head attention) models. The factory rejects runners where use_mla_backend is true because TRTLLMHAAttnBackend has no MLA-absorbed path.

Source

Thrown at python/sglang/srt/layers/attention/attention_registry.py:254

def create_flashattention_v4_backend(runner):
    from sglang.srt.layers.attention.flashattention_backend import (
        FlashAttentionBackend,
    )

    return FlashAttentionBackend(runner, fa_impl_ver=4)


@register_attention_backend("cutlass_mla")
def create_cutlass_mla_backend(runner):
    from sglang.srt.layers.attention.cutlass_mla_backend import CutlassMLABackend

    return CutlassMLABackend(runner)


@register_attention_backend("trtllm_mha")
def create_trtllm_mha_backend(runner):
    if runner.use_mla_backend:
        raise ValueError("trtllm_mha backend can only be used with non-MLA models.")
    from sglang.srt.layers.attention.trtllm_mha_backend import TRTLLMHAAttnBackend

    return TRTLLMHAAttnBackend(runner)


@register_attention_backend("hpc_ops")
def create_hpc_ops_backend(runner):
    if runner.use_mla_backend:
        raise ValueError("hpc_ops backend can only be used with non-MLA models.")
    if runner.model_config.is_encoder_decoder:
        raise ValueError(
            "Cross attention is not supported in the hpc_ops attention backend."
        )
    if get_spec().speculative_algorithm is not None:
        raise ValueError(
            "hpc_ops backend does not support speculative decoding for now."
        )
    from sglang.srt.layers.attention.hpc_ops_backend import HPCOpsAttnBackend

View on GitHub (pinned to 0132848349)

Solutions

1. Remove the --attention-backend trtllm_mha override for MLA models
2. Use an MLA-specific backend (trtllm_mla, cutedsl_mla, flashmla, etc.) for MLA checkpoints

Example fix

# before
--model DeepSeek-V3 --attention-backend trtllm_mha
# after
--model DeepSeek-V3 --attention-backend trtllm_mla

if model_runner.use_mla_backend and server_args.attention_backend == "trtllm_mha":
    raise SystemExit("trtllm_mha is for non-MLA models; use trtllm_mla")

def is_mla_model(runner) -> bool:
    return bool(getattr(runner, "use_mla_backend", False))

When it happens

Trigger: Selecting attention backend 'trtllm_mha' while the loaded model is MLA (use_mla_backend True, e.g. DeepSeek V2/V3).

Common situations: Tuning benchmarks on a non-MLA model then switching checkpoints to DeepSeek without updating --attention-backend; or explicitly requesting trtllm_mha expecting it to handle MLA.

Related errors

• tokenspeed_mla backend can only be used with MLA models.
• cutedsl_mla backend can only be used with MLA models.
• hpc_ops backend can only be used with non-MLA models.
• trtllm_mla cannot serve decode context parallelism with spec
• trtllm_mla does not forward the cyclic DCP metadata to its d

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0ae807a1c0d1702a. Report an issue: GitHub.