{"record":{"id":"0ae807a1c0d1702a","repo":"sgl-project/sglang","slug":"trtllm-mha-backend-can-only-be-used-with-non-mla-m","errorCode":null,"errorMessage":"trtllm_mha backend can only be used with non-MLA models.","messagePattern":"trtllm_mha backend can only be used with non-MLA models\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/attention_registry.py","lineNumber":254,"sourceCode":"def create_flashattention_v4_backend(runner):\n    from sglang.srt.layers.attention.flashattention_backend import (\n        FlashAttentionBackend,\n    )\n\n    return FlashAttentionBackend(runner, fa_impl_ver=4)\n\n\n@register_attention_backend(\"cutlass_mla\")\ndef create_cutlass_mla_backend(runner):\n    from sglang.srt.layers.attention.cutlass_mla_backend import CutlassMLABackend\n\n    return CutlassMLABackend(runner)\n\n\n@register_attention_backend(\"trtllm_mha\")\ndef create_trtllm_mha_backend(runner):\n    if runner.use_mla_backend:\n        raise ValueError(\"trtllm_mha backend can only be used with non-MLA models.\")\n    from sglang.srt.layers.attention.trtllm_mha_backend import TRTLLMHAAttnBackend\n\n    return TRTLLMHAAttnBackend(runner)\n\n\n@register_attention_backend(\"hpc_ops\")\ndef create_hpc_ops_backend(runner):\n    if runner.use_mla_backend:\n        raise ValueError(\"hpc_ops backend can only be used with non-MLA models.\")\n    if runner.model_config.is_encoder_decoder:\n        raise ValueError(\n            \"Cross attention is not supported in the hpc_ops attention backend.\"\n        )\n    if get_spec().speculative_algorithm is not None:\n        raise ValueError(\n            \"hpc_ops backend does not support speculative decoding for now.\"\n        )\n    from sglang.srt.layers.attention.hpc_ops_backend import HPCOpsAttnBackend","sourceCodeStart":236,"sourceCodeEnd":272,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/attention_registry.py#L236-L272","documentation":"The trtllm_mha backend is the mirror of the MLA backends: it only serves non-MLA (standard multi-head attention) models. The factory rejects runners where use_mla_backend is true because TRTLLMHAAttnBackend has no MLA-absorbed path.","triggerScenarios":"Selecting attention backend 'trtllm_mha' while the loaded model is MLA (use_mla_backend True, e.g. DeepSeek V2/V3).","commonSituations":"Tuning benchmarks on a non-MLA model then switching checkpoints to DeepSeek without updating --attention-backend; or explicitly requesting trtllm_mha expecting it to handle MLA.","solutions":["Remove the --attention-backend trtllm_mha override for MLA models","Use an MLA-specific backend (trtllm_mla, cutedsl_mla, flashmla, etc.) for MLA checkpoints"],"exampleFix":"# before\n--model DeepSeek-V3 --attention-backend trtllm_mha\n# after\n--model DeepSeek-V3 --attention-backend trtllm_mla","handlingStrategy":"validation","validationCode":"if model_runner.use_mla_backend and server_args.attention_backend == \"trtllm_mha\":\n    raise SystemExit(\"trtllm_mha is for non-MLA models; use trtllm_mla\")","typeGuard":"def is_mla_model(runner) -> bool:\n    return bool(getattr(runner, \"use_mla_backend\", False))","tryCatchPattern":null,"preventionTips":["Treat trtllm_mha vs trtllm_mla as model-dependent, never a global default","Add a config check that rejects backend names not in the model's supported set"],"tags":["attention-backend","trtllm-mha","mla","model-arch-mismatch","sglang"],"backgroundTag":"backend-model-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}