{"record":{"id":"1760fbb64ae487e4","repo":"sgl-project/sglang","slug":"trtllm-mla-backend-can-only-be-used-with-mla-model","errorCode":null,"errorMessage":"trtllm_mla backend can only be used with MLA models.","messagePattern":"trtllm_mla backend can only be used with MLA models\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/attention_registry.py","lineNumber":75,"sourceCode":"                not hasattr(runner, \"plan_stream_for_flashinfer\")\n                or not runner.plan_stream_for_flashinfer\n            ):\n                runner.plan_stream_for_flashinfer = torch.cuda.Stream()\n        return FlashInferAttnBackend(\n            runner, init_new_workspace=runner.init_new_workspace\n        )\n    else:\n        from sglang.srt.layers.attention.flashinfer_mla_backend import (\n            FlashInferMLAAttnBackend,\n        )\n\n        return FlashInferMLAAttnBackend(runner)\n\n\n@register_attention_backend(\"trtllm_mla\")\ndef create_trtllm_mla_backend(runner):\n    if not runner.use_mla_backend:\n        raise ValueError(\"trtllm_mla backend can only be used with MLA models.\")\n    if get_parallel().dcp_enabled and get_spec().speculative_algorithm is not None:\n        _, decode_backend = runner.server_args.get_attention_backends()\n        if decode_backend == \"trtllm_mla\":\n            raise ValueError(\n                \"trtllm_mla cannot serve decode context parallelism with speculative \"\n                \"decoding: it does not forward the cyclic DCP metadata to its decode \"\n                \"kernel and returns no rank-local LSE for the cross-rank merge. \"\n                \"Select cutedsl_mla or tokenspeed_mla.\"\n            )\n    from sglang.srt.layers.attention.trtllm_mla_backend import TRTLLMMLABackend\n\n    return TRTLLMMLABackend(runner)\n\n\n@register_attention_backend(\"tokenspeed_mla\")\ndef create_tokenspeed_mla_backend(runner):\n    if not runner.use_mla_backend:\n        raise ValueError(\"tokenspeed_mla backend can only be used with MLA models.\")","sourceCodeStart":57,"sourceCodeEnd":93,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/attention_registry.py#L57-L93","documentation":"The trtllm_mla attention backend is only valid for MLA-architecture models (DeepSeek-style multi-head latent attention). create_trtllm_mla_mla checks runner.use_mla_backend at registration time and raises ValueError if the model is not MLA.","triggerScenarios":"Starting sglang with --attention-backend trtllm_mla (or attn_backend='trtllm_mla') on a non-MLA model such as Llama/Qwen/Mistral, where use_mla_backend is False.","commonSituations":"Copy-pasting launch flags from a DeepSeek deployment to a GQA model; forcing a backend override via server_args without checking model architecture.","solutions":["Use trtllm_mla only with MLA models (DeepSeek V2/V3, etc.)","For non-MLA models switch to a supported backend: flashmla for MLA, or flashinfer/fa3/triton for GQA models","Remove the explicit --attention-backend trtllm_mla and let SGLang auto-select the backend for the model"],"exampleFix":"# before\npython -m sglang.launch_server --model qwen/Qwen2.5-7B --attention-backend trtllm_mla\n# after\npython -m sglang.launch_server --model qwen/Qwen2.5-7B --attention-backend flashinfer","handlingStrategy":"validation","validationCode":"from sglang.srt.server_args import ServerArgs\nif server_args.attention_backend == 'trtllm_mla' and not model_config.is_mla:\n    server_args.attention_backend = 'flashinfer'  # or raise early with a clear message","typeGuard":"def backend_ok_for_model(backend: str, use_mla: bool) -> bool:\n    return backend != 'trtllm_mla' or use_mla","tryCatchPattern":null,"preventionTips":["Only set --attention-backend trtllm_mla for DeepSeek-style MLA models","Validate backend-vs-model-architecture at startup in your launcher wrapper","Let SGLang auto-select the backend unless you have a measured reason to override"],"tags":["attention-backend","trtllm","mla","server-args","valueerror"],"backgroundTag":"incompatible-backend-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}