{"record":{"id":"1f2bd480c921bf78","repo":"sgl-project/sglang","slug":"tokenspeed-mla-backend-can-only-be-used-with-mla-m","errorCode":null,"errorMessage":"tokenspeed_mla backend can only be used with MLA models.","messagePattern":"tokenspeed_mla backend can only be used with MLA models\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/attention_registry.py","lineNumber":93,"sourceCode":"        raise ValueError(\"trtllm_mla backend can only be used with MLA models.\")\n    if get_parallel().dcp_enabled and get_spec().speculative_algorithm is not None:\n        _, decode_backend = runner.server_args.get_attention_backends()\n        if decode_backend == \"trtllm_mla\":\n            raise ValueError(\n                \"trtllm_mla cannot serve decode context parallelism with speculative \"\n                \"decoding: it does not forward the cyclic DCP metadata to its decode \"\n                \"kernel and returns no rank-local LSE for the cross-rank merge. \"\n                \"Select cutedsl_mla or tokenspeed_mla.\"\n            )\n    from sglang.srt.layers.attention.trtllm_mla_backend import TRTLLMMLABackend\n\n    return TRTLLMMLABackend(runner)\n\n\n@register_attention_backend(\"tokenspeed_mla\")\ndef create_tokenspeed_mla_backend(runner):\n    if not runner.use_mla_backend:\n        raise ValueError(\"tokenspeed_mla backend can only be used with MLA models.\")\n    from sglang.srt.layers.attention.tokenspeed_mla_backend import (\n        TokenspeedMLABackend,\n    )\n\n    return TokenspeedMLABackend(runner)\n\n\n@register_attention_backend(\"cutedsl_mla\")\ndef create_cutedsl_mla_backend(runner):\n    if not runner.use_mla_backend:\n        raise ValueError(\"cutedsl_mla backend can only be used with MLA models.\")\n    from sglang.srt.layers.attention.cutedsl_mla_backend import CuteDslMLABackend\n\n    return CuteDslMLABackend(runner)\n\n\n@register_attention_backend(\"aiter\")\ndef create_aiter_backend(runner):","sourceCodeStart":75,"sourceCodeEnd":111,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/attention_registry.py#L75-L111","documentation":"The tokenspeed_mla attention backend is only implemented for MLA (multi-head latent attention) model architectures. The factory checks runner.use_mla_backend and rejects non-MLA models before constructing TokenspeedMLABackend, because the kernels it dispatches to assume absorbed MLA KV projections.","triggerScenarios":"Creating a server/model runner with attention backend 'tokenspeed_mla' for a model whose architecture does not set use_mla_backend (any GQA/MHA model such as Llama, Qwen).","commonSituations":"Copying a launch flag from a DeepSeek/V3 setup to a non-MLA model, or explicitly forcing --attention-backend tokenspeed_mla on a standard transformer.","solutions":["Remove the explicit --attention-backend tokenspeed_mla and let SGLang auto-select a backend for the model","Switch to an MLA model (DeepSeek-V2/V3 family) if you specifically want tokenspeed_mla"],"exampleFix":"# before\n--model-model-path qwen2.5-7b --attention-backend tokenspeed_mla\n# after\n--model-model-path qwen2.5-7b  # let auto-selection pick a GQA backend","handlingStrategy":"validation","validationCode":"if not model_runner.use_mla_backend and server_args.attention_backend == \"tokenspeed_mla\":\n    raise SystemExit(\"tokenspeed_mla requires an MLA model; remove the override\")","typeGuard":"def is_mla_model(runner) -> bool:\n    return bool(getattr(runner, \"use_mla_backend\", False))","tryCatchPattern":null,"preventionTips":["Check use_mla_backend on the model config before setting MLA-only backend flags","Keep per-model launch scripts instead of one shared script with hardcoded backends"],"tags":["attention-backend","tokenspeed-mla","mla","model-arch-mismatch","sglang"],"backgroundTag":"backend-model-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}