{"record":{"id":"eeb796b18b85fa91","repo":"sgl-project/sglang","slug":"trtllm-mla-cannot-serve-decode-context-parallelism","errorCode":null,"errorMessage":"trtllm_mla cannot serve decode context parallelism with speculative decoding: it does not forward the cyclic DCP metadata to its decode kernel and returns no rank-local LSE for the cross-rank merge. Select cutedsl_mla or tokenspeed_mla.","messagePattern":"trtllm_mla cannot serve decode context parallelism with speculative decoding: it does not forward the cyclic DCP metadata to its decode kernel and returns no rank-local LSE for the cross-rank merge\\. Select cutedsl_mla or tokenspeed_mla\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/layers/attention/attention_registry.py","lineNumber":79,"sourceCode":"        return FlashInferAttnBackend(\n            runner, init_new_workspace=runner.init_new_workspace\n        )\n    else:\n        from sglang.srt.layers.attention.flashinfer_mla_backend import (\n            FlashInferMLAAttnBackend,\n        )\n\n        return FlashInferMLAAttnBackend(runner)\n\n\n@register_attention_backend(\"trtllm_mla\")\ndef create_trtllm_mla_backend(runner):\n    if not runner.use_mla_backend:\n        raise ValueError(\"trtllm_mla backend can only be used with MLA models.\")\n    if get_parallel().dcp_enabled and get_spec().speculative_algorithm is not None:\n        _, decode_backend = runner.server_args.get_attention_backends()\n        if decode_backend == \"trtllm_mla\":\n            raise ValueError(\n                \"trtllm_mla cannot serve decode context parallelism with speculative \"\n                \"decoding: it does not forward the cyclic DCP metadata to its decode \"\n                \"kernel and returns no rank-local LSE for the cross-rank merge. \"\n                \"Select cutedsl_mla or tokenspeed_mla.\"\n            )\n    from sglang.srt.layers.attention.trtllm_mla_backend import TRTLLMMLABackend\n\n    return TRTLLMMLABackend(runner)\n\n\n@register_attention_backend(\"tokenspeed_mla\")\ndef create_tokenspeed_mla_backend(runner):\n    if not runner.use_mla_backend:\n        raise ValueError(\"tokenspeed_mla backend can only be used with MLA models.\")\n    from sglang.srt.layers.attention.tokenspeed_mla_backend import (\n        TokenspeedMLABackend,\n    )\n","sourceCodeStart":61,"sourceCodeEnd":97,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/attention_registry.py#L61-L97","documentation":"The trtllm_mla attention backend factory refuses to run when decode context parallelism (DCP) is combined with speculative decoding. The TRTLLM MLA decode kernel neither receives the cyclic DCP metadata nor returns a rank-local LSE needed to merge results across CP ranks, so results would be silently wrong. The registry forces an explicit choice of a backend that supports the combination.","triggerScenarios":"Calling create_trtllm_mla_backend(runner) (or booting a server with --attention-backend trtllm_mla / decode backend trtllm_mla) when get_parallel().dcp_enabled is true and server_args.speculative_algorithm is set, and runner.server_args.get_attention_backends() resolves decode_backend == 'trtllm_mla'.","commonSituations":"Enabling --speculative-algorithm (e.g. EAGLE/NEXTN) together with DCP on DeepSeek-style MLA models while leaving trtllm_mla as the decode attention backend, or a config default that silently picks trtllm_mla on Hopper+ GPUs.","solutions":["Switch the decode attention backend to cutedsl_mla or tokenspeed_mla (e.g. --attention-backend cutedsl_mla)","Disable speculative decoding (--speculative-algorithm NONE) if DCP is required","Disable DCP if speculative decoding is required"],"exampleFix":"# before\n--attention-backend trtllm_mla --speculative-algorithm EAGLE  # with DCP enabled\n# after\n--attention-backend cutedsl_mla --speculative-algorithm EAGLE","handlingStrategy":"validation","validationCode":"from sglang.srt.distributed import get_parallel\nspec_alg = server_args.speculative_algorithm\n_, decode_backend = server_args.get_attention_backends()\nif get_parallel().dcp_enabled and spec_alg is not None and decode_backend == \"trtllm_mla\":\n    raise SystemExit(\"switch decode backend to cutedsl_mla or tokenspeed_mla\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate backend + DCP + speculative flag combinations in a startup config lint before launching","Pin attention backend explicitly per deployment instead of relying on defaults that change per GPU arch"],"tags":["attention-backend","trtllm-mla","context-parallelism","speculative-decoding","mla","sglang"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}