{"record":{"id":"a2ea5f67a54f0d90","repo":"sgl-project/sglang","slug":"trtllm-mla-does-not-forward-the-cyclic-dcp-metadat","errorCode":null,"errorMessage":"trtllm_mla does not forward the cyclic DCP metadata to its decode kernel and returns no rank-local LSE for the cross-rank merge; select cutedsl_mla or tokenspeed_mla for a DCP target-verify run","messagePattern":"trtllm_mla does not forward the cyclic DCP metadata to its decode kernel and returns no rank-local LSE for the cross-rank merge; select cutedsl_mla or tokenspeed_mla for a DCP target-verify run","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/attention/trtllm_mla_backend.py","lineNumber":859,"sourceCode":"        kv_cache: torch.Tensor,\n        block_tables: torch.Tensor,\n        seq_lens: torch.Tensor,\n        max_seq_len: int,\n        layer: RadixAttention,\n        *,\n        causal_seqs: Optional[torch.Tensor] = None,\n        cp_world: int = 1,\n        cp_rank: int = 0,\n        return_lse: bool = False,\n    ) -> torch.Tensor:\n        \"\"\"Hook for subclasses to swap the decode/spec-verify kernel.\n\n        The DCP arguments belong to the hook contract because forward_extend\n        passes them on the DCP target-verify path. This implementation does not\n        forward them to the kernel and returns no LSE, so only the DCP-capable\n        subclasses serve them.\"\"\"\n        if cp_world > 1 or return_lse:\n            raise NotImplementedError(\n                \"trtllm_mla does not forward the cyclic DCP metadata to its \"\n                \"decode kernel and returns no rank-local LSE for the cross-rank \"\n                \"merge; select cutedsl_mla or tokenspeed_mla for a DCP \"\n                \"target-verify run\"\n            )\n\n        # Scale computation for TRTLLM MLA kernel BMM1 operation:\n        # The final BMM1 scale is computed as: q_scale * k_scale * softmax_scale\n        # Scale components:\n        # - q_scale: Query scaling factor (set to 1.0 for both FP16/FP8 paths)\n        # - k_scale: Key scaling factor from model checkpoint. Only applied when KV cache\n        #   stores FP8-quantized values, to compensate for the quantization scaling.\n        #   For BF16/FP16 KV cache, k_scale must be 1.0 since values are unscaled.\n        # - softmax_scale: Attention softmax scaling = 1/sqrt(head_dim), pre-computed as layer.scaling\n        bmm1_scale = self._compute_decode_bmm1_scale(layer)\n        seq_lens_i32 = (\n            seq_lens if seq_lens.dtype == torch.int32 else seq_lens.to(torch.int32)\n        )","sourceCodeStart":841,"sourceCodeEnd":877,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/attention/trtllm_mla_backend.py#L841-L877","documentation":"trtllm_mla's decode path does not accept the cyclic-context-parallel (DCP) metadata arguments, and it never returns the rank-local LSE needed for the cross-rank merge in DCP target-verify runs. When forward_extend/forward_decode is invoked with cp_world > 1 or return_lse=True, the backend raises NotImplementedError rather than silently producing wrong results. The docstring states only DCP-capable subclasses (cutedsl_mla, tokenspeed_mla) can serve these calls.","triggerScenarios":"Running DeepSeek-style MTP/DCP target-verify (--cp-size > 1, i.e. cp_world > 1) or requesting return_lse=True while the attention backend is set to trtllm_mla; _run_decode_kernel checks these hook-contract arguments and raises immediately.","commonSituations":"Enabling context parallelism or speculative target-verify on a multi-GPU node with --attention-backend trtllm_mla; mixing DCP configurations copied from a cutedsl_mla/tokenspeed_mla deployment into a trtllm_mla run.","solutions":["Switch the MLA attention backend: use --attention-backend cutedsl_mla or tokenspeed_mla for DCP target-verify workloads.","Disable context parallelism (set cp_size=1) and do not request return_lse if you must keep trtllm_mla.","If you need trtllm_mla under DCP, implement forwarding of the cyclic DCP metadata into the decode kernel plus rank-local LSE output in a subclass and override _run_decode_kernel."],"exampleFix":"# before\npython -m sglang.launch_server --model deepseek-ai/DeepSeek-V3 --cp-size 2 --attention-backend trtllm_mla\n# after\npython -m sglang.launch_server --model deepseek-ai/DeepSeek-V3 --cp-size 2 --attention-backend cutedsl_mla","handlingStrategy":"validation","validationCode":"from sglang.srt.server_args import ServerArgs\nDCP_MLA_BACKENDS = {\"cutedsl_mla\", \"tokenspeed_mla\"}\nargs = parse_server_args()\nif getattr(args, \"cp_size\", 1) > 1 and args.attention_backend not in DCP_MLA_BACKENDS:\n    raise SystemExit(f\"cp_size>1 requires one of {DCP_MLA_BACKENDS}, got {args.attention_backend}\")","typeGuard":null,"tryCatchPattern":"try:\n    out = attn_backend.forward_decode(...)\nexcept NotImplementedError as e:\n    if \"DCP\" in str(e) or \"return_lse\" in str(e):\n        raise SystemExit(\"switch to cutedsl_mla/tokenspeed_mla for DCP runs\")\n    raise","preventionTips":["Pin the attention backend explicitly instead of relying on defaults when using --cp-size.","Keep a compatibility matrix of (cp_size, speculative mode) vs supported MLA backends.","Fail fast at startup: validate server args before model load rather than at the first decode step."],"tags":["sglang","attention-backend","context-parallelism","not-implemented","mla","speculative-decoding"],"backgroundTag":"unsupported-backend-feature","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}