sgl-project/sglang · critical · ValueError

trtllm_mla cannot serve decode context parallelism with spec

Error message

trtllm_mla cannot serve decode context parallelism with speculative decoding: it does not forward the cyclic DCP metadata to its decode kernel and returns no rank-local LSE for the cross-rank merge. Select cutedsl_mla or tokenspeed_mla.

What it means

The trtllm_mla attention backend factory refuses to run when decode context parallelism (DCP) is combined with speculative decoding. The TRTLLM MLA decode kernel neither receives the cyclic DCP metadata nor returns a rank-local LSE needed to merge results across CP ranks, so results would be silently wrong. The registry forces an explicit choice of a backend that supports the combination.

Source

Thrown at python/sglang/srt/layers/attention/attention_registry.py:79

        return FlashInferAttnBackend(
            runner, init_new_workspace=runner.init_new_workspace
        )
    else:
        from sglang.srt.layers.attention.flashinfer_mla_backend import (
            FlashInferMLAAttnBackend,
        )

        return FlashInferMLAAttnBackend(runner)


@register_attention_backend("trtllm_mla")
def create_trtllm_mla_backend(runner):
    if not runner.use_mla_backend:
        raise ValueError("trtllm_mla backend can only be used with MLA models.")
    if get_parallel().dcp_enabled and get_spec().speculative_algorithm is not None:
        _, decode_backend = runner.server_args.get_attention_backends()
        if decode_backend == "trtllm_mla":
            raise ValueError(
                "trtllm_mla cannot serve decode context parallelism with speculative "
                "decoding: it does not forward the cyclic DCP metadata to its decode "
                "kernel and returns no rank-local LSE for the cross-rank merge. "
                "Select cutedsl_mla or tokenspeed_mla."
            )
    from sglang.srt.layers.attention.trtllm_mla_backend import TRTLLMMLABackend

    return TRTLLMMLABackend(runner)


@register_attention_backend("tokenspeed_mla")
def create_tokenspeed_mla_backend(runner):
    if not runner.use_mla_backend:
        raise ValueError("tokenspeed_mla backend can only be used with MLA models.")
    from sglang.srt.layers.attention.tokenspeed_mla_backend import (
        TokenspeedMLABackend,
    )

View on GitHub (pinned to 0132848349)

Solutions

  1. Switch the decode attention backend to cutedsl_mla or tokenspeed_mla (e.g. --attention-backend cutedsl_mla)
  2. Disable speculative decoding (--speculative-algorithm NONE) if DCP is required
  3. Disable DCP if speculative decoding is required

Example fix

# before
--attention-backend trtllm_mla --speculative-algorithm EAGLE  # with DCP enabled
# after
--attention-backend cutedsl_mla --speculative-algorithm EAGLE
Defensive patterns

Strategy: validation

Validate before calling

from sglang.srt.distributed import get_parallel
spec_alg = server_args.speculative_algorithm
_, decode_backend = server_args.get_attention_backends()
if get_parallel().dcp_enabled and spec_alg is not None and decode_backend == "trtllm_mla":
    raise SystemExit("switch decode backend to cutedsl_mla or tokenspeed_mla")

Prevention

When it happens

Trigger: Calling create_trtllm_mla_backend(runner) (or booting a server with --attention-backend trtllm_mla / decode backend trtllm_mla) when get_parallel().dcp_enabled is true and server_args.speculative_algorithm is set, and runner.server_args.get_attention_backends() resolves decode_backend == 'trtllm_mla'.

Common situations: Enabling --speculative-algorithm (e.g. EAGLE/NEXTN) together with DCP on DeepSeek-style MLA models while leaving trtllm_mla as the decode attention backend, or a config default that silently picks trtllm_mla on Hopper+ GPUs.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/eeb796b18b85fa91. Report an issue: GitHub.