sgl-project/sglang · error · AssertionError

Decode attention backend for Kimi-K3 DCP must be 'cutedsl_ml

Error message

Decode attention backend for Kimi-K3 DCP must be 'cutedsl_mla' or 'tokenspeed_mla', got {decode_backend!r}.

What it means

Kimi-K3 DCP requires the decode attention backend to be either cutedsl_mla or tokenspeed_mla. The code handles each explicitly and any other value falls into an else branch raising AssertionError — effectively a programming/config invariant violation rather than a normal user path.

Source

Thrown at python/sglang/srt/arg_groups/overrides.py:673

                decode_attention_backend="cutedsl_mla",
            )
        elif decode_backend == "tokenspeed_mla":
            logger.info(
                "Kimi-K3 DCP overrides attention backends: "
                f"prefill={prefill_backend!r}, decode={decode_backend!r} -> "
                "'tokenspeed_mla'."
            )
            logger.info(
                "Kimi-K3 DCP with tokenspeed mla backend overrides KV cache dtype: "
                f"{cfg.kv_cache_dtype!r} -> 'fp8_e4m3'."
            )
            overrides.update(
                prefill_attention_backend="tokenspeed_mla",
                decode_attention_backend="tokenspeed_mla",
                kv_cache_dtype="fp8_e4m3",
            )
        else:
            raise AssertionError(
                f"Decode attention backend for Kimi-K3 DCP must be 'cutedsl_mla' or 'tokenspeed_mla', got {decode_backend!r}."
            )

        if cfg.dcp_replicate_q_proj is None:
            logger.info("Kimi-K3 DCP enables replicated Q projection by default.")
            overrides["dcp_replicate_q_proj"] = True

        device_name = get_device_name()
        dcp_comm_backend = "fi_a2a" if is_mnnvl_fabric_device() else "a2a"
        logger.info(
            "Kimi-K3 DCP selects communication backend on "
            f"{device_name!r}: {cfg.dcp_comm_backend!r} -> "
            f"{dcp_comm_backend!r}."
        )
        overrides["dcp_comm_backend"] = dcp_comm_backend
        return overrides

    if not (is_sm100_supported() and get_device_sm() in (100, 103)):

View on GitHub (pinned to 0132848349)

Solutions

Set --decode-attention-backend cutedsl_mla or tokenspeed_mla
Or remove the explicit flag to let Kimi-K3 defaults apply
Check other override hooks/scripts aren't rewriting decode_attention_backend

Example fix

# before
--decode-attention-backend flashinfer
# after
--decode-attention-backend cutedsl_mla

Defensive patterns

Strategy: validation

Validate before calling

assert decode_backend in ('cutedsl_mla', 'tokenspeed_mla'), f'bad decode backend for Kimi-K3 DCP: {decode_backend}'

Try / catch

except AssertionError as e:
    if 'cutedsl_mla' in str(e): server_args.decode_attention_backend = 'cutedsl_mla'; retry()
    raise

Prevention

Don't hardcode attention backends across models in shared scripts
Let Kimi-K3 defaults choose the decode backend

When it happens

Trigger: cfg.decode_attention_backend set to a third value while Kimi-K3 DCP handling routes into _kimi_k3_overrides without matching either branch (e.g. a stale custom backend string injected by an override hook).

Common situations: Custom launch wrappers forcing --decode-attention-backend flashinfer/trtllm_mla; version skew where a hook writes an unexpected backend value before this check.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2311bd20282a9819. Report an issue: GitHub.