sgl-project/sglang · error · RuntimeError

Kimi-K3 DCP with decode_attention_backend='cutedsl_mla' requ

Error message

Kimi-K3 DCP with decode_attention_backend='cutedsl_mla' requires FlashInfer 0.6.17 or newer with trtllm_batch_decode_with_kv_cache_mla exposing enable_dcp.

What it means

Kimi-K3 DCP (decoupled context parallelism) with the cutedsl_mla decode backend needs FlashInfer >= 0.6.17 where flashinfer.decode.trtllm_batch_decode_with_kv_cache_mla exists and can be introspected. If importing the symbol or reading its signature fails (ImportError/TypeError/ValueError), this RuntimeError is raised.

Source

Thrown at python/sglang/srt/arg_groups/overrides.py:598

    if backend == "trtllm_mla":
        return True
    if backend == "tokenspeed_mla":
        return kv_cache_dtype == "fp8_e4m3" and q_len <= 8
    if backend == "cutedsl_mla":
        # cute-dsl monolithic MLA decode folds the verify tokens into the head
        # dim (fold_sq), so it serves any DSPARK verify width. Needs flashinfer
        # >= 0.6.15 (older builds reject q_len >= 5).
        return True
    return False


def _require_kimi_k3_cutedsl_dcp_support() -> None:
    try:
        from flashinfer.decode import trtllm_batch_decode_with_kv_cache_mla

        parameters = inspect.signature(trtllm_batch_decode_with_kv_cache_mla).parameters
    except (ImportError, TypeError, ValueError) as exc:
        raise RuntimeError(
            "Kimi-K3 DCP with decode_attention_backend='cutedsl_mla' requires "
            "FlashInfer 0.6.17 or newer with "
            "trtllm_batch_decode_with_kv_cache_mla exposing enable_dcp."
        ) from exc

    if "enable_dcp" not in parameters:
        raise RuntimeError(
            "Kimi-K3 DCP with decode_attention_backend='cutedsl_mla' requires "
            "enable_dcp in the signature of "
            "flashinfer.decode.trtllm_batch_decode_with_kv_cache_mla; upgrade "
            "to FlashInfer 0.6.17 or newer."
        )


@_register_for("KimiK3ForConditionalGeneration")
def _kimi_k3_overrides(server_args: Any, hf_config: Any) -> dict:
    cfg = resolving_view(server_args)
    if cfg.dcp_size > 1:

View on GitHub (pinned to 0132848349)

Solutions

pip install -U 'flashinfer-python>=0.6.17' (match your CUDA/Python wheel)
Verify: python -c "from flashinfer.decode import trtllm_batch_decode_with_kv_cache_mla"
If you can't upgrade, switch decode_attention_backend away from cutedsl_mla or disable DCP

Example fix

# before
pip install flashinfer-python==0.6.10
# after
pip install -U 'flashinfer-python>=0.6.17'

Defensive patterns

Strategy: fallback

Validate before calling

try:
    from flashinfer.decode import trtllm_batch_decode_with_kv_cache_mla  # noqa
except ImportError:
    raise SystemExit('Upgrade: pip install -U "flashinfer-python>=0.6.17"')

Try / catch

except RuntimeError as e:
    if 'FlashInfer 0.6.17' in str(e): subprocess.run([sys.executable,'-m','pip','install','-U','flashinfer-python>=0.6.17']); retry()
    raise

Prevention

Pin flashinfer-python>=0.6.17 in requirements when using Kimi-K3 DCP
Add a startup dependency smoke-check for flashinfer symbols

When it happens

Trigger: Model Kimi-K3 with decode_attention_backend='cutedsl_mla' + DCP enabled while the installed flashinfer is older than 0.6.17 or the symbol is missing; raised inside _kimi_k3_overrides.

Common situations: Pinned older flashinfer in the environment; CI image with stale flashinfer; flashinfer installed but the decode module lacks the MLA API.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/16cfd30d73574a77. Report an issue: GitHub.