sgl-project/sglang · error · RuntimeError

Kimi-K3 DCP with decode_attention_backend='cutedsl_mla' requ

Error message

Kimi-K3 DCP with decode_attention_backend='cutedsl_mla' requires enable_dcp in the signature of flashinfer.decode.trtllm_batch_decode_with_kv_cache_mla; upgrade to FlashInfer 0.6.17 or newer.

What it means

Even when flashinfer's trtllm_batch_decode_with_kv_cache_mla is importable, Kimi-K3 DCP needs the enable_dcp parameter in its signature. Older FlashInfer builds expose the function without enable_dcp, so SGLang inspects the signature and raises this RuntimeError when the kwarg is absent.

Source

Thrown at python/sglang/srt/arg_groups/overrides.py:605

        # >= 0.6.15 (older builds reject q_len >= 5).
        return True
    return False


def _require_kimi_k3_cutedsl_dcp_support() -> None:
    try:
        from flashinfer.decode import trtllm_batch_decode_with_kv_cache_mla

        parameters = inspect.signature(trtllm_batch_decode_with_kv_cache_mla).parameters
    except (ImportError, TypeError, ValueError) as exc:
        raise RuntimeError(
            "Kimi-K3 DCP with decode_attention_backend='cutedsl_mla' requires "
            "FlashInfer 0.6.17 or newer with "
            "trtllm_batch_decode_with_kv_cache_mla exposing enable_dcp."
        ) from exc

    if "enable_dcp" not in parameters:
        raise RuntimeError(
            "Kimi-K3 DCP with decode_attention_backend='cutedsl_mla' requires "
            "enable_dcp in the signature of "
            "flashinfer.decode.trtllm_batch_decode_with_kv_cache_mla; upgrade "
            "to FlashInfer 0.6.17 or newer."
        )


@_register_for("KimiK3ForConditionalGeneration")
def _kimi_k3_overrides(server_args: Any, hf_config: Any) -> dict:
    cfg = resolving_view(server_args)
    if cfg.dcp_size > 1:
        overrides = {}
        if cfg.enable_symm_mem:
            logger.warning(
                "Kimi-K3 DCP disables --enable-symm-mem due to decode CUDA "
                "graph correctness issues."
            )
            overrides["enable_symm_mem"] = False

View on GitHub (pinned to 0132848349)

Solutions

Upgrade flashinfer to >= 0.6.17: pip install -U 'flashinfer-python>=0.6.17'
Check: python -c "import inspect, flashinfer.decode as d; print('enable_dcp' in inspect.signature(d.trtllm_batch_decode_with_kv_cache_mla).parameters)"
Otherwise switch decode_attention_backend or disable DCP

Example fix

# before
# flashinfer 0.6.15 (function exists, no enable_dcp)
# after
pip install -U 'flashinfer-python>=0.6.17'

Defensive patterns

Strategy: validation

Validate before calling

import inspect
from flashinfer.decode import trtllm_batch_decode_with_kv_cache_mla as f
assert 'enable_dcp' in inspect.signature(f).parameters, 'need flashinfer>=0.6.17'

Try / catch

except RuntimeError as e:
    if 'enable_dcp' in str(e): upgrade_flashinfer(); retry()
    raise

Prevention

Verify signature kwargs after any flashinfer upgrade/downgrade
Freeze the flashinfer version known to expose enable_dcp

When it happens

Trigger: FlashInfer installed with the symbol present but predating enable_dcp support (between versions), running Kimi-K3 DCP + cutedsl_mla; raised from _require_kimi_k3_cutedsl_dcp_support.

Common situations: A flashinfer version that added the function before adding enable_dcp; locally built flashinfer from an older branch; partial upgrade of the wheel.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/58f6db0acb10fb5d. Report an issue: GitHub.