{"record":{"id":"16cfd30d73574a77","repo":"sgl-project/sglang","slug":"kimi-k3-dcp-with-decode-attention-backend-cutedsl","errorCode":null,"errorMessage":"Kimi-K3 DCP with decode_attention_backend='cutedsl_mla' requires FlashInfer 0.6.17 or newer with trtllm_batch_decode_with_kv_cache_mla exposing enable_dcp.","messagePattern":"Kimi-K3 DCP with decode_attention_backend='cutedsl_mla' requires FlashInfer 0\\.6\\.17 or newer with trtllm_batch_decode_with_kv_cache_mla exposing enable_dcp\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/arg_groups/overrides.py","lineNumber":598,"sourceCode":"    if backend == \"trtllm_mla\":\n        return True\n    if backend == \"tokenspeed_mla\":\n        return kv_cache_dtype == \"fp8_e4m3\" and q_len <= 8\n    if backend == \"cutedsl_mla\":\n        # cute-dsl monolithic MLA decode folds the verify tokens into the head\n        # dim (fold_sq), so it serves any DSPARK verify width. Needs flashinfer\n        # >= 0.6.15 (older builds reject q_len >= 5).\n        return True\n    return False\n\n\ndef _require_kimi_k3_cutedsl_dcp_support() -> None:\n    try:\n        from flashinfer.decode import trtllm_batch_decode_with_kv_cache_mla\n\n        parameters = inspect.signature(trtllm_batch_decode_with_kv_cache_mla).parameters\n    except (ImportError, TypeError, ValueError) as exc:\n        raise RuntimeError(\n            \"Kimi-K3 DCP with decode_attention_backend='cutedsl_mla' requires \"\n            \"FlashInfer 0.6.17 or newer with \"\n            \"trtllm_batch_decode_with_kv_cache_mla exposing enable_dcp.\"\n        ) from exc\n\n    if \"enable_dcp\" not in parameters:\n        raise RuntimeError(\n            \"Kimi-K3 DCP with decode_attention_backend='cutedsl_mla' requires \"\n            \"enable_dcp in the signature of \"\n            \"flashinfer.decode.trtllm_batch_decode_with_kv_cache_mla; upgrade \"\n            \"to FlashInfer 0.6.17 or newer.\"\n        )\n\n\n@_register_for(\"KimiK3ForConditionalGeneration\")\ndef _kimi_k3_overrides(server_args: Any, hf_config: Any) -> dict:\n    cfg = resolving_view(server_args)\n    if cfg.dcp_size > 1:","sourceCodeStart":580,"sourceCodeEnd":616,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/arg_groups/overrides.py#L580-L616","documentation":"Kimi-K3 DCP (decoupled context parallelism) with the cutedsl_mla decode backend needs FlashInfer >= 0.6.17 where flashinfer.decode.trtllm_batch_decode_with_kv_cache_mla exists and can be introspected. If importing the symbol or reading its signature fails (ImportError/TypeError/ValueError), this RuntimeError is raised.","triggerScenarios":"Model Kimi-K3 with decode_attention_backend='cutedsl_mla' + DCP enabled while the installed flashinfer is older than 0.6.17 or the symbol is missing; raised inside _kimi_k3_overrides.","commonSituations":"Pinned older flashinfer in the environment; CI image with stale flashinfer; flashinfer installed but the decode module lacks the MLA API.","solutions":["pip install -U 'flashinfer-python>=0.6.17' (match your CUDA/Python wheel)","Verify: python -c \"from flashinfer.decode import trtllm_batch_decode_with_kv_cache_mla\"","If you can't upgrade, switch decode_attention_backend away from cutedsl_mla or disable DCP"],"exampleFix":"# before\npip install flashinfer-python==0.6.10\n# after\npip install -U 'flashinfer-python>=0.6.17'","handlingStrategy":"fallback","validationCode":"try:\n    from flashinfer.decode import trtllm_batch_decode_with_kv_cache_mla  # noqa\nexcept ImportError:\n    raise SystemExit('Upgrade: pip install -U \"flashinfer-python>=0.6.17\"')","typeGuard":null,"tryCatchPattern":"except RuntimeError as e:\n    if 'FlashInfer 0.6.17' in str(e): subprocess.run([sys.executable,'-m','pip','install','-U','flashinfer-python>=0.6.17']); retry()\n    raise","preventionTips":["Pin flashinfer-python>=0.6.17 in requirements when using Kimi-K3 DCP","Add a startup dependency smoke-check for flashinfer symbols"],"tags":["kimi-k3","flashinfer","version-mismatch","dcp","sglang"],"backgroundTag":"library-version-too-old","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}