{"record":{"id":"65f6e7ea18d78a6d","repo":"sgl-project/sglang","slug":"dspark-with-dp-attention-does-not-support-context","errorCode":null,"errorMessage":"DSpark with dp attention does not support context parallel (attn_cp_size={}).","messagePattern":"DSpark with dp attention does not support context parallel \\(attn_cp_size=(.+?)\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/arg_groups/speculative_hook.py","lineNumber":378,"sourceCode":"            raise ValueError(\n                \"DSpark with dp attention supports moe_a2a_backend 'none' \"\n                \"(built-in TP MoE) or 'megamoe', got \"\n                f\"{cfg.moe_a2a_backend!r}.\"\n            )\n        if not _is_npu and cfg.moe_a2a_backend != \"none\":\n            from sglang.srt.speculative.ragged_verify import (\n                RaggedVerifyMode,\n                read_ragged_verify_mode,\n            )\n\n            if read_ragged_verify_mode() is not RaggedVerifyMode.STATIC:\n                raise ValueError(\n                    \"DSpark with dp attention + \"\n                    f\"moe_a2a_backend={cfg.moe_a2a_backend!r} requires \"\n                    \"SGLANG_RAGGED_VERIFY_MODE=static.\"\n                )\n        if cfg.attn_cp_size > 1:\n            raise ValueError(\n                \"DSpark with dp attention does not support context parallel \"\n                f\"(attn_cp_size={cfg.attn_cp_size}).\"\n            )\n        if (\n            not _is_npu\n            and cfg.speculative_moe_a2a_backend is not None\n            and cfg.speculative_moe_a2a_backend != cfg.moe_a2a_backend\n        ):\n            raise ValueError(\n                \"DSpark ignores --speculative-moe-a2a-backend; with dp attention it \"\n                f\"must match the target moe_a2a_backend={cfg.moe_a2a_backend!r} \"\n                f\"(got {cfg.speculative_moe_a2a_backend!r}).\"\n            )\n\n    if cfg.pp_size != 1:\n        raise ValueError(\n            \"Currently DSpark speculative decoding only supports pp_size == 1.\"\n        )","sourceCodeStart":360,"sourceCodeEnd":396,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/arg_groups/speculative_hook.py#L360-L396","documentation":"DSpark with dp attention cannot run together with attention context parallelism. The hook rejects attn_cp_size > 1 under DP attention because CP-sharded attention is incompatible with the DSpark DP verification path.","triggerScenarios":"Launching DSpark with --enable-dp-attention, --dp-size>1, and --attn-cp-size (attn_cp_size) greater than 1.","commonSituations":"Long-context deployments (DeepSeek V4-style) that enable attention CP for ultra-long sequences also enable DP attention; adding DSpark breaks the combo.","solutions":["Set --attn-cp-size 1 (or drop the flag) when using DSpark + dp attention","Drop dp attention if context parallelism is required","Use a different speculative algorithm that supports CP"],"exampleFix":"# before\n--speculative-algorithm DSPARK --enable-dp-attention --dp-size 4 --attn-cp-size 2\n# after\n--speculative-algorithm DSPARK --enable-dp-attention --dp-size 4","handlingStrategy":"validation","validationCode":"if args.enable_dp_attention and args.dp_size > 1 and args.attn_cp_size > 1:\n    raise SystemExit('DSPARK + dp attention does not support attn_cp_size > 1')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Long-context CP configs and DSpark DP attention are mutually exclusive; keep separate presets"],"tags":["speculative-decoding","dspark","context-parallel","dp-attention"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}