{"record":{"id":"af12c88f3c53cbd9","repo":"sgl-project/sglang","slug":"currently-dflash-speculative-decoding-does-not-sup","errorCode":null,"errorMessage":"Currently DFLASH speculative decoding does not support dp attention.","messagePattern":"Currently DFLASH speculative decoding does not support dp attention\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/arg_groups/speculative_hook.py","lineNumber":195,"sourceCode":"        # fields on the record, so the writes are captured around the call.\n        declare_direct_writes(\n            server_args,\n            \"handle_speculative_decoding.custom_algo\",\n            algo.handle_server_args,\n        )\n\n\ndef _handle_dflash(server_args: ServerArgs) -> None:\n    cfg = resolving_view(server_args)\n    from sglang.srt.arg_groups.overrides import resolved_view\n\n    if not (cfg.device.startswith(\"cuda\") or cfg.device == \"npu\"):\n        raise ValueError(\n            \"DFLASH speculative decoding only supports CUDA and NPU devices.\"\n        )\n\n    if resolved_view(server_args).enable_dp_attention:\n        raise ValueError(\n            \"Currently DFLASH speculative decoding does not support dp attention.\"\n        )\n\n    if cfg.pp_size != 1:\n        raise ValueError(\n            \"Currently DFLASH speculative decoding only supports pp_size == 1.\"\n        )\n\n    if cfg.speculative_draft_model_path is None:\n        raise ValueError(\n            \"DFLASH speculative decoding requires setting --speculative-draft-model-path.\"\n        )\n\n    # DFLASH does not use EAGLE-style `num_steps`/`topk`, but those fields still\n    # affect generic scheduler/KV-cache accounting (buffer sizing, KV freeing,\n    # RoPE reservation). Force them to 1 to avoid surprising memory behavior.\n    #\n    # For DFlash, the natural unit is `block_size` (verify window length).","sourceCodeStart":177,"sourceCodeEnd":213,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/arg_groups/speculative_hook.py#L177-L213","documentation":"DFLASH speculative decoding is incompatible with data-parallel attention (dp attention). The arg-validation hook _handle_dflash rejects the combination at server-args processing time because DFLASH's draft/verify path was never implemented against the DP-attention request sharding machinery.","triggerScenarios":"Launching the server with speculative_algorithm=DFLASH together with --enable-dp-attention (or a config that resolves enable_dp_attention=True), e.g. `python -m sglang.launch_server --model ... --speculative-algorithm DFLASH --enable-dp-attention`.","commonSituations":"Users copy a DeepSeek-style DP-attention launch command and add DFLASH spec decoding on top; or a profile/config preset enables dp attention implicitly.","solutions":["Remove --enable-dp-attention (and any preset that turns it on) for DFLASH runs","Switch to a supported speculative algorithm (e.g. EAGLE/NEXTN) if dp attention is required","Track upstream support for DFLASH + dp attention before retrying"],"exampleFix":"# before\n--speculative-algorithm DFLASH --enable-dp-attention\n# after\n--speculative-algorithm DFLASH","handlingStrategy":"validation","validationCode":"from sglang.srt.server_args import ServerArgs\nargs = ServerArgs(model=..., speculative_algorithm='DFLASH')\nif args.enable_dp_attention:\n    raise SystemExit('DFLASH cannot run with dp attention; drop --enable-dp-attention')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Keep per-algorithm launch scripts; never mix DP-attention presets with DFLASH","Assert flag compatibility before launch in CI smoke tests"],"tags":["speculative-decoding","dflash","dp-attention","server-args"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}