{"record":{"id":"75c077c4341044a4","repo":"sgl-project/sglang","slug":"currently-dflash-speculative-decoding-only-support","errorCode":null,"errorMessage":"Currently DFLASH speculative decoding only supports pp_size == 1.","messagePattern":"Currently DFLASH speculative decoding only supports pp_size == 1\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/arg_groups/speculative_hook.py","lineNumber":200,"sourceCode":"        )\n\n\ndef _handle_dflash(server_args: ServerArgs) -> None:\n    cfg = resolving_view(server_args)\n    from sglang.srt.arg_groups.overrides import resolved_view\n\n    if not (cfg.device.startswith(\"cuda\") or cfg.device == \"npu\"):\n        raise ValueError(\n            \"DFLASH speculative decoding only supports CUDA and NPU devices.\"\n        )\n\n    if resolved_view(server_args).enable_dp_attention:\n        raise ValueError(\n            \"Currently DFLASH speculative decoding does not support dp attention.\"\n        )\n\n    if cfg.pp_size != 1:\n        raise ValueError(\n            \"Currently DFLASH speculative decoding only supports pp_size == 1.\"\n        )\n\n    if cfg.speculative_draft_model_path is None:\n        raise ValueError(\n            \"DFLASH speculative decoding requires setting --speculative-draft-model-path.\"\n        )\n\n    # DFLASH does not use EAGLE-style `num_steps`/`topk`, but those fields still\n    # affect generic scheduler/KV-cache accounting (buffer sizing, KV freeing,\n    # RoPE reservation). Force them to 1 to avoid surprising memory behavior.\n    #\n    # For DFlash, the natural unit is `block_size` (verify window length).\n    if cfg.speculative_num_steps is None:\n        declare_resolution(\n            server_args,\n            \"_handle_dflash\",\n            speculative_num_steps=1,","sourceCodeStart":182,"sourceCodeEnd":218,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/arg_groups/speculative_hook.py#L182-L218","documentation":"DFLASH speculative decoding only works with a single pipeline stage. _handle_dflash rejects cfg.pp_size != 1 because the DFLASH draft/verify flow is not implemented across pipeline-parallel ranks.","triggerScenarios":"Server launch with speculative_algorithm=DFLASH and --pipeline-parallel-size > 1 (pp_size resolved > 1).","commonSituations":"Reusing a multi-node PP launch script (e.g. pp=2 or 4 for large models) and turning on DFLASH spec decoding.","solutions":["Set --pipeline-parallel-size 1 for DFLASH runs","Use tensor parallelism to get the needed GPU count instead of PP","Use a different speculative algorithm if PP is mandatory"],"exampleFix":"# before\n--speculative-algorithm DFLASH --pp-size 2\n# after\n--speculative-algorithm DFLASH --pp-size 1 --tp-size 2","handlingStrategy":"validation","validationCode":"if args.pp_size != 1:\n    raise SystemExit('DFLASH requires pp_size == 1')","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Prefer TP over PP for DFLASH deployments","Grep launch scripts for pipeline-parallel flags before enabling speculation"],"tags":["speculative-decoding","dflash","pipeline-parallel","server-args"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}