sgl-project/sglang · error · ValueError

Currently DFLASH speculative decoding only supports pp_size

Error message

Currently DFLASH speculative decoding only supports pp_size == 1.

What it means

DFLASH speculative decoding only works with a single pipeline stage. _handle_dflash rejects cfg.pp_size != 1 because the DFLASH draft/verify flow is not implemented across pipeline-parallel ranks.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:200

        )


def _handle_dflash(server_args: ServerArgs) -> None:
    cfg = resolving_view(server_args)
    from sglang.srt.arg_groups.overrides import resolved_view

    if not (cfg.device.startswith("cuda") or cfg.device == "npu"):
        raise ValueError(
            "DFLASH speculative decoding only supports CUDA and NPU devices."
        )

    if resolved_view(server_args).enable_dp_attention:
        raise ValueError(
            "Currently DFLASH speculative decoding does not support dp attention."
        )

    if cfg.pp_size != 1:
        raise ValueError(
            "Currently DFLASH speculative decoding only supports pp_size == 1."
        )

    if cfg.speculative_draft_model_path is None:
        raise ValueError(
            "DFLASH speculative decoding requires setting --speculative-draft-model-path."
        )

    # DFLASH does not use EAGLE-style `num_steps`/`topk`, but those fields still
    # affect generic scheduler/KV-cache accounting (buffer sizing, KV freeing,
    # RoPE reservation). Force them to 1 to avoid surprising memory behavior.
    #
    # For DFlash, the natural unit is `block_size` (verify window length).
    if cfg.speculative_num_steps is None:
        declare_resolution(
            server_args,
            "_handle_dflash",
            speculative_num_steps=1,

View on GitHub (pinned to 0132848349)

Solutions

1. Set --pipeline-parallel-size 1 for DFLASH runs
2. Use tensor parallelism to get the needed GPU count instead of PP
3. Use a different speculative algorithm if PP is mandatory

Example fix

# before
--speculative-algorithm DFLASH --pp-size 2
# after
--speculative-algorithm DFLASH --pp-size 1 --tp-size 2

if args.pp_size != 1:
    raise SystemExit('DFLASH requires pp_size == 1')

When it happens

Trigger: Server launch with speculative_algorithm=DFLASH and --pipeline-parallel-size > 1 (pp_size resolved > 1).

Common situations: Reusing a multi-node PP launch script (e.g. pp=2 or 4 for large models) and turning on DFLASH spec decoding.

Related errors

• Currently DFLASH speculative decoding does not support dp at
• DFLASH speculative decoding only supports CUDA and NPU devic
• DFLASH speculative decoding requires setting --speculative-d
• DFLASH requires --speculative-dflash-block-size to be positi
• Both --speculative-num-draft-tokens and --speculative-dflash

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/75c077c4341044a4. Report an issue: GitHub.