sgl-project/sglang · error · ValueError

Currently DFLASH speculative decoding does not support dp at

Error message

Currently DFLASH speculative decoding does not support dp attention.

What it means

DFLASH speculative decoding is incompatible with data-parallel attention (dp attention). The arg-validation hook _handle_dflash rejects the combination at server-args processing time because DFLASH's draft/verify path was never implemented against the DP-attention request sharding machinery.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:195

        # fields on the record, so the writes are captured around the call.
        declare_direct_writes(
            server_args,
            "handle_speculative_decoding.custom_algo",
            algo.handle_server_args,
        )


def _handle_dflash(server_args: ServerArgs) -> None:
    cfg = resolving_view(server_args)
    from sglang.srt.arg_groups.overrides import resolved_view

    if not (cfg.device.startswith("cuda") or cfg.device == "npu"):
        raise ValueError(
            "DFLASH speculative decoding only supports CUDA and NPU devices."
        )

    if resolved_view(server_args).enable_dp_attention:
        raise ValueError(
            "Currently DFLASH speculative decoding does not support dp attention."
        )

    if cfg.pp_size != 1:
        raise ValueError(
            "Currently DFLASH speculative decoding only supports pp_size == 1."
        )

    if cfg.speculative_draft_model_path is None:
        raise ValueError(
            "DFLASH speculative decoding requires setting --speculative-draft-model-path."
        )

    # DFLASH does not use EAGLE-style `num_steps`/`topk`, but those fields still
    # affect generic scheduler/KV-cache accounting (buffer sizing, KV freeing,
    # RoPE reservation). Force them to 1 to avoid surprising memory behavior.
    #
    # For DFlash, the natural unit is `block_size` (verify window length).

View on GitHub (pinned to 0132848349)

Solutions

1. Remove --enable-dp-attention (and any preset that turns it on) for DFLASH runs
2. Switch to a supported speculative algorithm (e.g. EAGLE/NEXTN) if dp attention is required
3. Track upstream support for DFLASH + dp attention before retrying

Example fix

# before
--speculative-algorithm DFLASH --enable-dp-attention
# after
--speculative-algorithm DFLASH

from sglang.srt.server_args import ServerArgs
args = ServerArgs(model=..., speculative_algorithm='DFLASH')
if args.enable_dp_attention:
    raise SystemExit('DFLASH cannot run with dp attention; drop --enable-dp-attention')

When it happens

Trigger: Launching the server with speculative_algorithm=DFLASH together with --enable-dp-attention (or a config that resolves enable_dp_attention=True), e.g. `python -m sglang.launch_server --model ... --speculative-algorithm DFLASH --enable-dp-attention`.

Common situations: Users copy a DeepSeek-style DP-attention launch command and add DFLASH spec decoding on top; or a profile/config preset enables dp attention implicitly.

Related errors

• Currently DFLASH speculative decoding only supports pp_size
• DFLASH speculative decoding only supports CUDA and NPU devic
• DFLASH speculative decoding requires setting --speculative-d
• DFLASH requires --speculative-dflash-block-size to be positi
• Both --speculative-num-draft-tokens and --speculative-dflash

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/af12c88f3c53cbd9. Report an issue: GitHub.