sgl-project/sglang · error · ValueError

Currently DFLASH speculative decoding does not support dp…

Error message

Currently DFLASH speculative decoding does not support dp attention.

What it means

DFLASH speculative decoding is incompatible with data-parallel attention (dp attention). The arg-validation hook _handle_dflash rejects the combination at server-args processing time because DFLASH's draft/verify path was never implemented against the DP-attention request sharding machinery.

Solutions

1. Remove --enable-dp-attention (and any preset that turns it on) for DFLASH runs
2. Switch to a supported speculative algorithm (e.g. EAGLE/NEXTN) if dp attention is required
3. Track upstream support for DFLASH + dp attention before retrying

Example fix

# before
--speculative-algorithm DFLASH --enable-dp-attention
# after
--speculative-algorithm DFLASH

from sglang.srt.server_args import ServerArgs
args = ServerArgs(model=..., speculative_algorithm='DFLASH')
if args.enable_dp_attention:
    raise SystemExit('DFLASH cannot run with dp attention; drop --enable-dp-attention')

When it happens

Trigger: Launching the server with speculative_algorithm=DFLASH together with --enable-dp-attention (or a config that resolves enable_dp_attention=True), e.g. `python -m sglang.launch_server --model ... --speculative-algorithm DFLASH --enable-dp-attention`.

Common situations: Users copy a DeepSeek-style DP-attention launch command and add DFLASH spec decoding on top; or a profile/config preset enables dp attention implicitly.

Related errors

• Currently DFLASH speculative decoding only supports pp_size…
• Both --speculative-num-draft-tokens and…
• candidates and next_token_logits must be on the same…
• expected 2D positions, got shape= .
• mask/position shape mismatch: vs .

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/af12c88f3c53cbd9. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:195

        # fields on the record, so the writes are captured around the call.
        declare_direct_writes(
            server_args,
            "handle_speculative_decoding.custom_algo",
            algo.handle_server_args,
        )


def _handle_dflash(server_args: ServerArgs) -> None:
    cfg = resolving_view(server_args)
    from sglang.srt.arg_groups.overrides import resolved_view

    if not (cfg.device.startswith("cuda") or cfg.device == "npu"):
        raise ValueError(
            "DFLASH speculative decoding only supports CUDA and NPU devices."
        )

    if resolved_view(server_args).enable_dp_attention:
        raise ValueError(
            "Currently DFLASH speculative decoding does not support dp attention."
        )

    if cfg.pp_size != 1:
        raise ValueError(
            "Currently DFLASH speculative decoding only supports pp_size == 1."
        )

    if cfg.speculative_draft_model_path is None:
        raise ValueError(
            "DFLASH speculative decoding requires setting --speculative-draft-model-path."
        )

    # DFLASH does not use EAGLE-style `num_steps`/`topk`, but those fields still
    # affect generic scheduler/KV-cache accounting (buffer sizing, KV freeing,
    # RoPE reservation). Force them to 1 to avoid surprising memory behavior.
    #
    # For DFlash, the natural unit is `block_size` (verify window length).

View on GitHub (pinned to 0132848349)