sgl-project/sglang · error · ValueError

Both --speculative-num-draft-tokens and --speculative-dflash

Error message

Both --speculative-num-draft-tokens and --speculative-dflash-block-size are set but they differ. For DFLASH they must match. speculative_num_draft_tokens={}, speculative_dflash_block_size={}.

What it means

In DFLASH, --speculative-num-draft-tokens and --speculative-dflash-block-size denote the same quantity (the draft block size), so when both are supplied they must agree. Disagreement indicates a confused config and is rejected.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:257

            "DFLASH only supports speculative_eagle_topk == 1; overriding speculative_eagle_topk=%s to 1.",
            cfg.speculative_eagle_topk,
        )
        declare_resolution(
            server_args,
            "_handle_dflash",
            speculative_eagle_topk=1,
        )

    if cfg.speculative_dflash_block_size is not None:
        if int(cfg.speculative_dflash_block_size) <= 0:
            raise ValueError(
                "DFLASH requires --speculative-dflash-block-size to be positive, "
                f"got {cfg.speculative_dflash_block_size}."
            )
        if cfg.speculative_num_draft_tokens is not None and int(
            cfg.speculative_num_draft_tokens
        ) != int(cfg.speculative_dflash_block_size):
            raise ValueError(
                "Both --speculative-num-draft-tokens and --speculative-dflash-block-size are set "
                "but they differ. For DFLASH they must match. "
                f"speculative_num_draft_tokens={cfg.speculative_num_draft_tokens}, "
                f"speculative_dflash_block_size={cfg.speculative_dflash_block_size}."
            )
        declare_resolution(
            server_args,
            "_handle_dflash",
            speculative_num_draft_tokens=int(cfg.speculative_dflash_block_size),
        )

    if cfg.speculative_num_draft_tokens is None:
        from sglang.srt.speculative.dflash_utils import (
            parse_dflash_draft_config,
        )

        model_override_args = json.loads(cfg.json_model_override_args)
        inferred_block_size = None

View on GitHub (pinned to 0132848349)

Solutions

1. Make both values equal, e.g. both 16
2. Remove one of the two flags and let DFLASH infer/default it
3. Prefer setting only --speculative-dflash-block-size for DFLASH

Example fix

# before
--speculative-num-draft-tokens 8 --speculative-dflash-block-size 16
# after
--speculative-dflash-block-size 16

a, b = args.speculative_num_draft_tokens, args.speculative_dflash_block_size
if a is not None and b is not None and int(a) != int(b):
    raise SystemExit('num_draft_tokens must equal dflash_block_size')

When it happens

Trigger: Passing both --speculative-num-draft-tokens N and --speculative-dflash-block-size M with N != M.

Common situations: Copying EAGLE-style flags (num-draft-tokens) onto a DFLASH command line that also sets dflash-block-size; partial migration of old launch scripts.

Related errors

• DFLASH requires --speculative-dflash-block-size to be positi
• DFLASH speculative decoding only supports CUDA and NPU devic
• Currently DFLASH speculative decoding does not support dp at
• Currently DFLASH speculative decoding only supports pp_size
• DFLASH speculative decoding requires setting --speculative-d

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/bc9cae371f476d0c. Report an issue: GitHub.