sgl-project/sglang · error · ValueError

--speculative-draft-window-size must be >= --speculative-num

Error message

--speculative-draft-window-size must be >= --speculative-num-draft-tokens (block_size). window_size={}, block_size={}.

What it means

DFLASH's draft window must be at least as large as the draft block (num draft tokens), otherwise there is no room to draft even one block. The hook validates speculative_draft_window_size >= speculative_num_draft_tokens.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:310

                e,
            )

        if inferred_block_size is None:
            inferred_block_size = 16
            logger.warning(
                "speculative_num_draft_tokens is not set; defaulting to %d for DFLASH.",
                inferred_block_size,
            )
        declare_resolution(
            server_args,
            "_handle_dflash",
            speculative_num_draft_tokens=inferred_block_size,
        )

    if cfg.speculative_draft_window_size is not None:
        draft_tokens = int(cfg.speculative_num_draft_tokens)
        if cfg.speculative_draft_window_size < draft_tokens:
            raise ValueError(
                "--speculative-draft-window-size must be >= "
                "--speculative-num-draft-tokens (block_size). "
                f"window_size={cfg.speculative_draft_window_size}, block_size={draft_tokens}."
            )

    _resolve_dflash_draft_attention_backend(server_args)

    if cfg.max_running_requests is None:
        declare_resolution(
            server_args,
            "_handle_dflash",
            max_running_requests=48,
        )
        logger.warning(
            "Max running requests is reset to 48 for speculative decoding. You can override this by explicitly setting --max-running-requests."
        )

    if cfg.enable_mixed_chunk:

View on GitHub (pinned to 0132848349)

Solutions

1. Set --speculative-draft-window-size >= block size (e.g. block 16 -> window >= 16)
2. Omit --speculative-draft-window-size to use the default
3. If memory-constrained, lower both window and block size together

Example fix

# before
--speculative-dflash-block-size 32 --speculative-draft-window-size 16
# after
--speculative-dflash-block-size 32 --speculative-draft-window-size 32

if args.speculative_draft_window_size is not None:
    block = int(args.speculative_num_draft_tokens or 0)
    if args.speculative_draft_window_size < block:
        raise SystemExit('window_size must be >= block size')

When it happens

Trigger: Passing --speculative-draft-window-size smaller than the resolved --speculative-num-draft-tokens (block size).

Common situations: Tuning window size down for memory savings while keeping a larger block size; stale window value after increasing block size.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• --speculative-draft-window-size must be positive, got {}.
• DFLASH requires --speculative-dflash-block-size to be positi
• draft_token_num must be positive, got {draft_token_num}.
• Unknown match_type: '{match_type}'. Must be 'BFS' or 'PROB'.
• DFLASH speculative decoding only supports CUDA and NPU devic

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ff1f6c98d1be2305. Report an issue: GitHub.