sgl-project/sglang · error · ValueError

--speculative-draft-window-size must be >=…

Error message

--speculative-draft-window-size must be >= --speculative-num-draft-tokens (block_size). window_size={}, block_size={}.

What it means

DFLASH's draft window must be at least as large as the draft block (num draft tokens), otherwise there is no room to draft even one block. The hook validates speculative_draft_window_size >= speculative_num_draft_tokens.

Solutions

  1. Set --speculative-draft-window-size >= block size (e.g. block 16 -> window >= 16)
  2. Omit --speculative-draft-window-size to use the default
  3. If memory-constrained, lower both window and block size together

Example fix

# before
--speculative-dflash-block-size 32 --speculative-draft-window-size 16
# after
--speculative-dflash-block-size 32 --speculative-draft-window-size 32
Defensive patterns

Strategy: validation

Validate before calling

if args.speculative_draft_window_size is not None:
    block = int(args.speculative_num_draft_tokens or 0)
    if args.speculative_draft_window_size < block:
        raise SystemExit('window_size must be >= block size')

Prevention

When it happens

Trigger: Passing --speculative-draft-window-size smaller than the resolved --speculative-num-draft-tokens (block size).

Common situations: Tuning window size down for memory savings while keeping a larger block size; stale window value after increasing block size.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ff1f6c98d1be2305. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:310

                e,
            )

        if inferred_block_size is None:
            inferred_block_size = 16
            logger.warning(
                "speculative_num_draft_tokens is not set; defaulting to %d for DFLASH.",
                inferred_block_size,
            )
        declare_resolution(
            server_args,
            "_handle_dflash",
            speculative_num_draft_tokens=inferred_block_size,
        )

    if cfg.speculative_draft_window_size is not None:
        draft_tokens = int(cfg.speculative_num_draft_tokens)
        if cfg.speculative_draft_window_size < draft_tokens:
            raise ValueError(
                "--speculative-draft-window-size must be >= "
                "--speculative-num-draft-tokens (block_size). "
                f"window_size={cfg.speculative_draft_window_size}, block_size={draft_tokens}."
            )

    _resolve_dflash_draft_attention_backend(server_args)

    if cfg.max_running_requests is None:
        declare_resolution(
            server_args,
            "_handle_dflash",
            max_running_requests=48,
        )
        logger.warning(
            "Max running requests is reset to 48 for speculative decoding. You can override this by explicitly setting --max-running-requests."
        )

    if cfg.enable_mixed_chunk:

View on GitHub (pinned to 0132848349)