sgl-project/sglang · error · ValueError

speculative_ngram_external_sam_budget must be less than or e

Error message

speculative_ngram_external_sam_budget must be less than or equal to speculative_num_draft_tokens - 1 ({cfg.speculative_num_draft_tokens - 1}).

What it means

The external-corpus suffix-array budget reserves speculative_ngram_external_sam_budget of the per-step draft slots: at least one slot must remain for the request-local ngram matcher, so the budget cannot exceed speculative_num_draft_tokens - 1. SGLang enforces this bound with a ValueError in _handle_ngram.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:906

            speculative_num_steps=cfg.speculative_num_draft_tokens
            // cfg.speculative_eagle_topk,
        )
    if cfg.speculative_ngram_external_corpus_path is not None:
        if cfg.speculative_ngram_external_sam_budget <= 0:
            raise ValueError(
                "--speculative-ngram-external-sam-budget must be positive when "
                "--speculative-ngram-external-corpus-path is set."
            )
        if cfg.speculative_ngram_external_corpus_max_tokens <= 0:
            raise ValueError(
                "--speculative-ngram-external-corpus-max-tokens must be positive when "
                "--speculative-ngram-external-corpus-path is set."
            )
        if (
            cfg.speculative_ngram_external_sam_budget
            > cfg.speculative_num_draft_tokens - 1
        ):
            raise ValueError(
                "speculative_ngram_external_sam_budget must be less than or equal to "
                f"speculative_num_draft_tokens - 1 ({cfg.speculative_num_draft_tokens - 1})."
            )
    logger.warning(
        "The mixed chunked prefill are disabled because of "
        "using ngram speculative decoding."
    )

    from sglang.srt.arg_groups.overrides import resolved_view

    view = resolved_view(server_args)
    if (
        cfg.speculative_eagle_topk > 1
        and view.page_size > 1
        and view.attention_backend != "flashinfer"
    ):
        raise ValueError(
            f"speculative_eagle_topk({cfg.speculative_eagle_topk}) > 1 "

View on GitHub (pinned to 0132848349)

Solutions

1. Raise --speculative-num-draft-tokens to at least sam_budget + 1
2. Or lower --speculative-ngram-external-sam-budget to <= speculative_num_draft_tokens - 1
3. Remember both constraints together: 1 <= sam_budget <= num_draft_tokens - 1

Example fix

# before
--speculative-num-draft-tokens 4 --speculative-ngram-external-sam-budget 8
# after
--speculative-num-draft-tokens 8 --speculative-ngram-external-sam-budget 8

b = server_args.speculative_ngram_external_sam_budget
n = server_args.speculative_num_draft_tokens
if server_args.speculative_ngram_external_corpus_path is not None:
    assert 1 <= b <= n - 1, f"sam_budget {b} must be in [1, {n - 1}]"

When it happens

Trigger: Launching with --speculative-ngram-external-corpus-path where speculative_ngram_external_sam_budget > speculative_num_draft_tokens - 1, e.g. budget 8 with --speculative-num-draft-tokens 4.

Common situations: Raising the SAM budget for better draft hit-rate without also raising --speculative-num-draft-tokens; using a small draft-token count (e.g. 2 or 3) that leaves almost no room for external matches.

Related errors

• --speculative-ngram-external-sam-budget must be positive whe
• --speculative-ngram-external-corpus-max-tokens must be posit
• Ngram speculative decoding only supports CUDA or CPU devices
• External ngram corpus path does not exist: {path}
• This browser cannot encode H.264 MP4

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d861aacdfaf5bd3d. Report an issue: GitHub.