sgl-project/sglang · error · ValueError

--speculative-ngram-external-sam-budget must be positive whe

Error message

--speculative-ngram-external-sam-budget must be positive when --speculative-ngram-external-corpus-path is set.

What it means

When an external draft corpus is supplied via --speculative-ngram-external-corpus-path, SGLang builds a suffix-array model (SAM) over it for draft proposals. The SAM budget controls how many suffix-array matches are retained; a zero/negative budget makes the external corpus useless and indicates a misconfiguration, so the args hook requires it to be positive.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:893

        declare_resolution(
            server_args,
            "_handle_ngram",
            speculative_num_draft_tokens=12,
        )
        logger.warning(
            "speculative_num_draft_tokens is set to 12 by default for ngram speculative decoding. "
            "You can override this by explicitly setting --speculative-num-draft-tokens."
        )
    if cfg.speculative_num_steps is None:
        declare_resolution(
            server_args,
            "_handle_ngram",
            speculative_num_steps=cfg.speculative_num_draft_tokens
            // cfg.speculative_eagle_topk,
        )
    if cfg.speculative_ngram_external_corpus_path is not None:
        if cfg.speculative_ngram_external_sam_budget <= 0:
            raise ValueError(
                "--speculative-ngram-external-sam-budget must be positive when "
                "--speculative-ngram-external-corpus-path is set."
            )
        if cfg.speculative_ngram_external_corpus_max_tokens <= 0:
            raise ValueError(
                "--speculative-ngram-external-corpus-max-tokens must be positive when "
                "--speculative-ngram-external-corpus-path is set."
            )
        if (
            cfg.speculative_ngram_external_sam_budget
            > cfg.speculative_num_draft_tokens - 1
        ):
            raise ValueError(
                "speculative_ngram_external_sam_budget must be less than or equal to "
                f"speculative_num_draft_tokens - 1 ({cfg.speculative_num_draft_tokens - 1})."
            )
    logger.warning(
        "The mixed chunked prefill are disabled because of "

View on GitHub (pinned to 0132848349)

Solutions

1. Set --speculative-ngram-external-sam-budget to a positive value (also must be <= speculative_num_draft_tokens - 1)
2. Keep it within [1, speculative_num_draft_tokens - 1] to also satisfy the follow-up check
3. If you don't need the external corpus, remove --speculative-ngram-external-corpus-path

Example fix

# before
--speculative-ngram-external-corpus-path /data/corpus.bin
# after
--speculative-ngram-external-corpus-path /data/corpus.bin --speculative-ngram-external-sam-budget 4

if server_args.speculative_ngram_external_corpus_path is not None:
    assert server_args.speculative_ngram_external_sam_budget > 0
    assert server_args.speculative_ngram_external_sam_budget <= server_args.speculative_num_draft_tokens - 1

When it happens

Trigger: Launching with --speculative-ngram-external-corpus-path set and --speculative-ngram-external-sam-budget <= 0 (default 0 when unset). Checked inside _handle_ngram after the corpus-path branch.

Common situations: Enabling the external corpus path but forgetting to set the SAM budget (it doesn't default to a positive value); copying configs from a version where the budget defaulted differently; passing 0 thinking it means 'auto'.

Related errors

• --speculative-ngram-external-corpus-max-tokens must be posit
• speculative_ngram_external_sam_budget must be less than or e
• Ngram speculative decoding only supports CUDA or CPU devices
• External ngram corpus path does not exist: {path}
• This browser cannot encode H.264 MP4

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/097a54c1ad5bacfd. Report an issue: GitHub.