sgl-project/sglang · error · ValueError

--speculative-ngram-external-corpus-max-tokens must be posit

Error message

--speculative-ngram-external-corpus-max-tokens must be positive when --speculative-ngram-external-corpus-path is set.

What it means

When using an external ngram draft corpus, --speculative-ngram-external-corpus-max-tokens bounds how many tokens of the corpus are indexed; a zero/negative max means nothing would be indexed. SGLang's args hook validates that this value is positive whenever the corpus path is set.

Source

Thrown at python/sglang/srt/arg_groups/speculative_hook.py:898

        logger.warning(
            "speculative_num_draft_tokens is set to 12 by default for ngram speculative decoding. "
            "You can override this by explicitly setting --speculative-num-draft-tokens."
        )
    if cfg.speculative_num_steps is None:
        declare_resolution(
            server_args,
            "_handle_ngram",
            speculative_num_steps=cfg.speculative_num_draft_tokens
            // cfg.speculative_eagle_topk,
        )
    if cfg.speculative_ngram_external_corpus_path is not None:
        if cfg.speculative_ngram_external_sam_budget <= 0:
            raise ValueError(
                "--speculative-ngram-external-sam-budget must be positive when "
                "--speculative-ngram-external-corpus-path is set."
            )
        if cfg.speculative_ngram_external_corpus_max_tokens <= 0:
            raise ValueError(
                "--speculative-ngram-external-corpus-max-tokens must be positive when "
                "--speculative-ngram-external-corpus-path is set."
            )
        if (
            cfg.speculative_ngram_external_sam_budget
            > cfg.speculative_num_draft_tokens - 1
        ):
            raise ValueError(
                "speculative_ngram_external_sam_budget must be less than or equal to "
                f"speculative_num_draft_tokens - 1 ({cfg.speculative_num_draft_tokens - 1})."
            )
    logger.warning(
        "The mixed chunked prefill are disabled because of "
        "using ngram speculative decoding."
    )

    from sglang.srt.arg_groups.overrides import resolved_view

View on GitHub (pinned to 0132848349)

Solutions

1. Set --speculative-ngram-external-corpus-max-tokens to a positive token count covering your corpus
2. Size it >= the corpus token count if you want the whole corpus indexed
3. Drop the corpus path entirely if the external corpus isn't needed

Example fix

# before
--speculative-ngram-external-corpus-path /data/corpus.bin --speculative-ngram-external-sam-budget 4
# after
--speculative-ngram-external-corpus-path /data/corpus.bin --speculative-ngram-external-sam-budget 4 --speculative-ngram-external-corpus-max-tokens 100000000

if server_args.speculative_ngram_external_corpus_path is not None:
    assert server_args.speculative_ngram_external_corpus_max_tokens > 0

When it happens

Trigger: Launching with --speculative-ngram-external-corpus-path set and --speculative-ngram-external-corpus-max-tokens <= 0 (explicitly or via a stale default).

Common situations: Setting only the corpus path and SAM budget but not max-tokens; porting a config from an older SGLang where the parameter did not exist or defaulted to unlimited; passing 0 expecting 'no limit'.

Related errors

• --speculative-ngram-external-sam-budget must be positive whe
• speculative_ngram_external_sam_budget must be less than or e
• Ngram speculative decoding only supports CUDA or CPU devices
• External ngram corpus path does not exist: {path}
• This browser cannot encode H.264 MP4

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/69a96b9b2cfb830a. Report an issue: GitHub.