{"record":{"id":"d861aacdfaf5bd3d","repo":"sgl-project/sglang","slug":"speculative-ngram-external-sam-budget-must-be-less","errorCode":null,"errorMessage":"speculative_ngram_external_sam_budget must be less than or equal to speculative_num_draft_tokens - 1 ({cfg.speculative_num_draft_tokens - 1}).","messagePattern":"speculative_ngram_external_sam_budget must be less than or equal to speculative_num_draft_tokens - 1 \\((.+?)\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/arg_groups/speculative_hook.py","lineNumber":906,"sourceCode":"            speculative_num_steps=cfg.speculative_num_draft_tokens\n            // cfg.speculative_eagle_topk,\n        )\n    if cfg.speculative_ngram_external_corpus_path is not None:\n        if cfg.speculative_ngram_external_sam_budget <= 0:\n            raise ValueError(\n                \"--speculative-ngram-external-sam-budget must be positive when \"\n                \"--speculative-ngram-external-corpus-path is set.\"\n            )\n        if cfg.speculative_ngram_external_corpus_max_tokens <= 0:\n            raise ValueError(\n                \"--speculative-ngram-external-corpus-max-tokens must be positive when \"\n                \"--speculative-ngram-external-corpus-path is set.\"\n            )\n        if (\n            cfg.speculative_ngram_external_sam_budget\n            > cfg.speculative_num_draft_tokens - 1\n        ):\n            raise ValueError(\n                \"speculative_ngram_external_sam_budget must be less than or equal to \"\n                f\"speculative_num_draft_tokens - 1 ({cfg.speculative_num_draft_tokens - 1}).\"\n            )\n    logger.warning(\n        \"The mixed chunked prefill are disabled because of \"\n        \"using ngram speculative decoding.\"\n    )\n\n    from sglang.srt.arg_groups.overrides import resolved_view\n\n    view = resolved_view(server_args)\n    if (\n        cfg.speculative_eagle_topk > 1\n        and view.page_size > 1\n        and view.attention_backend != \"flashinfer\"\n    ):\n        raise ValueError(\n            f\"speculative_eagle_topk({cfg.speculative_eagle_topk}) > 1 \"","sourceCodeStart":888,"sourceCodeEnd":924,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/arg_groups/speculative_hook.py#L888-L924","documentation":"The external-corpus suffix-array budget reserves speculative_ngram_external_sam_budget of the per-step draft slots: at least one slot must remain for the request-local ngram matcher, so the budget cannot exceed speculative_num_draft_tokens - 1. SGLang enforces this bound with a ValueError in _handle_ngram.","triggerScenarios":"Launching with --speculative-ngram-external-corpus-path where speculative_ngram_external_sam_budget > speculative_num_draft_tokens - 1, e.g. budget 8 with --speculative-num-draft-tokens 4.","commonSituations":"Raising the SAM budget for better draft hit-rate without also raising --speculative-num-draft-tokens; using a small draft-token count (e.g. 2 or 3) that leaves almost no room for external matches.","solutions":["Raise --speculative-num-draft-tokens to at least sam_budget + 1","Or lower --speculative-ngram-external-sam-budget to <= speculative_num_draft_tokens - 1","Remember both constraints together: 1 <= sam_budget <= num_draft_tokens - 1"],"exampleFix":"# before\n--speculative-num-draft-tokens 4 --speculative-ngram-external-sam-budget 8\n# after\n--speculative-num-draft-tokens 8 --speculative-ngram-external-sam-budget 8","handlingStrategy":"validation","validationCode":"b = server_args.speculative_ngram_external_sam_budget\nn = server_args.speculative_num_draft_tokens\nif server_args.speculative_ngram_external_corpus_path is not None:\n    assert 1 <= b <= n - 1, f\"sam_budget {b} must be in [1, {n - 1}]\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Scale num_draft_tokens with sam_budget: draft tokens >= sam_budget + 1","Add a config lint step that validates ngram external-corpus bounds before launch"],"tags":["speculative-decoding","ngram","external-corpus","draft-tokens","server-args","validation"],"backgroundTag":"invalid-launch-argument-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}