{"record":{"id":"ec0add57316476a6","repo":"vllm-project/vllm","slug":"expected-num-speculative-tokens-to-be-greater-than","errorCode":null,"errorMessage":"Expected num_speculative_tokens to be greater than zero ({self.num_speculative_tokens}).","messagePattern":"Expected num_speculative_tokens to be greater than zero \\((.+?)\\)\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/speculative.py","lineNumber":1360,"sourceCode":"        return value\n\n    @model_validator(mode=\"after\")\n    def _verify_args(self) -> Self:\n        if self.tensor_parallel_size is not None:\n            raise ValueError(\n                \"'tensor_parallel_size' is not a valid argument in the \"\n                \"speculative_config. Please pass 'draft_tensor_parallel_size' instead.\"\n            )\n\n        if self.num_speculative_tokens is None:\n            raise ValueError(\n                \"num_speculative_tokens must be provided with \"\n                \"speculative model unless the draft model config contains an \"\n                \"n_predict parameter.\"\n            )\n\n        if self.num_speculative_tokens <= 0:\n            raise ValueError(\n                \"Expected num_speculative_tokens to be greater \"\n                f\"than zero ({self.num_speculative_tokens}).\"\n            )\n\n        if self.rejection_sample_method == \"synthetic\":\n            # Consolidate to per-position rates\n            self.synthetic_acceptance_rates = self._resolve_synthetic_acceptance_rates(\n                self.num_speculative_tokens,\n                self.synthetic_acceptance_rates,\n                self.synthetic_acceptance_length,\n            )\n            self.synthetic_acceptance_length = None\n        elif (\n            self.synthetic_acceptance_rates is not None\n            or self.synthetic_acceptance_length is not None\n        ):\n            raise ValueError(\n                \"synthetic_acceptance_rates / synthetic_acceptance_length \"","sourceCodeStart":1342,"sourceCodeEnd":1378,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/speculative.py#L1342-L1378","documentation":"Raised by the SpeculativeConfig._verify_args model_validator when num_speculative_tokens <= 0. The value sizes the draft loop and the verification token buffers; zero or negative tokens would mean no speculation at all or negative allocations, both invalid, so the config is rejected immediately after the presence check.","triggerScenarios":"Passing num_speculative_tokens=0 (a common attempt to 'disable' speculation while keeping the config) or a negative number via speculative_config or CLI.","commonSituations":"Trying to toggle speculation off per-request by zeroing the knob; arithmetic that computes the value from batch sizes or rates and can reach 0; sweeping parameter grids that include 0.","solutions":["Set num_speculative_tokens to >= 1 (e.g. 1 for minimal speculation)","To disable speculation entirely, remove the speculative_config / do not pass a speculative model, rather than zeroing the count","Fix generating arithmetic to clamp at 1"],"exampleFix":"# before\nspeculative_config={\"method\": \"ngram\", \"prompt_lookup_max\": 4, \"num_speculative_tokens\": 0}\n# after\nspeculative_config={\"method\": \"ngram\", \"prompt_lookup_max\": 4, \"num_speculative_tokens\": 3}","handlingStrategy":"validation","validationCode":"k = spec_cfg.get(\"num_speculative_tokens\")\nif k is not None:\n    assert k >= 1, f\"num_speculative_tokens={k} must be >= 1; omit the config to disable speculation\"","typeGuard":"def is_positive_token_count(k: int | None) -> bool:\n    return k is None or (isinstance(k, int) and k >= 1)","tryCatchPattern":null,"preventionTips":["Disable speculation by removing the speculative_config, never by zeroing num_speculative_tokens","Clamp computed token counts to >= 1 in tuning scripts"],"tags":["speculative-decoding","num-speculative-tokens","range-validation","pydantic"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}