{"record":{"id":"a559117c3c0e76fa","repo":"vllm-project/vllm","slug":"num-speculative-tokens-must-be-provided-with-specu","errorCode":null,"errorMessage":"num_speculative_tokens must be provided with speculative model unless the draft model config contains an n_predict parameter.","messagePattern":"num_speculative_tokens must be provided with speculative model unless the draft model config contains an n_predict parameter\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/speculative.py","lineNumber":1353,"sourceCode":"    @field_validator(\"attention_backend\", mode=\"before\")\n    @classmethod\n    def _parse_attention_backend(cls, value: Any) -> Any:\n        if isinstance(value, str):\n            if value.lower() == \"auto\":\n                return None\n            return AttentionBackendEnum[value.upper()]\n        return value\n\n    @model_validator(mode=\"after\")\n    def _verify_args(self) -> Self:\n        if self.tensor_parallel_size is not None:\n            raise ValueError(\n                \"'tensor_parallel_size' is not a valid argument in the \"\n                \"speculative_config. Please pass 'draft_tensor_parallel_size' instead.\"\n            )\n\n        if self.num_speculative_tokens is None:\n            raise ValueError(\n                \"num_speculative_tokens must be provided with \"\n                \"speculative model unless the draft model config contains an \"\n                \"n_predict parameter.\"\n            )\n\n        if self.num_speculative_tokens <= 0:\n            raise ValueError(\n                \"Expected num_speculative_tokens to be greater \"\n                f\"than zero ({self.num_speculative_tokens}).\"\n            )\n\n        if self.rejection_sample_method == \"synthetic\":\n            # Consolidate to per-position rates\n            self.synthetic_acceptance_rates = self._resolve_synthetic_acceptance_rates(\n                self.num_speculative_tokens,\n                self.synthetic_acceptance_rates,\n                self.synthetic_acceptance_length,\n            )","sourceCodeStart":1335,"sourceCodeEnd":1371,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/speculative.py#L1335-L1371","documentation":"Raised by the SpeculativeConfig._verify_args model_validator when num_speculative_tokens is None at field-validation time. This is the schema-level backstop: the constructor-level defaulting (e.g. from the draft's n_predict, covered elsewhere in __post_init__) has not run or was unavailable, so the value must be supplied explicitly unless the draft config carries n_predict.","triggerScenarios":"Constructing SpeculativeConfig (directly or via speculative_config dict) with no num_speculative_tokens where the draft does not expose n_predict; commonly with method='ngram' plus explicit prompt_lookup settings, or draft_model-style configs.","commonSituations":"Minimal configs copied from docs examples that relied on a draft checkpoint default; direct programmatic use of SpeculativeConfig.from_kwargs where the key was dropped.","solutions":["Add num_speculative_tokens (positive int, typically 1-5) to the speculative_config","Use a draft checkpoint whose hf_config declares n_predict so the default applies","If using ngram, ensure the method branch you intend is actually selected so its defaulting logic runs"],"exampleFix":"# before\nSpeculativeConfig.from_kwargs(method=\"ngram\", prompt_lookup_max=4)  # missing count\n# after\nSpeculativeConfig.from_kwargs(method=\"ngram\", prompt_lookup_max=4, num_speculative_tokens=3)","handlingStrategy":"validation","validationCode":"REQUIRED_UNLESS_NPREDICT = {\"num_speculative_tokens\"}\nif \"num_speculative_tokens\" not in spec_cfg:\n    from transformers import AutoConfig\n    if not hasattr(AutoConfig.from_pretrained(spec_cfg[\"model\"]), \"n_predict\"):\n        spec_cfg.setdefault(\"num_speculative_tokens\", 3)","typeGuard":"def spec_cfg_is_complete(spec_cfg: dict) -> bool:\n    if \"num_speculative_tokens\" in spec_cfg:\n        return True\n    from transformers import AutoConfig\n    return hasattr(AutoConfig.from_pretrained(spec_cfg[\"model\"]), \"n_predict\")","tryCatchPattern":null,"preventionTips":["Default num_speculative_tokens in your config layer so it is never omitted","Unit-test config construction for each supported speculative method in CI"],"tags":["speculative-decoding","num-speculative-tokens","required-field","pydantic"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}