{"record":{"id":"0ba60a6fb2376a2e","repo":"vllm-project/vllm","slug":"a-speculative-model-was-provided-but-num-specula","errorCode":null,"errorMessage":"A speculative model was provided, but `num_speculative_tokens` was not provided","messagePattern":"A speculative model was provided, but `num_speculative_tokens` was not provided","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/speculative.py","lineNumber":1084,"sourceCode":"                n_predict = getattr(\n                    self.draft_model_config.hf_config, \"n_predict\", None\n                )\n                if n_predict is not None:\n                    if self.num_speculative_tokens is None:\n                        # Default to max value defined in draft model config.\n                        self.num_speculative_tokens = n_predict\n                    elif (\n                        self.num_speculative_tokens > n_predict\n                        and self.num_speculative_tokens % n_predict != 0\n                    ):\n                        # Ensure divisibility for MTP module reuse.\n                        raise ValueError(\n                            f\"num_speculative_tokens:{self.num_speculative_tokens}\"\n                            f\" must be divisible by {n_predict=}\"\n                        )\n\n                if self.num_speculative_tokens is None:\n                    raise ValueError(\n                        \"A speculative model was provided, but \"\n                        \"`num_speculative_tokens` was not provided\"\n                    )\n\n                if self.dspark_draft_topk is not None and self.method != \"dspark\":\n                    raise ValueError(\"dspark_draft_topk is only supported by DSpark\")\n\n                dspark_draft_topk = None\n                if self.method == \"dspark\":\n                    hf_config = self.draft_model_config.hf_config\n                    dspark_draft_topk = self.dspark_draft_topk\n                    if dspark_draft_topk is None:\n                        dspark_draft_topk = getattr(\n                            hf_config, \"dspark_draft_topk\", None\n                        )\n                    if dspark_draft_topk is not None:\n                        draft_vocab_size = (\n                            getattr(hf_config, \"draft_vocab_size\", None)","sourceCodeStart":1066,"sourceCodeEnd":1102,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/speculative.py#L1066-L1102","documentation":"Raised when a speculative/draft model is configured but num_speculative_tokens is None after all defaulting: the user did not pass it and the draft model's hf_config has no n_predict attribute to fall back on. Without this count vLLM cannot size the draft loop or the verification buffers, so it fails fast during SpeculativeConfig.__post_init__.","triggerScenarios":"speculative_config={'method': 'eagle', 'model': '<draft>'} with no num_speculative_tokens key, where the draft checkpoint config lacks n_predict. Common with plain eagle/eagle3 draft heads and draft_model method.","commonSituations":"Following an EAGLE quick-start that omits num_speculative_tokens; assuming the draft checkpoint always carries n_predict (only MTP-style heads usually do).","solutions":["Add num_speculative_tokens to the speculative_config (typical values 1-5, e.g. 3 for EAGLE)","Use a draft checkpoint whose config declares n_predict so it can be defaulted","Pass --num-speculative-tokens N on the serve/serve CLI equivalent"],"exampleFix":"# before\nspeculative_config={\"method\": \"eagle\", \"model\": \"yuhuili/EAGLE-LLaMA3.1-Instruct-8B\"}\n# after\nspeculative_config={\"method\": \"eagle\", \"model\": \"yuhuili/EAGLE-LLaMA3.1-Instruct-8B\", \"num_speculative_tokens\": 3}","handlingStrategy":"validation","validationCode":"if spec_cfg.get(\"model\") and \"num_speculative_tokens\" not in spec_cfg:\n    from transformers import AutoConfig\n    if not hasattr(AutoConfig.from_pretrained(spec_cfg[\"model\"]), \"n_predict\"):\n        spec_cfg[\"num_speculative_tokens\"] = 3  # sensible default for eagle-style drafts","typeGuard":"def needs_explicit_token_count(spec_cfg: dict) -> bool:\n    if \"num_speculative_tokens\" in spec_cfg:\n        return False\n    from transformers import AutoConfig\n    return not hasattr(AutoConfig.from_pretrained(spec_cfg[\"model\"]), \"n_predict\")","tryCatchPattern":null,"preventionTips":["Always set num_speculative_tokens explicitly in shared config templates instead of relying on draft defaults","Wrap engine construction in a startup validator that checks required speculative keys per method"],"tags":["speculative-decoding","config","num-speculative-tokens","required-field"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}