{"record":{"id":"fd319e425108c8d7","repo":"vllm-project/vllm","slug":"num-speculative-tokens-self-num-speculative-token","errorCode":null,"errorMessage":"num_speculative_tokens:{self.num_speculative_tokens} must be divisible by {n_predict=}","messagePattern":"num_speculative_tokens:(.+?) must be divisible by (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/speculative.py","lineNumber":1078,"sourceCode":"                    self.draft_model_config.hf_config, \"num_lookahead_tokens\"\n                ):\n                    self.draft_model_config.hf_config.num_lookahead_tokens = (\n                        self.num_speculative_tokens\n                    )\n\n                n_predict = getattr(\n                    self.draft_model_config.hf_config, \"n_predict\", None\n                )\n                if n_predict is not None:\n                    if self.num_speculative_tokens is None:\n                        # Default to max value defined in draft model config.\n                        self.num_speculative_tokens = n_predict\n                    elif (\n                        self.num_speculative_tokens > n_predict\n                        and self.num_speculative_tokens % n_predict != 0\n                    ):\n                        # Ensure divisibility for MTP module reuse.\n                        raise ValueError(\n                            f\"num_speculative_tokens:{self.num_speculative_tokens}\"\n                            f\" must be divisible by {n_predict=}\"\n                        )\n\n                if self.num_speculative_tokens is None:\n                    raise ValueError(\n                        \"A speculative model was provided, but \"\n                        \"`num_speculative_tokens` was not provided\"\n                    )\n\n                if self.dspark_draft_topk is not None and self.method != \"dspark\":\n                    raise ValueError(\"dspark_draft_topk is only supported by DSpark\")\n\n                dspark_draft_topk = None\n                if self.method == \"dspark\":\n                    hf_config = self.draft_model_config.hf_config\n                    dspark_draft_topk = self.dspark_draft_topk\n                    if dspark_draft_topk is None:","sourceCodeStart":1060,"sourceCodeEnd":1096,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/speculative.py#L1060-L1096","documentation":"Raised when the draft model's hf_config declares n_predict (typical for MTP-style heads) and the user-supplied num_speculative_tokens is larger than n_predict but not an exact multiple of it. vLLD reuses the MTP module iteratively, which requires the token count to be a whole multiple of the module's native n_predict step; otherwise the loop would need a partial pass.","triggerScenarios":"SpeculativeConfig with method='mtp' (or a draft whose hf_config has n_predict, e.g. n_predict=3) plus --num-speculative-tokens 4, 5, 7, etc. — any value > n_predict that does not divide evenly.","commonSituations":"Tuning num_speculative_tokens upward for throughput on an MTP model without knowing the head's native step; mixing configs between models whose n_predict differs (e.g. 3 vs 5).","solutions":["Set num_speculative_tokens to a multiple of the draft's n_predict (e.g. n_predict=3 -> 3, 6, 9)","Set num_speculative_tokens <= n_predict (any value at or below n_predict is allowed)","Omit num_speculative_tokens so it defaults to n_predict from the draft config"],"exampleFix":"# before\nspeculative_config={\"method\": \"mtp\", \"model\": \"...\", \"num_speculative_tokens\": 4}  # n_predict=3\n# after\nspeculative_config={\"method\": \"mtp\", \"model\": \"...\", \"num_speculative_tokens\": 6}","handlingStrategy":"validation","validationCode":"n_predict = getattr(draft_hf_config, \"n_predict\", None)\nif n_predict and num_spec_tokens and num_spec_tokens > n_predict and num_spec_tokens % n_predict != 0:\n    num_spec_tokens = ((num_spec_tokens + n_predict - 1) // n_predict) * n_predict  # round up to a multiple","typeGuard":"def is_valid_mtp_token_count(n_predict: int | None, k: int | None) -> bool:\n    return k is None or n_predict is None or k <= n_predict or k % n_predict == 0","tryCatchPattern":null,"preventionTips":["Read n_predict from the draft checkpoint config.json before choosing num_speculative_tokens","Round requested token counts up to the next multiple of n_predict in your config generator"],"tags":["speculative-decoding","mtp","num-speculative-tokens","validation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}