vllm-project/vllm · error · ValueError

num_speculative_tokens:{self.num_speculative_tokens} must be

Error message

num_speculative_tokens:{self.num_speculative_tokens} must be divisible by {n_predict=}

What it means

Raised when the draft model's hf_config declares n_predict (typical for MTP-style heads) and the user-supplied num_speculative_tokens is larger than n_predict but not an exact multiple of it. vLLD reuses the MTP module iteratively, which requires the token count to be a whole multiple of the module's native n_predict step; otherwise the loop would need a partial pass.

Source

Thrown at vllm/config/speculative.py:1078

                    self.draft_model_config.hf_config, "num_lookahead_tokens"
                ):
                    self.draft_model_config.hf_config.num_lookahead_tokens = (
                        self.num_speculative_tokens
                    )

                n_predict = getattr(
                    self.draft_model_config.hf_config, "n_predict", None
                )
                if n_predict is not None:
                    if self.num_speculative_tokens is None:
                        # Default to max value defined in draft model config.
                        self.num_speculative_tokens = n_predict
                    elif (
                        self.num_speculative_tokens > n_predict
                        and self.num_speculative_tokens % n_predict != 0
                    ):
                        # Ensure divisibility for MTP module reuse.
                        raise ValueError(
                            f"num_speculative_tokens:{self.num_speculative_tokens}"
                            f" must be divisible by {n_predict=}"
                        )

                if self.num_speculative_tokens is None:
                    raise ValueError(
                        "A speculative model was provided, but "
                        "`num_speculative_tokens` was not provided"
                    )

                if self.dspark_draft_topk is not None and self.method != "dspark":
                    raise ValueError("dspark_draft_topk is only supported by DSpark")

                dspark_draft_topk = None
                if self.method == "dspark":
                    hf_config = self.draft_model_config.hf_config
                    dspark_draft_topk = self.dspark_draft_topk
                    if dspark_draft_topk is None:

View on GitHub (pinned to c794754062)

Solutions

1. Set num_speculative_tokens to a multiple of the draft's n_predict (e.g. n_predict=3 -> 3, 6, 9)
2. Set num_speculative_tokens <= n_predict (any value at or below n_predict is allowed)
3. Omit num_speculative_tokens so it defaults to n_predict from the draft config

Example fix

# before
speculative_config={"method": "mtp", "model": "...", "num_speculative_tokens": 4}  # n_predict=3
# after
speculative_config={"method": "mtp", "model": "...", "num_speculative_tokens": 6}

n_predict = getattr(draft_hf_config, "n_predict", None)
if n_predict and num_spec_tokens and num_spec_tokens > n_predict and num_spec_tokens % n_predict != 0:
    num_spec_tokens = ((num_spec_tokens + n_predict - 1) // n_predict) * n_predict  # round up to a multiple

def is_valid_mtp_token_count(n_predict: int | None, k: int | None) -> bool:
    return k is None or n_predict is None or k <= n_predict or k % n_predict == 0

When it happens

Trigger: SpeculativeConfig with method='mtp' (or a draft whose hf_config has n_predict, e.g. n_predict=3) plus --num-speculative-tokens 4, 5, 7, etc. — any value > n_predict that does not divide evenly.

Common situations: Tuning num_speculative_tokens upward for throughput on an MTP model without knowing the head's native step; mixing configs between models whose n_predict differs (e.g. 3 vs 5).

Related errors

• rejection_sample_method='synthetic' requires exactly one of
• synthetic_acceptance_rates must have length {n}, got {rates}
• synthetic_acceptance_rates entries must be in [0, 1], got {r
• synthetic_acceptance_rates must be non-increasing, got {rate
• synthetic_acceptance_length must be in [1, {n + 1}], got {le

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/fd319e425108c8d7. Report an issue: GitHub.