vllm-project/vllm · error · ValueError

num_speculative_tokens was provided but without speculative

Error message

num_speculative_tokens was provided but without speculative model.

What it means

Catch-all in SpeculativeConfig init: num_speculative_tokens was provided, no draft model was given, and method did not match any known self-drafting method (mtp, dspark, ngram, ngram_gpu, suffix, extract_hidden_states, custom_class). There is nothing vLLM can derive a draft from, so it errors instead of enabling broken speculation.

Source

Thrown at vllm/config/speculative.py:796

                    self.quantization = self.target_model_config.quantization
            elif self.method in ("ngram", "[ngram]"):
                self.model = "ngram"
            elif self.method == "ngram_gpu":
                self.model = "ngram_gpu"
            elif self.method == "suffix":
                self.model = "suffix"
            elif self.method == "extract_hidden_states":
                self.model = "extract_hidden_states"
            elif self.method == "custom_class":
                # method was set explicitly, but model should already contain the
                # custom module path. If not, this is a configuration error.
                if self.model is None:
                    raise ValueError(
                        "method='custom_class' requires 'model' to contain the "
                        "custom proposer module path (e.g., 'my_module.MyProposer')."
                    )
            else:
                raise ValueError(
                    "num_speculative_tokens was provided but without speculative model."
                )

        if self.method in ("ngram", "[ngram]"):
            self.method = "ngram"

        if self.method in ("ngram", "ngram_gpu"):
            # Set default values if not provided
            if self.prompt_lookup_min is None and self.prompt_lookup_max is None:
                # TODO(woosuk): Tune these values. They are arbitrarily chosen.
                self.prompt_lookup_min = 5
                self.prompt_lookup_max = 5
            elif self.prompt_lookup_min is None:
                if self.prompt_lookup_max is None:
                    raise ValueError(
                        "Either prompt_lookup_max or prompt_lookup_min must be "
                        "provided when using the ngram method."
                    )

View on GitHub (pinned to c794754062)

Solutions

1. Provide a draft: --speculative-model <draft checkpoint> (method defaults to draft_model)
2. Or set --speculative-method to a valid self-drafting method (mtp, ngram, ngram_gpu, suffix, extract_hidden_states, custom_class)
3. Check the method spelling against SpeculativeConfig docs/get_args

Example fix

# before
--num-speculative-tokens 3

# after
--num-speculative-tokens 3 --speculative-method ngram
# or
--num-speculative-tokens 3 --speculative-model Qwen2.5-0.5B

SELF_DRAFTING = {'mtp', 'dspark', 'ngram', 'ngram_gpu', 'suffix', 'extract_hidden_states', 'custom_class'}

def speculation_resolvable(method: str | None, model: str | None, n: int | None) -> bool:
    return n is None or model is not None or (method in SELF_DRAFTING)

null

null

When it happens

Trigger: --num-speculative-tokens 3 without --speculative-model; method misspelled (e.g. 'ngam', 'MTP') so it falls into the else branch; passing num_speculative_tokens via env/config while the method field was never set to a valid value.

Common situations: Assuming num_speculative-tokens alone enables speculation; typos in method strings; config keys silently not parsed in older versions so method stays None/draft_model.

Related errors

• target_model_config must be present for mtp
• target_model_config must be present for dspark
• method='custom_class' requires 'model' to contain the custom
• The model is not multimodal.
• quantization_config is only supported when quantization is o

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/023468e2210cc6f9. Report an issue: GitHub.