vllm-project/vllm · error · ValueError

Expected num_speculative_tokens to be greater than zero ({se

Error message

Expected num_speculative_tokens to be greater than zero ({self.num_speculative_tokens}).

What it means

Raised by the SpeculativeConfig._verify_args model_validator when num_speculative_tokens <= 0. The value sizes the draft loop and the verification token buffers; zero or negative tokens would mean no speculation at all or negative allocations, both invalid, so the config is rejected immediately after the presence check.

Source

Thrown at vllm/config/speculative.py:1360

        return value

    @model_validator(mode="after")
    def _verify_args(self) -> Self:
        if self.tensor_parallel_size is not None:
            raise ValueError(
                "'tensor_parallel_size' is not a valid argument in the "
                "speculative_config. Please pass 'draft_tensor_parallel_size' instead."
            )

        if self.num_speculative_tokens is None:
            raise ValueError(
                "num_speculative_tokens must be provided with "
                "speculative model unless the draft model config contains an "
                "n_predict parameter."
            )

        if self.num_speculative_tokens <= 0:
            raise ValueError(
                "Expected num_speculative_tokens to be greater "
                f"than zero ({self.num_speculative_tokens})."
            )

        if self.rejection_sample_method == "synthetic":
            # Consolidate to per-position rates
            self.synthetic_acceptance_rates = self._resolve_synthetic_acceptance_rates(
                self.num_speculative_tokens,
                self.synthetic_acceptance_rates,
                self.synthetic_acceptance_length,
            )
            self.synthetic_acceptance_length = None
        elif (
            self.synthetic_acceptance_rates is not None
            or self.synthetic_acceptance_length is not None
        ):
            raise ValueError(
                "synthetic_acceptance_rates / synthetic_acceptance_length "

View on GitHub (pinned to c794754062)

Solutions

1. Set num_speculative_tokens to >= 1 (e.g. 1 for minimal speculation)
2. To disable speculation entirely, remove the speculative_config / do not pass a speculative model, rather than zeroing the count
3. Fix generating arithmetic to clamp at 1

Example fix

# before
speculative_config={"method": "ngram", "prompt_lookup_max": 4, "num_speculative_tokens": 0}
# after
speculative_config={"method": "ngram", "prompt_lookup_max": 4, "num_speculative_tokens": 3}

k = spec_cfg.get("num_speculative_tokens")
if k is not None:
    assert k >= 1, f"num_speculative_tokens={k} must be >= 1; omit the config to disable speculation"

def is_positive_token_count(k: int | None) -> bool:
    return k is None or (isinstance(k, int) and k >= 1)

When it happens

Trigger: Passing num_speculative_tokens=0 (a common attempt to 'disable' speculation while keeping the config) or a negative number via speculative_config or CLI.

Common situations: Trying to toggle speculation off per-request by zeroing the knob; arithmetic that computes the value from batch sizes or rates and can reach 0; sweeping parameter grids that include 0.

Related errors

• num_speculative_tokens must be provided with speculative mod
• num_speculative_tokens:{self.num_speculative_tokens} must be
• A speculative model was provided, but `num_speculative_token
• dspark_draft_topk must be between 1 and the draft vocabulary
• suffix_decoding_max_tree_depth={self.suffix_decoding_max_tre

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/ec0add57316476a6. Report an issue: GitHub.