vllm-project/vllm · error · ValueError

Either prompt_lookup_max or prompt_lookup_min must be provid

Error message

Either prompt_lookup_max or prompt_lookup_min must be provided when using the ngram method.

What it means

Defensive branch while defaulting ngram window sizes: if prompt_lookup_min is None it is copied from prompt_lookup_max; the inner raise fires only if max were also None, which the preceding both-None branch (defaults 5/5) already handles — i.e. in the current code this specific raise is effectively unreachable. In practice, omitting both values gives you 5/5, not this error.

Source

Thrown at vllm/config/speculative.py:811

                        "custom proposer module path (e.g., 'my_module.MyProposer')."
                    )
            else:
                raise ValueError(
                    "num_speculative_tokens was provided but without speculative model."
                )

        if self.method in ("ngram", "[ngram]"):
            self.method = "ngram"

        if self.method in ("ngram", "ngram_gpu"):
            # Set default values if not provided
            if self.prompt_lookup_min is None and self.prompt_lookup_max is None:
                # TODO(woosuk): Tune these values. They are arbitrarily chosen.
                self.prompt_lookup_min = 5
                self.prompt_lookup_max = 5
            elif self.prompt_lookup_min is None:
                if self.prompt_lookup_max is None:
                    raise ValueError(
                        "Either prompt_lookup_max or prompt_lookup_min must be "
                        "provided when using the ngram method."
                    )
                self.prompt_lookup_min = self.prompt_lookup_max
            elif self.prompt_lookup_max is None:
                if self.prompt_lookup_min is None:
                    raise ValueError(
                        "Either prompt_lookup_max or prompt_lookup_min must be "
                        "provided when using the ngram method."
                    )
                self.prompt_lookup_max = self.prompt_lookup_min

            # Validate values
            if self.prompt_lookup_min > self.prompt_lookup_max:
                raise ValueError(
                    f"prompt_lookup_min={self.prompt_lookup_min} must "
                    f"be <= prompt_lookup_max={self.prompt_lookup_max}"
                )

View on GitHub (pinned to c794754062)

Solutions

1. Rely on the built-in default: pass neither prompt_lookup_min nor prompt_lookup_max (both become 5)
2. Or set at least one of prompt_lookup_min/prompt_lookup_max explicitly
3. If you maintain a fork, keep the both-None default branch ahead of this guard

Example fix

# before
SpeculativeConfig(method='ngram', num_speculative_tokens=3, prompt_lookup_min=None, prompt_lookup_max=None)  # on a fork without the default branch

# after
SpeculativeConfig(method='ngram', num_speculative_tokens=3, prompt_lookup_min=5, prompt_lookup_max=5)

def ngram_windows_ok(plmin: int | None, plmax: int | None) -> bool:
    return not (plmin is None and plmax is None) or True  # stock code defaults both to 5; guard only for forks

def ngram_windows_explicit(plmin: int | None, plmax: int | None) -> bool:
    return plmin is not None or plmax is not None

null

null

When it happens

Trigger: Only reachable if the both-None early branch is removed or the values are mutated between the checks (e.g. a subclass overriding __post_init__ behavior); with stock vLLM, normal ngram usage never triggers it.

Common situations: Forked/patched SpeculativeConfig where the defaulting order changed; reading old source where the guard was the primary path.

Related errors

• prompt_lookup_min={self.prompt_lookup_min} must be <= prompt
• rejection_sample_method='synthetic' requires exactly one of
• synthetic_acceptance_rates must have length {n}, got {rates}
• synthetic_acceptance_rates entries must be in [0, 1], got {r
• synthetic_acceptance_rates must be non-increasing, got {rate

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/65f78984440173c3. Report an issue: GitHub.