vllm-project/vllm · error · ValueError

prompt_lookup_min={self.prompt_lookup_min} must be <= prompt

Q: What causes "prompt_lookup_min={self.prompt_lookup_min} must be <= prompt_lookup_max={self.prompt_lookup_max}" in vllm-project/vllm?

Set prompt_lookup_max >= prompt_lookup_min (e.g. raise max or lower min) in the speculative_config dict/JSON

Error message

prompt_lookup_min={self.prompt_lookup_min} must be <= prompt_lookup_max={self.prompt_lookup_max}

What it means

Raised by SpeculativeConfig when using the 'ngram' speculative decoding method and prompt_lookup_min ends up strictly greater than prompt_lookup_max after the defaulting logic (unset max defaults to min, unset min defaults to max). The ngram worker uses both bounds to size its lookup window, so an inverted range is invalid. It is a plain ValueError thrown during engine/config construction, before any model loads.

Source

Thrown at vllm/config/speculative.py:826

                self.prompt_lookup_max = 5
            elif self.prompt_lookup_min is None:
                if self.prompt_lookup_max is None:
                    raise ValueError(
                        "Either prompt_lookup_max or prompt_lookup_min must be "
                        "provided when using the ngram method."
                    )
                self.prompt_lookup_min = self.prompt_lookup_max
            elif self.prompt_lookup_max is None:
                if self.prompt_lookup_min is None:
                    raise ValueError(
                        "Either prompt_lookup_max or prompt_lookup_min must be "
                        "provided when using the ngram method."
                    )
                self.prompt_lookup_max = self.prompt_lookup_min

            # Validate values
            if self.prompt_lookup_min > self.prompt_lookup_max:
                raise ValueError(
                    f"prompt_lookup_min={self.prompt_lookup_min} must "
                    f"be <= prompt_lookup_max={self.prompt_lookup_max}"
                )

            # TODO: current we still need extract vocab_size from target model
            # config, in future, we may try refactor it out, and set
            # draft related config as None here.
            self.draft_model_config = self.target_model_config
            self.draft_parallel_config = self.target_parallel_config
        elif self.method == "suffix":
            self._validate_suffix_decoding()
        elif self.method == "custom_class":
            # Custom class proposer does not need a draft model.
            # It will dynamically load the user-provided class at runtime.
            logger.warning_once(
                "Using a custom class-based proposer backend. This is an "
                "experimental feature and the proposer interface is subject to "
                "breaking changes in future vLLM releases."

View on GitHub (pinned to c794754062)

Solutions

Set prompt_lookup_max >= prompt_lookup_min (e.g. raise max or lower min) in the speculative_config dict/JSON
Omit prompt_lookup_min entirely so it defaults to prompt_lookup_max
Omit prompt_lookup_max so it defaults to prompt_lookup_min

Example fix

# before
speculative_config={"method": "ngram", "prompt_lookup_min": 10, "prompt_lookup_max": 4}
# after
speculative_config={"method": "ngram", "prompt_lookup_min": 4, "prompt_lookup_max": 10}

Defensive patterns

Strategy: validation

Validate before calling

def check_ngram_bounds(pl_min: int | None, pl_max: int | None) -> None:
    if pl_min is not None and pl_max is not None and pl_min > pl_max:
        raise ValueError(f"prompt_lookup_min={pl_min} must be <= prompt_lookup_max={pl_max}")

check_ngram_bounds(cfg.get("prompt_lookup_min"), cfg.get("prompt_lookup_max"))
speculative_config = cfg

Type guard

def is_valid_ngram_pair(pl_min: int | None, pl_max: int | None) -> bool:
    return pl_min is None or pl_max is None or pl_min <= pl_max

Prevention

Keep ngram lookup knobs in one place (a dataclass or single YAML block) so min/max cannot drift apart
Assert min <= max in your config loader before handing the dict to vLLM

When it happens

Trigger: Passing speculative_config='{"method": "ngram", "prompt_lookup_min": 5, "prompt_lookup_max": 3}' (or the equivalent CLI flags --speculative-config / --ngram-prompt-lookup-max with a smaller max than min). Any ngram config where min > max after defaults are applied.

Common situations: Copying an ngram config example and tuning the two knobs independently; swapping min/max values by hand; setting a large prompt_lookup_min hoping to raise the window while leaving an old smaller max in the config string.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/b5014bb1c395a08c. Report an issue: GitHub.