vllm-project/vllm · error · ValueError

suffix_decoding_min_token_prob=

Error message

suffix_decoding_min_token_prob={self.suffix_decoding_min_token_prob} must be in [0, 1]

What it means

Raised by _validate_suffix_decoding when suffix_decoding_min_token_prob is outside [0, 1]. The value is a probability threshold used to prune low-confidence draft tokens from the suffix tree; anything below 0 or above 1 is not a valid probability.

Solutions

1. Set suffix_decoding_min_token_prob to a fraction in [0, 1] (e.g. 0.05 to prune unlikely tokens)
2. Divide percentage values by 100 before passing
3. Remove the key to use the default

Example fix

# before
speculative_config={"method": "suffix", "suffix_decoding_min_token_prob": 5}
# after
speculative_config={"method": "suffix", "suffix_decoding_min_token_prob": 0.05}

p = spec_cfg.get("suffix_decoding_min_token_prob", 0.0)
assert 0.0 <= p <= 1.0, f"min_token_prob={p} must be a probability in [0, 1]"

def is_probability(p: float) -> bool:
    return 0.0 <= p <= 1.0

When it happens

Trigger: speculative_config={'method': 'suffix', 'suffix_decoding_min_token_prob': 1.5} or a negative value; passing a percentage (e.g. 50) instead of a fraction.

Common situations: Unit confusion (percent vs. probability); sweeping thresholds that overshoot the [0,1] interval; copy-paste from configs of systems that use log-probabilities.

Related errors

• suffix_decoding_max_cached_requests=
• suffix_decoding_max_spec_factor=
• suffix_decoding_max_tree_depth=
• Arctic Inference is required for suffix decoding. Install…
• dspark_draft_topk must be between 1 and the draft…

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/d6e875e328f3c71e. Report an issue: GitHub.

Appendix: source

Thrown at vllm/config/speculative.py:1177

            )
        # Validate values
        if self.suffix_decoding_max_tree_depth < 1:
            raise ValueError(
                f"suffix_decoding_max_tree_depth="
                f"{self.suffix_decoding_max_tree_depth} must be >= 1"
            )
        if self.suffix_decoding_max_cached_requests < 0:
            raise ValueError(
                f"suffix_decoding_max_cached_requests="
                f"{self.suffix_decoding_max_cached_requests} must be >= 0"
            )
        if self.suffix_decoding_max_spec_factor < 0:
            raise ValueError(
                f"suffix_decoding_max_spec_factor="
                f"{self.suffix_decoding_max_spec_factor} must be >= 0"
            )
        if not 0 <= self.suffix_decoding_min_token_prob <= 1:
            raise ValueError(
                f"suffix_decoding_min_token_prob="
                f"{self.suffix_decoding_min_token_prob} must be in [0, 1]"
            )

    @staticmethod
    def _maybe_override_draft_max_model_len(
        speculative_max_model_len: int | None,
        draft_max_model_len: int,
        target_max_model_len: int,
    ) -> int:
        """Determine the max sequence len for the draft model. This is usually
        the draft_max_model_len, but may be the target_max_model_len if it is
        less than the draft_max_model_len, or may be speculative_max_model_len
        if it is specified.

        This is necessary so that sequences do not exceed the capacity of the
        draft model or the target model.

View on GitHub (pinned to c794754062)