vllm-project/vllm · error · ValueError

suffix_decoding_max_tree_depth={self.suffix_decoding_max_tre

Error message

suffix_decoding_max_tree_depth={self.suffix_decoding_max_tree_depth} must be >= 1

What it means

Raised by _validate_suffix_decoding when suffix_decoding_max_tree_depth < 1. The tree depth bounds how deep the suffix-automaton draft tree grows per step; zero or negative depth yields an empty speculation tree, so it is rejected. Note when num_speculative_tokens is unset it is defaulted to this depth, which also makes a 0 value degenerate.

Source

Thrown at vllm/config/speculative.py:1162

        return self

    def _validate_suffix_decoding(self):
        if not has_arctic_inference():
            raise ImportError(
                "Arctic Inference is required for suffix decoding. "
                "Install via `pip install arctic-inference==0.1.1`."
            )
        if self.num_speculative_tokens is None:
            # Suffix decoding decides the actual number of speculative tokens
            # dynamically and treats num_speculative_tokens as a maximum limit.
            self.num_speculative_tokens = self.suffix_decoding_max_tree_depth
            logger.warning(
                "Defaulted num_speculative_tokens to %s for suffix decoding.",
                self.num_speculative_tokens,
            )
        # Validate values
        if self.suffix_decoding_max_tree_depth < 1:
            raise ValueError(
                f"suffix_decoding_max_tree_depth="
                f"{self.suffix_decoding_max_tree_depth} must be >= 1"
            )
        if self.suffix_decoding_max_cached_requests < 0:
            raise ValueError(
                f"suffix_decoding_max_cached_requests="
                f"{self.suffix_decoding_max_cached_requests} must be >= 0"
            )
        if self.suffix_decoding_max_spec_factor < 0:
            raise ValueError(
                f"suffix_decoding_max_spec_factor="
                f"{self.suffix_decoding_max_spec_factor} must be >= 0"
            )
        if not 0 <= self.suffix_decoding_min_token_prob <= 1:
            raise ValueError(
                f"suffix_decoding_min_token_prob="
                f"{self.suffix_decoding_min_token_prob} must be in [0, 1]"
            )

View on GitHub (pinned to c794754062)

Solutions

1. Set suffix_decoding_max_tree_depth to >= 1 (typical values are 4-64 depending on workload)
2. Remove the key to use the default depth

Example fix

# before
speculative_config={"method": "suffix", "suffix_decoding_max_tree_depth": 0}
# after
speculative_config={"method": "suffix", "suffix_decoding_max_tree_depth": 16}

if spec_cfg.get("method") == "suffix":
    assert (d := spec_cfg.get("suffix_decoding_max_tree_depth", 16)) >= 1, "tree depth must be >= 1"

def is_valid_tree_depth(depth: int) -> bool:
    return depth >= 1

When it happens

Trigger: speculative_config={'method': 'suffix', 'suffix_decoding_max_tree_depth': 0} or a negative value; env-driven config templating that zero-initializes tuning knobs.

Common situations: Disabling the feature by setting it to 0 instead of removing it; sweeping tuning parameters and passing 0 as the low end of the sweep.

Related errors

• suffix_decoding_max_spec_factor={self.suffix_decoding_max_sp
• suffix_decoding_max_cached_requests={self.suffix_decoding_ma
• suffix_decoding_min_token_prob={self.suffix_decoding_min_tok
• rejection_sample_method='synthetic' requires exactly one of
• synthetic_acceptance_rates must have length {n}, got {rates}

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/547176ea10d26eac. Report an issue: GitHub.