vllm-project/vllm · error · ValueError

suffix_decoding_max_cached_requests={self.suffix_decoding_ma

Error message

suffix_decoding_max_cached_requests={self.suffix_decoding_max_cached_requests} must be >= 0

What it means

Raised by _validate_suffix_decoding when suffix_decoding_max_cached_requests < 0. This parameter caps how many finished requests' suffixes are retained in the global cache; a negative cap is meaningless and rejected. Zero is legal and means the global cache is effectively disabled (only per-request suffixes are used).

Source

Thrown at vllm/config/speculative.py:1167

                "Arctic Inference is required for suffix decoding. "
                "Install via `pip install arctic-inference==0.1.1`."
            )
        if self.num_speculative_tokens is None:
            # Suffix decoding decides the actual number of speculative tokens
            # dynamically and treats num_speculative_tokens as a maximum limit.
            self.num_speculative_tokens = self.suffix_decoding_max_tree_depth
            logger.warning(
                "Defaulted num_speculative_tokens to %s for suffix decoding.",
                self.num_speculative_tokens,
            )
        # Validate values
        if self.suffix_decoding_max_tree_depth < 1:
            raise ValueError(
                f"suffix_decoding_max_tree_depth="
                f"{self.suffix_decoding_max_tree_depth} must be >= 1"
            )
        if self.suffix_decoding_max_cached_requests < 0:
            raise ValueError(
                f"suffix_decoding_max_cached_requests="
                f"{self.suffix_decoding_max_cached_requests} must be >= 0"
            )
        if self.suffix_decoding_max_spec_factor < 0:
            raise ValueError(
                f"suffix_decoding_max_spec_factor="
                f"{self.suffix_decoding_max_spec_factor} must be >= 0"
            )
        if not 0 <= self.suffix_decoding_min_token_prob <= 1:
            raise ValueError(
                f"suffix_decoding_min_token_prob="
                f"{self.suffix_decoding_min_token_prob} must be in [0, 1]"
            )

    @staticmethod
    def _maybe_override_draft_max_model_len(
        speculative_max_model_len: int | None,
        draft_max_model_len: int,

View on GitHub (pinned to c794754062)

Solutions

1. Set suffix_decoding_max_cached_requests to >= 0 (use 0 to disable cross-request caching)
2. Remove the key to accept the default
3. Fix the generating expression so it clamps at 0

Example fix

# before
speculative_config={"method": "suffix", "suffix_decoding_max_cached_requests": -1}
# after
speculative_config={"method": "suffix", "suffix_decoding_max_cached_requests": 0}

spec_cfg["suffix_decoding_max_cached_requests"] = max(0, computed_cache_requests)

def is_valid_cached_requests(n: int) -> bool:
    return n >= 0

When it happens

Trigger: speculative_config={'method': 'suffix', 'suffix_decoding_max_cached_requests': -1}; config generation code computing the value from an expression that can go negative (e.g. cache_size - requests).

Common situations: Attempting to size the suffix cache from a formula that underflows on small deployments; sign errors when porting configs.

Related errors

• suffix_decoding_max_tree_depth={self.suffix_decoding_max_tre
• suffix_decoding_max_spec_factor={self.suffix_decoding_max_sp
• suffix_decoding_min_token_prob={self.suffix_decoding_min_tok
• dspark_draft_topk must be between 1 and the draft vocabulary
• Arctic Inference is required for suffix decoding. Install vi

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/0b5e4a078e8af18a. Report an issue: GitHub.