{"record":{"id":"0b5e4a078e8af18a","repo":"vllm-project/vllm","slug":"suffix-decoding-max-cached-requests-self-suffix-d","errorCode":null,"errorMessage":"suffix_decoding_max_cached_requests={self.suffix_decoding_max_cached_requests} must be >= 0","messagePattern":"suffix_decoding_max_cached_requests=(.+?) must be >= 0","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"vllm/config/speculative.py","lineNumber":1167,"sourceCode":"                \"Arctic Inference is required for suffix decoding. \"\n                \"Install via `pip install arctic-inference==0.1.1`.\"\n            )\n        if self.num_speculative_tokens is None:\n            # Suffix decoding decides the actual number of speculative tokens\n            # dynamically and treats num_speculative_tokens as a maximum limit.\n            self.num_speculative_tokens = self.suffix_decoding_max_tree_depth\n            logger.warning(\n                \"Defaulted num_speculative_tokens to %s for suffix decoding.\",\n                self.num_speculative_tokens,\n            )\n        # Validate values\n        if self.suffix_decoding_max_tree_depth < 1:\n            raise ValueError(\n                f\"suffix_decoding_max_tree_depth=\"\n                f\"{self.suffix_decoding_max_tree_depth} must be >= 1\"\n            )\n        if self.suffix_decoding_max_cached_requests < 0:\n            raise ValueError(\n                f\"suffix_decoding_max_cached_requests=\"\n                f\"{self.suffix_decoding_max_cached_requests} must be >= 0\"\n            )\n        if self.suffix_decoding_max_spec_factor < 0:\n            raise ValueError(\n                f\"suffix_decoding_max_spec_factor=\"\n                f\"{self.suffix_decoding_max_spec_factor} must be >= 0\"\n            )\n        if not 0 <= self.suffix_decoding_min_token_prob <= 1:\n            raise ValueError(\n                f\"suffix_decoding_min_token_prob=\"\n                f\"{self.suffix_decoding_min_token_prob} must be in [0, 1]\"\n            )\n\n    @staticmethod\n    def _maybe_override_draft_max_model_len(\n        speculative_max_model_len: int | None,\n        draft_max_model_len: int,","sourceCodeStart":1149,"sourceCodeEnd":1185,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/speculative.py#L1149-L1185","documentation":"Raised by _validate_suffix_decoding when suffix_decoding_max_cached_requests < 0. This parameter caps how many finished requests' suffixes are retained in the global cache; a negative cap is meaningless and rejected. Zero is legal and means the global cache is effectively disabled (only per-request suffixes are used).","triggerScenarios":"speculative_config={'method': 'suffix', 'suffix_decoding_max_cached_requests': -1}; config generation code computing the value from an expression that can go negative (e.g. cache_size - requests).","commonSituations":"Attempting to size the suffix cache from a formula that underflows on small deployments; sign errors when porting configs.","solutions":["Set suffix_decoding_max_cached_requests to >= 0 (use 0 to disable cross-request caching)","Remove the key to accept the default","Fix the generating expression so it clamps at 0"],"exampleFix":"# before\nspeculative_config={\"method\": \"suffix\", \"suffix_decoding_max_cached_requests\": -1}\n# after\nspeculative_config={\"method\": \"suffix\", \"suffix_decoding_max_cached_requests\": 0}","handlingStrategy":"validation","validationCode":"spec_cfg[\"suffix_decoding_max_cached_requests\"] = max(0, computed_cache_requests)","typeGuard":"def is_valid_cached_requests(n: int) -> bool:\n    return n >= 0","tryCatchPattern":null,"preventionTips":["Clamp computed cache sizes at 0 in config-generation code","Treat 0 as the explicit 'off' value rather than using negatives"],"tags":["speculative-decoding","suffix-decoding","cache","range-validation"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}