{"record":{"id":"3cb4c10e110e1cfb","repo":"vllm-project/vllm","slug":"arctic-inference-is-required-for-suffix-decoding","errorCode":null,"errorMessage":"Arctic Inference is required for suffix decoding. Install via `pip install arctic-inference==0.1.1`.","messagePattern":"Arctic Inference is required for suffix decoding\\. Install via `pip install arctic-inference==0\\.1\\.1`\\.","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"vllm/config/speculative.py","lineNumber":1148,"sourceCode":"                        self.draft_model_config.max_model_len,\n                        self.target_model_config.max_model_len,\n                    )\n                )\n\n                self.draft_parallel_config = (\n                    SpeculativeConfig.create_draft_parallel_config(\n                        self.target_parallel_config, self.draft_tensor_parallel_size\n                    )\n                )\n\n        if self.method != \"dspark\" and self.enable_adaptive_verification:\n            raise ValueError(\"Adaptive verification only supported with DSpark\")\n\n        return self\n\n    def _validate_suffix_decoding(self):\n        if not has_arctic_inference():\n            raise ImportError(\n                \"Arctic Inference is required for suffix decoding. \"\n                \"Install via `pip install arctic-inference==0.1.1`.\"\n            )\n        if self.num_speculative_tokens is None:\n            # Suffix decoding decides the actual number of speculative tokens\n            # dynamically and treats num_speculative_tokens as a maximum limit.\n            self.num_speculative_tokens = self.suffix_decoding_max_tree_depth\n            logger.warning(\n                \"Defaulted num_speculative_tokens to %s for suffix decoding.\",\n                self.num_speculative_tokens,\n            )\n        # Validate values\n        if self.suffix_decoding_max_tree_depth < 1:\n            raise ValueError(\n                f\"suffix_decoding_max_tree_depth=\"\n                f\"{self.suffix_decoding_max_tree_depth} must be >= 1\"\n            )\n        if self.suffix_decoding_max_cached_requests < 0:","sourceCodeStart":1130,"sourceCodeEnd":1166,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/config/speculative.py#L1130-L1166","documentation":"Raised as ImportError by _validate_suffix_decoding when method='suffix' but the optional arctic-inference package is not importable (has_arctic_inference() is False). Suffix decoding delegates its suffix-automaton drafting to Arctic Inference, so vLLM refuses to construct the config rather than crashing later in the worker.","triggerScenarios":"speculative_config={'method': 'suffix'} on an environment where 'import arctic_inference' fails — package not installed, wrong version, or a broken install (missing native components).","commonSituations":"Trying suffix decoding after seeing it in docs without installing the extra dependency; arctic-inference installed for a different Python/torch ABI; CI images that strip optional deps.","solutions":["pip install arctic-inference==0.1.1 (match the version named in the message) into the same venv/interpreter vLLM runs on","If the install is present but broken, reinstall and verify 'python -c \"import arctic_inference\"' succeeds in that environment","If you cannot install it, switch to a method with no external dependency such as 'ngram'"],"exampleFix":"# before\nspeculative_config={\"method\": \"suffix\"}  # ImportError: Arctic Inference required\n# after\npip install arctic-inference==0.1.1\nspeculative_config={\"method\": \"suffix\"}","handlingStrategy":"try-catch","validationCode":"def has_arctic_inference() -> bool:\n    try:\n        import arctic_inference  # noqa: F401\n        return True\n    except ImportError:\n        return False\n\nif spec_cfg.get(\"method\") == \"suffix\" and not has_arctic_inference():\n    raise RuntimeError(\"install arctic-inference==0.1.1 before enabling suffix decoding\")","typeGuard":null,"tryCatchPattern":"try:\n    llm = LLM(model=..., speculative_config={\"method\": \"suffix\"})\nexcept ImportError as e:\n    if \"arctic-inference\" in str(e):\n        subprocess.run([sys.executable, \"-m\", \"pip\", \"install\", \"arctic-inference==0.1.1\"], check=True)\n        llm = LLM(model=..., speculative_config={\"method\": \"suffix\"})  # retry once after install\n    else:\n        raise","preventionTips":["Declare arctic-inference==0.1.1 in the deployment's dependency lockfile when suffix decoding is part of the config","Probe optional dependencies at startup and fail with an actionable message before the engine builds"],"tags":["speculative-decoding","suffix-decoding","dependency","import-error"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}