vllm-project/vllm · error · ImportError

Arctic Inference is required for suffix decoding. Install vi

Error message

Arctic Inference is required for suffix decoding. Install via `pip install arctic-inference==0.1.1`.

What it means

Raised as ImportError by _validate_suffix_decoding when method='suffix' but the optional arctic-inference package is not importable (has_arctic_inference() is False). Suffix decoding delegates its suffix-automaton drafting to Arctic Inference, so vLLM refuses to construct the config rather than crashing later in the worker.

Source

Thrown at vllm/config/speculative.py:1148

                        self.draft_model_config.max_model_len,
                        self.target_model_config.max_model_len,
                    )
                )

                self.draft_parallel_config = (
                    SpeculativeConfig.create_draft_parallel_config(
                        self.target_parallel_config, self.draft_tensor_parallel_size
                    )
                )

        if self.method != "dspark" and self.enable_adaptive_verification:
            raise ValueError("Adaptive verification only supported with DSpark")

        return self

    def _validate_suffix_decoding(self):
        if not has_arctic_inference():
            raise ImportError(
                "Arctic Inference is required for suffix decoding. "
                "Install via `pip install arctic-inference==0.1.1`."
            )
        if self.num_speculative_tokens is None:
            # Suffix decoding decides the actual number of speculative tokens
            # dynamically and treats num_speculative_tokens as a maximum limit.
            self.num_speculative_tokens = self.suffix_decoding_max_tree_depth
            logger.warning(
                "Defaulted num_speculative_tokens to %s for suffix decoding.",
                self.num_speculative_tokens,
            )
        # Validate values
        if self.suffix_decoding_max_tree_depth < 1:
            raise ValueError(
                f"suffix_decoding_max_tree_depth="
                f"{self.suffix_decoding_max_tree_depth} must be >= 1"
            )
        if self.suffix_decoding_max_cached_requests < 0:

View on GitHub (pinned to c794754062)

Solutions

pip install arctic-inference==0.1.1 (match the version named in the message) into the same venv/interpreter vLLM runs on
If the install is present but broken, reinstall and verify 'python -c "import arctic_inference"' succeeds in that environment
If you cannot install it, switch to a method with no external dependency such as 'ngram'

Example fix

# before
speculative_config={"method": "suffix"}  # ImportError: Arctic Inference required
# after
pip install arctic-inference==0.1.1
speculative_config={"method": "suffix"}

Defensive patterns

Strategy: try-catch

Validate before calling

def has_arctic_inference() -> bool:
    try:
        import arctic_inference  # noqa: F401
        return True
    except ImportError:
        return False

if spec_cfg.get("method") == "suffix" and not has_arctic_inference():
    raise RuntimeError("install arctic-inference==0.1.1 before enabling suffix decoding")

Try / catch

try:
    llm = LLM(model=..., speculative_config={"method": "suffix"})
except ImportError as e:
    if "arctic-inference" in str(e):
        subprocess.run([sys.executable, "-m", "pip", "install", "arctic-inference==0.1.1"], check=True)
        llm = LLM(model=..., speculative_config={"method": "suffix"})  # retry once after install
    else:
        raise

Prevention

Declare arctic-inference==0.1.1 in the deployment's dependency lockfile when suffix decoding is part of the config
Probe optional dependencies at startup and fail with an actionable message before the engine builds

When it happens

Trigger: speculative_config={'method': 'suffix'} on an environment where 'import arctic_inference' fails — package not installed, wrong version, or a broken install (missing native components).

Common situations: Trying suffix decoding after seeing it in docs without installing the extra dependency; arctic-inference installed for a different Python/torch ABI; CI images that strip optional deps.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/3cb4c10e110e1cfb. Report an issue: GitHub.