What causes "vllm is not installed. Please install it via `pip install vllm`." in docling-project/docling?

Install vllm in the same venv/interpreter Docling uses: pip install vllm (ensure CUDA setup matches)

docling-project/docling · error · ImportError

vllm is not installed. Please install it via `pip install vl

Error message

vllm is not installed. Please install it via `pip install vllm`.

What it means

VllmVlmModel imports vllm.LLM/SamplingParams at init; vllm is an optional, heavyweight dependency. On Python < 3.14 the ImportError is re-raised with the plain install instruction; on 3.14 a different message is used because no vllm wheel exists there yet.

Source

Thrown at docling/models/vlm_pipeline_models/vllm_model.py:114

        self.vlm_options: InlineVlmOptions = vlm_options

        self.llm = None
        self.sampling_params = None
        self.processor = None  # used for CHAT templating in formulate_prompt()
        self.device = "cpu"
        self.max_new_tokens = vlm_options.max_new_tokens
        self.temperature = vlm_options.temperature

        if not self.enabled:
            return

        from transformers import AutoProcessor

        try:
            from vllm import LLM, SamplingParams
        except ImportError:
            if sys.version_info < (3, 14):
                raise ImportError(
                    "vllm is not installed. Please install it via `pip install vllm`."
                )
            else:
                raise ImportError(
                    "vllm is not installed. It is not yet available on Python 3.14."
                )

        # Device selection
        self.device = decide_device(
            accelerator_options.device, supported_devices=vlm_options.supported_devices
        )
        _log.debug(f"Available device for VLM: {self.device}")

        # Resolve artifacts path / cache folder
        repo_cache_folder = vlm_options.repo_id.replace("/", "--")
        if artifacts_path is None:
            artifacts_path = self.download_models(
                self.vlm_options.repo_id, revision=self.vlm_options.revision

View on GitHub (pinned to 61d76f1ff3)

Solutions

Install vllm in the same venv/interpreter Docling uses: pip install vllm (ensure CUDA setup matches)
If installation is impossible (CPU box), select the Transformers or MLX engine instead
After installing, verify with python -c 'from vllm import LLM' to catch wheel/torch mismatches early

Example fix

# before
vlm_options.engine = "vllm"  # ImportError on python 3.12 without vllm
# after
# pip install vllm
vlm_options.engine = "vllm"

Defensive patterns

Strategy: validation

Validate before calling

import importlib.util

if vlm_options.engine == 'vllm' and importlib.util.find_spec('vllm') is None:
    raise ImportError('vLLM engine selected but vllm is not installed in this interpreter')

Try / catch

try:
    model = VllmVlmModel(...)
except ImportError as e:
    if 'pip install vllm' in str(e):
        vlm_options.engine = 'transformers'  # degrade to local engine
    else:
        raise

Prevention

Install vllm in the exact venv Docling runs in; verify with python -c 'from vllm import LLM'
Match vllm wheel version to your CUDA/torch stack
Probe for vllm at startup when the engine is configurable

When it happens

Trigger: Selecting the vLLM engine on a Python < 3.14 interpreter where vllm is not installed; also when vllm is installed in a different virtualenv than the one running Docling.

Common situations: Assuming vllm ships with docling; fresh environments without the vllm extra; vllm wheel uninstallable due to CUDA/torch mismatches so the import fails.

Related errors

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/423292575ec6d984. Report an issue: GitHub.