docling-project/docling · error · RuntimeError

Model not loaded. Ensure EngineModelConfig was provided duri

Error message

Model not loaded. Ensure EngineModelConfig was provided during initialization.

What it means

VllmVlmEngine.predict_batch() asserts that initialize() built its LLM instance, processor, and SamplingParams. These are populated only when a model_config with a repo_id was provided; without one the engine initializes device handling but never constructs an LLM, and this RuntimeError fires on the first batch.

Source

Thrown at docling/models/inference_engines/vlm/vllm_engine.py:274

        This method processes multiple images in a single batched vLLM call,
        which is much more efficient than processing them sequentially.

        Args:
            input_batch: List of inputs to process

        Returns:
            List of outputs, one per input
        """
        if not self._initialized:
            self.initialize()

        if not input_batch:
            return []

        # Model should already be loaded via initialize()
        if self.llm is None or self.processor is None or self.sampling_params is None:
            raise RuntimeError(
                "Model not loaded. Ensure EngineModelConfig was provided during initialization."
            )

        # Preprocess images
        images = preprocess_image_batch([inp.image for inp in input_batch])

        # Get prompt style from first input's extra config
        first_input = input_batch[0]
        prompt_style = first_input.extra_generation_config.get(
            "transformers_prompt_style",
            TransformersPromptStyle.CHAT,
        )

        # Format prompts
        prompts: List[str | None] = []
        for input_data in input_batch:
            formatted_prompt = format_prompt_for_vlm(
                prompt=input_data.prompt,

View on GitHub (pinned to 61d76f1ff3)

Solutions

Pass an EngineModelConfig with repo_id (or a VlmModelSpec through the factory) so initialize() builds the vLLM LLM
Check that model_config is not None and repo_id is set before the first predict_batch call
Note vLLM has no default model — an explicit repo is mandatory

Example fix

# before
engine = VllmVlmEngine(options=VllmVlmEngineOptions())
outputs = engine.predict_batch(inputs)  # RuntimeError

# after
engine = VllmVlmEngine(
    options=VllmVlmEngineOptions(),
    model_config=EngineModelConfig(repo_id='rednote-hilab/dots.mocr'),
)
outputs = engine.predict_batch(inputs)

Defensive patterns

Strategy: validation

Validate before calling

engine = VllmVlmEngine(options=opts, model_config=model_config, artifacts_path=None, accelerator_options=acc)
assert model_config is not None and model_config.repo_id, 'vLLM engine requires EngineModelConfig.repo_id'
engine.initialize()
assert engine.llm is not None and engine.processor is not None and engine.sampling_params is not None

Try / catch

try:
    outputs = engine.predict_batch(inputs)
except RuntimeError as e:
    if 'Model not loaded' in str(e):
        raise SystemExit('Attach an EngineModelConfig(repo_id=...) to the vLLM engine before inference') from e
    raise

Prevention

Always pass model_config with repo_id when building VllmVlmEngine — vLLM has no default model
Initialize eagerly at startup so missing-model errors surface before work is queued
Assert engine.llm is populated after initialize() in integration smoke tests

When it happens

Trigger: Creating VllmVlmEngine without model_config (or model_spec=None through create_vlm_engine), then calling predict_batch on a non-empty input list.

Common situations: Options-only setup where the model spec is never attached; assuming vLLM launches a default model from the local HuggingFace cache automatically.

Related errors

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/cbc35900fae36c5f. Report an issue: GitHub.