docling-project/docling · error · RuntimeError

Model not loaded. Ensure EngineModelConfig was provided duri

Error message

Model not loaded. Ensure EngineModelConfig was provided during initialization.

What it means

TransformersVlmEngine.predict_batch() verifies that initialize() loaded a vlm_model and processor. Those are only set when a model_config with a repo_id was provided at construction; without one, initialize() completes but no weights are in memory and this RuntimeError is raised on the first non-empty batch.

Source

Thrown at docling/models/inference_engines/vlm/transformers_engine.py:319

        This method processes multiple images in a single forward pass,
        which is much more efficient than processing them sequentially.

        Args:
            input_batch: List of inputs to process

        Returns:
            List of outputs, one per input
        """
        if not self._initialized:
            self.initialize()

        if not input_batch:
            return []

        # Model should already be loaded via initialize()
        if self.vlm_model is None or self.processor is None:
            raise RuntimeError(
                "Model not loaded. Ensure EngineModelConfig was provided during initialization."
            )

        # Get prompt style from first input's extra config
        first_input = input_batch[0]
        prompt_style = first_input.extra_generation_config.get(
            "transformers_prompt_style",
            TransformersPromptStyle.CHAT,
        )

        # Prepare images using shared utility
        images = preprocess_image_batch([inp.image for inp in input_batch])

        # Prepare prompts
        prompts = []
        for input_data in input_batch:
            # Format prompt
            if prompt_style == TransformersPromptStyle.CHAT:

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Pass an EngineModelConfig with repo_id (or a VlmModelSpec through create_vlm_engine) so initialize() downloads and loads weights
2. Assert model_config and model_config.repo_id are set right after engine construction, before the first batch
3. Do not rely on a default: the Transformers engine has no bundled model

Example fix

# before
engine = TransformersVlmEngine(options=TransformersVlmEngineOptions())
outputs = engine.predict_batch(inputs)  # RuntimeError

# after
engine = TransformersVlmEngine(
    options=TransformersVlmEngineOptions(),
    model_config=EngineModelConfig(repo_id='ds4sd/SmolDocling-256M-preview'),
)
outputs = engine.predict_batch(inputs)

engine = TransformersVlmEngine(options=opts, model_config=model_config, artifacts_path=None, accelerator_options=acc)
assert model_config is not None and model_config.repo_id, 'Transformers engine requires EngineModelConfig.repo_id'
engine.initialize()
assert engine.vlm_model is not None and engine.processor is not None

try:
    outputs = engine.predict_batch(inputs)
except RuntimeError as e:
    if 'Model not loaded' in str(e):
        raise SystemExit('Attach an EngineModelConfig(repo_id=...) to the Transformers engine before inference') from e
    raise

When it happens

Trigger: Building TransformersVlmEngine without model_config (or with repo_id None) via create_vlm_engine(model_spec=None), then calling predict_batch with at least one VlmEngineInput.

Common situations: Options-only pipelines where the model spec was never wired in; assuming a default model is auto-selected; passing model_spec=None to try to 'configure later'.

Related errors

• Model not loaded. Ensure EngineModelConfig was provided duri
• Model not loaded. Ensure EngineModelConfig was provided duri
• Expected TransformersVlmEngineOptions, got {type(options)}
• Neither processor.batch_decode nor tokenizer.batch_decode is
• No default options configured for {format}

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/37749ab05b302fac. Report an issue: GitHub.