docling-project/docling · error · RuntimeError

Model not loaded. Ensure EngineModelConfig was provided duri

Error message

Model not loaded. Ensure EngineModelConfig was provided during initialization.

What it means

MlxVlmEngine.predict_batch() asserts that initialize() actually loaded a model, processor, and config. These are only populated when model_config with a repo_id was supplied at construction; an engine created without them has nothing to run inference with, so this RuntimeError fires.

Source

Thrown at docling/models/inference_engines/vlm/mlx_engine.py:171

        processing is done sequentially. This method is provided for API
        consistency but does not provide performance benefits over sequential
        processing.

        Args:
            input_batch: List of inputs to process

        Returns:
            List of outputs, one per input
        """
        if not self._initialized:
            self.initialize()

        if not input_batch:
            return []

        # Model should already be loaded via initialize()
        if self.vlm_model is None or self.processor is None or self.config is None:
            raise RuntimeError(
                "Model not loaded. Ensure EngineModelConfig was provided during initialization."
            )

        _log.debug(
            f"MLX runtime processing batch of {len(input_batch)} images sequentially "
            "(MLX does not support batched inference)"
        )

        outputs: List[VlmEngineOutput] = []

        # MLX models are not thread-safe - use global lock to serialize access
        with _MLX_GLOBAL_LOCK:
            _log.debug("MLX model: Acquired global lock for thread safety")

            for input_data in input_batch:
                # Preprocess image
                images = preprocess_image_batch([input_data.image])
                image = images[0]

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Provide an EngineModelConfig with a repo_id (or a VlmModelSpec through the factory) so initialize() downloads and loads the model
2. Verify model_config is not None and model_config.repo_id is set before running predictions
3. Check the engine's constructor arguments — MLX has no bundled default weights

Example fix

# before
engine = MlxVlmEngine(options=MlxVlmEngineOptions())  # no model_config
outputs = engine.predict_batch(inputs)  # RuntimeError

# after
engine = MlxVlmEngine(
    options=MlxVlmEngineOptions(),
    model_config=EngineModelConfig(repo_id='ds4sd/SmolDocling-256M-preview', revision='main'),
)
outputs = engine.predict_batch(inputs)

engine = MlxVlmEngine(options=opts, model_config=model_config, artifacts_path=None)
assert model_config is not None and model_config.repo_id, 'MLX engine requires EngineModelConfig.repo_id'
engine.initialize()  # force load; fail fast here, not mid-batch
assert engine.vlm_model is not None and engine.processor is not None and engine.config is not None

try:
    outputs = engine.predict_batch(inputs)
except RuntimeError as e:
    if 'Model not loaded' in str(e):
        raise SystemExit('Attach an EngineModelConfig(repo_id=...) to the MLX engine before inference') from e
    raise

When it happens

Trigger: Constructing MlxVlmEngine without model_config (or with model_config.repo_id None), then calling predict_batch on a non-empty batch — initialize() returns without loading and the vlm_model/processor/config check fails.

Common situations: Assuming the engine pulls a default model on its own; wiring an options-only pipeline where the model spec was never attached; passing model_spec=None through create_vlm_engine.

Related errors

• Model not loaded. Ensure EngineModelConfig was provided duri
• Model not loaded. Ensure EngineModelConfig was provided duri
• Expected MlxVlmEngineOptions, got {type(options)}
• MLX models do not support HuggingFace StoppingCriteria insta
• No default options configured for {format}

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/fa472ccc52f72072. Report an issue: GitHub.