docling-project/docling · error · RuntimeError

Engine not initialized. Call initialize() first.

Error message

Engine not initialized. Call initialize() first.

What it means

predict_batch ran before the transformers engine finished initialization. The guard requires _model, _processor, and _device to be set; if initialize() was skipped or raised (e.g. the model failed to load), these are None and inference is refused with a clear message instead of an AttributeError.

Source

Thrown at docling/models/inference_engines/image_classification/transformers_engine.py:166

            raise RuntimeError(f"Failed to load model from {model_folder}: {exc}")

        self._initialized = True
        _log.info(
            "Transformers image-classification engine ready (device=%s, dtype=%s)",
            self._device,
            self._model.dtype,  # type: ignore[union-attr]
        )

    def predict_batch(
        self, input_batch: List[ImageClassificationEngineInput]
    ) -> List[ImageClassificationEngineOutput]:
        """Run inference on a batch of inputs."""
        import torch

        if not input_batch:
            return []
        if self._model is None or self._processor is None or self._device is None:
            raise RuntimeError("Engine not initialized. Call initialize() first.")

        images = [item.image.convert("RGB") for item in input_batch]
        inputs = self._processor(images=images, return_tensors="pt").to(self._device)

        with torch.inference_mode():
            outputs = self._model(**inputs)  # type: ignore[operator]
            probs_batch = torch.softmax(outputs.logits, dim=-1)

        batch_outputs: List[ImageClassificationEngineOutput] = []
        for input_item, probs_vector in zip(input_batch, probs_batch):
            # Use topk for efficiency when top_k is specified
            if self.options.top_k is not None:
                k = min(self.options.top_k, len(probs_vector))
                scores, labels = torch.topk(probs_vector, k=k)
            else:
                scores, labels = torch.sort(probs_vector, descending=True)

            batch_outputs.append(

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Call engine.initialize() before any predict_batch and let failures surface (fix the underlying load error rather than continuing).
2. Ensure the engine is re-initialized after a failure if you implement retry logic.
3. Structure usage as construct -> initialize -> predict, with no exception swallowing in between.

Example fix

# before
engine = TransformersImageClassificationEngine(...)
engine.predict_batch(batch)  # _model is None

# after
engine = TransformersImageClassificationEngine(...)
engine.initialize()
engine.predict_batch(batch)

if any(getattr(engine, attr, None) is None for attr in ("_model", "_processor", "_device")):
    engine.initialize()

try:
    engine.predict_batch(batch)
except RuntimeError as e:
    if "not initialized" in str(e):
        engine.initialize()
        engine.predict_batch(batch)
    else:
        raise

When it happens

Trigger: Calling TransformersImageClassificationEngine.predict_batch() when any of _model/_processor/_device is None — initialize() not called, or its exception (see 'Failed to load model' error) was caught and ignored before predicting.

Common situations: Broad try/except around initialize() in orchestration code; using the engine after an OOM or load failure; frameworks that lazily construct engines but forget the init step.

Related errors

• Engine not initialized. Call initialize() first.
• Engine not initialized. Call initialize() first.
• Engine not initialized. Call initialize() first.
• KServe v2 client is not initialized.
• KServe v2 client is not initialized.

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/dc6095b44b21c0d5. Report an issue: GitHub.