docling-project/docling · error · RuntimeError

Failed to load model from {model_folder}: {exc}

Error message

Failed to load model from {model_folder}: {exc}

What it means

A catch-all around the transformers model-loading block: from_preprocessor_config/from_pretrained loading, device placement, dtype casting, or optional torch.compile raised. The original exception is chained, so the message includes the underlying cause. It almost always reflects an environment or artifact problem (missing files, bad revision, unsupported dtype/device), not a docling logic bug.

Source

Thrown at docling/models/inference_engines/image_classification/transformers_engine.py:148

            self._model.eval()  # type: ignore[union-attr]

            # Optionally compile model for better performance (model must be in eval mode first)
            # Works for Python < 3.14 with any torch 2.x
            # Works for Python >= 3.14 with torch >= 2.10
            if self.options.compile_model:
                if sys.version_info < (3, 14):
                    self._model = torch.compile(self._model)  # type: ignore[arg-type,assignment]
                    _log.debug("Model compiled with torch.compile()")
                elif version.parse(torch.__version__) >= version.parse("2.10"):
                    self._model = torch.compile(self._model)  # type: ignore[arg-type,assignment]
                    _log.debug("Model compiled with torch.compile()")
                else:
                    _log.warning(
                        "Model compilation requested but not available "
                        "(requires Python < 3.14 or torch >= 2.10 for Python 3.14+)"
                    )
        except Exception as exc:
            raise RuntimeError(f"Failed to load model from {model_folder}: {exc}")

        self._initialized = True
        _log.info(
            "Transformers image-classification engine ready (device=%s, dtype=%s)",
            self._device,
            self._model.dtype,  # type: ignore[union-attr]
        )

    def predict_batch(
        self, input_batch: List[ImageClassificationEngineInput]
    ) -> List[ImageClassificationEngineOutput]:
        """Run inference on a batch of inputs."""
        import torch

        if not input_batch:
            return []
        if self._model is None or self._processor is None or self._device is None:
            raise RuntimeError("Engine not initialized. Call initialize() first.")

View on GitHub (pinned to 61d76f1ff3)

Solutions

1. Read the chained cause (raise ... from exc) — the '{exc}' part names the real failure; fix that first.
2. Pre-download the model (huggingface-cli download <repo_id>) or point to a local folder to rule out network/cache issues.
3. Verify accelerator/device settings match the environment (CPU-only host must not request CUDA) and the dtype is supported by the installed torch.
4. Disable torch.compile in options if the failure mentions compile/dynamo, or upgrade torch.

Example fix

# before: compile requested on unsupported combo
options.compilation = ...  # triggers torch.compile path

# after
# keep options defaults (no forced compile); ensure model folder/revision is valid
engine.initialize()  # if it fails, inspect __cause__ for the root error

from huggingface_hub import snapshot_download

path = snapshot_download(repo_id=repo_id, revision=revision)  # fails early with a clear HF error if unreachable
assert (Path(path) / "config.json").exists(), "model folder incomplete"

try:
    engine.initialize()
except RuntimeError as e:
    cause = e.__cause__
    if cause and ("ConnectionError" in type(cause).__name__ or "OfflineModeIsEnabled" in type(cause).__name__):
        # transient/network: pre-download then retry once
        snapshot_download(repo_id=repo_id, revision=revision)
        engine.initialize()
    else:
        raise

When it happens

Trigger: TransformersImageClassificationEngine.initialize() when any step inside the try block raises: downloading/loading the HF model folder, moving to device, casting dtype, or torch.compile — wrapped as RuntimeError(f"Failed to load model from {model_folder}: {exc}").

Common situations: No network / blocked HuggingFace access while fetching the model; wrong or unavailable revision; corrupted cache; CUDA requested but unavailable; unsupported torch dtype for the model; torch.compile incompatibility with the installed torch/Python version.

Related errors

• Failed to load image processor from {model_folder}: {exc}
• {type(self).__name__} requires model_config with repo_id
• Image processor config not found: {preprocessor_config}
• Failed to load label mapping from model config at {model_fol
• Failed to load model from {model_folder}: {e}

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/f003465997d8cbf6. Report an issue: GitHub.