docling-project/docling · error · RuntimeError

Failed to load model from {model_folder}: {e}

Error message

Failed to load model from {model_folder}: {e}

What it means

The Transformers engine wraps every exception thrown while loading the model/processor from the model folder into RuntimeError('Failed to load model from {model_folder}: {e}'). It is a wrapper: the root cause (corrupt weights, unsupported torch/transformers version, OOM, compile failure) is in the chained exception message.

Source

Thrown at docling/models/inference_engines/object_detection/transformers_engine.py:172

            self._model.eval()  # type: ignore[union-attr]

            # Optionally compile model for better performance (model must be in eval mode first)
            # Works for Python < 3.14 with any torch 2.x
            # Works for Python >= 3.14 with torch >= 2.10
            if self.options.compile_model:
                if sys.version_info < (3, 14):
                    self._model = torch.compile(self._model)  # type: ignore[arg-type,assignment]
                    _log.debug("Model compiled with torch.compile()")
                elif version.parse(torch.__version__) >= version.parse("2.10"):
                    self._model = torch.compile(self._model)  # type: ignore[arg-type,assignment]
                    _log.debug("Model compiled with torch.compile()")
                else:
                    _log.warning(
                        "Model compilation requested but not available "
                        "(requires Python < 3.14 or torch >= 2.10 for Python 3.14+)"
                    )
        except Exception as e:
            raise RuntimeError(f"Failed to load model from {model_folder}: {e}")

        self._initialized = True
        _log.info(
            f"Transformers engine ready (device={self._device}, dtype={self._model.dtype})"  # type: ignore[union-attr]
        )

    def predict_batch(
        self, input_batch: List[ObjectDetectionEngineInput]
    ) -> List[ObjectDetectionEngineOutput]:
        """Run inference on a batch of inputs.

        Args:
            input_batch: List of input images with metadata

        Returns:
            List of detection outputs
        """
        import torch

View on GitHub (pinned to 61d76f1ff3)

Solutions

  1. Read the full chained traceback — the '{e}' suffix names the underlying loader error; fix that first.
  2. If weights are corrupt/partial, delete the model folder in the cache/artifacts_path and re-download.
  3. Pin compatible torch/transformers versions for the model (check the model card); for compile issues, disable the compile option in the engine options.
  4. For OOM, choose a smaller device/dtype setting in accelerator options.

Example fix

# before
engine.initialize()  # raises generic 'Failed to load model from ...'

# after
# run with full traceback to see the cause:
#   python -X dev app.py   (or inspect __cause__ in except)
# then e.g. remove corrupt cache and retry:
#   rm -rf ~/.cache/huggingface/hub/models--BioMedClIP--... && rerun
Defensive patterns

Strategy: try-catch

Validate before calling

# fail fast on obviously broken artifacts before init
weights = list(model_folder.glob('*.safetensors')) + list(model_folder.glob('*.bin'))
assert weights, f"no weight files found in {model_folder}"

Try / catch

try:
    engine.initialize()
except RuntimeError as e:
    cause = e.__cause__ or e.__context__
    log.error("Model load failed: %s | root cause: %s", e, cause)
    raise

Prevention

When it happens

Trigger: Any exception inside the big load block: from_pretrained on a corrupt/incomplete download, torch.compile being attempted on unsupported Python/torch combos (guarded above but other compile errors possible), safetensors/pickle load errors, CUDA init failures, out-of-memory.

Common situations: Interrupted HF downloads leaving partial weight shards; transformers/torch version incompatibility with the model architecture; GPU OOM at load time; artifacts_path folder truncated during copy to a server.

Related errors

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/c02ff027fddaa602. Report an issue: GitHub.