docling-project/docling · error · RuntimeError

Nemotron OCR requires CUDA 13.x, but the current PyTorch run

Error message

Nemotron OCR requires CUDA 13.x, but the current PyTorch runtime reports CUDA {cuda_version!r}.

What it means

The Nemotron checkpoints are built for CUDA 13.x; validate_runtime reads the PyTorch runtime's reported CUDA version (torch.version.cuda) and raises RuntimeError when it is not a 13.x string. This means your torch wheel was compiled against a different CUDA major version than Nemotron requires.

Source

Thrown at docling/models/stages/ocr/nemotron_ocr_model.py:155

                    'via `pip install "docling[feat-ocr-nemotron]"` on Linux x86_64 with '
                    "Python 3.12 and CUDA 13.x."
                ) from exc

            # Resolve the request language
            language = resolve_nemotronocr_language(options.lang)

            # Initialize the model
            model_dir = self._resolve_model_dir(language, artifacts_path=artifacts_path)

            self.reader = NemotronOCRV2(
                model_dir=None if model_dir is None else str(model_dir),
                lang=language,
            )

    @staticmethod
    def _fail_runtime(message: str) -> None:
        _log.error(message)
        raise RuntimeError(message)

    @classmethod
    def validate_runtime(cls, accelerator_options: AcceleratorOptions) -> None:
        if sys.platform != "linux":
            cls._fail_runtime("Nemotron OCR is only supported on Linux.")

        if platform.machine() != "x86_64":
            cls._fail_runtime("Nemotron OCR is only supported on x86_64 machines.")

        if sys.version_info[:2] != (3, 12):
            cls._fail_runtime("Nemotron OCR requires Python 3.12.")

        requested_device = decide_device(accelerator_options.device)
        if not requested_device.startswith("cuda"):
            cls._fail_runtime(
                "Nemotron OCR requires a CUDA accelerator. Set "
                "`pipeline_options.accelerator_options.device` to CUDA or AUTO on a "
                "CUDA-enabled machine."

View on GitHub (pinned to 61d76f1ff3)

Solutions

Reinstall PyTorch from the cu130 index: `pip install torch --index-url https://download.pytorch.org/whl/cu130` (or the current CUDA 13.x channel).
Verify `python -c "import torch; print(torch.version.cuda)"` reports a 13.x version and torch.cuda.is_available() is True.
Update the NVIDIA driver to one supporting CUDA 13.x if the runtime check fails after reinstalling torch.

Example fix

# before
pip install torch  # or cu121 wheel -> torch.version.cuda == '12.1' -> RuntimeError

# after
pip install torch --index-url https://download.pytorch.org/whl/cu130
python -c "import torch; print(torch.version.cuda)"  # expect 13.x

Defensive patterns

Strategy: validation

Validate before calling

import torch

cuda = torch.version.cuda or ""
assert cuda.startswith("13."), f"Need CUDA 13.x torch, got {cuda!r}"

Prevention

Install torch from the cu130 wheel index; assert torch.version.cuda startswith '13.' at startup.
Upgrade the NVIDIA driver to a CUDA 13.x-capable version alongside the torch reinstall.
Freeze the torch+CUDA combination in a lockfile to prevent accidental upgrades.

When it happens

Trigger: torch installed with cu118/cu121/cu126 wheels while enabling Nemotron OCR; the reported torch.version.cuda is e.g. '12.4' or None (CPU build), failing the 13.x check.

Common situations: Reusing an existing environment with older CUDA torch; installing torch via a default index that pins a CUDA 12 wheel; mixing driver 13.x with torch cu12x.

Related errors

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/996f0e64a1253823. Report an issue: GitHub.