docling-project/docling · error · RuntimeError

Nemotron OCR requires a CUDA accelerator. Set `pipeline_opti

Error message

Nemotron OCR requires a CUDA accelerator. Set `pipeline_options.accelerator_options.device` to CUDA or AUTO on a CUDA-enabled machine.

What it means

Nemotron OCR requires GPU acceleration; validate_runtime inspects accelerator_options.device and raises RuntimeError unless it resolves to CUDA (directly or via AUTO on a CUDA machine). CPU, MPS, or an explicit non-CUDA device are rejected at construction time because the Nemotron checkpoints only run on CUDA 13.x.

Source

Thrown at docling/models/stages/ocr/nemotron_ocr_model.py:155

                    'via `pip install "docling[feat-ocr-nemotron]"` on Linux x86_64 with '
                    "Python 3.12 and CUDA 13.x."
                ) from exc

            # Resolve the request language
            language = resolve_nemotronocr_language(options.lang)

            # Initialize the model
            model_dir = self._resolve_model_dir(language, artifacts_path=artifacts_path)

            self.reader = NemotronOCRV2(
                model_dir=None if model_dir is None else str(model_dir),
                lang=language,
            )

    @staticmethod
    def _fail_runtime(message: str) -> None:
        _log.error(message)
        raise RuntimeError(message)

    @classmethod
    def validate_runtime(cls, accelerator_options: AcceleratorOptions) -> None:
        if sys.platform != "linux":
            cls._fail_runtime("Nemotron OCR is only supported on Linux.")

        if platform.machine() != "x86_64":
            cls._fail_runtime("Nemotron OCR is only supported on x86_64 machines.")

        if sys.version_info[:2] != (3, 12):
            cls._fail_runtime("Nemotron OCR requires Python 3.12.")

        requested_device = decide_device(accelerator_options.device)
        if not requested_device.startswith("cuda"):
            cls._fail_runtime(
                "Nemotron OCR requires a CUDA accelerator. Set "
                "`pipeline_options.accelerator_options.device` to CUDA or AUTO on a "
                "CUDA-enabled machine."

View on GitHub (pinned to 61d76f1ff3)

Solutions

Set accelerator_options.device = AcceleratorDevice.CUDA (or AUTO on a CUDA machine) before building the pipeline.
Ensure an NVIDIA GPU with CUDA 13.x drivers is actually available (nvidia-smi).
If no NVIDIA GPU exists, use a CUDA-capable host/container or fall back to a CPU-capable OCR engine (Tesseract, RapidOCR onnxruntime).

Example fix

# before
from docling.datamodel.accelerator_options import AcceleratorDevice, AcceleratorOptions
pipeline_options.accelerator_options = AcceleratorOptions(device=AcceleratorDevice.CPU)
pipeline_options.ocr_options = NemotronOcrOptions()  # RuntimeError

# after
pipeline_options.accelerator_options = AcceleratorOptions(device=AcceleratorDevice.CUDA)
pipeline_options.ocr_options = NemotronOcrOptions()

Defensive patterns

Strategy: validation

Validate before calling

from docling.datamodel.accelerator_options import AcceleratorDevice, AcceleratorOptions

options = AcceleratorOptions(device=AcceleratorDevice.CUDA)  # required for Nemotron

Prevention

Set accelerator_options.device to CUDA (or AUTO on GPU hosts) whenever Nemotron OCR is enabled.
Verify nvidia-smi works in the target environment before deploying.
Centralize accelerator settings in config so CPU defaults cannot silently reach the Nemotron pipeline.

When it happens

Trigger: Setting pipeline_options.accelerator_options.device = AcceleratorDevice.CPU (the docling default on many installs) while enabling NemotronOcrOptions; or AUTO on a machine with no NVIDIA GPU so it resolves to CPU.

Common situations: Default CPU pipeline options on GPU-less machines; users who copied a CPU config for other docling stages; GPU present but device left at CPU in a saved config.

Related errors

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/6e57e6aca7fa95de. Report an issue: GitHub.