{"record":{"id":"527823b7e2a1681e","repo":"docling-project/docling","slug":"nemotron-ocr-requires-cuda-at-initialization-time","errorCode":null,"errorMessage":"Nemotron OCR requires CUDA at initialization time, but `torch.cuda.is_available()` is false.","messagePattern":"Nemotron OCR requires CUDA at initialization time, but `torch\\.cuda\\.is_available\\(\\)` is false\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"docling/models/stages/ocr/nemotron_ocr_model.py","lineNumber":155,"sourceCode":"                    'via `pip install \"docling[feat-ocr-nemotron]\"` on Linux x86_64 with '\n                    \"Python 3.12 and CUDA 13.x.\"\n                ) from exc\n\n            # Resolve the request language\n            language = resolve_nemotronocr_language(options.lang)\n\n            # Initialize the model\n            model_dir = self._resolve_model_dir(language, artifacts_path=artifacts_path)\n\n            self.reader = NemotronOCRV2(\n                model_dir=None if model_dir is None else str(model_dir),\n                lang=language,\n            )\n\n    @staticmethod\n    def _fail_runtime(message: str) -> None:\n        _log.error(message)\n        raise RuntimeError(message)\n\n    @classmethod\n    def validate_runtime(cls, accelerator_options: AcceleratorOptions) -> None:\n        if sys.platform != \"linux\":\n            cls._fail_runtime(\"Nemotron OCR is only supported on Linux.\")\n\n        if platform.machine() != \"x86_64\":\n            cls._fail_runtime(\"Nemotron OCR is only supported on x86_64 machines.\")\n\n        if sys.version_info[:2] != (3, 12):\n            cls._fail_runtime(\"Nemotron OCR requires Python 3.12.\")\n\n        requested_device = decide_device(accelerator_options.device)\n        if not requested_device.startswith(\"cuda\"):\n            cls._fail_runtime(\n                \"Nemotron OCR requires a CUDA accelerator. Set \"\n                \"`pipeline_options.accelerator_options.device` to CUDA or AUTO on a \"\n                \"CUDA-enabled machine.\"","sourceCodeStart":137,"sourceCodeEnd":173,"githubUrl":"https://github.com/docling-project/docling/blob/61d76f1ff3f8428065465889f7b4577da7df704c/docling/models/stages/ocr/nemotron_ocr_model.py#L137-L173","documentation":"Beyond the configured device, validate_runtime verifies a CUDA runtime is actually usable via torch.cuda.is_available(). If torch cannot initialize CUDA (no driver, mismatched torch build, GPU in exclusive use), a RuntimeError is raised at initialization time. This catches configs that claim CUDA while the process cannot get it.","triggerScenarios":"device=CUDA/AUTO on a machine without the NVIDIA driver; CPU-only PyTorch wheel installed; driver/toolkit version too old for the torch build; GPU busy in exclusive compute mode.","commonSituations":"Containers without GPU passthrough (missing --gpus all); pip torch built for CPU only; CUDA driver version older than torch requires; CI runners without GPUs.","solutions":["Install GPU-enabled PyTorch matching your CUDA 13.x driver, and verify `python -c \"import torch; print(torch.cuda.is_available())\"` returns True.","For containers, pass the GPU through (docker run --gpus all ...) and install nvidia-container-toolkit.","If the host truly has no usable CUDA, run on a CUDA machine or fall back to a CPU OCR engine instead of Nemotron."],"exampleFix":"# before: CPU-only torch on a GPU host\npip install torch  # defaults to CPU wheel -> cuda.is_available() False\n\n# after\npip install torch --index-url https://download.pytorch.org/whl/cu130\npython -c \"import torch; assert torch.cuda.is_available()\"","handlingStrategy":"validation","validationCode":"import torch\n\nif not torch.cuda.is_available():\n    raise SystemExit(\"CUDA unavailable: fix driver/torch before using Nemotron OCR\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Smoke-test torch.cuda.is_available() in CI and at deploy time, not just at import time.","Install GPU torch builds (cu13x index) matching the NVIDIA driver.","Expose GPUs to containers with --gpus all and nvidia-container-toolkit."],"tags":["nemotron","ocr","cuda","pytorch","gpu"],"backgroundTag":null,"analyzedSha":"61d76f1ff3f8428065465889f7b4577da7df704c","analyzedAt":"2026-08-14T23:53:18.727Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}