docling-project/docling · error · AcceleratorDeviceNotAvailableError

CUDA is not available in the system. Please ensure PyTorch w

Error message

CUDA is not available in the system. Please ensure PyTorch with CUDA support is installed, or use --device auto/cpu.

What it means

AcceleratorDeviceNotAvailableError raised by decide_device() when the user explicitly requests a CUDA device but torch reports CUDA as unavailable (torch.backends.cuda.is_built() is false or torch.cuda.is_available() is false) — including the case where supported_devices removed CUDA. The machine/build cannot serve the request, so the error suggests installing a CUDA-enabled PyTorch or switching to auto/cpu.

Source

Thrown at docling/utils/accelerator_utils.py:85

            if len(parts) == 2 and parts[1].isdigit():
                # select cuda device's id
                cuda_index = int(parts[1])
                if cuda_index < torch.cuda.device_count():
                    device = f"cuda:{cuda_index}"
                else:
                    raise AcceleratorDeviceNotAvailableError(
                        f"CUDA device 'cuda:{cuda_index}' is not available. "
                        f"Available CUDA devices: 0-{torch.cuda.device_count() - 1}"
                    )
            elif len(parts) == 1:  # just "cuda"
                device = "cuda:0"
            else:
                raise AcceleratorDeviceNotAvailableError(
                    f"Invalid CUDA device format '{accelerator_device}'. "
                    f"Use 'cuda' or 'cuda:N' where N is a valid device index."
                )
        else:
            raise AcceleratorDeviceNotAvailableError(
                "CUDA is not available in the system. "
                "Please ensure PyTorch with CUDA support is installed, or use --device auto/cpu."
            )

    elif accelerator_device == AcceleratorDevice.MPS.value:
        if (
            supported_devices is not None
            and AcceleratorDevice.MPS not in supported_devices
        ):
            raise AcceleratorDeviceNotAvailableError(
                f"MPS is not supported by this model. Supported devices: {[d.value for d in supported_devices]}"
            )

        if has_mps:
            device = "mps"
        else:
            raise AcceleratorDeviceNotAvailableError(
                "MPS is not available in the system. "

View on GitHub (pinned to 61d76f1ff3)

Solutions

Verify the environment: python -c "import torch; print(torch.cuda.is_available(), torch.version.cuda)" and nvidia-smi
Install a CUDA-enabled PyTorch build matching your CUDA toolkit (e.g. the cu121 wheel index)
Or switch the config to accelerator_device='auto' or 'cpu' to run on CPU
In containers, ensure GPU passthrough (nvidia-container-toolkit / --gpus all) and that CUDA_VISIBLE_DEVICES is not empty

Example fix

# before
accelerator_options.accelerator_device = "cuda"  # CPU-only torch installed

# after
accelerator_options.accelerator_device = "auto"  # gracefully picks best available device

Defensive patterns

Strategy: validation

Validate before calling

import torch

cuda_ok = torch.backends.cuda.is_built() and torch.cuda.is_available()
if not cuda_ok:
    accelerator_options.accelerator_device = "cpu"  # or "auto"

Type guard

def cuda_available() -> bool:
    import torch
    return torch.backends.cuda.is_built() and torch.cuda.is_available()

Try / catch

from docling.exceptions import AcceleratorDeviceNotAvailableError

try:
    device = decide_device("cuda")
except AcceleratorDeviceNotAvailableError:
    device = decide_device("auto")  # CPU fallback

Prevention

Smoke-test torch.cuda.is_available() at startup when CUDA is requested
Install CUDA-enabled torch wheels matching your driver's CUDA version
In Docker, run with --gpus all / nvidia-container-toolkit
Check CUDA_VISIBLE_DEVICES is not empty or '-1' in the environment

When it happens

Trigger: Setting accelerator_device='cuda'/'cuda:N' while running a CPU-only PyTorch wheel (typical default pip install), on a machine without an NVIDIA GPU, with a missing NVIDIA driver, or with a CUDA version mismatch between torch and the driver.

Common situations: Installing docling via pip which pulls the CPU torch wheel, then enabling CUDA; CUDA driver too old for the installed torch CUDA runtime; GPU node drained/detached; running in a container without GPU passthrough (no nvidia-runtime); CUDA_VISIBLE_DEVICES set to empty string.

Related errors

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/7b83f702f97dc179. Report an issue: GitHub.