docling-project/docling · error · AcceleratorDeviceNotAvailableError

CUDA device 'cuda:{cuda_index}' is not available. Available

Error message

CUDA device 'cuda:{cuda_index}' is not available. Available CUDA devices: 0-{torch.cuda.device_count() - 1}

What it means

AcceleratorDeviceNotAvailableError raised by decide_device() when the user requests 'cuda:N' with a numeric index N that is >= torch.cuda.device_count(). CUDA is present, but the requested GPU index does not exist on this machine (valid indices are 0..device_count-1).

Source

Thrown at docling/utils/accelerator_utils.py:73

    elif accelerator_device.startswith("cuda"):
        if (
            supported_devices is not None
            and AcceleratorDevice.CUDA not in supported_devices
        ):
            raise AcceleratorDeviceNotAvailableError(
                f"CUDA is not supported by this model. Supported devices: {[d.value for d in supported_devices]}"
            )

        if has_cuda:
            # if cuda device index specified extract device id
            parts = accelerator_device.split(":")
            if len(parts) == 2 and parts[1].isdigit():
                # select cuda device's id
                cuda_index = int(parts[1])
                if cuda_index < torch.cuda.device_count():
                    device = f"cuda:{cuda_index}"
                else:
                    raise AcceleratorDeviceNotAvailableError(
                        f"CUDA device 'cuda:{cuda_index}' is not available. "
                        f"Available CUDA devices: 0-{torch.cuda.device_count() - 1}"
                    )
            elif len(parts) == 1:  # just "cuda"
                device = "cuda:0"
            else:
                raise AcceleratorDeviceNotAvailableError(
                    f"Invalid CUDA device format '{accelerator_device}'. "
                    f"Use 'cuda' or 'cuda:N' where N is a valid device index."
                )
        else:
            raise AcceleratorDeviceNotAvailableError(
                "CUDA is not available in the system. "
                "Please ensure PyTorch with CUDA support is installed, or use --device auto/cpu."
            )

    elif accelerator_device == AcceleratorDevice.MPS.value:
        if (

View on GitHub (pinned to 61d76f1ff3)

Solutions

Use a valid index: check torch.cuda.device_count() and pick 0..N-1, or just 'cuda' (defaults to cuda:0)
Inspect CUDA_VISIBLE_DEVICES in the environment — remapping may have hidden GPUs
Verify with nvidia-smi how many GPUs the process can actually see
Prefer 'cuda' without an index for portable configurations

Example fix

# before
accelerator_options.accelerator_device = "cuda:1"  # single-GPU machine

# after
accelerator_options.accelerator_device = "cuda"  # resolves to cuda:0

Defensive patterns

Strategy: validation

Validate before calling

import torch

if accelerator_options.accelerator_device.startswith("cuda:"):
    idx = int(accelerator_options.accelerator_device.split(":")[1])
    if idx >= torch.cuda.device_count():
        accelerator_options.accelerator_device = "cuda"  # clamp to cuda:0

Type guard

def valid_cuda_index(device: str) -> bool:
    if not device.startswith("cuda:"):
        return True
    suffix = device.split(":", 1)[1]
    return suffix.isdigit() and int(suffix) < torch.cuda.device_count()

Try / catch

from docling.exceptions import AcceleratorDeviceNotAvailableError

try:
    device = decide_device("cuda:2")
except AcceleratorDeviceNotAvailableError:
    device = decide_device("cuda")  # fall back to default GPU

Prevention

Use bare 'cuda' unless you specifically need multi-GPU sharding
Check torch.cuda.device_count() (after CUDA_VISIBLE_DEVICES remapping) before pinning indices
Never hardcode GPU indices in shared configs — make them env-driven
Remember CUDA_VISIBLE_DEVICES remaps indices: host GPU 2 may be cuda:0 inside the container

When it happens

Trigger: Setting accelerator_device='cuda:1' (or higher) on a machine with a single GPU; hardcoding a device index from a different node (multi-GPU box -> single-GPU container); CUDA_VISIBLE_DEVICES restricting visibility so device_count is smaller than the physical count.

Common situations: Porting configs between machines with different GPU counts; Docker with CUDA_VISIBLE_DEVICES=0 making 'cuda:1' invalid; driver/runtime problems reducing visible devices.

Related errors

AI-assisted analysis of docling-project/docling@61d76f1ff3 (2026-08-14). Data as JSON: /api/errors/6cefb8cf8ce8f2ce. Report an issue: GitHub.