invoke-ai/InvokeAI · error · ValueError

generation_devices requested '{device_str}', but no CUDA dev

Error message

generation_devices requested '{device_str}', but no CUDA device is available.

What it means

get_generation_devices normalizes each configured generation device and fails fast on explicitly requested CUDA devices when CUDA is entirely unavailable. This prevents workers from starting pinned to a device that would only fail at first tensor allocation.

Source

Thrown at invokeai/backend/util/devices.py:297

            if legacy_device != "auto":
                device_strs: list[str] = [legacy_device]
            else:
                device_strs = [str(device) for device in cls._auto_generation_devices()]
        elif not generation_devices:
            return []
        else:
            device_strs = list(generation_devices)

        devices: list[torch.device] = []
        seen: set[str] = set()
        for device_str in device_strs:
            device = cls.normalize(device_str)
            # Fail fast on a CUDA device that doesn't exist, rather than starting a worker pinned to
            # it that only errors cryptically at the first tensor allocation. ("auto" only generates
            # valid indices, so this just validates explicitly-configured devices.)
            if device.type == "cuda":
                if not torch.cuda.is_available():
                    raise ValueError(f"generation_devices requested '{device_str}', but no CUDA device is available.")
                if device.index is not None and device.index >= torch.cuda.device_count():
                    raise ValueError(
                        f"generation_devices requested '{device_str}', but only {torch.cuda.device_count()} "
                        f"CUDA device(s) are available (valid indices 0-{torch.cuda.device_count() - 1})."
                    )
            elif device.type == "xpu":
                if not _xpu_is_available():
                    raise ValueError(f"generation_devices requested '{device_str}', but no XPU device is available.")
                if device.index is not None and device.index >= torch.xpu.device_count():
                    raise ValueError(
                        f"generation_devices requested '{device_str}', but only {torch.xpu.device_count()} "
                        f"XPU device(s) are available (valid indices 0-{torch.xpu.device_count() - 1})."
                    )
            elif device.type == "mps" and not torch.backends.mps.is_available():
                raise ValueError(f"generation_devices requested '{device_str}', but MPS is not available.")
            if str(device) not in seen:
                seen.add(str(device))
                devices.append(device)

View on GitHub (pinned to 0b6a024f2f)

Solutions

Change generation_devices in the config to 'auto', 'cpu', or 'mps' as appropriate for the host
Install a CUDA-enabled torch build and the NVIDIA driver if a GPU is present
Check nvidia-smi and CUDA_VISIBLE_DEVICES; clear masking env vars
Update legacy pinned device settings after migrating hardware

Example fix

// before
generation_devices: ["cuda:0"]   # on CPU-only host
// after
generation_devices: ["auto"]

Defensive patterns

Strategy: validation

Validate before calling

import torch
if any(str(d).startswith("cuda") for d in generation_devices) and not torch.cuda.is_available():
    generation_devices = ["auto"]  # or ["cpu"]

Type guard

def is_cuda_config_valid(devices: list[str]) -> bool:
    import torch
    return all(not d.startswith("cuda") for d in devices) or torch.cuda.is_available()

Try / catch

try:
    devices = DeviceService.get_generation_devices(cfg.generation_devices)
except ValueError as e:
    if "no CUDA device is available" in str(e):
        log.warning("CUDA unavailable; falling back to CPU")
        devices = DeviceService.get_generation_devices(["auto"])
    else:
        raise

Prevention

Check torch.cuda.is_available() before pinning cuda devices in config
Avoid copying GPU configs across hosts; use 'auto' where possible
Verify drivers and CUDA-enabled torch builds (not CPU-only wheels)
Check CUDA_VISIBLE_DEVICES isn't masking GPUs

When it happens

Trigger: Configuring generation_devices to a cuda:* device string (via config or update_runtime_config) on a machine with no CUDA support — no NVIDIA driver, CPU-only torch build, or CUDA_VISIBLE_DEVICES masking all devices.

Common situations: Copying a GPU machine's config to a CPU-only host or CI runner; installing the CPU-only torch wheel; driver not loaded / nvidia modules missing; CUDA_VISIBLE_DEVICES="" set in the environment.

Related errors

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/d58a3c2e67618190. Report an issue: GitHub.