invoke-ai/InvokeAI · error · ValueError

generation_devices requested '{device_str}', but only {torch

Error message

generation_devices requested '{device_str}', but only {torch.cuda.device_count()} CUDA device(s) are available (valid indices 0-{torch.cuda.device_count() - 1}).

What it means

Same fail-fast validation as the no-CUDA case, but CUDA exists and the requested index is out of range: the explicitly configured cuda:N has N >= torch.cuda.device_count(). Raised with a message listing the valid index range.

Source

Thrown at invokeai/backend/util/devices.py:299

            else:
                device_strs = [str(device) for device in cls._auto_generation_devices()]
        elif not generation_devices:
            return []
        else:
            device_strs = list(generation_devices)

        devices: list[torch.device] = []
        seen: set[str] = set()
        for device_str in device_strs:
            device = cls.normalize(device_str)
            # Fail fast on a CUDA device that doesn't exist, rather than starting a worker pinned to
            # it that only errors cryptically at the first tensor allocation. ("auto" only generates
            # valid indices, so this just validates explicitly-configured devices.)
            if device.type == "cuda":
                if not torch.cuda.is_available():
                    raise ValueError(f"generation_devices requested '{device_str}', but no CUDA device is available.")
                if device.index is not None and device.index >= torch.cuda.device_count():
                    raise ValueError(
                        f"generation_devices requested '{device_str}', but only {torch.cuda.device_count()} "
                        f"CUDA device(s) are available (valid indices 0-{torch.cuda.device_count() - 1})."
                    )
            elif device.type == "xpu":
                if not _xpu_is_available():
                    raise ValueError(f"generation_devices requested '{device_str}', but no XPU device is available.")
                if device.index is not None and device.index >= torch.xpu.device_count():
                    raise ValueError(
                        f"generation_devices requested '{device_str}', but only {torch.xpu.device_count()} "
                        f"XPU device(s) are available (valid indices 0-{torch.xpu.device_count() - 1})."
                    )
            elif device.type == "mps" and not torch.backends.mps.is_available():
                raise ValueError(f"generation_devices requested '{device_str}', but MPS is not available.")
            if str(device) not in seen:
                seen.add(str(device))
                devices.append(device)
        return devices

View on GitHub (pinned to 0b6a024f2f)

Solutions

Set generation_devices to an in-range index (e.g. cuda:0) or 'auto'
Check torch.cuda.device_count() / nvidia-smi to see how many devices are visible
Fix CUDA_VISIBLE_DEVICES so the intended device index is exposed

Example fix

// before
generation_devices: ["cuda:1"]  # single-GPU host
// after
generation_devices: ["cuda:0"]

Defensive patterns

Strategy: validation

Validate before calling

import torch
for d in generation_devices:
    if d.startswith("cuda:"):
        idx = int(d.split(":")[1])
        if idx >= torch.cuda.device_count():
            raise SystemExit(f"{d} invalid: only {torch.cuda.device_count()} CUDA device(s)")

Type guard

def cuda_index_in_range(device_str: str) -> bool:
    import torch
    if not device_str.startswith("cuda:"):
        return True
    return int(device_str.split(":")[1]) < torch.cuda.device_count()

Try / catch

try:
    devices = DeviceService.get_generation_devices(cfg.generation_devices)
except ValueError as e:
    if "CUDA device(s) are available" in str(e):
        log.warning("Requested CUDA index out of range; using cuda:0/auto")
        devices = DeviceService.get_generation_devices(["auto"])
    else:
        raise

Prevention

Match device indices to the host's actual GPU count
Re-check config after hardware changes or GPU removal
Remember containers may expose a subset of GPUs via CUDA_VISIBLE_DEVICES
Prefer 'auto' in shared/portable configs

When it happens

Trigger: Configuring generation_devices 'cuda:1' (or higher) on a single-GPU machine, or after removing a GPU, since torch.cuda.device_count() is smaller than the requested index.

Common situations: Copying configs between machines with different GPU counts; GPU removed/failed at boot; container given only a subset of GPUs (CUDA_VISIBLE_DEVICES=0) while config still names cuda:1.

Related errors

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/b6bc5c2deab2c3ad. Report an issue: GitHub.