Lightning-AI/pytorch-lightning · error · MisconfigurationException

You requested gpu: {gpus} But your machine only has: {all_a

Error message

You requested gpu: {gpus}
 But your machine only has: {all_available_gpus}

What it means

Lightning validates the GPU indices you requested (e.g. devices=[3] or CUDA_VISIBLE_DEVICES-derived ids) against the GPUs actually visible to the process. If any requested id is not in the list returned by _get_all_available_gpus, it raises MisconfigurationException showing both lists. This usually means the ids don't exist in the current environment (wrong node, wrong CUDA_VISIBLE_DEVICES, or fewer GPUs than requested).

Source

Thrown at src/lightning/fabric/utilities/device_parser.py:136

    GPUs is not available.

    Args:
        gpus: List of ints corresponding to GPU indices

    Returns:
        Unmodified gpus variable

    Raises:
        MisconfigurationException:
            If machine has fewer available GPUs than requested.

    """
    if sum((include_cuda, include_mps)) == 0:
        raise ValueError("At least one gpu type should be specified!")
    all_available_gpus = _get_all_available_gpus(include_cuda=include_cuda, include_mps=include_mps)
    for gpu in gpus:
        if gpu not in all_available_gpus:
            raise MisconfigurationException(
                f"You requested gpu: {gpus}\n But your machine only has: {all_available_gpus}"
            )
    return gpus


def _normalize_parse_gpu_input_to_list(
    gpus: Union[int, list[int], tuple[int, ...]], include_cuda: bool, include_mps: bool
) -> Optional[list[int]]:
    assert gpus is not None
    if isinstance(gpus, (MutableSequence, tuple)):
        return list(gpus)

    # must be an int
    if not gpus:  # gpus==0
        return None
    if gpus == -1:
        return _get_all_available_gpus(include_cuda=include_cuda, include_mps=include_mps)

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Print torch.cuda.device_count() / check nvidia-smi and request valid indices (0..N-1)
2. Use devices='auto' or devices=N to let Lightning pick from available GPUs
3. Verify CUDA_VISIBLE_DEVICES on the target node and align requested ids with it

Example fix

# before
fabric = Fabric(accelerator="gpu", devices=[2, 3])  # node has 2 GPUs

# after
fabric = Fabric(accelerator="gpu", devices="auto")

import torch

def available_gpu_ids():
    if torch.cuda.is_available():
        return list(range(torch.cuda.device_count()))
    if getattr(torch.backends, "mps", None) and torch.backends.mps.is_available():
        return [0]
    return []

requested = [2, 3]
assert set(requested) <= set(available_gpu_ids()), f"requested {requested}, have {available_gpu_ids()}"

def is_valid_gpu_list(devices: object) -> bool:
    return (
        isinstance(devices, (list, tuple))
        and all(type(d) is int for d in devices)
        and len(devices) == len(set(devices))
    )

When it happens

Trigger: Passing devices=[1] or devices='1,' when only 1 GPU (index 0) is visible; setting devices=4 on a node with 2 GPUs; a CUDA_VISIBLE_DEVICES string like '2,3' on a machine with only 2 GPUs; requesting an MPS gpu id on a non-Apple machine.

Common situations: Multi-node jobs scheduled on nodes with heterogeneous GPU counts; Slurm/NGC containers restricting visible GPUs; hard-coded device indices moved between machines; typos in comma-separated device strings.

Related errors

• At least one gpu type should be specified!
• GPU parallel_devices set through {self._strategy_flag.__clas
• GPUs requested but none are available.
• Device ID's (GPU) must be unique.
• TensorRT only supports CUDA devices. The current device is {

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/330694801c524d66. Report an issue: GitHub.