{"record":{"id":"12e6ea462b0337ec","repo":"unslothai/unsloth","slug":"requested-gpu-wanted-but-none-of-them-are-visibl","errorCode":null,"errorMessage":"Requested GPU {wanted} but none of them are visible to this process (visible: {visible}). Clear the GPU selection to use the default device.","messagePattern":"Requested GPU (.+?) but none of them are visible to this process \\(visible: (.+?)\\)\\. Clear the GPU selection to use the default device\\.","errorType":"exception","errorClass":"ValueError","httpStatus":400,"severity":"error","filePath":"studio/backend/core/inference/diffusion_device.py","lineNumber":196,"sourceCode":"    \"\"\"\n    wanted = sorted({int(gpu_id) for gpu_id in gpu_ids or ()})\n    if not wanted:\n        return None\n    try:\n        from utils.hardware.hardware import (\n            get_parent_visible_gpu_ids,\n            resolve_requested_gpu_ids,\n        )\n    except Exception as exc:  # noqa: BLE001 -- without the hardware layer the mask is unknowable\n        raise ValueError(f\"GPU selection is unavailable on this host: {exc}\") from exc\n    allowed = resolve_requested_gpu_ids(wanted)\n    visible = get_parent_visible_gpu_ids()\n    # Torch enumerates the parent-visible list in order, so its ordinal for a physical id is that\n    # id's position in the mask. Unmasked, the layer reports range(physical count) and this is\n    # the identity mapping.\n    ordinals = [visible.index(gpu_id) for gpu_id in allowed if gpu_id in visible]\n    if not ordinals:\n        raise ValueError(\n            f\"Requested GPU {wanted} but none of them are visible to this process \"\n            f\"(visible: {visible}). Clear the GPU selection to use the default device.\"\n        )\n    if len(ordinals) == 1:\n        return ordinals[0]\n    if not allow_ranking:\n        return None\n\n    def _free_vram(ordinal: int) -> int:\n        try:\n            import torch\n            return int(torch.cuda.mem_get_info(ordinal)[0])\n        except Exception:  # noqa: BLE001 -- an unreadable card sorts last rather than failing the load\n            return -1\n\n    return max(ordinals, key = lambda ordinal: (_free_vram(ordinal), -ordinal))\n\n","sourceCodeStart":178,"sourceCodeEnd":214,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/core/inference/diffusion_device.py#L178-L214","documentation":"The requested physical GPU ids, after filtering through resolve_requested_gpu_ids and the parent's CUDA_VISIBLE_DEVICES mask, have no intersection with the ids actually visible to this process. The load is refused with the visible list rather than quietly running on a device the user did not choose.","triggerScenarios":"Calling the GPU-selection resolve with gpu_ids like [0,1] while CUDA_VISIBLE_DEVICES=4,5 (or unset-but-requested cards beyond the mask); requesting a card the container/host mask excludes; nvidia-smi reports the card but the process mask does not include it.","commonSituations":"Service launched under a container orchestrator or systemd unit that sets CUDA_VISIBLE_DEVICES; user selects GPU 0 in the UI not realizing the process only sees physical 4,5; mask changed after the UI read the host inventory.","solutions":["Pick one of the ids listed in the error's '(visible: ...)' set","Widen or clear CUDA_VISIBLE_DEVICES for the backend process so the requested card is visible, then restart it","Clear the GPU selection in the request (no gpu_ids) to fall back to the default device"],"exampleFix":"# before\n# CUDA_VISIBLE_DEVICES=4,5, request:\nordinal = resolve_gpu_ordinal(gpu_ids=[0])  # ValueError\n\n# after\nordinal = resolve_gpu_ordinal(gpu_ids=[4])   # a visible physical id\n# or unset the mask / clear the selection:\nordinal = resolve_gpu_ordinal(gpu_ids=None)","handlingStrategy":"validation","validationCode":"from utils.hardware.hardware import get_parent_visible_gpu_ids\n\ndef gpus_requestable(gpu_ids) -> bool:\n    if not gpu_ids:\n        return True\n    visible = get_parent_visible_gpu_ids()\n    return bool(set(map(int, gpu_ids)) & set(visible))","typeGuard":null,"tryCatchPattern":"try:\n    ordinal = resolve_gpu_ordinal(gpu_ids)\nexcept ValueError as e:\n    # message includes the visible list; offer those or the default device\n    return bad_request(str(e))","preventionTips":["Read CUDA_VISIBLE_DEVICES before offering GPU choices in the UI; show only parent-visible ids","Re-read the mask at request time — orchestrators can change it between inventory and use","Remember ids are PHYSICAL ids; torch ordinals are positions in the visible list, never send ordinals"],"tags":["gpu","cuda-visible-devices","environment","device-selection"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}