{"record":{"id":"80a4167e69c36453","repo":"invoke-ai/InvokeAI","slug":"attempted-to-configure-the-pytorch-cuda-memory-all","errorCode":null,"errorMessage":"Attempted to configure the PyTorch CUDA memory allocator, but no CUDA devices are available.","messagePattern":"Attempted to configure the PyTorch CUDA memory allocator, but no CUDA devices are available\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"invokeai/app/util/torch_cuda_allocator.py","lineNumber":38,"sourceCode":"                f\"PYTORCH_CUDA_ALLOC_CONF is already set to '{pytorch_cuda_alloc_conf}'. Skipping configuration.\"\n            )\n            return\n        else:\n            logger.warning(\n                f\"Attempted to configure the PyTorch CUDA memory allocator with '{pytorch_cuda_alloc_conf}', but PYTORCH_CUDA_ALLOC_CONF is already set to \"\n                f\"'{prev_cuda_alloc_conf}'. Skipping configuration.\"\n            )\n            return\n\n    # Configure the PyTorch CUDA memory allocator.\n    # NOTE: It is important that this happens before torch is imported.\n    os.environ[\"PYTORCH_CUDA_ALLOC_CONF\"] = pytorch_cuda_alloc_conf\n\n    import torch\n\n    # Relevant docs: https://pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf\n    if not torch.cuda.is_available():\n        raise RuntimeError(\n            \"Attempted to configure the PyTorch CUDA memory allocator, but no CUDA devices are available.\"\n        )\n\n    # Verify that the torch allocator was properly configured.\n    allocator_backend = torch.cuda.get_allocator_backend()\n    expected_backend = \"cudaMallocAsync\" if \"cudaMallocAsync\" in pytorch_cuda_alloc_conf else \"native\"\n    if allocator_backend != expected_backend:\n        raise RuntimeError(\n            f\"Failed to configure the PyTorch CUDA memory allocator. Expected backend: '{expected_backend}', but got \"\n            f\"'{allocator_backend}'. Verify that 1) the pytorch_cuda_alloc_conf is set correctly, and 2) that torch is \"\n            \"not imported before calling configure_torch_cuda_allocator().\"\n        )\n\n    logger.info(f\"PyTorch CUDA memory allocator: {torch.cuda.get_allocator_backend()}\")\n","sourceCodeStart":20,"sourceCodeEnd":53,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/app/util/torch_cuda_allocator.py#L20-L53","documentation":"After setting PYTORCH_CUDA_ALLOC_CONF and importing torch, the function verifies CUDA is available before validating the allocator backend. If torch.cuda.is_available() is False there is no CUDA allocator to configure, so it raises. Configuring a GPU allocator on a CPU-only environment is treated as a hard failure.","triggerScenarios":"Calling configure_torch_cuda_allocator() on a machine with no NVIDIA GPU, without CUDA-enabled drivers, with a CPU-only torch wheel, or with CUDA_VISIBLE_DEVICES set to hide all devices.","commonSituations":"Running a GPU-configured InvokeAI install on a CPU-only box or CI runner; installing torch from PyPI without the CUDA index URL; broken/mismatched NVIDIA driver; container without GPU passthrough.","solutions":["Verify nvidia-smi shows the GPU and drivers are working","Reinstall torch with CUDA support (e.g. pip install torch --index-url https://download.pytorch.org/whl/cu121)","Check CUDA_VISIBLE_DEVICES is not empty or hiding devices","If running CPU-only intentionally, do not call configure_torch_cuda_allocator() or guard the call on torch.cuda.is_available()"],"exampleFix":"// before\nconfigure_torch_cuda_allocator(\"backend:cudaMallocAsync\")\n\n// after\nimport torch\nif torch.cuda.is_available():\n    configure_torch_cuda_allocator(\"backend:cudaMallocAsync\")","handlingStrategy":"validation","validationCode":"import torch\nif not torch.cuda.is_available():\n    raise RuntimeError(\"CUDA unavailable; skipping allocator configuration\")\nconfigure_torch_cuda_allocator(conf)","typeGuard":"def cuda_ready() -> bool:\n    import torch\n    return torch.cuda.is_available() and torch.cuda.device_count() > 0","tryCatchPattern":"try:\n    configure_torch_cuda_allocator(conf)\nexcept RuntimeError as e:\n    if \"no CUDA devices are available\" in str(e):\n        logger.warning(\"No CUDA devices; running without allocator config\")\n    else:\n        raise","preventionTips":["Check nvidia-smi works before deploying GPU-configured builds","Install the CUDA-enabled torch wheel (use the pytorch index URL)","Audit CUDA_VISIBLE_DEVICES in containers and CI","Skip allocator config on CPU-only installs"],"tags":["pytorch","cuda","gpu","environment"],"backgroundTag":"cuda-not-available","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}