invoke-ai/InvokeAI · error · RuntimeError

Attempted to configure the PyTorch CUDA memory allocator, bu

Error message

Attempted to configure the PyTorch CUDA memory allocator, but no CUDA devices are available.

What it means

After setting PYTORCH_CUDA_ALLOC_CONF and importing torch, the function verifies CUDA is available before validating the allocator backend. If torch.cuda.is_available() is False there is no CUDA allocator to configure, so it raises. Configuring a GPU allocator on a CPU-only environment is treated as a hard failure.

Source

Thrown at invokeai/app/util/torch_cuda_allocator.py:38

                f"PYTORCH_CUDA_ALLOC_CONF is already set to '{pytorch_cuda_alloc_conf}'. Skipping configuration."
            )
            return
        else:
            logger.warning(
                f"Attempted to configure the PyTorch CUDA memory allocator with '{pytorch_cuda_alloc_conf}', but PYTORCH_CUDA_ALLOC_CONF is already set to "
                f"'{prev_cuda_alloc_conf}'. Skipping configuration."
            )
            return

    # Configure the PyTorch CUDA memory allocator.
    # NOTE: It is important that this happens before torch is imported.
    os.environ["PYTORCH_CUDA_ALLOC_CONF"] = pytorch_cuda_alloc_conf

    import torch

    # Relevant docs: https://pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf
    if not torch.cuda.is_available():
        raise RuntimeError(
            "Attempted to configure the PyTorch CUDA memory allocator, but no CUDA devices are available."
        )

    # Verify that the torch allocator was properly configured.
    allocator_backend = torch.cuda.get_allocator_backend()
    expected_backend = "cudaMallocAsync" if "cudaMallocAsync" in pytorch_cuda_alloc_conf else "native"
    if allocator_backend != expected_backend:
        raise RuntimeError(
            f"Failed to configure the PyTorch CUDA memory allocator. Expected backend: '{expected_backend}', but got "
            f"'{allocator_backend}'. Verify that 1) the pytorch_cuda_alloc_conf is set correctly, and 2) that torch is "
            "not imported before calling configure_torch_cuda_allocator()."
        )

    logger.info(f"PyTorch CUDA memory allocator: {torch.cuda.get_allocator_backend()}")

View on GitHub (pinned to 0b6a024f2f)

Solutions

1. Verify nvidia-smi shows the GPU and drivers are working
2. Reinstall torch with CUDA support (e.g. pip install torch --index-url https://download.pytorch.org/whl/cu121)
3. Check CUDA_VISIBLE_DEVICES is not empty or hiding devices
4. If running CPU-only intentionally, do not call configure_torch_cuda_allocator() or guard the call on torch.cuda.is_available()

Example fix

// before
configure_torch_cuda_allocator("backend:cudaMallocAsync")

// after
import torch
if torch.cuda.is_available():
    configure_torch_cuda_allocator("backend:cudaMallocAsync")

import torch
if not torch.cuda.is_available():
    raise RuntimeError("CUDA unavailable; skipping allocator configuration")
configure_torch_cuda_allocator(conf)

def cuda_ready() -> bool:
    import torch
    return torch.cuda.is_available() and torch.cuda.device_count() > 0

try:
    configure_torch_cuda_allocator(conf)
except RuntimeError as e:
    if "no CUDA devices are available" in str(e):
        logger.warning("No CUDA devices; running without allocator config")
    else:
        raise

When it happens

Trigger: Calling configure_torch_cuda_allocator() on a machine with no NVIDIA GPU, without CUDA-enabled drivers, with a CPU-only torch wheel, or with CUDA_VISIBLE_DEVICES set to hide all devices.

Common situations: Running a GPU-configured InvokeAI install on a CPU-only box or CI runner; installing torch from PyPI without the CUDA index URL; broken/mismatched NVIDIA driver; container without GPU passthrough.

Related errors

• configure_torch_cuda_allocator() must be called before impor
• Failed to configure the PyTorch CUDA memory allocator. Expec
• Tokenizer returned unexpected types.
• Unexpected cond image shape: {tuple(rgb_bchw_01.shape)} (exp
• Unexpected mask shape: {tuple(mask_b1hw_01.shape)} (expected

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/80a4167e69c36453. Report an issue: GitHub.