invoke-ai/InvokeAI · error · RuntimeError

Failed to configure the PyTorch CUDA memory allocator. Expec

Error message

Failed to configure the PyTorch CUDA memory allocator. Expected backend: '{expected_backend}', but got '{allocator_backend}'. Verify that 1) the pytorch_cuda_alloc_conf is set correctly, and 2) that torch is not imported before calling configure_torch_cuda_allocator().

What it means

After configuring, the function reads torch.cuda.get_allocator_backend() and compares it to the expected backend ('cudaMallocAsync' when that backend is requested, otherwise 'native'). A mismatch means the allocator was not applied as requested, so it raises with a diagnostic naming both backends.

Source

Thrown at invokeai/app/util/torch_cuda_allocator.py:46

            return

    # Configure the PyTorch CUDA memory allocator.
    # NOTE: It is important that this happens before torch is imported.
    os.environ["PYTORCH_CUDA_ALLOC_CONF"] = pytorch_cuda_alloc_conf

    import torch

    # Relevant docs: https://pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf
    if not torch.cuda.is_available():
        raise RuntimeError(
            "Attempted to configure the PyTorch CUDA memory allocator, but no CUDA devices are available."
        )

    # Verify that the torch allocator was properly configured.
    allocator_backend = torch.cuda.get_allocator_backend()
    expected_backend = "cudaMallocAsync" if "cudaMallocAsync" in pytorch_cuda_alloc_conf else "native"
    if allocator_backend != expected_backend:
        raise RuntimeError(
            f"Failed to configure the PyTorch CUDA memory allocator. Expected backend: '{expected_backend}', but got "
            f"'{allocator_backend}'. Verify that 1) the pytorch_cuda_alloc_conf is set correctly, and 2) that torch is "
            "not imported before calling configure_torch_cuda_allocator()."
        )

    logger.info(f"PyTorch CUDA memory allocator: {torch.cuda.get_allocator_backend()}")

View on GitHub (pinned to 0b6a024f2f)

Solutions

1. Confirm torch is not imported before configure_torch_cuda_allocator() (fix import order)
2. Check the conf string for typos and that the backend is supported by the installed torch version (print torch.__version__)
3. Simplify the conf to just the backend directive first, then add options back one at a time
4. Align the configured backend with what the torch build supports, or upgrade torch

Example fix

// before
configure_torch_cuda_allocator("backend:cudaMallocAsync,max_split_size_mb:512")  # torch 1.12 lacks cudaMallocAsync

// after
configure_torch_cuda_allocator("backend:native")  # or upgrade torch to >=2.0 for cudaMallocAsync

import torch
backend = torch.cuda.get_allocator_backend()
wanted = "cudaMallocAsync" if "cudaMallocAsync" in conf else "native"
assert backend == wanted, f"backend {backend} != {wanted}"

def allocator_matches(expected: str) -> bool:
    import torch
    return torch.cuda.get_allocator_backend() == expected

try:
    configure_torch_cuda_allocator(conf)
except RuntimeError as e:
    if "Failed to configure the PyTorch CUDA memory allocator" in str(e):
        logger.warning("allocator backend mismatch: %s", e)
    else:
        raise

When it happens

Trigger: Setting a PYTORCH_CUDA_ALLOC_CONF whose backend the installed torch version doesn't support; torch imported earlier so the env var was ignored while cudaMallocAsync was still the effective backend (or vice versa); typos in the conf string like 'backend:cudaMallocAsync' misspellings; older torch versions lacking cudaMallocAsync.

Common situations: Upgrading/downgrading PyTorch and the configured backend no longer exists; a stale cached import meant config never took effect; copying an alloc-conf snippet from docs incompatible with the installed torch.

Related errors

• configure_torch_cuda_allocator() must be called before impor
• Attempted to configure the PyTorch CUDA memory allocator, bu
• Tokenizer returned unexpected types.
• LoRA '{lora.lora.key}' has conflicting weights on the transf
• Model '{main_config.name}' is not a Krea-2 main model. Selec

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/d4f76a73e9372482. Report an issue: GitHub.