invoke-ai/InvokeAI · error · RuntimeError

configure_torch_cuda_allocator() must be called before impor

Error message

configure_torch_cuda_allocator() must be called before importing torch.

What it means

configure_torch_cuda_allocator() sets the PYTORCH_CUDA_ALLOC_CONF environment variable and then imports torch to verify the allocator backend. Because PyTorch reads PYTORCH_CUDA_ALLOC_CONF only at import time, the function refuses to run if torch is already in sys.modules. This guard exists to prevent silently ineffective allocator configuration.

Source

Thrown at invokeai/app/util/torch_cuda_allocator.py:13

import logging
import os
import sys


def configure_torch_cuda_allocator(pytorch_cuda_alloc_conf: str, logger: logging.Logger):
    """Configure the PyTorch CUDA memory allocator. See
    https://pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf for supported
    configurations.
    """

    if "torch" in sys.modules:
        raise RuntimeError("configure_torch_cuda_allocator() must be called before importing torch.")

    # Log a warning if the PYTORCH_CUDA_ALLOC_CONF environment variable is already set.
    prev_cuda_alloc_conf = os.environ.get("PYTORCH_CUDA_ALLOC_CONF", None)
    if prev_cuda_alloc_conf is not None:
        if prev_cuda_alloc_conf == pytorch_cuda_alloc_conf:
            logger.info(
                f"PYTORCH_CUDA_ALLOC_CONF is already set to '{pytorch_cuda_alloc_conf}'. Skipping configuration."
            )
            return
        else:
            logger.warning(
                f"Attempted to configure the PyTorch CUDA memory allocator with '{pytorch_cuda_alloc_conf}', but PYTORCH_CUDA_ALLOC_CONF is already set to "
                f"'{prev_cuda_alloc_conf}'. Skipping configuration."
            )
            return

    # Configure the PyTorch CUDA memory allocator.
    # NOTE: It is important that this happens before torch is imported.

View on GitHub (pinned to 0b6a024f2f)

Solutions

1. Move the configure_torch_cuda_allocator() call to the very top of the entry point, before any other imports that may pull in torch
2. Search the codebase for 'import torch' and defer it (move inside functions) until after configuration
3. Set the PYTORCH_CUDA_ALLOC_CONF environment variable yourself before launching the process so no runtime configuration is needed
4. In tests, ensure the function is called before torch is imported, or reload torch after setting the env var

Example fix

// before
import torch
from invokeai.app.util.torch_cuda_allocator import configure_torch_cuda_allocator
configure_torch_cuda_allocator("backend:cudaMallocAsync")
run_app()

// after
from invokeai.app.util.torch_cuda_allocator import configure_torch_cuda_allocator
configure_torch_cuda_allocator("backend:cudaMallocAsync")
import torch
run_app()

import sys
if "torch" in sys.modules:
    raise RuntimeError("torch already imported; configure the CUDA allocator first")
configure_torch_cuda_allocator(conf)

def torch_not_imported() -> bool:
    return "torch" not in sys.modules

try:
    configure_torch_cuda_allocator(conf)
except RuntimeError as e:
    if "must be called before importing torch" in str(e):
        # defer torch import: restructure entry point or set env var and relaunch
        ...

When it happens

Trigger: Calling configure_torch_cuda_allocator() after any module in the process has imported torch (directly or transitively, e.g. via diffusers, transformers, or another app module).

Common situations: Importing torch at module top-level in the entry script before calling the allocator config; calling run_app after other code triggered a torch import; test suites importing torch during collection before invoking the function.

Related errors

• Failed to configure the PyTorch CUDA memory allocator. Expec
• Attempted to configure the PyTorch CUDA memory allocator, bu
• Tokenizer returned unexpected types.
• LoRA '{lora.lora.key}' has conflicting weights on the transf
• Model '{main_config.name}' is not a Krea-2 main model. Selec

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/fe8238b7603e18ef. Report an issue: GitHub.