{"record":{"id":"fe8238b7603e18ef","repo":"invoke-ai/InvokeAI","slug":"configure-torch-cuda-allocator-must-be-called-be","errorCode":null,"errorMessage":"configure_torch_cuda_allocator() must be called before importing torch.","messagePattern":"configure_torch_cuda_allocator\\(\\) must be called before importing torch\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"invokeai/app/util/torch_cuda_allocator.py","lineNumber":13,"sourceCode":"import logging\nimport os\nimport sys\n\n\ndef configure_torch_cuda_allocator(pytorch_cuda_alloc_conf: str, logger: logging.Logger):\n    \"\"\"Configure the PyTorch CUDA memory allocator. See\n    https://pytorch.org/docs/stable/notes/cuda.html#optimizing-memory-usage-with-pytorch-cuda-alloc-conf for supported\n    configurations.\n    \"\"\"\n\n    if \"torch\" in sys.modules:\n        raise RuntimeError(\"configure_torch_cuda_allocator() must be called before importing torch.\")\n\n    # Log a warning if the PYTORCH_CUDA_ALLOC_CONF environment variable is already set.\n    prev_cuda_alloc_conf = os.environ.get(\"PYTORCH_CUDA_ALLOC_CONF\", None)\n    if prev_cuda_alloc_conf is not None:\n        if prev_cuda_alloc_conf == pytorch_cuda_alloc_conf:\n            logger.info(\n                f\"PYTORCH_CUDA_ALLOC_CONF is already set to '{pytorch_cuda_alloc_conf}'. Skipping configuration.\"\n            )\n            return\n        else:\n            logger.warning(\n                f\"Attempted to configure the PyTorch CUDA memory allocator with '{pytorch_cuda_alloc_conf}', but PYTORCH_CUDA_ALLOC_CONF is already set to \"\n                f\"'{prev_cuda_alloc_conf}'. Skipping configuration.\"\n            )\n            return\n\n    # Configure the PyTorch CUDA memory allocator.\n    # NOTE: It is important that this happens before torch is imported.","sourceCodeStart":1,"sourceCodeEnd":31,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/app/util/torch_cuda_allocator.py#L1-L31","documentation":"configure_torch_cuda_allocator() sets the PYTORCH_CUDA_ALLOC_CONF environment variable and then imports torch to verify the allocator backend. Because PyTorch reads PYTORCH_CUDA_ALLOC_CONF only at import time, the function refuses to run if torch is already in sys.modules. This guard exists to prevent silently ineffective allocator configuration.","triggerScenarios":"Calling configure_torch_cuda_allocator() after any module in the process has imported torch (directly or transitively, e.g. via diffusers, transformers, or another app module).","commonSituations":"Importing torch at module top-level in the entry script before calling the allocator config; calling run_app after other code triggered a torch import; test suites importing torch during collection before invoking the function.","solutions":["Move the configure_torch_cuda_allocator() call to the very top of the entry point, before any other imports that may pull in torch","Search the codebase for 'import torch' and defer it (move inside functions) until after configuration","Set the PYTORCH_CUDA_ALLOC_CONF environment variable yourself before launching the process so no runtime configuration is needed","In tests, ensure the function is called before torch is imported, or reload torch after setting the env var"],"exampleFix":"// before\nimport torch\nfrom invokeai.app.util.torch_cuda_allocator import configure_torch_cuda_allocator\nconfigure_torch_cuda_allocator(\"backend:cudaMallocAsync\")\nrun_app()\n\n// after\nfrom invokeai.app.util.torch_cuda_allocator import configure_torch_cuda_allocator\nconfigure_torch_cuda_allocator(\"backend:cudaMallocAsync\")\nimport torch\nrun_app()","handlingStrategy":"validation","validationCode":"import sys\nif \"torch\" in sys.modules:\n    raise RuntimeError(\"torch already imported; configure the CUDA allocator first\")\nconfigure_torch_cuda_allocator(conf)","typeGuard":"def torch_not_imported() -> bool:\n    return \"torch\" not in sys.modules","tryCatchPattern":"try:\n    configure_torch_cuda_allocator(conf)\nexcept RuntimeError as e:\n    if \"must be called before importing torch\" in str(e):\n        # defer torch import: restructure entry point or set env var and relaunch\n        ...","preventionTips":["Call configure_torch_cuda_allocator() as the first statement of the entry point","Never import torch at module top level in app code; import lazily inside functions","Grep for 'import torch' in startup imports when this error appears","Alternatively set PYTORCH_CUDA_ALLOC_CONF before process launch and skip runtime config"],"tags":["pytorch","cuda","import-order","configuration"],"backgroundTag":"torch-import-order","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}