vllm-project/vllm · error · RuntimeError

torch.xpu.memory is not available

Error message

torch.xpu.memory is not available

What it means

Raised by _xpu_memory_module in vllm/device_allocator/xpumem.py when torch.xpu.memory is absent. The XPU sleep-mode allocator needs torch's XPU memory APIs (MemPool/use_mem_pool/XPUPluggableAllocator); if the attribute is missing, the installed torch either has no XPU support or predates these APIs.

Source

Thrown at vllm/device_allocator/xpumem.py:37

MEMCPY_DEVICE_TO_HOST = 1
MEMCPY_DEVICE_TO_DEVICE = 2

xpumem_available = False
xpumem_allocator: Any = None

try:
    from vllm_xpu_kernels import xpumem_allocator as _xpumem_allocator

    xpumem_allocator = _xpumem_allocator
    xpumem_available = True
except ImportError:
    xpumem_allocator = None


def _xpu_memory_module() -> Any:
    mem_mod = getattr(torch.xpu, "memory", None)
    if mem_mod is None:
        raise RuntimeError("torch.xpu.memory is not available")
    return mem_mod


def _supports_xpu_mem_pool(mem_mod: Any) -> bool:
    return hasattr(mem_mod, "MemPool") and hasattr(mem_mod, "use_mem_pool")


def _xpu_memcpy_sync(
    dst_ptr: int,
    src_ptr: int,
    n_bytes: int,
    kind: int,
    device: int,
) -> None:
    def _to_i64_ptr(ptr: int) -> int:
        # torch custom-op `int` arguments are signed int64.
        # data_ptr() may return a uint64 value above 2^63-1, so normalize it.
        return ptr if ptr < (1 << 63) else ptr - (1 << 64)

View on GitHub (pinned to c794754062)

Solutions

  1. Install an Intel XPU build of PyTorch that exposes torch.xpu.memory (check torch.xpu.is_available() and hasattr(torch.xpu, 'memory')).
  2. Verify the vLLM XPU requirements (requirements/xpu.txt or docs) and match the pinned torch version.
  3. Confirm the process actually sees the XPU devices (oneAPI level-zero/GPU drivers loaded).

Example fix

# before: stock torch, torch.xpu.memory is None
# after
pip install --index-url https://download.pytorch.org/whl/xpu torch  # XPU-enabled build
python -c "import torch; print(hasattr(torch.xpu, 'memory'))"  # True
Defensive patterns

Strategy: type-guard

Validate before calling

import torch

if torch.xpu.is_available() and getattr(torch.xpu, "memory", None) is None:
    raise SystemExit("torch build lacks torch.xpu.memory; install an XPU-enabled torch")

Type guard

def xpu_memory_api_present() -> bool:
    import torch
    return getattr(torch.xpu, "memory", None) is not None

Try / catch

try:
    allocator = XpuMemAllocator.get_instance()
except RuntimeError as e:
    if "torch.xpu.memory is not available" in str(e):
        report_torch_xpu_version_and_abort()
    raise

Prevention

When it happens

Trigger: XpuMemAllocator code paths (sleep mode on XPU) call _xpu_memory_module(); it fails when getattr(torch.xpu, "memory", None) is None — a torch build without XPU support, an old torch, or a non-XPU machine reaching this code.

Common situations: Using a stock PyPI torch (no XPU) with an XPU-enabled vLLM; torch version older than the one that introduced torch.xpu.memory.MemPool; misconfigured environment where XPUs exist but the wrong torch is active.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/ff62dc439e7bffc1. Report an issue: GitHub.