vllm-project/vllm · error · RuntimeError

torch.xpu.memory.XPUPluggableAllocator is not available

Error message

torch.xpu.memory.XPUPluggableAllocator is not available

What it means

Raised by get_pluggable_allocator when torch.xpu.memory exists but lacks the XPUPluggableAllocator class. The xpumem extension is present and initialized, but the torch XPU build's memory module does not provide the pluggable-allocator API, so the allocator object cannot be created.

Source

Thrown at vllm/device_allocator/xpumem.py:77

        _to_i64_ptr(src_ptr),
        n_bytes,
        kind,
        device,
    )


def get_pluggable_allocator(
    python_malloc_fn: Callable[[HandleType], None],
    python_free_func: Callable[[int], HandleType],
) -> Any:
    if not xpumem_available or xpumem_allocator is None:
        raise RuntimeError("xpumem allocator extension is not available")

    xpumem_allocator.init_module(python_malloc_fn, python_free_func)
    mem_mod = _xpu_memory_module()
    alloc_cls = getattr(mem_mod, "XPUPluggableAllocator", None)
    if alloc_cls is None:
        raise RuntimeError("torch.xpu.memory.XPUPluggableAllocator is not available")

    lib_name = xpumem_allocator.__file__
    return alloc_cls(lib_name, "my_malloc", "my_free")


def create_and_allocate(allocation_handle: HandleType) -> None:
    if not xpumem_available or xpumem_allocator is None:
        raise RuntimeError("xpumem allocator extension is not available")
    xpumem_allocator.python_create_and_allocate(*allocation_handle)


def unmap_and_release(allocation_handle: HandleType) -> None:
    if not xpumem_available or xpumem_allocator is None:
        raise RuntimeError("xpumem allocator extension is not available")
    xpumem_allocator.python_unmap_and_release(*allocation_handle)


@contextmanager

View on GitHub (pinned to c794754062)

Solutions

Upgrade torch to the XPU version pinned by your vLLM release (see requirements/xpu.txt).
Downgrade/align if a nightly renamed the class — match the torch/vLLM pair that is known to work.
Confirm with python -c "import torch; print(hasattr(torch.xpu.memory,'XPUPluggableAllocator'))" before launching sleep mode on XPU.

Example fix

# before: torch.xpu.memory without XPUPluggableAllocator
# after: upgrade to the pinned XPU torch
pip install --index-url https://download.pytorch.org/whl/xpu torch==<pinned version>
python -c "import torch; print(hasattr(torch.xpu.memory,'XPUPluggableAllocator'))"  # True

Defensive patterns

Strategy: type-guard

Validate before calling

import torch

mem = getattr(torch.xpu, "memory", None)
if mem is None or not hasattr(mem, "XPUPluggableAllocator"):
    raise SystemExit("torch lacks torch.xpu.memory.XPUPluggableAllocator; upgrade torch-XPU")

Type guard

def pluggable_allocator_available() -> bool:
    import torch
    mem = getattr(torch.xpu, "memory", None)
    return mem is not None and hasattr(mem, "XPUPluggableAllocator")

Try / catch

try:
    alloc = get_pluggable_allocator(malloc, free)
except RuntimeError as e:
    if "XPUPluggableAllocator" in str(e):
        upgrade_torch_xpu_to_pinned_version()
    raise

Prevention

Check hasattr(torch.xpu.memory, 'XPUPluggableAllocator') in node provisioning scripts.
Keep torch, vllm, and vllm-xpu-kernels versions locked as a tested triple.

When it happens

Trigger: get_pluggable_allocator reaches getattr(mem_mod, "XPUPluggableAllocator", None) and gets None — typically a torch-XPU version that predates (or renamed) XPUPluggableAllocator.

Common situations: torch too old for the pluggable allocator API; a custom/older Intel torch build; API renamed in a newer torch while vLLM expects the older name.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/74e701fd13eb5d18. Report an issue: GitHub.