vllm-project/vllm · error · RuntimeError

xpumem allocator extension is not available

Error message

xpumem allocator extension is not available

What it means

Raised by get_pluggable_allocator in xpumem.py when the vllm_xpu_kernels.xpumem_allocator extension is unavailable (import failed at module load, leaving xpumem_available False). This extension supplies the my_malloc/my_free native functions the pluggable XPU allocator needs, so without it the allocator cannot be constructed.

Source

Thrown at vllm/device_allocator/xpumem.py:71

        # torch custom-op `int` arguments are signed int64.
        # data_ptr() may return a uint64 value above 2^63-1, so normalize it.
        return ptr if ptr < (1 << 63) else ptr - (1 << 64)

    torch.ops._C.xpu_memcpy_sync(
        _to_i64_ptr(dst_ptr),
        _to_i64_ptr(src_ptr),
        n_bytes,
        kind,
        device,
    )


def get_pluggable_allocator(
    python_malloc_fn: Callable[[HandleType], None],
    python_free_func: Callable[[int], HandleType],
) -> Any:
    if not xpumem_available or xpumem_allocator is None:
        raise RuntimeError("xpumem allocator extension is not available")

    xpumem_allocator.init_module(python_malloc_fn, python_free_func)
    mem_mod = _xpu_memory_module()
    alloc_cls = getattr(mem_mod, "XPUPluggableAllocator", None)
    if alloc_cls is None:
        raise RuntimeError("torch.xpu.memory.XPUPluggableAllocator is not available")

    lib_name = xpumem_allocator.__file__
    return alloc_cls(lib_name, "my_malloc", "my_free")


def create_and_allocate(allocation_handle: HandleType) -> None:
    if not xpumem_available or xpumem_allocator is None:
        raise RuntimeError("xpumem allocator extension is not available")
    xpumem_allocator.python_create_and_allocate(*allocation_handle)


def unmap_and_release(allocation_handle: HandleType) -> None:

View on GitHub (pinned to c794754062)

Solutions

1. Install/upgrade the vllm-xpu-kernels package matching your vLLM and torch-XPU versions.
2. Verify the import directly (python -c "from vllm_xpu_kernels import xpumem_allocator; print(xpumem_allocator.__file__)") and fix any underlying .so/loader errors it reports.
3. If the extension is genuinely unavailable in your build, avoid XPU sleep mode.

Example fix

# before: vllm_xpu_kernels missing -> RuntimeError
# after
pip install vllm-xpu-kernels>=<version required by your vllm>
python -c "from vllm_xpu_kernels import xpumem_allocator as a; print(a.__file__)"

try:
    from vllm_xpu_kernels import xpumem_allocator  # noqa: F401
except ImportError:
    raise SystemExit("vllm-xpu-kernels with xpumem_allocator is required for XPU sleep mode")

def xpumem_extension_available() -> bool:
    try:
        from vllm_xpu_kernels import xpumem_allocator  # noqa: F401
        return True
    except ImportError:
        return False

try:
    alloc = get_pluggable_allocator(malloc, free)
except RuntimeError as e:
    if "xpumem allocator extension" in str(e):
        install_vllm_xpu_kernels_and_restart()
    raise

When it happens

Trigger: Any call to get_pluggable_allocator (XPU sleep-mode memory pool setup) when vllm_xpu_kernels is not installed or does not expose xpumem_allocator.

Common situations: vLLM XPU deployment where the companion vllm-xpu-kernels wheel was not installed or is an old version without xpumem; a broken extension import (missing native .so) that was silently swallowed by the except ImportError at module top.

Related errors

• Sleep mode allocator is not available on platform {type(curr
• torch.xpu.memory is not available
• Sleep-mode backend '{name}' is already registered.
• Unsupported sleep-mode backend '{name}'. Registered backends
• Sleep-mode backend '{name}' is not supported on this platfor

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/c9a06ea0b4ead8cb. Report an issue: GitHub.