vllm-project/vllm · error · RuntimeError

torch.xpu.memory MemPool APIs are not available (need MemPoo

Error message

torch.xpu.memory MemPool APIs are not available (need MemPool and use_mem_pool).

What it means

Raised by use_memory_pool_with_allocator in xpumem.py when torch.xpu.memory exists but is missing MemPool or use_mem_pool. The context manager swaps the XPU allocator via these pool APIs; a torch XPU build without them cannot host the pluggable allocator, so the guard fails before any allocation happens.

Source

Thrown at vllm/device_allocator/xpumem.py:102

    if not xpumem_available or xpumem_allocator is None:
        raise RuntimeError("xpumem allocator extension is not available")
    xpumem_allocator.python_create_and_allocate(*allocation_handle)


def unmap_and_release(allocation_handle: HandleType) -> None:
    if not xpumem_available or xpumem_allocator is None:
        raise RuntimeError("xpumem allocator extension is not available")
    xpumem_allocator.python_unmap_and_release(*allocation_handle)


@contextmanager
def use_memory_pool_with_allocator(
    python_malloc_fn: Callable[[HandleType], None],
    python_free_func: Callable[[int], HandleType],
) -> Iterator[tuple[Any, Any]]:
    mem_mod = _xpu_memory_module()
    if not _supports_xpu_mem_pool(mem_mod):
        raise RuntimeError(
            "torch.xpu.memory MemPool APIs are not available "
            "(need MemPool and use_mem_pool)."
        )
    new_alloc = get_pluggable_allocator(python_malloc_fn, python_free_func)
    mem_pool = mem_mod.MemPool(new_alloc._allocator)
    with mem_mod.use_mem_pool(mem_pool):
        yield mem_pool, new_alloc


class XpuMemAllocator:
    """A singleton pluggable allocator helper for XPU.

    Note:
    Sleep will offload selected payloads to CPU or discard and unmap XPU
    physical memory. Wake-up remaps physical memory back to the same
    reserved virtual address and restores payload.
    """

View on GitHub (pinned to c794754062)

Solutions

  1. Upgrade to the torch XPU version pinned by your vLLM release.
  2. Preflight the APIs: python -c "import torch; m=torch.xpu.memory; print(hasattr(m,'MemPool'), hasattr(m,'use_mem_pool'))" — both must be True.
  3. If upgrading is impossible, disable XPU sleep mode on this stack.

Example fix

# before: torch without MemPool APIs -> RuntimeError
# after
pip install --index-url https://download.pytorch.org/whl/xpu torch==<pinned>
python -c "import torch; m=torch.xpu.memory; print(hasattr(m,'MemPool') and hasattr(m,'use_mem_pool'))"  # True
Defensive patterns

Strategy: type-guard

Validate before calling

import torch

mem = getattr(torch.xpu, "memory", None)
ok = mem is not None and hasattr(mem, "MemPool") and hasattr(mem, "use_mem_pool")
if not ok:
    raise SystemExit("torch.xpu.memory MemPool/use_mem_pool missing; upgrade torch-XPU")

Type guard

def xpu_mempool_api_present() -> bool:
    import torch
    mem = getattr(torch.xpu, "memory", None)
    return mem is not None and hasattr(mem, "MemPool") and hasattr(mem, "use_mem_pool")

Try / catch

try:
    with use_memory_pool_with_allocator(malloc, free) as (pool, alloc):
        ...
except RuntimeError as e:
    if "MemPool APIs" in str(e):
        abort_with_torch_upgrade_guidance()
    raise

Prevention

When it happens

Trigger: Entering the XpuMemAllocator memory-pool context (XPU sleep mode allocation phase) with a torch whose xpu.memory module lacks MemPool/use_mem_pool — usually an older or trimmed torch-XPU build.

Common situations: Version skew between vLLM and Intel torch: vLLM expects the MemPool APIs added in a given torch release, the deployed image has an older one; a custom torch build compiled without those bindings.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/0f4da3f0150338c2. Report an issue: GitHub.