vllm-project/vllm · error · RuntimeError

Sleep mode allocator is not available on platform {type(curr

Error message

Sleep mode allocator is not available on platform {type(current_platform).__name__} (device_type={current_platform.device_type}).

What it means

Raised by vllm.device_allocator.get_mem_allocator_instance when sleep mode (--sleep-mode) requests a memory allocator on a platform that is neither CUDA-like nor XPU. The CuMemAllocator and XpuMemAllocator are the only implementations, so any other platform (CPU, ROCm without cuda-alike semantics, etc.) fails at allocator creation.

Source

Thrown at vllm/device_allocator/__init__.py:46

    def sleep(self, offload_tags: tuple[str, ...] | str | None = None) -> None: ...

    def wake_up(self, tags: list[str] | None = None) -> None: ...

    def get_current_usage(self) -> int: ...


def get_mem_allocator_instance() -> MemAllocator:
    if current_platform.is_cuda_alike():
        from vllm.device_allocator.cumem import CuMemAllocator

        return CuMemAllocator.get_instance()

    if current_platform.is_xpu():
        from vllm.device_allocator.xpumem import XpuMemAllocator

        return XpuMemAllocator.get_instance()

    raise RuntimeError(
        "Sleep mode allocator is not available on platform "
        f"{type(current_platform).__name__} "
        f"(device_type={current_platform.device_type})."
    )

View on GitHub (pinned to c794754062)

Solutions

Disable sleep mode (remove --sleep-mode) on this platform.
If you expect CUDA or XPU, verify the correct platform plugin/GPU stack is installed (drivers, torch build) so current_platform resolves properly.
Check whether a platform-specific allocator exists in your vLLM version before attempting sleep mode there.

Example fix

# before
vllm serve model --sleep-mode ...  # on a CPU-only host
# after
vllm serve model  # sleep mode unsupported here; omit it

Defensive patterns

Strategy: type-guard

Validate before calling

from vllm.platforms import current_platform

if not (current_platform.is_cuda_alike() or current_platform.is_xpu()):
    if sleep_mode_enabled(args):
        raise SystemExit("sleep mode requires CUDA-like or XPU platform")

Type guard

def sleep_mode_supported() -> bool:
    from vllm.platforms import current_platform
    return current_platform.is_cuda_alike() or current_platform.is_xpu()

Try / catch

try:
    allocator = get_mem_allocator_instance()
except RuntimeError as e:
    if "Sleep mode allocator" in str(e):
        disable_sleep_mode_and_relaunch()
    raise

Prevention

Gate --sleep-mode flags on a platform capability check in deployment templates.
Run smoke tests of sleep mode on the target image before rolling it out.

When it happens

Trigger: Starting vLLM with sleep mode enabled (e.g. --sleep-mode flush_discard_kvs / level 1+) on a machine whose current_platform is not cuda-alike and not XPU.

Common situations: Testing sleep mode on a CPU-only dev box or an unsupported accelerator; running a container where the platform plugin resolved to something unexpected.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/267cced5dc7a4805. Report an issue: GitHub.