vllm-project/vllm · error · RuntimeError

Insufficient space in {shm_path}: {required_bytes / mib:.0f}

Error message

Insufficient space in {shm_path}: {required_bytes / mib:.0f} MiB required, {free_bytes / mib:.0f} MiB free. Increase {shm_path} (e.g. --shm-size or --ipc=host).

What it means

Before creating a POSIX shared-memory broadcast buffer, ShmBroadcast checks free space under the mount backing /dev/shm (skipped if the path doesn't exist). If the required segment size exceeds shutil.disk_usage(shm_path).free, it raises RuntimeError telling you exactly how many MiB are needed vs free — the classic Docker default 64MB /dev/shm problem for multi-worker vLLM.

Source

Thrown at vllm/distributed/device_communicators/shm_broadcast.py:243

def check_shm_free_space(required_bytes: int, shm_path: str = SHM_PATH) -> None:
    """Raise if ``shm_path`` cannot fit a ``required_bytes`` shared segment.

    Args:
        required_bytes: Size of the shared-memory segment to be created.
        shm_path: Mount point backing POSIX shared memory; skipped if absent.

    Raises:
        RuntimeError: If ``required_bytes`` exceeds the free space.
    """
    if not os.path.isdir(shm_path):
        return
    free_bytes = shutil.disk_usage(shm_path).free
    if required_bytes <= free_bytes:
        return
    mib = 1 << 20
    raise RuntimeError(
        f"Insufficient space in {shm_path}: {required_bytes / mib:.0f} MiB "
        f"required, {free_bytes / mib:.0f} MiB free. Increase {shm_path} "
        "(e.g. --shm-size or --ipc=host)."
    )


class ShmRingBuffer:
    def __init__(
        self,
        n_reader: int,
        max_chunk_bytes: int,
        max_chunks: int,
        name: str | None = None,
    ):
        """
        A shared memory ring buffer implementation for broadcast communication.
        Essentially, it is a queue where only one will `enqueue` and multiple
        will `dequeue`. The max size of each item, together with the max number

View on GitHub (pinned to c794754062)

Solutions

1. Restart the container with a larger shared-memory allowance: docker run --shm-size=2g (or --ipc=host)
2. In Kubernetes, mount an emptyDir with medium: Memory and a sizeLimit covering the required MiB
3. Check and clean stale segments: df -h /dev/shm and remove leaked /dev/shm/* vLLM files from crashed runs
4. Reduce the buffer requirement (smaller max_token_num / chunk config) if the budget cannot grow

Example fix

# before
docker run --gpus all -p 8000:8000 vllm/vllm-openai --tensor-parallel-size 4
# RuntimeError: Insufficient space in /dev/shm: ... MiB required, 64 MiB free

# after
docker run --gpus all --shm-size=8g -p 8000:8000 vllm/vllm-openai --tensor-parallel-size 4

import shutil
free = shutil.disk_usage('/dev/shm').free
assert required_bytes <= free, (
    f"need {required_bytes >> 20} MiB in /dev/shm, only {free >> 20} MiB free; raise --shm-size")

try:
    ShmBroadcast(...)
except RuntimeError as e:
    if "Insufficient space" in str(e):
        raise SystemExit("restart container with --shm-size >= required MiB or --ipc=host")

When it happens

Trigger: Creating a ShmBroadcast / MessageQueue with a large max_chunk_bytes * max_chunks payload inside a container whose /dev/shm is the Docker default 64 MiB; or on a host where tmpfs is nearly exhausted by other processes.

Common situations: docker run without --shm-size (default 64MB) launching multiple vLLM workers; --shm-size set but multiple processes share the same segment budget; k8s pods missing emptyDir medium:Memory with adequate sizeLimit; leftover segments from crashed runs filling /dev/shm.

Related errors

• Only readers can dequeue
• Not enough space in the data buffer, try calling free_buf()
• chat template looks like a file path but does not exist
• `{executor_backend}` does not support async scheduling yet.
• torch_shm is known to fail without VLLM_WORKER_MULTIPROC_MET

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/9681c077ec2140f4. Report an issue: GitHub.