{"record":{"id":"9681c077ec2140f4","repo":"vllm-project/vllm","slug":"insufficient-space-in-shm-path-required-bytes","errorCode":null,"errorMessage":"Insufficient space in {shm_path}: {required_bytes / mib:.0f} MiB required, {free_bytes / mib:.0f} MiB free. Increase {shm_path} (e.g. --shm-size or --ipc=host).","messagePattern":"Insufficient space in (.+?): (.+?) MiB required, (.+?) MiB free\\. Increase (.+?) \\(e\\.g\\. --shm-size or --ipc=host\\)\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"vllm/distributed/device_communicators/shm_broadcast.py","lineNumber":243,"sourceCode":"\n\ndef check_shm_free_space(required_bytes: int, shm_path: str = SHM_PATH) -> None:\n    \"\"\"Raise if ``shm_path`` cannot fit a ``required_bytes`` shared segment.\n\n    Args:\n        required_bytes: Size of the shared-memory segment to be created.\n        shm_path: Mount point backing POSIX shared memory; skipped if absent.\n\n    Raises:\n        RuntimeError: If ``required_bytes`` exceeds the free space.\n    \"\"\"\n    if not os.path.isdir(shm_path):\n        return\n    free_bytes = shutil.disk_usage(shm_path).free\n    if required_bytes <= free_bytes:\n        return\n    mib = 1 << 20\n    raise RuntimeError(\n        f\"Insufficient space in {shm_path}: {required_bytes / mib:.0f} MiB \"\n        f\"required, {free_bytes / mib:.0f} MiB free. Increase {shm_path} \"\n        \"(e.g. --shm-size or --ipc=host).\"\n    )\n\n\nclass ShmRingBuffer:\n    def __init__(\n        self,\n        n_reader: int,\n        max_chunk_bytes: int,\n        max_chunks: int,\n        name: str | None = None,\n    ):\n        \"\"\"\n        A shared memory ring buffer implementation for broadcast communication.\n        Essentially, it is a queue where only one will `enqueue` and multiple\n        will `dequeue`. The max size of each item, together with the max number","sourceCodeStart":225,"sourceCodeEnd":261,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/device_communicators/shm_broadcast.py#L225-L261","documentation":"Before creating a POSIX shared-memory broadcast buffer, ShmBroadcast checks free space under the mount backing /dev/shm (skipped if the path doesn't exist). If the required segment size exceeds shutil.disk_usage(shm_path).free, it raises RuntimeError telling you exactly how many MiB are needed vs free — the classic Docker default 64MB /dev/shm problem for multi-worker vLLM.","triggerScenarios":"Creating a ShmBroadcast / MessageQueue with a large max_chunk_bytes * max_chunks payload inside a container whose /dev/shm is the Docker default 64 MiB; or on a host where tmpfs is nearly exhausted by other processes.","commonSituations":"docker run without --shm-size (default 64MB) launching multiple vLLM workers; --shm-size set but multiple processes share the same segment budget; k8s pods missing emptyDir medium:Memory with adequate sizeLimit; leftover segments from crashed runs filling /dev/shm.","solutions":["Restart the container with a larger shared-memory allowance: docker run --shm-size=2g (or --ipc=host)","In Kubernetes, mount an emptyDir with medium: Memory and a sizeLimit covering the required MiB","Check and clean stale segments: df -h /dev/shm and remove leaked /dev/shm/* vLLM files from crashed runs","Reduce the buffer requirement (smaller max_token_num / chunk config) if the budget cannot grow"],"exampleFix":"# before\ndocker run --gpus all -p 8000:8000 vllm/vllm-openai --tensor-parallel-size 4\n# RuntimeError: Insufficient space in /dev/shm: ... MiB required, 64 MiB free\n\n# after\ndocker run --gpus all --shm-size=8g -p 8000:8000 vllm/vllm-openai --tensor-parallel-size 4","handlingStrategy":"validation","validationCode":"import shutil\nfree = shutil.disk_usage('/dev/shm').free\nassert required_bytes <= free, (\n    f\"need {required_bytes >> 20} MiB in /dev/shm, only {free >> 20} MiB free; raise --shm-size\")","typeGuard":null,"tryCatchPattern":"try:\n    ShmBroadcast(...)\nexcept RuntimeError as e:\n    if \"Insufficient space\" in str(e):\n        raise SystemExit(\"restart container with --shm-size >= required MiB or --ipc=host\")","preventionTips":["Always launch containers with --shm-size or --ipc=host for multi-worker vLLM","In k8s use emptyDir medium:Memory with a sizeLimit","Clean leaked /dev/shm segments after crashes","Check df -h /dev/shm in startup health checks"],"tags":["shared-memory","docker","distributed","resource-limits"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}