{"record":{"id":"cdd27e7dda0bc139","repo":"sgl-project/sglang","slug":"failed-to-create-shm-file-e","errorCode":null,"errorMessage":"Failed to create shm file: {e}","messagePattern":"Failed to create shm file: (.+?)","errorType":"exception","errorClass":"OSError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/mem_cache/storage/mmap/mmap_allocator.py","lineNumber":178,"sourceCode":"    # Create an anonymous shared memory file descriptor via memfd_create\n    fd = None\n    try:\n        # MFD_CLOEXEC is standard on Linux 3.17+\n        fd = os.memfd_create(\n            f\"sglang_host_pool_{uuid.uuid4().hex}\",\n            flags=getattr(os, \"MFD_CLOEXEC\", 1),\n        )\n    except (AttributeError, OSError):\n        # Fallback to creating a file in /dev/shm if memfd_create is not supported\n        shm_path = f\"/dev/shm/sglang_host_pool_{uuid.uuid4().hex}.mmap\"\n        try:\n            fd = os.open(shm_path, os.O_CREAT | os.O_RDWR | os.O_TRUNC, 0o600)\n            try:\n                os.unlink(shm_path)\n            except OSError:\n                pass\n        except Exception as e:\n            raise OSError(f\"Failed to create shm file: {e}\")\n\n    try:\n        os.ftruncate(fd, alloc_bytes)\n        mm = mmap.mmap(\n            fd,\n            alloc_bytes,\n            flags=mmap.MAP_SHARED | _MAP_POPULATE,\n            prot=mmap.PROT_READ | mmap.PROT_WRITE,\n        )\n        try:\n            # MADV_POPULATE_WRITE guarantees pages are populated and writable,\n            # throwing an error on failure (e.g. out of memory).\n            mm.madvise(_MADV_POPULATE_WRITE)\n        except OSError:\n            # Fall back to MAP_POPULATE if MADV_POPULATE_WRITE is not supported (<5.14 kernel).\n            pass\n    except Exception as e:\n        if fd is not None:","sourceCodeStart":160,"sourceCodeEnd":196,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/storage/mmap/mmap_allocator.py#L160-L196","documentation":"os.open('/dev/shm/...') failed while creating the shared-memory backing file for the host KV cache pool. The original exception is wrapped into OSError with 'Failed to create shm file'. Typical underlying causes are ENOSPC (tmpfs full) or EACCES/ENOENT on /dev/shm.","triggerScenarios":"alloc_shm() opening /dev/shm/<name> with O_CREAT|O_RDWR|O_TRUNC when /dev/shm has no free space, is not mounted, or the process lacks write permission; too many stale files with the same name.","commonSituations":"hicache-size larger than the container's /dev/shm limit (common in Docker, default 64MB); read-only or missing /dev/shm in minimal containers; leftover shm files from crashed runs filling tmpfs.","solutions":["Enlarge tmpfs: mount -o remount,size=... /dev/shm or docker --shm-size=","Lower --hicache-size below the /dev/shm capacity","Clean stale files in /dev/shm","Verify /dev/shm is mounted rw (cat /proc/mounts | grep shm)"],"exampleFix":"# before\ndocker run ... # default 64MB shm; hicache-size 10gb -> OSError\n# after\ndocker run --shm-size=32gb ... sglang.launch_server --enable-hicache --hicache-size 20gb","handlingStrategy":"validation","validationCode":"import shutil, os\ndef shm_can_hold(n_bytes: int) -> bool:\n    total, used, free = shutil.disk_usage('/dev/shm')\n    return free > n_bytes\nassert shm_can_hold(alloc_bytes), '/dev/shm too small; raise --shm-size or lower hicache-size'","typeGuard":null,"tryCatchPattern":"try:\n    t = alloc_shm(name, size)\nexcept OSError as e:\n    if 'Failed to create shm file' in str(e):\n        # inspect root cause via __cause__; enlarge /dev/shm or shrink pool\n        raise","preventionTips":["Set docker --shm-size to at least the hicache host pool size","Periodically clean stale /dev/shm entries","Alert on /dev/shm usage before it fills"],"tags":["shm","tmpfs","disk-full","hicache","docker"],"backgroundTag":"tmpfs-no-space-left","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}