{"record":{"id":"e49dc4e12102c3e9","repo":"sgl-project/sglang","slug":"short-read-for-suffixed","errorCode":null,"errorMessage":"Short read for {suffixed}","messagePattern":"Short read for (.+?)","errorType":"error_code","errorClass":"IOError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/hicache_storage.py","lineNumber":478,"sourceCode":"            stem = fn[:-4]\n            # Only files belonging to this rank/model.\n            if stem.endswith(self.config_suffix):\n                self.metadata_cache.add(stem)\n\n    def get(\n        self,\n        key: str,\n        target_location: torch.Tensor,\n        target_sizes: Optional[Any] = None,\n    ) -> torch.Tensor | None:\n        suffixed = self._get_suffixed_key(key)\n        tensor_path = os.path.join(self.file_path, f\"{suffixed}.bin\")\n        try:\n            expected = target_location.numel() * target_location.element_size()\n            with open(tensor_path, \"rb\", buffering=0) as f:\n                buf = memoryview(target_location.view(torch.uint8).contiguous().numpy())\n                if f.readinto(buf) != expected:\n                    raise IOError(f\"Short read for {suffixed}\")\n            self._evictor.touch(suffixed, tensor_path)\n            if self.metadata_cache is not None:\n                self.metadata_cache.add(suffixed)\n            return target_location\n        except FileNotFoundError:\n            if self.metadata_cache is not None:\n                self.metadata_cache.remove(suffixed)\n            logger.warning(f\"Failed to fetch {key} from HiCacheFile storage.\")\n            return None\n\n    def batch_get(\n        self,\n        keys: List[str],\n        target_locations: List[torch.Tensor],\n        target_sizes: Optional[Any] = None,\n    ) -> List[torch.Tensor | None]:\n        return [\n            self.get(key, target_location)","sourceCodeStart":460,"sourceCodeEnd":496,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/hicache_storage.py#L460-L496","documentation":"Raised by HiCache file storage backend (hicache_storage.py) when reading a tensor page from a .bin file on disk returns fewer bytes than the tensor's expected size (numel * element_size). It means the backing file is truncated, corrupted, or was written with a different layout/dtype than the reader expects. The read is done via raw f.readinto into the target tensor's byte buffer, so any size mismatch surfaces as a short read.","triggerScenarios":"Calling get()/batch_get() on FileSystemBackend (or _read_page) for a suffixed page whose {suffixed}.bin file is smaller than target_location.numel()*element_size(); e.g. file truncated mid-write by a crash, page saved under a different dtype/page size and reused after config change, or disk/NFS returning partial data.","commonSituations":"Reusing a persisted HiCache dump after changing --page-size, kv dtype (fp8 vs bf16), or tensor layout; a server crash while writing pages leaving truncated .bin files; copying cache directories incompletely; multiple writers with different model configs sharing one hicache directory.","solutions":["Delete or regenerate the affected cache files/directory so pages are rewritten with the current layout (page contents are reconstructable, they are just a cache)","Verify hicache configuration (page size, dtype, layer count, mem layout) matches what was used when the dump was written","Check filesystem integrity/free space and ensure the writer fully flushes/closes files (fsync) before readers access them","If sharing a cache directory across models/runs, give each config its own --hicache-dir to avoid layout mismatches"],"exampleFix":"# before: reusing stale dump after changing page size / dtype\nserver_args = ServerArgs(..., hicache_dir=\"/cache/shared\", page_size=64)\n\n# after: separate directory per layout, or wipe stale dump\nimport shutil; shutil.rmtree(\"/cache/shared\", ignore_errors=True)\nserver_args = ServerArgs(..., hicache_dir=\"/cache/mha-bf16-p64\", page_size=64)","handlingStrategy":"retry","validationCode":"import os\nexpected = target.numel() * target.element_size()\npath = os.path.join(hicache_dir, f\"{suffixed}.bin\")\nif os.path.exists(path) and os.path.getsize(path) != expected:\n    os.remove(path)  # stale/corrupt page; it will be recomputed","typeGuard":null,"tryCatchPattern":"try:\n    tensor = backend.get(suffixed, target)\nexcept IOError as e:\n    if \"Short read\" in str(e):\n        # treat as cache miss: delete corrupt file and recompute page\n        os.remove(os.path.join(backend.file_path, f\"{suffixed}.bin\"))\n        tensor = None\n    else:\n        raise","preventionTips":["Use a dedicated --hicache-dir per (model, page-size, dtype) combination","Ensure writers fsync/close page files before readers can see them","Periodically validate cache file sizes against expected page bytes"],"tags":["hicache","file-io","cache-corruption","truncated-file"],"backgroundTag":"file-read-truncated","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}