{"record":{"id":"92dbea609232f1bc","repo":"sgl-project/sglang","slug":"not-enough-host-memory-for-dsa-indexer-hierarchica","errorCode":null,"errorMessage":"Not enough host memory for DSA indexer hierarchical cache. Requesting {requested_bytes / 1e9:.2f} GB but only have {available_bytes / 1e9:.2f} GB free.","messagePattern":"Not enough host memory for DSA indexer hierarchical cache\\. Requesting (.+?) GB but only have (.+?) GB free\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/srt/mem_cache/pool_host/dsa.py","lineNumber":99,"sourceCode":"            + self.index_head_dim // self.indexer_quant_block_size * 4\n        )\n        self.size = anchor_host.size\n        self.page_num = anchor_host.page_num\n\n        self.indexer_page_stride_size = (\n            self.indexer_size_per_token * self.page_size * self.indexer_dtype.itemsize\n        )\n        self.indexer_layout_dim = self.indexer_page_stride_size * self.layer_num\n        self.indexer_page_num = (self.size + self.page_size + 1) // self.page_size\n        self.size_per_token = (\n            self.indexer_size_per_token * self.layer_num * self.indexer_dtype.itemsize\n        )\n\n        buf_elem_size = self.page_num * self.layer_num * self.indexer_page_stride_size\n        requested_bytes = buf_elem_size * self.indexer_dtype.itemsize\n        available_bytes = host_memory_budget_bytes()\n        if requested_bytes > available_bytes:\n            raise ValueError(\n                f\"Not enough host memory for DSA indexer hierarchical cache. \"\n                f\"Requesting {requested_bytes / 1e9:.2f} GB but only have \"\n                f\"{available_bytes / 1e9:.2f} GB free.\"\n            )\n        draft_layer_num = self.layer_num - self.target_layer_num\n        if draft_layer_num > 0:\n            logger.info(\n                \"Allocating %.2f GB host memory for DSA indexer (layout=%s), \"\n                \"packed MTP layers: \"\n                \"target_layers=%d, draft_layers=%d, total_layers=%d.\",\n                requested_bytes / 1e9,\n                layout,\n                self.target_layer_num,\n                draft_layer_num,\n                self.layer_num,\n            )\n        else:\n            logger.info(","sourceCodeStart":81,"sourceCodeEnd":117,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/pool_host/dsa.py#L81-L117","documentation":"The DSA (DeepSeek Sparse Attention) indexer hierarchical cache host pool sizes its index buffer from page_num * layer_num * page stride * dtype itemsize, and checks it against the free host memory budget before allocating. If the required buffer exceeds available host RAM it refuses to construct. This protects the process from an OOM kill at allocation time.","triggerScenarios":"Constructing DSATokenToKVPoolHost (dsa.py __init__) with a large --hp-size / page count, many layers, or a wide indexer dtype on a machine whose host_memory_budget_bytes() reports less free memory than the buffer needs.","commonSituations":"Setting a very large hierarchical cache size (hp-size) relative to physical RAM; running alongside other processes consuming host memory; fp8 vs bf16 indexer dtype changes altering itemsize.","solutions":["Reduce hierarchical cache size (e.g. lower --hp-size / max host pages)","Free host memory or increase the machine's RAM / cgroup limit","Use a smaller indexer dtype if supported","Compute page_num*layer_num*stride*itemsize up front and validate against psutil free memory before launching"],"exampleFix":"# before\nserver_args.hierarchical_cache_size_gb = 200  # exceeds free RAM\n# after\nserver_args.hierarchical_cache_size_gb = 50   # fit within host_memory_budget_bytes()","handlingStrategy":"validation","validationCode":"import psutil\nrequested = page_num * layer_num * page_stride * dtype_itemsize\navail = psutil.virtual_memory().available\nassert requested <= avail, f\"need {requested/1e9:.1f}GB, have {avail/1e9:.1f}GB\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Size hierarchical cache GB against actual free RAM before launch","Monitor host memory when co-locating other processes"],"tags":["memory","dsa-hicache","host-memory","startup"],"backgroundTag":"out-of-host-memory","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}