{"record":{"id":"16306fca9be32d72","repo":"sgl-project/sglang","slug":"cpu-offloading-is-unsupported-under-the-page-major","errorCode":null,"errorMessage":"CPU offloading is unsupported under the page-major layout (TODO: split token ids into page/slot for the 4-D index).","messagePattern":"CPU offloading is unsupported under the page-major layout \\(TODO: split token ids into page/slot for the 4-D index\\)\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/memory_pool.py","lineNumber":3320,"sourceCode":"            tgt_loc,\n            src_loc,\n            page_size=self.page_size,\n        )\n\n    # The methods below assume the per-layer contiguous 3-D layout. The 4-D\n    # strided envelope views have no per-layer contiguous region (their bytes are\n    # interleaved layer-major within each page) and index page-major, not\n    # token-major. Inheriting them would silently mis-index; fail loudly instead.\n\n    def get_contiguous_buf_infos(self):\n        raise NotImplementedError(\n            \"page-major layout has no per-layer contiguous regions; KV transfer / \"\n            \"disaggregation is unsupported (TODO: expose the single _raw buffer \"\n            \"with a page-aware transfer scheme).\"\n        )\n\n    def get_cpu_copy(self, indices, mamba_indices=None):\n        raise NotImplementedError(\n            \"CPU offloading is unsupported under the page-major layout \"\n            \"(TODO: split token ids into page/slot for the 4-D index).\"\n        )\n\n    def load_cpu_copy(self, kv_cache_cpu, indices, mamba_indices=None):\n        raise NotImplementedError(\n            \"CPU offloading is unsupported under the page-major layout \"\n            \"(TODO: split token ids into page/slot for the 4-D index).\"\n        )\n\n    def set_kv_buffer_prefix_valid(self, *args, **kwargs):\n        raise NotImplementedError(\n            \"prefix-valid commit is unsupported under the page-major layout \"\n            \"(_set_kv_buffer_prefix_valid_impl assumes 3-D contiguous + row_dim).\"\n        )\n\n\nclass MHATokenToKVPoolMXFP8(MHATokenToKVPool):","sourceCodeStart":3302,"sourceCodeEnd":3338,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/memory_pool.py#L3302-L3338","documentation":"CPU offloading (get_cpu_copy) requires copying per-layer contiguous KV slices indexed token-major, but the page-major 4-D layout interleaves layers within pages, so the inherited implementation would copy wrong bytes. The method raises NotImplementedError until a page/slot-splitting index is implemented.","triggerScenarios":"Allocating the page-major KV pool and enabling CPU offloading of KV cache (hi-CPU offload / hierarchical cache), which calls pool.get_cpu_copy(indices, mamba_indices).","commonSituations":"Combining a paged page-major KV layout config with --enable-hierarchical-cache or CPU offload settings; recent versions introducing the page-major envelope layout while offload code still assumes 3-D.","solutions":["Disable CPU offloading / hierarchical cache when using the page-major layout","Switch the pool to the token-major 3-D layout (page_size=1) to keep CPU offloading usable","Implement page/slot decomposition for the 4-D index before calling get_cpu_copy"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"def offload_supported(pool) -> bool:\n    return not getattr(pool, 'page_size', 1) > 1  # token-major 3-D pools only","typeGuard":null,"tryCatchPattern":"try:\n    cpu_copy = pool.get_cpu_copy(indices)\nexcept NotImplementedError:\n    logger.warning('CPU offload unavailable for this layout; skipping offload')","preventionTips":["Validate enable-hierarchical-cache/offload flags against pool layout at launch","Keep a compatibility matrix of layout vs offload features in CI","Fail fast at startup instead of at first offload attempt"],"tags":["kv-cache","cpu-offload","page-major","not-implemented"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}