{"record":{"id":"e2630f9b18a650b3","repo":"sgl-project/sglang","slug":"prefix-valid-commit-is-unsupported-under-the-page","errorCode":null,"errorMessage":"prefix-valid commit is unsupported under the page-major layout (_set_kv_buffer_prefix_valid_impl assumes 3-D contiguous + row_dim).","messagePattern":"prefix-valid commit is unsupported under the page-major layout \\(_set_kv_buffer_prefix_valid_impl assumes 3-D contiguous \\+ row_dim\\)\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/memory_pool.py","lineNumber":3332,"sourceCode":"            \"page-major layout has no per-layer contiguous regions; KV transfer / \"\n            \"disaggregation is unsupported (TODO: expose the single _raw buffer \"\n            \"with a page-aware transfer scheme).\"\n        )\n\n    def get_cpu_copy(self, indices, mamba_indices=None):\n        raise NotImplementedError(\n            \"CPU offloading is unsupported under the page-major layout \"\n            \"(TODO: split token ids into page/slot for the 4-D index).\"\n        )\n\n    def load_cpu_copy(self, kv_cache_cpu, indices, mamba_indices=None):\n        raise NotImplementedError(\n            \"CPU offloading is unsupported under the page-major layout \"\n            \"(TODO: split token ids into page/slot for the 4-D index).\"\n        )\n\n    def set_kv_buffer_prefix_valid(self, *args, **kwargs):\n        raise NotImplementedError(\n            \"prefix-valid commit is unsupported under the page-major layout \"\n            \"(_set_kv_buffer_prefix_valid_impl assumes 3-D contiguous + row_dim).\"\n        )\n\n\nclass MHATokenToKVPoolMXFP8(MHATokenToKVPool):\n    \"\"\"MHA KV cache pool for MXFP8 block-scaled FP8.\n\n    K/V data is stored as FP8 E4M3. Per-32-element UE8M0 scale factors are\n    stored beside it and passed to the FA4 MXFP8 kernel.\n    \"\"\"\n\n    MXFP8_SCALE_BLOCK_SIZE = 32\n\n    def _create_buffers(self):\n        with self.memory_saver_adapter.region(GPU_MEMORY_TYPE_KV_CACHE):\n            with (\n                torch.cuda.use_mem_pool(self.custom_mem_pool)","sourceCodeStart":3314,"sourceCodeEnd":3350,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/memory_pool.py#L3314-L3350","documentation":"set_kv_buffer_prefix_valid commits prefix-cached KV rows via _set_kv_buffer_prefix_valid_impl, which assumes a 3-D contiguous buffer with a row_dim. The page-major 4-D strided envelope has no such per-layer contiguous rows, so the call raises NotImplementedError rather than mis-committing bytes.","triggerScenarios":"Using the page-major KV layout together with prefix caching / speculative prefix-valid commit logic that calls pool.set_kv_buffer_prefix_valid(...) after a verified prefix hit.","commonSituations":"Enabling prefix caching (radix cache) or the prefix-valid optimization on a config that selects the page-major KV pool layout; newer SGlang versions where prefix-valid commit was added but not ported to the 4-D layout.","solutions":["Disable prefix caching / the prefix-valid commit path when using the page-major layout","Fall back to the token-major 3-D pool (page_size=1) to use prefix-valid commits","Port _set_kv_buffer_prefix_valid_impl to handle the 4-D page-major envelope before enabling it"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"def prefix_valid_supported(pool) -> bool:\n    return hasattr(pool, '_set_kv_buffer_prefix_valid_impl')","typeGuard":null,"tryCatchPattern":"try:\n    pool.set_kv_buffer_prefix_valid(layer, loc)\nexcept NotImplementedError:\n    logger.warning('prefix-valid commit unsupported; falling back to full write')\n    pool.set_kv_buffer(layer, loc, k, v)","preventionTips":["Check pool capabilities before enabling prefix-valid optimizations","Gate prefix-cache features on pool layout in server args validation"],"tags":["kv-cache","prefix-cache","page-major","not-implemented"],"backgroundTag":"unsupported-feature-combination","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}