{"record":{"id":"64b40021e9b9be6e","repo":"sgl-project/sglang","slug":"prefix-valid-commit-is-unsupported-for-mxfp8-kv-ca","errorCode":null,"errorMessage":"prefix-valid commit is unsupported for MXFP8 KV cache (it does not carry the scale buffers).","messagePattern":"prefix-valid commit is unsupported for MXFP8 KV cache \\(it does not carry the scale buffers\\)\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/memory_pool.py","lineNumber":3680,"sourceCode":"\n    def get_kv_scale_buf_infos(self):\n        \"\"\"(ptrs, lens, item_lens) for the UE8M0 scale buffers, k then v.\n\n        The interleaved layout puts pages on the leading axis, so a page's\n        scales are one contiguous row; the flat layout is per slot.\n        \"\"\"\n        tensors = self.k_scale_buffer + self.v_scale_buffer\n        ptrs = [t.data_ptr() for t in tensors]\n        lens = [t.nbytes for t in tensors]\n        row_bytes = [t[0].nbytes for t in tensors]\n        if self.mxfp8_sf_interleaved:\n            item_lens = row_bytes\n        else:\n            item_lens = [rb * self.page_size for rb in row_bytes]\n        return ptrs, lens, item_lens\n\n    def set_kv_buffer_prefix_valid(self, *args, **kwargs):\n        raise NotImplementedError(\n            \"prefix-valid commit is unsupported for MXFP8 KV cache \"\n            \"(it does not carry the scale buffers).\"\n        )\n\n    def get_kv_size_bytes(self):\n        k_size_bytes = 0\n        v_size_bytes = 0\n        for k_cache in self.k_buffer:\n            k_size_bytes += get_tensor_size_bytes(k_cache)\n        for k_scale in self.k_scale_buffer:\n            k_size_bytes += get_tensor_size_bytes(k_scale)\n        for v_cache in self.v_buffer:\n            v_size_bytes += get_tensor_size_bytes(v_cache)\n        for v_scale in self.v_scale_buffer:\n            v_size_bytes += get_tensor_size_bytes(v_scale)\n        return k_size_bytes, v_size_bytes\n\n","sourceCodeStart":3662,"sourceCodeEnd":3698,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/memory_pool.py#L3662-L3698","documentation":"MHAMXFP8TokenToKVPool deliberately does not implement set_kv_buffer_prefix_valid, the prefix-valid commit path that copies raw KV buffers. Because the MXFP8 pool stores fp8 payloads with separate scale buffers, a raw-copy commit would drop the scales and corrupt the cache, so the operation is refused.","triggerScenarios":"Any code path that commits KV entries via set_kv_buffer_prefix_valid (prefix caching reuse / transfer flows that mark prefixes valid by pointer copy) while the memory pool is an MXFP8 KV pool.","commonSituations":"Enabling prefix caching or a KV-transfer/HiCache feature that uses prefix-valid commits together with MXFP8 KV cache quantization; newer SGLang versions adding new prefix-commit codepaths that reach this pool.","solutions":["Disable the feature that uses prefix-valid commits (e.g. turn off the specific prefix-transfer option or hierarchical cache mode) when using MXFP8 KV cache","Use a non-MXFP8 kv-cache-dtype if you need prefix-valid commits","Wait for / implement scale-aware prefix commit in MHAMXFP8TokenToKVPool instead of the raw copy"],"exampleFix":"// before\n--kv-cache-dtype mxfp8 --enable-hierarchical-cache  # triggers set_kv_buffer_prefix_valid\n// after\n--kv-cache-dtype fp8_e4m3 --enable-hierarchical-cache","handlingStrategy":"validation","validationCode":"if isinstance(pool, MHAMXFP8TokenToKVPool):\n    raise SystemExit('prefix-valid commit unsupported with MXFP8 KV cache; disable it or change kv-cache-dtype')","typeGuard":"def supports_prefix_valid(pool) -> bool:\n    return not type(pool).__name__.startswith('MHAMXFP8')","tryCatchPattern":"try:\n    pool.set_kv_buffer_prefix_valid(*args)\nexcept NotImplementedError as e:\n    logger.warning('skipping prefix-valid commit: %s', e)","preventionTips":["Document the MXFP8 + prefix-transfer incompatibility in deployment runbooks","Add a startup config check that rejects the combination"],"tags":["mxfp8","kv-cache","prefix-caching","not-implemented","sglang"],"backgroundTag":"unsupported-operation-for-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}