{"record":{"id":"e2574b3655dd88f0","repo":"sgl-project/sglang","slug":"use-get-key-buffer-instead","errorCode":null,"errorMessage":"Use get_key_buffer instead.","messagePattern":"Use get_key_buffer instead\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/deepseek_v4_memory_pool.py","lineNumber":171,"sourceCode":"        return fused_store_cache(\n            input=cache_k,\n            cache=self.kv_buffer[layer_id],\n            indices=loc,\n            page_size=self.page_size,\n            type=\"flashmla\",\n        )\n\n    def get_key_buffer(self, layer_id: int):\n        if self.store_dtype != self.dtype:\n            return self.kv_buffer[layer_id - self.start_layer].view(self.dtype)\n\n        return self.kv_buffer[layer_id]\n\n    def set_kv_buffer(self, *args, **kwargs) -> None:\n        raise NotImplementedError()\n\n    def get_value_buffer(self, layer_id: int) -> torch.Tensor:\n        raise NotImplementedError(\"Use get_key_buffer instead.\")\n\n    def get_kv_buffer(self, layer_id: int) -> Tuple[torch.Tensor, torch.Tensor]:\n        raise NotImplementedError(\"Use get_key_buffer instead.\")\n\n\nclass HiSparseC4DevicePool(DeepSeekV4SingleKVPool):\n\n    def __init__(\n        self,\n        size: int,\n        page_size: int,\n        dtype: torch.dtype,\n        qk_nope_head_dim: int,\n        qk_rope_head_dim: int,\n        layer_num: int,\n        device: str,\n        enable_memory_saver: bool,\n        start_layer: int | None = None,","sourceCodeStart":153,"sourceCodeEnd":189,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/deepseek_v4_memory_pool.py#L153-L189","documentation":"The DeepSeek-V4 single-KV pool stores keys and values in one shared kv_buffer and has no separate value buffer, so get_value_buffer and get_kv_buffer deliberately raise NotImplementedError telling callers to use get_key_buffer instead (set_kv_buffer is likewise not supported).","triggerScenarios":"Calling pool.get_value_buffer(layer_id) or pool.get_kv_buffer(layer_id) on a DeepSeekV4SingleKVPool / HiSparseC4DevicePool instance — e.g. generic attention backend or custom kernel code that assumes the standard MHATokenToKVPool interface.","commonSituations":"Porting an attention backend or profiler written against standard KV pools to DSv4's single-KV layout; third-party code calling get_kv_buffer unconditionally over all pool types.","solutions":["Replace get_value_buffer()/get_kv_buffer() calls with get_key_buffer(layer_id), which returns the shared kv_buffer for this pool","Add an isinstance/hasattr check before calling the generic accessors","Extend DeepSeekV4SingleKVPool in a subclass if you genuinely need a separate value view"],"exampleFix":"# before\nk, v = pool.get_kv_buffer(layer_id)\n# after\nk = pool.get_key_buffer(layer_id)  # single-KV pool: shared buffer","handlingStrategy":"type-guard","validationCode":"if isinstance(pool, DeepSeekV4SingleKVPool):\n    buf = pool.get_key_buffer(layer_id)\nelse:\n    k, v = pool.get_kv_buffer(layer_id)","typeGuard":"from sglang.srt.mem_cache.deepseek_v4_memory_pool import DeepSeekV4SingleKVPool\n\ndef is_single_kv_pool(pool) -> bool:\n    return isinstance(pool, DeepSeekV4SingleKVPool) or type(pool).get_value_buffer is DeepSeekV4SingleKVPool.get_value_buffer","tryCatchPattern":"try:\\n    k, v = pool.get_kv_buffer(layer_id)\\nexcept NotImplementedError:\\n    k = pool.get_key_buffer(layer_id)  # single-KV layout","preventionTips":["Query the pool class before using standard KV accessors","Write backend code against an accessor helper that dispatches on pool type","Read the pool class docstring before extending attention backends"],"tags":["deepseek-v4","memory-pool","api-misuse","not-implemented"],"backgroundTag":"unsupported-api-call","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}