{"record":{"id":"5bdcab7958c6ce30","repo":"sgl-project/sglang","slug":"mhatokentokonlypool-does-not-allocate-v","errorCode":null,"errorMessage":"MHATokenToKOnlyPool does not allocate V","messagePattern":"MHATokenToKOnlyPool does not allocate V","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/mem_cache/memory_pool.py","lineNumber":4783,"sourceCode":"    def get_key_buffer(self, layer_id: int):\n        if self.layer_transfer_counter is not None:\n            self.layer_transfer_counter.wait_until(layer_id - self.start_layer)\n        return self._get_key_buffer(layer_id)\n\n    def set_k_buffer(\n        self,\n        layer_id: int,\n        loc: torch.Tensor,\n        cache_k: torch.Tensor,\n    ) -> None:\n        if cache_k.dtype != self.dtype:\n            cache_k = cache_k.to(self.dtype)\n        if self.store_dtype != self.dtype:\n            cache_k = cache_k.view(self.store_dtype)\n        self.k_buffer[layer_id][loc] = cache_k\n\n    def get_value_buffer(self, layer_id: int) -> torch.Tensor:\n        raise NotImplementedError(\"MHATokenToKOnlyPool does not allocate V\")\n\n    def get_kv_buffer(self, layer_id: int) -> Tuple[torch.Tensor, torch.Tensor]:\n        raise NotImplementedError(\"MHATokenToKOnlyPool does not allocate V\")\n\n    def set_kv_buffer(\n        self,\n        layer: RadixAttention,\n        loc: torch.Tensor,\n        cache_k: torch.Tensor,\n        cache_v: torch.Tensor,\n        k_scale: Optional[float] = None,\n        v_scale: Optional[float] = None,\n        layer_id_override: Optional[int] = None,\n    ) -> None:\n        # Routed through MiniMaxSparseKVPool.set_index_k_buffer instead.\n        raise NotImplementedError(\n            \"MHATokenToKOnlyPool: use set_index_k_buffer on the parent \"\n            \"MiniMaxSparseKVPool — this pool does not store V\"","sourceCodeStart":4765,"sourceCodeEnd":4801,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/mem_cache/memory_pool.py#L4765-L4801","documentation":"MHATokenToKOnlyPool is a K-only sparse pool that never allocates a V buffer; calling get_value_buffer on it is unsupported by design. V for these layers lives in the parent MiniMaxSparseKVPool's index-V pool, not here.","triggerScenarios":"Calling pool.get_value_buffer(layer_id) where pool is (or delegates to) an MHATokenToKOnlyPool — e.g. generic code that assumes every KV pool can return both K and V buffers.","commonSituations":"Running MiniMax-style sparse-attention models with generic memory-pool tooling, metrics collection, or custom attention backends that iterate get_value_buffer over all layers.","solutions":["Use the parent MiniMaxSparseKVPool.get_index_kv_buffer / the pool API that routes V lookups to the index-V pool","Guard calls: only call get_value_buffer for layers with a V cache (check the layer id mapping)","Use get_key_buffer for K-only layers"],"exampleFix":"// before\nv = pool.get_value_buffer(layer_id)  # pool is MHATokenToKOnlyPool\n// after\nk = pool.get_key_buffer(layer_id)\nv = parent_sparse_pool.get_index_kv_buffer(index_kv_layer_id)","handlingStrategy":"type-guard","validationCode":"from sglang.srt.mem_cache.memory_pool import MHATokenToKOnlyPool\nif isinstance(pool, MHATokenToKOnlyPool):\n    raise TypeError('K-only pool has no V buffer; use parent sparse pool accessors')","typeGuard":"def has_value_buffer(pool) -> bool:\n    from sglang.srt.mem_cache.memory_pool import MHATokenToKOnlyPool\n    return not isinstance(pool, MHATokenToKOnlyPool)","tryCatchPattern":"try:\n    v = pool.get_value_buffer(layer_id)\nexcept NotImplementedError:\n    v = None  # K-only layer; V lives in the index-V pool","preventionTips":["Route all buffer access through the parent MiniMaxSparseKVPool dispatchers","Never assume get_value_buffer exists for every sub-pool"],"tags":["kv-cache","k-only-pool","sparse-attention","minimax","sglang"],"backgroundTag":"unsupported-operation-for-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}