sgl-project/sglang · error · ValueError

layer_id={layer_id} does not have an index V cache (either d

Error message

layer_id={layer_id} does not have an index V cache (either dense, or in the K-only group). index_kv layers: {list(self.index_kv_layer_id_mapping.keys())}

What it means

MiniMaxSparseKVPool.get_index_kv_buffer only serves layers registered in index_kv_layer_id_mapping (sparse layers that keep both an index K and an index V cache). The requested layer_id is either a dense layer or a K-only sparse layer, so there is no index-V cache to return.

Source

Thrown at python/sglang/srt/mem_cache/memory_pool.py:4958

            self.layer_transfer_counter.wait_until(layer_id - self.start_layer)

    def get_key_buffer(self, layer_id: int) -> torch.Tensor:
        self._wait_for_layer(layer_id)
        return self.main_pool.get_key_buffer(layer_id)

    def get_value_buffer(self, layer_id: int) -> torch.Tensor:
        self._wait_for_layer(layer_id)
        return self.main_pool.get_value_buffer(layer_id)

    def get_kv_buffer(self, layer_id: int) -> Tuple[torch.Tensor, torch.Tensor]:
        self._wait_for_layer(layer_id)
        return self.main_pool.get_kv_buffer(layer_id)

    def get_index_kv_buffer(self, layer_id: int) -> Tuple[torch.Tensor, torch.Tensor]:
        self._wait_for_layer(layer_id)
        mapped_id = self.index_kv_layer_id_mapping.get(layer_id)
        if mapped_id is None:
            raise ValueError(
                f"layer_id={layer_id} does not have an index V cache "
                f"(either dense, or in the K-only group). "
                f"index_kv layers: {list(self.index_kv_layer_id_mapping.keys())}"
            )
        return self.index_kv_pool.get_kv_buffer(mapped_id)

    def get_index_k_buffer(self, layer_id: int) -> torch.Tensor:
        self._wait_for_layer(layer_id)
        # First try the K-only pool; fall back to the index_kv pool's K side
        # so callers that just need K work for both sparse subgroups.
        mapped_id = self.index_k_layer_id_mapping.get(layer_id)
        if mapped_id is not None:
            return self.index_k_pool.get_key_buffer(mapped_id)
        mapped_id = self.index_kv_layer_id_mapping.get(layer_id)
        if mapped_id is not None:
            return self.index_kv_pool.get_key_buffer(mapped_id)
        raise ValueError(
            f"layer_id={layer_id} is not a sparse attention layer; "

View on GitHub (pinned to 0132848349)

Solutions

1. Log list(self.index_kv_layer_id_mapping.keys()) and only call this API for those layer ids
2. For dense layers use main_pool.get_kv_buffer; for K-only layers use get_key_buffer
3. Fix the model config / pool construction so the layer is registered as index-KV if it should carry a V cache

Example fix

// before
k, v = sparse_pool.get_index_kv_buffer(layer_id)  # dense or K-only layer
// after
if layer_id in sparse_pool.index_kv_layer_id_mapping:
    k, v = sparse_pool.get_index_kv_buffer(layer_id)
else:
    k, v = sparse_pool.main_pool.get_kv_buffer(layer_id)

if layer_id not in sparse_pool.index_kv_layer_id_mapping:
    raise ValueError(f'{layer_id} has no index-V cache; use main_pool.get_kv_buffer')

def is_index_kv_layer(sparse_pool, layer_id: int) -> bool:
    return sparse_pool.index_kv_layer_id_mapping.get(layer_id) is not None

When it happens

Trigger: Calling get_index_kv_buffer(layer_id) where layer_id maps to None in index_kv_layer_id_mapping — e.g. passing a dense attention layer id or a K-only group layer id.

Common situations: Custom backends iterating all layers with the index-KV accessor; mismatch between the model's sparse layer partition (K-only vs K+V) and caller assumptions; misconfigured sparse layer lists in the model config.

Related errors

• layer_id={layer_id} is not a sparse attention layer; sparse
• layer.layer_id={layer.layer_id} does not have an index V cac
• layer.layer_id={layer.layer_id} is not in the K-only sparse
• {layer_id=} not in full attention layers: {self.full_attenti
• MHATokenToKOnlyPool does not allocate V

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d66172781ba25644. Report an issue: GitHub.