sgl-project/sglang · error · ValueError

layer_id={layer_id} is not a sparse attention layer; sparse

Error message

layer_id={layer_id} is not a sparse attention layer; sparse layers: {list(self.sparse_layer_id_mapping.keys())}

What it means

MiniMaxSparseKVPool.get_key_buffer raised because the layer_id appears in neither index_k_layer_id_mapping (K-only sparse group) nor index_kv_layer_id_mapping (index K+V group) — i.e. it is not a sparse attention layer at all, so the sparse pool has no key buffer for it.

Source

Thrown at python/sglang/srt/mem_cache/memory_pool.py:4975

        if mapped_id is None:
            raise ValueError(
                f"layer_id={layer_id} does not have an index V cache "
                f"(either dense, or in the K-only group). "
                f"index_kv layers: {list(self.index_kv_layer_id_mapping.keys())}"
            )
        return self.index_kv_pool.get_kv_buffer(mapped_id)

    def get_index_k_buffer(self, layer_id: int) -> torch.Tensor:
        self._wait_for_layer(layer_id)
        # First try the K-only pool; fall back to the index_kv pool's K side
        # so callers that just need K work for both sparse subgroups.
        mapped_id = self.index_k_layer_id_mapping.get(layer_id)
        if mapped_id is not None:
            return self.index_k_pool.get_key_buffer(mapped_id)
        mapped_id = self.index_kv_layer_id_mapping.get(layer_id)
        if mapped_id is not None:
            return self.index_kv_pool.get_key_buffer(mapped_id)
        raise ValueError(
            f"layer_id={layer_id} is not a sparse attention layer; "
            f"sparse layers: {list(self.sparse_layer_id_mapping.keys())}"
        )

    def set_kv_buffer(
        self,
        layer: RadixAttention,
        loc: torch.Tensor,
        cache_k: torch.Tensor,
        cache_v: torch.Tensor,
        k_scale: Optional[float] = None,
        v_scale: Optional[float] = None,
    ) -> None:
        """Write main K/V at `loc`. Works for any layer (dense or sparse).

        Scale semantics follow MHATokenToKVPool: None means unit scale;
        a non-None scale is applied with an in-place div_ before the fp8 cast.
        """

View on GitHub (pinned to 0132848349)

Solutions

Check membership in index_k_layer_id_mapping / index_kv_layer_id_mapping before calling; dense layers must use main_pool.get_key_buffer
Verify the sparse layer lists used to build the pool match the model's actual layer partition
Use absolute layer ids consistently (no remapped compact ids) when calling this API

Example fix

// before
k = sparse_pool.get_key_buffer(layer_id)  # dense layer -> ValueError
// after
if layer_id in sparse_pool.index_k_layer_id_mapping or layer_id in sparse_pool.index_kv_layer_id_mapping:
    k = sparse_pool.get_key_buffer(layer_id)
else:
    k = sparse_pool.main_pool.get_key_buffer(layer_id)

Defensive patterns

Strategy: validation

Validate before calling

if layer_id not in sparse_pool.index_k_layer_id_mapping and layer_id not in sparse_pool.index_kv_layer_id_mapping:
    k = sparse_pool.main_pool.get_key_buffer(layer_id)  # dense layer

Type guard

def is_sparse_layer(sparse_pool, layer_id: int) -> bool:
    return layer_id in sparse_pool.index_k_layer_id_mapping or layer_id in sparse_pool.index_kv_layer_id_mapping

Prevention

Branch dense vs sparse before buffer access
Keep layer ids absolute and consistent between pool construction and callers

When it happens

Trigger: Calling get_key_buffer on the sparse pool with a dense-attention layer id; or a layer-id enumeration mismatch where the sparse layer sets were built from a different layer ordering than the caller uses.

Common situations: Custom attention backends or profiling code iterating every layer through the sparse pool; model config changes that moved layers between dense and sparse groups; off-by-one or absolute-vs-remapped layer id confusion.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1884cb912fb10d5b. Report an issue: GitHub.