sgl-project/sglang · error · ValueError

layer.layer_id={layer.layer_id} is not in the K-only sparse

Error message

layer.layer_id={layer.layer_id} is not in the K-only sparse group. K-only layers: {list(self.index_k_layer_id_mapping.keys())}

What it means

set_index_k_buffer on MiniMaxSparseKVPool writes only into the K-only sub-pool (index_k_pool) and requires the layer to be in index_k_layer_id_mapping. The supplied layer.layer_id is not in the K-only sparse group, so the write is rejected rather than silently writing to the wrong slot.

Source

Thrown at python/sglang/srt/mem_cache/memory_pool.py:5038

            layer,
            loc,
            cache_idx_k,
            cache_idx_v,
            k_scale,
            v_scale,
            layer_id_override=mapped_id,
        )

    def set_index_k_buffer(
        self,
        layer: RadixAttention,
        loc: torch.Tensor,
        cache_idx_k: torch.Tensor,
        k_scale: Optional[float] = None,
    ) -> None:
        mapped_id = self.index_k_layer_id_mapping.get(layer.layer_id)
        if mapped_id is None:
            raise ValueError(
                f"layer.layer_id={layer.layer_id} is not in the K-only "
                f"sparse group. K-only layers: "
                f"{list(self.index_k_layer_id_mapping.keys())}"
            )
        sub_pool = self.index_k_pool
        if cache_idx_k.dtype != sub_pool.dtype:
            if k_scale is not None:
                cache_idx_k = cache_idx_k / k_scale
        sub_pool.set_k_buffer(mapped_id, loc, cache_idx_k)

    def _can_fuse_kv_index_store(
        self,
        index_pool: MHATokenToKVPool,
        cache_k: torch.Tensor,
        cache_idx_k: torch.Tensor,
    ) -> bool:
        """Fast-path precondition: CUDA, no per-store quantization, and a uniform
        head byte size shared by main and index caches."""

View on GitHub (pinned to 0132848349)

Solutions

1. Use set_index_kv_buffer for layers in index_kv_layer_id_mapping and the main pool for dense layers
2. Check layer.layer_id in sparse_pool.index_k_layer_id_mapping before calling
3. Validate the K-only vs index-KV layer split used at pool construction matches the model

Example fix

# before
sparse_pool.set_index_k_buffer(layer, loc, cache_k, k_scale)  # index-KV layer
# after
if layer.layer_id in sparse_pool.index_k_layer_id_mapping:
    sparse_pool.set_index_k_buffer(layer, loc, cache_k, k_scale)
else:
    sparse_pool.set_index_kv_buffer(layer, loc, cache_k, cache_v, k_scale, v_scale)

if layer.layer_id not in sparse_pool.index_k_layer_id_mapping:
    raise ValueError(f'{layer.layer_id} not in K-only group; use set_index_kv_buffer')

def is_k_only_layer(sparse_pool, layer) -> bool:
    return sparse_pool.index_k_layer_id_mapping.get(layer.layer_id) is not None

When it happens

Trigger: Calling set_index_k_buffer(layer, loc, cache_idx_k, k_scale) where layer.layer_id maps to None in index_k_layer_id_mapping — e.g. an index-KV layer or a dense layer.

Common situations: Model code routing all sparse layers through the K-only setter; layer group misclassification in the model config; refactors that swapped the set_index_k_buffer / set_index_kv_buffer call sites.

Related errors

• layer.layer_id={layer.layer_id} does not have an index V cac
• layer_id={layer_id} does not have an index V cache (either d
• layer_id={layer_id} is not a sparse attention layer; sparse
• MHATokenToKOnlyPool does not allocate V
• MHATokenToKOnlyPool: use set_index_k_buffer on the parent Mi

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0f381726e0cb23d2. Report an issue: GitHub.