sgl-project/sglang · error · ValueError

{debug_name}: cache_head_start required for head slice

Error message

{debug_name}: cache_head_start required for head slice

What it means

Same head-slice contract as the dense cache, but for QVGPackedCausalKVCache: if the input key's head count differs from the cache's num_heads, a cache_head_start offset must be supplied to locate the head slice to write.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/kvcache/qvg_packed_cache.py:286

                    self._pack(s)

    # -------------------------------------------------------------- contract
    def update_and_get_attention_kv(
        self,
        *,
        key: torch.Tensor,
        value: torch.Tensor,
        current_chunk_start: int,
        cache_head_start: int | None = None,
        recent_window_tokens: int | None = None,
        debug_name: str = "QVG packed KV cache",
    ) -> CausalAttentionKVView:
        num_new = key.shape[1]
        num_input_heads = key.shape[2]
        head_slice = None
        if num_input_heads != self.num_heads:
            if cache_head_start is None:
                raise ValueError(
                    f"{debug_name}: cache_head_start required for head slice"
                )
            head_slice = slice(cache_head_start, cache_head_start + num_input_heads)
        cend = current_chunk_start + num_new

        if self._cur is not None and current_chunk_start == self._cur.g0:
            # rewrite current chunk in place (denoise step)
            if cend != self._cur.g1:
                raise NotImplementedError(
                    f"{debug_name}: current-chunk rewrite size changed"
                )
            self._write(self._cur, key, value, head_slice)
        elif current_chunk_start == self._global_end:
            # advance: finalize current chunk, start a new one
            if self._cur is not None:
                self._segments.append(self._cur)
            is_sink = current_chunk_start < self.sink_tokens
            if self._chunk_tokens == 0:

View on GitHub (pinned to 0132848349)

Solutions

1. Pass cache_head_start (first cache-head index for this writer)
2. Match input head count to the cache's num_heads if slicing is unintended
3. Check cache allocation uses the maximum head count across contributing layers

Example fix

# before
cache.update_and_get_attention_kv(k, v, current_chunk_start=s)
# after
cache.update_and_get_attention_kv(k, v, current_chunk_start=s, cache_head_start=off)

if key.shape[2] != cache.num_heads:
    assert cache_head_start is not None

When it happens

Trigger: update_and_get_attention_kv on the packed cache with num_input_heads != self.num_heads and cache_head_start=None.

Common situations: GQA/MQA-style layers or shared packed caches across layers with different KV head counts; new model wiring that forgot the head offset argument.

Related errors

• {debug_name} requires cache_head_start when cache heads ({nu
• k_pool has incompatible shape {k_pool.shape}
• recent_window_tokens must be non-negative or None
• recent_window_tokens must be >= 0 or None
• v_cache must be provided

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4f75640cdd72e5b7. Report an issue: GitHub.