sgl-project/sglang · error · ValueError

{debug_name} requires cache_head_start when cache heads ({nu

Error message

{debug_name} requires cache_head_start when cache heads ({num_cache_heads}) differ from input heads ({num_input_heads}).

What it means

CausalAttentionKVCache.update_and_get_attention_kv raises this when the incoming key tensor's head count differs from the cache's head count (e.g. GQA/MQA where the layer projects fewer KV heads) but no cache_head_start offset was provided, so the code cannot know which slice of cache heads to write.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/kvcache/causal_attention_cache.py:165

        Args:
            current_chunk_start: the global position of the start of the chunk
            cache_head_start: first cache head for key/value when they only
                carry a local slice of the cache heads; other heads are left untouched
            recent_window_tokens: recent-window attention size. ``None``
                returns the full visible attention window. ``0`` keeps only sink
                tokens plus the current chunk. A positive value keeps sink tokens,
                up to that many tokens before the current chunk, and the current
                chunk. Negative values are invalid.

        """
        num_new_tokens = key.shape[1]
        num_input_heads = key.shape[2]
        num_cache_heads = self.k.shape[2]
        cache_head_slice = None
        if num_cache_heads != num_input_heads:
            if cache_head_start is None:
                raise ValueError(
                    f"{debug_name} requires cache_head_start when cache heads "
                    f"({num_cache_heads}) differ from input heads ({num_input_heads})."
                )
            cache_head_slice = slice(
                cache_head_start, cache_head_start + num_input_heads
            )
        current_chunk_end = current_chunk_start + num_new_tokens
        kv_cache_size = self.cache_size
        sink_tokens = self._effective_sink_tokens()
        global_end_index, local_end_index_prev = self._read_indices()

        # local_start(/end)_index: the local position of the start/end of current chunk
        # updated_local_end: the updated local end
        # updated_global_end: the updated global end

        # the global position of the start of the buffer
        window_start = global_end_index - local_end_index_prev

View on GitHub (pinned to 0132848349)

Solutions

1. Pass cache_head_start indicating the first cache head index for this layer's KV write
2. Align num_input_heads with the cache's num heads if no slicing was intended (fix layer head config)
3. Verify the cache was allocated with the total (max) head count expected across layers

Example fix

# before
cache.update_and_get_attention_kv(key, value, debug_name="layer0")
# after
cache.update_and_get_attention_kv(key, value, cache_head_start=head_offset, debug_name="layer0")

if key.shape[2] != cache.k.shape[2]:
    assert cache_head_start is not None, 'cache_head_start required for head slice'

When it happens

Trigger: Calling update_and_get_attention_kv with key.shape[2] != self.k.shape[2] and cache_head_start=None; typical for multi-head attention layers writing into a shared multi-head cache.

Common situations: Migrating a model where some layers have partial head counts; forgetting to pass cache_head_start when wiring a new attention layer with head slicing; mismatched head configs between cache allocation and layer projection.

Related errors

• {debug_name}: cache_head_start required for head slice
• k_pool has incompatible shape {k_pool.shape}
• kv d_qk must match q d_qk={d_qk}, got {kv_d_qk}
• kv-canary: launch_canary_plan_kernels_torch_reference verify
• {debug_name}: current-chunk rewrite size changed

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/dd8b8f13ad1ba8d6. Report an issue: GitHub.