{"record":{"id":"dd8b8f13ad1ba8d6","repo":"sgl-project/sglang","slug":"debug-name-requires-cache-head-start-when-cache","errorCode":null,"errorMessage":"{debug_name} requires cache_head_start when cache heads ({num_cache_heads}) differ from input heads ({num_input_heads}).","messagePattern":"(.+?) requires cache_head_start when cache heads \\((.+?)\\) differ from input heads \\((.+?)\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/layers/kvcache/causal_attention_cache.py","lineNumber":165,"sourceCode":"\n        Args:\n            current_chunk_start: the global position of the start of the chunk\n            cache_head_start: first cache head for key/value when they only\n                carry a local slice of the cache heads; other heads are left untouched\n            recent_window_tokens: recent-window attention size. ``None``\n                returns the full visible attention window. ``0`` keeps only sink\n                tokens plus the current chunk. A positive value keeps sink tokens,\n                up to that many tokens before the current chunk, and the current\n                chunk. Negative values are invalid.\n\n        \"\"\"\n        num_new_tokens = key.shape[1]\n        num_input_heads = key.shape[2]\n        num_cache_heads = self.k.shape[2]\n        cache_head_slice = None\n        if num_cache_heads != num_input_heads:\n            if cache_head_start is None:\n                raise ValueError(\n                    f\"{debug_name} requires cache_head_start when cache heads \"\n                    f\"({num_cache_heads}) differ from input heads ({num_input_heads}).\"\n                )\n            cache_head_slice = slice(\n                cache_head_start, cache_head_start + num_input_heads\n            )\n        current_chunk_end = current_chunk_start + num_new_tokens\n        kv_cache_size = self.cache_size\n        sink_tokens = self._effective_sink_tokens()\n        global_end_index, local_end_index_prev = self._read_indices()\n\n        # local_start(/end)_index: the local position of the start/end of current chunk\n        # updated_local_end: the updated local end\n        # updated_global_end: the updated global end\n\n        # the global position of the start of the buffer\n        window_start = global_end_index - local_end_index_prev\n","sourceCodeStart":147,"sourceCodeEnd":183,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/layers/kvcache/causal_attention_cache.py#L147-L183","documentation":"CausalAttentionKVCache.update_and_get_attention_kv raises this when the incoming key tensor's head count differs from the cache's head count (e.g. GQA/MQA where the layer projects fewer KV heads) but no cache_head_start offset was provided, so the code cannot know which slice of cache heads to write.","triggerScenarios":"Calling update_and_get_attention_kv with key.shape[2] != self.k.shape[2] and cache_head_start=None; typical for multi-head attention layers writing into a shared multi-head cache.","commonSituations":"Migrating a model where some layers have partial head counts; forgetting to pass cache_head_start when wiring a new attention layer with head slicing; mismatched head configs between cache allocation and layer projection.","solutions":["Pass cache_head_start indicating the first cache head index for this layer's KV write","Align num_input_heads with the cache's num heads if no slicing was intended (fix layer head config)","Verify the cache was allocated with the total (max) head count expected across layers"],"exampleFix":"# before\ncache.update_and_get_attention_kv(key, value, debug_name=\"layer0\")\n# after\ncache.update_and_get_attention_kv(key, value, cache_head_start=head_offset, debug_name=\"layer0\")","handlingStrategy":"validation","validationCode":"if key.shape[2] != cache.k.shape[2]:\n    assert cache_head_start is not None, 'cache_head_start required for head slice'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always pass cache_head_start for GQA/MQA writers","Assert cache head count >= max layer head count at init","Unit-test head-sliced writes per layer"],"tags":["kv-cache","gqa","head-slicing","shape-mismatch"],"backgroundTag":"kv-cache-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}