{"record":{"id":"4f5e700e7b1750df","repo":"sgl-project/sglang","slug":"k-pool-has-incompatible-shape-k-pool-shape","errorCode":null,"errorMessage":"k_pool has incompatible shape {k_pool.shape}","messagePattern":"k_pool has incompatible shape (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/aot/python/sgl_kernel/metal.py","lineNumber":90,"sourceCode":"    positions_shape = tuple(positions.shape)\n    slots_shape = tuple(slots.shape)\n    k_pool_shape = tuple(k_pool.shape)\n    v_pool_shape = tuple(v_pool.shape)\n\n    if q_shape != (q_shape[0], num_qo_heads, head_dim):\n        raise ValueError(\n            \"q shape must be [num_tokens, num_qo_heads, head_dim], \" f\"got {q.shape}\"\n        )\n    if k_shape != (q_shape[0], num_kv_heads, head_dim):\n        raise ValueError(\n            \"k shape must be [num_tokens, num_kv_heads, head_dim], \" f\"got {k.shape}\"\n        )\n    if v_shape != k_shape:\n        raise ValueError(f\"v shape must match k shape, got {v.shape} vs {k.shape}\")\n    if positions_shape != (q_shape[0],) or slots_shape != (q_shape[0],):\n        raise ValueError(\"positions/slots must have one entry per token\")\n    if k_pool_shape[1:] != (num_kv_heads, head_dim):\n        raise ValueError(f\"k_pool has incompatible shape {k_pool.shape}\")\n    if v_pool_shape != k_pool_shape:\n        raise ValueError(\n            f\"v_pool shape must match k_pool shape, got {v_pool.shape} vs {k_pool.shape}\"\n        )\n    if q.dtype != k.dtype or q.dtype != v.dtype:\n        raise ValueError(\"q/k/v dtypes must match\")\n    if k_pool.dtype != q.dtype or v_pool.dtype != q.dtype:\n        raise ValueError(\"pool dtypes must match q/k/v dtype\")\n\n    return _metal.rope_pool_fused(\n        q,\n        k,\n        v,\n        positions,\n        slots,\n        k_pool,\n        v_pool,\n        head_dim,","sourceCodeStart":72,"sourceCodeEnd":108,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/aot/python/sgl_kernel/metal.py#L72-L108","documentation":"The k_pool tensor must have trailing dimensions [num_kv_heads, head_dim]; its leading (pool_size) dimension is free. This error fires when k_pool.shape[1:] != (num_kv_heads, head_dim), i.e. the pool was allocated with head layout inconsistent with the declared KV configuration.","triggerScenarios":"Passing a pool allocated as [pool_size, num_kv_heads*head_dim], [pool_size, total_heads, head_dim] with MHA head count, or with head_dim of a different config.","commonSituations":"Reusing a cache allocated for another model or backend; allocating pools from a generic cache manager that flattens heads; config change (e.g. rope scaling variant) altering head_dim after pool allocation.","solutions":["Reallocate k_pool as torch.empty(pool_size, num_kv_heads, head_dim, dtype=q.dtype)","If the pool is flat, reshape: k_pool = k_pool.view(pool_size, num_kv_heads, head_dim) (only if layout permits)","Verify num_kv_heads/head_dim passed to the call equal the values used at pool allocation"],"exampleFix":"# before\nk_pool = torch.empty(pool_size, num_kv_heads * head_dim, dtype=dtype)\n\n# after\nk_pool = torch.empty(pool_size, num_kv_heads, head_dim, dtype=dtype)","handlingStrategy":"validation","validationCode":"assert k_pool.shape[1:] == (num_kv_heads, head_dim), (k_pool.shape, num_kv_heads, head_dim)","typeGuard":"def pool_layout_ok(k_pool, num_kv_heads, head_dim) -> bool:\n    return k_pool.ndim == 3 and tuple(k_pool.shape[1:]) == (num_kv_heads, head_dim)","tryCatchPattern":null,"preventionTips":["Allocate pools from the same config used for the rope call","Never flatten the head/head_dim dims of the pool"],"tags":["shape-validation","kv-cache","gqa","metal","sgl-kernel"],"backgroundTag":"tensor-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}