sgl-project/sglang · error · ValueError

v_pool shape must match k_pool shape, got {v_pool.shape} vs

Error message

v_pool shape must match k_pool shape, got {v_pool.shape} vs {k_pool.shape}

What it means

rope_pool_fused requires v_pool to have exactly the same shape as k_pool ([pool_size, num_kv_heads, head_dim]). This error fires when the V pool's shape (usually pool_size) differs from the K pool, meaning the two halves of the KV cache are inconsistent.

Source

Thrown at python/sglang/kernels/aot/python/sgl_kernel/metal.py:92

    k_pool_shape = tuple(k_pool.shape)
    v_pool_shape = tuple(v_pool.shape)

    if q_shape != (q_shape[0], num_qo_heads, head_dim):
        raise ValueError(
            "q shape must be [num_tokens, num_qo_heads, head_dim], " f"got {q.shape}"
        )
    if k_shape != (q_shape[0], num_kv_heads, head_dim):
        raise ValueError(
            "k shape must be [num_tokens, num_kv_heads, head_dim], " f"got {k.shape}"
        )
    if v_shape != k_shape:
        raise ValueError(f"v shape must match k shape, got {v.shape} vs {k.shape}")
    if positions_shape != (q_shape[0],) or slots_shape != (q_shape[0],):
        raise ValueError("positions/slots must have one entry per token")
    if k_pool_shape[1:] != (num_kv_heads, head_dim):
        raise ValueError(f"k_pool has incompatible shape {k_pool.shape}")
    if v_pool_shape != k_pool_shape:
        raise ValueError(
            f"v_pool shape must match k_pool shape, got {v_pool.shape} vs {k_pool.shape}"
        )
    if q.dtype != k.dtype or q.dtype != v.dtype:
        raise ValueError("q/k/v dtypes must match")
    if k_pool.dtype != q.dtype or v_pool.dtype != q.dtype:
        raise ValueError("pool dtypes must match q/k/v dtype")

    return _metal.rope_pool_fused(
        q,
        k,
        v,
        positions,
        slots,
        k_pool,
        v_pool,
        head_dim,
        num_qo_heads,
        num_kv_heads,

View on GitHub (pinned to 0132848349)

Solutions

1. Allocate both pools together with identical shape: torch.empty(pool_size, num_kv_heads, head_dim, dtype) for both k_pool and v_pool
2. If resizing, resize both: v_pool = v_pool[: k_pool.shape[0]]
3. Assert v_pool.shape == k_pool.shape at allocation and before the call

Example fix

# before
k_pool = torch.empty(pool_size, kv_heads, D)
v_pool = torch.empty(pool_size - 1, kv_heads, D)  # sizing bug

# after
k_pool = torch.empty(pool_size, kv_heads, D)
v_pool = torch.empty(pool_size, kv_heads, D)
assert v_pool.shape == k_pool.shape

assert v_pool.shape == k_pool.shape, (v_pool.shape, k_pool.shape)

def pools_match(k_pool, v_pool) -> bool:
    return k_pool.shape == v_pool.shape

When it happens

Trigger: Allocating v_pool with a different pool_size than k_pool (off-by-one or different sizing formula); resizing one pool but not the other; passing pools from different layers/models.

Common situations: Cache sizing bug where v_pool uses a different token-capacity formula; memory-pressure code that shrinks only one pool; typos in allocation loops producing mismatched pool sizes.

Related errors

• rope_pool_fused expects pool tensors to be 3-D
• rope_pool_fused expects q/k/v to be 3-D
• rope_pool_fused expects positions/slots to be 1-D
• q shape must be [num_tokens, num_qo_heads, head_dim], got {q
• k shape must be [num_tokens, num_kv_heads, head_dim], got {k

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/693f579470ce65b1. Report an issue: GitHub.