sgl-project/sglang · error · ValueError

v shape must match k shape, got {v.shape} vs {k.shape}

Error message

v shape must match k shape, got {v.shape} vs {k.shape}

What it means

rope_pool_fused requires v to have exactly the same shape as k ([num_tokens, num_kv_heads, head_dim]). This error fires when v's shape differs from k, meaning the K and V projections are inconsistent for this call.

Source

Thrown at python/sglang/kernels/aot/python/sgl_kernel/metal.py:86

        raise ValueError("rope_pool_fused expects pool tensors to be 3-D")
    q_shape = tuple(q.shape)
    k_shape = tuple(k.shape)
    v_shape = tuple(v.shape)
    positions_shape = tuple(positions.shape)
    slots_shape = tuple(slots.shape)
    k_pool_shape = tuple(k_pool.shape)
    v_pool_shape = tuple(v_pool.shape)

    if q_shape != (q_shape[0], num_qo_heads, head_dim):
        raise ValueError(
            "q shape must be [num_tokens, num_qo_heads, head_dim], " f"got {q.shape}"
        )
    if k_shape != (q_shape[0], num_kv_heads, head_dim):
        raise ValueError(
            "k shape must be [num_tokens, num_kv_heads, head_dim], " f"got {k.shape}"
        )
    if v_shape != k_shape:
        raise ValueError(f"v shape must match k shape, got {v.shape} vs {k.shape}")
    if positions_shape != (q_shape[0],) or slots_shape != (q_shape[0],):
        raise ValueError("positions/slots must have one entry per token")
    if k_pool_shape[1:] != (num_kv_heads, head_dim):
        raise ValueError(f"k_pool has incompatible shape {k_pool.shape}")
    if v_pool_shape != k_pool_shape:
        raise ValueError(
            f"v_pool shape must match k_pool shape, got {v_pool.shape} vs {k_pool.shape}"
        )
    if q.dtype != k.dtype or q.dtype != v.dtype:
        raise ValueError("q/k/v dtypes must match")
    if k_pool.dtype != q.dtype or v_pool.dtype != q.dtype:
        raise ValueError("pool dtypes must match q/k/v dtype")

    return _metal.rope_pool_fused(
        q,
        k,
        v,
        positions,

View on GitHub (pinned to 0132848349)

Solutions

1. Assert v.shape == k.shape before the call and reshape/slice v to match k
2. Fix QKV split indices: k = qkv[..., q_size:q_size+kv_size].reshape(...); v = qkv[..., q_size+kv_size:].reshape(...) with identical kv sizes
3. Log both shapes to find which dimension diverges

Example fix

# before
k = qkv[:, :kv_size].view(N, kv_heads, D)
v = qkv[:, kv_size:].view(N, kv_heads, D)  # off-by-one slice -> wrong token/head count

# after
k = qkv[..., q_size:q_size+kv_size].reshape(N, kv_heads, D)
v = qkv[..., q_size+kv_size:].reshape(N, kv_heads, D)
assert v.shape == k.shape

assert v.shape == k.shape, (v.shape, k.shape)

def kv_same_shape(k, v) -> bool:
    return k.shape == v.shape

When it happens

Trigger: Passing v with a different token count, head count, or head_dim than k; passing a v sliced differently from k; misconfigured projection producing mismatched K/V widths.

Common situations: Splitting a fused QKV projection with wrong slice boundaries so V gets a different width than K; partial slicing during chunked prefill where v was sliced but k was not; copy-paste of shapes from a different layer.

Related errors

• rope_pool_fused expects q/k/v to be 3-D
• rope_pool_fused expects positions/slots to be 1-D
• rope_pool_fused expects pool tensors to be 3-D
• q shape must be [num_tokens, num_qo_heads, head_dim], got {q
• k shape must be [num_tokens, num_kv_heads, head_dim], got {k

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/bf6412564fa65ee3. Report an issue: GitHub.